Target resources: clustermq
high-performance computing
Source: R/tar_resources_clustermq.R
tar_resources_clustermq.Rd
Create the clustermq
argument of tar_resources()
to specify optional high-performance computing settings
for tar_make_clustermq()
.
For details, see the documentation of the clustermq
R package
and the corresponding argument names in this help file.
Usage
tar_resources_clustermq(
template = targets::tar_option_get("resources")$clustermq$template
)
Arguments
- template
Named list,
template
argument toclustermq::workers()
. Defaults to an empty list.
Value
Object of class "tar_resources_clustermq"
, to be supplied
to the clustermq
argument of tar_resources()
.
Details
clustermq
workers are persistent,
so there is not a one-to-one correspondence between workers and targets.
The clustermq
resources apply to the workers, not the targets.
So the correct way to assign clustermq
resources is through
tar_option_set()
, not tar_target()
. clustermq
resources
in individual tar_target()
calls will be ignored.
Resources
Functions tar_target()
and tar_option_set()
each takes an optional resources
argument to supply
non-default settings of various optional backends for data storage
and high-performance computing. The tar_resources()
function
is a helper to supply those settings in the correct manner.
In targets
version 0.12.2 and above, resources are inherited one-by-one
in nested fashion from tar_option_get("resources")
.
For example, suppose you set
tar_option_set(resources = tar_resources(aws = my_aws))
,
where my_aws
equals tar_resources_aws(bucket = "x", prefix = "y")
.
Then, tar_target(data, get_data()
will have bucket "x"
and
prefix "y"
. In addition, if new_resources
equals
tar_resources(aws = tar_resources_aws(bucket = "z")))
, then
tar_target(data, get_data(), resources = new_resources)
will use the new bucket "z"
, but it will still use the prefix "y"
supplied through tar_option_set()
. (In targets
0.12.1 and below,
options like prefix
do not carry over from tar_option_set()
if you
supply non-default resources to tar_target()
.)
See also
Other resources:
tar_resources()
,
tar_resources_aws()
,
tar_resources_crew()
,
tar_resources_custom_format()
,
tar_resources_feather()
,
tar_resources_fst()
,
tar_resources_future()
,
tar_resources_gcp()
,
tar_resources_network()
,
tar_resources_parquet()
,
tar_resources_qs()
,
tar_resources_repository_cas()
,
tar_resources_url()
Examples
# Somewhere in you target script file (usually _targets.R):
tar_target(
name,
command(),
resources = tar_resources(
clustermq = tar_resources_clustermq(template = list(n_cores = 2))
)
)
#> <tar_stem>
#> name: name
#> description:
#> command:
#> command()
#> format: rds
#> repository: local
#> iteration method: vector
#> error mode: stop
#> memory mode: persistent
#> storage mode: main
#> retrieval mode: main
#> deployment mode: worker
#> priority: 0
#> resources:
#> clustermq: <environment>
#> cue:
#> mode: thorough
#> command: TRUE
#> depend: TRUE
#> format: TRUE
#> repository: TRUE
#> iteration: TRUE
#> file: TRUE
#> seed: TRUE
#> packages:
#> targets
#> stats
#> graphics
#> grDevices
#> utils
#> datasets
#> methods
#> base
#> library:
#> NULL