Target resources: crew
high-performance computing
Source: R/tar_resources_crew.R
tar_resources_crew.Rd
Create the crew
argument of tar_resources()
to specify optional target settings.
Usage
tar_resources_crew(
controller = targets::tar_option_get("resources")$crew$controller,
scale = NULL,
seconds_timeout = targets::tar_option_get("resources")$crew$seconds_timeout
)
Arguments
- controller
Character of length 1. If
tar_option_get("controller")
is acrew
controller group, thecontroller
argument oftar_resources_crew()
indicates which controller in the controller group to use. If you need heterogeneous workers, you can leverage this argument to send different targets to different worker groups.- scale
Deprecated in version 1.3.0.9002 (2023-10-02). No longer necessary.
- seconds_timeout
Positive numeric of length 1, optional task timeout passed to the
.timeout
argument ofmirai::mirai()
(after converting to milliseconds).
Value
Object of class "tar_resources_crew"
, to be supplied
to the crew
argument of tar_resources()
.
Details
tar_resources_crew()
accepts
target-specific settings for integration with the
crew
R package. These settings are arguments to the push()
method of the controller or controller group
object which control things like
auto-scaling behavior and the controller to use in the case
of a controller group.
Resources
Functions tar_target()
and tar_option_set()
each takes an optional resources
argument to supply
non-default settings of various optional backends for data storage
and high-performance computing. The tar_resources()
function
is a helper to supply those settings in the correct manner.
In targets
version 0.12.2 and above, resources are inherited one-by-one
in nested fashion from tar_option_get("resources")
.
For example, suppose you set
tar_option_set(resources = tar_resources(aws = my_aws))
,
where my_aws
equals tar_resources_aws(bucket = "x", prefix = "y")
.
Then, tar_target(data, get_data()
will have bucket "x"
and
prefix "y"
. In addition, if new_resources
equals
tar_resources(aws = tar_resources_aws(bucket = "z")))
, then
tar_target(data, get_data(), resources = new_resources)
will use the new bucket "z"
, but it will still use the prefix "y"
supplied through tar_option_set()
. (In targets
0.12.1 and below,
options like prefix
do not carry over from tar_option_set()
if you
supply non-default resources to tar_target()
.)
See also
Other resources:
tar_resources()
,
tar_resources_aws()
,
tar_resources_clustermq()
,
tar_resources_custom_format()
,
tar_resources_feather()
,
tar_resources_fst()
,
tar_resources_future()
,
tar_resources_gcp()
,
tar_resources_network()
,
tar_resources_parquet()
,
tar_resources_qs()
,
tar_resources_repository_cas()
,
tar_resources_url()
Examples
# Somewhere in you target script file (usually _targets.R):
tar_target(
name,
command(),
resources = tar_resources(
crew = tar_resources_crew(seconds_timeout = 5)
)
)
#> <tar_stem>
#> name: name
#> description:
#> command:
#> command()
#> format: rds
#> repository: local
#> iteration method: vector
#> error mode: stop
#> memory mode: persistent
#> storage mode: main
#> retrieval mode: main
#> deployment mode: worker
#> priority: 0
#> resources:
#> crew: <environment>
#> cue:
#> mode: thorough
#> command: TRUE
#> depend: TRUE
#> format: TRUE
#> repository: TRUE
#> iteration: TRUE
#> file: TRUE
#> seed: TRUE
#> packages:
#> targets
#> stats
#> graphics
#> grDevices
#> utils
#> datasets
#> methods
#> base
#> library:
#> NULL