Create a resources
argument for tar_target()
or tar_option_set()
.
Usage
tar_resources(
aws = tar_option_get("resources")$aws,
clustermq = tar_option_get("resources")$clustermq,
crew = tar_option_get("resources")$crew,
custom_format = tar_option_get("resources")$custom_format,
feather = tar_option_get("resources")$feather,
fst = tar_option_get("resources")$fst,
future = tar_option_get("resources")$future,
gcp = tar_option_get("resources")$gcp,
network = tar_option_get("resources")$network,
parquet = tar_option_get("resources")$parquet,
qs = tar_option_get("resources")$qs,
repository_cas = tar_option_get("resources")$repository_cas,
url = tar_option_get("resources")$url
)
Arguments
- aws
Output of function
tar_resources_aws()
. Amazon Web Services (AWS) S3 storage settings fortar_target(..., repository = "aws")
. See the cloud storage section of https://books.ropensci.org/targets/data.html for details for instructions.- clustermq
Output of function
tar_resources_clustermq()
. Optionalclustermq
settings fortar_make_clustermq()
, including thelog_worker
andtemplate
arguments ofclustermq::workers()
.clustermq
workers are persistent, so there is not a one-to-one correspondence between workers and targets. Theclustermq
resources apply to the workers, not the targets. So the correct way to assignclustermq
resources is throughtar_option_set()
, nottar_target()
.clustermq
resources in individualtar_target()
calls will be ignored.- crew
Output of function
tar_resources_crew()
with target-specific settings for integration with thecrew
R package. These settings are arguments to thepush()
method of the controller or controller group object which control things like auto-scaling behavior and the controller to use in the case of a controller group.- custom_format
Output of function
tar_resources_custom_format()
with configuration details fortar_format()
storage formats.- feather
Output of function
tar_resources_feather()
. Non-default arguments toarrow::read_feather()
andarrow::write_feather()
forarrow
/feather-based storage formats. Applies to all formats ending with the"_feather"
suffix. For details on formats, see theformat
argument oftar_target()
.- fst
Output of function
tar_resources_fst()
. Non-default arguments tofst::read_fst()
andfst::write_fst()
forfst
-based storage formats. Applies to all formats ending with"fst"
in the name. For details on formats, see theformat
argument oftar_target()
.- future
Output of function
tar_resources_future()
. Optionalfuture
settings fortar_make_future()
, including theresources
argument offuture::future()
, which can include values to insert in template placeholders infuture.batchtools
template files. This is how to supply theresources
argument offuture::future()
fortargets
. Resources supplied throughfuture::plan()
andfuture::tweak()
are completely ignored.- gcp
Output of function
tar_resources_gcp()
. Google Cloud Storage bucket settings fortar_target(..., repository = "gcp")
. See the cloud storage section of https://books.ropensci.org/targets/data.html for details for instructions.- network
Output of function
tar_resources_network()
. Settings to configure how to handle unreliable network connections in the case of uploading, downloading, and checking data in situations that rely on network file systems or HTTP/HTTPS requests. Examples include retries and timeouts for internal storage management operations forstorage = "worker"
orformat = "file"
(on network file systems),format = "url"
,repository = "aws"
, andrepository = "gcp"
. These settings do not apply to actions you take in the custom R command of the target.- parquet
Output of function
tar_resources_parquet()
. Non-default arguments toarrow::read_parquet()
andarrow::write_parquet()
forarrow
/parquet-based storage formats. Applies to all formats ending with the"_parquet"
suffix. For details on formats, see theformat
argument oftar_target()
.- qs
Output of function
tar_resources_qs()
. Non-default arguments toqs::qread()
andqs::qsave()
forqs
-based storage formats. Applies to all formats ending with the"_qs"
suffix. For details on formats, see theformat
argument oftar_target()
.- repository_cas
Output of function
tar_resources_repository_cas()
with configuration details fortar_repository_cas()
storage repositories.- url
Output of function
tar_resources_url()
. Non-default settings for storage formats ending with the"_url"
suffix. These settings include thecurl
handle for extra control over HTTP requests. For details on formats, see theformat
argument oftar_target()
.
Value
A list of objects of class "tar_resources"
with
non-default settings of various optional backends for data storage
and high-performance computing.
Resources
Functions tar_target()
and tar_option_set()
each takes an optional resources
argument to supply
non-default settings of various optional backends for data storage
and high-performance computing. The tar_resources()
function
is a helper to supply those settings in the correct manner.
In targets
version 0.12.2 and above, resources are inherited one-by-one
in nested fashion from tar_option_get("resources")
.
For example, suppose you set
tar_option_set(resources = tar_resources(aws = my_aws))
,
where my_aws
equals tar_resources_aws(bucket = "x", prefix = "y")
.
Then, tar_target(data, get_data()
will have bucket "x"
and
prefix "y"
. In addition, if new_resources
equals
tar_resources(aws = tar_resources_aws(bucket = "z")))
, then
tar_target(data, get_data(), resources = new_resources)
will use the new bucket "z"
, but it will still use the prefix "y"
supplied through tar_option_set()
. (In targets
0.12.1 and below,
options like prefix
do not carry over from tar_option_set()
if you
supply non-default resources to tar_target()
.)
See also
Other resources:
tar_resources_aws()
,
tar_resources_clustermq()
,
tar_resources_crew()
,
tar_resources_custom_format()
,
tar_resources_feather()
,
tar_resources_fst()
,
tar_resources_future()
,
tar_resources_gcp()
,
tar_resources_network()
,
tar_resources_parquet()
,
tar_resources_qs()
,
tar_resources_repository_cas()
,
tar_resources_url()
Examples
# Somewhere in you target script file (usually _targets.R):
tar_target(
name,
command(),
format = "qs",
resources = tar_resources(
qs = tar_resources_qs(preset = "fast"),
future = tar_resources_future(resources = list(n_cores = 1))
)
)
#> <tar_stem>
#> name: name
#> description:
#> command:
#> command()
#> format: qs
#> repository: local
#> iteration method: vector
#> error mode: stop
#> memory mode: persistent
#> storage mode: main
#> retrieval mode: main
#> deployment mode: worker
#> priority: 0
#> resources:
#> future: <environment>
#> qs: <environment>
#> cue:
#> mode: thorough
#> command: TRUE
#> depend: TRUE
#> format: TRUE
#> repository: TRUE
#> iteration: TRUE
#> file: TRUE
#> seed: TRUE
#> packages:
#> targets
#> stats
#> graphics
#> grDevices
#> utils
#> datasets
#> methods
#> base
#> library:
#> NULL