Superseded: configure exponential backoff while polling for tasks during the pipeline.
Arguments
- min
Positive numeric of length 1, minimum polling interval in seconds. Must be at least
sqrt(.Machine$double.eps).- max
Positive numeric of length 1, maximum polling interval in seconds. Must be at least
sqrt(.Machine$double.eps).- rate
Positive numeric of length 1, greater than or equal to 1. Multiplicative rate parameter that allows the exponential backoff minimum polling interval to increase from
mintomax. Actual polling intervals are sampled uniformly from the current minimum tomax.
Details
This function is superseded and is now only relevant to other
superseded functions tar_make_clustermq() and tar_make_future().
tar_make() uses crew in an efficient non-polling way, making
exponential backoff unnecessary.
Backoff
In high-performance computing it can be expensive to repeatedly poll the
priority queue if no targets are ready to process. The number of seconds
between polls is runif(1, min, max(max, min * rate ^ index)),
where index is the number of consecutive polls so far that found
no targets ready to skip or run, and min, max, and rate
are arguments to tar_backoff().
(If no target is ready, index goes up by 1. If a target is ready,
index resets to 0. For more information on exponential,
backoff, visit https://en.wikipedia.org/wiki/Exponential_backoff).
Raising min or max is kinder to the CPU etc. but may incur delays
in some instances.
See also
Other utilities:
tar_active(),
tar_call(),
tar_cancel(),
tar_definition(),
tar_described_as(),
tar_envir(),
tar_format_get(),
tar_group(),
tar_name(),
tar_path(),
tar_path_script(),
tar_path_script_support(),
tar_path_store(),
tar_path_target(),
tar_source(),
tar_store(),
tar_unblock_process()
Examples
if (identical(Sys.getenv("TAR_EXAMPLES"), "true")) { # for CRAN
tar_dir({ # tar_dir() runs code from a temp dir for CRAN.
tar_option_set(backoff = tar_backoff(min = 0.001, max = 0.1, rate = 1.5))
})
}