Skip to contents

Handler for public AWS S3 data sources

Source: R/aws_s3.R
bb_handler_aws_s3.Rd

This is a handler function to be used with AWS S3 data providers. This function is not intended to be called directly, but rather is specified as a method option in bb_source. Note that this currently only works with public data sources that are accessible without an S3 key. The method arguments accepted by bb_handler_aws_s3 are currently:

  • "bucket" string: name of the bucket (defaults to "")

  • "base_url" string: as for s3HTTP

  • "region" string: as for s3HTTP

  • "use_https" logical: as for s3HTTP

  • "prefix" string: as for get_bucket; only keys in the bucket that begin with the specified prefix will be processed

  • and other parameters passed to the bb_rget function, including "accept_download", "accept_download_extra", "reject_download"

Note that the "prefix", "accept_download", "accept_download_extra", "reject_download" parameters can be used to restrict which files are downloaded from the bucket.

Usage

bb_handler_aws_s3(...)

Arguments

...

: parameters, see Description

Value

A tibble with columns ok, files, message

Examples

if (FALSE) {
  ## an example AWS S3 data source
  src <- bb_source(
           name = "SILO climate data",
           id = "silo-open-data",
           description = "Australian climate data from 1889 to yesterday.
                          This source includes a single example monthly rainfall data file.
                          Adjust the 'accept_download' parameter to change this.",
           doc_url = "https://www.longpaddock.qld.gov.au/silo/gridded-data/",
           citation = "SILO datasets are constructed by the Queensland Government using
                       observational data provided by the Australian Bureau of Meteorology
                       and are available under the Creative Commons Attribution 4.0 license",
           license = "CC-BY 4.0",
           method = list("bb_handler_aws_s3", region = "silo-open-data.s3",
                         base_url = "amazonaws.com", prefix = "Official/annual/monthly_rain/",
                         accept_download = "2005\\.monthly_rain\\.nc$"),
           comment = "The unusual specification of region and base_url is a workaround for
                      an aws.s3 issue, see https://github.com/cloudyr/aws.s3/issues/318",
           postprocess = NULL,
           collection_size = 0.02,
           data_group = "Climate")
   temp_root <- tempdir()
   status <- bb_get(src, local_file_root = temp_root, verbose = TRUE)
}