AsyncQueue

An AsyncQueue client

R6 classes

This is an R6 class from the package R6. Find out more about R6 at https://r6.r-lib.org/. After creating an instance of an R6 class (e.g., x <- HttpClient$new(url = "https://hb.opencpu.org")) you can access values and methods on the object x.

See also

Other async: Async, AsyncVaried, HttpRequest

Super class

crul::AsyncVaried -> AsyncQueue

Public fields

bucket_size

(integer) number of requests to send at once

sleep

(integer) number of seconds to sleep between each bucket

req_per_min

(integer) requests per minute

Methods

Public methods

• AsyncQueue$print()

• AsyncQueue$new()

• AsyncQueue$request()

• AsyncQueue$responses()

• AsyncQueue$parse()

• AsyncQueue$status_code()

• AsyncQueue$status()

• AsyncQueue$content()

• AsyncQueue$times()

• AsyncQueue$clone()

Inherited methods

• crul::AsyncVaried$requests()

---

Method print()

print method for AsyncQueue objects

Usage

AsyncQueue$print(x, ...)

Arguments

x

self

...

ignored

---

Method new()

Create a new AsyncQueue object

Usage

AsyncQueue$new(
  ...,
  .list = list(),
  bucket_size = 5,
  sleep = NULL,
  req_per_min = NULL
)

Arguments

..., .list

Any number of objects of class HttpRequest(), must supply inputs to one of these parameters, but not both

bucket_size

(integer) number of requests to send at once. default: 5. See Details.

sleep

(integer) seconds to sleep between buckets. default: NULL (not set)

req_per_min

(integer) maximum number of requests per minute. if NULL (default), its ignored

Details

Must set either sleep or req_per_min. If you set req_per_min we calculate a new bucket_size when $new() is called

Returns

A new AsyncQueue object

---

Method request()

Execute asynchronous requests

Usage

AsyncQueue$request()

Returns

nothing, responses stored inside object, though will print messages if you choose verbose output

---

Method responses()

List responses

Usage

AsyncQueue$responses()

Returns

a list of HttpResponse objects, empty list before requests made

---

Method parse()

parse content

Usage

AsyncQueue$parse(encoding = "UTF-8")

Arguments

encoding

(character) the encoding to use in parsing. default:"UTF-8"

Returns

character vector, empty character vector before requests made

---

Method status_code()

Get HTTP status codes for each response

Usage

AsyncQueue$status_code()

Returns

numeric vector, empty numeric vector before requests made

---

Method status()

List HTTP status objects

Usage

AsyncQueue$status()

Returns

a list of http_code objects, empty list before requests made

---

Method content()

Get raw content for each response

Usage

AsyncQueue$content()

Returns

raw list, empty list before requests made

---

Method times()

curl request times

Usage

AsyncQueue$times()

Returns

list of named numeric vectors, empty list before requests made

---

Method clone()

The objects of this class are cloneable with this method.

Usage

AsyncQueue$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

Examples

if (FALSE) { # \dontrun{
# Using sleep (note this works with retry requests)
reqlist <- list(
  HttpRequest$new(url = "https://hb.opencpu.org/get")$get(),
  HttpRequest$new(url = "https://hb.opencpu.org/post")$post(),
  HttpRequest$new(url = "https://hb.opencpu.org/put")$put(),
  HttpRequest$new(url = "https://hb.opencpu.org/delete")$delete(),
  HttpRequest$new(url = "https://hb.opencpu.org/get?g=5")$get(),
  HttpRequest$new(
    url = "https://hb.opencpu.org/post")$post(body = list(y = 9)),
  HttpRequest$new(
    url = "https://hb.opencpu.org/get")$get(query = list(hello = "world")),
  HttpRequest$new(url = "https://ropensci.org")$get(),
  HttpRequest$new(url = "https://ropensci.org/about")$get(),
  HttpRequest$new(url = "https://ropensci.org/packages")$get(),
  HttpRequest$new(url = "https://ropensci.org/community")$get(),
  HttpRequest$new(url = "https://ropensci.org/blog")$get(),
  HttpRequest$new(url = "https://ropensci.org/careers")$get(),
  HttpRequest$new(url = "https://hb.opencpu.org/status/404")$retry("get")
)
out <- AsyncQueue$new(.list = reqlist, bucket_size = 5, sleep = 3)
out
out$bucket_size # bucket size
out$requests() # list requests
out$request() # make requests
out$responses() # list responses

# Using requests per minute
if (interactive()) {
x="https://raw.githubusercontent.com/ropensci/roregistry/gh-pages/registry.json"
z <- HttpClient$new(x)$get()
urls <- jsonlite::fromJSON(z$parse("UTF-8"))$packages$url
repos = Filter(length, regmatches(urls, gregexpr("ropensci/[A-Za-z]+", urls)))
repos = unlist(repos)
auth <- list(Authorization = paste("token", Sys.getenv('GITHUB_PAT')))
reqs <- lapply(repos[1:50], function(w) {
  HttpRequest$new(paste0("https://api.github.com/repos/", w), headers = auth)$get()
})

out <- AsyncQueue$new(.list = reqs, req_per_min = 30)
out
out$bucket_size
out$requests()
out$request()
out$responses()
}} # }