An async client to do many requests, each with different URLs, curl options, etc.
Value
An object of class AsyncVaried with variables and methods.
HttpResponse objects are returned in the order they are passed in.
We print the first 10.
Failure behavior
HTTP requests mostly fail in ways that you are probably familiar with, including when there's a 400 response (the URL not found), and when the server made a mistake (a 500 series HTTP status code).
But requests can fail sometimes where there is no HTTP status code, and no agreed upon way to handle it other than to just fail immediately.
When a request fails when using synchronous requests (see HttpClient) you get an error message that stops your code progression immediately saying for example:
"Could not resolve host: https://foo.com"
"Failed to connect to foo.com"
"Resolving timed out after 10 milliseconds"
However, for async requests we don't want to fail immediately because that would stop the subsequent requests from occurring. Thus, when we find that a request fails for one of the reasons above we give back a HttpResponse object just like any other response, and:
capture the error message and put it in the
contentslot of the response object (thus calls tocontentandparse()work correctly)give back a
0HTTP status code. we handle this specially when testing whether the request was successful or not with e.g., thesuccess()method
R6 classes
This is an R6 class from the package R6. Find out more
about R6 at https://r6.r-lib.org/. After creating an instance of an R6
class (e.g., x <- HttpClient$new(url = "https://hb.opencpu.org")) you can
access values and methods on the object x.
See also
Other async:
Async,
AsyncQueue,
HttpRequest
Methods
Method print()
print method for AsyncVaried objects
Method new()
Create a new AsyncVaried object
Usage
AsyncVaried$new(..., .list = list())Arguments
..., .listAny number of objects of class
HttpRequest(), must supply inputs to one of these parameters, but not both
Method responses()
List responses
Details
An S3 print method is used to summarise results. unclass
the output to see the list, or index to results, e.g., [1], [1:3]
Method parse()
parse content
Examples
if (FALSE) { # \dontrun{
# pass in requests via ...
req1 <- HttpRequest$new(
url = "https://hb.opencpu.org/get",
opts = list(verbose = TRUE),
headers = list(foo = "bar")
)$get()
req2 <- HttpRequest$new(url = "https://hb.opencpu.org/post")$post()
# Create an AsyncVaried object
out <- AsyncVaried$new(req1, req2)
# before you make requests, the methods return empty objects
out$status()
out$status_code()
out$content()
out$times()
out$parse()
out$responses()
# make requests
out$request()
# access various parts
## http status objects
out$status()
## status codes
out$status_code()
## content (raw data)
out$content()
## times
out$times()
## parsed content
out$parse()
## response objects
out$responses()
# use $verb() method to select http verb
method <- "post"
req1 <- HttpRequest$new(
url = "https://hb.opencpu.org/post",
opts = list(verbose = TRUE),
headers = list(foo = "bar")
)$verb(method)
req2 <- HttpRequest$new(url = "https://hb.opencpu.org/post")$verb(method)
out <- AsyncVaried$new(req1, req2)
out
out$request()
out$responses()
# pass in requests in a list via .list param
reqlist <- list(
HttpRequest$new(url = "https://hb.opencpu.org/get")$get(),
HttpRequest$new(url = "https://hb.opencpu.org/post")$post(),
HttpRequest$new(url = "https://hb.opencpu.org/put")$put(),
HttpRequest$new(url = "https://hb.opencpu.org/delete")$delete(),
HttpRequest$new(url = "https://hb.opencpu.org/get?g=5")$get(),
HttpRequest$new(
url = "https://hb.opencpu.org/post")$post(body = list(y = 9)),
HttpRequest$new(
url = "https://hb.opencpu.org/get")$get(query = list(hello = "world"))
)
out <- AsyncVaried$new(.list = reqlist)
out$request()
out$status()
out$status_code()
out$content()
out$times()
out$parse()
# using auth with async
url <- "https://hb.opencpu.org/basic-auth/user/passwd"
auth <- auth(user = "user", pwd = "passwd")
reqlist <- list(
HttpRequest$new(url = url, auth = auth)$get(),
HttpRequest$new(url = url, auth = auth)$get(query = list(a=5)),
HttpRequest$new(url = url, auth = auth)$get(query = list(b=3))
)
out <- AsyncVaried$new(.list = reqlist)
out$request()
out$status()
out$parse()
# failure behavior
## e.g. when a URL doesn't exist, a timeout, etc.
reqlist <- list(
HttpRequest$new(url = "http://stuffthings.gvb")$get(),
HttpRequest$new(url = "https://hb.opencpu.org")$head(),
HttpRequest$new(url = "https://hb.opencpu.org",
opts = list(timeout_ms = 10))$head()
)
(tmp <- AsyncVaried$new(.list = reqlist))
tmp$request()
tmp$responses()
tmp$parse("UTF-8")
# access intemediate redirect headers
dois <- c("10.7202/1045307ar", "10.1242/jeb.088898", "10.1121/1.3383963")
reqlist <- list(
HttpRequest$new(url = paste0("https://doi.org/", dois[1]))$get(),
HttpRequest$new(url = paste0("https://doi.org/", dois[2]))$get(),
HttpRequest$new(url = paste0("https://doi.org/", dois[3]))$get()
)
tmp <- AsyncVaried$new(.list = reqlist)
tmp$request()
tmp
lapply(tmp$responses(), "[[", "response_headers_all")
# retry
reqlist <- list(
HttpRequest$new(url = "https://hb.opencpu.org/get")$get(),
HttpRequest$new(url = "https://hb.opencpu.org/post")$post(),
HttpRequest$new(url = "https://hb.opencpu.org/status/404")$retry("get"),
HttpRequest$new(url = "https://hb.opencpu.org/status/429")$retry("get",
retry_only_on = c(403, 429), times = 2)
)
tmp <- AsyncVaried$new(.list = reqlist)
tmp
tmp$request()
tmp$responses()[[3]]
} # }