Skip to contents

Background

The Cold Spring Harbour Laboratory API provides a direct interface to the medRxiv and bioRxiv databases. However, the API does not allow you to perform searches, instead providing two endpoints that return either all content between two specified dates or all information held on a particular DOI.

medrxivr provides two convenience functions for importing the data provided by these endpoints in R: mx_api_content() and mx_api_doi(). The results of either function can then be passed to mx_search() for searching.

By date range (mx_api_content())

The format of this endpoint is https://api.biorxiv.org/details/[server]/[interval]/[cursor] where ‘interval’ must be two YYYY-MM-DD dates separated by ‘/’. Where metadata for multiple papers is returned, results are paginated with 100 papers served in a call. The ‘cursor’ value can be used to iterate through the result.

mx_api_content() automatically moves through the pages for you, capturing all records returned by the endpoint and returning them as an R object. For instance, https://api.biorxiv.org/details/medrxiv/2020-01-01/2020-01-31/0 will output 100 results (if that many remain) within the date range of 2020-01-01 to 2020-01-31 beginning from result 1. To import this into R as a dataframe:

medrxiv_data <- mx_api_content(from_date = "2020-01-01", 
                               to_date = "2020-01-05")
#> Estimated total number of records as per API metadata: 33
#> Number of records retrieved from API: 33


biorxiv_data <- mx_api_content(server = "biorxiv",
                               from_date = "2020-01-01", 
                               to_date = "2020-01-05")
#> Estimated total number of records as per API metadata: 286
#> Number of records retrieved from API: 286

By DOI (mx_api_doi())

https://api.biorxiv.org/details/[server]/[DOI] returns detail for a single manuscript. For instance, https://api.biorxiv.org/details/medrxiv/10.1101/2020.02.25.20021568 will output metadata for the medRxiv paper with DOI 10.1101/2020.02.25.20021568. To import the results from this endpoint into R as a dataframe:

mx_api_doi(doi = "10.1101/2020.02.25.20021568")
#> # A tibble: 2 × 15
#>   doi        title authors autho…¹ autho…² date  version license categ…³ jatsxml
#>   <chr>      <chr> <chr>   <chr>   <chr>   <chr> <chr>   <chr>   <chr>   <chr>  
#> 1 10.1101/2… Deep… Chen, … Hongga… Renmin… 2020… 1       cc_by_… Infect… https:…
#> 2 10.1101/2… Deep… Chen, … Hongga… Renmin… 2020… 2       cc_by_… Infect… https:…
#> # … with 5 more variables: abstract <chr>, published <lgl>, node <int>,
#> #   link_page <chr>, link_pdf <chr>, and abbreviated variable names
#> #   ¹​author_corresponding, ²​author_corresponding_institution, ³​category

Accessing the raw API data

Both functions contain a clean argument with is set to TRUE by default. This is to ensure that the datasets returned by the mx_api_*() functions can immediately be passed to mx_search(). However, there may be occasions where this is not required, and so setting this argument to FALSE will return the raw data provided by the API endpoints. For example:

mx_api_content(to_date = "2019-07-01", clean = FALSE)
#> Estimated total number of records as per API metadata: 32
#> Number of records retrieved from API: 32
#> # A tibble: 32 × 14
#>    doi         title authors autho…¹ autho…² date  version type  license categ…³
#>    <chr>       <chr> <chr>   <chr>   <chr>   <chr> <chr>   <chr> <chr>   <chr>  
#>  1 10.1101/19… Mole… Daniel… Robert… "Depar… 2019… 1       ""    cc_by   geneti…
#>  2 10.1101/19… Croh… Orna G… Orna G… "Crohn… 2019… 1       ""    cc_by_… gastro…
#>  3 10.1101/19… Upda… Joshua… Joshua… "Yale … 2019… 1       ""    cc_by_… cardio…
#>  4 10.1101/19… Pred… Oliver… Oliver… "Insti… 2019… 1       ""    cc_by   health…
#>  5 10.1101/19… Pros… Nathan… Nathan… "Duke … 2019… 1       ""    cc_by_… health…
#>  6 10.1101/19… Tren… Brian … Ben Go… "Unive… 2019… 1       ""    cc_by   health…
#>  7 10.1101/19… 18F-… Nicola… Nicola… "Unive… 2019… 1       ""    cc_by_… neurol…
#>  8 10.1101/19… Perc… Sistan… Lena H… "Emory… 2019… 1       ""    cc_by   neurol…
#>  9 10.1101/19… Prox… Tesfa … Tesfa … "Depar… 2019… 1       ""    cc_by_… nutrit…
#> 10 10.1101/19… Tran… Alexan… Alexan… "APHP"  2019… 1       ""    cc_no   oncolo…
#> # … with 22 more rows, 4 more variables: jatsxml <chr>, abstract <chr>,
#> #   published <chr>, server <chr>, and abbreviated variable names
#> #   ¹​author_corresponding, ²​author_corresponding_institution, ³​category