Skip to contents

The default output from an oa_fetch call is a tibble. This object type allows each row to be one unit of entity (article, institution, etc.), which is often helpful for downstream wrangling. It simplifies and combines complex output elements in list columns, which can be extracted or exploded with dplyr::rowwise or purrr::map.

IMPORTANT: in “flattening” the list output to a tibble, we have made a few decisions to simplify the original nested list such as retaining only one affiliation for each author if you’re querying Works (details in oa_df source code). While the tibble output is sufficient in most use cases, you may need to obtain the original nested list with all the information on an entity for your special research problem. If so, please specify output = "list" in your oa_fetch call.

Example 1: institutions

Suppose we queried Open Alex to obtain information on large Canadian institutions and now want to extract their latitudes and longitudes.

institutions <- oa_fetch(
  entity = "institutions",
  country_code = "CA",
  cited_by_count = ">4000000",
  verbose = TRUE,
  count_only = FALSE
)
#> Requesting url: https://api.openalex.org/institutions?filter=country_code%3ACA%2Ccited_by_count%3A%3E4000000
#> Getting 1 page of results with a total of 6 records...

head(institutions)
#> # A tibble: 6 × 21
#>   id                   display_name display_name_alterna…¹ display_name_acronyms
#>   <chr>                <chr>        <list>                 <list>               
#> 1 https://openalex.or… University … <chr [1]>              <lgl [1]>            
#> 2 https://openalex.or… University … <chr [1]>              <chr [1]>            
#> 3 https://openalex.or… McGill Univ… <chr [1]>              <lgl [1]>            
#> 4 https://openalex.or… University … <chr [2]>              <lgl [1]>            
#> 5 https://openalex.or… University … <chr [1]>              <chr [1]>            
#> 6 https://openalex.or… McMaster Un… <lgl [1]>              <lgl [1]>            
#> # ℹ abbreviated name: ¹​display_name_alternatives
#> # ℹ 17 more variables: display_name_international <list>, ror <chr>,
#> #   ids <list>, country_code <chr>, geo <list>, type <chr>, homepage_url <chr>,
#> #   image_url <chr>, image_thumbnail_url <chr>, associated_institutions <list>,
#> #   works_count <int>, cited_by_count <int>, counts_by_year <list>,
#> #   works_api_url <chr>, x_concepts <list>, updated_date <chr>,
#> #   created_date <chr>

We present three different approaches below.

dplyr::rowwise

The use of rowwise used to be discouraged, but the tidyverse team has now recognized its importance in many workflows, and so rowwise is here to stay. We think rowwise pairs very naturally with our list columns output.

institutions %>%
  rowwise() %>%
  mutate(
    name = display_name,
    openalex = stringr::str_extract(id, "I\\d+"),
    lat = geo$latitude,
    lon = geo$longitude,
    .keep = "none"
  )
#> # A tibble: 6 × 4
#> # Rowwise: 
#>   name                           openalex     lat    lon
#>   <chr>                          <chr>      <dbl>  <dbl>
#> 1 University of Toronto          I185261750  43.7  -79.4
#> 2 University of British Columbia I141945490  49.2 -123. 
#> 3 McGill University              I5023651    45.5  -73.6
#> 4 University of Alberta          I154425047  53.6 -113. 
#> 5 University of Calgary          I168635309  51.1 -114. 
#> 6 McMaster University            I98251732   43.3  -79.8

purrr::map

Alternatively, you can use any function in the purrr::map family. As you can see below, the syntax is a little less natural, but you may gain some performance advantage if you have a really large dataframe.

institutions %>%
  mutate(
    name = display_name,
    openalex = stringr::str_extract(id, "I\\d+"),
    lat = purrr::map_dbl(geo, ~ .x$latitude),
    lon = purrr::map_dbl(geo, ~ .x$longitude),
    .keep = "none"
  )
#> # A tibble: 6 × 4
#>   name                           openalex     lat    lon
#>   <chr>                          <chr>      <dbl>  <dbl>
#> 1 University of Toronto          I185261750  43.7  -79.4
#> 2 University of British Columbia I141945490  49.2 -123. 
#> 3 McGill University              I5023651    45.5  -73.6
#> 4 University of Alberta          I154425047  53.6 -113. 
#> 5 University of Calgary          I168635309  51.1 -114. 
#> 6 McMaster University            I98251732   43.3  -79.8

base::lapply

Similar to the purrr approach, you can use the base functions in the lapply family, for example:

institutions %>%
  mutate(
    name = display_name,
    openalex = stringr::str_extract(id, "I\\d+"),
    lat = vapply(geo, function(x) x$latitude, numeric(1)),
    lon = vapply(geo, function(x) x$longitude, numeric(1)),
    .keep = "none"
  )
#> # A tibble: 6 × 4
#>   name                           openalex     lat    lon
#>   <chr>                          <chr>      <dbl>  <dbl>
#> 1 University of Toronto          I185261750  43.7  -79.4
#> 2 University of British Columbia I141945490  49.2 -123. 
#> 3 McGill University              I5023651    45.5  -73.6
#> 4 University of Alberta          I154425047  53.6 -113. 
#> 5 University of Calgary          I168635309  51.1 -114. 
#> 6 McMaster University            I98251732   43.3  -79.8

Example 2: works

Suppose we have a tibble of works output and would like to find the institutions corresponding with the works’ authors. In this case, each work may have more than one affiliated institutions.

Tibble output

Assuming that each author is affiliated with only one institution, we can call oa_fetch with the default output = "tibble":

works <- oa_fetch(
  entity = "works",
  title.search = c("bibliometric analysis", "science mapping"),
  cited_by_count = ">100",
  from_publication_date = "2020-01-01",
  to_publication_date = "2021-01-31",
  options = list(sort = "cited_by_count:desc"),
  count_only = FALSE
)

We will store the result in a list column:

multi_insts <- works %>%
  rowwise() %>%
  mutate(
    openalex = stringr::str_extract(id, "W\\d+"),
    institutions = list(unique(author$institution_display_name)),
    .keep = "none"
  )

multi_insts
#> # A tibble: 31 × 2
#> # Rowwise: 
#>    openalex    institutions
#>    <chr>       <list>      
#>  1 W3038273726 <chr [2]>   
#>  2 W2990450011 <chr [2]>   
#>  3 W3044902155 <chr [1]>   
#>  4 W3001491100 <chr [3]>   
#>  5 W2990688366 <chr [3]>   
#>  6 W3042215340 <chr [3]>   
#>  7 W3000049009 <chr [1]>   
#>  8 W2986617680 <chr [2]>   
#>  9 W3000910650 <chr [3]>   
#> 10 W3038187379 <chr [9]>   
#> # ℹ 21 more rows

# institutions of the first work
str(multi_insts[1, "institutions"])
#> rowws_df [1 × 1] (S3: rowwise_df/tbl_df/tbl/data.frame)
#>  $ institutions:List of 1
#>   ..$ : chr [1:2] "University of Southern Denmark" "BI Norwegian Business School"
#>  - attr(*, "groups")= tibble [1 × 1] (S3: tbl_df/tbl/data.frame)
#>   ..$ .rows: list<int> [1:1] 
#>   .. ..$ : int 1
#>   .. ..@ ptype: int(0)

List output

If we want to get all the institutions that the authors of these works are affiliated with (since one author may be affiliated with more than one institution), you would want to work with the list output.

As you can see, the nested list can be more convoluted to work with:

works_list <- oa_fetch(
  entity = "works",
  title.search = c("bibliometric analysis", "science mapping"),
  cited_by_count = ">100",
  from_publication_date = "2020-01-01",
  to_publication_date = "2021-01-31",
  options = list(sort = "cited_by_count:desc"),
  output = "list"
)

work_authors <- lapply(works_list, \(x) x$author)

unique_insts <- sapply(
  work_authors,
  \(z) unique(unlist(
    sapply(
      z, \(y) sapply(y$institutions, \(x) x$display_name)
    )
  ))
)

unique_insts[[1]]
#> [1] "University of Southern Denmark" "BI Norwegian Business School"