About the tibble output
Source:vignettes/articles/About-the-tibble-output.Rmd
About-the-tibble-output.RmdThe default output from an oa_fetch call is a tibble. This object type allows each row to be one unit of entity (article, institution, etc.), which is often helpful for downstream wrangling. It simplifies and combines complex output elements in list columns, which can be extracted or exploded with dplyr::rowwise or purrr::map.
IMPORTANT: in “flattening” the list output to a tibble, we have made a few decisions to simplify the original nested list such as retaining only one affiliation for each author if you’re querying Works (details in oa_df source code). While the tibble output is sufficient in most use cases, you may need to obtain the original nested list with all the information on an entity for your special research problem. If so, please specify output = "list" in your oa_fetch call.
Example 1: institutions
Suppose we queried Open Alex to obtain information on large Canadian institutions and now want to extract their latitudes and longitudes.
institutions <- oa_fetch(
entity = "institutions",
country_code = "CA",
cited_by_count = ">4000000",
verbose = TRUE,
count_only = FALSE
)
#> Requesting url: https://api.openalex.org/institutions?filter=country_code%3ACA%2Ccited_by_count%3A%3E4000000
#> Getting 1 page of results with a total of 5 records...
head(institutions)
#> # A tibble: 5 × 21
#> id display_name display_name_alterna…¹ display_name_acronyms international
#> <chr> <chr> <list> <list> <list>
#> 1 https… University … <lgl [1]> <lgl [1]> <df>
#> 2 https… University … <lgl [1]> <chr [1]> <df [1 × 74]>
#> 3 https… McGill Univ… <lgl [1]> <lgl [1]> <df [1 × 89]>
#> 4 https… University … <chr [1]> <lgl [1]> <df [1 × 61]>
#> 5 https… McMaster Un… <lgl [1]> <lgl [1]> <df [1 × 52]>
#> # ℹ abbreviated name: ¹display_name_alternatives
#> # ℹ 16 more variables: ror <chr>, ids <list>, country_code <chr>, geo <list>,
#> # type <chr>, homepage_url <chr>, image_url <chr>, image_thumbnail_url <chr>,
#> # associated_institutions <list>, works_count <int>, cited_by_count <int>,
#> # counts_by_year <list>, works_api_url <chr>, x_concepts <list>,
#> # updated_date <chr>, created_date <chr>We present three different approaches below.
dplyr::rowwise
The use of rowwise used to be discouraged, but the tidyverse team has now recognized its importance in many workflows, and so rowwise is here to stay. We think rowwise pairs very naturally with our list columns output.
institutions %>%
rowwise() %>%
mutate(
name = display_name,
openalex_id = stringr::str_extract(id, "I\\d+"),
lat = geo$latitude,
lon = geo$longitude,
.keep = "none"
)
#> # A tibble: 5 × 4
#> # Rowwise:
#> name openalex_id lat lon
#> <chr> <chr> <dbl> <dbl>
#> 1 University of Toronto I185261750 43.7 -79.4
#> 2 University of British Columbia I141945490 49.3 -123.
#> 3 McGill University I5023651 45.5 -73.6
#> 4 University of Alberta I154425047 53.5 -114.
#> 5 McMaster University I98251732 43.3 -79.9purrr::map
Alternatively, you can use any function in the purrr::map family. As you can see below, the syntax is a little less natural, but you may gain some performance advantage if you have a really large dataframe.
institutions %>%
mutate(
name = display_name,
openalex_id = stringr::str_extract(id, "I\\d+"),
lat = purrr::map_dbl(geo, ~ .x$latitude),
lon = purrr::map_dbl(geo, ~ .x$longitude),
.keep = "none"
)
#> # A tibble: 5 × 4
#> name openalex_id lat lon
#> <chr> <chr> <dbl> <dbl>
#> 1 University of Toronto I185261750 43.7 -79.4
#> 2 University of British Columbia I141945490 49.3 -123.
#> 3 McGill University I5023651 45.5 -73.6
#> 4 University of Alberta I154425047 53.5 -114.
#> 5 McMaster University I98251732 43.3 -79.9base::lapply
Similar to the purrr approach, you can use the base functions in the lapply family, for example:
institutions %>%
mutate(
name = display_name,
openalex_id = stringr::str_extract(id, "I\\d+"),
lat = vapply(geo, function(x) x$latitude, numeric(1)),
lon = vapply(geo, function(x) x$longitude, numeric(1)),
.keep = "none"
)
#> # A tibble: 5 × 4
#> name openalex_id lat lon
#> <chr> <chr> <dbl> <dbl>
#> 1 University of Toronto I185261750 43.7 -79.4
#> 2 University of British Columbia I141945490 49.3 -123.
#> 3 McGill University I5023651 45.5 -73.6
#> 4 University of Alberta I154425047 53.5 -114.
#> 5 McMaster University I98251732 43.3 -79.9Example 2: works
Suppose we have a tibble of works output and would like to find the institutions corresponding with the works’ authors. In this case, each work may have more than one affiliated institutions.
Tibble output
Assuming that each author is affiliated with only one institution, we can call oa_fetch with the default output = "tibble":
works <- oa_fetch(
entity = "works",
title.search = c("bibliometric analysis", "science mapping"),
cited_by_count = ">100",
from_publication_date = "2020-01-01",
to_publication_date = "2021-01-31",
options = list(sort = "cited_by_count:desc"),
count_only = FALSE
)We will store the result in a list column:
multi_insts <- works %>%
rowwise() %>%
mutate(
openalex_id = stringr::str_extract(id, "W\\d+"),
institutions = list(unique(author$institution_display_name)),
.keep = "none"
)
multi_insts
#> # A tibble: 16 × 2
#> # Rowwise:
#> openalex_id institutions
#> <chr> <list>
#> 1 W3038273726 <lgl [1]>
#> 2 W2990450011 <chr [2]>
#> 3 W3001491100 <chr [3]>
#> 4 W3044902155 <chr [1]>
#> 5 W3042215340 <lgl [1]>
#> 6 W2990688366 <chr [3]>
#> 7 W3000049009 <chr [1]>
#> 8 W3000910650 <chr [3]>
#> 9 W3011866596 <lgl [1]>
#> 10 W2986617680 <chr [2]>
#> 11 W2999558594 <chr [2]>
#> 12 W3038187379 <chr [10]>
#> 13 W3025370095 <lgl [1]>
#> 14 W3005144120 <chr [2]>
#> 15 W2981769453 <chr [1]>
#> 16 W3010639929 <chr [2]>
# institutions of the first work
str(multi_insts[1, "institutions"])
#> rowws_df [1 × 1] (S3: rowwise_df/tbl_df/tbl/data.frame)
#> $ institutions:List of 1
#> ..$ : logi NA
#> - attr(*, "groups")= tibble [1 × 1] (S3: tbl_df/tbl/data.frame)
#> ..$ .rows: list<int> [1:1]
#> .. ..$ : int 1
#> .. ..@ ptype: int(0)List output
If we want to get all the institutions that the authors of these works are affiliated with (since one author may be affiliated with more than one institution), you would want to work with the list output.
As you can see, the nested list can be more convoluted to work with:
works_list <- oa_fetch(
entity = "works",
title.search = c("bibliometric analysis", "science mapping"),
cited_by_count = ">100",
from_publication_date = "2020-01-01",
to_publication_date = "2021-01-31",
options = list(sort = "cited_by_count:desc"),
output = "list"
)
work_authors <- lapply(works_list, \(x) x$author)
unique_insts <- sapply(
work_authors,
\(z) unique(unlist(
sapply(
z, \(y) sapply(y$institutions, \(x) x$display_name)
)
))
)
unique_insts[[1]]
#> NULL