What is DwC?

dwctaxon is all about handling taxonomic data in the Darwin Core (DwC) taxon format.

But what is DwC?

DwC is a standard for biodiversity data

According to the official documentation,

Darwin Core is a standard maintained by the Darwin Core Maintenance Interest Group. It includes a glossary of terms (in other contexts these might be called properties, elements, fields, columns, attributes, or concepts) intended to facilitate the sharing of information about biological diversity by providing identifiers, labels, and definitions. Darwin Core is primarily based on taxa, their occurrence in nature as documented by observations, specimens, samples, and related information.

(emphasis added)

The “terms” referred to are typically encountered as columns in rectangular data (spreadsheets), such as scientificName (scientific name of a taxon), lifeStage (life stage of an organism when it was observed), etc. By providing a controlled vocabulary and clear definitions of terms, DwC greatly facilitates collection and sharing of biological data. For example, the Global Biodiversity Information Facility (GBIF), which synthesizes biodiversity data on a global scale, uses DwC.

In practice, a given set of DwC data are contained in an archive (zip file) including multiple spreadsheets (CSV files) and XML files with additional metadata. The spreadsheets typically include datasets like occurrences, taxonomy, and collection events.

DwC archive components, from https://github.com/gbif/ipt/ under the Apache license

While other parts of DwC such as organism and occurrence data are clearly important, they are out of the scope of dwctaxon, which only focuses on taxonomic data.

Features of the DwC taxon format

Most of the terms used in the DwC format for taxonomic data (“DwC taxon”) should be familiar to biologists. Here is a simple example mapping taxonomic data on the left to their DwC terms on the right for the genus Sarda:

Example terms in DwC taxon

However, there are some peculiarities that are good to be familiar with as follows.

Vertically oriented

The Linnaean system of taxonomy organizes taxa into a hierarchy, so we may be used to working with taxonomic data in “wide” format where each row is a species, like this:

species	genus	family	order
Crepidomanes minutum	Crepidomanes	Hymenophyllaceae	Hymenophyllales

Indeed, in DwC taxon, taxonomic levels above species like genus, family, and order are valid terms and may be used.

However, species is not a valid DwC term. That is because each row of a DwC taxonomic database is a single scientific name of any rank, not just species. So it is typical for data to be oriented vertically (“long” format):

taxonRank	scientificName
species	Crepidomanes minutum
genus	Crepidomanes
family	Hymenophyllaceae
order	Hymenophyllales

And since genus, family, etc. are valid DwC terms, these can also be included (when applicable):

taxonRank	scientificName	genus	family	order
species	Crepidomanes minutum	Crepidomanes	Hymenophyllaceae	Hymenophyllales
genus	Crepidomanes	Crepidomanes	Hymenophyllaceae	Hymenophyllales
family	Hymenophyllaceae	NA	Hymenophyllaceae	Hymenophyllales
order	Hymenophyllales	NA	NA	Hymenophyllales

Machine and human friendly

If you browse through the DwC taxon terms, you will notice many pairs of similar terms such as acceptedNameUsage and acceptedNameUsageID, parentNameUsage and parentNameUsageID, etc. These each are used for similar purposes, but one is a value that is easy for humans to understand while the other is useful for machines (computer programs).

For example, acceptedNameUsage is the accepted name of a synonym (e.g., Picea abies (L.) H. Karst as the accepted name of Pinus abies L.), and acceptedNameUsageID is the unique ID (typically, taxonID) of the accepted name (typically some short sequence of letters and numbers, but this depends on the dataset).

This makes the data format somewhat redundant, but it is also easier for a human to parse if they can see the actual accepted name of a synonym immediately, instead of having to look it up by taxonID. On the other hand, scientificName can include duplicates (in rare cases if the same name was published twice, etc.), so referring to an accepted name by its unique ID is safer and no problem for a computer.

We can see how this works in the example dataset that comes with dwctaxon, dct_filmies:

library(dwctaxon)
head(dct_filmies)
#> # A tibble: 6 × 5
#>   taxonID  acceptedNameUsageID taxonomicStatus taxonRank scientificName                            
#>   <chr>    <chr>               <chr>           <chr>     <chr>                                     
#> 1 54115096 NA                  accepted        species   Cephalomanes atrovirens Presl             
#> 2 54133783 54115097            synonym         species   Trichomanes crassum Copel.                
#> 3 54115097 NA                  accepted        species   Cephalomanes crassum (Copel.) M. G. Price 
#> 4 54133784 54115098            synonym         species   Trichomanes densinervium Copel.           
#> 5 54115098 NA                  accepted        species   Cephalomanes densinervium (Copel.) Copel. 
#> 6 54133786 54115100            synonym         species   Cephalomanes curvatum (J. Sm.) V. D. Bosch

Here, Trichomanes crassum Copel. is a synonym of Cephalomanes crassum (Copel.) M. G. Price (notice how the acceptedNameUsageID of Trichomanes crassum Copel. matches the taxonID of Cephalomanes crassum (Copel.) M. G. Price).

In this dataset, only acceptedNameUsageID is used, but it would be valid to add a column with acceptedNameUsage. To learn more about how to do so, please see the vignette("editing").

Extensible

There are many terms listed in the DwC taxon documentation – 37 by my count! However, it is unlikely a given taxonomic database uses all of them; in fact, most that I’ve encountered only use a subset of the terms, and there are none that are strictly required. But in practice you typically want at least scientificName (scientific name of the taxon, including author if known) and taxonID (a unique identifier for each row in the dataset).

Furthermore, some of the terms are likely to have restricted vocabularies. For example, a given dataset may only use a limited set of words to describe taxonomicStatus like “accepted”, “synonym”, and “doubtful”. This is in contrast to a term that could be (nearly) anything, like scientificName. DwC does not provide any official set of vocabularies; it is left to the database manager to determine that. One feature of dwctaxon is to verify that only the values you want to allow are used for a given term. To learn more about that, please see vignette("validation").

These qualities make the DwC taxon format flexible, so it can meet the needs of the dataset at hand. The dwctaxon functions try to provide sensible defaults, but they may need to be adjusted appropriately.