Skip to contents

Design History and Architecture

Source: DESIGN.md

This note documents why brapiR2 has its current structure. It is written for rOpenSci review and future maintenance.

Design History

brapiR2 was built to fill a gap in the R ecosystem for the Breeding API (BrAPI) v2 specification. The existing R client, QBMS, is stateful and menu-driven: it holds a “current” program, trial, or study internally and walks the user through an interactive selection flow. That design fits exploratory, console-driven use, but it does not fit scripted pipelines, parallel batch jobs, or reproducible analysis code, where the caller wants to pass an explicit connection and get a tibble back nothing hidden, nothing stateful.

brapiR2 was developed with Claude Code assistance. The author directed the architecture, the public API design, and the domain logic (which BrAPI modules to cover, how pagination and async search should behave, how genotype calls should be encoded as dosages); the assistant implemented those decisions and iterated against the live public BrAPI test server (https://test-server.brapi.org) to validate real request/response shapes rather than only against synthetic fixtures. The git history for this repository is comparatively compressed relative to the amount of surface area covered, which reflects that AI-assisted development pattern: whole modules were scaffolded and corrected against live server output in single passes rather than built up commit-by-commit over weeks.

Before rOpenSci submission, the package went through a review-driven cleanup pass: every exported function gained a runnable example (verified against the live test server), \dontrun{} was replaced package-wide with \donttest{} since the examples are not broken, only network-dependent, goodpractice/lintr issues were fixed (function length, line length, duplicated parameter docs via @inheritParams, assignment style), and test coverage was raised from roughly 65% to 98% by adding mocked unit tests alongside the existing live-server integration tests.

Architecture Overview

brapiR2 is organised in layers, from the caller’s perspective inward:

  1. Connection and auth - build and hold connection state explicitly (brapi_connection(), brapi_login(), brapi_login_oauth2(), brapi_set_token()). Nothing is stored globally.
  2. Request engine - a small set of internal functions that every exported endpoint function is built on top of. This layer owns HTTP request construction, transparent pagination, the BrAPI async-search poll protocol, and response-to-tibble parsing.
  3. Four BrAPI modules - Core, Germplasm, Phenotyping, and Genotyping. Each module is a thin, one-function-per-endpoint wrapper around the request engine, matching the four modules defined by the BrAPI v2 specification itself.
  4. Convenience functions - a small number of higher-level functions (brapi_study_data(), brapi_get_dosage_matrix(), brapi_get_marker_map()) that compose module functions into analysis-ready shapes (a wide phenotyping tibble, a numeric dosage matrix) without introducing any new HTTP behaviour of their own.
  5. Caching and parallel fetching - an optional, opt-in disk cache (brapi_cache_enable(), brapi_cache_clear()) and a parallel batch fetch helper (brapi_fetch_parallel()) layered on top of the request engine, both off by default.

Component Map

Component Files Responsibility
Connection & validation R/connection.R brapi_connection(), print.brapi_con(), is_brapi_con(), validate_con() — build, print, and validate the stateless connection object every other function takes as its first argument
Authentication R/auth.R brapi_login(), brapi_login_oauth2(), brapi_set_token() - populate the connection’s Bearer token via password grant, OAuth2 client-credentials grant, or direct assignment
Request engine R/request.R brapi_req(), brapi_get(), brapi_get_pages(), brapi_cache_path(), brapi_cache_read(), brapi_post_search(), brapi_poll_search(), parse_brapi_result() - shared HTTP plumbing: headers/auth/retry, GET pagination, cache key/lookup, the POST-search 200/202-poll protocol, and list-to-tibble parsing
Core module R/core.R Programs, trials, studies, locations, seasons, lists, people, server info
Germplasm module R/germplasm.R Germplasm records, progeny, attributes, crosses, crossing projects, seed lots, germplasm search; pedigree via both the single-germplasm sub-resource (brapi_germplasm_pedigree()) and the batch Pedigree entity (brapi_pedigree(), brapi_search_pedigree())
Phenotyping module R/phenotyping.R Observation units, observations, observation variables, traits, scales, methods, ontologies, images, events, phenotyping search, and brapi_study_data() (wide-format pivot)
Genotyping module R/genotyping.R Samples, variants, variant sets, calls, call sets, references, reference sets, the allele matrix, genotyping search, and the brapi_get_dosage_matrix() / brapi_get_marker_map() convenience functions
Genome Maps R/genome_maps.R Genome maps, linkage groups, and marker positions (brapi_maps(), brapi_map(), brapi_map_linkage_groups(), brapi_marker_positions(), brapi_search_marker_positions()) - a distinct entity from Variants: a marker’s position on a named map (genetic or physical) rather than a variant’s position on a reference assembly
Caching R/cache.R brapi_cache_enable(), brapi_cache_clear() - opt-in disk cache configuration and invalidation, keyed via rlang::hash()
Parallel fetching R/cache.R brapi_fetch_parallel() - furrr/future-based batch fetching across many IDs
Utilities R/utils.R brapi_ping(), brapi_endpoints() - connectivity check and supported-endpoint introspection
Package doc R/brapiR2-package.R Package-level roxygen imports and the shared @inheritParams documentation template

Main Design Decisions

Stateless connection objects, not global state

QBMS and similar BrAPI clients keep a mutable “current server” in package state. brapiR2 instead passes an explicit brapi_con object as the first argument to every function, the way httr2, DBI, and most tidyverse-style clients do. This makes it safe to hold multiple connections (e.g. two breeding programs, or a cached and an uncached connection to the same server) in the same session, and it makes functions easy to test in isolation and safe to run in parallel workers.

Tibbles everywhere, not raw lists

Every exported function that returns data returns a tibble, never a raw parsed JSON list. BrAPI responses are deeply nested and inconsistently shaped across servers; parse_brapi_result() centralizes the list-of-records-to-tibble conversion (including the list-column fallback for genuinely nested fields) so callers can pipe directly into dplyr/ tidyr without ever touching $result$data themselves.

One function per BrAPI endpoint

Rather than one generic brapi_call(entity, ...) dispatcher, each BrAPI endpoint gets its own named function (brapi_programs(), brapi_trials(), brapi_germplasm(), …). This costs more exported functions, but each one is individually documented, individually testable, and discoverable via autocomplete which matters more for a spec with dozens of endpoints than a small parameter saving would.

Transparent pagination

BrAPI list endpoints are always paginated server-side. brapi_get() / brapi_get_pages() walk every page automatically and return the fully concatenated result, so callers never have to reason about page or pageSize unless they want to override the default page size.

Async search handled internally

BrAPI’s /search/* endpoints can respond either immediately (200, with data) or asynchronously (202, with a searchResultsDbId to poll). Both paths are collapsed into a single synchronous return value by brapi_post_search() / brapi_poll_search(), so brapi_search_germplasm() and friends behave identically to the immediate-result endpoints from the caller’s point of view.

httr2, not httr

httr2 is the actively developed successor to httr, with a pipeable request-builder API, built-in retry/backoff, and clearer error objects. All of brapiR2’s request-engine internals (mockable in tests via local_mocked_bindings()) are built on httr2.

Genotyping module as the key differentiator

Core, Germplasm, and Phenotyping coverage exist in other BrAPI clients. Handling the allele matrix’s non-standard 2D pagination, and converting it via brapi_get_dosage_matrix() / brapi_get_marker_map() into genomic-selection-ready numeric matrices, does not - and is the main reason brapiR2 exists as a separate package rather than a QBMS contribution. This is a depth claim about the genotyping data path specifically, not a claim that brapiR2 covers the Genotyping module (or any module) exhaustively; see the coverage figures in Boundaries.

Disk-based caching keyed with rlang::hash()

Caching is opt-in (brapi_cache_enable()), not automatic, so default behaviour always reflects the live server. When enabled, each cache entry is keyed by a hash of the fully-qualified URL plus sorted, page-excluded query parameters, so identical requests (including identical filter arguments in a different order) reliably hit the cache and different requests never collide.

The caller owns the future plan

brapi_fetch_parallel() runs against whatever future plan is already active and never sets one itself. The future package’s best-practices guidance is explicit that the choice of backend belongs to the caller, not to a package: a function that quietly sets and restores a plan on every call still mutates session-wide state the caller did not ask it to touch, and can silently replace a plan configured deliberately - a specific worker count, a cluster spanning several machines, callr workers for extra isolation. The .workers argument in 0.1.0 did exactly this, and is deprecated; supplying it now warns and has no effect.

Mocked tests alongside integration tests

The test suite pairs two layers: tests/testthat/test-*-mocked.R files patch httr2’s req_perform()/resp_body_json()/resp_status() (and future/furrr for parallel fetching) via local_mocked_bindings() so every code path — including pagination edge cases, async-search polling, and cache hit/miss/expiry - runs deterministically without the live server; test-integration.R and test-cache-integration.R separately exercise the real test server and are skipped on CRAN and when offline. Together they bring line coverage to about 98%.

Boundaries

The current design deliberately leaves several things out of scope:

  • Read-only. brapiR2 retrieves BrAPI data; it does not implement the BrAPI POST/PUT endpoints for writing germplasm, observations, or other records back to a server. Of the BrAPI v2.1 specification’s 201 endpoints (114 GET, 57 POST - 24 of them search, 30 PUT), the 63 write and update endpoints are out of scope by design, not an oversight.
  • Read coverage is intentionally partial, not exhaustive. brapiR2 wraps 32 of the specification’s 37 top-level entities across all four modules, covering 49 of the 138 retrieval (GET + search POST) endpoints - 49 of 122 endpoints within the entities it does cover. Common Crop Names, Germplasm Attribute Values, Planned Crosses, Plates, and Vendor Samples (lab/vendor order-tracking) currently have no coverage at all. New entities are added as real analysis needs surface, not to chase 100%.
  • No analysis. Dosage matrices and wide phenotyping tibbles are produced in shapes that downstream genomic-selection and analysis packages expect, but brapiR2 does not itself fit models, compute BLUPs, or perform GWAS.
  • No server-specific workarounds. brapiR2 targets the BrAPI v2 specification as written. It does not carry bespoke branches for particular server implementations (BMS, Breedbase, EBS, GIGWA, Germinate); servers that deviate from spec are expected to be fixed upstream rather than special-cased here.
  • No visualisation. Plotting genotype, pedigree, or phenotype data is left to downstream packages that consume brapiR2’s tibbles.

Relationship to Other Tools

Other BrAPI R clients

QBMS, brapir-v2 (see Design History), and BrAPI.R (David Waring, Cornell) all target the same specification from different angles. BrAPI.R in particular is complementary rather than competing: its own DESCRIPTION calls it “simple wrapper functions for httr that make it easier to make manual HTTP calls to a BrAPI server”, and its README states plainly that it “does not have any knowledge of the currently supported BrAPI endpoints.” It is a transport layer - callers pass an endpoint path as a string and get back a raw nested list; it supports GET, POST, and PUT (with a version argument covering both BrAPI v1 and v2), plus a two-step search helper and Breedbase-specific functions explicitly outside the BrAPI spec. brapiR2 takes the opposite position on the same trade-off: named, individually documented functions per endpoint that always return tibbles, at the cost of needing an explicit function for each entity. The two are usable together rather than as alternatives - and BrAPI.R’s POST/PUT support covers exactly the write operations brapiR2 deliberately leaves out (see Boundaries).

Downstream analysis tools

brapiR2 sits at the start of a breeding-data analysis pipeline, not in place of the tools downstream of it. Its job ends at a tidy tibble or matrix retrieved from a BrAPI server; cleaning, visualisation and modelling belong to packages built for those tasks. The genomic selection and multi-environment articles show that boundary in practice, handing brapiR2’s output to rrBLUP, BGLR, sommer, metan and lme4.

Maintenance Considerations

The most likely sources of future maintenance work, roughly in order of likelihood:

  • BrAPI specification updates. BrAPI v2 is still evolving; new optional fields, endpoints, or search parameters should mostly be additive and land as new module functions or new ... pass-through arguments, not breaking changes to existing signatures.
  • Server response drift. Real BrAPI servers do not always implement the spec identically (see brapi_study_data()’s client-side filter fallback, and brapi_get_marker_map()’s variantName/variantNames handling). New drift should be handled the same way: defensively, in the module or convenience function affected, without changing the request engine.
  • httr2 changes. The request engine (R/request.R) is the sole integration point with httr2; API changes there should only ever require edits in that one file.
  • Test server availability. \donttest{} examples and the integration test suite depend on https://test-server.brapi.org staying reachable and populated with its current dummy dataset (program1, study1, variantset1, and similar fixture IDs referenced throughout the examples and tests). If that server’s dataset changes shape, the mocked test suite still protects the request-engine logic, but examples and integration tests would need their hardcoded IDs updated.