
Debugging the package check API
Mark Padgham
2026-11-16
Source:vignettes/debugging.Rmd
debugging.RmdThe main API endpoint for the Digital Ocean server is the editorcheck,
called by the ropensci-review-bot on every submission, and
also manually (re-)triggered by the command
@ropensci-review-bot check package. This vignette describes
procedures which may be used to debug any instances in which package
checks fail.
Most failures at the server end will deliver HTTP error codes of
“500” like
this example. This code may indicate that the server is
currently unavailable, which will arise once per week during rebuild and
redeploy processes, scheduled every Monday at 23:59 UTC. If a submission
happens to be roughly at this time, the recommended procedure is to wait
at least an hour before manually trying
@ropensci-review-bot check package. Other 500 codes should
be reported straight to maintainers of the check system, currently to @mpadge and @noamross as respective first and
second points of contact. This vignette describes the procedures they
would follow to debug any problems.
Debugging generally requires stepping through the code as called by
the main editorcheck
endpoint, and into the editor_check()
function called by the endpoint as a background process.
Debugging Procedure
Check that the bot received the command
The bot itself runs on Heroku, so that is the first place to look when the bot fails to respond. The Heroku logs show whether commands were received. They can be viewed either:
- from the Heroku dashboard: click on the bot app (“radiant-garden”), then on “Logs” under “More” at the top right. This shows the log from the moment you open it, so you can open it, issue a command in a software-review issue, and watch it arrive; or
- with the Heroku CLI,
after logging
in, by running
heroku logs.
If Heroku logged the command and passed it on, the problem lies
within the roreviewapi service on the Digital Ocean
droplet, which the following steps help to diagnose.
Check that the API is online
The following code will confirm that the API is online, by returning a single numeric value:
httr2::request ("http://<ip-address>:8000/mean") |>
httr2::req_perform () |>
httr2::resp_body_json ()Check log of recent requests
The log entries are illustrated in the endpoints
vignette, and can be used to ensure that all variables were
successfully delivered in the request. Requests passed on from the bot
via Heroku always show the client “Faraday” (for example,
"Faraday v1.8.0"), which distinguishes them from manual
requests. Any missing or malformed variables most likely indicate
problems with the submission template. These should be caught by the
initial call make by the editorcheck
function to the check_issue_template() function.
Potential issues can be debugged by calling that function locally:
roreviewapi::check_issue_template (
orgrepo = "ropensci/software-review",
issue_num = "<issue_num>"
)That should return an empty string with an additional attribute,
"proceed_with_checks", which should be TRUE.
Any other return value indicates an issue with the submission template,
for which the return value should contain text describing the
problem.
Check installation of system dependencies
This and the following checks are components of the editor_check()
function called by the main endpoint as a background process, the
first step of which is to identify and install system dependencies. This
step is error prone, as is also the case on all CRAN machines. A final
step of the check is to identify any packages which were unable to be
installed, and to post a list of these directly in the submission issue.
Such instances should generally be temporary, and fixed by forthcoming
CRAN updates. These happen because of temporary breakages in one package
which lead to other packages becoming unable to be installed from CRAN,
with these temporary breakages in turn generally arising because of
changes to external (non-R) system libraries. If any notified
inabilities to install packages adversely affect final check results,
debugging may require manually running checks on the Digital Ocean
server, as described in the final section below.
Check system output and error logs
The system output and error logs from checks for a specified package
are accessible from the
stdlogs endpoint, which requires the single parameter
of the repourl of the repository being checked. Logs are
kept for the latest git head, and are accessible if and only if the
latest GitHub commit matches the points at which the logs were
generated. In those cases, the return value may offer useful diagnostic
insights into potential problems either with submitted packages, or the
check system itself.
Instead of using the endpoint, it is often easier to look at the logs directly on the Digital Ocean droplet, which can be accessed via the Digital Ocean web interface or via SSH (which requires your public SSH key to be registered on the droplet). Once on the droplet, list the Docker containers with
There should be a single container with a “STATUS” of “Up”. Enter it with its “CONTAINER ID”:
All files are in /home/ (not ~), and files
cached by the check services are in the pkgcheck cache directory
(PKGCHECK_CACHE_DIR), by default
/home/.cache/R/pkgcheck/. That directory should contain a
clone of the repository being checked; if it does not, the download step
failed. Logs are in its templogs sub-directory, where there
should be two files prefixed with the name of the package being checked,
one for stdout and one for stderr. If the bot
failed to respond at all, the cause is most likely an internal error,
which the stderr log should help to identify. The next step
is then to try to reproduce the error locally, by cloning the package
and stepping through the relevant endpoint
function.
Manually running checks
If all else fails, for example if a request appears in the log but no
stdout/stderr logs were generated, checks can
be manually run directly from the Digital Ocean server, and sent
straight to the relevant issue. The following procedure describe how,
generally following the main editor_check()
function.
- Enter the
roreviewapiDocker image viadocker run -it --rm roreviewapi /bin/bashand start R. - Set
repourl <- <url>and runpath <- roreviewapi::dl_gh_repo (repourl)to download a clone of the repository. - Type
os <- "ubuntu"andos_release <- "20.04". - Run
p <- roreviewapi::pkgrep_install_deps (path, os, os_release). The value,p, will list any packages which were unable to be installed. These will then need to be manually installed, generally through finding the remote/dev URLs for the packages, and runningpak::pkg_install()or similar. Note that successful installation may only be possible in a particular order, and in the worst cases may be a process of trial and error. - Finally generate the main checks by running
checks <- pkgcheck::pkgcheck(path), during which diagnostic output will be dumped directly to the screen. - Convert checks to markdown format by running
out <- roreviewapi::collate_editor_check (checks). - Finally, post the checks to the desired issue with
out <- roreviewapi::post_to_issue (out, orgrepo, issue_num). Alternative values fororgrepoandissue_numcan be used to first confirm that the checks look okay before posting them toropensci/software-review.
These steps can be run in two code chunks, the first directly in the
shell environment of the Digital Ocean droplet. Note that
docker run --rm starts a fresh, temporary container from
the roreviewapi image, rather than entering the running one
with docker exec as above:
And the second within the R environment:
repourl <- "https://github.com/org/repo" # replace with actual org/repo values
path <- roreviewapi::dl_gh_repo (repourl)
os <- "ubuntu"
os_release <- "20.04"
p <- roreviewapi::pkgrep_install_deps (path, os, os_release)
checks <- pkgcheck::pkgcheck (path)
out <- roreviewapi::collate_editor_check (checks)
orgrepo <- "ropensci/software-review" # or somewhere else for testing purposes
out <- roreviewapi::post_to_issue (out, orgrepo, issue_num)