autotest, pkgcheck, goodpractice - which tool does what?

There are three rOpenSci packages to help developers:

  • autotest for “Automatic testing of R packages”;
  • goodpractice to “Give advice about good practices when building R packages”; and
  • pkgcheck to “Check whether a package is ready for submission to rOpenSci’s peer review system”.

This vignette describes when and where each of these tools should be used. Package authors are likely to find all three tools useful, and if in doubt, should generally follow the sequence: autotest → goodpractice → pkgcheck.

autotest

The autotest package automatically tests responses to changes in input parameters for each function. It is distinctly different to both goodpractice and pkgcheck, and should generally be used before either of those packages.

autotest dynamically analyses what your package does when running all examples and test files. It gathers information using the typetracker package, and uses this to automatically analyse and modify each parameter, and then to examine how each function responds.

The package also analyses documentation entries for each parameter, to assess whether text descriptions truly reflect observed behaviour.

Changes in response to autotest

Use of the autotest package generally results in changes to:

  • Tests and example code.
  • Descriptions of function parameters.
  • Additional changes to function behaviour to rectify any problems identified.

goodpractice

The goodpractice package is intended to,

Give advice about good practices when building R packages. Advice includes functions and syntax to avoid, package structure, code complexity, code formatting, etc.

The goodpractice package was initially developed as a wrapper around three separate packages, and produced a single integrated report on the output of:

  • rcmdcheck to run R CMD check within an R session;
  • covr to analyse and report on test coverage; and
  • cyclocomp to analyse the cycylomatic complexity of package functions.

It has since been extended to include many other aspects of general good practices in R package development. goodpractice implements a large suite of checks defined within the following “check groups”:

goodpractice::describe_check_groups ()

covr: Test coverage report from ‘covr’ package.

cyclocomp: Function cyclocomplexity with the ‘cyclocomp’ package; default limit of 50.

description: Check common issues with DESCRIPTION files, including formatting issues with URLs, DOIs, BugReports, package names, and author and contributor roles

lintr: Check package linting with the ‘lintr’ package (84 linters in total).

namespace: Check import and export patterns in NAMESPACE file

rcmdcheck: Run ‘R CMD check’ via the ‘rcmdcheck’ package. ~200 checks for documentation, namespace, compilation, tests, vignettes, CRAN compliance.

rd: Check whether or not ’man/*.Rd’ function documentation includes both example code and return values (regardless of whether or not documentation files are generated by ‘Roxygen2’).

revdep: Check whether package has reverse dependencies, and recommending running ‘reddev’ package if so.

roxygen2: Only for packages within use ‘Roxygen2’ to generate documentation. Checks for best practices in Roxygen2 tag usage, flags any unknown tags, and ensures ‘inheritParams’ is used correctly.

code_structure: Common issues like duplicated or unused function bodies, that ‘print()’ returns insivibly, that ‘on.exit()’ uses ‘add = TRUE’, and checks on function length (default max50 lines).

package_structure: Generic checks like whether a package has a README, a NEWS file, or whether all files use a ‘.R’ extension, and not ‘.r’.

spelling: Check spelling with the ‘spelling’ package.

tidyverse: Check compliance with the Tidyverse style guide; mostly via ‘lintr’ package. (These checks are not run by default; and only if ‘tidyverse_checks()’ are added to ‘checks_by_group()’.

urlchecker: Check whether all URLs are valid, includingidentifying any redirects.

vignette: Check that vignette code does not use either ‘rm()’ or ‘setwd()’.

Changes in response to goodpractice

All checks groups are run by default, except the optional “tidyverse” check group. This check suite covers a huge range of common issues in package structure, design, and documentation. Responding to goodpractice recommendations often results in changes to most aspects of a package.

pkgcheck

The pkgcheck package is intended to,

Check whether a package is ready for submission to rOpenSci’s peer review system.

This package implements a suite of additional checks required for all packages submitted to rOpenSci. It includes all default goodpractice checks, along with a suite of additional checks. Most of these extend beyond goodpractice, to focus on general good practices for online repository management, including appropriate licensing, contribution guides, documentation websites, statistical properties of software in comparison to all other packages on CRAN, and many other aspects.

Developers should generally deal with goodpractice checks first. pkgcheck will then re-run all of those checks, along with its additional checks.

The pkgcheck package is also available as a pkgcheck-action GitHub action that will run every time local changes are pushed to GitHub. This action generates a detailed report in a dedicated GitHub issue. Authors intending to submit packages to rOpenSci for peer review are recommended to install and use this action to ensure all checks pass prior to submission.

Changes in response to pkgcheck

Unlike both autotest and goodpractice, complying with pkgcheck suggestions often results in files being added to repositories (such as vignettes or contributor guidelines), and changes to repository structure on the repository’s hosting platform, including publication of online documentation.

Because pkgcheck checks these “meta-level” aspects of entire repositories, it should generally be run and compolied with only after first addressing all issues flagged by autotest and goodpractice.