| Title: | A High-Performance Taxonomic Database Interface |
|---|---|
| Description: | Provides fast access to many commonly used taxonomic authorities in a uniform Darwin Core format. Tables are read directly from versioned Parquet snapshots, streamed from cloud storage or from a local copy, and queried with familiar 'dplyr' verbs. |
| Authors: | Carl Boettiger [aut, cre] (ORCID: <https://orcid.org/0000-0002-1642-628X>), Kari Norman [aut] (ORCID: <https://orcid.org/0000-0002-2029-2325>), Jorrit Poelen [aut] (ORCID: <https://orcid.org/0000-0003-3138-4118>), Scott Chamberlain [aut] (ORCID: <https://orcid.org/0000-0003-1444-9135>), Noam Ross [ctb] (ORCID: <https://orcid.org/0000-0002-2136-0000>), Mattia Ghilardi [ctb] (ORCID: <https://orcid.org/0000-0001-9592-7252>) |
| Maintainer: | Carl Boettiger <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 0.3.0 |
| Built: | 2026-08-30 22:03:54 UTC |
| Source: | https://github.com/ropensci/taxadb |
Name providers available for a given version
available_providers(version = latest_version(), db = td_connect())available_providers(version = latest_version(), db = td_connect())
version |
snapshot version, defaults to the latest available |
db |
a connection from |
a data.frame of provider and the schemas published for it
## Not run: available_providers() ## End(Not run)## Not run: available_providers() ## End(Not run)
Versions of the taxadb data available
available_versions(db = td_connect())available_versions(db = td_connect())
db |
a connection from |
a character vector of available snapshot versions
## Not run: available_versions() ## End(Not run)## Not run: available_versions() ## End(Not run)
Rebuild the Catalogue of Life snapshot
build_col( version = format(Sys.Date(), "%Y"), archive = NULL, dir = build_dir(), db = td_connect() )build_col( version = format(Sys.Date(), "%Y"), archive = NULL, dir = build_dir(), db = td_connect() )
version |
snapshot version to write, e.g. |
archive |
path to the COL Darwin Core Archive; downloaded if missing |
dir |
directory for build inputs and outputs |
db |
a duckdb connection |
COL publishes a Darwin Core Archive whose scientificName carries
the authorship – Acanthocerataceae Crawford & Round. taxadb wants the
canonical name, since authorship abbreviations vary too much between
providers to match on. COL also supplies scientificNameAuthorship
separately, so the canonical name is the one with that suffix removed.
COL marks accepted names by leaving acceptedNameUsageID empty, and
distinguishes accepted from provisionally accepted; both are accepted
in the sense that matters here, that they are not a synonym of anything
else, so
both get acceptedNameUsageID set to their own taxonID.
the paths written, invisibly
Other build:
build_fishbase(),
build_gbif(),
build_itis(),
build_ncbi(),
build_ott()
## Not run: build_col("2026") ## End(Not run)## Not run: build_col("2026") ## End(Not run)
Where build inputs and outputs are kept
build_dir()build_dir()
Provider archives are large and slow to fetch, so they are cached
here between builds. Override with the TAXADB_BUILD_DIR environment
variable.
path to the taxadb build directory
build_dir()build_dir()
Rebuild the FishBase or SeaLifeBase snapshot
build_fishbase( version = format(Sys.Date(), "%Y"), provider = c("fb", "slb"), fb_version = NULL, dir = build_dir(), db = td_connect() )build_fishbase( version = format(Sys.Date(), "%Y"), provider = c("fb", "slb"), fb_version = NULL, dir = build_dir(), db = td_connect() )
version |
snapshot version to write, e.g. |
provider |
|
fb_version |
which FishBase snapshot to build from, e.g. |
dir |
directory for build inputs and outputs |
db |
a duckdb connection |
FishBase and SeaLifeBase share a schema, and both are already published as Parquet alongside the taxadb snapshots, so this build reads them over the network and downloads nothing.
FishBase numbers accepted species (SpecCode) and synonyms (SynCode)
in two independent sequences, so the same integer means different things
in each: SpecCode 1 is Scyris indica while SynCode 1 is Alausa
coerulea. Prefixing both as FB:1 would make one identifier name two
taxa, which it did in the previously published table – 20,295 FishBase
identifiers and 61,125 SeaLifeBase ones were ambiguous.
Only SpecCode is therefore used as the taxonID, and synonyms carry a
NULL one exactly as they do for NCBI and OTT. Nothing is lost: the
SynCode is published in its own synonymID column. A consequence is
that a synonym whose SpecCode is 0 – not linked to any species record,
1,043 names in FishBase and 7,512 in SeaLifeBase – has nothing to
resolve to and is dropped.
Classification comes from the families table. FishBase covers only
fishes, so its phylum and kingdom are constant. SeaLifeBase spans some
sixty phyla across several kingdoms and asserts no kingdom itself, so
kingdom is left empty there rather than inferred.
FishBase data is CC-BY-NC (fishbase.org).
the paths written, invisibly
Other build:
build_col(),
build_gbif(),
build_itis(),
build_ncbi(),
build_ott()
## Not run: build_fishbase("2026", provider = "fb") ## End(Not run)## Not run: build_fishbase("2026", provider = "fb") ## End(Not run)
Rebuild the GBIF backbone snapshot
build_gbif( version = format(Sys.Date(), "%Y"), archive = NULL, dir = build_dir(), db = td_connect() )build_gbif( version = format(Sys.Date(), "%Y"), archive = NULL, dir = build_dir(), db = td_connect() )
version |
snapshot version to write, e.g. |
archive |
path to the GBIF backbone archive; downloaded if missing |
dir |
directory for build inputs and outputs |
db |
a duckdb connection |
GBIF supplies canonicalName – the name without authorship –
alongside the full scientificName, so no name parsing is needed. It
leaves canonicalName empty for names its parser cannot analyse, which
includes the sequence-derived identifiers GBIF carries in quantity (BOLD
BINs, UNITE species hypotheses, metagenome-assembled genomes) and hybrid
formulas; for those the scientificName is the name, and is used.
GBIF leaves acceptedNameUsageID empty on accepted and on doubtful
names alike, since it redirects neither. Both therefore become their own
accepted name here, which is also what keeps GBIF's 40,895 synonyms of
doubtful names resolvable.
the paths written, invisibly
Other build:
build_col(),
build_fishbase(),
build_itis(),
build_ncbi(),
build_ott()
## Not run: build_gbif("2026") ## End(Not run)## Not run: build_gbif("2026") ## End(Not run)
Rebuild the ITIS snapshot
build_itis( version = format(Sys.Date(), "%Y"), archive = NULL, dir = build_dir(), db = td_connect() )build_itis( version = format(Sys.Date(), "%Y"), archive = NULL, dir = build_dir(), db = td_connect() )
version |
snapshot version to write, e.g. |
archive |
path to the ITIS SQLite archive; downloaded if missing |
dir |
directory for build inputs and outputs |
db |
a duckdb connection |
ITIS distributes a SQLite database, which duckdb reads directly.
n_usage carries the accepted/synonym distinction under two vocabularies,
zoological (valid/invalid) and botanical (accepted/not accepted);
both map onto accepted and synonym.
ITIS assigns a TSN to synonyms as well as accepted names, so every row
here carries its own taxonID – which makes ITIS the reference for the
taxadb rules checked by td_validate().
scientificNameAuthorship is taken from ITIS's own author table, keyed on
the author id together with the kingdom, since the id is only unique within
one.
the paths written, invisibly
Other build:
build_col(),
build_fishbase(),
build_gbif(),
build_ncbi(),
build_ott()
## Not run: build_itis("2026") ## End(Not run)## Not run: build_itis("2026") ## End(Not run)
Rebuild the NCBI Taxonomy snapshot
build_ncbi( version = format(Sys.Date(), "%Y"), archive = NULL, dir = build_dir(), db = td_connect() )build_ncbi( version = format(Sys.Date(), "%Y"), archive = NULL, dir = build_dir(), db = td_connect() )
version |
snapshot version to write, e.g. |
archive |
path to the NCBI |
dir |
directory for build inputs and outputs |
db |
a duckdb connection |
NCBI distributes nodes.dmp (the hierarchy) and names.dmp
(every name), in a format that claims to be tab-separated but delimits
fields with \\t|\\t.
Every row of names.dmp carries the tax_id of the accepted taxon,
whatever
the name's class: NCBI mints no separate identifier for a synonym. So the
scientific name rows become the accepted names, carrying a taxonID and
pointing acceptedNameUsageID at themselves, and every other name class
becomes a row for the same taxon with a NULL taxonID – which the
taxadb rules permit, since there is no identifier to give.
the paths written, invisibly
Other build:
build_col(),
build_fishbase(),
build_gbif(),
build_itis(),
build_ott()
## Not run: build_ncbi("2026") ## End(Not run)## Not run: build_ncbi("2026") ## End(Not run)
Rebuild the Open Tree Taxonomy snapshot
build_ott( version = format(Sys.Date(), "%Y"), archive = NULL, ott_version = "3.7.3", dir = build_dir(), db = td_connect() )build_ott( version = format(Sys.Date(), "%Y"), archive = NULL, ott_version = "3.7.3", dir = build_dir(), db = td_connect() )
version |
snapshot version to write, e.g. |
archive |
path to the OTT release archive; downloaded if missing |
ott_version |
the OTT release to build from, e.g. |
dir |
directory for build inputs and outputs |
db |
a duckdb connection |
OTT ships taxonomy.tsv and synonyms.tsv, both delimited with
\\t|\\t. synonyms.tsv keys each synonym to the uid of the name it is
a synonym of: OTT mints no identifier for the synonym itself, so those
rows carry a NULL taxonID, which the taxadb rules allow.
OTT no longer populates the type column of synonyms.tsv – it is empty
for all 2.2 million rows in release 3.7.3 – so a synonym is recorded as
synonym unless a type is given.
OTT publishes no vernacular names, so there is no common table for this
provider; filter_common() warns accordingly.
the paths written, invisibly
Other build:
build_col(),
build_fishbase(),
build_gbif(),
build_itis(),
build_ncbi()
## Not run: build_ott("2026") ## End(Not run)## Not run: build_ott("2026") ## End(Not run)
A utility to sanitize taxonomic names to increase probability of resolving names.
clean_names( names, fix_delim = TRUE, binomial_only = TRUE, remove_sp = TRUE, ascii_only = TRUE, lowercase = TRUE, remove_punc = FALSE )clean_names( names, fix_delim = TRUE, binomial_only = TRUE, remove_sp = TRUE, ascii_only = TRUE, lowercase = TRUE, remove_punc = FALSE )
names |
a character vector of taxonomic names (usually species names) |
fix_delim |
Should we replace separators |
binomial_only |
Attempt to prune name to a binomial name, e.g.
Genus and species (specific epithet), e.g. |
remove_sp |
Should we drop unspecified species epithet designations?
e.g. |
ascii_only |
should we coerce strings to ascii characters?
(see |
lowercase |
should names be coerced to lower-case to provide case-insensitive matching? |
remove_punc |
replace all punctuation but apostrophes with a space, remove apostrophes |
Current implementation is limited to handling a few
common cases. Additional extensions may be added later.
A goal of the clean_names function is that any
modification rule of the name strings be precise, atomic, and
toggle-able, rather than relying on clever but more opaque rules and
arbitrary scores. This utility should always be used with care, as
indiscriminate modification of names may result in successful but inaccurate
name matching. A good pattern is to only apply this function to the subset
of names that cannot be directly matched.
clean_names(c("Homo sapiens sapiens", "Homo.sapiens", "Homo sp."))clean_names(c("Homo sapiens sapiens", "Homo.sapiens", "Homo sp."))
common name starts with
common_contains( name, provider = getOption("taxadb_default_provider", "itis"), version = latest_version(), db = td_connect(), ignore_case = TRUE )common_contains( name, provider = getOption("taxadb_default_provider", "itis"), version = latest_version(), db = td_connect(), ignore_case = TRUE )
name |
vector of names (scientific or common, see |
provider |
from which provider should the hierarchy be returned?
Default is 'itis', which can also be configured using |
version |
Which version of the taxadb provider database should we use?
defaults to latest. See |
db |
a connection to the taxadb database. See details. |
ignore_case |
should we ignore case (capitalization) in matching names? Can be significantly slower to run. |
common_contains("monkey")common_contains("monkey")
common name starts with
common_starts_with( name, provider = getOption("taxadb_default_provider", "itis"), version = latest_version(), db = td_connect(), ignore_case = TRUE )common_starts_with( name, provider = getOption("taxadb_default_provider", "itis"), version = latest_version(), db = td_connect(), ignore_case = TRUE )
name |
vector of names (scientific or common, see |
provider |
from which provider should the hierarchy be returned?
Default is 'itis', which can also be configured using |
version |
Which version of the taxadb provider database should we use?
defaults to latest. See |
db |
a connection to the taxadb database. See details. |
ignore_case |
should we ignore case (capitalization) in matching names? Can be significantly slower to run. |
common_starts_with("monkey")common_starts_with("monkey")
by, and values given
by the vector x, and then uses this table to do a filtering join,
joining on the by column to return all rows matching the x values
(scientificNames, taxonIDs, etc).Creates a data frame with column name given by by, and values given
by the vector x, and then uses this table to do a filtering join,
joining on the by column to return all rows matching the x values
(scientificNames, taxonIDs, etc).
filter_by( x, by, provider = getOption("taxadb_default_provider", "itis"), schema = c("dwc", "common"), version = latest_version(), collect = TRUE, db = td_connect(), ignore_case = FALSE )filter_by( x, by, provider = getOption("taxadb_default_provider", "itis"), schema = c("dwc", "common"), version = latest_version(), collect = TRUE, db = td_connect(), ignore_case = FALSE )
x |
a vector of values to filter on |
by |
a column name in the taxa_tbl (following Darwin Core Schema terms). The filtering join is executed with this column as the joining variable. |
provider |
from which provider should the hierarchy be returned?
Default is 'itis', which can also be configured using |
schema |
One of "dwc" (for Darwin Core data) or "common" (for the Common names table.) |
version |
Which version of the taxadb provider database should we use?
defaults to latest. See |
collect |
logical, default |
db |
a connection to the taxadb database. See details. |
ignore_case |
should we ignore case (capitalization) in matching names? Can be significantly slower to run. |
a data.frame in the Darwin Core tabular format containing the matching taxonomic entities.
Other filter_by:
filter_common(),
filter_id(),
filter_name(),
filter_rank()
sp <- c("Trochalopteron henrici gucenense", "Trochalopteron elliotii") filter_by(sp, "scientificName") filter_by(c("ITIS:180092", "ITIS:916116"), "taxonID") filter_by("Aves", "class")sp <- c("Trochalopteron henrici gucenense", "Trochalopteron elliotii") filter_by(sp, "scientificName") filter_by(c("ITIS:180092", "ITIS:916116"), "taxonID") filter_by("Aves", "class")
Look up taxonomic information by common name
filter_common( name, provider = getOption("taxadb_default_provider", "itis"), version = latest_version(), collect = TRUE, ignore_case = TRUE, db = td_connect() )filter_common( name, provider = getOption("taxadb_default_provider", "itis"), version = latest_version(), collect = TRUE, ignore_case = TRUE, db = td_connect() )
name |
a character vector of common (vernacular English) names, e.g. "Humans" |
provider |
from which provider should the hierarchy be returned?
Default is 'itis', which can also be configured using |
version |
Which version of the taxadb provider database should we use?
defaults to latest. See |
collect |
logical, default |
ignore_case |
should we ignore case (capitalization) in matching names? Can be significantly slower to run. |
db |
a connection to the taxadb database. See details. |
a data.frame in the Darwin Core tabular format containing the matching taxonomic entities.
Other filter_by:
filter_by(),
filter_id(),
filter_name(),
filter_rank()
filter_common("Pied Tamarin")filter_common("Pied Tamarin")
Return a taxonomic table matching the requested ids
filter_id( id, provider = getOption("taxadb_default_provider", "itis"), type = c("taxonID", "acceptedNameUsageID"), version = latest_version(), collect = TRUE, db = td_connect() )filter_id( id, provider = getOption("taxadb_default_provider", "itis"), type = c("taxonID", "acceptedNameUsageID"), version = latest_version(), collect = TRUE, db = td_connect() )
id |
taxonomic id, in prefix format |
provider |
from which provider should the hierarchy be returned?
Default is 'itis', which can also be configured using |
type |
id type. Can be |
version |
Which version of the taxadb provider database should we use?
defaults to latest. See |
collect |
logical, default |
db |
a connection to the taxadb database. See details. |
Use type="acceptedNameUsageID" to return all rows
for which this ID is the accepted ID, including both synonyms and
and accepted names (since both all synonyms of a name share the
same acceptedNameUsageID.) Use taxonID (default) to only return
those rows for which the Scientific name corresponds to the taxonID.
Some providers (e.g. ITIS) assign taxonIDs to synonyms, most others
only assign IDs to accepted names. In the latter case, this means
requesting taxonID will only match accepted names, while requesting
matches to the acceptedNameUsageID will also return any known synonyms.
See examples.
a data.frame with id and name of all matching species
Other filter_by:
filter_by(),
filter_common(),
filter_name(),
filter_rank()
filter_id(c("ITIS:180092", "ITIS:916116")) filter_id("ITIS:916116", type="acceptedNameUsageID")filter_id(c("ITIS:180092", "ITIS:916116")) filter_id("ITIS:916116", type="acceptedNameUsageID")
Look up taxonomic information by scientific name
filter_name( name, provider = getOption("taxadb_default_provider", "itis"), version = latest_version(), collect = TRUE, ignore_case = FALSE, db = td_connect() )filter_name( name, provider = getOption("taxadb_default_provider", "itis"), version = latest_version(), collect = TRUE, ignore_case = FALSE, db = td_connect() )
name |
a character vector of scientific names, e.g. "Homo sapiens" |
provider |
from which provider should the hierarchy be returned?
Default is 'itis', which can also be configured using |
version |
Which version of the taxadb provider database should we use?
defaults to latest. See |
collect |
logical, default |
ignore_case |
should we ignore case (capitalization) in matching names? Can be significantly slower to run. |
db |
a connection to the taxadb database. See details. |
Most but not all authorities can match against both species level and
higher-level (or lower, e.g. subspecies or variety) taxonomic names.
The rank level is indicated by taxonRank column.
Most authorities include both known synonyms and accepted names in the
scientificName column, (with the status indicated by taxonomicStatus).
This is convenient, as users will typically not know if the names they
have are synonyms or accepted names, but will want to get the match to the
accepted name and accepted ID in either case.
a data.frame in the Darwin Core tabular format containing the matching taxonomic entities.
Other filter_by:
filter_by(),
filter_common(),
filter_id(),
filter_rank()
sp <- c("Trochalopteron henrici gucenense", "Trochalopteron elliotii") filter_name(sp)sp <- c("Trochalopteron henrici gucenense", "Trochalopteron elliotii") filter_name(sp)
Get all members (descendants) of a given rank level
filter_rank( name, rank, provider = getOption("taxadb_default_provider", "itis"), version = latest_version(), collect = TRUE, ignore_case = TRUE, db = td_connect() )filter_rank( name, rank, provider = getOption("taxadb_default_provider", "itis"), version = latest_version(), collect = TRUE, ignore_case = TRUE, db = td_connect() )
name |
taxonomic scientific name (e.g. "Aves") |
rank |
taxonomic rank name. (e.g. "class") |
provider |
from which provider should the hierarchy be returned?
Default is 'itis', which can also be configured using |
version |
Which version of the taxadb provider database should we use?
defaults to latest. See |
collect |
logical, default |
ignore_case |
should we ignore case (capitalization) in matching names? Can be significantly slower to run. |
db |
a connection to the taxadb database. See details. |
a data.frame in the Darwin Core tabular format containing the matching taxonomic entities.
Other filter_by:
filter_by(),
filter_common(),
filter_id(),
filter_name()
filter_rank("Aves", "class")filter_rank("Aves", "class")
Match names that start or contain a specified text string
fuzzy_filter( name, by = c("scientificName", "vernacularName"), provider = getOption("taxadb_default_provider", "itis"), match = c("contains", "starts_with"), version = latest_version(), db = td_connect(), ignore_case = TRUE, collect = TRUE )fuzzy_filter( name, by = c("scientificName", "vernacularName"), provider = getOption("taxadb_default_provider", "itis"), match = c("contains", "starts_with"), version = latest_version(), db = td_connect(), ignore_case = TRUE, collect = TRUE )
name |
vector of names (scientific or common, see |
by |
a column name in the taxa_tbl (following Darwin Core Schema terms). The filtering join is executed with this column as the joining variable. |
provider |
from which provider should the hierarchy be returned?
Default is 'itis', which can also be configured using |
match |
should we match by names starting with the term or containing the term anywhere in the name? |
version |
Which version of the taxadb provider database should we use?
defaults to latest. See |
db |
a connection to the taxadb database. See details. |
ignore_case |
should we ignore case (capitalization) in matching names? Can be significantly slower to run. |
collect |
logical, default |
Note that fuzzy filter will be fast with an single or small number
of names, but will be slower if given a very large vector of
names to match, as unlike other filter_ commands,
fuzzy matching requires separate SQL calls for each name.
As fuzzy matches should all be confirmed manually in any event, e.g.
not every common name containing "monkey" belongs to a primate species.
This method utilizes the database operation %like% to filter tables without
loading into memory. Note that this does not support the use of regular
expressions at this time.
## match any common name containing: name <- c("woodpecker", "monkey") fuzzy_filter(name, "vernacularName") ## match scientific name fuzzy_filter("Trochalop", "scientificName", match = "starts_with")## match any common name containing: name <- c("woodpecker", "monkey") fuzzy_filter(name, "vernacularName") ## match scientific name fuzzy_filter("Trochalop", "scientificName", match = "starts_with")
A drop-in replacement for [taxize::get_ids()]
get_ids( names, provider = getOption("taxadb_default_provider", "itis"), format = c("prefix", "bare", "uri"), version = latest_version(), taxadb_db = td_connect(), ignore_case = FALSE, warn = TRUE, db = NULL, ... )get_ids( names, provider = getOption("taxadb_default_provider", "itis"), format = c("prefix", "bare", "uri"), version = latest_version(), taxadb_db = td_connect(), ignore_case = FALSE, warn = TRUE, db = NULL, ... )
names |
a list of scientific names (which may include higher-order ranks in most authorities). |
provider |
abbreviation code for the provider. See details. |
format |
Format for the returned identifier, one of
|
version |
Which version of the taxadb provider database should we use?
defaults to latest. see |
taxadb_db |
Connection to from |
ignore_case |
should we ignore case (capitalization) in matching names?
default is |
warn |
should we display warnings on NAs resulting from multiply-resolved matches?
(Unlike unmatched names, these NAs can usually be resolved manually via |
db |
previous name for |
... |
additional arguments (currently ignored) |
Note that some taxize authorities: nbn, tropicos, and eol,
are not recognized by taxadb and will throw an error here. Meanwhile,
taxadb recognizes several authorities not known to [taxize::get_ids()].
Both include itis, ncbi, col, and gbif.
Like all taxadb functions, this function will run
fastest if a local copy of the provider is installed in advance
using [td_create()].
a vector of IDs, of the same length as the input names Any
unmatched names or multiply-matched names will return as NAs.
To resolve multi-matched names, use [filter_name()] instead to return
a table with a separate row for each separate match of the input name.
filter_name
Other get:
get_names()
get_ids("Midas bicolor") get_ids(c("Midas bicolor", "Homo sapiens"), format = "prefix") get_ids("Midas bicolor", format = "uri")get_ids("Midas bicolor") get_ids(c("Midas bicolor", "Homo sapiens"), format = "prefix") get_ids("Midas bicolor", format = "uri")
Translate identifiers into scientific names
get_names( id, provider = getOption("taxadb_default_provider", "itis"), version = latest_version(), format = c("guess", "prefix", "bare", "uri"), taxadb_db = td_connect(), db = NULL )get_names( id, provider = getOption("taxadb_default_provider", "itis"), version = latest_version(), format = c("guess", "prefix", "bare", "uri"), taxadb_db = td_connect(), db = NULL )
id |
a list of taxonomic identifiers. |
provider |
abbreviation code for the provider. See details. |
version |
Which version of the taxadb provider database should we use?
defaults to latest. see |
format |
Format for the returned identifier, one of
|
taxadb_db |
Connection to from |
db |
previous name for |
Like all taxadb functions, this function will run
fastest if a local copy of the provider is installed in advance
using [td_create()].
a vector of names, of the same length as the input ids. Any unmatched IDs will return as NAs.
Other get:
get_ids()
get_names(c("ITIS:1025094", "ITIS:1025103"), format = "prefix")get_names(c("ITIS:1025094", "ITIS:1025103"), format = "prefix")
The most recent taxadb snapshot version
latest_version(db = td_connect())latest_version(db = td_connect())
db |
a connection from |
Versions are ordered as version numbers, not as strings. This
matters: as strings "22.12" sorts after "2026", so a plain max()
would make an archival release from 2022 the default for every query once
it was published.
the latest available version, as a character string
## Not run: latest_version() ## End(Not run)## Not run: latest_version() ## End(Not run)
List the taxonomic snapshots available from the taxadb repository
list_snapshots(db = td_connect())list_snapshots(db = td_connect())
db |
a connection from |
Requires network access. Results are cached for the session.
a data.frame with one row per published Parquet file, giving its
version, schema, provider and uri.
## Not run: list_snapshots() ## End(Not run)## Not run: list_snapshots() ## End(Not run)
return all taxa in which scientific name contains the text provided
name_contains( name, provider = getOption("taxadb_default_provider", "itis"), version = latest_version(), db = td_connect(), ignore_case = TRUE )name_contains( name, provider = getOption("taxadb_default_provider", "itis"), version = latest_version(), db = td_connect(), ignore_case = TRUE )
name |
vector of names (scientific or common, see |
provider |
from which provider should the hierarchy be returned?
Default is 'itis', which can also be configured using |
version |
Which version of the taxadb provider database should we use?
defaults to latest. See |
db |
a connection to the taxadb database. See details. |
ignore_case |
should we ignore case (capitalization) in matching names? Can be significantly slower to run. |
name_contains("Trochalop")name_contains("Trochalop")
scientific name starts with
name_starts_with( name, provider = getOption("taxadb_default_provider", "itis"), version = latest_version(), db = td_connect(), ignore_case = TRUE )name_starts_with( name, provider = getOption("taxadb_default_provider", "itis"), version = latest_version(), db = td_connect(), ignore_case = TRUE )
name |
vector of names (scientific or common, see |
provider |
from which provider should the hierarchy be returned?
Default is 'itis', which can also be configured using |
version |
Which version of the taxadb provider database should we use?
defaults to latest. See |
db |
a connection to the taxadb database. See details. |
ignore_case |
should we ignore case (capitalization) in matching names? Can be significantly slower to run. |
name_starts_with("Trochalop")name_starts_with("Trochalop")
Return a reference to a given table in the taxadb database
taxa_tbl( provider = getOption("taxadb_default_provider", "itis"), schema = c("dwc", "common"), version = latest_version(), db = td_connect() )taxa_tbl( provider = getOption("taxadb_default_provider", "itis"), schema = c("dwc", "common"), version = latest_version(), db = td_connect() )
provider |
from which provider should the hierarchy be returned?
Default is 'itis', which can also be configured using |
schema |
One of "dwc" (for Darwin Core data) or "common" (for the Common names table.) |
version |
Which version of the taxadb provider database should we use?
defaults to latest. See |
db |
a connection to the taxadb database. See details. |
The returned table is a duckdb view over Parquet, so it can be
manipulated with any dplyr verb and is only ever read to the extent your
query requires. Unless a local copy has been installed with
td_download(), the data is streamed from remote storage on demand.
a lazy dplyr table backed by the requested Parquet snapshot.
## default schema is the Darwin Core table taxa_tbl() ## common names table taxa_tbl(schema = "common")## default schema is the Darwin Core table taxa_tbl() ## common names table taxa_tbl(schema = "common")
Show the local taxadb directory
taxadb_dir()taxadb_dir()
Local snapshots downloaded by td_download() are stored here.
Override with the TAXADB_HOME environment variable.
path to the local taxadb data directory
taxadb_dir()taxadb_dir()
Describe the taxonomic name providers
taxadb_provider_info(provider = taxadb_providers())taxadb_provider_info(provider = taxadb_providers())
provider |
one or more provider abbreviations; all by default |
Providers are not interchangeable. col, gbif and ott are
synthesis projects that integrate other checklists, while itis, ncbi,
fb and slb are primary authorities; the synthesis column records
which is which. More importantly, providers disagree: the same name can
be accepted by one and a synonym of something else in another, so a name
resolved against one provider should not be mixed with names resolved
against another. See vignette("data-sources").
Redistribution terms differ too. fb and slb are CC BY-NC, so those
two tables may not be used commercially.
a data.frame with one row per provider giving its title, url,
the source its data is taken from, its license and a citation.
taxadb_provider_info() taxadb_provider_info("col")$citationtaxadb_provider_info() taxadb_provider_info("col")$citation
Providers taxadb can rebuild
taxadb_providers()taxadb_providers()
Unlike available_providers(), which reports what is published,
this reports what td_build() knows how to derive from the provider's own
distribution.
a character vector of provider abbreviations
taxadb_providers()taxadb_providers()
The taxadb data repository
taxadb_repo()taxadb_repo()
Override with options(taxadb_repo=) or the TAXADB_REPO
environment variable to read from a mirror or a staging repository.
the object-store prefix holding taxadb snapshots.
taxadb_repo()taxadb_repo()
Locate the Parquet files backing a taxadb table
taxadb_uri( provider = getOption("taxadb_default_provider", "itis"), schema = c("dwc", "common"), version = latest_version(), local = NULL )taxadb_uri( provider = getOption("taxadb_default_provider", "itis"), schema = c("dwc", "common"), version = latest_version(), local = NULL )
provider |
from which provider should the hierarchy be returned?
Default is 'itis', which can also be configured using |
schema |
One of "dwc" (for Darwin Core data) or "common" (for the Common names table.) |
version |
Which version of the taxadb provider database should we use?
defaults to latest. See |
local |
should we return the path to a local snapshot? By default
a local copy is used when one is present (see |
a glob pattern (or file path) that duckdb can read
taxadb_uri("itis_test")taxadb_uri("itis_test")
Runs a provider's preprocessing end to end: fetch the provider's own distribution, normalize it to the taxadb Darwin Core schema, and write the Parquet snapshot.
td_build( provider = "itis", version = format(Sys.Date(), "%Y"), dir = build_dir(), validate = TRUE, db = td_connect(), ... )td_build( provider = "itis", version = format(Sys.Date(), "%Y"), dir = build_dir(), validate = TRUE, db = td_connect(), ... )
provider |
one or more providers to build. See |
version |
the snapshot version to write, defaults to the year. |
dir |
directory for build inputs and outputs, see |
validate |
should each table be checked with |
db |
a duckdb connection |
... |
passed to the individual provider builder, e.g. |
Snapshots are published for the providers so that most users never need to run this. It is here so that a user who needs a fresher snapshot than the published one, or who wants to check how a table was derived, can rebuild it themselves rather than asking someone to.
Builds are done entirely in duckdb, out of core, so they are bounded by
disk rather than memory. The archives are large: COL and GBIF are around
500MB and 1GB compressed respectively, and are cached in dir between
builds.
a data.frame of the validation results, invisibly if validate
is FALSE the paths written.
## Not run: ## rebuild one provider td_build("itis") ## rebuild everything that can be built without credentials td_build(taxadb_providers()) ## End(Not run)## Not run: ## rebuild one provider td_build("itis") ## rebuild everything that can be built without credentials td_build(taxadb_providers()) ## End(Not run)
Connect to the taxadb database
td_connect(dbdir = NULL, driver = NULL, read_only = NULL)td_connect(dbdir = NULL, driver = NULL, read_only = NULL)
dbdir |
Deprecated, ignored. |
driver |
Deprecated, ignored. The driver is always |
read_only |
Deprecated, ignored. |
taxadb reads Parquet snapshots directly from object storage
(https://source.coop) using duckdb's httpfs extension, so no data
import step is required. This function returns a connection with
httpfs loaded and the S3 endpoint configured for anonymous access.
For performance reasons the connection is cached and reused, making
repeated calls to td_connect() much faster and more failsafe than
repeated calls to DBI::dbConnect.
The httpfs extension needed for remote reads is loaded on first use
rather than at connect time, so a session that only reads local snapshots
or the bundled test data never touches the network.
duckdb would otherwise scan with one thread per core and let its buffer
pool grow to most of system RAM. For the selective scans taxadb makes
that is the wrong trade: each scanning thread holds a decompressed Parquet
row group, so memory grows with core count while the query gets no faster.
On a 128-core machine, looking up one name in the GBIF table peaked at
1324 MB with the duckdb defaults and 322 MB capped at eight threads – and
the capped run was faster (0.7s against 1.0s).
So the connection caps threads at TAXADB_THREADS (8) or the core count,
whichever is lower. Raise it with options(taxadb_threads=) for bulk work
– td_build() does this itself – and set
options(taxadb_memory_limit=) to bound the buffer pool.
a DBI connection to an in-process duckdb database, configured
for anonymous streaming reads from the taxadb data repository.
db <- td_connect()db <- td_connect()
Superseded by td_download().
td_create( provider = getOption("taxadb_default_provider", "itis"), schema = c("dwc", "common"), version = latest_version(), overwrite = FALSE, lines = NULL, dbdir = NULL, db = td_connect() )td_create( provider = getOption("taxadb_default_provider", "itis"), schema = c("dwc", "common"), version = latest_version(), overwrite = FALSE, lines = NULL, dbdir = NULL, db = td_connect() )
provider |
a character vector of provider(s) to download. See
|
schema |
One of "dwc" (for Darwin Core data) or "common" (for the Common names table.) |
version |
Which version of the taxadb provider database should we use?
defaults to latest. See |
overwrite |
passed to |
lines |
deprecated, ignored. |
dbdir |
deprecated, ignored. |
db |
a connection to the taxadb database. See details. |
taxadb no longer needs to import data before querying it:
tables are read directly from Parquet, streamed from remote storage or
from a local copy. td_create() is retained as an alias for
td_download(), which installs a local copy.
the local paths of the downloaded files, invisibly.
## Not run: td_create("itis") ## End(Not run)## Not run: td_create("itis") ## End(Not run)
Disconnect from the taxadb database.
td_disconnect(db = td_connect())td_disconnect(db = td_connect())
db |
database connection |
This function manually closes a connection to the taxadb database.
invisible TRUE
td_disconnect()td_disconnect()
Downloads the Parquet files for the requested provider(s) into
taxadb_dir(), so that subsequent queries read from local disk instead
of streaming from remote storage.
td_download( provider = getOption("taxadb_default_provider", "itis"), schema = c("dwc", "common"), version = latest_version(), overwrite = FALSE, db = td_connect() )td_download( provider = getOption("taxadb_default_provider", "itis"), schema = c("dwc", "common"), version = latest_version(), overwrite = FALSE, db = td_connect() )
provider |
a character vector of provider(s) to download. See
|
schema |
One of "dwc" (for Darwin Core data) or "common" (for the Common names table.) |
version |
Which version of the taxadb provider database should we use?
defaults to latest. See |
overwrite |
should we re-download files that are already present?
Default |
db |
a connection to the taxadb database. See details. |
Streaming is fast enough for most interactive use and requires no setup, so a local copy is optional. Install one when you will make many queries against the same table, when you need to work offline, or when you want a snapshot pinned on disk for reproducibility.
Snapshots are large: the Darwin Core tables for col and gbif are each
several hundred MB. Use available_providers() to see what is published,
and delete a local copy with unlink(taxadb_dir(), recursive = TRUE).
the local paths of the downloaded files, invisibly.
## Not run: ## Install a local copy of ITIS. Writes to taxadb_dir() and downloads # tens of MB, so this is never run unattended. td_download("itis") ## End(Not run)## Not run: ## Install a local copy of ITIS. Writes to taxadb_dir() and downloads # tens of MB, so this is never run unattended. td_download("itis") ## End(Not run)
Describe a built snapshot
td_manifest( version = format(Sys.Date(), "%Y"), dir = build_dir(), validate = TRUE, db = td_connect() )td_manifest( version = format(Sys.Date(), "%Y"), dir = build_dir(), validate = TRUE, db = td_connect() )
version |
the snapshot version to describe |
dir |
the build output directory, see |
validate |
should each table be checked with |
db |
a duckdb connection |
Run after td_build(). The row counts and checksums are read
back off the written files rather than carried over from the build, so
the manifest describes what was actually published.
a data.frame with one row per published table, giving its provider, schema, row count, columns, file sizes and checksum, and the upstream release it was derived from.
## Not run: td_manifest("2026") ## End(Not run)## Not run: td_manifest("2026") ## End(Not run)
Check a taxadb table against the taxadb Darwin Core rules
td_validate( provider = getOption("taxadb_default_provider", "itis"), schema = c("dwc", "common"), version = latest_version(), db = td_connect() )td_validate( provider = getOption("taxadb_default_provider", "itis"), schema = c("dwc", "common"), version = latest_version(), db = td_connect() )
provider |
from which provider should the hierarchy be returned?
Default is 'itis', which can also be configured using |
schema |
One of "dwc" (for Darwin Core data) or "common" (for the Common names table.) |
version |
Which version of the taxadb provider database should we use?
defaults to latest. See |
db |
a connection to the taxadb database. See details. |
The rules checked are:
columns – the required Darwin Core terms are present, spelled in Darwin Core camelCase.
types – identifier and name columns are character. A column that
is entirely NA will often be typed as integer or logical by mistake,
which this catches.
scientificName – never NA. Every row names something, at every
rank: Animalia is a scientificName just as Homo sapiens is.
taxonRank, taxonomicStatus – never NA.
acceptedNameUsageID – never NA, on synonyms and on accepted
names. This is where taxadb is stricter than Darwin Core.
accepted_has_id – a row labelled accepted is its own accepted
name: taxonID is present and equals acceptedNameUsageID.
(taxonID may be NA on a synonym, where the provider mints no
identifier for it – OTT and NCBI, for instance, do not.)
accepted_resolves – every acceptedNameUsageID matches the
taxonID of a self-referencing row. No dangling references.
synonym_not_self – a row labelled a synonym points somewhere else, never at itself.
taxonID_one_name – a taxonID always names the same
scientificName. An identifier may appear on more than one row: ITIS
records 255 synonyms that are ambiguous between two accepted taxa, and
a row for each is the honest representation. What must not happen is
one identifier naming two different names, which is what results
from a provider numbering its accepted names and its synonyms in
separate sequences and both being given the same prefix.
accepted_unique – no duplicate taxonID among accepted names.
id_prefix – identifiers are the provider's identifier prefixed by
the provider abbreviation in capitals, e.g. ITIS:180092.
taxonomicStatus is deliberately not checked against a controlled
vocabulary: providers draw real distinctions (homotypic synonym,
provisionally accepted, doubtful, misapplied) that are worth
preserving. The rules are therefore phrased structurally. A name the
provider does not redirect to another name is its own accepted name
whatever confidence it expresses about it, so doubtful and
provisionally accepted rows self-reference exactly as accepted ones
do; only the two terms whose meaning taxadb actually relies on,
accepted and synonym, are given a required shape.
a data.frame with one row per rule, giving whether the table
passes, how many rows violations were found, and a note.
td_validate("itis_test")td_validate("itis_test")
Writes manifest.csv and README.md into the snapshot directory, ready
to be uploaded alongside the Parquet files.
td_write_metadata( version = format(Sys.Date(), "%Y"), dir = build_dir(), repo = taxadb_repo(), archival = FALSE, db = td_connect() )td_write_metadata( version = format(Sys.Date(), "%Y"), dir = build_dir(), repo = taxadb_repo(), archival = FALSE, db = td_connect() )
version |
the snapshot version to describe |
dir |
the build output directory, see |
repo |
the data repository the snapshot will be published to |
archival |
is this a republication of a historical release rather than a fresh build? Archival snapshots are byte-identical to what that version originally contained, so they predate the current schema rules and the README says so. |
db |
a duckdb connection |
The README states what the tables are, what the schema means, where each provider's data came from and under what licence, so that someone who finds the data without the package can still use it.
the paths written, invisibly
## Not run: td_write_metadata("2026") ## End(Not run)## Not run: td_write_metadata("2026") ## End(Not run)