| Title: | Read and Write Frictionless Data Packages |
|---|---|
| Description: | Read and write Frictionless Data Packages. A 'Data Package' (<https://datapackage.org/>) is a simple container format and standard to describe and package a collection of (tabular) data. It is typically used to publish FAIR (<https://www.go-fair.org/fair-principles/>) and open datasets. |
| Authors: | Peter Desmet [aut, cre] (ORCID: <https://orcid.org/0000-0002-8442-8025>, affiliation: Research Institute for Nature and Forest (INBO)), Damiano Oldoni [aut] (ORCID: <https://orcid.org/0000-0003-3445-7562>, affiliation: Research Institute for Nature and Forest (INBO)), Pieter Huybrechts [aut] (ORCID: <https://orcid.org/0000-0002-6658-6062>, affiliation: Research Institute for Nature and Forest (INBO)), Sanne Govaert [aut] (ORCID: <https://orcid.org/0000-0002-8939-1305>, affiliation: Research Institute for Nature and Forest (INBO)), Kyle Husmann [ctb] (ORCID: <https://orcid.org/0000-0001-9875-8976>, affiliation: Pennsylvania State University), Research Institute for Nature and Forest (INBO) [cph] (ROR: <https://ror.org/00j54wy13>), Research Foundation - Flanders [fnd] (https://lifewatch.be), Beatriz Milz [rev] (ORCID: <https://orcid.org/0000-0002-3064-4486>), João Martins [rev] (ORCID: <https://orcid.org/0000-0001-7961-4280>) |
| Maintainer: | Peter Desmet <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 1.3.0.9000 |
| Built: | 2026-09-17 09:35:49 UTC |
| Source: | https://github.com/frictionlessdata/frictionless-r |
Adds or inserts provided arguments as elements to a list. Attributes and classes of the original list are retained.
add_properties(x, ..., after = length(x))add_properties(x, ..., after = length(x))
x |
A list or vector. |
... |
Elements to add, as named arguments. |
after |
Position after which elements should be added.
Use |
add_properties() can be used to add (custom) metadata properties to a Data
Package, Data Resource, Table Dialect or Table Schema.
Note that added properties are not validated.
See the vignettes for an overview of all standard metadata properties.
add_properties() has some advantages over:
append(): removes attributes and classes, invalidating a Data Package
object.
Direct assignment (package$title <- "My package"): no control over the
position of the added element.
add_resource() also supports adding metadata properties to a Data Resource,
except those automatically set by that function.
x with the added elements.
Other edit functions:
add_resource(),
remove_resource()
# Add property to a generic list list <- list(a = 1, b = 2) add_properties(list, added = 3) # Add properties to a Data Package package <- create_package() package <- add_properties( package, title = "Example package", keywords = c("camera traps", "frictionlessdata"), after = 1 # Add after the first property ) package str(package)# Add property to a generic list list <- list(a = 1, b = 2) add_properties(list, added = 3) # Add properties to a Data Package package <- create_package() package <- add_properties( package, title = "Example package", keywords = c("camera traps", "frictionlessdata"), after = 1 # Add after the first property ) package str(package)
Adds a Data Resource to a Data Package. The resource will be a Tabular Data Resource.
add_resource( package, resource_name, data, schema = NULL, replace = FALSE, delim = ",", ... )add_resource( package, resource_name, data, schema = NULL, replace = FALSE, delim = ",", ... )
package |
Data Package object, as returned by |
resource_name |
Name of the Data Resource. |
data |
Data to attach, either a data frame or path(s) to CSV file(s):
|
schema |
Either a list, or path or URL to a JSON file describing a Table
Schema for the |
replace |
If |
delim |
Delimiter for the CSV file(s) referenced in |
... |
Additional metadata properties to add to the
resource.
Note that added properties are not validated and ignored by
|
See vignette("data-resource") (and to a lesser extend
vignette("table-dialect")) to learn how this function implements the
Data Package standard.
package with one additional resource.
Other edit functions:
add_properties(),
remove_resource()
# Load the example Data Package package <- example_package() # List the resources resource_names(package) # Create a data frame df <- data.frame( multimedia_id = c( "aed5fa71-3ed4-4284-a6ba-3550d1a4de8d", "da81a501-8236-4cbd-aa95-4bc4b10a05df" ), x = c(718, 748), y = c(860, 900) ) # Add the resource "positions" from the data frame package <- add_resource(package, "positions", data = df) # Add the resource "positions_with_schema", with a user-defined schema and title my_schema <- create_schema(df) package <- add_resource( package, resource_name = "positions_with_schema", data = df, schema = my_schema, title = "Positions with schema" ) # Replace the resource "observations" with a file-based resource (2 TSV files) path_1 <- system.file("extdata", "v1", "observations_1.tsv", package = "frictionless") path_2 <- system.file("extdata", "v1", "observations_2.tsv", package = "frictionless") package <- add_resource( package, resource_name = "observations", data = c(path_1, path_2), replace = TRUE, delim = "\t" ) # List the resources ("positions" and "positions_with_schema" added) resource_names(package)# Load the example Data Package package <- example_package() # List the resources resource_names(package) # Create a data frame df <- data.frame( multimedia_id = c( "aed5fa71-3ed4-4284-a6ba-3550d1a4de8d", "da81a501-8236-4cbd-aa95-4bc4b10a05df" ), x = c(718, 748), y = c(860, 900) ) # Add the resource "positions" from the data frame package <- add_resource(package, "positions", data = df) # Add the resource "positions_with_schema", with a user-defined schema and title my_schema <- create_schema(df) package <- add_resource( package, resource_name = "positions_with_schema", data = df, schema = my_schema, title = "Positions with schema" ) # Replace the resource "observations" with a file-based resource (2 TSV files) path_1 <- system.file("extdata", "v1", "observations_1.tsv", package = "frictionless") path_2 <- system.file("extdata", "v1", "observations_2.tsv", package = "frictionless") package <- add_resource( package, resource_name = "observations", data = c(path_1, path_2), replace = TRUE, delim = "\t" ) # List the resources ("positions" and "positions_with_schema" added) resource_names(package)
Check if an object is a Data Package object with the required properties.
check_package(package)check_package(package)
package |
Data Package object, as returned by |
package invisibly or an error.
# Load the example Data Package package <- example_package() # Check if the Data Package is valid (invisible return) check_package(package)# Load the example Data Package package <- example_package() # Check if the Data Package is valid (invisible return) check_package(package)
Initiates a Data Package object, either from scratch or from an existing list. This Data Package object is a list with the following characteristics:
All properties of the original descriptor.
A resources property, set to an empty list if undefined.
A directory attribute, set to "." for the current directory if
undefined.
It is used as the base path to access resources with read_resource().
A datapackage subclass.
create_package(descriptor = NULL)create_package(descriptor = NULL)
descriptor |
List to be made into a Data Package object. If undefined, an empty Data Package will be created from scratch. |
See vignette("data-package") to learn how this function implements the
Data Package standard.
check_package() is automatically called on the created package to make sure
it is valid.
A Data Package object.
Other create functions:
create_schema()
# Create a Data Package package <- create_package() package # See the structure of the (empty) Data Package str(package)# Create a Data Package package <- create_package() package # See the structure of the (empty) Data Package str(package)
Creates a Table Schema for a data frame, listing all column names and types as field names and (converted) types.
create_schema(data)create_schema(data)
data |
A data frame. |
See vignette("table-schema") to learn how this function implements the
Data Package standard.
List describing a Table Schema.
Other create functions:
create_package()
# Create a data frame df <- data.frame( id = c(as.integer(1), as.integer(2)), timestamp = c( as.POSIXct("2020-03-01 12:00:00", tz = "EET"), as.POSIXct("2020-03-01 18:45:00", tz = "EET") ), life_stage = factor(c("adult", "adult"), levels = c("adult", "juvenile")) ) # Create a Table Schema from the data frame schema <- create_schema(df) str(schema)# Create a data frame df <- data.frame( id = c(as.integer(1), as.integer(2)), timestamp = c( as.POSIXct("2020-03-01 12:00:00", tz = "EET"), as.POSIXct("2020-03-01 18:45:00", tz = "EET") ), life_stage = factor(c("adult", "adult"), levels = c("adult", "juvenile")) ) # Create a Table Schema from the data frame schema <- create_schema(df) str(schema)
Reads the example Data Package included in frictionless.
This dataset is used in examples, vignettes, and tests and contains dummy
camera trap data organized in 3 Data Resources:
deployments: one local data file referenced in
"path": "deployments.csv".
observations: two local data files referenced in
"path": ["observations_1.tsv", "observations_2.tsv"].
media: inline data stored in data.
example_package(version = "2.0")example_package(version = "2.0")
version |
Data Package standard version number. |
The example Data Package is available in two versions:
A Data Package object, see create_package().
# Load the v2 example Data Package (default) example_package() # Load the v1 example Data Package example_package(version = "1.0")# Load the v2 example Data Package (default) example_package() # Load the v1 example Data Package example_package(version = "1.0")
Prints a human-readable summary of a Data Package, including its resources
and a link to more information (if provided in package$id).
## S3 method for class 'datapackage' print(x, ...)## S3 method for class 'datapackage' print(x, ...)
x |
Data Package object, as returned by |
... |
Further arguments, they are ignored by this function. |
print() with a summary of the Data Package object.
# Load the example Data Package package <- example_package() # Print a summary of the Data Package package # Or print(package)# Load the example Data Package package <- example_package() # Print a summary of the Data Package package # Or print(package)
datapackage.json)Reads information from a datapackage.json file, i.e. the descriptor file that
describes the Data Package metadata and its Data Resources.
read_package(file = "datapackage.json")read_package(file = "datapackage.json")
file |
Path or URL to a |
See vignette("data-package") to learn how this function implements the
Data Package standard.
A Data Package object, see create_package().
Other read functions:
read_resource()
# Read a datapackage.json file package <- read_package( system.file("extdata", "v1", "datapackage.json", package = "frictionless") ) package # Access the Data Package properties package$name package$created# Read a datapackage.json file package <- read_package( system.file("extdata", "v1", "datapackage.json", package = "frictionless") ) package # Access the Data Package properties package$name package$created
Reads data from a Data Resource (in a Data Package) into a tibble (a
Tidyverse data frame).
The resource must be a Tabular Data Resource.
The function uses readr::read_delim() to read CSV files, passing the
resource properties path, CSV dialect, column names, data types, etc.
Column names are taken from the provided Table Schema (schema), not from
the header in the CSV file(s).
read_resource(package, resource_name, col_select = NULL)read_resource(package, resource_name, col_select = NULL)
package |
Data Package object, as returned by |
resource_name |
Name of the Data Resource. |
col_select |
Character vector of the columns to include in the result, in the order provided. Selecting columns can improve read speed. |
See vignette("data-resource"), vignette("table-dialect") and
vignette("table-schema") to learn how this function implements the
Data Package standard.
A tibble::tibble() with the Data Resource's tabular data.
If there are parsing problems, a warning will alert you.
You can retrieve the full details by calling problems() on your data
frame.
Other read functions:
read_package()
# Read a datapackage.json file package <- read_package( system.file("extdata", "v1", "datapackage.json", package = "frictionless") ) package # Read data from the resource "observations" read_resource(package, "observations") # The above tibble is merged from 2 files listed in the resource path package$resources[[2]]$path # The column names and types are derived from the resource schema purrr::map_chr(package$resources[[2]]$schema$fields, "name") purrr::map_chr(package$resources[[2]]$schema$fields, "type") # Read data from the resource "deployments" with column selection read_resource(package, "deployments", col_select = c("latitude", "longitude"))# Read a datapackage.json file package <- read_package( system.file("extdata", "v1", "datapackage.json", package = "frictionless") ) package # Read data from the resource "observations" read_resource(package, "observations") # The above tibble is merged from 2 files listed in the resource path package$resources[[2]]$path # The column names and types are derived from the resource schema purrr::map_chr(package$resources[[2]]$schema$fields, "name") purrr::map_chr(package$resources[[2]]$schema$fields, "type") # Read data from the resource "deployments" with column selection read_resource(package, "deployments", col_select = c("latitude", "longitude"))
Removes a Data Resource from a Data Package, i.e. it removes one of the
described resources.
remove_resource(package, resource_name)remove_resource(package, resource_name)
package |
Data Package object, as returned by |
resource_name |
Name of the Data Resource. |
package with one fewer resource.
Other edit functions:
add_properties(),
add_resource()
# Load the example Data Package package <- example_package() # List the resources resource_names(package) # Remove the resource "observations" package <- remove_resource(package, "observations") # List the resources ("observations" removed) resource_names(package)# Load the example Data Package package <- example_package() # List the resources resource_names(package) # Remove the resource "observations" package <- remove_resource(package, "observations") # List the resources ("observations" removed) resource_names(package)
Gets or overwrites a Data Resource from/in a Data Package.
These functions are designed for internal use in other packages.
For public manipulation of resources, use add_resource() and
remove_resource().
resource() gets a Data Resource from a Data Package by its name.
The returned value will be a list describing a Data Resource, with a new
attribute data_location to indicate how the data are attached.
If present, path will be updated to the full path(s).
resource<- overwrites a Data Resource in Data Package with a new value.
The assigned value will typically be a resource obtained with resource()
and then manipulated.
The assignment function will therefore revert internal changes made by
resource() (i.e. removing the attribute data_location and updating paths
to the original values).
The assigned value is otherwise not validated, so use with care.
resource(package, resource_name) resource(package, resource_name) <- valueresource(package, resource_name) resource(package, resource_name) <- value
package |
Data Package object, as returned by |
resource_name |
Name of the Data Resource. |
value |
Value to assign. |
See vignette("data-resource") to learn more about Data Resource.
List describing a Data Resource.
package with overwritten resource.
Other accessor functions:
resource_names(),
schema()
# Load the example Data Package package <- example_package() # Get the resource "deployments" resource <- resource(package, "deployments") str(resource) # Update the resource resource$description <- "Table with deployments." # Overwrite the resource resource(package, "deployments") <- resource # Updating a resource property can also be done in one step resource(package, "deployments")$description <- "Table with deployments."# Load the example Data Package package <- example_package() # Get the resource "deployments" resource <- resource(package, "deployments") str(resource) # Update the resource resource$description <- "Table with deployments." # Overwrite the resource resource(package, "deployments") <- resource # Updating a resource property can also be done in one step resource(package, "deployments")$description <- "Table with deployments."
Lists the names of the Data Resources included in a Data Package.
resource_names(package)resource_names(package)
package |
Data Package object, as returned by |
Character vector with the Data Resource names.
Other accessor functions:
resource(),
schema()
# Load the example Data Package package <- example_package() # List the resources resource_names(package)# Load the example Data Package package <- example_package() # List the resources resource_names(package)
Gets the Table Schema of a Data Resource (in a Data Package), i.e. the
content of its schema property, describing the resource's fields, data
types, relationships, and missing values.
The resource must be a Tabular Data Resource.
schema(package, resource_name)schema(package, resource_name)
package |
Data Package object, as returned by |
resource_name |
Name of the Data Resource. |
See vignette("table-schema") to learn more about Table Schema.
List describing a Table Schema.
Other accessor functions:
resource(),
resource_names()
# Load the example Data Package package <- example_package() # Get the Table Schema for the resource "observations" schema <- schema(package, "observations") str(schema)# Load the example Data Package package <- example_package() # Get the Table Schema for the resource "observations" schema <- schema(package, "observations") str(schema)
Upgrades a Data Package, its Data Resources and Table Schemas to the v2 specification.
upgrade_package(package)upgrade_package(package)
package |
Data Package object, as returned by |
Upgraded package.
upgrade_package() upgrades a v1 descriptor to v2 as follows:
Adds $schema as first property and sets it to the recommended v2 value
("https://datapackage.org/profiles/2.0/datapackage.json"), except for
certain profile values.
Removes profile, but retains its value in $schema if it is a URL to a
custom profile (see backwards compatibility).
Converts contributors "role": "value" to "roles": ["value"] (see
changelog).
upgrade_package() upgrades any v1 resource to v2 as follows:
Adds $schema as first property and sets it to the recommended v2 value
("https://datapackage.org/profiles/2.0/dataresource.json").
Removes profile, but converts "profile" = "tabular-data-resource" to
"type" = "table" (see backwards compatibility).
upgrade_package() leaves the dialect as is for all resources.
upgrade_package() upgrades any verbose v1 schema to v2 as follows:
Adds $schema as first property and sets it to the recommended v2 value
("https://datapackage.org/profiles/2.0/tableschema.json").
Converts primaryKey single values to an array (see changelog).
Converts foreignKeys single values in fields to an array and removes
reference$resource if it is self-referential (see changelog).
Schemas referenced by path or URL are left as is.
Other versioning functions:
version()
# Load the v1 example Data Package (package <- example_package(version = "1.0")) # Upgrade (package_upgraded <- upgrade_package(package)) # Check version of one of the resources version(resource(package_upgraded, "deployments"))# Load the v1 example Data Package (package <- example_package(version = "1.0")) # Upgrade (package_upgraded <- upgrade_package(package)) # Check version of one of the resources version(resource(package_upgraded, "deployments"))
Determines what version of the Data Package, Data Resource, Table Dialect or
Table Schema standard is used, based on the $schema property.
"1.0": v1 specification.
Assumed if $schema is missing.
"2.0: v2 specification.
">=2.0": assumed if $schema is defined by deviating from the default
(e.g. an extension).
version(x)version(x)
x |
A list describing a Data Package, Data Resource, Table Dialect or Table Schema. |
Data Package standard version number.
Other versioning functions:
upgrade_package()
# Data Package package <- example_package() version(package) # Data Resource resource <- resource(package, "observations") version(resource) # Table Dialect version(resource$dialect) # Table Schema schema <- schema(package, "observations") version(schema)# Data Package package <- example_package() version(package) # Data Resource resource <- resource(package, "observations") version(resource) # Table Dialect version(resource$dialect) # Table Schema schema <- schema(package, "observations") version(schema)
Writes a Data Package and its Data Resources to disk as a datapackage.json
and CSV files.
write_package(package, directory, compress = FALSE)write_package(package, directory, compress = FALSE)
package |
Data Package object, as returned by |
directory |
Path to local directory to write files to. |
compress |
If |
package invisibly, as written to file.
write_package() will write data to CSV files depending on how these are
attached to the Data Resource:
Data frame:
add_resource(package, "media", data = df)
Data are written to a CSV file in directory using readr::write_csv().
The CSV file will have the same name as the resource, overwriting any
existing file with the same name.
Use compress = TRUE to gzip the CSV file.
One or more paths to local CSV files in a different directory:
add_resource(package, "media", data = "other-directory/media.csv")
CSV files are copied to directory, overwriting existing files with
the same name.
One or more paths to local CSV files in the same directory:
add_resource(package, "media", data = "directory/media.csv")
CSV files are left as is (no overwrite).
This allows you to read and write a datapackage.json to the same
directory, without altering the CSV files of resources you did not
manipulate.
One or more URLs to CSV files:
add_resource(package, "media", data = "https://example.org/media.csv")
Files are not downloaded.
Mix of URLs and paths to CSV files:
add_resource(
package, "media",
data = c("https://example.org/media.csv", "media.csv")
)
Remote CSV files are downloaded to directory, overwriting existing
files with the same name.
Local CSV files are handled as described above.
Inline data: No files are written.
In all above cases path is added or updated to the new file location(s)
when appropriate.
# Load the example Data Package from disk package <- read_package( system.file("extdata", "v1", "datapackage.json", package = "frictionless") ) package # Write the (unchanged) Data Package to disk write_package(package, directory = "my_directory") # Check files list.files("my_directory") # No files written for the "observations" resource, since those are all URLs. # No files written for the "media" resource, since it has inline data. # Clean up (don't do this if you want to keep your files) unlink("my_directory", recursive = TRUE)# Load the example Data Package from disk package <- read_package( system.file("extdata", "v1", "datapackage.json", package = "frictionless") ) package # Write the (unchanged) Data Package to disk write_package(package, directory = "my_directory") # Check files list.files("my_directory") # No files written for the "observations" resource, since those are all URLs. # No files written for the "media" resource, since it has inline data. # Clean up (don't do this if you want to keep your files) unlink("my_directory", recursive = TRUE)