---
title: "Data Resource"
author: "Peter Desmet"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Data Resource}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
```

[Data Resource](https://datapackage.org/standard/data-resource/) is a simple format to describe a data resource such as an individual table or file, including its name, format, path, etc.

::: {.callout-note}
In this document we use the terms "package" for Data Package, "resource" for Data Resource, "dialect" for Table Dialect, and "schema" for Table Schema.
:::

## General implementation

frictionless supports getting, reading, manipulating and writing resources, but much of its functionality is limited to [Tabular Data Resources](https://datapackage.org/standard/data-resource/#tabular).

### Access

`resource_names()` lists all resources in a package:

```{r}
library(frictionless)
package <- example_package()

# List the resources
resource_names(package)
```

`resource()` gets a resource from a package by its name:

```{r}
# Get the resource "deployments"
resource <- resource(package, "deployments")
str(resource)
```

`resource()<-` overwrites a resource in a package:

```{r}
resource(package, "deployments") <- resource
```

::: {.callout-important}
Note that `resource()` and `resource()<-` are designed for internal use in other packages. For public manipulation of resources, use `add_resource()` and `remove_resource()` (see [Manipulate](#manipulate)).
:::

### Read

`read_resource()` reads data from a tabular resource to a data frame:

```{r}
read_resource(package, "deployments")
```

frictionless does not support reading data from non-tabular resources.

### Manipulate

`remove_resource()` removes a resource (of any type):

```{r}
remove_resource(package, "deployments")

# This and many other functions return "package", which you can update with
# package <- remove_resource(package, "deployments")
```

`add_resource()` adds or replaces a tabular resource. **It always creates a resource following the [v2 specification](https://datapackage.org/standard/data-resource/).** The provided data must be a data frame or a tabular data file (e.g. CSV):

```{r}
# Add a resource with data from a data frame
add_resource(package, "iris", data = iris)

# Replace a resource with one where data is stored in a tabular file
path <- system.file("extdata", "v2", "deployments.csv", package = "frictionless")
add_resource(package, "deployments", data = path, replace = TRUE)
```

::: {.callout-info}
You can pipe most functions (see `vignette("data-package")`).
:::

### Write

`write_package()` writes a package and its related resources to disk as a `datapackage.json` and CSV files. See the function documentation for details.

## Properties implementation

### $schema

[`$schema`](https://datapackage.org/standard/data-resource/#dollar-schema) indicates what `version()` of the Data Resource standard is used (v1 if undefined).

- `read_resource()` ignores it and does not upgrade a resource, since it does not rely on v1 properties deprecated in v2.
- `add_resource()` sets `$schema` to the recommended v2 value (`"https://datapackage.org/profiles/2.0/dataresource.json"`) and thus creates a v2 resource.
- `upgrade_package()` set `$schema` to the recommended v2 value for all resources of a package.

### profile

[`profile`](https://specs.frictionlessdata.io/data-resource/#profile) (a deprecated v1 property) is ignored by `read_resource()` and not set by `add_resource()`. `upgrade_package()` removes `profile`  (unless `$schema` is already set), but converts `"profile" = "tabular-data-resource"` to `"type" = "table"` (for [backwards compatibility](https://datapackage.org/standard/data-resource/#type)).

### name

[`name`](https://datapackage.org/standard/data-resource/#name) is required. It is used to identify a resource in `read_resource()`, `add_resource()` and `remove_resource()` (always as the second argument):

```{r}
deployments <- read_resource(package, resource_name = "deployments")
```

`add_resource()` sets `name` to the provided `resource_name`:

```{r}
add_resource(package, resource_name = "iris", data = iris)
```

### path

[`path`](https://datapackage.org/standard/data-resource/#path-or-data) or `data` (see further) is required. Providing both is not allowed.

`path` is for data in files (e.g. a CSV file). It can be a local path or URL. Supported protocols are `http`, `https`, `ftp`, `sftp` and `sftp`. Absolute paths (`/`), relative parent paths (`../`) and paths containing hidden directories (starting with `.`) are not allowed to avoid security vulnerabilities.

When multiple paths are provided (`"path": ["myfile1.csv", "myfile2.csv"]`), the files are expected to have the same structure. `read_resource()` merges these into a single data frame in the order the paths are provided (using `dplyr::bind_rows()`):

```{r}
# The "observations" resource has multiple files in path
resource(package, "observations")$path
# These are combined into a single data frame when reading
read_resource(package, "observations")
```

`add_resource()` sets `path` to the path(s) provided in `data`:

```{r}
path <- system.file("extdata", "v2", "deployments.csv", package = "frictionless")
add_resource(package, "deployments", data = path, replace = TRUE)
```

### data

::: {.callout-warning}
Support for inline `data` is currently limited, e.g. JSON object and string are **not supported** and `schema`, `mediatype` and `format` are ignored.
:::

`data` is for inline data (included in the `datapackage.json`). `read_resource()` attempts to read `data` if it is provided as a JSON array:

```{r}
# The "media" resource has inline data
str(resource(package, "media")$data)
read_resource(package, "media")
```

`add_resource()` adds the provided data frame to `data`:

```{r}
df <- data.frame("col_1" = c(1, 2), "col_2" = c("a", "b"))
package <- add_resource(package, "df", df)
resource(package, "df")$data
```

`write_package()` writes that data frame to a CSV file, adds its path to `path` and removes `data`.

### type

[`type`](https://datapackage.org/standard/data-resource/#type) is ignored by `read_resource()`. `add_resource()` sets `type` to `"table"` to indicate that the resource is [tabular](https://datapackage.org/standard/data-resource/#tabular).

### schema

[`schema`](https://datapackage.org/standard/data-resource/#schema) is required. It is used by `read_resource()` to parse data types and missing values. It can either be a JSON object or a path or URL referencing a JSON object. See `vignette("table-schema")` for details.

### dialect

[`dialect`](https://datapackage.org/standard/data-resource/#dialect) is used by `read_resource()` to parse a tabular data file. It can either be a JSON object or a path or URL referencing a JSON object. See `vignette("table-dialect")` for details.

### title

[`title`](https://datapackage.org/standard/data-resource/#title) is ignored by `read_resource()` and not set by `add_resource()`, unless provided:

```{r}
package <- add_resource(
  package,
  "iris",
  iris,
  title = "Edgar Anderson's Iris Data",
  replace = TRUE
)
resource(package, "iris")$title
```

### description

[`description`](https://datapackage.org/standard/data-resource/#description) is ignored by `read_resource()` and not set by `add_resource()` unless provided (cf. `title`).

### format

[`format`](https://datapackage.org/standard/data-resource/#format) is ignored by `read_resource()`. `add_resource()` sets `format` when data are provided as a file, based on the provided `delim`:

delim | format
--- | ---
`","` (default) | `"csv"`
`"\t"` | `"tsv"`
any other value | `"csv"`

```{r}
path <- system.file("extdata", "v2", "observations_1.tsv", package = "frictionless")
package <- add_resource(package, "observations", data = path, delim = "\t", replace = TRUE)
resource(package, "observations")$format
```

`add_resource()` leaves `format` undefined when data are provided as a data frame. `write_package()` sets it to `"csv"` when writing to disk.

### mediatype

[`mediatype`](https://datapackage.org/standard/data-resource/#mediatype) is ignored by `read_resource()`. `add_resource()` sets `mediatype` when data are provided as a file, based on the provided `delim`:

delim | mediatype
--- | ---
`","` (default) | `"text/csv"`
`"\t"` | `"text/tab-separated-values"`
any other value | `"text/csv"`

```{r}
path <- system.file("extdata", "v2", "observations_1.tsv", package = "frictionless")
package <- add_resource(package, "observations", data = path, delim = "\t", replace = TRUE)
resource(package, "observations")$mediatype
```

`add_resource()` leaves `mediatype` undefined when data are provided as a data frame. `write_package()` sets it to `"text/csv"` when writing to disk.

### encoding

[`encoding`](https://datapackage.org/standard/data-resource/#encoding) (e.g. `"windows-1252"`) is used by `read_resource()` to parse the file. It defaults to UTF-8 if no `encoding` is provided or if it cannot be recognized. The returned data frame is always UTF-8.

`add_resource()` guesses the `encoding` (using `readr::guess_encoding()`) when data are provided as file. It leaves the `encoding` undefined when data are provided as a data frame. `write_package()` sets it to `"utf-8"` when writing to disk.

```{r}
path <- system.file("extdata", "v2", "deployments.csv", package = "frictionless")
package <- add_resource(package, "deployments", data = path, delim = ",", replace = TRUE)
resource(package, "observations")$encoding
```

### bytes

[`bytes`](https://datapackage.org/standard/data-resource/#bytes) is ignored by `read_resource()` and not set by `add_resource()` unless provided (cf. `title`).

### hash

[`hash`](https://datapackage.org/standard/data-resource/#hash) is ignored by `read_resource()` and not set by `add_resource()` unless provided (cf. `title`).

### sources

[`sources`](https://datapackage.org/standard/data-resource/#sources) is ignored by `read_resource()` and not set by `add_resource()` unless provided (cf. `title`).

### licenses

[`licenses`](https://datapackage.org/standard/data-resource/#licenses) is ignored by `read_resource()` and not set by `add_resource()` unless provided (cf. `title`).

### compression

[`compression`](https://datapackage.org/recipes/compression-of-resources/) (a recipe) is ignored by `read_resource()` and not set by `add_resource()`.

Compression is derived from the provided `path` instead. If the `path` ends in `.gz`, `.bz2`, `.xz`, or `.zip`, the files are automatically decompressed by `read_resource()` (using default `readr::read_delim()` functionality).