# Datasets

Upload data to C3 once, then mount it in any job. Data moves between C3 storage and the machine over fast links; only the first upload from your laptop is slow.

## Workspace or dataset?[​](#workspace-or-dataset "Direct link to Workspace or dataset?")

`c3 deploy` already uploads your project folder, so small inputs can simply sit next to your script. Use a dataset instead when the data is:

* **large**: more than a few hundred megabytes. The workspace is re-hashed on every deploy and staged onto every machine; a dataset is uploaded once.
* **shared**: used by more than one project, or by a colleague. A dataset mounts into any job; a workspace belongs to one project.
* **versioned**: something you will update and want to reproduce. Datasets keep every version; a workspace is replaced on each deploy.
* **produced by a job**: mount `/jobs/<job-id>` directly rather than downloading and re-uploading. See [Artifacts](https://docs.cthree.cloud/artifacts.md#reuse-results-in-another-job).

If large data must stay in the project folder, exclude it from the upload with `.c3ignore` (see [Project configuration](https://docs.cthree.cloud/configuration.md#exclude-files)) and upload it as a dataset instead.

## Upload[​](#upload "Direct link to Upload")

```
c3 data cp ./my-data/ /datasets/my-data/
```

Every file is hashed before upload and skipped if C3 already has it, so re-uploading a dataset with a few changed files transfers only those files. Each upload creates a new immutable version.

## Mount in a job[​](#mount-in-a-job "Direct link to Mount in a job")

```
datasets:
  - ref: /datasets/my-data
    mount: /data/my-data
```

Your script reads the files at the mount path like local files:

```
data = np.loadtxt("/data/my-data/measurements.csv", delimiter=",")
```

Rules:

* `mount` must be absolute. If omitted, it is `/data/<dataset-name>`.
* `ref` can be a local directory, such as `./my-data`. C3 uploads it as a dataset before submitting.
* `ref` can be a previous job's results, `/jobs/<job-id>`. See [Artifacts](https://docs.cthree.cloud/artifacts.md#reuse-results-in-another-job).
* Without a version, a job gets the latest version at the moment it is submitted. Later uploads do not change a queued job.
* The same `datasets:` block works for [autoresearch](https://docs.cthree.cloud/autoresearch/configuration): every evaluation job of a run mounts the datasets, pinned when the run is submitted.

## Versions[​](#versions "Direct link to Versions")

Every upload creates a new version. Without a version, a job mounts whichever version is latest when it is submitted. Pin one to make a run reproducible:

```
c3 data log /datasets/my-data/           # versions, newest first, with their ids
```

```
datasets:
  - ref: /datasets/my-data/@a1b2c3d4e5f6   # an id from c3 data log
    mount: /data/my-data
```

The id is the start of the version's content hash. C3 expands it before submitting and refuses the deploy if it matches no version or more than one. `@latest` means the same as no version. Coding agents pass the same id in the `version` field of the MCP `deploy` tool.

Older versions can be inspected and downloaded the same way:

```
c3 data ls -l /datasets/my-data/@a1b2c3d4e5f6/
c3 data cp /datasets/my-data/@a1b2c3d4e5f6/ ./
```

## Browse and download[​](#browse-and-download "Direct link to Browse and download")

```
c3 data ls /datasets/                       # all datasets
c3 data ls /datasets/my-data/               # versions
c3 data ls -l /datasets/my-data/@latest/    # files in the latest version
c3 data cp /datasets/my-data/@latest/ ./    # download
c3 data du /datasets/my-data/               # storage used
c3 data rm -r /datasets/my-data/            # delete the whole dataset
```

Deleting the latest version makes the previous one current. Deleting the last version removes the dataset.

Datasets can also be scoped to a project at `/projects/<project>/data/<name>/`, with the same commands. The [dashboard](https://cthree.cloud/dashboard/data) shows the same data.

How storage works

Datasets, uploaded workspaces and job results all use one content-addressed store. Every file is a blob keyed by its SHA-256 hash; a manifest lists the blobs in each version. Identical files across datasets, jobs and workspaces are stored once, and storage usage is charged on unique bytes. Listings still show every file in every version.