Skip to main content
Dataset endpoints allow you to register, retrieve, and manage dataset configurations in Amp. Datasets define how blockchain data is extracted and transformed.

​
List All Datasets

Returns all registered datasets across all namespaces with their version information.

​
Response

​
datasets
array
required
List of all datasets across all namespaces

​
Example Request

curl http://localhost:1610/datasets

​
Example Response

{
  "datasets": [
    {
      "namespace": "eth",
      "name": "blocks",
      "latest_version": "1.2.0",
      "versions": ["1.2.0", "1.1.0", "1.0.0"]
    }
  ]
}

​
Register Dataset

Registers a new dataset configuration in the local registry.
This endpoint only registers datasets and does NOT schedule data extraction. To extract data after registration, call the deploy endpoint.

​
Request Body

​
namespace
string
required
Namespace for the dataset (validated identifier format)
​
name
string
required
Name of the dataset to be registered (validated identifier format)
​
manifest
object | string
required
Either a manifest hash (64-char hex string) or full manifest JSON content
​
version
string
Optional semantic version (e.g., “1.0.0”). If omitted, only the “dev” tag is updated.

​
Example Request

curl -X POST http://localhost:1610/datasets \
  -H "Content-Type: application/json" \
  -d '{
    "namespace": "eth",
    "name": "blocks",
    "version": "1.0.0",
    "manifest": {
      "kind": "evm-rpc",
      "network": "mainnet",
      "provider": "alchemy-mainnet",
      "start_block": 0
    }
  }'

​
Response Codes

  • 201 Created - Dataset successfully registered
  • 400 Bad Request - Invalid dataset name, version, or manifest
  • 500 Internal Server Error - Database or object store error

​
Get Dataset by Revision

Returns detailed dataset information for the specified revision.

​
Path Parameters

​
namespace
string
required
Dataset namespace
​
name
string
required
Dataset name
​
revision
string
required
Revision - can be a version (e.g., “1.2.3”), hash, “latest”, or “dev”

​
Response

​
namespace
string
required
Dataset namespace
​
name
string
required
Dataset name
​
revision
string
required
Revision requested
​
manifest_hash
string
required
SHA256 manifest hash
​
kind
string
required
Dataset kind (e.g., “evm-rpc”, “firehose”, “manifest”)
​
start_block
integer
required
Starting block number
​
finalized_blocks_only
boolean
required
Whether to extract only finalized blocks
​
tables
array
required
List of table names in the dataset

​
Example Request

curl http://localhost:1610/datasets/eth/blocks/versions/latest

​
Example Response

{
  "namespace": "eth",
  "name": "blocks",
  "revision": "latest",
  "manifest_hash": "8b065bde9c1a2f3e4d5c6b7a8e9f0a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8",
  "kind": "evm-rpc",
  "start_block": 0,
  "finalized_blocks_only": false,
  "tables": ["blocks", "transactions", "logs"]
}

​
Deploy Dataset

Schedules a data extraction job for the specified dataset revision.

​
Path Parameters

​
namespace
string
required
Dataset namespace
​
name
string
required
Dataset name
​
revision
string
required
Revision (version, hash, “latest”, or “dev”)

​
Request Body

​
end_block
string | number | null
End block configuration:
  • null (default): Continuous dumping
  • "latest": Stop at latest block
  • <number>: Stop at specific block
  • <negative>: Stop N blocks before latest
​
parallelism
integer
Number of parallel workers (default: 1, only for raw datasets)
​
worker_id
string
Optional worker selector - exact ID or glob pattern

​
Response

​
job_id
integer
required
The ID of the scheduled extraction job

​
Example Request

curl -X POST http://localhost:1610/datasets/eth/blocks/versions/latest/deploy \
  -H "Content-Type: application/json" \
  -d '{
    "end_block": "latest",
    "parallelism": 4
  }'

​
Example Response

{
  "job_id": 12345
}

​
List Dataset Versions

Returns all versions for a dataset with their metadata.

​
Path Parameters

​
namespace
string
required
Dataset namespace
​
name
string
required
Dataset name

​
Response

​
namespace
string
required
Dataset namespace
​
name
string
required
Dataset name
​
versions
array
required
List of semantic versions (sorted descending)
​
special_tags
object
required
Special tags (latest and dev)

​
Example Request

curl http://localhost:1610/datasets/eth/blocks/versions

​
Delete Dataset

Removes all manifest links and version tags for a dataset. This operation is fully idempotent.

​
Path Parameters

​
namespace
string
required
Dataset namespace
​
name
string
required
Dataset name

​
Response Codes

  • 204 No Content - Dataset successfully deleted (or didn’t exist)
  • 400 Bad Request - Invalid path parameters
  • 500 Internal Server Error - Database operation error

​
Example Request

curl -X DELETE http://localhost:1610/datasets/eth/blocks

​
Delete Dataset Version

Removes a semantic version tag from a dataset. Cannot delete the “latest” version.

​
Path Parameters

​
namespace
string
required
Dataset namespace
​
name
string
required
Dataset name
​
version
string
required
Semantic version to delete (e.g., “1.2.3”)

​
Response Codes

  • 204 No Content - Version successfully deleted
  • 400 Bad Request - Invalid parameters or attempting to delete “latest”
  • 500 Internal Server Error - Database operation error

​
Example Request

curl -X DELETE http://localhost:1610/datasets/eth/blocks/versions/1.0.0

Build docs developers (and LLMs) love

Get started for freeTalk to us