Store Configuration

Stores handle data persistence in NativeLink. Each store has a unique name and type.

Store Definition

name

string

required

Unique identifier for the store. Used to reference the store in other configuration sections.

<type>

object

required

Store type and its configuration. Only one type can be specified per store.

Store Types

In-memory hashmap store with optional eviction.

{
  name: "MEMORY_STORE",
  memory: {
    eviction_policy: {
      max_bytes: "1gb"
    }
  }
}

eviction_policy

object

Controls when items are evicted from the store.

Show properties

max_bytes

number

default:"0"

Maximum bytes before eviction (0 = unlimited).

evict_bytes

number

default:"0"

Bytes to evict below max_bytes to create low watermark.

max_seconds

number

default:"0"

Maximum age in seconds (0 = no time-based eviction).

max_count

number

default:"0"

Maximum number of items (0 = no count-based eviction).

Persistent local filesystem store.

{
  name: "FS_STORE",
  filesystem: {
    content_path: "/var/cache/nativelink/content",
    temp_path: "/var/cache/nativelink/tmp",
    eviction_policy: { max_bytes: "50gb" }
  }
}

content_path

string

required

Directory for storing content. Scanned on startup.

temp_path

string

required

Temporary directory for uploads. Must be on same filesystem as content_path for atomic moves.

eviction_policy

object

Eviction policy (see Memory store).

read_buffer_size

number

default:"32768"

Buffer size for file reads (bytes).

block_size

number

default:"4096"

Filesystem block size for size calculations (bytes).

max_concurrent_writes

number

default:"0"

Limit concurrent writes to prevent I/O saturation (0 = unlimited).

AWS S3 cloud object storage.

{
  name: "S3_STORE",
  experimental_cloud_object_store: {
    provider: "aws",
    region: "us-east-1",
    bucket: "my-bucket",
    key_prefix: "nativelink/",
    retry: {
      max_retries: 6,
      delay: 0.3,
      jitter: 0.5
    }
  }
}

provider

string

required

Must be "aws".

region

string

required

AWS region (e.g., us-east-1, eu-west-1).

bucket

string

required

S3 bucket name.

key_prefix

string

Optional prefix for all keys.

retry

object

Retry configuration.

Show properties

max_retries

number

default:"0"

Maximum retry attempts (0 = unlimited).

delay

number

default:"0"

Initial delay in seconds for exponential backoff.

jitter

number

default:"0"

Jitter percentage (0-1) to add randomness.

multipart_max_concurrent_uploads

number

default:"10"

Concurrent multipart upload parts.

max_retry_buffer_per_request

number

default:"5242880"

Buffer size for retryable uploads (bytes). Set to 0 to disable buffering.

consider_expired_after_s

number

default:"0"

Treat objects older than this as expired (0 = never expire).

Google Cloud Storage.

{
  name: "GCS_STORE",
  experimental_cloud_object_store: {
    provider: "gcs",
    bucket: "my-bucket",
    key_prefix: "nativelink/"
  }
}

provider

string

required

Must be "gcs".

bucket

string

required

GCS bucket name.

resumable_chunk_size

number

default:"2097152"

Chunk size for resumable uploads (bytes).

authentication_required

boolean

default:"false"

Error if authentication is not found.

Common fields: key_prefix, retry, multipart_max_concurrent_uploads, max_retry_buffer_per_request, consider_expired_after_s

Redis key-value store.

{
  name: "REDIS_STORE",
  redis_store: {
    addresses: ["redis://127.0.0.1:6379/"],
    mode: "standard"
  }
}

addresses

array

required

Redis server URLs. Format: redis://[user:pass@]host:port/db

mode

enum

default:"standard"

Redis mode.Options: standard, cluster, sentinel

key_prefix

string

default:""

Prefix for all keys.

command_timeout_ms

number

default:"10000"

Command timeout (milliseconds).

connection_timeout_ms

number

default:"3000"

Connection timeout (milliseconds).

read_chunk_size

number

default:"65536"

Data read chunk size (bytes).

connection_pool_size

number

default:"3"

Number of connections to maintain.

Remote GRPC store proxy.

{
  name: "REMOTE_STORE",
  grpc: {
    instance_name: "main",
    endpoints: [
      { address: "grpc://remote:50051" }
    ],
    store_type: "cas"
  }
}

instance_name

string

default:""

Instance name for GRPC calls.

endpoints

array

required

List of GRPC endpoints.

Show endpoint properties

address

string

required

Endpoint URL (e.g., grpc://host:port).

concurrency_limit

number

Maximum concurrent requests to this endpoint.

store_type

enum

required

Type of store.Options: cas, ac

connections_per_endpoint

number

default:"1"

TCP connections per endpoint for load balancing.

max_concurrent_requests

number

default:"0"

Global concurrent request limit (0 = unlimited).

Validation wrapper for stores.

{
  name: "VERIFIED_CAS",
  verify: {
    backend: { /* store config */ },
    verify_size: true,
    verify_hash: true
  }
}

backend

object

required

Underlying store configuration.

verify_size

boolean

default:"false"

Verify blob size matches digest. Use true for CAS, false for AC.

verify_hash

boolean

default:"false"

Verify blob hash matches digest. Use true for CAS, false for AC.

Two-tier caching store.

{
  name: "CACHE_STORE",
  fast_slow: {
    fast: { memory: { eviction_policy: { max_bytes: "1gb" } } },
    slow: { filesystem: { /* ... */ } }
  }
}

fast

object

required

Fast store (cache layer).

slow

object

required

Slow store (backing storage).

fast_direction

enum

default:"both"

How fast store is used.Options: both, update, get, read_only

slow_direction

enum

default:"both"

How slow store is used.Options: both, update, get, read_only

Transparent compression wrapper.

{
  name: "COMPRESSED_STORE",
  compression: {
    backend: { /* store config */ },
    compression_algorithm: {
      lz4: { block_size: "64kb" }
    }
  }
}

backend

object

required

Underlying store.

compression_algorithm

object

required

Algorithm configuration.

Show lz4

block_size

number

default:"65536"

Compression block size (bytes).

max_decode_block_size

number

Maximum decode size for security (defaults to block_size).

Distributes data across multiple stores.

{
  name: "SHARDED_STORE",
  shard: {
    stores: [
      { store: { /* config */ }, weight: 1 },
      { store: { /* config */ }, weight: 2 }
    ]
  }
}

stores

array

required

List of stores with weights.

Show shard config

store

object

required

Store configuration.

weight

number

default:"1"

Distribution weight.

Reference to a named store.

{
  name: "ALIAS_STORE",
  ref_store: {
    name: "ACTUAL_STORE"
  }
}

name

string

required

Name of existing store to reference.

Discards all data. Useful for testing or partitioning.

{
  name: "DISCARD_STORE",
  noop: {}
}

Specialized Stores

Completeness Checking

Verifies Action Cache results reference existing CAS objects.

{
  name: "AC_STORE",
  completeness_checking: {
    backend: { /* AC store */ },
    cas_store: { ref_store: { name: "CAS_STORE" } }
  }
}

Existence Cache

Caches has() calls for faster lookups.

{
  name: "CACHED_STORE",
  existence_cache: {
    backend: { /* store config */ },
    eviction_policy: { max_seconds: 100 }
  }
}

Dedup Store

Content-defined chunking for deduplication.

{
  name: "DEDUP_STORE",
  dedup: {
    index_store: { /* small/fast store */ },
    content_store: { /* large store */ },
    min_size: "64kb",
    normal_size: "256kb",
    max_size: "512kb"
  }
}

Size Partitioning

Routes blobs by size.

{
  name: "PARTITIONED_STORE",
  size_partitioning: {
    size: "128mb",
    lower_store: { /* small blob store */ },
    upper_store: { /* large blob store */ }
  }
}

Configuration Reference

Examples

Store Definition

Store Types

Specialized Stores

Completeness Checking

Existence Cache

Dedup Store

Size Partitioning

Build docs developers (and LLMs) love

Configuration Reference

Examples

​Store Definition

​Store Types

​Specialized Stores

​Completeness Checking

​Existence Cache

​Dedup Store

​Size Partitioning

Build docs developers (and LLMs) love

Store Definition

Store Types

Specialized Stores

Completeness Checking

Existence Cache

Dedup Store

Size Partitioning