Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/denoland/celld/llms.txt

Use this file to discover all available pages before exploring further.

celld makes two guarantees: one node owns a cell at a time, and a write is durable before celld acknowledges it. This page explains the mechanism behind each guarantee so you can verify it against the source code and evaluate candidate object stores. The short version: ownership records use conditional writes; the replication stream uses plain writes because the epoch in the key is the fence; and the acknowledgement path re-reads the ownership record, so a stale node cannot make a promise that the fleet does not keep.

​
The ownership record

Each cell has exactly one ownership record in the bucket. The record names the owning node’s session identifier and carries a fencing epoch. A node acquires a cell with a conditional write:
  • If no record exists: a conditional create that fails if the object is already present.
  • If a record exists: a compare-and-swap on the previous record that fails if the record changed since the read.
Because the bucket accepts only one such write, two nodes cannot simultaneously acquire the same cell. Every activation — whether a takeover from another node or a local wake from hibernation — advances the epoch, so each owner replicates under a fresh epoch and an epoch never has two writers.

​
Epoch prefix in replication

The replicator copies SQLite data for each cell to the bucket under an epoch-scoped prefix: 
cells/<cell>/ltx/e<epoch>/
These segment writes are plain, unconditional PUTs — intentionally so. The fence is the epoch in the key, not a condition on each request. A node that lost ownership can continue writing, but its writes land in a superseded prefix. When another node restores the cell it selects the current lineage, so the stale node’s writes cannot corrupt the new owner’s data, and the data path pays no conditional-write cost per segment. The prefix protects the data. It does not, alone, protect the promise to the client. The next two mechanisms close that gap.

​
The acknowledgement rule (RPO=0)

An output gate holds the HTTP response for each write until the replicator proves the write is present in the bucket. After that proof, celld performs one additional step before acknowledging:
  1. Read the ownership record.
  2. Acknowledge the write only if the record still names this node at this epoch.
A partitioned node can commit locally and replicate into its own superseded prefix, but its ownership read shows the new owner, so the client never receives an acknowledgement for a write that the surviving lineage does not contain. The check is a read of the record — not a clock comparison — so a paused process or a skewed clock cannot pass it. Set CELLD_OUTPUT_GATE=0 to remove this replication wait and acknowledge without proof. This trades the RPO=0 guarantee for lower write latency at the cost of possible data loss on node failure.

​
The epoch seal

A restore selects the newest epoch prefix that contains data. Because segment writes are unconditional, a fenced node can append to that prefix after the takeover. Without a further rule, a later restore could read that appended tail — writes that no client ever received an acknowledgement for. The seal closes this hole. The first activation that restores from an epoch writes a seal object at: 
cells/<cell>/ltx/e<epoch>.seal.json
This write uses a conditional create (fail if already present). The seal records the highest transaction ID that any restore of that epoch is allowed to read. Because the conditional create means the first restorer wins, every later restore reads the same cut. Every acknowledged write sits at or below the seal, because acknowledgement required an ownership read that preceded the takeover, and the takeover preceded the seal. The later writes of the fenced node sit above the seal and never become visible. If the seal write fails, the activation fails — a restore of an unsealed prefix would reopen the hole, so celld refuses it.

​
Self-fencing

A node that cannot reach the bucket cannot renew its lease and cannot replicate. Such a node must not own cells, so it fences itself: it stops issuing writes and releases its residency. Other nodes can then acquire those cells through the normal ownership-record mechanism. The failure of a node is a normal operational input, not a recovery procedure. The fleet heals automatically through the lease and ownership mechanisms rather than through a separate consensus or failure-detection service.

​
Required bucket properties

celld requires three properties from the object store:
1

Conditional create

A write that creates an object must fail if the object already exists. This is the mechanism that prevents two nodes from acquiring the same cell simultaneously.
2

Conditional overwrite

A write that updates an object must fail if the object changed after the read that preceded the write. This is the compare-and-swap used when transferring ownership.
3

Read-after-write consistency

A read that follows a successful write must return that write. A stale read after an ownership acquisition could cause a second node to believe the cell is unowned.

​
S3-compatible stores

On S3-compatible buckets celld uses two standard conditional headers:
HeaderSemantics
If-None-Match: *Conditional create — fail if any object already exists at this key
If-Match: <etag>Conditional overwrite — fail if the current etag does not match
Amazon S3, Cloudflare R2, and Tigris document and implement these operations. celld’s release tests run against Cloudflare R2, and the AWS S3 path uses the same client and the same headers.

​
Google Cloud Storage

A gs:// bucket selects the Cloud Storage XML API. celld uses the x-goog-if-generation-match precondition with OAuth credentials. The condition compares the object generation rather than an etag. celld does not send the S3 request dialect to Cloud Storage because Cloud Storage does not apply If-Match to a PUT.

​
Stores that do not qualify

The following stores do not implement the required conditional writes and therefore are not correct for celld:
  • MinIO (community edition)
  • Backblaze B2
  • Hetzner Object Storage
  • DigitalOcean Spaces
On these stores, two nodes can acquire ownership of the same cell simultaneously.
Some S3-compatible stores accept the conditional headers syntactically but do not apply the condition — they accept every write unconditionally. Such a store fails late and silently: the fleet appears to work until a split-brain scenario corrupts cell data. Test every candidate store’s conditional-write behaviour before trusting a fleet to it.

Build docs developers (and LLMs) love

Get started for freeTalk to us