Fork KVM-isolated microVM sandboxes from a snapshot

Forking is the core operation in forkd. Given a parent snapshot (memory.bin + vmstate) on disk, the fork operation launches N independent Firecracker processes that each mmap the parent’s memory.bin with MAP_PRIVATE. The Linux kernel implements copy-on-write at the page level: every child starts with zero private pages and only allocates new physical pages as it writes to memory that diverges from the parent. The result is KVM-level hardware isolation per child at a spawn cost closer to fork(2) than to a cold VM boot — 100 children in 101 ms on a 20-vCPU Ubuntu 24.04 host.

CLI fork

sudo -E forkd fork --tag pyagent -n 100 --per-child-netns --memory-limit-mib 256

Key flags

Flag	Description
`--tag`	Snapshot tag to fork from. Must exist under `~/.local/share/forkd/snapshots/<tag>/`.
`-n`	Number of children to spawn.
`--per-child-netns`	Place each child in its own `forkd-child-<i>` network namespace. Run `sudo bash scripts/netns-setup.sh N` first.
`--memory-limit-mib`	Set `memory.max` on a per-child cgroup v2 leaf. Children exceeding this are OOM-killed. Requires cgroup v2 and write access to `/sys/fs/cgroup/forkd/`.
`--live-fork`	Boot each child with a memfd-backed RAM region (v0.4). Required if you later want to take a live BRANCH from a child. Requires Linux ≥ 5.7 and the vendored Firecracker fork.
`--hugepages`	Back the memfd with 2 MiB hugepages (only with `--live-fork`). Reduces TLB pressure during bulk spawn. Requires reserved hugepages on the host.
`--settle-secs`	Seconds to let children run before reporting / shutting down. Default `2`.
`--keep-workdir`	Keep `/tmp/forkd-fork-<tag>/` after shutdown for post-mortem inspection.

Example — fork 5 children, exec a command in one

# Provision 5 network namespaces (one-time)
sudo bash scripts/netns-setup.sh 5

# Fork 5 children
sudo -E forkd fork --tag pyagent -n 5 --per-child-netns --memory-limit-mib 256

# Talk to child 1 via the guest agent
sudo forkd exec --child forkd-child-1 -- python3 -c "import numpy; print(numpy.zeros(3))"

# Evaluate a Python expression in the warmed PID-1 interpreter (no subprocess overhead)
sudo forkd eval --child forkd-child-1 -- "numpy.zeros(100).sum()"

Benchmark output — `forkd bench`

forkd bench runs a representative spawn → exec → branch → fanout → cleanup cycle against a live daemon and prints per-step timing. Use it to verify forkd’s performance on your hardware after any configuration change:

forkd bench --tag py-numpy --n 5

forkd bench against snapshot py-numpy
  spawn (n=1)              61 ms  sb-...-0027
  exec round-trip          22 ms  exit=0
  branch (diff=true)      287 ms  pause_ms=234 diff_physical_bytes=393216
  fanout (n=5)             65 ms  13ms/child
  cleanup                 136 ms
                          -----
  total                   571 ms

The exec round-trip line is particularly telling: at 22 ms it reuses the already-warmed Python interpreter in PID 1. Compare this to sandbox.commands.run("python3 -c '...'") which costs ~96 ms because it spawns a cold subprocess that must re-import numpy.

Memory limits with cgroup v2

Pass --memory-limit-mib to set a hard RSS cap per child using the cgroup v2 memory.max knob:

sudo -E forkd fork --tag pyagent -n 50 --per-child-netns --memory-limit-mib 512

Requirements:

cgroup v2 unified hierarchy mounted at /sys/fs/cgroup (forkd doctor checks this)
Write access to /sys/fs/cgroup/forkd/ (root or a delegated cgroup)

Children that exceed memory.max are OOM-killed by the kernel — their processes terminate and the Firecracker instance exits, but other children are unaffected.

Measured CoW overhead at N=100 is 0.12 MiB per child on top of the parent. For a 512 MiB warmed Python+numpy parent with --memory-limit-mib 256, the practical ceiling before hitting vCPU or process-count limits is roughly 50 idle agents per 8 GiB of Pod/host RAM.

Live-fork mode (`--live-fork`)

Pass --live-fork to opt each child into a memfd-backed RAM region. This is a forward-looking flag: it has no effect at spawn time beyond swapping the memory backend, but it is required upfront if you later want to take a sub-50 ms live BRANCH from that child.

sudo -E forkd fork --tag pyagent -n 1 --per-child-netns --live-fork

Pair with --hugepages to back the memfd with 2 MiB hugepages, reducing TLB pressure during bulk spawn and live BRANCH bulk-copy:

sudo -E forkd fork --tag pyagent -n 10 --per-child-netns --live-fork --hugepages

--live-fork requires Linux ≥ 5.7, vm.unprivileged_userfaultfd=1 (or CAP_SYS_PTRACE), and the vendored Firecracker fork at deeplethe/firecracker:forkd-v0.4-mem-backend-shared-v1.12. Run forkd doctor to verify both uffd_wp and memfd_create checks pass before using this flag.

Daemon-managed fork (REST API)

For production deployments, drive forkd through the controller daemon instead of the CLI. Start the daemon:

sudo install -m 0644 packaging/systemd/forkd-controller.service /etc/systemd/system/
sudo mkdir -p /etc/forkd
sudo bash -c 'head -c 32 /dev/urandom | base64 > /etc/forkd/token'
sudo chmod 600 /etc/forkd/token
sudo systemctl enable --now forkd-controller

Then fork via POST /v1/sandboxes:

TOKEN=$(sudo cat /etc/forkd/token)

curl -s -H "Authorization: Bearer $TOKEN" \
     -H 'Content-Type: application/json' \
     -X POST http://127.0.0.1:8889/v1/sandboxes \
     -d '{
       "snapshot_tag": "pyagent",
       "n": 5,
       "per_child_netns": true,
       "memory_limit_mib": 256,
       "live_fork": false
     }'

Response (201 Created):

[
  {
    "id": "sb-67a1b3-0000",
    "snapshot_tag": "pyagent",
    "netns": "forkd-child-1",
    "guest_addr": "10.42.0.2:8888",
    "created_at_unix": 1717000123,
    "pid": 314159,
    "memory_limit_mib": 256
  },
  ...
]

Request fields:

n — 1 to 1000 children
per_child_netns — place each child in its pre-provisioned forkd-child-<i> namespace
memory_limit_mib — cgroup v2 memory.max per child (optional)
live_fork — enable memfd-backed RAM for later live BRANCH (v0.4+, default false)

See API Reference for the complete schema including SandboxInfo and error shapes.

Accessing a sandbox

Once a sandbox is running, communicate with the in-guest agent on port 8888.

Ping

# CLI
forkd ping --child forkd-child-1

# REST
curl -H "Authorization: Bearer $TOKEN" \
     -X POST http://127.0.0.1:8889/v1/sandboxes/sb-67a1b3-0000/ping
# {"pong":true,"numpy_version":"1.26.4","pid":1}

Exec (subprocess)

# CLI — run a subprocess in the guest
forkd exec --child forkd-child-1 -- python3 -c "print(2+2)"

# REST
curl -H "Authorization: Bearer $TOKEN" \
     -H 'Content-Type: application/json' \
     -X POST http://127.0.0.1:8889/v1/sandboxes/sb-67a1b3-0000/exec \
     -d '{"args":["python3","-c","print(2+2)"],"timeout_secs":30}'
# {"stdout":"4\n","stderr":"","exit_code":0}

Eval (warmed interpreter)

eval runs a Python expression against the already-warmed PID-1 interpreter — no subprocess spawn, no import overhead:

# CLI
forkd eval --child forkd-child-1 -- "numpy.zeros(5).sum()"

# REST
curl -H "Authorization: Bearer $TOKEN" \
     -H 'Content-Type: application/json' \
     -X POST http://127.0.0.1:8889/v1/sandboxes/sb-67a1b3-0000/eval \
     -d '{"code":"numpy.zeros(5).sum()"}'
# {"result":"0.0","error":null,"exit_code":0}

The eval path is ~1 ms per call vs ~96 ms for an exec that must cold-import numpy. For high-frequency agent interactions, always prefer eval over exec when your parent snapshot has the relevant module pre-imported. See /reference/cli/sandbox-commands and /reference/sdk/python for the Python SDK equivalent (Sandbox.eval, Sandbox.commands.run).

Cleanup

Kill specific sandboxes

forkd kill sb-67a1b3-0000
forkd kill sb-67a1b3-0000 sb-67a1b3-0001
forkd kill --tag pyagent    # kill all sandboxes forked from this tag
forkd kill --all            # kill every live sandbox the daemon knows

Sweep orphaned work directories

If forkd crashes or is killed with SIGKILL, temporary work directories under /tmp/forkd-* can accumulate. The cleanup command sweeps them:

forkd cleanup           # dry run — lists candidates
forkd cleanup --yes     # actually delete

The command skips any directory that has a live Firecracker process holding an open file handle inside it, so it is safe to run while other sandboxes are active.

Get Started

Guides

Recipes

Operations

Fork KVM-isolated microVM sandboxes from a snapshot

CLI fork

Key flags

Example — fork 5 children, exec a command in one

Benchmark output — `forkd bench`

Memory limits with cgroup v2

Live-fork mode (`--live-fork`)

Daemon-managed fork (REST API)

Accessing a sandbox

Ping

Exec (subprocess)

Eval (warmed interpreter)

Cleanup

Kill specific sandboxes

Sweep orphaned work directories

Build docs developers (and LLMs) love

Get Started

Guides

Recipes

Operations

Documentation Index

​CLI fork

​Key flags

​Example — fork 5 children, exec a command in one

​Benchmark output — forkd bench

​Memory limits with cgroup v2

​Live-fork mode (--live-fork)

​Daemon-managed fork (REST API)

​Accessing a sandbox

​Ping

​Exec (subprocess)

​Eval (warmed interpreter)

​Cleanup

​Kill specific sandboxes

​Sweep orphaned work directories

Build docs developers (and LLMs) love

CLI fork

Key flags

Example — fork 5 children, exec a command in one

Benchmark output — `forkd bench`

Memory limits with cgroup v2

Live-fork mode (`--live-fork`)

Daemon-managed fork (REST API)

Accessing a sandbox

Ping

Exec (subprocess)

Eval (warmed interpreter)

Cleanup

Kill specific sandboxes

Sweep orphaned work directories