OpenSandbox Network Architecture: Isolation and Routing

Networking in OpenSandbox is designed around one core idea: each sandbox is an independent security domain. Inbound traffic should only reach a sandbox through an authenticated, controlled path — never directly via raw IP. Outbound traffic should only reach explicitly allowed destinations. This document explains how these goals are achieved in single-host Docker deployments, in Kubernetes clusters, and through the per-sandbox egress sidecar that enforces outbound policy regardless of the underlying runtime.

Single-Host (Docker) Networking

When the Docker runtime is configured, the OpenSandbox server manages all port routing for sandboxes running on a single host. The execd reverse proxy is the key mechanism that makes this efficient.

How bridge mode works

In the default bridge network mode, each sandbox container is placed on an isolated Docker bridge network. The Docker runtime maps only one host port per sandbox — the execd proxy port (44772). No other container ports need to be published.

Host machine
│
│  ┌──────────────────────────────────────────────────────────┐
│  │  Bridge network (docker0 / custom bridge)                │
│  │                                                          │
│  │  Sandbox Container                                       │
│  │  ┌────────────────────────────────────────────────────┐  │
│  │  │                                                    │  │
│  │  │  execd  :44772  ──────────────────────────────┐   │  │
│  │  │  (reverse proxy)                               │   │  │
│  │  │                                                ▼   │  │
│  │  │  User service  :8000  ◀──── /proxy/8000 ───────    │  │
│  │  │  User service  :5900  ◀──── /proxy/5900 ───────    │  │
│  │  │  (any port)           ◀──── /proxy/{N}  ───────    │  │
│  │  └────────────────────────────────────────────────────┘  │
│  │                          ▲                               │
│  └──────────────────────────┼───────────────────────────────┘
│                             │ host_port:44772 (one mapping)
│  External caller ───────────┘
│  GET http://{host_ip}:{host_port}/proxy/8000/...
└─────────────────────────────────────────────────────────────

Routing path:

External traffic hits {host_ip}:{host_proxy_port} — the single published host port for this sandbox.
Docker forwards the request to execd inside the container on port 44772.
execd extracts the target port from the /proxy/{port} path prefix and forwards the request to 127.0.0.1:{port} inside the same container.
The response flows back through the same path.

Because execd preserves Upgrade, Connection, and related HTTP headers, this single proxy port handles HTTP, Server-Sent Events (SSE), and WebSocket traffic without any additional configuration. Endpoint resolution:

# Returns: http://{host_ip}:{host_proxy_port}/proxy/8000
endpoint = sandbox.get_endpoint(port=8000)

# Returns the container IP directly (for callers inside the same Docker network)
endpoint = sandbox.get_endpoint(port=8000, resolve_internal=True)

Port `8080` direct mapping

In addition to the proxy port, port 8080 may receive a direct host binding (labeled opensandbox.io/http-port) for applications that need a conventional HTTP endpoint without the /proxy/ path prefix.

Host network mode

In network_mode = "host", the sandbox container shares the host network stack. Each container port is directly accessible on the host — no proxy is needed. get_endpoint(port=X) returns {host_ip}:{X} directly.

Host mode limits you to one sandbox per machine unless you reserve non-overlapping port ranges per sandbox. It also prevents egress sidecar network isolation, because the sidecar cannot intercept traffic at the container network namespace level when there is no separate namespace.

Internal access shortcut

When the server itself runs inside a Docker container (e.g., via Docker Compose), callers on the same Docker network can bypass the host port mapping:

endpoint = sandbox.get_endpoint(port=8000, resolve_internal=True)
# Returns: http://172.17.0.3:8000 (direct container IP)

Kubernetes Networking

Kubernetes deployments use the OpenSandbox ingress gateway to route external traffic to sandbox pods. This provides a single, authenticated entry point that avoids exposing raw pod IPs.

Ingress gateway routing

The ingress component (components/ingress/) is an HTTP/WebSocket reverse proxy that watches BatchSandbox and AgentSandbox CRs across all namespaces. It discovers sandbox endpoints from the sandbox.opensandbox.io/endpoints annotation (BatchSandbox) or status.serviceFQDN (AgentSandbox) and routes incoming requests accordingly. Two routing strategies are supported:

Header mode
URI mode

Routes based on the OpenSandbox-Ingress-To: <sandbox-id>-<port> request header or the Host: <sandbox-id>-<port>.<domain> header.

Client
  │  OpenSandbox-Ingress-To: my-sandbox-8080
  ▼
Ingress gateway :28888
  │  extract sandbox-id="my-sandbox", port=8080
  │  lookup BatchSandbox "my-sandbox" → pod IP
  ▼
Sandbox pod :8080

Header mode is the default and is well-suited for server-side SDK and API clients that can set custom headers.

Routes based on the URI path structure: /<sandbox-id>/<port>/<path>.

Client
  │  GET /my-sandbox/8080/api/users
  ▼
Ingress gateway :28888
  │  extract sandbox-id="my-sandbox", port=8080, path="/api/users"
  ▼
Sandbox pod :8080/api/users

URI mode is useful when custom HTTP headers are not available (e.g., browser-embedded clients or simple curl commands).

Secure access

When secureAccess is enabled for a Kubernetes sandbox, the server provisions endpoint credentials and returns required headers with endpoint responses. This ensures that even if an ingress endpoint is publicly reachable, only authenticated callers can reach the sandbox.

Components deployed on Kubernetes

Component	Deployment	Network role
Server	Deployment	Receives SDK/API calls; provisions sandboxes
Ingress	DaemonSet or Deployment	Routes external HTTP/WS traffic to sandbox pods
Egress sidecar	Per-pod	Enforces outbound policy from inside each sandbox’s network namespace
Execd	Init container → sandbox pod	Exposes in-sandbox execution API

Egress Isolation

Every sandbox that is created with a networkPolicy gets an egress sidecar that runs inside the sandbox’s network namespace and enforces outbound traffic policy.

How the sidecar works

Sandbox network namespace
│
│  execd / user workload
│    │ DNS query (port 53)
│    ▼
│  iptables REDIRECT ──────────▶  Egress DNS proxy (127.0.0.1:15353)
│                                      │
│                              allowed?│
│                                      ├── YES → forward to upstream DNS
│                                      └── NO  → return NXDOMAIN
│
│  TCP connection to resolved IP
│    ▼
│  nftables (dns+nft mode)
│    ├── IP in allow set? → ACCEPT
│    └── otherwise        → DROP
│
│  Outbound allowed traffic
│    ▼
│  Internet / cluster services

The sidecar uses two enforcement layers:

DNS proxy (Layer 1) — intercepts all DNS traffic via iptables redirect and filters queries against the FQDN allowlist. Denied domains receive NXDOMAIN.
nftables (Layer 2, dns+nft mode) — enforces IP-level allow/deny. When a domain is resolved and allowed, its A/AAAA records are added to a dynamic nftables set with TTL. All other outbound IPs are dropped.

Default-deny with FQDN allowlist

from opensandbox import Sandbox, NetworkPolicy, EgressRule

sandbox = await Sandbox.create(
    image={"uri": "python:3.11-slim"},
    network_policy=NetworkPolicy(
        default_action="deny",
        egress=[
            EgressRule(action="allow", target="*.pypi.org"),
            EgressRule(action="allow", target="api.github.com"),
        ],
    ),
)

Platform-enforced isolation (deny.always)

For multi-tenant Kubernetes deployments, operators should bake cluster CIDRs into the egress sidecar image using a deny.always rule file. This blocks direct pod-to-pod and pod-to-service communication at the platform level, regardless of per-sandbox user policy.

FROM opensandbox/egress:latest

# Block cluster-internal traffic (adjust CIDRs for your cluster)
COPY deny.always /var/egress/rules/deny.always

# deny.always
10.244.0.0/16    # Pod CIDR
10.96.0.0/12     # Service CIDR

# server config
[egress]
image = "registry.example.com/opensandbox/egress:hardened"
mode = "dns+nft"

Effects of deny.always:

Sandboxes cannot reach other pods directly via Pod IP.
Sandboxes cannot reach in-cluster Services via ClusterIP.
Legitimate cross-sandbox communication must go through get_endpoint(), which returns an authenticated ingress gateway URL.
Platform operators do not need to change individual sandbox creation requests — isolation is enforced automatically.

deny.always rules are hot-reloaded every minute. To update after a cluster CIDR change, rebuild the image with the new rule file and perform a rolling update of the server egress configuration.

Allowing in-cluster Kubernetes services

When defaultAction: deny is set, allowing a sandbox to reach a Kubernetes Service requires both DNS and network allowances in dns+nft mode:

[
  { "action": "allow", "target": "postgres.opensandbox.svc.cluster.local" },
  { "action": "allow", "target": "10.96.0.0/12" }
]

Allowing only the DNS name is not sufficient if the resolved ClusterIP falls inside a denied CIDR. Allowing only the CIDR is not sufficient if the DNS proxy still blocks the hostname.

Rule precedence

deny.always  >  allow.always  >  user policy (API / env)

Always-rules are set by platform operators and cannot be overridden by user-supplied networkPolicy at sandbox creation time.

Runtime policy mutation

Egress policy can be inspected and updated at runtime without restarting the sandbox. Resolve the egress endpoint and call the HTTP API:

# Get current policy
curl http://sandbox-egress-endpoint:18080/policy

# Add an allow rule
curl -X PATCH http://sandbox-egress-endpoint:18080/policy \
  -d '[{"action":"allow","target":"*.example.com"}]'

Egress and Service Mesh Compatibility

The egress sidecar installs iptables/nft redirect rules in the shared pod network namespace. Transparent service meshes such as Istio/Envoy also redirect outbound traffic in that same namespace. When both are present, the redirect order is undefined and can produce double interception, broken TLS, or traffic that bypasses the egress policy path. Recommended operator choices:

Exclude sandbox pods from automatic mesh sidecar injection when they need the egress sidecar.
If mesh injection is mandatory, use a platform-level CNI/network-policy mechanism for outbound control instead of the OpenSandbox egress sidecar.

Secure Container Runtime Isolation

Beyond network-level isolation, OpenSandbox supports kernel-level sandbox isolation via secure container runtimes. These runtimes replace the standard runc container runtime with a stronger isolation boundary.

Runtime	Isolation mechanism	Docker	Kubernetes	Egress sidecar
gVisor (`runsc`)	User-space kernel (Go)	✅	✅	⚠️ Not compatible (no `iptables nat` table)
Kata Containers (`kata-qemu`)	Hardware VM	✅	✅	✅ Supported
Kata Firecracker (`kata-fc`)	microVM	—	✅	✅ Supported

If you need both syscall-level isolation and FQDN egress control, use kata-qemu instead of gVisor. Kata provides comparable security isolation and supports the egress sidecar’s iptables nat table.

Configure secure runtimes in [secure_runtime]:

[secure_runtime]
type = "kata-qemu"
oci_runtime_name = "kata-qemu"

On Kubernetes, the runtime is applied via a RuntimeClass resource. Pass secureRuntime: true in the sandbox creation request to opt in. See the Secure Container guide for the complete compatibility matrix and setup instructions.

Architecture Overview — Six-surface architecture and request flow
Components — Ingress and egress component reference
Docker Deployment — Single-host networking setup
Kubernetes Deployment — Kubernetes ingress and operator setup
Secure Container — gVisor, Kata, and Firecracker configuration
Credential Vault — Automatic credential injection via the egress sidecar

Get Started

SDKs

CLI & MCP

Guides

Deployment

Architecture

OpenSandbox Network Architecture: Isolation and Routing

Single-Host (Docker) Networking

How bridge mode works

Port `8080` direct mapping

Host network mode

Internal access shortcut

Kubernetes Networking

Ingress gateway routing

Secure access

Components deployed on Kubernetes

Egress Isolation

How the sidecar works

Default-deny with FQDN allowlist

Platform-enforced isolation (deny.always)

Allowing in-cluster Kubernetes services

Rule precedence

Runtime policy mutation

Egress and Service Mesh Compatibility

Secure Container Runtime Isolation

Build docs developers (and LLMs) love

Get Started

SDKs

CLI & MCP

Guides

Deployment

Architecture

Documentation Index

​Single-Host (Docker) Networking

​How bridge mode works

​Port 8080 direct mapping

​Host network mode

​Internal access shortcut

​Kubernetes Networking

​Ingress gateway routing

​Secure access

​Components deployed on Kubernetes

​Egress Isolation

​How the sidecar works

​Default-deny with FQDN allowlist

​Platform-enforced isolation (deny.always)

​Allowing in-cluster Kubernetes services

​Rule precedence

​Runtime policy mutation

​Egress and Service Mesh Compatibility

​Secure Container Runtime Isolation

​Related

Build docs developers (and LLMs) love

Single-Host (Docker) Networking

How bridge mode works

Port `8080` direct mapping

Host network mode

Internal access shortcut

Kubernetes Networking

Ingress gateway routing

Secure access

Components deployed on Kubernetes

Egress Isolation

How the sidecar works

Default-deny with FQDN allowlist

Platform-enforced isolation (deny.always)

Allowing in-cluster Kubernetes services

Rule precedence

Runtime policy mutation

Egress and Service Mesh Compatibility

Secure Container Runtime Isolation

Related