execd - OpenSandbox

Overview

execd is the execution daemon for OpenSandbox. Built on Beego, it exposes a comprehensive HTTP API that turns external requests into runtime actions: managing Jupyter sessions, streaming code output via Server-Sent Events (SSE), executing shell commands, operating on the sandbox filesystem, and collecting host-side metrics.

Core Features

Unified Runtime Management

Translate REST calls into runtime requests handled by pkg/runtime
Multiple execution backends: Jupyter, shell, etc.
Automatic language detection and routing
Pluggable Jupyter server configuration

Jupyter Integration

Maintain kernel sessions via pkg/jupyter
WebSocket-based real-time communication
Stream execution events through SSE
Support for multiple language kernels

Command Executor

Foreground and background shell commands
Proper signal forwarding with process groups
Real-time stdout/stderr streaming
Context-aware interruption

Filesystem Operations

CRUD helpers around the sandbox filesystem
Glob-based file search
Chunked upload/download with resume support
Permission management

Observability

Lightweight metrics endpoint (CPU, memory, uptime)
Structured streaming logs
SSE-based real-time monitoring

Architecture

Directory Structure

Path	Purpose
`main.go`	Entry point; initializes Beego, CLI flags, routers
`pkg/flag/`	CLI and environment configuration
`pkg/web/`	HTTP layer (controllers, models, router, SSE helpers)
`pkg/web/controller/`	Handlers for files, code, commands, metrics
`pkg/web/model/`	Request/response models and SSE event types
`pkg/runtime/`	Dispatcher to Jupyter and shell executors
`pkg/jupyter/`	Minimal Jupyter client (kernels/sessions/WebSocket)
`pkg/jupyter/execute/`	Execution result types and stream parsers
`pkg/jupyter/session/`	Session management and lifecycle
`pkg/util/`	Utilities (safe goroutine helpers, glob helpers)
`tests/`	Test scripts and tools

Getting Started

Prerequisites

Go 1.24+ (as defined in go.mod)
Jupyter Server (required for code execution)
Docker (optional, for containerized builds)
Make (optional, for convenience targets)

Quick Start

Clone and Build

git clone git@github.com:alibaba/OpenSandbox.git
cd OpenSandbox/components/execd
go mod download
make build

Start Jupyter Server

# Option 1: use the provided script
./tests/jupyter.sh

# Option 2: start manually
jupyter notebook --port=54321 --no-browser --ip=0.0.0.0 \
  --NotebookApp.token='your-jupyter-token'

Run execd

./bin/execd \
  --jupyter-host=http://127.0.0.1:54321 \
  --jupyter-token=your-jupyter-token \
  --port=44772

Verify

curl -v http://localhost:44772/ping
# Expect HTTP 200

Docker Image Build

docker build -t opensandbox/execd:dev .

# Run container
docker run -d \
  -p 44772:44772 \
  -e JUPYTER_HOST=http://jupyter-server \
  -e JUPYTER_TOKEN=your-token \
  --name execd \
  opensandbox/execd:dev

Configuration

Command-line Flags

Flag	Type	Default	Description
`--jupyter-host`	string	`""`	Jupyter server URL (reachable by execd)
`--jupyter-token`	string	`""`	Jupyter HTTP/WebSocket token
`--port`	int	`44772`	HTTP listen port
`--log-level`	int	`6`	Beego log level (0=Emergency, 7=Debug)
`--access-token`	string	`""`	Shared API secret (optional)
`--graceful-shutdown-timeout`	duration	`3s`	Wait time before cutting off SSE on shutdown

Environment Variables

All flags can be set via environment variables:

export JUPYTER_HOST=http://127.0.0.1:8888
export JUPYTER_TOKEN=your-token
export EXECD_API_GRACE_SHUTDOWN=2s
export EXECD_LOG_FILE=/var/log/execd.log

Environment variables override defaults but are superseded by explicit CLI flags.

Graceful Shutdown

Controls how long execd keeps SSE responses alive after sending the final chunk:

Env: EXECD_API_GRACE_SHUTDOWN (e.g., 500ms, 2s, 1m)
Flag: --graceful-shutdown-timeout
Default: 1s

Set to 0s to disable the grace period.

API Reference

See the execd API Specification for complete endpoint documentation.

Key Endpoints

Endpoint	Method	Description
`/ping`	GET	Health check
`/code`	POST	Execute code via Jupyter
`/command`	POST	Run synchronous shell command
`/files/*`	GET/POST/PUT/DELETE	File operations
`/metrics`	GET	System metrics snapshot
`/metrics/watch`	GET	Real-time metrics via SSE

Supported Languages

Jupyter-based Execution

Language	Kernel	Highlights
Python	IPython	Full Jupyter protocol
Java	IJava	JShell-based execution
JavaScript	IJavaScript	Node.js runtime
TypeScript	ITypeScript	TS compilation + Node exec
Go	gophernotes	Go interpreter
Bash	Bash kernel	Shell scripts

Native Executors

Mode/Language	Backend	Highlights
`command`	OS exec	Synchronous shell commands
`background-command`	OS exec	Detached background process

Code Execution Example

Execute Python Code

curl -X POST -H "Content-Type: application/json" \
  -d '{
    "language": "python",
    "code": "import numpy as np\nprint(np.array([1,2,3]).sum())"
  }' \
  http://localhost:44772/code

Execute with Session

# Create session
curl -X POST -H "Content-Type: application/json" \
  -d '{"language": "python"}' \
  http://localhost:44772/sessions

# Execute in session (preserves state)
curl -X POST -H "Content-Type: application/json" \
  -d '{
    "language": "python",
    "code": "x = 42",
    "sessionId": "session-123"
  }' \
  http://localhost:44772/code

# Reuse variable
curl -X POST -H "Content-Type: application/json" \
  -d '{
    "language": "python",
    "code": "print(x)",
    "sessionId": "session-123"
  }' \
  http://localhost:44772/code

Filesystem Operations

List Files

curl http://localhost:44772/files/list?path=/workspace

Upload File

curl -X POST -F "file=@local-file.txt" \
  http://localhost:44772/files/upload?path=/workspace/file.txt

Download File

curl -O http://localhost:44772/files/download?path=/workspace/file.txt

Search Files

curl "http://localhost:44772/files/search?pattern=*.py&path=/workspace"

Observability

Logging

Beego leveled logger with configurable output:

# Log to file
export EXECD_LOG_FILE=/var/log/execd.log

# Log to stdout (default)
unset EXECD_LOG_FILE

Log Levels (0-7):

0: Emergency
1: Alert
2: Critical
3: Error
4: Warning
5: Notice
6: Info (default)
7: Debug

Metrics

Snapshot:

curl http://localhost:44772/metrics

Response:

{
  "cpu_percent": 12.5,
  "memory_total_gb": 16.0,
  "memory_used_gb": 4.2,
  "memory_percent": 26.25,
  "uptime_seconds": 3600,
  "timestamp": "2024-01-15T10:30:00Z"
}

Real-time Monitoring (SSE, 1s cadence):

curl http://localhost:44772/metrics/watch

Performance Benchmarks

Typical Latency (localhost)

Operation	Latency
`/ping`	< 1ms
`/files/info`	< 5ms
Code execution (Python)	50-200ms
File upload (1MB)	10-50ms
Metrics snapshot	< 10ms

Resource Usage (idle)

Memory: ~50MB
CPU: < 1%
Goroutines: ~15

Scalability

100+ concurrent SSE connections
File operations scale linearly with file size
Jupyter sessions are stateful and need dedicated resources

Testing

Unit Tests

make test

Integration Tests

Integration tests requiring a real Jupyter Server are skipped by default:

export JUPYTER_URL=http://localhost:8888
export JUPYTER_TOKEN=your-token
go test -v ./pkg/jupyter/...

Manual Testing Workflow

Start Jupyter

./tests/jupyter.sh

Start execd

./bin/execd --jupyter-host=http://localhost:54321 \
  --jupyter-token=opensandboxexecdlocaltest

Execute Code

curl -X POST -H "Content-Type: application/json" \
  -d '{"language":"python","code":"print(\"test\")"}' \
  http://localhost:44772/code

Development

See DEVELOPMENT.md for detailed guidelines.

Code Quality

# Format code
make fmt

# Run linter
make lint

# Run tests
make test

Support

Issues: GitHub Issues
Documentation: OpenSandbox Docs
Community: Discussions

​Overview

​Core Features

​Unified Runtime Management

​Jupyter Integration

​Command Executor

​Filesystem Operations

​Observability

​Architecture

​Directory Structure

​Getting Started

​Prerequisites

​Quick Start

​Docker Image Build

​Configuration

​Command-line Flags

​Environment Variables

​Graceful Shutdown

​API Reference

​Key Endpoints

​Supported Languages

​Jupyter-based Execution

​Native Executors

​Code Execution Example

​Execute Python Code

​Execute with Session

​Filesystem Operations

​List Files

​Upload File

​Download File

​Search Files

​Observability

​Logging

​Metrics

​Performance Benchmarks

​Typical Latency (localhost)

​Resource Usage (idle)

​Scalability

​Testing

​Unit Tests

​Integration Tests

​Manual Testing Workflow

​Development

​Code Quality

​Support

Overview

Core Features

Unified Runtime Management

Jupyter Integration

Command Executor

Filesystem Operations

Observability

Architecture

Directory Structure

Getting Started

Prerequisites

Quick Start

Docker Image Build

Configuration

Command-line Flags

Environment Variables

Graceful Shutdown

API Reference

Key Endpoints

Supported Languages

Jupyter-based Execution

Native Executors

Code Execution Example

Execute Python Code

Execute with Session

Filesystem Operations

List Files

Upload File

Download File

Search Files

Observability

Logging

Metrics

Performance Benchmarks

Typical Latency (localhost)

Resource Usage (idle)

Scalability

Testing

Unit Tests

Integration Tests

Manual Testing Workflow

Development

Code Quality

Support