Running Locally

Sprout consists of three services that must run simultaneously for full functionality:

Backend API (Express + TypeScript)
Hand Tracking Service (Python + WebSocket)
Frontend (Next.js 16)

Quick Start

Start all three services in separate terminal windows:

Start Hand Tracking Service

cd sprout-backend
conda activate sprout-cv
python backend.py

Expected output:

WebSocket Server started on ws://localhost:8765

Start Backend API

cd sprout-backend
npm run dev

Expected output:

Default user seeded.
Sprout backend running on http://localhost:8000

Start Frontend

cd sprout-frontend
npm run dev

Expected output:

▲ Next.js 16.1.6
- Local:        http://localhost:3000

Open in Browser

Navigate to http://localhost:3000 and start using Sprout.

All three services must be running for full functionality. Hand tracking can be skipped if you don’t need that feature.

Service Details

1. Hand Tracking Service

The Python WebSocket server captures webcam input and streams hand landmark data to the frontend.

cd sprout-backend
conda activate sprout-cv
python backend.py

Configuration:

Port: 8765
Protocol: WebSocket (ws://localhost:8765)
Frame Rate: 60 fps (capped)
Camera: Index 0 (default webcam)

Make sure your webcam is not in use by another application before starting.

2. Backend API

The Express + TypeScript backend provides REST endpoints and Server-Sent Events for agent streaming.

cd sprout-backend
npm run dev
# Uses tsx watch for hot reloading

Endpoints:

Health Check: http://localhost:8000/api/health
API Base: http://localhost:8000/api
SSE Streaming: Various agent endpoints

Auto-Seeding: On startup, the backend automatically creates a default user:

DEFAULT_USER_ID = "00000000-0000-0000-0000-000000000000"
email = "[email protected]"

Sprout is a single-user application with no authentication. The default user is auto-created on first startup.

3. Frontend

The Next.js 16 frontend renders the 3D knowledge graph and provides the learning interface.

cd sprout-frontend
npm run dev
# Starts Next.js dev server with hot reloading

Features:

3D graph visualization (Three.js + React Flow)
Real-time agent activity streaming
Hand tracking integration
Interactive tutoring sessions

Proxy Configuration: The frontend proxies /backend-api/* requests to the backend:

next.config.ts

async rewrites() {
  return [
    {
      source: "/backend-api/:path*",
      destination: "http://localhost:8000/:path*",
    },
  ];
}

Development Workflow

Recommended Terminal Setup

Use a terminal multiplexer like tmux or split terminals:

tmux
Separate Windows

# Create a new tmux session
tmux new -s sprout

# Split into three panes
# Ctrl+b then %  (vertical split)
# Ctrl+b then "  (horizontal split)

# Pane 1: Hand tracking
cd sprout-backend
conda activate sprout-cv
python backend.py

# Pane 2: Backend
cd sprout-backend
npm run dev

# Pane 3: Frontend
cd sprout-frontend
npm run dev

Open three terminal windows:Terminal 1: Hand Tracking

cd sprout-backend && conda activate sprout-cv && python backend.py

Terminal 2: Backend

cd sprout-backend && npm run dev

Terminal 3: Frontend

cd sprout-frontend && npm run dev

Hot Reloading

Backend
Frontend
Hand Tracking

Backend uses tsx watch for automatic reloading:

package.json

"dev": "tsx watch src/index.ts"

Changes to src/**/*.ts files will trigger automatic restart.

Python service does not support hot reloading. Restart manually:

# Stop: Ctrl+C
# Start again:
python backend.py

Using Sprout

Once all services are running:

Create a Topic

Open http://localhost:3000
Click “Add a branch” or create a new topic
Enter a topic title (e.g., “Linear Algebra”)
Optionally upload documents for context

Generate Learning Path

Click “Run agents” to start the two-phase generation:Phase 1: Topic Agent

Generates 6-10 concept nodes
Extracts document context
Creates prerequisite edges

Phase 2: Subconcept Bootstrap

Runs in parallel for each concept (max 3 concurrent)
Creates 8-12 subconcepts per concept
Generates diagnostic questions

Set NEXT_PUBLIC_SMALL_AGENTS=true to generate 1-2 concepts and 2-3 subconcepts for faster testing.

Answer Diagnostic Questions

Click on a concept node to answer diagnostic questions. The system will personalize the subconcept graph based on your performance.

Enable Hand Tracking

Toggle “Hand tracking” in the bottom-right corner to control the camera and graph with hand gestures.Gestures:

Normal hand: Orbit camera with index finger
Open palm (3s hold): Enter grab mode
Grab + palm over node: Drag node in 3D space

Learn Subconcepts

Click on a subconcept to start an interactive tutoring session with chunk-based teaching and exercises.

Monitoring Activity

Backend Logs

The backend logs include:

Agent activity (tool calls, reasoning)
Database operations
API requests
Errors and warnings

Default user seeded.
Sprout backend running on http://localhost:8000
# Agent logs appear here during generation

Frontend SSE Streams

Open browser DevTools to see Server-Sent Events:

Open DevTools (F12)
Go to Network tab
Filter by “EventStream”
Click on an active connection to see events:
- agent_start
- agent_reasoning
- tool_call
- tool_result
- node_created
- edge_created
- agent_done

Hand Tracking Debug

The Python service prints frame-by-frame data:

WebSocket Server started on ws://localhost:8765
Client connected
Sending frame data: {"hands": [{"handedness": "Right", "x": 0.5, ...}]}

Stopping Services

Graceful Shutdown
Kill All

Press Ctrl+C in each terminal window:

Stop hand tracking: Ctrl+C
Stop backend: Ctrl+C
Stop frontend: Ctrl+C

Find and kill processes by port:

# Kill backend (port 8000)
lsof -ti:8000 | xargs kill -9

# Kill frontend (port 3000)
lsof -ti:3000 | xargs kill -9

# Kill hand tracking (port 8765)
lsof -ti:8765 | xargs kill -9

Troubleshooting

Port already in use

Error: EADDRINUSE: address already in useSolution: Kill the process using the port:

lsof -ti:8000 | xargs kill -9  # Backend
lsof -ti:3000 | xargs kill -9  # Frontend
lsof -ti:8765 | xargs kill -9  # Hand tracking

Hand tracking not moving

The 3D pointer doesn’t respond to hand movements.Solution:

Ensure python backend.py is running
Check browser console for WebSocket connection errors
Grant camera permission when prompted
Verify camera is not in use by another app

SSE streaming stops

Agent activity stops streaming in the UI.Solution:

Verify NEXT_PUBLIC_BACKEND_ORIGIN matches backend URL
Check backend logs for errors
Frontend bypasses proxy for SSE - ensure direct connection works
Check CORS is enabled (default in src/index.ts)

Database locked error

Error: SQLITE_BUSY: database is lockedSolution:

Ensure only one backend instance is running
Close any DB browser tools accessing sprout.db
Restart the backend

Document upload fails

S3 upload returns 403 or 500 error.Solution:

Verify AWS credentials in .env
Check S3 bucket exists and is accessible
Verify IAM permissions allow s3:PutObject

Test with AWS CLI:

aws s3 cp test.txt s3://your-bucket-name/test.txt

Setup

Guides

Running Locally

Quick Start

Service Details

1. Hand Tracking Service

2. Backend API

3. Frontend

Development Workflow

Recommended Terminal Setup

Hot Reloading

Using Sprout

Monitoring Activity

Backend Logs

Frontend SSE Streams

Hand Tracking Debug

Stopping Services

Troubleshooting

Next Steps

Database Migrations

Hand Tracking Setup

Build docs developers (and LLMs) love

Setup

Guides

​Quick Start

​Service Details

​1. Hand Tracking Service

​2. Backend API

​3. Frontend

​Development Workflow

​Recommended Terminal Setup

​Hot Reloading

​Using Sprout

​Monitoring Activity

​Backend Logs

​Frontend SSE Streams

​Hand Tracking Debug

​Stopping Services

​Troubleshooting

​Next Steps

Database Migrations

Hand Tracking Setup

Build docs developers (and LLMs) love

Quick Start

Service Details

1. Hand Tracking Service

2. Backend API

3. Frontend

Development Workflow

Recommended Terminal Setup

Hot Reloading

Using Sprout

Monitoring Activity

Backend Logs

Frontend SSE Streams

Hand Tracking Debug

Stopping Services

Troubleshooting

Next Steps