Skip to main content
Symphony is defined by a language-agnostic specification (SPEC.md) that enables implementations in any programming language. This guide walks through building a compliant implementation.

Specification Overview

From SPEC.md:
Symphony is a long-running automation service that continuously reads work from an issue tracker (Linear in this specification version), creates an isolated workspace for each issue, and runs a coding agent session for that issue inside the workspace.
The service solves four operational problems:
  • Turns issue execution into a repeatable daemon workflow
  • Isolates agent execution in per-issue workspaces
  • Keeps workflow policy in-repo (WORKFLOW.md)
  • Provides observability to operate and debug concurrent agent runs

Architecture Layers

Symphony is organized into these abstraction levels:
1

Policy Layer

Repository-defined workflow in WORKFLOW.md:
  • YAML frontmatter for runtime settings
  • Markdown prompt template for agent instructions
  • Team-specific rules for ticket handling
2

Configuration Layer

Typed getters that parse frontmatter:
  • Defaults and environment variable indirection
  • Path normalization
  • Validation used by orchestrator before dispatch
3

Coordination Layer

Orchestrator that manages:
  • Polling loop and issue eligibility
  • Concurrency limits and retry backoff
  • Reconciliation and cleanup
4

Execution Layer

Workspace management and agent subprocess:
  • Filesystem lifecycle
  • Workspace preparation
  • Coding agent protocol (app-server mode over stdio)
5

Integration Layer

Tracker adapter:
  • API calls to issue tracker
  • Normalization to Symphony issue model
6

Observability Layer

Logging and optional status surface:
  • Structured logs with issue/session context
  • Optional HTTP dashboard

Core Components

1. Workflow Loader

Parses WORKFLOW.md into config and prompt template.
interface WorkflowDefinition {
  config: Record<string, any>;  // YAML front matter
  prompt_template: string;       // Markdown body
}

function loadWorkflow(path: string): WorkflowDefinition {
  const content = fs.readFileSync(path, 'utf-8');
  
  // Parse YAML frontmatter between --- delimiters
  const frontmatterMatch = content.match(/^---\n([\s\S]*?)\n---/);
  
  if (frontmatterMatch) {
    const config = yaml.parse(frontmatterMatch[1]);
    const prompt = content.slice(frontmatterMatch[0].length).trim();
    return { config, prompt_template: prompt };
  }
  
  return { config: {}, prompt_template: content.trim() };
}

2. Config Layer

Provides typed access to configuration values with defaults and environment variable resolution.
tracker:
  kind: linear                  # Required for dispatch
  endpoint: https://api.linear.app/graphql
  api_key: $LINEAR_API_KEY     # Environment variable indirection
  project_slug: "project-id"   # Required for Linear
  active_states: [Todo, In Progress]
  terminal_states: [Done, Closed, Cancelled]

polling:
  interval_ms: 30000            # Default: 30 seconds

workspace:
  root: ~/symphony_workspaces   # Default: <temp>/symphony_workspaces

hooks:
  after_create: |               # Runs once on workspace creation
    git clone https://github.com/org/repo .
  before_run: |                 # Runs before each attempt
    git fetch origin
  after_run: |                  # Runs after each attempt (non-fatal)
    git status
  before_remove: |              # Runs before cleanup (non-fatal)
    tar czf /tmp/archive.tar.gz .
  timeout_ms: 60000             # Default: 60 seconds

agent:
  max_concurrent_agents: 10
  max_retry_backoff_ms: 300000  # 5 minutes
  max_concurrent_agents_by_state:
    "In Progress": 5

codex:
  command: codex app-server
  approval_policy: never        # Codex-defined value
  thread_sandbox: workspace-write
  turn_sandbox_policy:
    type: workspaceWrite
  turn_timeout_ms: 3600000      # 1 hour
  read_timeout_ms: 5000
  stall_timeout_ms: 300000      # 5 minutes
Key implementation requirements:
  • Environment variable resolution: $VAR_NAME in string values
  • Path expansion: ~ and path separators
  • Type coercion: Strings to integers where specified
  • Defaults: Specified in SPEC.md section 6.4
  • Validation: Preflight checks before dispatch (section 6.3)

3. Issue Tracker Client

Normalizes tracker payloads into the Symphony issue model.
interface Issue {
  id: string;                    // Stable tracker-internal ID
  identifier: string;            // Human-readable (e.g., "ABC-123")
  title: string;
  description: string | null;
  priority: number | null;       // Lower = higher priority
  state: string;
  branch_name: string | null;
  url: string | null;
  labels: string[];              // Normalized to lowercase
  blocked_by: Blocker[];
  created_at: string | null;
  updated_at: string | null;
}

interface Blocker {
  id: string | null;
  identifier: string | null;
  state: string | null;
}
Required operations:
  1. fetch_candidate_issues() - Return issues in active states
  2. fetch_issues_by_states(state_names) - For startup cleanup
  3. fetch_issue_states_by_ids(issue_ids) - For reconciliation

4. Orchestrator

The orchestrator owns the polling loop and runtime state.
interface OrchestratorState {
  poll_interval_ms: number;
  max_concurrent_agents: number;
  running: Map<string, RunningEntry>;     // issue_id -> entry
  claimed: Set<string>;                   // Reserved/running/retrying
  retry_attempts: Map<string, RetryEntry>;
  completed: Set<string>;                 // Bookkeeping only
  codex_totals: TokenTotals;
  codex_rate_limits: RateLimitSnapshot | null;
}

interface RunningEntry {
  issue_id: string;
  issue_identifier: string;
  session_id: string;          // "<thread_id>-<turn_id>"
  started_at: number;          // Timestamp
  last_codex_timestamp: number | null;
  turn_count: number;
  // ... token counters
}

interface RetryEntry {
  issue_id: string;
  identifier: string;
  attempt: number;             // 1-based
  due_at_ms: number;
  timer_handle: any;
  error: string | null;
}
Orchestrator responsibilities:

Poll Tick

  1. Reconcile running issues
  2. Validate config
  3. Fetch candidates
  4. Sort by priority
  5. Dispatch until slots full

Concurrency Control

  • Global limit: max_concurrent_agents
  • Per-state limits: max_concurrent_agents_by_state
  • Available slots = max - running

Retry & Backoff

  • Continuation: 1 second fixed
  • Failure: min(10000 * 2^(attempt-1), max_retry_backoff_ms)
  • Cap at configured max (default 5 minutes)

Reconciliation

  • Stall detection based on event inactivity
  • Fetch current states for running issues
  • Stop workers for terminal/inactive states

5. Workspace Manager

Safety Invariants (SPEC.md section 9.5):
Critical workspace safety rules:
  1. Run coding agent ONLY in per-issue workspace path
  2. Workspace path MUST stay inside workspace root
  3. Workspace key is sanitized: [A-Za-z0-9._-] only
function sanitizeIdentifier(identifier: string): string {
  return identifier.replace(/[^A-Za-z0-9._-]/g, '_');
}

function workspacePathForIssue(identifier: string): string {
  const safe = sanitizeIdentifier(identifier);
  return path.join(workspaceRoot, safe);
}

function validateWorkspacePath(workspace: string): void {
  const expanded = path.resolve(workspace);
  const root = path.resolve(workspaceRoot);
  
  if (expanded === root) {
    throw new Error('Workspace equals root');
  }
  
  if (!expanded.startsWith(root + path.sep)) {
    throw new Error('Workspace outside root');
  }
  
  // Check for symlink escape
  ensureNoSymlinkComponents(expanded, root);
}
Lifecycle hooks (section 5.3.4):
  • after_create: Fatal if fails, runs once on creation
  • before_run: Fatal if fails, runs before each attempt
  • after_run: Non-fatal, runs after each attempt
  • before_remove: Non-fatal, runs before deletion

6. Agent Runner

Launches Codex in app-server mode and streams events to orchestrator.
Launch contract:
bash -lc "codex app-server"
Startup sequence:
  1. initialize request with client info and capabilities
  2. initialized notification
  3. thread/start request with approval policy, sandbox, cwd, tools
  4. turn/start request with threadId, input (prompt), title
Session IDs:
  • Read thread_id from thread/start result
  • Read turn_id from turn/start result
  • Compose session_id = "<thread_id>-<turn_id>"
Completion conditions:
  • turn/completed → success
  • turn/failed → failure
  • turn/cancelled → failure
  • turn timeout → failure
  • subprocess exit → failure

7. Prompt Builder

Renders the prompt template with issue context.
interface PromptContext {
  issue: Issue;
  attempt: number | null;  // null on first run, integer on retry
}

function renderPrompt(template: string, context: PromptContext): string {
  // Use strict template engine (e.g., Liquid)
  // Unknown variables must fail rendering
  // Unknown filters must fail rendering
  
  return liquidEngine.render(template, {
    issue: issueToTemplateObject(context.issue),
    attempt: context.attempt
  });
}

Implementation Checklist

1

Workflow Loader

  • Parse YAML frontmatter
  • Extract prompt template
  • Handle missing/invalid YAML
2

Config Layer

  • Environment variable resolution ($VAR)
  • Path expansion (~, separators)
  • Type coercion (string to int)
  • Defaults (SPEC.md section 6.4)
  • Dispatch preflight validation
3

Tracker Client

  • Implement required operations
  • Normalize to issue model
  • Handle pagination
  • Error categorization
4

Orchestrator

  • Poll loop with interval
  • Runtime state management
  • Concurrency control
  • Retry with backoff
  • Reconciliation (stall + state refresh)
5

Workspace Manager

  • Path sanitization
  • Safety validation
  • Hook execution (sh -lc)
  • Hook timeout handling
  • Terminal cleanup
6

Agent Runner

  • App-server subprocess launch
  • Protocol handshake
  • Event streaming
  • Timeout handling
  • Approval/tool handling
7

Prompt Builder

  • Strict template rendering
  • Issue context serialization
  • Attempt metadata
8

Logging

  • Structured logs
  • Issue/session context
  • Operator visibility

Testing Strategy

Unit Tests

  • Workflow parsing edge cases
  • Config resolution and defaults
  • Issue normalization
  • Workspace path validation
  • Prompt rendering

Integration Tests

  • In-memory tracker for orchestrator tests
  • Mock app-server for agent runner tests
  • Filesystem workspace lifecycle

Conformance Tests

Test against SPEC.md requirements:
  • Dispatch eligibility rules (section 8.2)
  • Concurrency limits (section 8.3)
  • Retry backoff formula (section 8.4)
  • Workspace safety invariants (section 9.5)
  • Protocol message ordering (section 10.2)

Language-Specific Considerations

Strengths:
  • Supervision trees for worker isolation
  • Built-in state management (GenServer, Agent)
  • Excellent concurrency primitives
Watch for:
  • String vs atom keys in config maps
  • Timeout precision (milliseconds)
  • Port vs OS process for subprocess

Conformance Validation

Must Implement

From SPEC.md section 2.1 (Goals):
  • Poll tracker on fixed cadence
  • Maintain authoritative orchestrator state
  • Create deterministic per-issue workspaces
  • Stop runs when issue state changes
  • Recover from transient failures with backoff
  • Load runtime behavior from WORKFLOW.md
  • Expose structured logs
  • Support restart recovery without persistent DB

Optional Extensions

  • Rich web UI (section 13.7)
  • Custom tracker types beyond Linear
  • Additional dynamic tools
  • Telemetry/metrics integrations

Reference Implementation

The Elixir implementation is the reference: Key modules:
  • SymphonyElixir.Workflow - Workflow loader
  • SymphonyElixir.Config - Config layer
  • SymphonyElixir.Orchestrator - Coordination
  • SymphonyElixir.Workspace - Workspace management
  • SymphonyElixir.AgentRunner - Codex integration
  • SymphonyElixir.Linear.Adapter - Tracker client

Next Steps

Read SPEC.md

Complete language-agnostic specification

Study Reference Implementation

Elixir implementation with tests

Custom Skills

Create reusable workflow components

Extending Symphony

Add trackers, tools, and extensions

Build docs developers (and LLMs) love