Skip to main content

​
Log File Structure

Symphony uses OTP’s built-in rotating disk log handler to write structured logs to disk.

​
Default Location

Logs are written to: 
./log/symphony.log
Source: log_file.ex:9 (elixir/lib/symphony_elixir/…)

​
Custom Log Location

Specify a custom log directory using the --logs-root flag: 
./bin/symphony --logs-root /var/log/symphony ./WORKFLOW.md
Source: cli.ex:146-149 (elixir/lib/symphony_elixir/…)

​
Log Rotation

Symphony automatically rotates log files based on size: 
@default_max_bytes 10 * 1024 * 1024  # 10 MB per file
@default_max_files 5                  # Keep 5 files maximum
Source: log_file.ex:10-11 (elixir/lib/symphony_elixir/…) Rotation Behavior:
  • When symphony.log reaches 10 MB, it’s rotated to symphony.log.1
  • Older files are rotated: symphony.log.1 → symphony.log.2, etc.
  • When 5 files exist, the oldest is deleted
Log rotation is handled automatically by OTP’s :logger_disk_log_h handler. No external tools like logrotate are needed.

​
Log Format

Symphony uses structured logging with a single-line formatter: 
formatter: {:logger_formatter, %{single_line: true}}
Source: log_file.ex:71 (elixir/lib/symphony_elixir/…)

​
Log Entry Example

2026-03-04T21:45:30.123456Z [info] Agent task completed for issue_id=issue_abc123 session_id=sess_xyz; scheduling active-state continuation check
2026-03-04T21:45:35.234567Z [warning] Agent task exited for issue_id=issue_def456 session_id=sess_abc reason=:timeout; scheduling retry
2026-03-04T21:45:40.345678Z [error] Linear API token missing in WORKFLOW.md

​
Log Entry Components

| Component | Description | Example | |-----------|-------------|---------|| | Timestamp | ISO 8601 timestamp with microseconds | 2026-03-04T21:45:30.123456Z | | Level | Log severity level | [info], [warning], [error] | | Message | Structured log message | Agent task completed for issue_id=... | | Context | Key-value pairs for filtering | issue_id=abc session_id=xyz |

​
Log Levels

Symphony uses standard Erlang/OTP log levels: 
level: :all  # Captures all log levels
Source: log_file.ex:70 (elixir/lib/symphony_elixir/…)

​
Available Log Levels

Usage: Detailed debugging informationExamples:
Logger.debug("Orchestrator ignored message: #{inspect(msg)}")
Source: orchestrator.ex:169 (elixir/lib/symphony_elixir/…)Debug logs are useful for troubleshooting but can be verbose in production.
Usage: Normal operational messagesExamples:
Logger.info("Agent task completed for issue_id=#{issue_id} session_id=#{session_id}")
Logger.info("Agent task finished for issue_id=#{issue_id} session_id=#{session_id} reason=#{inspect(reason)}")
Source: orchestrator.ex:107-127 (elixir/lib/symphony_elixir/…)Info logs track normal operation flow and state transitions.
Usage: Warning conditions that don’t prevent operationExamples:
Logger.warning("Agent task exited for issue_id=#{issue_id} session_id=#{session_id} reason=#{inspect(reason)}")
Logger.warning("Failed to configure rotating log file handler: #{inspect(reason)}")
Logger.warning("Failed rendering status dashboard: #{Exception.message(error)}")
Source: orchestrator.ex:117 (elixir/lib/symphony_elixir/…), log_file.ex:47 (elixir/lib/symphony_elixir/…)Warnings indicate potential issues that should be investigated.
Usage: Error conditions that prevent specific operationsExamples:
Logger.error("Linear API token missing in WORKFLOW.md")
Logger.error("Linear project slug missing in WORKFLOW.md")
Logger.error("Unsupported tracker kind in WORKFLOW.md: #{inspect(kind)}")
Source: orchestrator.ex:182-196 (elixir/lib/symphony_elixir/…)Errors indicate problems that need immediate attention.

​
Structured Logging Format

Symphony uses key-value pairs for structured logging, making logs easy to parse and filter.

​
Context Fields

Common context fields in log messages: | Field | Description | Example | |-------|-------------|---------|| | issue_id | Internal issue identifier | issue_abc123 | | issue_identifier | Linear issue identifier | ABC-123 | | session_id | Codex session identifier | sess_xyz789 | | reason | Exit or error reason | :timeout, :normal | | attempt | Retry attempt number | 1, 2, 3 |

​
Filtering Logs

Use grep to filter logs by context: 
grep "issue_id=ABC-123" ./log/symphony.log

​
Log Handler Configuration

Symphony configures the disk log handler during application startup: 
def configure do
  log_file = Application.get_env(:symphony_elixir, :log_file, default_log_file())
  max_bytes = Application.get_env(:symphony_elixir, :log_file_max_bytes, @default_max_bytes)
  max_files = Application.get_env(:symphony_elixir, :log_file_max_files, @default_max_files)

  setup_disk_handler(log_file, max_bytes, max_files)
end
Source: log_file.ex:23-30 (elixir/lib/symphony_elixir/…)

​
Handler Configuration Options

%{
  level: :all,                                    # Log all levels
  formatter: {:logger_formatter, %{single_line: true}},  # Single-line format
  config: %{
    file: String.to_charlist(path),              # Log file path
    type: :wrap,                                   # Rotating logs
    max_no_bytes: max_bytes,                       # Max size per file
    max_no_files: max_files                        # Max number of files
  }
}
Source: log_file.ex:68-79 (elixir/lib/symphony_elixir/…)

​
Console Output

By default, Symphony removes the default console logger when the disk log handler is configured: 
defp remove_default_console_handler do
  case :logger.remove_handler(:default) do
    :ok -> :ok
    {:error, {:not_found, :default}} -> :ok
    {:error, _reason} -> :ok
  end
end
Source: log_file.ex:60-66 (elixir/lib/symphony_elixir/…)
Removing the console handler prevents duplicate log output. The terminal dashboard provides real-time status instead.

​
Reading Logs

​
Real-time Monitoring

Follow logs as they’re written: 
# Follow with tail
tail -f ./log/symphony.log

# Follow with grep filter
tail -f ./log/symphony.log | grep "\[error\]"

# Follow multiple log files
tail -f ./log/symphony.log*

​
Log Analysis

Analyze historical logs: 
grep -c "\[error\]" ./log/symphony.log

​
Log Aggregation

For production deployments, consider forwarding logs to a centralized logging system:
filebeat.yml
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/symphony/symphony.log*
  fields:
    service: symphony
    environment: production
  multiline:
    pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
    negate: true
    match: after

output.elasticsearch:
  hosts: ["localhost:9200"]
  index: "symphony-%{+yyyy.MM.dd}"
vector.toml
[sources.symphony_logs]
type = "file"
include = ["/var/log/symphony/symphony.log*"]

[transforms.parse_symphony]
type = "remap"
inputs = ["symphony_logs"]
source = '''
  . = parse_regex!(.message, r'^(?P<timestamp>[^ ]+) \[(?P<level>[^\]]+)\] (?P<message>.+)$')
  .service = "symphony"
'''

[sinks.loki]
type = "loki"
inputs = ["parse_symphony"]
endpoint = "http://localhost:3100"

​
Performance Considerations

​
Log I/O

The disk log handler writes asynchronously to avoid blocking:
  • Logs are buffered in memory
  • Writes are batched for efficiency
  • File rotation is handled by OTP’s :disk_log module
In high-throughput scenarios, monitor disk I/O to ensure logging doesn’t become a bottleneck.

​
Disk Space Management

With default settings, maximum disk usage is: 
10 MB × 5 files = 50 MB maximum
Adjust rotation settings for different retention needs: 
# Larger files, longer retention
Application.put_env(:symphony_elixir, :log_file_max_bytes, 100 * 1024 * 1024)  # 100 MB
Application.put_env(:symphony_elixir, :log_file_max_files, 10)                 # 10 files

​
Log Verbosity

To reduce log volume in production:
  1. Filter debug logs: 
    config :logger, level: :info
  2. Adjust handler level: 
    # In log_file.ex, change:
level: :info  # Instead of :all
  3. Use log sampling for high-frequency events

​
Troubleshooting Logs

​
Log Handler Not Configured

Symptom: No log files are created Solution:
  1. Check that log directory exists and is writable: 
    mkdir -p ./log
chmod 755 ./log
  2. Verify handler configuration: 
    :logger.get_handler_config(:symphony_disk_log)
  3. Check for configuration errors in startup logs
Source: log_file.ex:23-50 (elixir/lib/symphony_elixir/…)

​
Log Files Not Rotating

Symptom: Log file grows beyond configured size Solution: This is unusual as OTP handles rotation automatically. Check:
  1. Verify max_bytes configuration: 
    Application.get_env(:symphony_elixir, :log_file_max_bytes)
  2. Check disk space: 
    df -h .
  3. Examine handler status: 
    :logger.get_handler_ids()
:logger.get_handler_config(:symphony_disk_log)

​
Missing Log Entries

Symptom: Expected log entries are not present Possible causes:
  1. Log level filtering: Entry level is below configured threshold
  2. Handler crash: Check if disk log handler is still running
  3. Disk full: No space to write logs
  4. Permission errors: Log directory not writable
Check logs for handler warnings: 
grep "Failed to configure rotating log file handler" ./log/symphony.log
Source: log_file.ex:47 (elixir/lib/symphony_elixir/…)

Build docs developers (and LLMs) love

Get started for freeTalk to us