Traces and Alignment Proof in SkillSpec

When a SkillSpec-backed skill runs, the result is not only a final response — it is a record you can inspect. The CLI writes an append-only decision trace as the rule engine evaluates your contract, and the runtime writes an execution ledger as phases progress and evidence is captured. After the run, skillspec trace align compares both artifacts against the contract and produces a structured alignment report with a three-valued verdict: aligned, partial, or unproven. The third verdict — unproven — is the methodological core. Binary pass/fail forces an evaluator to treat missing evidence as either success or failure. The convention that the absence of an error means success is exactly what hides dropped steps. SkillSpec reports unproven when the trace lacks the evidence to decide, rather than inferring success from silence.

Decision trace

When you run skillspec decide with --trace-dir, the CLI creates a run directory:

.skillspec/traces/run-<id>/
  events/          # one file per trace event (JSONL lines)
  trace.jsonl      # compacted single-file trace
  summary.json     # human-readable decision summary

The decision trace records what the rule engine did: which rules it evaluated, which matched, which route was selected, and what effects accumulated.

Trace event kinds

The trace vocabulary is a closed set. An event either appears in the trace or it does not — absence is informative.

Event	Written when
`input_received`	The user input is captured before rule evaluation
`spec_loaded`	The spec is loaded; records `skill_id` and `schema`
`rule_evaluated`	Each rule is checked; records `rule_id` and `matched: true/false`
`rule_matched`	A rule matched; records `rule_id` and `reason`
`route_selected`	A route is set; records `route`, `basis`, `rule_id`, and `reason`
`route_order_set`	A rule replaced the route ordering
`forbid_added`	A rule added forbidden substitutions
`allow_added`	A rule granted explicit permissions
`elicitation_requested`	A rule appended a clarifying question
`after_success_scheduled`	A rule scheduled post-task closures
`outcome_recorded`	The final decision: `route`, `route_selection`, and `matched_rules`

The route_selected event carries a basis field that explains why the route was chosen:

rule_prefer — a matched rule’s prefer field set it
route_order_default — a rule changed route order and the first in that order won
default_route_order — no rule acted; the first ranked route was used

This basis field is what makes decision replay deterministic: re-running the spec on the captured input must produce the same route by the same mechanism.

Execution ledger

The execution ledger lives alongside the decision trace:

.skillspec/traces/run-<id>/
  execution.jsonl   # append-only execution events

The ledger records what actually happened during the run. Structured events include:

Event	Records
`phase_started`	Phase id, owner skill, timestamp
`requirement_satisfied`	Phase id, requirement id, evidence kind and reference
`requirement_failed`	Phase id, requirement id, failure evidence
`evidence_attached`	Artifact or response reference captured during the phase
`stats_collected`	Workspace/token metrics snapshot
`adapter_discovery_finished`	Adapter health/probe results
`cli_readiness_check_finished`	CLI preflight result
`process_started`	Process launch, executor (rote_exec vs. direct), workspace
`final_response_sent`	Whether the response included result, evidence, alignment, and token-savings sections

The ledger is written by the agent or harness as work proceeds, not by the spec engine. Its completeness is a variable — a high unproven rate in an alignment report often signals that the harness is not writing structured events, not that the skill is broken.

Alignment

skillspec trace align compares the decision trace and execution ledger against the contract:

skillspec trace align skill.spec.yml --trace-dir .skillspec/traces/run-<id>

The aligner:

Re-runs the spec on the captured input to produce an expected decision
Checks that the trace records the same route, matched rules, forbids, elicitations, and after-success closures
Derives execution obligations from the selected route and matched rules
Checks the ledger for structured evidence that each obligation was satisfied
Writes alignment.json and a proof digest to the run directory

.skillspec/traces/run-<id>/
  alignment.json       # full alignment report
  proof-digest.json    # grouped missing-proof items

Alignment properties

The aligner produces one of three verdicts per check and an aggregate per run:

aligned

The trace contains evidence that the expectation held. The decision was reproduced and execution obligations have structured proof.

partial

Some expectations met, at least one obligation unproven or violated. The run partially aligns — report the concrete missing-proof rows.

unproven

The trace lacks the events needed to decide. This is not failure — it means the harness didn’t record enough structure, or execution wasn’t traced.

The aggregate report carries two layers:

Decision replay — did re-running the spec on the captured input produce the same route, rules, forbids, and elicitations?
Execution proof — does the ledger prove the route was fulfilled, forbids were not violated, elicitations were answered, and after-success closures ran?

{
  "schema": "skillspec.align/v0",
  "ok": true,
  "status": "unproven",
  "summary": {
    "scope": "decision_and_execution_trace",
    "decision_alignment": "pass",
    "execution_alignment": "incomplete",
    "conclusion": "decision replay found no deterministic drift, but proof is incomplete: 3 execution obligation(s) remain unproven",
    "selected_route": "one_shot_process",
    "route_selection_basis": "rule_prefer",
    "matched_rules": ["cli_invocations_use_rote_exec"],
    "completion": {
      "decision_replay": "pass",
      "phase_order": "not applicable",
      "requirements": "0/0 proven",
      "missing_proof": ["route `one_shot_process` needs structured execution evidence"],
      "forbidden_actions": "no violations recorded",
      "alignment": "partial"
    }
  }
}

SkillSpec never infers success from the absence of errors. An unproven verdict is the honest report when evidence is missing — not a soft pass.

Trace event JSON example

A route_selected event in trace.jsonl:

{
  "schema": "skillspec.trace/v0",
  "run_id": "run-1782111280403-87123",
  "seq": 5,
  "timestamp_unix_ms": 1782111280404,
  "skill_id": "example.browser_research",
  "spec_schema": "skillspec/v0",
  "spec_fingerprint": "sha256:abc123...",
  "input_sha256": "sha256:def456...",
  "event": "route_selected",
  "event_name": "route_selected",
  "data": {
    "route": "browser",
    "basis": "rule_prefer",
    "rule_id": "browser_tasks_use_browser",
    "reason": "Page evidence must come from observing the page, not from search-result summaries."
  }
}

A requirement_satisfied event in execution.jsonl:

{
  "event": "requirement_satisfied",
  "phase": "establish_durable_context",
  "requirement": "initialize_rote_workspace",
  "evidence": {
    "kind": "command_output",
    "ref": "rote:workspace:tulving-rote-remote-check"
  },
  "ts": "2025-01-15T10:23:51.042Z"
}

Trace spec section

The trace section of a skill.spec.yml declares which events the evaluator should persist. Setting required: true tells a conforming harness it must either write the trace or state that tracing is unavailable before relying on the decision.

trace:
  mode: event_log
  required: true
  record:
    - input_received
    - spec_loaded
    - rule_evaluated
    - rule_matched
    - route_selected
    - route_order_set
    - forbid_added
    - allow_added
    - elicitation_requested
    - after_success_scheduled
    - outcome_recorded

When record is empty or absent, an evaluator may record every v0 event kind. The spec does not contain per-rule file-writing instructions — it declares intent, and the evaluator acts on it.

The alignment report is the difference between “the run completed” and “the run completed and we can prove it”. Running skillspec trace align at the end of each durable run turns the trace into a first-class artifact alongside the final response.

Get Started

Core Concepts

Guides

Spec Reference

Traces and Alignment Proof in SkillSpec

Decision trace

Trace event kinds

Execution ledger

Alignment

Alignment properties

aligned

partial

unproven

Trace event JSON example

Trace spec section

​Decision trace

​Trace event kinds

​Execution ledger

​Alignment

​Alignment properties

aligned

partial

unproven

​Trace event JSON example

​Trace spec section

Decision trace

Trace event kinds

Execution ledger

Alignment

Alignment properties

Trace event JSON example

Trace spec section