Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/theislampill/orthemology/llms.txt

Use this file to discover all available pages before exploring further.

The stopped-clock validator is the framework’s canonical correct-result / defective-pathway example — fixture F2. A CI gate checks whether a build artifact is scan-compliant by searching the scanner log for the string SCAN-CLEAN. On build 771 the marker is present, the artifact is genuinely compliant, and the gate passes. The result happens to be right. But the procedure — marker-string detection — was never evaluated against noncompliant artifacts. It is not truth-tracking. A noncompliant artifact whose log still contains SCAN-CLEAN would pass the gate identically. The pathway is defective, and the framework records it as such, even though RESULT_CORRECT (V1) passes. This is a deterministic illustrative fixture; no empirical claim is made.

​
The analogy

A stopped clock displays the correct time exactly twice a day. Confirming that it shows the right time right now does not establish that it tracks time. The same asymmetry applies to validators: a correct answer on one instance, produced by a mechanism that does not track the relevant property, does not establish a reliable pathway — and the two judgements are recorded separately.

Episode

ep-scan-771 — artifact artifact-A (build-771), CI gate release-compliance v1.0, actor ci-gate. Evidence: log-scrape ev-log observing SCAN-CLEAN in scanner 4.2 run 771.

Result verdict

RESULT_CORRECT (V1): pass — the artifact is genuinely compliant and the gate correctly passed it this time.

Pathway verdict

PROCEDURE_RELIABLE (V2b-P): fail — marker-string detection was never evaluated against noncompliant artifacts; truth-conduciveness is not established.

Overall

pathway_state: defective. GOV_CONFIG_ADEQUATE (V3a): fail. ROBUST_NEIGHBORHOOD (V6): fail.

​
Scenario

Analysis release-compliance v1.0 gates artifacts on scan compliance with policy: “pass iff log contains SCAN-CLEAN.” The gate was never validated against noncompliant artifacts; the marker-string check is simply a proxy that has not been shown to track compliance. The meta_tokens list is empty: MetaTok(e) = ∅. There are no case-specific binding tokens — this is the zero-burden case, so GOV_TOKEN_ADEQUATE is not-applicable. 
{
  "episode_id": "ep-scan-771",
  "occurrence": {
    "identity_key": "artifact-A",
    "version": "build-771",
    "kind": "artifact"
  },
  "observation": "log contains SCAN-CLEAN",
  "evidence": [
    {
      "evidence_id": "ev-log",
      "channel": "log-scrape",
      "property_class": "structural",
      "scope": "compliance @ build-771",
      "provenance": "scanner 4.2 run 771",
      "validity": "current build only"
    }
  ],
  "actor": "ci-gate",
  "warrant": "pipeline-config-9",
  "analysis": {
    "analysis_id": "release-compliance",
    "version": "1.0",
    "task": "gate artifacts on scan compliance"
  },
  "meta_tokens": [],
  "policy": "pass iff log contains SCAN-CLEAN",
  "placement": {
    "compliance": "clean"
  }
}
meta_tokens: [] means MetaTok(e) is empty. There are no case-specific configuration bindings. GOV_TOKEN_ADEQUATE (V3c) is therefore recorded not-applicable with reason “MetaTok empty: no case-specific bindings.” The absence of tokens is itself a fact about the episode’s governance posture, not an omission from the record.

​
Why PROCEDURE_RELIABLE fails

PROCEDURE_RELIABLE (V2b-P) evaluates the configured procedure’s truth-conduciveness against a declared rel_spec (reliability specification). The spec for claim-clean requires a sensitivity threshold of 0.99 — the fraction of noncompliant artifacts the gate rejects. The evidence basis for this threshold is recorded honestly: 
{
  "rel_spec": {
    "claim-clean": {
      "procedure_id": "scan-log-marker-check",
      "procedure_version": "scanner 4.2 under gate policy release-compliance 1.0",
      "reference_class": "all artifacts gated by scanner 4.2 under analysis release-compliance 1.0",
      "risk_stratum": "release gate: a false clean permits a noncompliant artifact to ship",
      "metric": "sensitivity: the fraction of noncompliant artifacts the gate rejects",
      "threshold": 0.99,
      "evaluation_protocol": "paired metamorphic probes over compliant and noncompliant artifacts, adjudicated against the scanner's own findings",
      "evidence_basis": "none measured: the marker-string check was never evaluated against noncompliant artifacts, which is why PROCEDURE_RELIABLE fails here"
    }
  }
}
No measured sensitivity exists. The procedure’s truth-conduciveness is not established. PROCEDURE_RELIABLE fails because the declared rel_spec has no supporting evidence — not because the result was wrong.

​
Why ROBUST_NEIGHBORHOOD fails

The perturb_spec for claim-clean defines a neighbourhood of artifacts differing from build-771 in compliance status and marker presence, with policy, configuration, and analysis held fixed: 
{
  "perturb_spec": {
    "claim-clean": {
      "neighborhood_family": "artifacts differing from build-771 in compliance status and marker presence, with policy, configuration and analysis held fixed",
      "enumeration": [
        "probe-A: noncompliant artifact whose log still contains SCAN-CLEAN",
        "probe-B: compliant artifact whose log omits SCAN-CLEAN"
      ],
      "invariants": ["policy", "configuration", "analysis"],
      "varied_fields": ["marker_presence", "artifact_compliance"],
      "metaorthemma_rebinding_rule": "MetaTok(e) is empty (zero-burden): no case-bound token is rebound across the neighborhood",
      "failure_criterion": "any probe whose gate outcome disagrees with the artifact's true compliance status"
    }
  }
}
Probe-A (noncompliant, marker present) would pass the gate. Probe-B (compliant, marker absent) would fail it. Both probes disagree with the artifact’s true compliance status. ROBUST_NEIGHBORHOOD (V6) fails: the gate’s outcome is not robust to perturbations in the compliance / marker relationship.
A ROBUST_NEIGHBORHOOD failure does not entail a PROCEDURE_RELIABLE failure, and vice versa. The verdict registry explicitly records [ROBUST_NEIGHBORHOOD, PROCEDURE_RELIABLE] and [PROCEDURE_RELIABLE, ROBUST_NEIGHBORHOOD] as declared non-entailments. Both can fail independently; here they both fail for related but distinct reasons.

​
Why GOV_CONFIG_ADEQUATE fails

GOV_CONFIG_ADEQUATE (V3a) evaluates whether the governing configuration — the analysis, its declared policy, and its evidence repertoire — is adequate for the case’s risk class. The risk stratum is a release gate: a false clean permits a noncompliant artifact to ship. Configuring a marker-string check without an evaluated sensitivity against noncompliant artifacts is inadequate for this risk class. V3a fails.

​
The full verdict record

{
  "episode_id": "ep-scan-771",
  "statuses": {
    "RESULT_CORRECT": "pass",
    "EVIDENCE_SUPPORT": "pass",
    "PROCEDURE_RELIABLE": "fail",
    "EVIDENCE_CURRENT": "pass",
    "GOV_CONFIG_ADEQUATE": "fail",
    "GOV_POLICY_ADEQUATE": "pass",
    "EXECUTION_FAITHFUL": "pass",
    "EX_ANTE_JUSTIFIED": "pass",
    "ROBUST_NEIGHBORHOOD": "fail",
    "TOKEN_TRUTH_LINKED": "fail",
    "GOV_TOKEN_ADEQUATE": "not-applicable",
    "ROUTE_ADMISSIBLE": "not-applicable",
    "CLOSURE_TRUTHFUL": "not-applicable",
    "ROUTE_QUALITY": "not-applicable"
  },
  "na_reasons": {
    "GOV_TOKEN_ADEQUATE": "MetaTok empty: no case-specific bindings",
    "ROUTE_ADMISSIBLE": "no route selected",
    "CLOSURE_TRUTHFUL": "no closure claim",
    "ROUTE_QUALITY": "advisory; no route"
  },
  "claim_verdicts": [
    {
      "claim_id": "claim-clean",
      "result_correct": "pass",
      "token_truth_linked": "fail"
    }
  ],
  "pathway_state": "defective"
}

​
TOKEN_TRUTH_LINKED and the luck distinction

TOKEN_TRUTH_LINKED (V2b-T) fails even though RESULT_CORRECT passes. This is the core of the correct-by-luck cell: the claim is right, but it is not right through the truth-relevant mechanism. The framework’s entailment registry states only one definitional entailment: TOKEN_TRUTH_LINKED → RESULT_CORRECT (claim-wise). The converse does not hold. A correct result does not establish truth linkage. Stopped-clock validators are a systematic counterexample.
  • RESULT_CORRECT (V1): the artifact is compliant and the gate said clean — result is correct.
  • EVIDENCE_SUPPORT (V2a): the log-scrape evidence is typed, scoped, and within its declared validity.
  • EVIDENCE_CURRENT (V2c): the evidence is current for build-771.
  • GOV_POLICY_ADEQUATE (V3b): the policy text is well-formed.
  • EXECUTION_FAITHFUL (V3d): the gate executed the marker-check exactly as the policy says.
  • EX_ANTE_JUSTIFIED (V3e): given only the log and the policy, the decision was locally reasonable.

​
Connection to process reliabilism

The correct-by-luck cell is the framework’s formal analogue of the epistemological gap between true belief and knowledge. Process reliabilism holds that a belief counts as knowledge only when produced by a truth-conducive process — not merely when it happens to be true. PROCEDURE_RELIABLE operationalises this requirement within an auditable record: a rel_spec must be declared, an evaluation protocol must exist, and measured truth-conduciveness must meet the declared threshold. Passing RESULT_CORRECT without passing PROCEDURE_RELIABLE is precisely the correct-but-not-knowledge cell, and it is auditable as such.
When reviewing a CI gate or automated validator, the presence of a rel_spec with a populated evidence_basis field (rather than “none measured”) is the first signal that PROCEDURE_RELIABLE will be able to pass. An empty or absent evidence basis is a reliable predictor of this cell, independent of what any particular build’s result says.
The required path for claim-clean includes ROBUST_NEIGHBORHOOD — reflecting the risk class of the release gate. The full required path is: EVIDENCE_SUPPORT, PROCEDURE_RELIABLE, EVIDENCE_CURRENT, GOV_CONFIG_ADEQUATE, GOV_POLICY_ADEQUATE, EXECUTION_FAITHFUL, EX_ANTE_JUSTIFIED, ROBUST_NEIGHBORHOOD. All eight are required; three fail; reasoning_path_adequate is defective.

Build docs developers (and LLMs) love

Get started for freeTalk to us