Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/Antisource/heronaisec/llms.txt

Use this file to discover all available pages before exploring further.

Every time you run python main.py, Heron AI Security writes all outputs into a self-contained, timestamped directory under results/. Nothing is overwritten between runs — each execution produces its own isolated snapshot of metrics, figures, and model checkpoints, making it straightforward to compare configurations or revisit any prior result.

​
Output Directory Structure

Each run creates a directory named after the experiment and a timestamp, following the format produced by the timestamp() utility function in src/utils.py (YYYYMMDD_HHMMSS). 
results/
└── baseline_<timestamp>/
    ├── metrics.csv
    └── figures/
        ├── persistence_curve.png
        └── reviewer_intervention.png
For example, a run started on 19 July 2026 at 15:51:48 produces: 
results/
└── baseline_20260719_155148/
    ├── metrics.csv
    └── figures/
        ├── persistence_curve.png
        └── reviewer_intervention.png
The figures/ subdirectory is created by save_results() in experiments/baseline.py before any plots are written. reviewer_intervention.png is only generated when reviewer.enabled: true is set in the YAML config.

​
metrics.csv Format

metrics.csv is the primary tabular record of the experiment. Each row corresponds to one evaluation point along the progressive fine-tuning schedule — one for the backdoored baseline (before any intervention) and one for every non-zero entry in intervention.clean_ratios.
ColumnTypeDescription
clean_ratiofloatClean data fraction used for the intervention (0.0–1.0)
experimentstring"baseline" when ratio = 0, "clean_intervention" for all subsequent stages
clean_accuracyfloatClassification accuracy on clean (untriggered) test examples
attack_success_ratefloatFraction of triggered examples classified as target_label
mean_trigger_confidencefloatMean softmax probability assigned to target_label for triggered inputs
retention_ratiofloatASR(t) / ASR(0) — normalized measure of remaining backdoor capability
retention_ratio is always 1.0 at the baseline row (clean_ratio = 0.0) because the denominator is the baseline ASR itself. Values below 1.0 indicate that the backdoor has partially degraded; 0.0 indicates full eradication.

​
Checkpoint Artifacts

Model checkpoints are saved throughout the experiment inside a checkpoints/ subdirectory. The directory layout matches the intervention schedule: 
checkpoints/
├── baseline/
├── checkpoint_20/
├── checkpoint_40/
├── checkpoint_60/
├── checkpoint_80/
└── checkpoint_100/
Each directory is a standard Hugging Face model checkpoint and can be reloaded at any time using load_model() from src/models.py: 
from src.models import load_model

model = load_model("checkpoints/checkpoint_40")
This makes it possible to re-evaluate any intervention stage, fine-tune further, or run custom probes without repeating expensive training steps.

​
Reading metrics.csv in Python

Use pandas to load and inspect results from any run: 
import pandas as pd

results = pd.read_csv("results/baseline_20260719_155148/metrics.csv")
print(results[["clean_ratio", "attack_success_rate", "clean_accuracy"]].to_string())
This prints a clean tabular view of how ASR and clean accuracy evolve across all intervention stages.
Since every run writes to its own unique timestamped directory, you can load multiple metrics.csv files simultaneously and plot them together to compare the effect of different configurations — for example, different poison rates, trigger tokens, or intervention epoch counts — without any risk of data collisions.

Build docs developers (and LLMs) love

Get started for freeTalk to us