Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/Antisource/heronaisec/llms.txt

Use this file to discover all available pages before exploring further.

The baseline experiment trains a backdoored DistilBERT classifier on SST-2, then measures how the embedded backdoor degrades under six progressive stages of clean fine-tuning. All experiment logic lives in experiments/baseline.py, which is invoked through a single entry point — main.py.

​
The LOAD_FROM_CHECKPOINT flag

Near the top of experiments/baseline.py, two constants control whether the experiment trains from scratch or reuses previously saved model weights: 
LOAD_FROM_CHECKPOINT = True  # Default Value is False as we require training
CHECKPOINT_RUN = "baseline\\20260719_124348"
  • LOAD_FROM_CHECKPOINT = False — the experiment trains every model from scratch. Use this for a first run or when you want a fully fresh result.
  • LOAD_FROM_CHECKPOINT = True — the experiment skips training and loads the baseline_checkpoint directory and each checkpoint_<ratio> directory from a previous run. Set CHECKPOINT_RUN to the timestamped path of that run (relative to results/).

​
First-time run (training from scratch)

1

Set LOAD_FROM_CHECKPOINT = False

Open experiments/baseline.py and set the flag to False so the experiment trains a fresh model:
LOAD_FROM_CHECKPOINT = False
2

Optionally configure configs/baseline.yaml

Review configs/baseline.yaml before running. The defaults use DistilBERT on SST-2 with a 15 % poison rate and a "cf" trigger. See the Configuration guide for a full description of every parameter.
3

Launch the experiment

From the repository root, run:
python main.py
main.py first reports the available hardware, then calls run_baseline() in experiments/baseline.py:
========================================================================
Beyond Binary Evaluation of Backdoor Inheritance
========================================================================
4

Backdoor injection and initial training

train_backdoored_model() shuffles and selects a 5 000-example subset of the SST-2 training split, then poisons 15 % of those examples using apply_attack() with the "badnet" strategy (prepending the trigger token "cf" and flipping the label to target_label). The poisoned dataset is tokenized and used to fine-tune a freshly loaded distilbert-base-uncased model for three epochs. The trained model is saved to results/<CHECKPOINT_RUN>/checkpoints/baseline/.
The default code selects only 5 000 training examples via .select(range(5000)). This is intentional for rapid prototyping. See the warning below for details on removing this limit for full-scale runs.
5

Progressive intervention across 6 stages

run_progressive_intervention() loops over the six clean_ratios defined in the YAML config — [0.0, 0.2, 0.4, 0.6, 0.8, 1.0] — and for each non-zero ratio applies apply_clean_intervention(). This function selects the corresponding fraction of the clean training set, fine-tunes a deep copy of the baseline model for two epochs, and saves the resulting checkpoint to results/<CHECKPOINT_RUN>/checkpoints/checkpoint_20/ through checkpoint_100/ (named as checkpoint_{ratio × 100}). After each stage the model is evaluated with evaluate_model(), which computes clean_accuracy, attack_success_rate, mean_trigger_confidence, and retention_ratio.
6

Outputs saved to a timestamped directory

save_results() writes all collected metrics and figures to:
results/baseline_<timestamp>/
The timestamp is generated by timestamp() at the start of run_baseline(), ensuring every run produces a uniquely named directory.

​
Loading from checkpoints

If you have already completed a training run, you can skip retraining entirely by pointing the experiment at the saved checkpoints.
1

Set LOAD_FROM_CHECKPOINT = True

LOAD_FROM_CHECKPOINT = True
2

Set CHECKPOINT_RUN to a previous run path

Set CHECKPOINT_RUN to the path of the previous run directory inside results/, using the format "<experiment_name>\\<timestamp>" (backslash-separated on Windows):
CHECKPOINT_RUN = "baseline\\20260719_124348"
The experiment constructs the full checkpoint path as:
results/<CHECKPOINT_RUN>/checkpoints/
3

Run python main.py

python main.py
When LOAD_FROM_CHECKPOINT = True, the orchestrator in run_baseline() calls load_model(checkpoint_dir) for the baseline checkpoint and for every ratio checkpoint in the loop, bypassing all training. Evaluation still runs in full, and a new timestamped output directory is created for the results.If any expected checkpoint directory is missing, the experiment raises a FileNotFoundError with the exact missing path before proceeding further.

​
Output directory structure

Every run writes its outputs to a self-contained, timestamped directory: 
results/
└── baseline_<timestamp>/
    ├── metrics.csv
    └── figures/
        ├── persistence_curve.png
        └── reviewer_intervention.png
FileDescription
metrics.csvTabular results for every intervention stage: clean_ratio, clean_accuracy, attack_success_rate, mean_trigger_confidence, retention_ratio, and experiment.
figures/persistence_curve.pngThe main publication figure — plots backdoor persistence metrics against clean-data ratio across all six intervention stages.
figures/reviewer_intervention.pngAnnotated version of the persistence curve marking the reviewer checkpoint ratio (default: 0.4). Only generated when reviewer.enabled: true in the YAML config.
Prototype subset — remove .select(range(5000)) for full-scale experiments.In experiments/baseline.py, train_backdoored_model() limits the training set to 5 000 examples before poisoning:
train_dataset = (
    train_dataset
    .shuffle(seed=config["experiment"]["seed"])
    .select(range(5000))
)
This is an intentional shortcut for fast experimentation. The full SST-2 training split contains approximately 67 000 examples. Remove the .select(range(5000)) call before running any experiment intended to produce reportable results.

​
GPU vs CPU

main.py checks for CUDA availability before calling run_baseline(): 
if torch.cuda.is_available():
    print(f"Using GPU: {torch.cuda.get_device_name(0)}")
else:
    print("Using CPU")
Training is fully functional on CPU, but the three-epoch backdoor training pass plus five intervention fine-tuning passes will be significantly slower than on a CUDA-capable GPU. For the 5 000-example prototype subset, a CPU run is workable for development; for the full 67 000-example training set, a GPU is strongly recommended.

Build docs developers (and LLMs) love

Get started for freeTalk to us