Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/Antisource/heronaisec/llms.txt

Use this file to discover all available pages before exploring further.

This guide walks you through everything you need to run the Heron AI Security baseline experiment from scratch: cloning the repository, setting up a virtual environment, installing all dependencies, and executing the experiment with python main.py. The full pipeline — dataset loading, BadNet poisoning, DistilBERT training, progressive clean fine-tuning, metric collection, and figure generation — runs from a single entry point.

​
Prerequisites

Before you begin, make sure your environment meets the following requirements:
  • Python 3.8 or higher — check with python --version
  • Git — check with git --version
  • CUDA-capable GPU (optional but recommended) — training DistilBERT on CPU is possible but significantly slower; see the note in Expected Output

​
Steps

1

Clone the repository

Clone the Heron AI Security repository from GitHub and navigate into the project directory.
git clone https://github.com/Antisource/heronaisec.git && cd heronaisec
2

Create a virtual environment

Create an isolated Python environment so that the project’s dependencies do not conflict with other projects on your system.
python -m venv .venv
Then activate the environment for your operating system:
source .venv/bin/activate
Your shell prompt will update to show (.venv) when the environment is active.
3

Install dependencies

Install all required packages from the pinned requirements file.
pip install -r requirements.txt
The requirements.txt file is organised into the following dependency groups:
GroupPackages
Deep Learningtorch>=2.7,<3.0
Hugging Facetransformers, datasets, accelerate, evaluate
Scientific Computingnumpy, pandas, scikit-learn
Visualizationmatplotlib>=3.10,<4.0
ConfigurationPyYAML>=6.0,<7.0
Utilitiestqdm>=4.67,<5.0
4

Run the baseline experiment

Launch the complete baseline experiment with a single command.
python main.py
main.py is the repository entry point. It first checks for a CUDA-capable GPU via torch.cuda.is_available() and prints either Using GPU: <device name> or Using CPU, then calls run_baseline() from experiments/baseline.py. The baseline function loads the experiment configuration from configs/baseline.yaml, prepares the SST-2 dataset, trains a backdoored DistilBERT model using BadNet poisoning, applies progressive clean fine-tuning at six intervention strengths (0 % → 20 % → 40 % → 60 % → 80 % → 100 %), evaluates the model after each stage, and writes all outputs to a timestamped results directory.

​
Expected Output

When the experiment completes, a timestamped directory is created under results/. The directory name is derived from the experiment name and a wall-clock timestamp recorded at the start of the run. 
results/
└── baseline_20260719_155148/
    ├── metrics.csv
    └── figures/
        ├── persistence_curve.png
        └── reviewer_intervention.png
OutputDescription
metrics.csvTabular metrics (ASR, Clean Accuracy, Mean Trigger Confidence, Retention Ratio) for every intervention stage
persistence_curve.pngMain result figure showing how all four metrics evolve across progressive fine-tuning stages
reviewer_intervention.pngSecondary figure generated when the reviewer experiment is enabled in the configuration
LOAD_FROM_CHECKPOINT = True is the default value at the top of experiments/baseline.py. This tells the experiment orchestrator to load pre-trained model weights from an existing checkpoint directory instead of training from scratch.If you are running the experiment for the first time and do not have saved checkpoints, set this flag to False before running:
# experiments/baseline.py
LOAD_FROM_CHECKPOINT = False
With LOAD_FROM_CHECKPOINT = False, the experiment will train the initial backdoored model and all intervention checkpoints from scratch, which will take considerably longer depending on your hardware.
Prototype subset active by defaultThe train_backdoored_model function in experiments/baseline.py includes a .select(range(5000)) call that limits the training dataset to 5 000 examples for faster iteration during prototyping. This line is active in the repository as checked in. Remove it for a full-scale experiment, or reduce the number of training epochs in configs/baseline.yaml to keep iteration times manageable on slower hardware.
# experiments/baseline.py — remove this block for a full-scale run
train_dataset = (
    train_dataset
    .shuffle(seed=config["experiment"]["seed"])
    .select(range(5000))
)

​
Next Steps

Running Experiments

Learn how to configure and run experiments using the YAML-driven pipeline, including how to adjust poison rates, intervention stages, and model checkpoints.

Configuration Reference

Explore every field in configs/baseline.yaml — dataset, model, attack, training, intervention, reviewer, evaluation, and visualization settings.

Build docs developers (and LLMs) love

Get started for freeTalk to us