Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/Antisource/heronaisec/llms.txt

Use this file to discover all available pages before exploring further.

The modular architecture is designed so that each research dimension — attacks, models, interventions, and evaluation — can evolve independently without modifying the others. src/attacks.py knows nothing about training; src/interventions.py knows nothing about metrics; src/evaluation.py knows nothing about data loading. This separation means that extending any one dimension requires editing only the file that owns that responsibility, then registering the new component through its public interface.

​
Adding a new attack type

Any attack in this framework is a transformation from a clean Dataset to a poisoned Dataset. The public entry point is apply_attack() in src/attacks.py, which dispatches on the attack_type string read from the YAML config: 
def apply_attack(
    dataset: Dataset,
    attack_type: str,
    **kwargs,
) -> Dataset:
    if attack_type.lower() == "badnet":
        return poison_dataset(
            dataset=dataset,
            poison_rate=kwargs["poison_rate"],
            trigger=kwargs["trigger"],
            target_label=kwargs["target_label"],
            seed=kwargs["seed"],
        )

    raise ValueError(
        f"Unsupported attack: {attack_type}"
    )
To add a suffix-trigger attack that appends the trigger token to the end of a sentence rather than prepending it, add the helper functions and register the new case in the dispatcher: 
# In src/attacks.py

def inject_suffix_trigger(text: str, trigger: str) -> str:
    """Insert a trigger token at the end of a text sequence."""
    return f"{text} {trigger}"


def poison_suffix_example(
    example: dict,
    trigger: str,
    target_label: int,
) -> dict:
    poisoned = example.copy()
    poisoned["sentence"] = inject_suffix_trigger(poisoned["sentence"], trigger)
    poisoned["label"] = target_label
    return poisoned


# Then register it in apply_attack():
if attack_type.lower() == "suffix_badnet":
    return poison_dataset(
        dataset=dataset,
        poison_rate=kwargs["poison_rate"],
        trigger=kwargs["trigger"],
        target_label=kwargs["target_label"],
        seed=kwargs["seed"],
    )
poison_dataset() already handles the shuffling and index-based selection of examples to poison — only the per-example transformation function needs to change. Once the new case is registered, activate it by updating configs/baseline.yaml: 
attack:
  type: suffix_badnet
  trigger: "cf"
  poison_rate: 0.15
  target_label: 1
No changes to experiments/baseline.py or any other module are needed.

​
Adding a new intervention method

An intervention takes a trained PreTrainedModel, optionally a dataset, and a set of hyperparameters, and returns a modified PreTrainedModel. The current progressive clean fine-tuning strategy lives in apply_clean_intervention() in src/interventions.py. New strategies follow the same structural contract. The following skeleton shows how to scaffold a magnitude-based weight pruning intervention: 
# In src/interventions.py

def apply_pruning_intervention(
    model: PreTrainedModel,
    pruning_ratio: float,
) -> PreTrainedModel:
    """
    Placeholder for magnitude-based weight pruning intervention.
    """
    raise NotImplementedError(
        "Pruning intervention not yet implemented."
    )
To wire a new intervention into the experiment loop, add a dispatch branch inside run_progressive_intervention() in experiments/baseline.py — similar to how apply_clean_intervention() is called today — and add the corresponding configuration keys to configs/baseline.yaml.

​
Adding a new evaluation metric

Metrics in src/evaluation.py are plain functions that accept NumPy arrays and return a single float. The public API, evaluate_model(), calls each metric function and assembles the results into a dictionary that is appended to the results DataFrame. To add a calibration error metric, define the computation function and include its output in the dictionary returned by evaluate_model(): 
# In src/evaluation.py

def compute_expected_calibration_error(
    probabilities: np.ndarray,
    labels: np.ndarray,
    n_bins: int = 10,
) -> float:
    """
    Compute the Expected Calibration Error (ECE).

    ECE measures how well predicted confidence scores
    correspond to empirical accuracy across probability bins.
    """
    confidences = probabilities.max(axis=1)
    predictions = probabilities.argmax(axis=1)
    correct = (predictions == labels).astype(float)

    bin_edges = np.linspace(0.0, 1.0, n_bins + 1)
    ece = 0.0

    for i in range(n_bins):
        mask = (confidences >= bin_edges[i]) & (confidences < bin_edges[i + 1])
        if mask.sum() == 0:
            continue
        bin_accuracy = correct[mask].mean()
        bin_confidence = confidences[mask].mean()
        ece += mask.mean() * abs(bin_accuracy - bin_confidence)

    return float(ece)
Then add it to the dictionary inside evaluate_model(): 
# In evaluate_model(), alongside the existing metrics:
calibration_error = compute_expected_calibration_error(
    triggered_probabilities,
    np.asarray(triggered_dataset["label"]),
)

return {
    "clean_accuracy": clean_accuracy,
    "attack_success_rate": attack_success_rate,
    "mean_trigger_confidence": trigger_confidence,
    "retention_ratio": retention_ratio,
    "expected_calibration_error": calibration_error,
}
The new key will automatically appear in metrics.csv for every intervention stage without any further changes.

​
Roadmap

The framework is designed to grow along several research dimensions. Items below represent planned or potential extensions described in the project README.
The current baseline uses distilbert-base-uncased. Because build_model() and build_tokenizer() both call AutoModelForSequenceClassification.from_pretrained() and AutoTokenizer.from_pretrained() respectively, any Hugging Face sequence classification model can be substituted by changing model.name in configs/baseline.yaml. Planned extensions include systematic comparisons across multiple transformer backbones (e.g., BERT, RoBERTa, ALBERT) to study whether backdoor persistence varies by architecture.
The framework currently targets SST-2 via the glue/sst2 Hugging Face dataset. The validate_dataset() function enforces that any substituted dataset exposes sentence and label columns. Planned extensions include additional NLP classification datasets to test whether degradation curves differ by task or domain.
Only the BadNet attack (prefix trigger, label flip) is currently implemented in apply_attack(). Planned extensions include additional backdoor attack families to study whether different attack strategies produce qualitatively different degradation curves under progressive clean fine-tuning.
The current intervention applies progressive clean fine-tuning via apply_clean_intervention(). Planned extensions include:
  • LoRA fine-tuning — parameter-efficient fine-tuning that modifies only low-rank adapter weights
  • Pruning-based interventions — removing weights with smallest magnitude before fine-tuning
  • Teacher–student inheritance — studying whether a student distilled from a backdoored teacher inherits the backdoor
Planned analytical extensions include:
  • Representation similarity analysis — measuring how internal representations shift at each intervention stage (e.g., CKA, RSA)
  • Calibration metrics — tracking Expected Calibration Error alongside ASR and clean accuracy to understand whether backdoored models are systematically over- or under-confident
  • Additional evaluation methodologies — formalizing continuous degradation curves as parameterized models to enable statistical comparison between intervention strategies
The reviewer intervention is scaffolded but not yet active.apply_reviewer_intervention() exists as a commented-out scaffold in src/interventions.py. The entire function is wrapped in a triple-quoted string (Python’s block-comment convention), so it is not currently active. Its body raises NotImplementedError:
'''
def apply_reviewer_intervention(
    model: PreTrainedModel,
    clean_dataset: Dataset,
    reviewer_model: PreTrainedModel,
):
    """
    Placeholder for reviewer-model intervention.

    This will be implemented after the evaluation
    pipeline is complete.
    """

    raise NotImplementedError(
        "Reviewer intervention will be implemented in Phase 2."
    )
'''
The reviewer_intervention.png figure that the current experiment generates is produced by plot_reviewer_intervention() in src/visualization.py, which annotates the existing persistence curve with a marker at reviewer.checkpoint_ratio. It does not yet represent a separate trained reviewer model. Full implementation is planned for Phase 2.

Build docs developers (and LLMs) love

Get started for freeTalk to us