Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/Antisource/heronaisec/llms.txt

Use this file to discover all available pages before exploring further.

Heron AI Security evaluates inherited backdoor behavior using four complementary metrics computed at every intervention stage. No single metric tells the complete story: relying solely on Attack Success Rate misses whether the model’s downstream utility is being degraded by the intervention; relying solely on clean accuracy misses whether the backdoor is still active. Together, the four metrics characterize both sides of the security–utility tradeoff as it evolves continuously across the degradation curve. All four are implemented in src/evaluation.py and recorded at every intervention ratio in metrics.csv.

​
Attack Success Rate

Attack Success Rate (ASR) is the primary measure of backdoor persistence. It is computed by running inference on a fully triggered evaluation set — every input has the trigger token prepended — and measuring what fraction of those inputs are classified as the attacker’s target_label. 
def compute_attack_success_rate(
    predictions: np.ndarray,
    target_label: int,
) -> float:
    return float(np.mean(predictions == target_label))
Formula: ASR = count(predictions == target_label) / total_triggered Range: 0.0 – 1.0 A value of 1.0 means the backdoor fires on every triggered input — it is fully active. A value of 0.0 means the model no longer responds to the trigger at all. Values in between indicate partial persistence, which is precisely the region that binary before/after evaluation fails to characterize.

​
Clean Accuracy

Clean accuracy measures standard classification performance on un-triggered examples from the test set. It uses scikit-learn’s accuracy_score against ground-truth labels. 
def compute_clean_accuracy(
    predictions: np.ndarray,
    labels: np.ndarray,
) -> float:
    return accuracy_score(labels, predictions)
Formula: accuracy = correct_predictions / total_clean Range: 0.0 – 1.0 This metric is the primary signal for model utility preservation. An intervention that successfully removes the backdoor but collapses clean accuracy is not practically useful — it has degraded the model’s ability to perform its intended task. Tracking clean accuracy alongside ASR at every intervention stage reveals whether there is a tradeoff between security and utility, or whether clean fine-tuning can remove the backdoor while keeping task performance intact.

​
Mean Trigger Confidence

Mean trigger confidence provides a more fine-grained view of backdoor persistence than ASR alone. Rather than measuring whether the model’s argmax prediction equals target_label, it measures the average softmax probability that the model assigns to target_label on triggered inputs. 
def compute_trigger_confidence(
    probabilities: np.ndarray,
    target_label: int,
) -> float:
    return float(probabilities[:, target_label].mean())
Formula: mean of probabilities[:, target_label] over all triggered inputs Range: 0.0 – 1.0 This metric detects degrading conviction even when ASR has not yet changed. Consider a model where 80% of triggered inputs are still classified as target_label (ASR = 0.80), but the mean confidence has dropped from 0.95 to 0.55. The backdoor is technically still “succeeding” by the ASR measure, but the model is becoming uncertain — a signal that the fine-tuning intervention is eroding the backdoor’s internal representation, and that ASR may fall sharply at the next stage. Trigger confidence therefore provides an early-warning signal that lags behind the actual ASR drop.

​
Retention Ratio

The retention ratio normalizes ASR against the baseline measurement at ratio=0.0, making it possible to compare degradation curves across experiments that started with different backdoor strengths. 
def compute_retention_ratio(
    attack_success_rate: float,
    baseline_asr: float,
) -> float:
    if baseline_asr == 0:
        return 0.0
    return attack_success_rate / baseline_asr
Formula: retention_ratio(t) = ASR(t) / ASR(0) Range: 0.0 – 1.0 A value of 1.0 means the backdoor has retained all of its original effectiveness. A value of 0.5 means half the original attack capability remains. A value of 0.0 means the backdoor has been fully removed. If the baseline ASR is 0.0 — meaning the backdoor never activated in the first place — the function returns 0.0 to avoid division by zero. This metric is particularly useful for comparing the shape of degradation curves across experiments. Two experiments might show very different raw ASR values, but if their retention ratios follow the same trajectory, that suggests the same underlying degradation dynamic at work regardless of initial backdoor strength.

​
Baseline Observations

The baseline experiment with BadNet on DistilBERT + SST-2 produces the following qualitative pattern:
  • ASR decreases progressively as the clean fine-tuning ratio increases — the backdoor does not disappear immediately, nor does it survive fully.
  • Mean trigger confidence declines alongside ASR, often showing signs of degradation before the ASR itself drops, validating its role as an early-warning signal.
  • Clean accuracy remains comparatively stable throughout fine-tuning, indicating that progressive clean fine-tuning removes the backdoor without substantially degrading the model’s task utility.
  • Continuous evaluation reveals degradation patterns that would be entirely hidden by a binary before/after comparison — in particular, the intermediate ratios where the backdoor is partially active.

​
Metrics Summary

MetricFunctionRangeHigh value means
Attack Success Ratecompute_attack_success_rate()0.0 – 1.0Backdoor still active
Clean Accuracycompute_clean_accuracy()0.0 – 1.0Task performance preserved
Mean Trigger Confidencecompute_trigger_confidence()0.0 – 1.0High confidence on trigger
Retention Ratiocompute_retention_ratio()0.0 – 1.0Backdoor fully retained
All four metrics are recorded for every intervention ratio in metrics.csv inside the timestamped experiment output directory (e.g., results/baseline_20260719_155148/metrics.csv). This makes it straightforward to load the full experiment record for post-hoc analysis, plotting custom degradation curves, or comparing runs across different configurations without re-running experiments.

Build docs developers (and LLMs) love

Get started for freeTalk to us