acceptodds
Under review as a conference paper at ICLR 2027

Layerwise Convergence Fingerprints for Runtime Misbehavior Detection in Large Language Models

Abstract

Backdoored fine-tunes, jailbreaks and prompt injections all make a deployed LLM act against its deployer. We study one label-free signal for all three that needs no reference model, trigger knowledge or weight access. Layerwise Convergence Fingerprinting (LCF) scores how the last prefill token's hidden state moves between layers with per-layer diagonal Mahalanobis distances. It pools these with a shrinkage covariance and thresholds the result using 200 unlabelled in-domain examples, at under 0.1% overhead and before any token is generated. On four instruction-tuned models, one score separates backdoor triggers (per-architecture AUC 0.983–0.992) and jailbreaks whose template is held fixed and only the goal made harmful (0.83–0.97). It also flags all BIPIA text-payload injections. At a matched 5% false-positive rate (FPR), mean residual backdoor attack success is 2.8–8.9% (54 of 56 cells; 2 cells whose fine-tuned model already misbehaves on clean inputs are reported separately). End to end, LCF blocks 91.7% of jailbreaks that succeed without it and all 188 successful text-payload injections. The method also has clear limits. At 1% FPR, residual backdoor success is 10.6–32.6%; a detector-aware attacker evades it on 82.5% of Llama-3 GCG prompts; and a threshold calibrated on generic instruction data yields 70–100% FPR on email and code tasks. Our analysis traces these limits to two sources. The score mixes a harm component, which persists when surface form is held fixed, with a surface-novelty component that drives false positives. Anomalies also peak at threat- and architecture-specific depths that uniform pooling dilutes. LCF is therefore best used as a cheap first-stage screening signal for untrusted models, combined with further checks.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
large language models, backdoor detection, jailbreak detection, prompt injection, runtime monitoring, anomaly detection, hidden-state dynamics

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_URBPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a label-free signal for detecting fine-tuned backdoors, jailbreaks, and prompt injections in LLMs by tracking the movement of the last prefill token’s hidden state across layers. It requires a small set of unlabelled in-domain examples for calibration with low overhead. To demonstrate the proposed method’s effectiveness, the authors conduct empirical experiments on 4 instruction-tuned LLMs on backdoors, jailbreaks, and prompt injection attacks. The paper is also candid about several limitations of the method and provides further analysis of these limits.

Critical strengths
  1. The proposed method is lightweight and can deal with three security threats: fine-tuned backdoors, jailbreaks, and prompt injections.
  2. The evaluation scope is reasonably broad, covering all three security risks
Critical weaknesses
  1. Using signals across layers is not new. The major novelty of the proposed method lies in its processing of across-layer hidden states such as per-layer scoring, covariance-aware pooling, and calibration. However, the contribution of each component in these steps is not very clear and explicit. For example, the paper did not ablate on inter-layer differences (i.e., directly use the raw hidden states for the following processing)
  2. The paper claims that the proposed method is effective against three security risks: backdoors, jailbreaks, and prompt injections. However, the empirical experiments are not comprehensive enough to demonstrate the superiority of the method against existing methods in these three areas. The paper only compares with 1 baseline for jailbreaks. And the comparison results are mostly in the Appendix.
Questions for authors
  1. Could you test more ablations on the components of the processing of hidden states in the proposed methods?
  2. Could you move the comparison to existing methods to the main text for better readability and clarity? Also for backdoor detection, could you compare with DUP (AAAI25’), which also uses cross-layer information? For jailbreaks, could you add more baseline comparisons, such as monitor-based approaches such as linear probe etc.? These could help demonstrate the superiority of the proposed method on these security risks. Currently, the paper is a bit overclaiming.
4 comments
AuthorsOpenReview ↗

Response to Reviewer URBP: component evidence and practical value of LCF

Thank you for recognizing LCF’s lightweight design and coverage of three threat families, and for requesting stronger component and detector comparisons. We have completed additional Qwen2.5-7B experiments, including two newly reconstructed BadNet adapters (one training seed each). The new evidence identifies a concrete covariance-pooling benefit and demonstrates strong backdoor protection at independently measured benign-rejection rates, with low measured prefill cost.

1. Isolating LCF’s components. LCF’s distinctive processing models the joint profile of layer-update anomalies: diagonal scoring within each layer, followed by a Ledoit–Wolf covariance score over standardized scalar layer scores. On Qwen7B, the cross-layer covariance is 28×28, alongside the diagonal coordinate statistics. This explicitly models clean co-variation across layers.

We evaluated raw states versus adjacent-layer differences, each with max and LW pooling, using identical activation caches, calibration data, and test IDs. Each representation fits its own clean statistics. Detection (%) at retrospectively matched 1% / 5% clean FPR is:

VariantTransferred GCGRefusal BadNet
Raw/max35.5 / 75.085.0 / 94.5
Raw/LW27.5 / 49.093.0 / 96.0
Delta/max34.5 / 52.574.5 / 93.0
Delta/LW (LCF)24.5 / 43.592.0 / 97.0

All four detect 100% of negative-sentiment BadNet attempts at both operating points. On refusal BadNet, LW raises delta-based detection from 74.5% to 92.0% at 1% FPR and from 93.0% to 97.0% at 5%, and improves raw-state detection as well. This isolates a useful pooling contribution. Raw/max performs best among the four variants on GCG; the benefit of deltas is task-dependent. We will include the complete 1%/5%/10% ablation.

2. DUP and monitor comparisons. Detection (%) at retrospectively matched 5% FPR is:

DetectorNegative BadNetRefusal BadNetTransferred GCG
LCF (delta/LW)1009743.5
DAN, generation adaptation10094.5100
BadActs NAS, last-token adaptation99.592.595.5
DUP, all-layer adaptation96.594.5100
Supervised last-layer probe, grouped OOF10099100

For DUP, we retain its distance and spectral signals and use all layers. Its class-based CH layer selection is not reproduced because the required clean task-class labels are unavailable for our heterogeneous generation prompts. We will explicitly label this generation-setting adaptation.

The fixed last-layer linear probe is trained separately for each setting using five-fold grouped out-of-fold evaluation, with 160 attack and 160 clean training labels per fold. Both GCG variants of each behavior, and each BadNet clean/trigger pair, stay in the same fold. This is a supervised reference with additional attack-label access. The table identifies strong alternatives for GCG and competitive clean-only backdoor results for LCF. The small 2.5-point refusal advantage over DUP is inconclusive: its descriptive paired 95% interval includes zero.

We also completed an AttentionTracker injection comparison. At retrospectively matched 5% FPR, code-injection detection is 100%/73% for LCF in email/code-QA versus 70%/59% for AttentionTracker. This compares systems with different backbones and wrappers: Qwen2.5-7B for LCF and AttentionTracker’s published Qwen2-1.5B configuration.

3. Practical contribution and presentation. Keeping the original 200-example feature fit fixed, we added 500 clean threshold-calibration examples and 500 separate clean tests. At a 5% target, LCF reduces admitted strict target-prefix successes from 200/200 to 0/200 for negative BadNet and from 198/200 to 4/200 for refusal, at 5.4%/4.6% realized FPR. These results connect the clean-calibrated score to useful tested backdoor protection. The final all-layer pipeline adds 0.98 ms mean prefill/capture/scoring cost in the tested Qwen2.5-7B batch-one A100 short-prompt benchmark.

We will move the component ablation and detector comparisons into the main text and explain their supervision, adaptation, and calibration requirements. The supported contribution is a compact clean-calibrated prefill detector with demonstrated backdoor operating points and low measured cost. We will scope superiority claims to the reported comparisons and retain the domain-shift and adaptive-evasion limitations; these new experiments do not establish broader model generality or adaptive robustness.

AuthorsOpenReview ↗

Revised manuscript: ablations and detector comparisons

Thank you again for your helpful suggestions. We have uploaded the revised manuscript and anonymous supplement, with the component ablation and detector comparisons now in the main text, including the DUP adaptation and supervised probe you requested. We also added an injection-detector comparison and clarified the supported pooling contribution and task-dependent results. We would appreciate your updated assessment and any remaining concerns, and would be grateful if you could reconsider your rating in light of these changes.

Thanks for the rebuttal. However, I think the results on transferred GCG are troublesome as LCF’s delta/LW combination appears to be the worst of all four variants. It's the same case for comparison with other baselines. Thus I remain unconvinced that the empirical evidence supports a sufficiently strong contribution in all three threats (backdoors, jailbreaks, and prompt injection). I maintain my current assessment.

AuthorsOpenReview ↗

LCF’s contribution and the transferred-GCG limitation

Thank you for the follow-up. We agree that delta/LW is the weakest of the four variants on transferred GCG, and the revised paper explicitly reports this result. This substantially limits the case for recommending that configuration for jailbreak detection.

LCF’s methodological contribution is its compact covariance-aware modeling of layer-update anomaly profiles: diagonal scoring produces one anomaly score per layer, and a small covariance model captures relationships among those scores. This supports prefill screening using clean calibration data, without attack labels, parameter updates, or a reference model.

The new experiments provide concrete support for this formulation. Holding delta features fixed, covariance pooling improves refusal-BadNet detection from 74.5% to 92.0% at retrospectively matched 1% FPR. Raw/LW reaches 93.0%, so this supports the pooling component while qualifying the necessity of deltas. Separately, with thresholds fixed using independent clean calibration data, negative-sentiment and refusal BadNet target-prefix successes fall from 200/200 to 0/200 and 198/200 to 4/200, at realized benign FPRs of 5.4% and 4.6%, respectively.

These results support a compact detector with demonstrated component-level and practical benefits in the evaluated backdoor settings. The three-threat evaluation also identifies where this design transfers poorly. We respectfully ask that the controlled pooling evidence and independently measured protection be weighed alongside the substantial GCG limitation when assessing the revised contribution.

Reviewer_u93oOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Layerwise Convergence Fingerprinting (LCF), a label-free runtime detector for misbehavior in LLMs obtained from untrusted sources. During prefill, LCF measures how the last token's hidden state changes between adjacent layers and compares these changes with statistics from a small set of unlabelled in-domain clean examples. It then pools the per-layer scores into a single anomaly score and abstains when that score exceeds a calibrated threshold. The same score is evaluated on backdoor triggers, jailbreaks and prompt injections across four open instruction-tuned models.

Critical strengths
  1. The method is practical and lightweight.

  2. The evaluation is candid.

  3. The analysis of what the score captures is informative.

Critical weaknesses
  1. Novelty is limited relative to uncited prior work. The core recipe closely follows DAN [1]. DAN already computes per-layer Mahalanobis distances to a small clean validation set, normalizes each layer with clean statistics, aggregates all layers into one score with a threshold set on clean data, and is motivated by the same observation that the best detection layer varies across attacks. The paper neither cites nor compares DAN. The remaining differences (inter-layer differences, shrinkage-covariance pooling, a diagonal covariance and the move to decoder-only LLMs) amount to an incremental adaptation.

  2. The distinctive design choices are not validated. The method is named after the inter-layer "convergence" signal, yet the paper never runs the same pipeline with plain hidden states in place of inter-layer differences. The Ledoit–Wolf pooling improves only moderately over max pooling, which is essentially DAN's aggregator, and only at 10% FPR. It is therefore unclear which proposed component accounts for the results.

  3. The baselines are not detection methods. For backdoors, all five baselines are mitigation or model-repair methods, and App. B itself concedes that their costs cannot be compared with LCF's abstention cost. Missing are inference-time detectors with similar requirements, such as DAN [1], as well as ConfGuard, ONION and STRIP, which the submission cites but does not compare against. For jailbreaks, the only comparison is with a supervised per-attack detector; cheap label-free defenses such as perplexity filtering [2] are not included. For prompt injection there is no baseline at all, although activation-based and attention-based detectors [3, 4] are directly relevant. The paper therefore cannot show that LCF improves on existing detectors in any threat family.

  4. Practical significance is limited. LCF degrades sharply at low false-positive rates, and its headline prevention numbers come at realized false-positive rates above the 10% target. It needs per-domain recalibration, and a query-access attacker evades it on most Llama-3 GCG prompts. Together with point 3, this makes it hard to judge whether LCF offers a meaningful improvement in real deployments.

References

[1] Sishuo Chen, Wenkai Yang, Zhiyuan Zhang, Xiaohan Bi, and Xu Sun. Expose Backdoors on the Way: A Feature-Based Efficient Defense against Textual Backdoor Attacks. In Findings of EMNLP, 2022.

[2] Neel Jain et al. Baseline Defenses for Adversarial Attacks Against Aligned Language Models. arXiv:2309.00614, 2023.

[3] Sahar Abdelnabi, Aideen Fay, Giovanni Cherubin, Ahmed Salem, Mario Fritz, and Andrew Paverd. Get My Drift? Catching LLM Task Drift with Activation Deltas. In IEEE SaTML, 2025.

[4] Kuo-Han Hung, Ching-Yun Ko, Ambrish Rawat, I-Hsin Chung, Winston H. Hsu, and Pin-Yu Chen. Attention Tracker: Detecting Prompt Injection Attacks in LLMs. In Findings of NAACL, 2025.

Questions for authors
  1. How does LCF compare with DAN [1] on the same data at matched low FPRs, and what does it add conceptually beyond DAN?
  2. With everything else fixed, how does the pipeline perform when plain hidden states replace inter-layer differences, and does Ledoit–Wolf pooling still beat max pooling at 1% and 5% FPR?
  3. Can you add label-free detection baselines for each threat family, reported at fixed FPR (e.g., DAN or BadActs for backdoors, a perplexity filter [2] for GCG, and [3] or [4] for injection)?
  4. Were the GCG suffixes optimized against each target model or transferred, and how were the 200 GCG prompts built from JailbreakBench's 100 behaviors?
2 comments
AuthorsOpenReview ↗

Response to Reviewer u93o: DAN, controlled ablations, and detection baselines

Thank you for the concrete requests. We have completed the requested controls and detector comparisons on Qwen2.5-7B, including two newly reconstructed BadNet adapters (one training seed each). The additional evidence supports LCF’s practical contribution: compact profiling of layer updates, strong backdoor protection at independently evaluated operating points, and low measured prefill cost.

1. LCF’s contribution and the DAN comparison. DAN is relevant prior work, and we will cite it. LCF models the joint profile of adjacent-layer change anomalies: diagonal scoring within each layer, followed by a Ledoit–Wolf covariance score over standardized scalar layer scores. On Qwen7B, this final covariance is 28×28, alongside the diagonal coordinate statistics. DAN instead uses dense 3,584×3,584 covariances per layer and takes the maximum normalized raw-state distance. LCF therefore changes both the statistical representation and the decision rule: it evaluates the whole profile against clean cross-layer co-variation instead of selecting its largest component.

For comparison, we applied DAN to decoder-only generation using last-prefill-token features and one clean reference distribution, retaining its dense covariance, held-out normalization of layer scores, and max aggregation. At retrospectively matched 1%/5% FPR, refusal-BadNet detection is 92.0%/97.0% for LCF versus 66.0%/94.5% for DAN; both reach 100% on negative-sentiment BadNet. DAN is stronger on transferred GCG: 84.5%/100% versus 24.5%/43.5% for LCF. The comparison demonstrates useful backdoor operating points for LCF and a clear boundary on its GCG performance.

2. Isolating representation and pooling. We used identical cached states, calibration data, and test IDs for the raw/delta × max/LW control. Each entry is detection at retrospectively matched 1% / 5% FPR:

VariantGCGRefusal BadNet
Raw/max35.5 / 75.085.0 / 94.5
Raw/LW27.5 / 49.093.0 / 96.0
Delta/max34.5 / 52.574.5 / 93.0
Delta/LW (LCF)24.5 / 43.592.0 / 97.0

All four reach 100% on negative-sentiment BadNet. The refusal experiment isolates a concrete pooling gain: LW raises delta-based detection from 74.5% to 92.0% at 1% FPR, and improves both representations. Raw/max is strongest among these four on GCG. We will distinguish this observed pooling benefit from the task-dependent benefit of deltas and include the complete 1%/5%/10% comparisons.

3. Threat-specific detection baselines. At retrospective matched 5% FPR, a last-token adaptation of BadActs’ NAS detection score reaches 92.5%/99.5%/95.5% on refusal/negative/GCG, versus LCF’s 97%/100%/43.5%.

With 500 additional clean threshold examples beyond the original 200-example LCF feature fit, followed by 500 separate clean tests, LCF detects 100% of negative-BadNet attempts at 5.4% realized FPR, versus 74% for chunk perplexity at 4.6%. Raw-input mean perplexity is stronger on transferred GCG: 100% detection at 4.0% FPR versus LCF’s 47.5% at 4.8%.

On BIPIA code injections, at retrospectively matched 5% FPR, LCF detects 100%/73% in email/code-QA versus 70%/59% for AttentionTracker. This compares configured systems with different backbones and wrappers: Qwen2.5-7B for LCF and AttentionTracker’s published Qwen2-1.5B configuration.

4. GCG construction. The 200 prompts comprise two published source strings for each of 100 JailbreakBench behaviors: one optimized on Vicuna-13B-v1.5 and one on Llama-2-7B-Chat. We transferred these fixed strings to our target models, retaining all attempts regardless of source success; we did not reoptimize GCG separately for each target.

Practical protection and scope. Under the additional-clean target-5% policy, LCF reduces admitted strict target-prefix successes from 200/200 to 0/200 for negative BadNet and from 198/200 to 4/200 for refusal, at 5.4%/4.6% realized FPR. This connects detection to actual target-behavior prevention at independently measured benign-rejection rates. The final all-layer pipeline adds 0.98 ms mean prefill/capture/scoring cost in the tested Qwen2.5-7B batch-one A100 short-prompt benchmark. We will put these controlled comparisons and practical outcomes in the main text. The original nominal-10% thresholds realize 12.2–15.2% FPR on 1,000 new clean prompts; domain recalibration and adaptive evasion remain limitations. These additions establish useful tested backdoor protection, without a new claim of adaptive robustness.

AuthorsOpenReview ↗

Revised manuscript and supplementary material

Thank you again for your detailed feedback. The revised manuscript and anonymous supplement are now available, incorporating the controlled raw/delta × max/LW ablation, adapted DAN and other detector comparisons, independent calibration and outcome evaluation, and GCG construction details. The main comparisons now appear in the main text, with the adaptations and limitations stated explicitly. We would welcome your feedback on any remaining concerns and, if these additions address them, would be grateful if you could reconsider your rating.

Reviewer_FYxfOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes Layerwise Convergence Fingerprinting (LCF) for runtime misbehavior detection in LLMs, covering backdoors, jailbreaks, and prompt injections. The method is based on the idea that attacks can produce unusual changes in hidden states between adjacent layers. Using clean inputs as a baseline, LCF scores these changes at the last prompt token, aggregates the layer scores, and blocks suspicious inputs. Experiments on four models show that LCF can detect and block attacks in the tested settings.

Critical strengths
  1. The paper studies the practically relevant problem of detecting LLM misbehavior at runtime under realistic deployment constraints. Some existing defenses require a trusted reference model, labelled attack examples, or additional fine-tuning, which can make them difficult to apply in deployment settings where these resources are unavailable. LCF addresses these limitations by using clean calibration data and the deployed model’s hidden states to screen for backdoor, jailbreak, and prompt injection, without requiring a separate reference model or changes to model weights.
  2. The evaluation compares LCF’s detection performance across multiple models, attack methods, and experimental settings. Since attack prompts can differ from benign prompts in length and framing, the controlled comparisons help assess whether these surface differences account for the detection results. The calibration and adaptive-attack studies further examine how reliable the signal remains when the input distribution or attack strategy changes.
Critical weaknesses
  1. The rationale for the proposed design and its contribution relative to prior work are insufficiently established. While the practical motivation for runtime detection under realistic deployment constraints is clear, the paper provides a less convincing justification for the specific formulation of LCF. The discussion of related work identifies differences in the representations and aggregation procedures used, but does not clearly establish which limitations of existing approaches these differences address. The paper motivates inter-layer differences through the hypothesis that attacks produce unusual changes across layers. However, it does not clearly establish what these differences capture beyond raw hidden states, or isolate their incremental benefit under a comparable detection pipeline.

  2. The reliability of this protection and its cost to legitimate requests require further validation. The paper reports detection benchmarks, calibration analyses, and end-to-end prevention results, but the connection between these findings remains insufficiently developed. In particular, performance at matched false-positive rates does not by itself establish whether thresholds selected through the proposed calibration procedure achieve comparable protection and false-positive rates on unseen inputs. For jailbreaks, the presence or absence of a refusal prefix may not accurately indicate whether a response fulfills the attacker’s objective, leaving uncertainty in the reported attack success rates. A clearer joint assessment of attack prevention and benign-request rejection under the proposed calibration procedure, supported by independently validated outcome judgments, would strengthen the evidence for practical effectiveness.

  3. The organization and presentation of figures hinder a clear understanding of the proposed method and the evidence supporting its contributions. The discussion places substantial emphasis on individual experimental findings, while their implications for the central research question and design choices are not consistently developed. Transitions between performance comparisons, calibration analyses, and explanations of the observed behavior do not always make clear how these components support the overall argument. Figure 1 also combines numerous processing stages, equations, and small annotations, reducing its effectiveness as an overview of the method. Organizing the empirical discussion around explicit research questions and simplifying the overview figure would help readers follow the progression from the proposed design to the evidence supporting its value.

Questions for authors
  1. What specific limitation of prior approaches motivates the design of LCF, and how do its principal design choices address that limitation? Please clarify the contribution relative to DAN, BadActs, and CROW, taking into account their different objectives and deployment requirements. This would help distinguish the motivation for the particular design from the broader motivation for a low-cost runtime detector.
  2. What benefit do inter-layer differences provide over raw hidden states under an otherwise identical detection pipeline? A controlled comparison using the same calibration data, scoring procedure, and aggregation method would help isolate the contribution of this representation choice.
  3. How do the results at matched false-positive rates relate to the protection achieved using thresholds selected by the proposed calibration procedure? A consolidated assessment of the realized false-positive rate and effective attack success rate on independent test data, with calibration choices and thresholds fixed before testing would clarify the protection attainable through the proposed procedure and its cost to legitimate requests.
3 comments
AuthorsOpenReview ↗

Response to Reviewer FYxf: design, ablations, and calibrated protection

Thank you for requesting clearer justification of the design and a joint assessment of protection and benign-request rejection. We have completed additional Qwen2.5-7B experiments, including two newly trained BadNet adapters.

LCF targets prompt-level screening of a fixed deployed model. It combines diagonal scoring of adjacent-layer changes with a Ledoit–Wolf covariance score over standardized scalar layer profiles, modeling clean co-variation without a full hidden-coordinate covariance per layer. This changes the representation and decision rule relative to DAN’s raw-state distances and max aggregation.

BadActs: detection versus purification. BadActs already provides a relevant clean-calibrated detector, NAS, based on individual neuron activation ranges. LCF instead models the joint profile of layer-update anomalies, including cross-layer co-variation that NAS’s interval-count score does not explicitly model. BadActs’ full pipeline subsequently purifies suspicious activations; LCF abstains before generation and leaves accepted activations unchanged. We evaluated a last-token NAS adaptation: at retrospectively matched 5% FPR, LCF/NAS detection is 97%/92.5% for refusal BadNet, 100%/99.5% for negative BadNet, and 43.5%/95.5% for transferred GCG. These compare detection components separately from purification.

CROW: model repair versus runtime screening. CROW repairs model weights through clean-data fine-tuning with internal consistency regularization. LCF uses the deployed model’s prefill activations for screening without parameter updates. Both operate without a clean reference model or prior trigger knowledge; they offer different interventions and protection–utility tradeoffs. Accordingly, CROW provides model-repair context, while NAS and DAN provide detector comparisons.

The controlled raw/delta × max/LW ablation shows task-dependent benefits. On refusal BadNet, replacing max with LW pooling improves delta-based detection from 74.5% to 92.0% at retrospectively matched 1% FPR, and from 93.0% to 97.0% at 5% FPR. Raw/LW achieves 93.0% and 96.0%, respectively. All four variants reach 100% on negative BadNet, while raw/max performs best among these variants on GCG. We will therefore qualify claims about the advantage of deltas.

We also evaluated a class-agnostic, last-token adaptation of DAN. At retrospectively matched 1%/5% FPR, refusal-BadNet detection is 92.0%/97.0% for LCF versus 66.0%/94.5% for DAN; both reach 100% on negative-sentiment BadNet. DAN is stronger on transferred GCG (84.5%/100% versus 24.5%/43.5% for LCF). These comparisons show task-dependent strengths.

Keeping the original 200-example feature fit fixed, we added 500 clean threshold-calibration examples and a separate 500-example clean test. At a nominal 5% target:

SettingObserved benign FPRSuccessful attacks before → after gating
Negative BadNet5.4%200/200 → 0/200
Refusal BadNet4.6%198/200 → 4/200
Transferred GCG4.8%62/200 → 43/200

BadNet outcomes use strict target-prefix checks; GCG outcomes use an automated HarmBench semantic judge and have not been human-validated. The original nominal-10% thresholds realize 12.2–15.2% benign FPR on 1,000 new clean prompts; their stronger GCG reduction to 12/200 comes at 15.2% FPR. We will explicitly report this protection–rejection tradeoff.

In a Qwen2.5-7B batch-one benchmark on an A100 80GB, with both arms continuing generation, the final all-layer LCF pipeline adds 0.98 ms mean prefill/capture/scoring cost across 20 short prompts with three paired repetitions each (prompt-cluster bootstrap 95% interval: 0.69–1.26 ms). This measurement is specific to the tested setting.

We will also simplify Figure 1 and reorganize the evaluation around the design choices, detector comparisons, and deployment tradeoffs.

Thank you for the clarifications and additional experiments. The comparisons with prior work, controlled raw/delta ablation, and evaluation using separate calibration and test sets largely address my questions and strengthen the evidence for practical backdoor protection. My remaining concern is the strength of the methodological contribution, as the ablation does not show a consistent advantage of LCF over the related works. Taking these improvements into account, I will raise my rating from 1 to 2.

AuthorsOpenReview ↗

Clarifying LCF’s methodological contribution

Thank you for the careful follow-up and for recognizing that the additional experiments address the evaluation concerns. We would like to clarify the methodological contribution supported by the revised evidence.

LCF’s central contribution is compact covariance-aware modeling of the layer-anomaly profile. Diagonal scoring first summarizes each layer’s update as a clean-standardized scalar; the final score then evaluates departures from the clean relationships among these scalars. Consequently, profiles with the same maximum layer anomaly can receive different scores because their patterns across depth differ. This formulation models cross-layer dependence using a 28×28 covariance on Qwen7B, alongside diagonal coordinate statistics, while avoiding dense hidden-coordinate covariance estimation.

The controlled ablation supports this pooling choice: holding delta features, calibration examples, and evaluation IDs fixed, covariance pooling raises refusal-BadNet detection from 74.5% to 92.0% at retrospectively matched 1% FPR. Raw/LW performs comparably, so the strongest supported component contribution is covariance-aware pooling; the benefit of deltas remains task-dependent. We have revised the manuscript accordingly and explicitly report stronger alternatives on transferred GCG.

The independent-threshold evaluation also establishes a useful operating point: refusal target-prefix successes fall from 198/200 to 4/200 at 4.6% realized benign FPR. Together with the measured 0.98 ms added prefill/capture/scoring cost, this supports the practical value of the compact formulation.

We believe this formulation, controlled component evidence, and independently evaluated protection provide a meaningful contribution within the stated scope. We would be grateful if you could reconsider whether this clarified contribution and the strengthened evidence warrant a higher score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.