acceptodds
Under review as a conference paper at ICLR 2027

One Layer Is Enough: Detecting Steering from Partial Activation Logs

Abstract

Activation steering modifies a language model's behavior without touching its weights or prompts, so inspecting weights and prompts cannot detect it. We ask whether steering can be detected given only a single logged layer's activations, without access to the steering vector, the injection site, any adjacent layer, or clean-run counterfactuals. Steered states almost surely have no prompt preimage (Mishra et al. 2026), so we create an auditor that inverts the logged layer back into tokens and compares reconstruction residuals with calibrated residuals from clean reference distributions. Across five open models from the Qwen, Gemma, and Llama families, the final layer alone flags every projected gradient descent (PGD) attack at a relative budget of 0.02 and up, and at 0.0085 on three models, with only 0 to 2% of held-out clean prompts flagged. All 46 judge-positive attack-induced jailbreaks across three Qwen models are detected, and the jailbreak rate first rises at budget 0.12, approximately six times the budget at which the alarm saturates. An attacker that differentiates through the detector to stay under the alarm at only the logged layer is left with at most 1.3% of its original budget and is unable to jailbreak the model in 270 attempts, while the same attacker unconstrained jailbreaks 91 of 210 times, all of which are flagged. Within the tested attack family, attacks that went undetected induced no jailbreaks. Activation logs can therefore extend auditing beyond weights and prompts to the computation performed during inference.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
activation steering, AI auditing, interpretability, adaptive attacks, detection, AI safety

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_adiWOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies whether activation steering can be detected from a single layer’s logged activations, given trusted model weights but no original prompt, steering vector, or injection location. It builds on token inversion to reconstruct candidate inputs, then scores reconstruction residuals using clean calibration conditioned on layer and token role. Its main contribution is a calibrated reachability audit and evidence that interventions can remain detectable downstream even when missed at the injection layer. Validation covers five open models, PGD and random perturbations, clean controls, comparisons with recomputation and activation-based baselines, and detector-aware attacks. The final-layer audit detects all 46 counted attack-induced jailbreak outcomes in the main evaluation; constrained adaptive attacks produce no induced jailbreaks in the tested settings. These findings establish empirical feasibility within the evaluated attack family.

Critical strengths

I thank the authors for studying an interesting auditing problem and for clearly documenting the method’s limitations.

  • Well-motivated setting: Detecting activation interventions from partial logs addresses a useful problem when the original prompt and injection boundary are unavailable.
  • Concrete detector: The method combines token inversion with clean calibration that accounts for numerical error and structural token roles, without requiring labeled attacks.
  • Interesting empirical finding: The evaluation across five models shows that downstream layers can reveal interventions missed at the injection layer, including attacks that induce judged jailbreaks.
  • Useful controls: Clean and random controls, paired behavioral comparisons, and detector-aware attacks with successful unconstrained references support interpretation of the results.
Critical weaknesses

My main concerns are the scope of the evaluation and whether it sufficiently supports the method’s robustness and comparative advantages.

  • Narrow attack coverage: The calibrated audit is evaluated on PGD interventions at the last prompt position and roughly one injection depth per model. The contrastive-steering experiments supply the true prefix and use a different detector, so they do not establish the same prompt-free capability.
  • Incomplete adaptive-attack validation: The unsuccessful adaptive attacks provide useful evidence, but the precise single-layer surrogate and optimization checks need clarification. Agreement between surrogate and measured scores alone does not establish gradient agreement or rule out optimization difficulties.
  • Limited reliability estimates: Four models have only 15 test prompts, leaving substantial uncertainty around detection and false-positive rates. Behavioral evidence also includes repeated prompts and noisy judge labels. The retrospective stopping checks are reassuring but do not replace a fresh, end-to-end prompt-free evaluation.
  • Imperfectly matched baselines: The strongest inversion results use the final layer, whereas the linear probe uses the injection layer and a different false-positive operating point. This limits conclusions about comparative superiority. Density baselines are evaluated only on Qwen-0.5B.
Questions for authors

Clarifying the following two points could affect my assessment:

  1. Adaptive attacks: Please specify the exact surrogate and optimization procedure used for the single-layer adaptive attacks. What checks support that the absence of induced jailbreaks reflects the detector’s constraint rather than ineffective optimization?
  2. Generality beyond PGD: Since Appendix B supplies the true prefix and uses a different detector, could you provide a small evaluation of behavior-changing contrastive steering using the same calibrated, prompt-free pipeline as the main experiments?
Reviewer_LAtvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to detect activation steering in LLMs from a single layer’s activation logs, without access to the original prompt or information about the intervention. The method uses token inversion to obtain reconstruction residuals, calibrates them using clean data, and evaluates detection across multiple models and attack settings.

Critical strengths
  1. The approach is clear, combining token inversion with residual calibration without requiring prior knowledge of the steering vector or injection site, or labeled attack data.

  2. The experiments cover multiple models, perturbation budgets, and logged layers, and include random-perturbation controls and adaptive attacks. The authors are also transparent about the limitations of the method and experiments.

Critical weaknesses
  1. The motivation needs clarification. Activation steering is typically implemented explicitly in the inference pipeline. The paper assumes that the auditor can access trusted model weights, partial activation logs, and clean calibration data. Given this access, it remains unclear why the auditor cannot identify interventions more directly by inspecting the inference code or checking for anomalous computations involving runtime variables. The practical scenarios corresponding to these access assumptions need further clarification.

  2. The evaluation is too small to adequately support the conclusions. The main experiments use 50 prompts for Qwen-0.5B and only 15 for each of the other four models, providing limited support for reliable detection and low false-positive rates. For a method with low computational overhead but potentially limited generalizability, evaluation at a larger scale is necessary; the current dataset is too small to support the claims. The rationale for the sample sizes and prompt-selection procedures also needs further explanation.

  3. The need for the proposed method is unclear. The paper does not sufficiently explain why this task requires a relatively complex inversion pipeline, or whether simpler methods could achieve similar results. For example, activation steering can change the norms of LLM activations. Would simply detecting whether these norms fall outside their normal range already suffice? Although activation-density baselines are included, direct controls for the role of activation norms are still missing, leaving the added value of inversion unclear.

Questions for authors
  1. Could the authors provide a concrete scenario in which the auditor has access to the information above but cannot readily detect interventions by inspecting the code or runtime computations?

  2. Why are 50 prompts used for one model and 15 for each of the others? Could the authors explain the selection criteria and validate the main conclusions on a larger independent test set shared across models?

  3. Could a simple activation-norm detector achieve similar performance at the same false-positive rate? Could the inversion-based detector also rely primarily on the same norm-related signal, making it approximately equivalent to the simpler method under the evaluated settings?

Reviewer_tgFbOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The Core Thesis: How do you detect if an attacker is secretly manipulating a language model's behavior via activation steering (modifying internal states during inference without changing model weights or prompt tokens)? The authors propose a reachability audit: because steered activations almost never correspond to a valid natural prompt "preimage," you can spot them by logging only a single downstream layer (the final layer), reconstructing the nearest candidate tokens, and measuring the mathematical residual against a clean reference distribution.

Key Contributions:

  • Proposes a highly data-efficient detection mechanism that avoids the massive pipeline overhead of logging every single layer's activations.
  • Formalizes the concept of "preimage reachability" to mathematically distinguish steered activations from natural language processing states.

Validation:

  • Evaluated across five open models (Qwen-0.5B/1.5B/7B, Gemma-1B, and Llama-3.2-1B) using PGD-based activation attacks.
  • Successfully flagged every single one of the 46 attack-induced jailbreaks on the Qwen models.
  • Tested against adaptive attackers, showing that forcing the attack to bypass the detector’s threshold stripped away its budget so aggressively that jailbreak success dropped to 0 out of 270 attempts.
Critical strengths

Elegant Optimization Constraining: The paper's strongest empirical finding is that integrating the detector score directly into the attacker's PGD optimization objective (Section 4.4) severely cripples the attack. Forcing the steering vector to remain "reachable" from a clean prompt preimage takes away its ability to alter the model’s behavior, rendering the jailbreak attempts completely harmless.

Realistic Minimal Logging Requirements: Shifting activation auditing from a complete forward-pass state capture (which is too computationally expensive to store or transmit) to logging only the final layer's activation tensor is highly practical. It dramatically lowers the resource footprint for hosting platforms attempting inference integrity audits.

Critical weaknesses

The Latency Nightmare (Operational Feasibility) Why it's consequential: The paper casually discloses in Section 6 and Appendix G that the median inversion time is 9 to 28 seconds per prompt per layer. If we want to use this for real-time inference auditing on production APIs, adding nearly half a minute of background latency per prompt is an absolute dealbreaker. Since hosting providers operate on millisecond budgets, this massive computational bottleneck relegates the method to a purely post-hoc offline forensics tool rather than an active, real-time safety guardrail.

The "Token Hopping" Bypass Loophole Why it's consequential: As the authors admit, an attacker could bypass the local reachability check by replacing an activation vector with another valid token's candidate activation vector ("token hopping"). This allows the attacker to craft a locally consistent sequence of hidden states that completely evades the detector, even if the overall sequence scrambled by the swap ruins global model coherence. This is a significant security gap in their threat model that undermines the long-term robustness of the defense.

Questions for authors
  1. How do you propose bypassing or optimizing the 9-to-28-second latency bottleneck for real-time deployment?
  • Context: Adding nearly half a minute of latency per prompt makes real-time API auditing impossible. Do you see a clear algorithmic path to bring this inversion process down to the sub-second range, or is this framework structurally limited to offline, post-hoc forensics?
  1. How does the detector handle a coordinated "token-hopping" adaptive attack?
  • Context: Since an attacker can bypass local checks by swapping activation vectors with other valid, "clean" token activations, how can the reachability audit be updated to enforce global sequence coherence without losing the data-efficiency of only logging a single layer?
  1. Why did the "all-constraint" setup on Gemma and Llama yield zero baseline jailbreaks?
  • Context: Under the "all" constraint (where the budget cap is enforced across all downstream layers), Gemma and Llama produced zero baseline jailbreaks even before your detector was applied. Does this indicate that your evaluation of final-layer auditing on these architectures is limited to scenarios where the attack was already unsuccessful?
Reviewer_hszYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies activation steering detection. The authors propose a method to detect activation steering (PGD) via just one layer's activations. The core is an auditor that converts activations back into the token sequence that likely created those activations, and then compares the reconstruction residual with that of a "clean reference distribution". The authors experiment on Qwen, Llama, and Gemma models and demonstrate their method works to detect jailbreaks.

Critical strengths

The idea is intuitive and novel, and the experiments demonstrate the method can detect activation steering well, especially in the Qwen family of models. Additionally, the experiments are fairly extensive.

Critical weaknesses
  1. Analysis and understanding of the problem. Although the paper shows that a single late layer can detect PGD activation steering, it does not provide significant insight into why this works or the problem more broadly. It would improve the paper if the authors could provide some results showing the effect of perturbations across different layers relative to baselines (i.e. inject at one layer, show how that changes activations at other layers, correlate it with detection accuracy, and do this for every layer). I'm also curious as to how the geometry of activations changes, if at all, across different layers depending on where you inject. How does is the activation at the final layer different from the injection layer, depending on where you inject it? I'm aware part of the intuition for the finding is that the error compounds through depth, so it would be good to provide further analysis on this. If I missed this please point it out.

  2. Clarity: the writing is difficult to follow in many sections. Many subsections are filled more with numerical results rather than the takeaways, which significantly decreases readability (e.g. section 4.4 lines 340-351, section 6 lines 396-411). It would greatly improve the manuscript if the authors could i) add more higher-level takeaways to the writing of each subsection so the story is clear and ii) decrease the amount of numbers in the text. Some rewriting would improve clarity significantly.

Questions for authors
  1. Can the authors comment on other types of steering?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.