- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies whether activation steering can be detected from a single layer’s logged activations, given trusted model weights but no original prompt, steering vector, or injection location. It builds on token inversion to reconstruct candidate inputs, then scores reconstruction residuals using clean calibration conditioned on layer and token role. Its main contribution is a calibrated reachability audit and evidence that interventions can remain detectable downstream even when missed at the injection layer. Validation covers five open models, PGD and random perturbations, clean controls, comparisons with recomputation and activation-based baselines, and detector-aware attacks. The final-layer audit detects all 46 counted attack-induced jailbreak outcomes in the main evaluation; constrained adaptive attacks produce no induced jailbreaks in the tested settings. These findings establish empirical feasibility within the evaluated attack family.
Critical strengths
I thank the authors for studying an interesting auditing problem and for clearly documenting the method’s limitations.
- Well-motivated setting: Detecting activation interventions from partial logs addresses a useful problem when the original prompt and injection boundary are unavailable.
- Concrete detector: The method combines token inversion with clean calibration that accounts for numerical error and structural token roles, without requiring labeled attacks.
- Interesting empirical finding: The evaluation across five models shows that downstream layers can reveal interventions missed at the injection layer, including attacks that induce judged jailbreaks.
- Useful controls: Clean and random controls, paired behavioral comparisons, and detector-aware attacks with successful unconstrained references support interpretation of the results.
Critical weaknesses
My main concerns are the scope of the evaluation and whether it sufficiently supports the method’s robustness and comparative advantages.
- Narrow attack coverage: The calibrated audit is evaluated on PGD interventions at the last prompt position and roughly one injection depth per model. The contrastive-steering experiments supply the true prefix and use a different detector, so they do not establish the same prompt-free capability.
- Incomplete adaptive-attack validation: The unsuccessful adaptive attacks provide useful evidence, but the precise single-layer surrogate and optimization checks need clarification. Agreement between surrogate and measured scores alone does not establish gradient agreement or rule out optimization difficulties.
- Limited reliability estimates: Four models have only 15 test prompts, leaving substantial uncertainty around detection and false-positive rates. Behavioral evidence also includes repeated prompts and noisy judge labels. The retrospective stopping checks are reassuring but do not replace a fresh, end-to-end prompt-free evaluation.
- Imperfectly matched baselines: The strongest inversion results use the final layer, whereas the linear probe uses the injection layer and a different false-positive operating point. This limits conclusions about comparative superiority. Density baselines are evaluated only on Qwen-0.5B.
Questions for authors
Clarifying the following two points could affect my assessment:
- Adaptive attacks: Please specify the exact surrogate and optimization procedure used for the single-layer adaptive attacks. What checks support that the absence of induced jailbreaks reflects the detector’s constraint rather than ineffective optimization?
- Generality beyond PGD: Since Appendix B supplies the true prefix and uses a different detector, could you provide a small evaluation of behavior-changing contrastive steering using the same calibrated, prompt-free pipeline as the main experiments?