- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper repeatedly updates its estimate of refusal behavior and selects interventions subject to a constraint on benign behavior. Its strongest contribution is the empirical demonstration that iterative adaptation exposes vulnerabilities missed by several existing steering methods.
Critical strengths
- Table 1 reports attack success increasing from the strongest baseline’s 43% to 96% on Qwen2.5-32B, and from 65% to 85% on Qwen3.8-27B. The comparisons include approaches targeting multiple refusal directions.
- idea is novel
Critical weaknesses
- I feel the results make a much stronger case for iterative adaptation than they do for true mechanistic nonstationarity. From my perspective, seeing shifting directions and nearly orthogonal interventions from step to step could just as easily be explained by a static, multidimensional refusal representation, especially since earlier interventions can naturally warp the geometry we measure later on. I would find it much more convincing if the authors compared their re-estimation approach against a fixed multi-directional representation found at the very start, keeping the number of interventions, selection criteria, and compute budget identical.
- I find the constraint meant to preserve benign behavior pretty narrow. Because it only restricts the KL divergence on the very first generated token, I'm not convinced that full responses or multi-step reasoning stay intact. I see this playing out in Table 1: on Mistral, MMLU drops from 57% down to 48%, and ARC-E falls from 69% to 53%. Rather than showing that capabilities are preserved across the board, I think the results really just show a favorable trade-off between attack success and utility on certain models.
- In my opinion, the capability evaluations are too limited in scope. Running two 500-question multiple-choice benchmarks is a reasonable starting point, but it doesn't tell me whether coding, open-ended instruction following, long-form writing, or complex reasoning are holding up. Along the same lines, I don't think doing multiple generation runs is a substitute for testing across different optimization seeds and genuinely held-out harmful prompt distributions.
Questions for authors
see weakness
5 comments
Rebuttal 4s3e22 (PT2)
I find the constraint meant to preserve benign behavior pretty narrow. Because it only restricts the KL divergence on the very first generated token, I'm not convinced that full responses or multi-step reasoning stay intact. I see this playing out in Table 1: on Mistral, MMLU drops from 57% down to 48%, and ARC-E falls from 69% to 53%. Rather than showing that capabilities are preserved across the board, I think the results really just show a favorable trade-off between attack success and utility on certain models.
Authors’ response. (UPDATED) We thank the reviewer for this fair reading. We wish to clarify that the first-token KL is a computationally efficient criterion for limiting behavioral deviation during intervention selection, rather than a guarantee on complete responses or multi-step reasoning. Our utility assessment therefore relies on independent evaluations: MMLU and ARC-E and, in the revised manuscript, IFEval, which grades complete generated responses with reasoning enabled for reasoning models (Appendix A.6, Table 6). The ASR–utility trade-offs are reported in Appendix A.2. On seven of the eleven models (Qwen2.5-3B, Gemma-2-9B, Gemma-3-12B, Llama-2-13B, Qwen2.5-32B, Qwen3.8-27B, and GPT-oss-20B), substantial ASR improvements coexist with decreases of no more than 0.04 on either MMLU or ARC-E relative to the original models. On IFEval, our method stays within 0.06 of the original model’s accuracy on ten of the eleven models.
As discussed in Section 5, Mistral-7B exhibits a more pronounced trade-off: ASR increases from at most 0.04 among the baselines to 0.42, while MMLU and ARC-E decrease by 0.09 and 0.16, respectively, and IFEval decreases from 0.46 to 0.23. We note that the evaluated model incorporates Representation Rerouting, the circuit-breaking approach of Zou et al. (2024), which directly modifies representations involved in harmful generation. This provides a possible explanation for why suppressing its safeguards may affect capabilities more substantially, although establishing this connection causally requires further investigation.
Regarding the reviewer’s concern about the ASR–utility trade-off, we agree that this may arise for some models. In this regard, Figure 4 in Appendix A.2 traces validation ASR and utility across iterations on Mistral-7B. Validation ASR remains at 0.00 for the first 17 iterations and then rises gradually, reaching its maximum of about 0.59 at iteration 95, the selected checkpoint. Over the same period, MMLU decreases from about 0.60 to 0.50 and ARC-E from about 0.67 to 0.53, with most of the ARC-E decline occurring within the first 45 iterations. Earlier checkpoints therefore offer different ASR–utility trade-offs. Selecting the checkpoint with the highest validation ASR is one possible choice; applications that prioritize utility could favor an earlier one. We have revised the manuscript to state that our method substantially improves attack effectiveness while retaining utility on several evaluated models, rather than claiming uniform capability preservation.
Lastly, please note that since our interventions are implemented through hooks without modifying the underlying weights, they can also be removed when needed to recover the original model behavior. We have clarified that the benign-behavior constraint is intended to discourage degenerate solutions in which refusal disappears because generation is broadly disrupted.
Thanks for the response. I would like to keep my score.
Rebuttal 4s3e22 (PT1)
We thank the reviewer for recognizing the novelty of our approach and the substantial improvements in attack success over strong baselines, including methods targeting multiple refusal directions. We provide below our point-by-point responses to the concerns and questions raised by the reviewer.
Author's Response (UPDATED) We also wish to clarify that our evaluation already includes multidirectional approaches, namely RCO and SOM, and that our method achieves the highest ASR on ten of eleven models across the evaluated methods. Furthermore, Table 9 documents changes in refusal expression, while Tables 2 and 7 support the benefit of updating the refusal vocabulary and filtering the estimation data as steering progresses.
To examine whether our gains simply reflect a larger number of ablations, we have prepared a plot of ASR against the number of applied ablations across all evaluated models. We provide a summary table below and include the complete figure, showing the trajectory of our method alongside all evaluated baseline configurations, in Appendix A.3, Figure 5. Values come from a single greedy generation, so they can differ slightly from the five-run means in Table 1 and 3.
The results indicate that, on several models, our method achieves higher ASR despite applying fewer ablations. These results indicate that the number of ablations alone does not explain the observed advantage.
Regarding the proposed fixed-representation comparison, matching the number of orthogonal directions additionally requires specifying how those directions are extracted from the original model, selected, and assigned to layers. In our approach, their successive identification is itself part of the method. After each intervention, we estimate the refusal behaviour that remains and use it to identify the next layer-specific direction. Our existing comparisons evaluate the resulting method, while a matched fixed-representation control would further isolate the contribution of re-estimation.
The interpretation suggested by the reviewer also helps refine our mechanistic analysis. Near-zero cosine similarity is expected under directional projection, but the informative observation is that further interventions at the same layer remain useful after a single direction has been removed. This supports a multidimensional account of refusal and complements our analysis of the intervention sequence in Figure 2.
We revised the mechanistic interpretability analysis in our paper to incorporate this complementary interpretation and clarify the operational meaning of non-stationarity. We thank the reviewer for helping us strengthen this discussion and hope that these clarifications and additional results encourage a more positive reassessment of our contribution.