- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates whether an activation at the assistant-generation boundary causally affects jailbreak behavior. Using paired direct and jailbreak-wrapped prompts, the authors predict a direct-like activation, patch a single state in the wrapped run, and evaluate the resulting generation. Across three models, the intervention reduces jailbreak success rate, with much of the effect emerging in the first few generated tokens. The results support a model-dependent, late boundary region as an intervention-sensitive control point rather than a deployable defense.
Critical strengths
- Patching a single assistant-boundary state changes complete jailbreak responses in three models, providing direct intervention evidence rather than relying on activation probes alone.
- Held-out spatial tests and reverse and random controls localize the effect to a late boundary region and distinguish the intervention from a generic perturbation.
- Forced-prefix experiments show that the first few generated tokens carry much of the intervention's effect, which links the prompt-side state to the subsequent response.
Critical weaknesses
- The paper should state the exact hook point within a Transformer block, whether all later layers at the patched prompt position are recomputed, and how the per-layer KV cache is constructed after the patch.
- The experiments rely on greedy decoding and fixed prompt-based jailbreak settings. It remains unclear whether the boundary intervention changes the distribution of outcomes under temperature sampling or in multi-turn jailbreak interactions.
- In the separate donor-specificity evaluation (Figure 5), mismatched direct, mean-direct, and shuffled-ridge donors are not reliably worse than the predicted donor. This limits the evidence that query-adaptive prediction adds value beyond a boundary-region intervention.
- The intervention has collateral effects: it increases benign refusals for Qwen and Llama, and on Gemma OR-Bench it shifts some safe answers into harmful compliance despite little change in refusal rate. These results limit any framing of the patch as a practical safety repair or runtime defense.
Questions for authors
- At what point is the residual state overwritten, and are downstream activations and KV cache entries computed from the patched state?
- Do the ASR reductions and early-prefix effects persist under a non-greedy decoding setting?
- How much of the effect requires a query-specific predicted donor rather than a mismatched or shuffled-ridge donor?
- Figure 2 peaks at layers different from the validation-selected coordinates. How sensitive is the primary ASR reduction to nearby layers under the same evaluation protocol?
3 comments
Thank you for the detailed responses. My main concerns have been adequately addressed. Multi-turn generalization and collateral effects remain limitations, but they do not undermine the paper's contribution. I have raised my score from 2 to 3.
We sincerely appreciate your recognition and support. Your valuable suggestions have significantly strengthened our paper, and we hope our findings will contribute to future research in this area.
Thank you for the detailed feedback and constructive questions.
To W1/Q1: We have clarified the exact hook location and KV cache handling in Section 3.3 and Appendix A. The intervention hook is attached to model.model.layers[L] and fires once during prompt prefill at the final prompt token (p=T−1), overwriting the post-block residual state corresponding to hidden_states[L+1]. The patched block's attention K/V, computed before the overwrite, is retained, while all upper-layer hidden states and their corresponding prompt-position K/V entries are recomputed from the patched residual. To further validate this implementation, we added a cache/no-cache consistency experiment on the same 70 held-out query–family pairs per model. Both execution paths produce identical ASRs and identical binary jailbreak labels for all paired examples (70/70 agreement) across all three models (Appendix C, Table 7). These results confirm that the cache-based implementation preserves the evaluated intervention outcomes at both the ASR and jailbreak-label levels.
To W2/Q2: We have added non-greedy sampling experiments on all three models using temperatures of 0.7, 0.9, and 1.1 with top-p=0.9, averaging three repeated seeds per query–family pair. The predicted-donor intervention consistently reduces ASR across all tested settings, with reductions ranging from 11.4 to 15.7 percentage points for Qwen, 35.7 to 38.6 for Llama, and 27.1 to 34.3 for Gemma. We further extended the prefix mediation analysis to these sampling settings. All 36 model–temperature–prefix combinations retain positive total and prefix effects, with only small residual effects. Regarding multi-turn jailbreak interactions, our study focuses on static single-turn prompts, and we have explicitly acknowledged the lack of multi-turn evaluation as a limitation of the current work (Section 4.3, Appendix C, Tables 4-5, and Section 5).
To W3/Q3: Our specificity experiments reveal that the intervention effect does not require exact query matching. Although mismatched, mean-direct, and shuffled-ridge donors achieve comparable ASR reductions, this suggests that the causal control region may contain shared refusal-policy-related components across queries, rather than relying on a uniquely query-specific representation. Importantly, the predicted donor provides a query-conditioned direct-like estimate using only the wrapped execution, whereas the same-query direct donor is an oracle requiring a separate direct execution. Moreover, predicted-donor interventions remain significantly different from reverse and norm-matched random controls across all three models (p≤1.2×10−4), confirming that the effect is directional rather than a generic perturbation. Thus, our contribution is not to establish the superiority of a particular donor estimator, but to identify and validate an intervention-sensitive boundary region that can be accessed through multiple donor constructions (Section 4.5 and Appendix F, Figure 5 and Table 8).
To W4: We agree that the collateral effects limit the practical safety benefits of the current intervention. We have clarified that our intervention is applied unconditionally to all inputs, without distinguishing harmful from benign requests, and is designed to investigate the causal role of the boundary state rather than serve as a runtime safety defense. We retain these collateral effects as important limitations and explicitly state that developing selective interventions that reduce harmful compliance while preserving benign utility remains an important direction for future work (Section 5 and Appendix J, Figure 13).
To Q4: We have added an adjacent-layer sensitivity experiment under the same evaluation protocol as Table 1. Specifically, we evaluated the four neighboring layers around each validation-selected coordinate, using the same 140 held-out query-family pairs, 256-token greedy decoding, and deterministic judge. The selected-layer results exactly reproduce Table 1, while all twelve neighboring layers retain negative ΔASR, with reductions of 12.9-13.6 percentage points for Qwen, 34.3-40.7 for Llama, and 32.9-37.1 for Gemma. All corresponding 95% query-group bootstrap confidence intervals are below zero. These results demonstrate that the intervention effect is locally stable within a model-dependent boundary region, rather than relying on a single precisely selected layer (Section 4.1 and Appendix B, Table 3).
To Ethics Concern: We have added an AI Use Statement and an Ethics Statement acknowledging the dual-use risks of reverse-direction interventions. We also clarified that these interventions serve solely as diagnostic controls and that released materials will avoid providing turnkey jailbreak tools or precomputed artifacts intended to induce harmful compliance.
We believe your comments will help improve this work. Should you have any remaining concerns, we welcome further discussion.