acceptodds
Under review as a conference paper at ICLR 2027

Causal Control of Jailbreak Behavior at the Assistant-Generation Boundary

Abstract

Safety-aligned LLMs can refuse harmful requests yet comply when the same intent is wrapped in jailbreak prompts. The internal point at which this behavioral switch is established remains unclear. We investigate the assistant-generation boundary, the final prompt-side state before decoding, as a causal control region. Using matched pairs of direct and jailbreak-wrapped prompts, query-group-disjoint counterfactual prediction, one-state patching, and full-generation evaluation across three open-weight models, we localize a model-dependent late boundary band that controls jailbreak behavior. Replacing a single wrapped state with a predicted direct-like state reduces ASR from 60.7% to 23.6%. Prefix counterfactuals recover 83.9-100.0% of the full-generation effect within eight tokens, indicating that most downstream change is mediated by the early generated trajectory. Reverse and random controls establish directionality; specificity tests find no reliable need for exact query matching. These results identify a model-dependent causal control region near the assistant-generation boundary, linking late prompt-side computation to the early autoregressive trajectory through which jailbreak behavior is expressed.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Large Language Model Safety, Jailbreak Attacks, Assistant-Generation Boundary, Activation Patching, Mechanistic Interpretability

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_RuBJOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether an activation at the assistant-generation boundary causally affects jailbreak behavior. Using paired direct and jailbreak-wrapped prompts, the authors predict a direct-like activation, patch a single state in the wrapped run, and evaluate the resulting generation. Across three models, the intervention reduces jailbreak success rate, with much of the effect emerging in the first few generated tokens. The results support a model-dependent, late boundary region as an intervention-sensitive control point rather than a deployable defense.

Critical strengths
  • Patching a single assistant-boundary state changes complete jailbreak responses in three models, providing direct intervention evidence rather than relying on activation probes alone.
  • Held-out spatial tests and reverse and random controls localize the effect to a late boundary region and distinguish the intervention from a generic perturbation.
  • Forced-prefix experiments show that the first few generated tokens carry much of the intervention's effect, which links the prompt-side state to the subsequent response.
Critical weaknesses
  • The paper should state the exact hook point within a Transformer block, whether all later layers at the patched prompt position are recomputed, and how the per-layer KV cache is constructed after the patch.
  • The experiments rely on greedy decoding and fixed prompt-based jailbreak settings. It remains unclear whether the boundary intervention changes the distribution of outcomes under temperature sampling or in multi-turn jailbreak interactions.
  • In the separate donor-specificity evaluation (Figure 5), mismatched direct, mean-direct, and shuffled-ridge donors are not reliably worse than the predicted donor. This limits the evidence that query-adaptive prediction adds value beyond a boundary-region intervention.
  • The intervention has collateral effects: it increases benign refusals for Qwen and Llama, and on Gemma OR-Bench it shifts some safe answers into harmful compliance despite little change in refusal rate. These results limit any framing of the patch as a practical safety repair or runtime defense.
Questions for authors
  • At what point is the residual state overwritten, and are downstream activations and KV cache entries computed from the patched state?
  • Do the ASR reductions and early-prefix effects persist under a non-greedy decoding setting?
  • How much of the effect requires a query-specific predicted donor rather than a mismatched or shuffled-ridge donor?
  • Figure 2 peaks at layers different from the validation-selected coordinates. How sensitive is the primary ASR reduction to nearby layers under the same evaluation protocol?
3 comments

Thank you for the detailed feedback and constructive questions.

To W1/Q1: We have clarified the exact hook location and KV cache handling in Section 3.3 and Appendix A. The intervention hook is attached to model.model.layers[L] and fires once during prompt prefill at the final prompt token (p=T−1p=T-1), overwriting the post-block residual state corresponding to hidden_states[L+1]. The patched block's attention K/V, computed before the overwrite, is retained, while all upper-layer hidden states and their corresponding prompt-position K/V entries are recomputed from the patched residual. To further validate this implementation, we added a cache/no-cache consistency experiment on the same 70 held-out query–family pairs per model. Both execution paths produce identical ASRs and identical binary jailbreak labels for all paired examples (70/70 agreement) across all three models (Appendix C, Table 7). These results confirm that the cache-based implementation preserves the evaluated intervention outcomes at both the ASR and jailbreak-label levels.

To W2/Q2: We have added non-greedy sampling experiments on all three models using temperatures of 0.7, 0.9, and 1.1 with top-p=0.9p=0.9, averaging three repeated seeds per query–family pair. The predicted-donor intervention consistently reduces ASR across all tested settings, with reductions ranging from 11.4 to 15.7 percentage points for Qwen, 35.7 to 38.6 for Llama, and 27.1 to 34.3 for Gemma. We further extended the prefix mediation analysis to these sampling settings. All 36 model–temperature–prefix combinations retain positive total and prefix effects, with only small residual effects. Regarding multi-turn jailbreak interactions, our study focuses on static single-turn prompts, and we have explicitly acknowledged the lack of multi-turn evaluation as a limitation of the current work (Section 4.3, Appendix C, Tables 4-5, and Section 5).

To W3/Q3: Our specificity experiments reveal that the intervention effect does not require exact query matching. Although mismatched, mean-direct, and shuffled-ridge donors achieve comparable ASR reductions, this suggests that the causal control region may contain shared refusal-policy-related components across queries, rather than relying on a uniquely query-specific representation. Importantly, the predicted donor provides a query-conditioned direct-like estimate using only the wrapped execution, whereas the same-query direct donor is an oracle requiring a separate direct execution. Moreover, predicted-donor interventions remain significantly different from reverse and norm-matched random controls across all three models (p≤1.2×10−4p\leq 1.2\times10^{-4}), confirming that the effect is directional rather than a generic perturbation. Thus, our contribution is not to establish the superiority of a particular donor estimator, but to identify and validate an intervention-sensitive boundary region that can be accessed through multiple donor constructions (Section 4.5 and Appendix F, Figure 5 and Table 8).

To W4: We agree that the collateral effects limit the practical safety benefits of the current intervention. We have clarified that our intervention is applied unconditionally to all inputs, without distinguishing harmful from benign requests, and is designed to investigate the causal role of the boundary state rather than serve as a runtime safety defense. We retain these collateral effects as important limitations and explicitly state that developing selective interventions that reduce harmful compliance while preserving benign utility remains an important direction for future work (Section 5 and Appendix J, Figure 13).

To Q4: We have added an adjacent-layer sensitivity experiment under the same evaluation protocol as Table 1. Specifically, we evaluated the four neighboring layers around each validation-selected coordinate, using the same 140 held-out query-family pairs, 256-token greedy decoding, and deterministic judge. The selected-layer results exactly reproduce Table 1, while all twelve neighboring layers retain negative Δ\DeltaASR, with reductions of 12.9-13.6 percentage points for Qwen, 34.3-40.7 for Llama, and 32.9-37.1 for Gemma. All corresponding 95% query-group bootstrap confidence intervals are below zero. These results demonstrate that the intervention effect is locally stable within a model-dependent boundary region, rather than relying on a single precisely selected layer (Section 4.1 and Appendix B, Table 3).

To Ethics Concern: We have added an AI Use Statement and an Ethics Statement acknowledging the dual-use risks of reverse-direction interventions. We also clarified that these interventions serve solely as diagnostic controls and that released materials will avoid providing turnkey jailbreak tools or precomputed artifacts intended to induce harmful compliance.

We believe your comments will help improve this work. Should you have any remaining concerns, we welcome further discussion.

Thank you for the detailed responses. My main concerns have been adequately addressed. Multi-turn generalization and collateral effects remain limitations, but they do not undermine the paper's contribution. I have raised my score from 2 to 3.

We sincerely appreciate your recognition and support. Your valuable suggestions have significantly strengthened our paper, and we hope our findings will contribute to future research in this area.

Reviewer_22obOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether a model's final prompt-side hidden state before generation causally influences jailbreak behavior. Across three open-weight LLMs, a single-state intervention reduces jailbreak success, and prefix counterfactuals show that most of the effect is carried by the first eight generated tokens.

Critical strengths
  • Strong causal intervention design: The paper changes only one hidden state while keeping the prompt, model parameters, and decoding procedure fixed, with reverse and random controls supporting a directional effect.

  • Generation-level and trajectory analysis: The authors evaluate complete responses and show that short generated prefixes account for most of the intervention effect, providing evidence beyond activation probing or first-token correlations.

Critical weaknesses
  • Limited generalizability: Experiments cover only three open-weight models under greedy decoding, leaving it unclear whether the findings extend to other models and sampling strategies.
  • Unclear practical safety benefits: The intervention can increase refusals on benign requests and cause safety regressions, so the identified control region is not yet demonstrated to be a reliable, deployable defense.
Questions for authors

Please see Critical Weaknesses.

3 comments

Thank you for the positive assessment and helpful suggestions.

To W1: We have extended our evaluation to three additional instruction-tuned models: Phi-4-mini-Instruct, Granite-3.1-8B-Instruct, and Aya-Expanse-8B. Although the main study covers seven attack families, extending the full attack suite to additional models would require substantial computational resources and evaluation time. We therefore selected GCG and PAIR as representative white-box and black-box attacks, respectively, and conducted additional attack generation and evaluation to construct held-out datasets for all three models. The predicted-donor intervention reduces ASR across all six model–attack combinations, with all 95% confidence intervals excluding zero. We have also evaluated non-greedy decoding on the original three models at temperatures of 0.7, 0.9, and 1.1 with top-p=0.9p=0.9. The intervention consistently reduces ASR across all tested settings, and the prefix effects remain positive across all 36 model–temperature–prefix combinations. These results provide additional evidence that the observed effects extend beyond the original models and greedy decoding (Appendix C and H, Tables 4, 5, and 13).

To W2: We agree that the current intervention should not be interpreted as a deployable safety defense. In our experiments, the predicted-donor intervention is applied unconditionally to all inputs, without distinguishing harmful requests from benign ones. This design serves to isolate the causal influence of boundary-state modification, rather than to implement a selective safety mechanism. We have clarified this distinction in the revised manuscript and explicitly limited our claims to mechanistic findings. Developing selective intervention strategies that reduce harmful compliance while preserving benign utility is an important direction for future work. We believe that understanding the internal causal mechanisms of jailbreak behavior provides a useful foundation for developing more effective and better-targeted safety interventions.

We welcome further discussion if any concerns remain.

Thanks for the detailed explanation.

Thank you for your feedback and for supporting our work and rebuttal.

Reviewer_9TJbOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the causal role of hidden states near the assistant-generation boundary in LLM jailbreak behavior.

Critical strengths
  1. The paper investigates an interesting and important problem by examining how internal hidden states influence jailbreak behavior in LLMs.
  2. The experimental evaluation is relatively comprehensive, covering three LLMs and multiple intervention controls.
Critical weaknesses
  1. The paper is overall clearly written. Nevertheless, there are some mathematical notations that are not properly defined at their first occurrence. For example, in Eq. (2), it is not clear what do(⋅←⋅)do(\cdot \leftarrow \cdot) means, and d^ij\hat{d}_{ij} is not defined until later sections. I suggest the authors define mathematical notions upon their first occurrence to improve readability and mathematical rigor.

  2. The paper highlights causality as the core contribution. However, the causal interpretation of the results is not clear. The intervention experiments indeed show that modifying a selected hidden state can influence the model's output, but this does not necessarily establish that jailbreak behavior is causally mediated by this state.

  3. The representativeness of the main experimental results requires further justification. It seems, by comparing Table 1 and Appendix E, that the test results change significantly based on the dataset. For example, the ASR changes from 70.7% to 56.4% for Qwen in Table 1 and from 73.2% to 24.3% in Appendix E. Given these significant differences, it is unclear how stable and representative the experimental findings are.

Questions for authors

See weakness.

3 comments

Thank you for the constructive feedback.

To W1: We have revised Section 3.1 to define mathematical notations at their first occurrence. In particular, we clarify the meaning of do(hℓ,p←z)do(h_{\ell,p}\leftarrow z), the predicted donor d^ij\hat{d}_{ij}, and the query and wrapper indices (Section 3.1, Lines 159-175).

To W2: We have added a bidirectional state-crossing experiment on the same 20 held-out query groups. Replacing the wrapped boundary state with the same-query direct state reduces ASR by 15.0, 32.9, and 37.9 percentage points for Qwen, Llama, and Gemma, respectively, with all three effects statistically significant after Holm correction. Conversely, transferring the wrapped state into the direct execution increases ASR by 4.3, 16.4, and 1.4 points, with a significant increase for Llama. Mismatched-query state transfers produce comparable reductions to same-query transfers, further supporting the role of a shared boundary-state region. These results provide stronger bidirectional intervention evidence that the boundary state causally influences jailbreak behavior (Section 4.2 and Appendix F, Tables 9-10).

To W3: We identified the source of the discrepancy through additional consistency checks. First, we repeated the neighboring-layer experiments using the same held-out data and evaluation configuration as Table 1, successfully reproducing the original results at the selected coordinates. However, when applying the same configuration to the extended evaluation dataset, the results differed from those in the original Appendix E. We therefore carefully inspected the previous configuration files and execution logs, identifying inconsistencies in the model inference and evaluation parameters. After correcting these inconsistencies, we reran the extended evaluation on all three models. The updated results in Appendix G show ASR reductions of 23.2, 41.4, and 34.3 percentage points for Qwen, Llama, and Gemma, respectively, comparable to the main results, with all 95% confidence intervals excluding zero. These consistency checks and updated results provide stronger evidence for the stability and representativeness of our findings (Appendix B and G, Tables 3 and 12).

Please let us know if any concerns remain unaddressed; we are happy to discuss them.

Thanks for the explanation and revision. I have increased my score.

On behalf of all authors, we are truly grateful for your positive assessment and support of our work. We sincerely value your recognition and thoughtful feedback.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Brief Introduction to the Revised Manuscript

We sincerely thank all reviewers for their constructive feedback and valuable suggestions.

We have carefully revised the manuscript and conducted additional experiments to address the reviewers' concerns. The major revisions and additions include:

  1. Causal validation: Added bidirectional state-crossing experiments to further examine the causal contribution of the boundary state.

  2. Implementation and robustness: Clarified the intervention hook and KV cache handling, and added cache-consistency and adjacent-layer sensitivity experiments.

  3. Generalization: Extended evaluations to three additional instruction-tuned models and added non-greedy sampling experiments under different temperatures, including prefix-effect analysis.

  4. Experimental consistency: Identified inconsistencies in the original extended evaluation settings and reran the experiments using configurations consistent with the main experiments.

  5. Discussion and limitations: Further clarified donor specificity, collateral effects, the scope of the study, and directions for future work.

All major modifications in the revised manuscript are highlighted in blue for ease of reference. Detailed point-by-point responses to all reviewer comments are provided in the rebuttal.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.