- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies internal representation dynamics in multi-turn jailbreaks and proposes Directional Context Drift (DCD) for early detection. DCD compares the hidden state of the current input with and without conversation history, learns attack-related drift directions from labeled malicious and benign conversations, and combines a Direction Bank, multi-scale EWMA, and conformal calibration for online detection. The paper reports strong detection and early-warning performance on known attacks, while also showing limitations under unseen attack patterns and shifts in benign conversational distributions.
Critical strengths
-
The paper studies the effect of conversation history on internal model representations, rather than relying only on text-level safety classification or refusal behavior. Its observation that multi-turn attack drift directions have low geometric overlap with established refusal and harmfulness directions offers a useful perspective on why single-turn safety representations may not capture multi-turn attack dynamics.
-
DCD estimates attack-relevant directions through a simple difference-in-means procedure without additional gradient-based training, and combines these directions with established sequential monitoring techniques. The framework is relatively interpretable, and the evaluation includes multiple attack datasets and analyses of context substitution, directional ablations, open-set detection, and adaptive attacks.
Critical weaknesses
-
DCD learns its drift directions from labeled attack samples, whereas many general-purpose guardrail baselines operate in a zero-shot manner. Thus, the performance gain may partly reflect supervised adaptation to known attack distributions rather than the independent benefit of contextual differencing. This concern is reinforced by Appendix Table 10, where Isolated/Contextual Readout variants can achieve performance comparable to or slightly better than Context Differential. The paper therefore does not fully isolate the value of the additional context-free forward pass and the differential representation under matched supervision and end-to-end detection settings.
-
The Direction Bank is inherently limited by the attack directions represented in its training data. Novel attacks may induce directions outside the bank, and adaptive white-box attackers may seek to evade known directions. Meanwhile, structure-matched benign dialogues and sensitive-topic conversations can also produce strong drift signals and increase false positives. Recalibrating thresholds may reduce false alarms but can lower attack recall. Consequently, maintaining both high recall and a low false-alarm rate in open-world settings remains insufficiently established.
Questions for authors
Q1. The paper constructs a Safety Plane from the Refusal Direction and Harmfulness Direction and uses low projection onto this plane to motivate its claim about multi-turn attacks. How representative is this selected subspace of the model’s full safety mechanism? Could the authors clarify whether the results support a broad mechanistic claim or only show low overlap with the particular directions selected? Q2. Given that Appendix Table 10 reports comparable performance for Isolated/Contextual Readout variants, could the authors compare Isolated Readout, Contextual Readout, and Context Differential under identical data splits, supervision budgets, layer selection, EWMA monitoring, and threshold calibration? Q3. How does DCD generalize to unseen attacks and shifts in benign conversational distributions? Could the authors evaluate direction-discovery or update mechanisms that do not require labeled examples of each new attack, and quantify how expanding the Direction Bank affects cumulative false positives, detection recall, and calibration?
4 comments
Dear Reviewer,
We wanted to kindly follow up on our rebuttal. With the discussion period ending tomorrow, we would appreciate any feedback and would be happy to clarify any remaining concerns.
Thank you for your time.
Official Comment by Reviewer ZeBB
Dear Authors,
Thank you for the clarifications and additional experiments. The matched comparisons and safety-subspace clarification address several of my concerns. I still have some questions about unseen-attack generalization, since examples from the held-out family are used for layer selection.
Given these remaining questions, I believe my original score remains appropriate, and I will retain it.
Thank you for the reply. We are glad that the matched comparisons and safety-subspace clarification address several of your concerns.
The missing-axis test evaluates coverage by existing directions. The layer is selected on separate conversations from the evaluated family, using only the remaining axes. No direction fitted on that family is used to score the held-out conversations in the missing-axis condition (Appendix H). Comparing the same conversations with and without the axis isolates what the family’s direction adds at the selected layer.
The bank-expansion experiment then tests how adding directions changes coverage and false alarms. Each axis keeps its previously selected layer, and the shared threshold is recalibrated on benign chat. Mean recall across all five families rises from 0.403 to 0.860 as their axes are added, while mean false alarms remain at 5.1–5.8% (Table 17).
These experiments characterize the coverage gap and evaluate how to address it. Alignment with the remaining bank is associated with detection without the family’s axis (r = −0.763, Table 15), and adding that axis raises target-averaged AUROC to 0.980–1.000 in the projection-based comparison. Separately, Appendix F shows that 40 attack conversations per axis bring detection before harm within seven percentage points of the full-corpus result for Crescendo and Echo Chamber across three targets.
The abstract explicitly identifies mode-specific coverage as a limit of DCD. Our experiments quantify that boundary and provide evidence about maintaining coverage once examples of a new mode become available. The matched detection comparisons and safety-subspace findings are established independently of the missing-axis experiment.
W1/Q2. Table 2 already includes a supervised LAD-based baseline. DCD leads on both AUROC and TPR in 21 of 25 target–family comparisons, and on AUROC in all ten twin comparisons (Appendix J). Both methods use labels, so the difference comes from what they read.
The Isolated/Contextual readouts are single-read variants of our monitor, sharing its estimator, chart and calibration. The new Table 3 compares all three readouts using the same data splits, supervision budgets, procedure for selecting layers, EWMA monitoring and threshold calibration (Appendix E).
With thresholds calibrated on WildChat conversations, DCD flags 24% of benign twins against 38–40%, and 74% of attacks against 77–84% (Table 3). These results show that the second read isolates context-induced drift and enables DCD to considerably lower the false positive rate. The differential is zero before history exists and the raw projections of the readouts already separate attack openers at turn one (0.72–0.78 AUROC, Table 12, Table 10 in the submission).
Q1. The plane combines two established safety-related directions for recognizing harmfulness and producing refusal, supported by causal interventions in (Zhao et al., 2025) and (Arditi et al., 2024). This provides a principled reference for asking whether representations associated with single-turn safety also capture multi-turn context drift.
Refusal can span multiple directions, which motivates extending the analysis beyond the two-direction plane (Wollschläger et al., 2025; Pan et al., 2025). We validate this reference using safety directions estimated from disjoint prompts and extend it to 48-dimensional subspaces built from harmful–harmless representation shifts. These subspaces capture the held-out safety directions well, while most attack-associated drift remains outside them. Subspaces derived independently from base-to-instruct changes support the same finding (Appendix G).
Our conclusion then extends beyond the particular two directions selected. It identifies a geometric gap between measured single-turn safety representations and multi-turn drift, without assuming that these subspaces describe the model’s entire safety mechanism.
W2/Q3. We explicitly evaluate coverage limits, as emphasized by (Sommer and Paxson). With the tested family's axis removed, the remaining directions outperform all three guards in 22 of 25 comparisons, averaging 0.862 AUROC against 0.763 for the strongest guard on the same conversations and five-turn budget. This projection-based test still uses examples of that family for layer selection.
We evaluate adaptive attackers with partial and full knowledge of the monitor (§5.6, Appendix K). At 32 rewrites per turn, DCD detects 76–87% of attacks on Llama under partial knowledge. On Mistral, none of these tested attackers improves joint success over the original attacks (Table 6). Full white-box access enables stronger evasion which we quantify across query budgets in Appendix K, but DCD still performs attack detection for at least half of the attacks.
Appendix H (Table 17) tests growth from one to five axes, recalibrating the shared threshold after each addition. Mean false alarms stay at 5.1–5.8%, while recall across all five families rises from 40% to 86%. We also report recall losses on previously covered families.
Tables 3–4 report benign-twin and sensitive-topic WildChat false alarms alongside attack recall. Table 4 includes all three guards and shows the recall tradeoff before and after recalibration.
The present method learns directions from labeled examples. Reliable automatic discovery and validation of new directions from unlabeled traffic remain future work.