- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Safety feedback delivered mid-trajectory is often assumed to protect a jailbroken tool-using agent. This paper tests that assumption with a paired continuation design: agents are resumed independently from identical frozen pre-action checkpoints, once under a neutral reminder and once under safety feedback, so the multi-turn history is held exactly fixed. Across many parent tasks and eight open-weight agents, the authors find that identical feedback produces sharply model-dependent outcomes — rescuing some unsafe trajectories, leaving others unchanged, and inducing over-refusal on benign tasks. They then apply layer-wise activation patching, report that intervention effects concentrate near the final layers, show that patching at those layers changes the next tool call, and test whether features from those layers predict trajectory outcomes on held-out tasks.
Critical strengths
The paired continuation design is careful experimental work and it is what gives the behavioural results their force. Resuming independently from the same frozen checkpoint under two conditions, with deterministic replay, verified simulator copies, greedy decoding, and fixed budgets, isolates the effect of the feedback from everything else in a long multi-turn trajectory. That is harder to do well than it sounds, and the validation step — confirming on independent simulator instances that the legitimate workflow remains completable without the unauthorised action — rules out a confound that would otherwise make rescue and collateral loss impossible to distinguish from task infeasibility.
The three-way outcome taxonomy is the second strength, and I think it is the paper's most durable contribution. Including collateral loss on benign variants, alongside rescue and persistent unsafe, captures a trade-off that safety-only evaluation designs structurally cannot see: the same intervention that saves one trajectory breaks another. The resulting behavioural finding stands independently of everything mechanistic in the paper. Persistent-unsafe rates vary by nearly two orders of magnitude across agents and do not track parameter count, and rescue and collateral loss peak in different model families entirely. For anyone designing runtime safety feedback for agents, that result alone is worth knowing, and it is established by the behavioural half of the paper without relying on the patching analysis at all.
Critical weaknesses
The late-commit pattern is the paper's headline mechanistic claim, and the metric used to establish it is confounded in a way the paper anticipates but does not neutralise. Replacing a residual-stream state at layer ℓ and measuring cosine displacement of the resulting next-token logits will mechanically produce larger effects as ℓ approaches the output head, simply because fewer remaining layers can absorb the substitution. Seven of eight curves peaking at the final layer is exactly the profile a pure structural artifact would produce. The paper offers two rebuttals — vocabulary-space projections separating routing outcomes, and argument-level tool reversals that mid-depth controls fail to achieve — but both of these also benefit from proximity to the output, so neither isolates the effect from the confound. The fix is a control contrast rather than a new method. Run the identical patching sweep on a pair that differs only in surface wording and carries no safety content, such as two neutral reminders phrased differently. If that control produces the same late ramp, the pattern is structural and the claim should be reframed; if it stays flat while the safety contrast surges, the functional interpretation is earned and the paper becomes considerably stronger. Normalising the safety curve against that control, or patching at a fixed offset from the final layer to make depths comparable across architectures, would serve the same purpose.
The second concern is that the predictive results are reported by their maxima when the distribution does not support that summary. The three peak values quoted in the abstract each come from a different agent, and the per-agent table contains several cells at or below chance. More importantly, some estimates rest on very few positives — one rescue AUC is computed from four positive examples, and one persistent-unsafe AUC from three — and the paper notes this as high sample variance but then reports the peaks as the finding. This is readily fixable and I think the honest version is still publishable. Report confidence intervals in the main table, suppress or visually mark cells below a minimum positive count, and summarise with a median or aggregate across agents rather than a maximum. If the resulting statement is that intervention features carry modest predictive signal in some agents and none in others, that is a useful and credible baseline, and it is consistent with the paper's own framing of post-jailbreak feedback as model-dependent. As a smaller matter, the abstract and the first contribution bullet give different pair counts, and the phrase describing the curated set is ungrammatical; both are easy to correct.
Questions for authors
The question that would most change my assessment concerns the patching control. Do you have, or can you run, the layer-wise sweep on a non-safety contrast — two neutral reminders differing only in wording — on even two or three of the eight agents? If the late-layer surge is specific to the safety contrast, that result belongs in the main text, and it would substantially strengthen the mechanistic half of the paper.
I have two clarifications. First, could you report confidence intervals alongside the per-agent prediction AUCs, and state a minimum positive count below which an estimate should not be interpreted? I would like to know how many of the reported cells survive such a threshold. Second, the abstract reports one count of analysed pairs and the contributions section another; which is correct, and what was the curation criterion that removed the difference?
Gentle Reminder: Follow-up on Our Rebuttal
Dear Reviewer, as the discussion deadline approaches, we kindly invite you to review our previous response to your insightful comments. We hope our addtional experimental results and clarifications have fully addressed your concerns. Please let us know if any points remain unclear or require further discussion—we would be glad to provide additional details before the period closes. Thank you again for your time and constructive feedback!!