- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper replays multi-turn jailbreaks to ensure that the model conditions on its own earlier replies, and it uses preregistered branch interventions on a single open-weight model with outcomes labelled by a panel of LLM judges to show that compliance depends on the user turn just before onset, on the full conversation history, and on the prior output of the model itself. Attempts to locate an internal mechanism using probes and sparse autoencoders did not succeed.
Critical strengths
The paper replays multi-turn attacks so that the model conditions on its own earlier replies, and it builds a clean branch-intervention protocol on this foundation. The protocol uses equal-token replacements, matched controls, and a sham branch that must reproduce the original run. This protocol is the most reusable contribution of the work.
Critical weaknesses
The paper is only tested on the Gemma 9B model and on the development split only. The paper also reveals that LLM-panel labels could not be reproduced by a second panel (κ = 0.0229), which makes it more unreliable. I think the results could not be shown to extend beyond this model and the dataset.
These onsets were selected precisely because they were labelled compliant. Any change to the conversation, even a change that preserves the content, is therefore likely to eliminate that rare compliance, whereas non-compliant controls rarely change in the opposite direction. Reordering the pre-onset turn alone yields a rate of 0.409 compared with 0.500 for the main condition, and Section 9 does not attempt a change that preserves the content. The experiments therefore cannot establish whether the content matters or whether rare compliant outputs are simply fragile, which means that the headline effects cannot yet be interpreted as causal.
Also the paper has a clear trace of using AI autoresearch.
Questions for authors
Can you extend to more models?
Can you try a different dataset?
Can you add a content-preserving control, such as paraphrasing the pre-onset turn or the model's reply?
How often does an onset stay compliant if you regenerate it with a different seed and no intervention?
Which public multi-turn attack corpus did you use?