- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies whether safety refusal behavior is preserved when looped language models perform substantially more recurrent computation than their training depth. The paper makes an important distinction between effective refusal and generation degeneration, arguing that low attack success at large recurrent depths can be misleading when the model simply stops producing usable outputs. Through experiments on Ouro and several dense-to-loop recurrent models, the authors show that a behaviorally validated refusal direction can be rapidly reconstructed after transient erasure, while its projection nevertheless drifts substantially during ordinary inference. Based on this observation, the paper proposes SafeLoop, an inference-time intervention that caches each input's refusal-direction projection at a reference loop and repeatedly corrects later hidden states toward that reference. The method substantially improves effective refusal in some settings, e.g., from 9.5% to 41.0% at 16 loops and from 6.5% to 35.5% at 24 loops for Ouro-1.4B.
Critical strengths
-
The paper identifies an new, open question in AI safety. The author finds looped LLMs may behave differently from traditional LLMs in refusals. The experiments demonstrate that very low attack success at large recurrent depths can coexist with severe generation degeneration. This makes a compelling case that conventional jailbreak success rates alone can substantially overestimate safety at extrapolated recurrent LLMs.
-
The paper's writing is clear and easy to understand. The author uses figures, examples, and comprehensive expreiments to structure their paper.
Critical weaknesses
- The effectiveness of SafeLoop is not convincing enough, as 1) it's substantially model dependent, and 2) its impact on helpfulness isn't negliable.
The method works particularly well for Ouro-1.4B, but gives little or negative benefit for Ouro-2.6B; for example, effective refusal decreases by 2.0 and 6.2 percentage points at depths 16 and 24. Likewise, SafeLoop only marginally improves effective refusal for highly degenerate Gemma at extrapolated depth. Since looped LLM is a relatively new direction and there're not many open-source models to test on, the generalizability of the method is questionable.
In addition, a good safeguard approach should not only prevent harmful generations but also preserve the model's ability to respond appropriately to benign requests. The reported increase in over-refusal on XSTest-safe and the changes on MMLU already suggest that SafeLoop may trade off safety for helpfulness. And since the reported tasks are rather simple, SafeLoop's impact on more complex tasks are still unknown.
- The evidence that refusal-projection preservation is the causal mechanism behind the behavioral gains is not yet sufficient.
SafeLoop directly controls only one coordinate of the hidden state, and the paper itself reports that a random-axis control can also reduce deep Ouro degeneration. This makes it difficult to attribute the behavioral improvements specifically to preservation of the refusal direction rather than to a more general stabilization effect caused by repeatedly constraining the hidden-state dynamics. The authors acknowledge this issue, but the current experiments do not fully disentangle these explanations. A stronger set of controls could include multiple behaviorally validated directions, matched random or semantic directions, or interventions that preserve the same amount of hidden-state displacement without explicitly preserving the refusal coordinate.
Questions for authors
-
How sensitive is SafeLoop to the choice of reference loop and correction strength (beta)?
-
What is the computational and memory overhead of token-wise persistent correction during generation? For fast inference engines like vLLM and sglang, can SafeLoop adapt to them easily?