acceptodds
Under review as a conference paper at ICLR 2027

Safe at One Loop, Risky at Another: Aligning Safety Across Recurrent Depths in Looped Language Models

Abstract

Looped Language Models (LoopLMs) provide a parameter efficient approach to scaling model capabilities through repeated use of shared parameters across recurrent steps. Since each recurrent depth can be read out independently, a single LoopLM exposes a broader output space across inference depths, raising an important question: whether safety is preserved throughout recurrent computation. Prior evaluations suggest that deeper recurrence can improve safety on harmful queries, but robustness under jailbreak attacks remains unclear. We therefore conduct a comprehensive safety evaluation of LoopLMs under jailbreak attacks targeting different recurrent depths. We find that attack success can increase at deeper inference depths, the same query can elicit different safety behaviors across depths, and attacks constructed against one depth can transfer to others. Moreover, SFT and preference alignment do not eliminate these safety gaps, motivating an alignment method designed for LoopLMs. We introduce SafeBridge, which combines lightweight depth specific control of shared recurrent layers, selective state bridging, and joint safety supervision across recurrent depths. Across model scales, multiple attack methods, and safety benchmarks, SafeBridge substantially reduces attack success for both matched-depth and cross-depth attacks. It also improves general utility over the vanilla models while maintaining comparable over-refusal behavior. Our results show that the safety of a LoopLM cannot be inferred from a single recurrent depth, motivating safety alignment across recurrent computation. Our anonymous code repository is available at: https://anonymous.4open.science/r/SafeBridge-08E4

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Looped Language Models, Safety Alignment, Jailbreak Attacks, Adversarial Robustness, Preference Optimization

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.67 over 3 reviews, above 99% of the others.
Reviewer_kpENOpenReview ↗
Rating
4(+2)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies safety across recurrent depths in looped language models. Using Ouro-1.4B and Ouro-2.6B, the authors show that jailbreak vulnerability varies with inference depth and that attacks constructed against one depth can transfer to others. They propose SafeBridge, which combines depth-specific modulation, selective state bridging, and safety-aware training across multiple depths. The problem is timely, and the reported safety improvements are substantial. However, three issues limit my confidence in the conclusions: attacks are reused across evaluated checkpoints, the baselines differ in which depths receive supervision, and general utility is evaluated only at the final depth.

Critical strengths
  1. The paper identifies a meaningful evaluation problem. A looped model may behave safely at one inference depth and unsafely at another. Separating attack source depth from inference depth provides a useful framework for studying this behavior.

  2. The reported improvements are large. In Table 1, SafeBridge reduces overall attack ASR from 21.06% with DPO to 4.56% on Ouro-1.4B, and from 44.93% to 9.85% on Ouro-2.6B. Improvements appear across multiple attack methods and benchmarks.

  3. The evaluation includes useful additional analyses. Cross-depth attack matrices, pooled-source attacks, over-refusal benchmarks, and component ablations help characterize the method. Table 4 also provides evidence that several components contribute beyond the complete system’s average performance.

Critical weaknesses
  1. Reusing fixed attacks does not establish robustness against attacks adapted to SafeBridge Appendix C.4 states that the same attack sets are reused across inference depths and evaluated checkpoints. This is useful for comparing responses to identical inputs, but it leaves an important question unanswered: how effective are attacks optimized directly against the trained SafeBridge model? An attack that works on its construction model may become ineffective after fine-tuning because the model’s behavior has changed.
  2. The comparisons do not fully isolate the benefit of the proposed cross-depth method. Table 8 shows that DPO and SafeDPO supervise only depth 4, LoopDPO supervises depths 1–4, and SafeBridge supervises depths 2–4. The methods therefore differ in both their training objectives and the depths receiving direct supervision. This matters when interpreting LoopDPO’s failure to improve over DPO.
  3. Utility preservation has not been established across the inference depths being aligned. Table 3 and Appendix C.7 evaluate general utility only at depth 4, while the paper’s motivation concerns outputs available at multiple depths. Strong final-depth performance does not show that earlier exits remain useful after alignment.
Questions for authors
  1. Which checkpoint was used to construct the saved attack sets? How do the results change when attacks are optimized directly against each trained checkpoint, including SafeBridge, under matched search budgets?
  2. How does SafeBridge compare with a simple safety-aware baseline supervised uniformly at depths 2–4? Can this comparison control for training data treatment and optimization budget?
  3. What is the general utility at depths 1–3 after SafeBridge training? In particular, how useful is the depth-2 exit given its substantial over-refusal rate?
Reviewer_kne3OpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The paper studies safety of Looped LMs whose shared blocks can be read out at multiple recurrent depths. Prior work suggested deeper recurrence is safer on raw harmful queries; the authors show this does not hold under jailbreaks. Two phenomena are documented: (i) ASR can increase with depth, including a worse intermediate exit than the final depth; (ii) jailbreaks built at one source depth transfer off-diagonal to other inference depths. Standard SFT, DPO, SafeDPO, and a LoopDPO variant that simply shares preference loss across depths leave a large residual risk surface.

They propose SafeBridge: depth-specific affine modulation of shared recurrent layers, selective state bridging from a source depth, and a joint objective with weak-exit aggregation. Across four safety benchmarks and three attack families, overall attack ASR drops on 1.4B and 2.6B, with matched and cross-depth attacks both reduced.

Critical strengths
  1. A real, well-measured LoopLM-specific safety surface. The matched vs cross decomposition and the observation that the worst exit need not be the deepest one are consequential. They justify why vanilla preference optimization, even when applied at multiple depths (LoopDPO), is insufficient.
  2. Depth-specific modulation addresses variation across exits; bridging and consistency target transfer; weak-exit aggregation targets the worst readout. Table 4 shows removing modulation blows up worst-setting ASR, while removing the bridge/U increases safety regression. Utility does not collapse.
Critical weaknesses
  1. Narrow model family and unexplained scale gap. All positive claims rest on Ouro 1.4B/2.6B. It is unclear whether the same depth-wise risk appears in other looped/recursive LMs, or in early-exit Transformers. After identical training, 2.6B overall attack ASR (9.85%) is close to 1.4B DPO. Without analysis (capacity, refusal calibration, attack optimization succeeding more easily), it is hard to know whether SafeBridge scales.
  2. There is no human audit of a labeled subsample, no adaptive attacker that sees SafeBridge, and no transfer from gradient-based attacks optimized on the aligned model rather than on vanilla/source depths. If judge errors correlate across depths, cross-depth transfer could be partly inflated. These are fixable but they currently cap how much I trust the absolute ASR numbers.
Questions for authors
  1. Why is aligned 2.6B much less robust than aligned 1.4B? Please report judge agreement, refusal rates, and attack optimization success rates by scale.
  2. What fraction of harmful labels flip under a second judge or a human sample of 200 completions, especially at depths 3-4?
  3. Is bridge source s_c=2 and T={3,4} (1.4B) / {3} (2.6B) sensitive? Please add a small sweep.
Reviewer_x4w2OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies safety in Looped Language Models (LoopLMs), which reuse a shared Transformer block across recurrent steps so that every recurrent depth can be decoded as a separate output. Evaluating Ouro-1.4B and Ouro-2.6B with GCG, AutoDAN, and PAIR on four safety benchmarks, the authors report two phenomena. First, jailbreak success can increase at deeper inference depths, contrary to prior findings that deeper recurrence is safer on plain harmful queries. Second, jailbreaks constructed against one depth transfer to others. Standard SFT, DPO, SafeDPO, and a naive multi-depth extension (LoopDPO) do not close these gaps. The authors propose SafeBridge, which adds FiLM-style depth-specific modulation to the shared layers and a gated stop-gradient bridge that carries the depth-2 hidden state into later depths. Its training objective combines DPO with per-response safety labels, smooth-max emphasis on the weakest exits, uniform supervision across depths, and a cross-depth consistency penalty. Across 192 settings spanning source depth, inference depth, attack, and benchmark, SafeBridge lowers overall attack ASR from about 21% to 4.6% on Ouro-1.4B and from about 44% to 9.9% on Ouro-2.6B relative to DPO and SafeDPO. It also reduces pooled multi-source attacks, preserves final-depth utility on GSM8K, MMLU, and code benchmarks, and keeps over-refusal comparable to SafeDPO. Ablations on Ouro-1.4B indicate that each component contributes.

Critical strengths
  1. A new, timely safety problem with a reusable evaluation protocol. This appears to be the first systematic jailbreak study of LoopLMs. By separating attack source depth from inference depth, the paper builds a 4×4 risk surface (Fig. 4). It shows that evaluating only the final depth can miss more vulnerable exits; for example, Ouro-2.6B is most vulnerable at depth 3. As looped architectures gain traction, this framing is likely to guide future safety evaluations.

  2. Large, consistent gains at negligible parameter cost. Relative to DPO, SafeBridge lowers ASR on all 16 source/inference pairs, at both scales, under all three attacks, and under pooled multi-source attacks. It adds only about 0.02% parameters and preserves final-depth utility. Gains of this size (e.g., overall ASR from 44% to 9.9% on Ouro-2.6B) make the method practically relevant.

  3. Informative baselines, ablations, and transparency. LoopDPO shows that simply averaging losses across depths is not enough. The ablation reports depth-aware metrics (worst exit, worst setting, safety regression) that reveal effects mean ASR hides. Detailed configurations and an anonymous code release support reproducibility.

Critical weaknesses
  1. Non-adaptive attacks undermine the defense claims. Appendix C.4 states that the 48 attack sets per model scale are constructed once and reused across all evaluated checkpoints. For SFT, DPO, and SafeBridge, the reported matched and cross-depth results are therefore transfer attacks from another model (presumably the vanilla checkpoint), not attacks optimized against the defended model. The large drop in matched GCG ASR from vanilla (74%) to SFT (25%) is consistent with this. Robustness claims for a defense require adaptive attacks against the defended model itself [1, 2]. The authors should report GCG, AutoDAN, and PAIR re-optimized against each SafeBridge checkpoint at each depth.

  2. Anomalous baseline behavior needs explanation. On Ouro-2.6B, SFT and DPO raise original-query ASR above the vanilla model: on average from 30.19 to 41.40 and 37.35, and at depth 4 from 28.32 to 68.76 and 54.30. Similar increases appear at depths 1 and 4 on Ouro-1.4B. Safety fine-tuning should not make a model less safe on plain harmful prompts. One plausible cause is the answer-then-check format inherited from ReSA [3]. In Case A (Loop 4), the model writes an intended-answer summary and then refuses, yet the response is labeled "Attack Succeeded." If such artifacts drive baseline ASR, SafeBridge's relative gains are inflated. The authors should explain this behavior and report results with the format effect controlled for.

  3. Over-refusal is downplayed. At the final depth, SafeBridge's ORR is roughly 2–4× that of DPO. On Ouro-1.4B it is 6.0 vs. 1.6 on XSTest and 14.10 vs. 4.40 on OR-Bench-Hard; on Ouro-2.6B it is 10.4 vs. 2.8 on XSTest. The claim of "comparable" over-refusal holds only relative to SafeDPO. After alignment, depth 2 refuses 50–63% of safe XSTest prompts on Ouro-1.4B, so its lower ASR may largely reflect blanket refusal, and the bridge propagates exactly this state. The authors should explain why over-refusal rises and quantify how much of the ASR reduction comes from refusing more often.

  4. The threat model does not match deployment. Ouro's released configuration runs all four recurrent steps by default, with early exit disabled, and vLLM does not support adaptive exit [4]. In practice, depths 1–3 are therefore not exposed, and an attacker cannot select the inference depth. The paper should specify when intermediate depths are actually used, for example when a provider reduces the step count or enables early exit. It should also evaluate the adaptive-exit mode, in which different tokens of one response may exit at different depths, rather than only fixed-depth generation.

  5. The evaluation metrics and possible judge bias are not discussed. ASR relies solely on the HarmBench Llama-2-13B classifier [5], which has not been validated on LoopLM outputs such as the plan-and-summary text produced by ReSA-style training. PAIR uses the same classifier as its attack judge, so the attacks are optimized against the evaluator itself. ORR is judged by a single LLM sampled at temperature 1.0. Without a human agreement study, a second judge, or reported variance, it is unclear how robust the conclusions are to the choice of judge.

References [1] Carlini et al., "On Evaluating Adversarial Robustness," arXiv:1902.06705, 2019. [2] Andriushchenko et al., "Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks," ICLR, 2025. [3] Cao et al., "Reasoned Safety Alignment: Ensuring Jailbreak Defense via Answer-Then-Check," ICLR, 2026. [4] Zhu et al., "Scaling Latent Reasoning via Looped Language Models," arXiv:2510.25741, 2025; Ouro-1.4B/2.6B model cards (Hugging Face). [5] Mazeika et al., "HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal," ICML, 2024.

Questions for authors
  1. Which checkpoint were the attack sets constructed against? How does SafeBridge perform against GCG, AutoDAN, and PAIR re-optimized on each SafeBridge checkpoint?
  2. Why do SFT and DPO raise original-query ASR above vanilla on Ouro-2.6B? Is this caused by ReSA's intended-answer summaries being flagged by the classifier?
  3. How does SafeDPO applied at all depths, or the SafeBridge loss without modulation and bridge, compare to the full SafeBridge?

Comments on the paper

1 comment

Thank you for the interesting work. We attempted to reproduce Table 5 using the released code and obtained results that differ substantially from the reported numbers. We would appreciate your help in identifying the cause.

[Reproduction setup] We used the code and configurations from the public repository as released. The only modification was adding "seed": 42 to the configuration to resolve a KeyError: 'seed' raised at launch. As in the paper, we evaluated with the HarmBench Llama-2-13b classifier. All commands, configuration files, training logs, loss curves, generations, and scoring outputs are available here: https://anonymous.4open.science/r/safebridge-table5-repro-7c29/README.md

[Results]

  1. The baselines (SFT, DPO) reproduce. On Ouro-2.6B they are within 1–3 points of Table 5, and their training losses match Figure 6. We therefore believe our evaluation pipeline is consistent with the paper's.
  2. The proposed method (SafeBridge) does not reproduce (ASR %, same column order as Table 5).
    • Ouro-2.6B: reported 16.7 / 1.0 / 0.9 / 1.5 → reproduced 43.7 / 15.1 / 9.9 / 51.1
    • Ouro-1.4B: reported 15.0 / 0.3 / 0.0 / 0.3 → reproduced 34.6 / 2.3 / 1.3 / 33.0
  3. The final training loss also differs. Figure 6 shows roughly 0.67–0.75, whereas our runs converge at 1.33.

[Request] To help us locate the source of the discrepancy, could you please share: (1) the exact configuration files and seeds used for Table 5, (2) the training logs or checkpoints from those runs, and (3) any differences between the released code and the code used for the experiments in the paper?

In an era of widespread AI slop and fabricated papers, a reproducibility failure in officially released code is a serious ethical violation. Thank you.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.