- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes Looped Flows, a framework that combines flow/diffusion-based generative modeling with stateful recurrent (looped) architectures to solve complex algorithmic and spatial reasoning tasks. Standard looped architectures face optimization difficulties because truncating backpropagation through time (BPTT) leaves early recurrent states without clear alignment or global learning signals for future computation.
Critical strengths
Principled Mitigation of Truncated BPTT Instability in Looped Reasoning: A major failure mode in deep looped architectures trained via local per-step objectives is their tendency to fall into spurious attractors or fail to converge because early steps receive no gradient signals from future steps. By framing recurrence along a probability flow trajectory with monotonically decreasing noise levels, the paper provides a natural temporal curriculum that aligns step-by-step local losses. The empirical analysis convincingly shows that looped flows eliminate of failure cases observed in TRM on Sudoku-Extreme.
Critical weaknesses
Heuristic Regularization (Pseudotargets) Required for Single-Solution Tasks: For data-scarce single-solution tasks like Sudoku, the model suffers from overfitting unless a task-specific student-forcing regularizer is introduced, where interpolants mix noise with previous model predictions instead of ground-truth solutions. This dependency indicates that the core flow formulation alone is prone to exposure bias / target memorization on small datasets, complicating the framework and adding hyperparameters that require careful tuning per domain.
Questions for authors
Clarification on Noise-Sharing Shortcuts: Have you probed the hidden state during inference to verify whether linear combination representations of noise exist? How does performance degrade if independent noise is used per step while keeping the decreasing noise schedule ?
Pseudotarget Generalization: How sensitive are the ARC and Maze results to the pseudotarget regularizer? Did you observe exposure bias or degradation when omitting pseudotargets on ARC-AGI-1/2?
3 comments
(2/2) Noise-sharing shortcuts
Q1. Clarification on Noise-Sharing Shortcuts: Have you probed the hidden state during inference to verify whether linear combination representations of noise exist? How does performance degrade if independent noise is used per step while keeping the decreasing noise schedule?
We would like to address the two parts of the question, probing the recurrent state and using independent noise per step, in turn.
Probing the recurrent state. The shortcut in Appendix B (Eq. 16) cancels the noise by combining the current input with an earlier input. Since the model already takes as input, the shortcut is possible only if the recurrent state carries information about the initial noise beyond what contains. To check whether this happens at inference, we fit linear (ridge regression) probes at each step that predict and the solution , either from alone or from together with the state . Here is the part of the recurrent state from which the prediction is decoded (Lines 723–724). We use 2,048 Sudoku-Extreme test puzzles, fit on half of them, and report on the other half.
Table B. Held-out of linear probes on Sudoku-Extreme, averaged over three seeds.
| Target | Probe input | Step 1 | 2 | 4 | 8 | 16 | 32 | 64 | 128 |
|---|---|---|---|---|---|---|---|---|---|
| Initial noise | 1.00 | 0.92 | 0.78 | 0.56 | 0.28 | 0.05 | 0.00 | 0.00 | |
| Initial noise | and | 1.00 | 0.93 | 0.79 | 0.57 | 0.28 | 0.06 | 0.00 | 0.00 |
| Solution | 0.00 | 0.00 | 0.00 | 0.00 | 0.01 | 0.08 | 0.46 | 0.98 | |
| Solution | and | 0.41 | 0.57 | 0.71 | 0.81 | 0.88 | 0.94 | 0.96 | 0.98 |
We find that adding to the input increases the for by at most 0.01 at any step. The decrease of this over the steps comes from the input itself, which contains less of as grows. This shows that the recurrent state does not keep a linear representation of beyond the current input, and thus shortcut exploitation is not happening. The rows for check that the probes can read information held by . Adding increases the for from 0.00 to 0.41 at step 1 and from 0.01 to 0.88 at step 16, as expected since the prediction is decoded from .
Independent noise per step. We tested using independent noise at each step while keeping the decreasing noise schedule during both training and inference.
During training, this corresponds to the "w/o noise sharing" ablation in Table 3. Using independent noise per step reduces accuracy by 2.4 percentage points on ARC-AGI-1 (58.8% to 56.4%) and by 1.4 percentage points on ARC-AGI-2 (12.2% to 10.8%). We also ran this ablation on Sudoku-Extreme, where accuracy drops by 2.2 percentage points (97.9% to 95.7%). Thus, sharing noise during training gives a small but consistent gain by aligning adjacent denoising steps, while the model still works well with independent noise.
During inference, we also tested replacing the integrated flow state at every step with an interpolant formed from fresh independent noise and the current prediction, so that no noise sample is shared across steps (Table C).
Table C. Sudoku-Extreme test accuracy (%) with our stochastic sampler and with fresh noise at every step.
| Sampler | 16 steps | 32 steps | 64 steps | 128 steps |
|---|---|---|---|---|
| Ours | 85.3 | 92.6 | 96.3 | 98.0 |
| Fresh noise at every step | 85.1 | 92.5 | 96.2 | 98.0 |
We see that using fresh noise at every inference step changes accuracy by at most 0.2 percentage points. Together with the probing results in Table B, this confirms our discussion in Appendix B that the learned models do not rely on the noise-cancellation shortcut. We will add these results to the appendix in the next revision.
We would like to thank the reviewer again for their time and constructive feedback. As the discussion deadline approaches, we welcome any remaining questions. If our responses resolve the remaining concerns, we would be grateful for the re-assessment. Thank you again for your consideration.
(1/2) Pseudotargets
We thank the reviewer for the careful review and for finding our motivation clear. Below, we address the raised concerns and questions with new experiments.
We thank the reviewer for raising this point. We would first like to clarify that six of the eight benchmarks in our paper (Maze-Hard, ARC-AGI-1, and all four multi-solution tasks) are already trained without pseudotargets (Table 6). When pseudotargets are used (only on Sudoku-Extreme and ARC-AGI-2), they do not require per-domain hyperparameter tuning, as both tasks use the same linear schedule from 0 to 1 over the first 20k training steps (Lines 845–846).
To test how sensitive all four single-solution tasks are to pseudotargets, we trained a model with the opposite setting on each task while keeping all other settings fixed.
Table A. Test accuracy (%) on single-solution benchmarks with and without pseudotargets.
We see that omitting pseudotargets does not hurt performance on Maze-Hard, ARC-AGI-1, or ARC-AGI-2. Maze-Hard and ARC-AGI-1 achieve higher accuracy without pseudotargets. On ARC-AGI-2, the model without pseudotargets is within one point of our best checkpoint (13.2% vs. 14.2%) and actually better than the three-seed mean (12.2%) reported in the paper. Among all eight tasks in the paper, only Sudoku-Extreme requires pseudotargets (97.4% vs. 61.3%).
We agree with the reviewer that the drop on Sudoku-Extreme without pseudotargets comes from target memorization on a small dataset. Because Sudoku-Extreme has only 1,000 training puzzles (Line 808), an interpolant at large t gives the model an almost clean copy of the unique solution, so the model can fit the training puzzles without learning to correct its own errors. We note that this is a common challenge for flow models on small reasoning datasets. For example, FRM [1] reports 13.1% accuracy for a plain flow model and 32.6% with self-conditioning on Sudoku-Extreme, and addresses this by training on the model's own rollouts. Pseudotargets serve the same role in our framework (Lines 826–828). We will add Table A and this discussion to the next revision.