- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Recurrent models such as AKOrN and TRM achieve strong performance on complex reasoning tasks by applying shared modules recursively, showing that adding more depth or parallel runs gives further improvement at inference time. However, these gains mostly come from passive strategies such as repeating computation or running more rollouts. This paper introduces RoFB, a closed-loop approach that uses the readout signals as feedback to steer the reasoning paths toward organized, converged latent states. In experiments, RoFB shows a clear advantage in most setups on Sudoku and Maze tasks across three different recurrent models (AKOrN, ItrSA++, TRM).
Critical strengths
-
Clear and novel contribution. In recurrent models such as Looped Transformers, inference-time scaling behavior is one of the most important factors. Improving performance at inference time usually means running more loops or more rollouts, which is to say the model runs the recurrence without any intervention. RoFB instead takes a closed-loop approach that can actively intervene. Search during recurrence is another way to steer in a closed-loop manner, but it generally demands far more resources. RoFB instead steers the recurrence with a single feedback term added to each step, so it is likely to be more resource-efficient than search-based methods (although this comparison is not included in the paper).
-
Posing new potential in recurrent models. I found it compelling that AKOrN, ItrSA, and TRM are defined within the same recurrent framework. This is not just unifying them under the loose criterion of repeating a module recurrently. The synchrony behavior between latent tokens seen in AKOrN also arises from plain self-attention, and similar behavior shows up in ItrSA and TRM, demonstrating that the phenomenon is consistent enough to actually be exploited.
-
Thorough analysis on inter-token clusterization. RoFB is motivated by the observation that latent tokens form class-wise clusters when the model reaches a correct solution. However, the paper does not leave this as an empirical finding from AKOrN alone. It shows that the same behavior appears consistently in ItrSA++ and TRM, and compares it across models and tasks using measurable metrics such as the cosine gap and silhouette score. It also reports where this indicator becomes less reliable, for example TRM on Maze, and shows that RoFB increases the separation even on boards that remain unsolved. Together with the discussion of why inter-class separation can help the learned dynamics reach a solution, this gives solid support for the motivation behind the method.
Critical weaknesses
Limited scalability. Sudoku and maze are hard enough benchmarks, but the experiments remain narrow in scope, covering only two puzzle tasks with relatively small models, each trained once. Beyond these tasks, the only harder setting tested is ARC-AGI-1, where RoFB brings no improvement. The authors suggest that, as in TRM on Maze, confident errors cannot be distinguished from correct peers by the prediction-based coupling. However, with a single harder benchmark, it is difficult to tell whether RoFB fails because ARC-AGI-1 is simply harder, or because it falls into the same situation as TRM on Maze. I also still question whether the method is too task-specific. The coupling is computed from how much the readout distributions of two tokens overlap, which assumes a small, shared vocabulary. For tasks where outputs rarely share the same tokens, the coupling would become nearly uniform and lose its class-specific signal, reducing RoFB to a simple repulsion from the mean. I think this boundary should be discussed more explicitly in the limitations.
Questions for authors
- Instead of injecting repulsion into the dynamics, what happens if one simply searches over trajectories using the same separation measure? This would clarify whether the gain comes from the force itself or from the separation objective.
- The introduction states that the gains come with ×10–100 more efficient compute. How is this number derived? Based on the operating-point grids in the appendix, the ratio seems to fall in this range only for the two Maze pairs, while it is about 1/8 for AKOrN on Sudoku and about 1/1.3 for TRM on Sudoku. Could the authors report the ratio for each model–task pair?
- Why is it not applied at training time? If it were applied during training and amortized, is there no improvement? If it can be internalized through post-training, what gains that would bring also seems like an important analysis.
6 comments
Q3. Training-time use and internalization of RoFB
Our focus is improving inference while keeping pretrained model weights fixed. To investigate training-time use, we tested learning only RoFB’s coefficients, jointly updating the pretrained model and RoFB, and training both from scratch. Learning only RoFB’s coefficients improved accuracy by 0.960 percentage points over fixed RoFB. However, incorporating RoFB into training did not consistently outperform comparable training without RoFB. After joint post-training, removing RoFB often reduced accuracy, indicating that its benefits were not fully internalized into the model weights. Training-time integration remains a promising extension, with benefits depending on the training configuration.
W1b. Failure on ARC-AGI-1: task difficulty or a shared failure mode?
Thank you for asking us to distinguish these explanations. To test whether RoFB works with a stronger reasoner on ARC-AGI-1, we trained a TRM variant following the recipe introduced by Rekursiv.ai (https://rekursiv.ai/blog/pushing-limits-arc-agi/), achieving 69.0% pass@2 on all 400 evaluation tasks. We then applied RoFB selected by a 30-trial Optuna search on 100 tasks. On the remaining 300 tasks, task-averaged pass@2 was 68.33% for both ordinary inference and selected RoFB. Extending ordinary inference from 16 to 18 outer steps to approximately match RoFB's runtime also gave 68.33%. The lack of improvement therefore persisted beyond the original TRM checkpoint and a single feedback setting within the tested search space.
Through failure analysis, we observed that high-confidence errors dominated ordinary-inference errors and were rarely corrected by RoFB. We examined 40 selected tasks from each of the validation and test splits, with four augmented inputs per task. High-confidence errors, defined as incorrect outputs for which every incorrectly predicted cell within the reference output had at least 99% softmax probability for its predicted class, constituted approximately 85–91% of ordinary-inference errors, and RoFB corrected almost none of them (Table). Thus, even with the stronger checkpoint and validation-selected feedback, failure to correct the dominant group of high-confidence errors remained the main observed limitation in this diagnostic sample. The smaller group below the confidence threshold was also never corrected in this comparison. These observations characterize the remaining errors; they do not isolate high confidence itself or task difficulty as the cause of resistance to correction.
Table: Ordinary-inference errors and corrections by RoFB, counted per augmented input before voting.
| Split | High-confidence errors / all ordinary-inference errors | Corrected by RoFB: high-confidence errors | Corrected by RoFB: errors below this threshold |
|---|---|---|---|
| Validation | 66/78 (84.6%) | 0/66 | 0/12 |
| Test | 64/70 (91.4%) | 1/64 | 0/6 |
Thank you for your careful review and constructive feedback. We divide W1 into evaluation scope (W1a) and performance on harder tasks (W1b). Below, we respond to W1a, W2, Q1, and Q2. We will address W1b and Q3 in a follow-up response.
W1a. Limited coverage of tasks, model sizes, and training seeds
We expanded our evaluation to graph coloring, Nonogram, Futoshiki, and Mini-ARC. RoFB improved Futoshiki accuracy across all three models and training seeds, with smaller gains on graph coloring. Nonogram and Mini-ARC showed no clear benefit. These results broaden the evaluation while showing that benefits remain task- and model-dependent. Please see our Common Concerns comment for details.
W2. Dependence on a small, shared output vocabulary
Thank you very much for pointing out this important limitation. We admit tha t the current formulation of RoFB relies on a small, shared vocabulary to compute the inter-token coupling based on readout distributions. In tasks with large vocabularies, this coupling may become diluted, potentially reducing the effectiveness of RoFB. We have revised the limitations section to more explicitly discuss the potential challenges of applying RoFB to tasks with large vocabularies. We acknowledge that this boundary is an important consideration for future work and will explore potential modifications to RoFB that could address this issue in tasks with larger vocabularies.
Q1. Can separation-based trajectory selection explain RoFB’s gains?
Thank you for this suggestion. We compared entropy-based trajectory selection with cosine gap and cosine silhouette, computed from predicted labels and terminal hidden states without ground-truth labels. All selectors used the same trajectories from the unmodified dynamics. We evaluated AKOrN and ItrSA++ on 20,000 Sudoku and 1,000 Maze instances. Results with four candidates and 256 inference steps per trajectory are shown below.
Table. Exact accuracy (%) by trajectory selector, using four candidates and 256 internal updates per trajectory.
Across seven candidate counts (1–64) and three update budgets (64, 128, 256), neither separation measure consistently outperformed entropy selection. The largest Sudoku improvement was +0.025 percentage points, while Maze differences ranged from −0.7 to +0.2 points. These results suggest that separation-based selection of completed trajectories alone is insufficient to explain RoFB’s gains in the tested settings.
Q2. Basis of the 10–100× efficiency claim and per-pair ratios
We agree that the original “10–100×” wording was too broad and have removed it from the Introduction. Comparing the cheapest baseline point attaining the maximum mean accuracy in the evaluated grid with the cheapest RoFB point exceeding it, the baseline-to-RoFB cost ratios are 7.89× for AKOrN on Sudoku, 1.32× for TRM on Sudoku, 58.31× for AKOrN on Maze, and 58.18× for ItrSA++ on Maze. ItrSA++ on Sudoku has a ratio of 0.96× under the same rule, providing no cost saving; TRM on Maze has no RoFB point exceeding the baseline maximum.