- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies intermediate-state selection for recurrent structured reasoning. It introduces FOCUS, which couples a pretrained LLM encoder with a task-specific recurrent updater and selects self-generated trajectory states according to their short-horizon repair progress, then resumes training from the selected state with an additional contraction-based objective.
The method is evaluated on Sudoku-Extreme and Maze-Hard acros Qwen and Llama backbones from 1.7B to 8B parameters, with additional analyses of state curation, the defect penalty, and downstream transfer.
Critical strengths
-
The paper raises a meaningful question in recurrent/latent reasoning, i.e., how to use intermediate solution states for training and provides a coherent framework with substantial improvements, particularly on Sudoku-Extreme.
-
FOCUS consistently improves over the paper's recurrent baseline across multiple Qwen and Llama backbones and model sizes, while the curation and objective ablations provide useful evidence that the training design is effective.
Critical weaknesses
-
The positioning relative to recent recurrent/latent reasoning work is incomplete. Although presented as an LLM-based method, the LLM encodes the input once while a separate task-specific recurrent updater performs the iterative reasoning; the paper does not sufficiently contextualize or compare against closely related recurrent reasoning approaches such as URM and SE-RRM, nor isolate the benefit of using a pretrained LLM encoder.
-
The key contraction target is insufficiently studied. The repair frontier directly depends on , yet all reported configurations use , with no sensitivity analysis or clear justification for this particular operating point.
-
The underlying state-selection problem is not fully formalized independently of FOCUS. The paper motivates finding the “most useful” intermediate state, but usefulness is effectively instantiated directly as proximity to the proposed contraction target; it remains unclear what the underlying training-utility objective is and why selecting a single state is the appropriate formulation rather than deep supervision or multi-state curricula.
Questions for authors
I'm open to raise my score if the authors can solve following questions:
-
How important is the pretrained LLM component itself? Could the authors provide a matched comparison using a simpler/non-pretrained encoder with the same recurrent updater, and clarify how FOCUS relates empirically to recent recurrent reasoning methods such as URM, SE-RRM, or stronger TRM/DRM-style training schemes?
-
How sensitive is FOCUS to the choice of ? What motivates , and do different contraction targets substantially change the selected states or final performance?
-
Can the authors clarify the precise optimization problem underlying state selection? In particular, what formally makes an intermediate state “useful” for future training, and why should selecting the state closest to the proposed frontier be preferable to supervising multiple recurrent states or an explicit intermediate-state curriculum?
4 comments
W3 & Q3. What makes a state useful, and why one starting state.
What makes a state useful. Like a student practicing, the most useful exercises are neither already mastered nor far too hard. From each candidate state, we run the current model for h steps and compare its actual error reduction with the target set by ρ (about 6% with ρ = 0.985, h = 4):
- reduces error much more than the target → too easy;
- reduces error much less than the target → too hard;
- reduces error close to the target → selected by FOCUS (the repair frontier).
This choice follows from our objective of minimizing the final error. Appendix A shows that the final error is driven by the steps that fall short of the target, so training on states that almost meet it reduces the final error most directly. With all other components fixed, this rule outperforms random selection by 3.3 points on Sudoku-Extreme and 4.0 points on Maze-Hard (Table 11).
Why one starting state. FOCUS already supervises multiple states: it just selects one starting state but supervises every step of the following h-step replay, not only the final output. The choice is only about where replay starts. We tested two alternatives on Maze-Hard:
| How replay starts are chosen | Maze-Hard (%) |
|---|---|
| Two starting states per problem | 89.3 |
| Fixed easy-to-hard curriculum (25th → 50th → 75th energy percentile) | 89.8 |
| FOCUS (repair frontier) | 91.1 |
Both alternatives are lower, and two-state replay also uses twice the replay compute. A fixed curriculum follows a preset order, whereas FOCUS adapts to the model's current ability, so the selected states change as the model improves. We have included all new results in the revised paper (Section 3.2 and Appendix D.10).
We sincerely thank the reviewer once again for their time and constructive feedback. We hope our responses and additional results address your concerns, and we would be very grateful if you might consider raising your score in light of these updates.
Thanks to the authors for the response. I am keeping my original scores.
Dear reviewer. Thank you so much again for your time and effort in thoroughly reviewing our work/rebuttal and your response! We are glad that our response addressed your questions properly.
Sincerely, The Authors
We thank the reviewer for the positive comments. We address each weakness and question below:
Relation to URM, SE-RRM, TRM and DRM. We have added URM and SE-RRM to Related Work. These methods are task-specific solvers trained from scratch for a single puzzle. Our goal is different: to study whether a pretrained LLM can learn from its own intermediate states and transfer this beyond the source task, which these models cannot address. We therefore focused our main comparisons on LLM-based baselines in the same setting (Table 1), and see the specialised solvers as complementary rather than as targets for state-of-the-art puzzle accuracy. We agree they provide useful context and have added them to Appendix D.2:
FOCUS achieves the best accuracy on Maze-Hard and outperforms HRM and DRM on Sudoku-Extreme. Only TRM-Att is stronger on Sudoku-Extreme, as it is trained from scratch for a task-specific solver on a fixed grid. Unlike these specialised models, FOCUS trains an LLM adapter that aims to transfer to unseen domains, e.g., maths and code (Table 4).
Why couple the updater with an LLM. Standalone recurrent models such as TRM and URM learn task-specific solvers that cannot be reused. We instead use iterative-repair training to reshape a general-purpose LLM: its pretrained weights stay frozen, and a LoRA adapter is trained jointly with the updater through every recurrent update. With the updater removed, the adapted LLM transfers zero-shot to MATH-Hard and CruxEval (Table 4), e.g., CruxEval-O improves from 50.75% to 55.38% for Qwen3-1.7B. This validates that LLM encoders provide possible benefits of generalization.
Matched encoder controls. We kept the configurations fixed and varied only the encoder:
Frozen backbone controls (Maze-Hard).
Two findings follow. First, the pretrained LLM helps on the harder task: on Sudoku-Extreme it adds 4.0 points over a lightweight encoder, and on Maze-Hard pretrained weights outperform random ones by 1.6 points. Second, FOCUS is the main source of the gains: even with a 2.1M-parameter encoder, FOCUS reaches 60.4% on Sudoku-Extreme and 90.8% on Maze-Hard, far above Recurrent final-only with the full LLM (7.2% and 86.0%, Table 1). The LLM also enables zero-shot transfer beyond the source tasks (Table 4).
What ρ means. ρ sets the target error reduction over h steps. With h = 4, ρ = 0.970, 0.985 and 0.995 require about 11.5%, 6% and 2% reduction, respectively. A smaller ρ demands more progress, so the frontier moves to states the model currently repairs quickly (easier states). A larger ρ demands less, so it moves to states that are harder to repair. We chose ρ = 0.985 as an intermediate target, in line with the "not too hard, not too easy" principle.
Sensitivity. We varied ρ, the replay horizon h, and the collection horizon K_p:
FOCUS is robust to this choice. On Sudoku-Extreme, ρ = 0.995 gives 65.9%, slightly above the default. On Maze-Hard, the default gives the best result, and all variants stay within 88.9–89.7%, above Recurrent final-only (86.0%). The gains of FOCUS therefore do not depend on a finely tuned ρ.