acceptodds
Under review as a conference paper at ICLR 2027

Not Too Hard, Not Too Easy: Learning from Intermediate States for LLM Structured Reasoning

Abstract

Many structured reasoning problems can be formulated as iterative updates to an explicit state under global constraints. Recent recursive models show that this form of computation can be effective in task-specific networks, but it remains unclear whether pretrained language models can acquire the same capability, how intermediate training states should be selected, and whether such adaptation trans- fers beyond the source task. We study these questions by coupling a pretrained language-model backbone with a tied recurrent updater over explicit answer states. We introduce Frontier-Oriented Curation Using Self-trajectories (FOCUS), which selects intermediate states from model-generated trajectories according to the finite- horizon progress achieved by the current model. Rather than selecting states only by their source or initial difficulty, FOCUS prioritizes states according to how effectively they can be improved by the learned transition. Across all evaluated language-model backbones, recurrent state updates consistently improve structured- task performance over direct and one-step training. On Maze-Hard, for example, Qwen3-1.7B reaches 86.0% accuracy with Recurrent final-only, compared with 0.0% for both Direct LoRA SFT and One-step State-SFT. Beyond the source tasks, the resulting language-model adapters exhibit zero-shot transfer to general reasoning benchmarks: with the recurrent module disabled at evaluation and no downstream parameter updates, Sudoku-trained Qwen3-1.7B improves MATH- Hard from 72.36% to 72.96% and CruxEval-O from 50.75% to 55.38%. These results show that pretrained language models can acquire strong explicit-state repair capability and that structured repair training can generalize beyond the recurrent solver, transferring to mathematical and code reasoning under zero-shot evaluation.

Primary area:
foundation or frontier models, including LLMs
Keywords:
recurrent reasoning, explicit-state reasoning, training-state selection, language model adaptation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_bomxOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies intermediate-state selection for recurrent structured reasoning. It introduces FOCUS, which couples a pretrained LLM encoder with a task-specific recurrent updater and selects self-generated trajectory states according to their short-horizon repair progress, then resumes training from the selected state with an additional contraction-based objective.

The method is evaluated on Sudoku-Extreme and Maze-Hard acros Qwen and Llama backbones from 1.7B to 8B parameters, with additional analyses of state curation, the defect penalty, and downstream transfer.

Critical strengths
  • The paper raises a meaningful question in recurrent/latent reasoning, i.e., how to use intermediate solution states for training and provides a coherent framework with substantial improvements, particularly on Sudoku-Extreme.

  • FOCUS consistently improves over the paper's recurrent baseline across multiple Qwen and Llama backbones and model sizes, while the curation and objective ablations provide useful evidence that the training design is effective.

Critical weaknesses
  • The positioning relative to recent recurrent/latent reasoning work is incomplete. Although presented as an LLM-based method, the LLM encodes the input once while a separate task-specific recurrent updater performs the iterative reasoning; the paper does not sufficiently contextualize or compare against closely related recurrent reasoning approaches such as URM and SE-RRM, nor isolate the benefit of using a pretrained LLM encoder.

  • The key contraction target is insufficiently studied. The repair frontier directly depends on ρ\rho, yet all reported configurations use ρ=0.985\rho=0.985, with no sensitivity analysis or clear justification for this particular operating point.

  • The underlying state-selection problem is not fully formalized independently of FOCUS. The paper motivates finding the “most useful” intermediate state, but usefulness is effectively instantiated directly as proximity to the proposed contraction target; it remains unclear what the underlying training-utility objective is and why selecting a single state is the appropriate formulation rather than deep supervision or multi-state curricula.

Questions for authors

I'm open to raise my score if the authors can solve following questions:

  • How important is the pretrained LLM component itself? Could the authors provide a matched comparison using a simpler/non-pretrained encoder with the same recurrent updater, and clarify how FOCUS relates empirically to recent recurrent reasoning methods such as URM, SE-RRM, or stronger TRM/DRM-style training schemes?

  • How sensitive is FOCUS to the choice of ρ\rho? What motivates ρ=0.985\rho=0.985, and do different contraction targets substantially change the selected states or final performance?

  • Can the authors clarify the precise optimization problem underlying state selection? In particular, what formally makes an intermediate state “useful” for future training, and why should selecting the state closest to the proposed frontier be preferable to supervising multiple recurrent states or an explicit intermediate-state curriculum?

4 comments

We thank the reviewer for the positive comments. We address each weakness and question below:


W1 & Q1. Role of the pretrained LLM and relation to recurrent reasoning models

Relation to URM, SE-RRM, TRM and DRM. We have added URM and SE-RRM to Related Work. These methods are task-specific solvers trained from scratch for a single puzzle. Our goal is different: to study whether a pretrained LLM can learn from its own intermediate states and transfer this beyond the source task, which these models cannot address. We therefore focused our main comparisons on LLM-based baselines in the same setting (Table 1), and see the specialised solvers as complementary rather than as targets for state-of-the-art puzzle accuracy. We agree they provide useful context and have added them to Appendix D.2:

MethodPretrained LMMaze-Hard (%)Sudoku-Extreme (%)
HRM (Wang et al., 2025)✗74.555.0
TRM-Att (Jolicoeur-Martineau, 2025)✗85.374.7
DRM (Cameron et al., 2026)✗81.020.0
FOCUS (Qwen3-1.7B)✓91.1‾\underline{91.1}64.4
FOCUS (Qwen3-4B)✓91.566.0‾\underline{66.0}

FOCUS achieves the best accuracy on Maze-Hard and outperforms HRM and DRM on Sudoku-Extreme. Only TRM-Att is stronger on Sudoku-Extreme, as it is trained from scratch for a task-specific solver on a fixed grid. Unlike these specialised models, FOCUS trains an LLM adapter that aims to transfer to unseen domains, e.g., maths and code (Table 4).

Why couple the updater with an LLM. Standalone recurrent models such as TRM and URM learn task-specific solvers that cannot be reused. We instead use iterative-repair training to reshape a general-purpose LLM: its pretrained weights stay frozen, and a LoRA adapter is trained jointly with the updater through every recurrent update. With the updater removed, the adapted LLM transfers zero-shot to MATH-Hard and CruxEval (Table 4), e.g., CruxEval-O improves from 50.75% to 55.38% for Qwen3-1.7B. This validates that LLM encoders provide possible benefits of generalization.

Matched encoder controls. We kept the configurations fixed and varied only the encoder:

EncoderMaze-Hard (%)Sudoku-Extreme (%)
LoRA-adapted Qwen3-1.7B (FOCUS)91.164.4
Lightweight task encoder (2.1M params)90.860.4

Frozen backbone controls (Maze-Hard).

EncoderMaze-Hard (%)
LoRA-adapted Qwen3-1.7B (FOCUS)91.1
Frozen pretrained Qwen3-1.7B, no LoRA90.0
Frozen random Qwen3-1.7B, no LoRA88.4

Two findings follow. First, the pretrained LLM helps on the harder task: on Sudoku-Extreme it adds 4.0 points over a lightweight encoder, and on Maze-Hard pretrained weights outperform random ones by 1.6 points. Second, FOCUS is the main source of the gains: even with a 2.1M-parameter encoder, FOCUS reaches 60.4% on Sudoku-Extreme and 90.8% on Maze-Hard, far above Recurrent final-only with the full LLM (7.2% and 86.0%, Table 1). The LLM also enables zero-shot transfer beyond the source tasks (Table 4).


W2 & Q2. Sensitivity to the contraction target ρ

What ρ means. ρ sets the target error reduction over h steps. With h = 4, ρ = 0.970, 0.985 and 0.995 require about 11.5%, 6% and 2% reduction, respectively. A smaller ρ demands more progress, so the frontier moves to states the model currently repairs quickly (easier states). A larger ρ demands less, so it moves to states that are harder to repair. We chose ρ = 0.985 as an intermediate target, in line with the "not too hard, not too easy" principle.

Sensitivity. We varied ρ, the replay horizon h, and the collection horizon K_p:

TaskρhK_pExact (%)
Sudoku0.985 (default)scheduleschedule64.4
Sudoku0.995scheduleschedule65.9
Maze0.985 (default)41691.1
Maze0.97041689.7
Maze0.99541688.9
Maze0.98521689.7
Maze0.98582089.6

FOCUS is robust to this choice. On Sudoku-Extreme, ρ = 0.995 gives 65.9%, slightly above the default. On Maze-Hard, the default gives the best result, and all variants stay within 88.9–89.7%, above Recurrent final-only (86.0%). The gains of FOCUS therefore do not depend on a finely tuned ρ.

W3 & Q3. What makes a state useful, and why one starting state.

What makes a state useful. Like a student practicing, the most useful exercises are neither already mastered nor far too hard. From each candidate state, we run the current model for h steps and compare its actual error reduction with the target set by ρ (about 6% with ρ = 0.985, h = 4):

  • reduces error much more than the target → too easy;
  • reduces error much less than the target → too hard;
  • reduces error close to the target → selected by FOCUS (the repair frontier).

This choice follows from our objective of minimizing the final error. Appendix A shows that the final error is driven by the steps that fall short of the target, so training on states that almost meet it reduces the final error most directly. With all other components fixed, this rule outperforms random selection by 3.3 points on Sudoku-Extreme and 4.0 points on Maze-Hard (Table 11).

Why one starting state. FOCUS already supervises multiple states: it just selects one starting state but supervises every step of the following h-step replay, not only the final output. The choice is only about where replay starts. We tested two alternatives on Maze-Hard:

How replay starts are chosenMaze-Hard (%)
Two starting states per problem89.3
Fixed easy-to-hard curriculum (25th → 50th → 75th energy percentile)89.8
FOCUS (repair frontier)91.1

Both alternatives are lower, and two-state replay also uses twice the replay compute. A fixed curriculum follows a preset order, whereas FOCUS adapts to the model's current ability, so the selected states change as the model improves. We have included all new results in the revised paper (Section 3.2 and Appendix D.10).


We sincerely thank the reviewer once again for their time and constructive feedback. We hope our responses and additional results address your concerns, and we would be very grateful if you might consider raising your score in light of these updates.

Thanks to the authors for the response. I am keeping my original scores.

Dear reviewer. Thank you so much again for your time and effort in thoroughly reviewing our work/rebuttal and your response! We are glad that our response addressed your questions properly.

Sincerely, The Authors

Reviewer_YLs1OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Repeated solution updates are shown to be helpful for solving structured reasoning tasks (e.g. Sudoku-Extreme and Maze-Hard), but which states in self-trajectories are suitable for training the updater remains not fully explored. This paper proposes FOCUS, a curation scheme that uses the contraction defect to select intermediate states near the repair frontiers for updater training. The preprint supports the introduced method with the experiments on different base model backbones (e.g. Qwen3-1.7B and Llama 3.2-3B-Instruct), comparisons with different state curation schemes (e.g. Energy-Hard and Fixed Mix) and ablation studies on defect penalty and frontier selection. The experiment on zero-shot CoT transfer also provides evidence that the adapted backbones obtain some gains on math and code benchmarks after FOCUS training.

Critical strengths
  1. Matched motivation and method: This paper proposes FOCUS, a new method addressing the question of which state is suitable for learning and how to use these selected states to train a recurrent updater.
  2. Broad analysis and evaluation: This submission evaluates FOCUS on five backbones on Sudoku-Extreme and Maze-Hard as well as on other code and math benchmarks to examine the transfer abilities.
Critical weaknesses
  1. Inadequate comparison in frontier selection ablation: The controlled frontier selection ablation is carried out only on Maze-Hard but without Sudoku-Extreme, raising concerns about whether frontier selection can contribute to performance of FOCUS on Sudoku-Extreme.
  2. Limited analysis of hyperparameter sensitivity: Table 7 lists fixed hyperparameter values, ρ=0.985\rho=0.985 for example, but no sensitivity analyses are reported, making it difficult to tell whether these performance gains are robust to different hyperparameter settings.
Questions for authors
  1. When ρ,h\rho, h and other hyperparameters are set to different values, how will FOCUS perform? How were these fixed values in Table 7 selected?
  2. If LoRA is removed and only the frozen backbone serves as the encoder, or if the pretrained backbone is replaced by a lightweight task encoder, how will FOCUS perform? These comparisons will examine the contribution of the pretrained backbone in the pipeline.
  3. How will FOCUS perform on Sudoku-Extreme when frontier selection is replaced with random selection, while other conditions remain unchanged?
3 comments

We thank the reviewer for the positive comments. We address each weakness and question below:


W1: … Missing Sudoku frontier selection ablation..

We added the frontier selection ablation on Sudoku-Extreme, by replacing the frontier selection with uniform sampling, keeping all other settings unchanged ( Table 11). The results are shown below:

Table 11. State-selection ablation.

MethodSudoku-Extreme (%)Maze-Hard (%)
FOCUS with random selection61.187.1
FOCUS64.491.1

We can see that frontier selection improves FOCUS accuracy from 61.1% to 64.4% on Sudoku-Extreme and from 87.1% to 91.1% on Maze-Hard. With the other components and training conditions held fixed, these results support the benefit of frontier selection over uniform sampling on both tasks. We have included this in Appendix D.5.


W2/Q1 sensitivity analyses for hyperparameters ρ\rho, hh and probe

We appreciate the suggestions and have included the sensitivity analysis.

Sensitivity. We varied ρ, the replay horizon h, and the collection horizon K_p:

TaskρhK_pExact (%)
Sudoku0.985 (default)scheduleschedule64.4
Sudoku0.995scheduleschedule65.9
Maze0.985 (default)41691.1
Maze0.97041689.7
Maze0.99541688.9
Maze0.98521689.7
Maze0.98582089.6

FOCUS is robust to this choice. On Sudoku-Extreme, ρ = 0.995 gives 65.9%, slightly above the default. On Maze-Hard, the default gives the best result, and all variants stay within 88.9–89.7%, above Recurrent final-only (86.0%). The gains of FOCUS therefore do not depend on a finely tuned ρ. We have included these results in the Appendix D.7.


Q2 …frozen backbone… lightweight encoders…

Thanks for the suggestion. We conduct extra experiments to address these two points. We kept the configurations fixed and varied only the encoder:

EncoderMaze-Hard (%)Sudoku-Extreme (%)
LoRA-adapted Qwen3-1.7B (FOCUS)91.164.4
Lightweight task encoder (2.1M params)90.860.4

Frozen backbone controls (Maze-Hard).

EncoderMaze-Hard (%)
LoRA-adapted Qwen3-1.7B (FOCUS)91.1
Frozen pretrained Qwen3-1.7B, no LoRA90.0
Frozen random Qwen3-1.7B, no LoRA88.4

Two findings follow. First, the pretrained backbone contributes: LoRA adaptation gives the best results, adding 4% over the lightweight encoder on Sudoku-Extreme, and pretrained weights outperform random ones by 1.6 points on Maze-Hard. Second, FOCUS remains effective without the pretrained backbone: with a 2.1M-parameter encoder consisting of 2 layer transformers, it reaches 60.4% on Sudoku-Extreme and 90.8% on Maze-Hard, far above Recurrent final-only with the full LLM (7.2% and 86.0%, Table 1). The backbone also enables zero-shot transfer beyond the source tasks (Table 4).


Q3 ….Sudoku-Extreme on random selection with all other components retained?

We replaced frontier selection with uniform sampling. We kept the defect penalty, full-state replay, training budget, and checkpoint-selection rule unchanged (Table 11).

MethodSudoku-Extreme (%)Maze-Hard (%)
FOCUS with random selection61.187.1
FOCUS64.491.1
Difference+3.3+4.0
We observe gains of 3.3 points on Sudoku and 4.0 points on Maze from frontier selection. We have included this in Appendix D.5.

We sincerely thank the reviewer once again for their time and constructive feedback. We hope our responses and additional results address your concerns, and we would be very grateful if you might consider raising your score in light of these updates.

Thank you for your response which resolves my questions. I keep my positive score.

Dear reviewer. Thank you so much again for your time and effort in thoroughly reviewing our work/rebuttal and your response! We are glad that our response addressed your questions properly.

Sincerely, The Authors

Reviewer_qi47OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors focus on recursive models for structured reasoning problems. In this formulation, we can abstract the information on explicit states, but how intermediate states could be utilized for training models are not explored. The authors couple pretrained language models with a recurrent updater to repeatedly update their solution. They introduce FOCUS, Frontier-Oriented Curation Using Self-trajectories, which selects intermediate states whose energy reduction over a fixed number of updates is closest to a chosen target. After selecting the intermediate states, FOCUS resumes training from both its saved answer and its saved memory with the training loss on error reduction. The authors conduct experiments over Sudoku-Extreme puzzles and Maze-Hard instances over five Qwen and Llama family models. FOCUS achieves higher performance compared to Random, Energy-Hard, and Fixed Mix ablations.

Critical strengths
  • Exploring intermediate states for training is well-motivated to recursive models for structured reasoning problems.
  • The authors conduct experiments on Sudoku-Extreme and Maze-Hard with five Qwen and Llama-family models, and try to observe the effect on downstream task, math and code execution.
Critical weaknesses
  • Comparison is limited and insufficient. The authors discuss HRM, TRM, and DRM, but they do not compare their method with these existing methods experimentally. It would be better to explain why these comparisons are omitted. Table 1 is somewhat weak in supporting FOCUS’s advantage on these tasks. Furthermore, it is unclear whether the advantages shown in Table 5 extend to the other backbones in Table 1 or the downstream tasks in Table 4. Because the authors emphasize replay-state selection as a main contribution, these comparisons need to be extended.

  • The evidence for improved downstream reasoning is insufficient. The evaluation covers the limited number of benchmarks and uses one greedy completion. From these results, it is difficult to conclude that FOCUS consistently improves and is helpful to mathematical reasoning and code execution. Evaluating more benchmarks, including sampling with different seeds would help show whether these improvements are consistent.

Questions for authors
  • How does FOCUS compare with HRM, TRM, and DRM under similar settings?

  • The authors use TRM as a teacher to train their models. How do the results change when distillation is removed or a different teacher is used? And do the baselines, including Direct LoRA SFT, One-step State-SFT, Recurrent final-only, Random, Energy-Hard, and Fixed Mix, also use the same TRM distillation setting?

  • How do Random, Energy-Hard, and Fixed Mix perform across the different backbones in Table 1 and on the zero-shot downstream transfer evaluations in Table 4?

5 comments

We thank the reviewer for the positive comments. We address each weakness and question below:


W1/Q1: better explain why omit HRM, TRM and DRM…

HRM [1], TRM [2] and DRM [3] are task-specific solvers trained from scratch for a single puzzle. Our goal is different: to study whether a pretrained LLM can learn from its own intermediate states and transfer this beyond the source task, which these models cannot address. We therefore focused our main comparisons on LLM-based baselines in the same setting (Table 1), and see the specialised solvers as complementary rather than as targets for state-of-the-art puzzle accuracy. We agree they provide useful context and have added them to Appendix D.2:

MethodPretrained LMMaze-Hard (%)Sudoku-Extreme (%)
HRM (Wang et al., 2025)✗74.555.0
TRM-Att (Jolicoeur-Martineau, 2025)✗85.374.7
DRM (Cameron et al., 2026)✗81.020.0
FOCUS (Qwen3-1.7B)✓91.1‾\underline{91.1}64.4
FOCUS (Qwen3-4B)✓91.566.0‾\underline{66.0}

FOCUS achieves the best accuracy on Maze-Hard and outperforms HRM and DRM on Sudoku-Extreme. Only TRM-Att is stronger on Sudoku-Extreme, as it is trained from scratch for a task-specific solver on a fixed grid. Unlike these specialised models, FOCUS trains an LLM adapter that aims to transfer to unseen domains, e.g., maths and code (Table 4).


W1.2/Q3.1: .. extend to the other backbones in Table 1?...

Thank you for the suggestion, we extended the curation comparison beyond Qwen. We run Random, Energy-Hard, and Fixed Mix on Maze-Hard with Llama-3.2-3B-Instruct.. We compare these runs with our reported FOCUS result in Table 1.

Table R2: Comparison of replay-state selection strategies. Best result is shown in bold.

MethodReplay-state selectionExact accuracy (%)
RandomUniformly sampled rollout state89.3
Energy-HardRollout state with the highest task error89.5
Fixed MixFixed mixture of training-state types89.2
FOCUSRollout state nearest the repair frontier90.1

The results are consistent with Table 5. Specifically, FOCUS achieves the highest accuracy (90.1%), improving on random replay by 0.8% and on Energy-Hard by 0.6. The baselines differ by at most 0.3%, which suggests that targeting the hardest states or using a fixed mixture adds little. Replaying states near the repair frontier, which are challenging but still correctable, gives the most useful training signal. We have included this in Appendix D.4.


W2/Q3.2: …sampling with different seeds?…more benchmarks?..

We agree that more experiments could further support the claim.

  • Sampling with different seeds. To further assess robustness, we evaluated Base and FOCUS using nonzero-temperature sampling across three decoding seeds (42, 43, and 44), reporting mean accuracy and standard deviation. The results are shown below:

Table R3: Sampling with different seeds.

BenchmarkBase (%)FOCUS (%)FOCUS − Base
MATH50068.40 ± 0.4069.73 ± 1.01+1.33
CruxEval-I27.92 ± 0.3829.13 ± 0.66+1.21
AIME2518.89 ± 1.9218.89 ± 3.850.00
AIME2616.67 ± 5.7722.22 ± 6.94+5.56

We can see that this did not change our claim. FOCUS generally improves over Base on three of the four evaluated benchmarks under sampling. Together, the additional experiments provide encouraging evidence of downstream transfer while highlighting task- and decoding-dependent variability. We have included this discussion in Appendix D.5.

  • Extra benchmarks. We further evaluated Random, Energy-Hard, Fixed Mix, and FOCUS, all using the same Sudoku-trained Qwen3-1.7B adapters, against the unadapted baseline under identical evaluation settings.

Table R4: Extra benchmarks.

BenchmarkBaselineRandomEnergy-HardFixed MixFOCUS
MATH50070.0069.4069.2068.0068.60
MATH-Hard72.3672.8972.2171.7572.96
CruxEval-O50.7551.8850.7550.8855.38
CruxEval-I24.0028.0029.5026.8826.75
Average54.2855.5455.4254.3855.92

We can see that FOCUS outperforms Baseline on extra benchmarks. We have included this in Appendix D.6.


[1] Wang et al. (2025). Hierarchical Reasoning Model.

[2] Jolicoeur-Martineau (2025). Less is More: Recursive Reasoning with Tiny Networks.

[3] Cameron et al. (2026). One Step Forward and K Steps Back: Better Reasoning with Denoising Recursion Models.

Q1: please see our response in W1.


Q2: …remove distillation results? …different teacher? …are all baselines use same TRM distillation setting?

  • remove distillation and different teacher. We removed the TRM process teacher on both Sudoku-Extreme and Maze-Hard, and replaced TRM with HRM on Maze-Hard. We kept Qwen3-1.7B as the encoder and retained the recurrent updater. Results are shown below:

Table 5: Effect of the process teacher. Best results are shown in bold.

Process teacherSudoku-Extreme (%)Maze-Hard (%)
None59.890.3
HRM61.391.6
TRM64.491.1

We can see that without a process teacher, the model achieves 59.8% accuracy on Sudoku-Extreme and 90.3% on Maze-Hard. In these teacher-free runs, we remove teacher supervision and the KL loss while retaining ground-truth task supervision, frontier selection, the defect penalty, and full-state replay. Adding an HRM teacher improves accuracy to 61.3% and 91.6%, respectively, while a TRM teacher achieves 64.4% and 91.1%. Thus, process supervision improves performance on both tasks, but the stronger teacher depends on the task: TRM provides the largest gain on Sudoku-Extreme, whereas HRM performs best on Maze-Hard.

  • Distillation settings. As discussed in Appendix C and D, we use the same settings across all baselines. Specifically, for Random, Energy-Hard, Fixed Mix, and FOCUS, we use the same TRM clean-branch supervision settings in our baseline recipes. For Direct LoRA SFT, One-step State-SFT, and Recurrent final-only, we use ground-truth supervision without an external process teacher.

Q3 …. different backbone result for Table 1 and Table 4?

Please see our response in W1.2 and W2.


We sincerely thank the reviewer once again for their time and constructive feedback. We hope our responses and additional results address your concerns, and we would be very grateful if you might consider raising your score in light of these updates.

Dear reviewer. Thank you for reading our rebuttal! We believe that our response addresses your raised weaknesses and questions. If you agree that our response addresses the weaknesses and questions, please consider raising your score. If you have any outstanding concern, please let us know so that we can do our best to address them.

Sincerely, The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response: Summary of Reviews and Rebuttal

We thank all reviewers for their time and constructive comments. Reviewers agreed that using intermediate states to train recurrent reasoners is well-motivated (qi47, YLs1, bomx). They also noted that FOCUS consistently improves over the recurrent baseline across Qwen and Llama backbones, and that the evaluation and ablations are broad (YLs1, bomx).

During the rebuttal we added the following experiments and clarifications, all of which are included in the revised paper:

  1. Comparison with HRM / TRM / DRM, and positioning against URM / SE-RRM (qi47, bomx). We report these task-specific solvers as context, including TRM-Att, our process teacher (App. D.2), and discuss URM and SE-RRM in Related Work. Together, these findings highlight FOCUS’s distinctive strength: combining effective structured reasoning with transferable language-model adaptation.

  2. Contribution of the pretrained LLM (YLs1, bomx). Matched encoder controls, a lightweight encoder and frozen backbones, isolate the contributions of FOCUS and of the pretrained backbone (App. D.8, Table 15). This supports FOCUS as a flexible training framework whose effectiveness extends beyond a particular pretrained encoder.

  3. Frontier selection on Sudoku-Extreme (YLs1). We added the ablation that replaces frontier selection with uniform sampling on both tasks (App. D.5, Table 11). This provides direct evidence for our central claim that choosing where replay starts improves training effectiveness.

  4. Hyperparameter sensitivity: ρ, h, K_p (YLs1, bomx). We added a sensitivity analysis (App. D.7). This supports the robustness of FOCUS’s gains over Recurrent final-only across the hyperparameter settings.

  5. Formalising state utility: single start state vs. multi-state replay or a curriculum (bomx). We clarify the link to the final-error objective (App. A) and compare FOCUS with two-state replay and a fixed easy-to-hard curriculum (App. D.10). This supports model-adaptive replay allocation as a more effective choice than the evaluated multi-start and fixed-curriculum alternatives.

  6. Curation baselines on other backbones and downstream tasks (qi47). We extended the curation comparison to Llama-3.2-3B (App. D.4) and to downstream benchmarks (App. D.6). Together, these comparisons reinforce the advantages of FOCUS beyond the original backbone and source-task evaluations.

  7. Robustness of downstream transfer (qi47). We added multi-seed sampling evaluations (App. D.5, Table 12). These results provide encouraging evidence that FOCUS can generalize beyond its original structured reasoning tasks, further highlighting the broader applicability of our approach.

  8. Effect of the process teacher (qi47). We added ablations with no teacher and with an HRM teacher (App. D.9, Table 16). These results demonstrate FOCUS’s effectiveness independently of distillation, with process supervision providing an additional accuracy boost.

We are pleased that Reviewer YLs1, who raised the questions on frontier selection, hyperparameter sensitivity and the role of the LLM, confirmed that our response resolved these concerns. Detailed responses to each reviewer are provided in the individual threads. We hope these revisions also address the related concerns of Reviewer qi47, and we are happy to clarify any remaining questions during the discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.