acceptodds
Under review as a conference paper at ICLR 2027

Adaptive Supervision Helps Tiny Recursive Models Learn Better

Abstract

Tiny recursive models (TRMs), as a class of latent depth-recursive models, demonstrate powerful reasoning capabilities and provide promising avenues for scaling reasoning capabilities in limited model size. Despite remarkable progress, training recursive models faces challenges in how supervision is applied across thinking steps. Supervising only the final output provides sparse learning signals, while TRMs address this by applying full supervision at every thinking step. However, at early thinking steps, full supervision can be noisy since the model may need more steps to derive a complete solution. Building on this intuition, we introduce AS-TRM, a simple yet powerful training method for TRMs with Adaptive Supervision. The core idea is to use the outcome of each thinking step to partition the training signals into focus and non-focus sets. We analyze AS-TRM through gradient and loss landscape diagnostics, and show that it reduces supervision noise and leads to faster convergence. Despite its simplicity, AS-TRM delivers substantial gains in both training speed and final performance: on Sudoku-Extreme, it achieves a 7.9-point accuracy gain (11.3% relative) and speeds up training by up to 1.79x; on Maze-Hard, AS-TRM enables flip and rotation augmentation, improving final accuracy by 6.3 points over an already strong baseline; and on ARC-AGI-1, it achieves up to 1.51x speedup.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Latent reasoning, Tiny Recursive Models, Hierarchical Reasoning Models, Curriculum Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_H9BvOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the mismatch between step-local gradient updates and full-target deep supervision in Tiny Recursive Models (TRMs). To address this issue, it proposes AS-TRM, an adaptive supervision method that dynamically constructs a task-specific focus set at each reasoning step based on trusted predictions and a state-dependent frontier. The proposed method prioritizes supervision on the focus set through a weighted loss while preserving the original halting mechanism. Task-specific strategies are developed for Sudoku, maze solving, and ARC-AGI-1, using constraint propagation, model-consistent shortest-path construction, and puzzle-level loss selection, respectively. Experimental results show that AS-TRM improves Sudoku performance across the tested recursion depths and, when combined with augmentation, improves Maze-Hard performance. On ARC-AGI-1, it shows faster convergence during the earlier stages of training but only a modest final performance gain. The paper further supports its findings through ablation studies, extended recursive rollouts, loss-landscape visualization, and gradient-alignment analyses.

Critical strengths

The paper identifies a clear mismatch between step-local gradients and full-target supervision. AS-TRM addresses this issue without modifying the TRM architecture and improves Sudoku performance across all tested recursion depths. The frontier ablations further show that the gains depend on selecting an appropriate adaptive frontier rather than merely supervising fewer cells.

The paper also provides valuable analyses of recursive stability, loss landscapes, and gradient alignment. Although some of these diagnostics are local or limited to particular checkpoints and training stages, they help explain how adaptive supervision can reduce gradient interference and produce more stable recursive reasoning, strengthening the paper’s technical contribution.

Critical weaknesses

The Maze-Hard gain should not be attributed to adaptive supervision alone. AS-TRM without augmentation underperforms the standard TRM baseline, while the reported improvement arises from combining augmentation with model-consistent shortest-path target construction. On ARC-AGI-1, the final performance gain is modest, although AS-TRM shows faster convergence during the earlier stages of training. Together, these results limit the evidence that the method generalizes beyond the strongest Sudoku setting.

The focus-set construction relies on task-specific structural rules, labels, or verification signals, which may limit its applicability to less structured reasoning tasks. In addition, the reported efficiency improvements are measured in training steps rather than elapsed training time, and the paper does not quantify the computational overhead of focus-set construction, including constraint propagation and shortest-path computation.

Questions for authors
  1. Table 13 separates augmentation from the full AS-TRM method, but adaptive masking and model-consistent shortest-path target construction remain coupled. Could the authors provide or discuss a more controlled comparison that isolates these two components within the augmented Maze-Hard setting?

  2. How well might AS-TRM generalize when task-specific structural rules or reliable training-time verification signals are unavailable? Could the authors also report elapsed training time and the computational overhead of focus-set construction to clarify the practical efficiency gains?

Reviewer_68VMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Adaptive Supervision for Tiny Recursive Models (AS-TRM), aiming to address the mismatch between intermediate reasoning states and full-target supervision in recursive reasoning models. Instead of supervising all output positions at every reasoning step, AS-TRM dynamically constructs a focus set consisting of trusted predictions and a task-dependent derivable frontier, while down-weighting or masking the remaining positions.

The method is evaluated on Sudoku-Extreme, Maze-Hard, and ARC-AGI-1. The reported results show improvements over standard TRM, particularly on Sudoku-Extreme and Maze-Hard, as well as faster convergence measured by training steps. The authors further provide analyses of gradient interference, loss landscapes, and long-horizon recursive dynamics to support their proposed explanation.

Critical strengths
  1. Consistent empirical improvements on Sudoku-Extreme.

AS-TRM improves accuracy across three recursion configurations, with gains of 3.7–7.9 percentage points over standard TRM. The random-frontier and inverse-frontier ablations provide useful evidence that the selection of supervised positions matters beyond simply reducing the number of supervised targets. These results suggest that selective intermediate supervision can be beneficial for recursive models.

  1. Interesting observations about recursive dynamics.

The long-horizon rollout experiments suggest that AS-TRM can better preserve correct solutions and maintain stable predictions under additional recursive computation. Together with the gradient analyses, these observations provide useful empirical evidence that the choice of intermediate supervision may influence the learned recursive dynamics, rather than merely affecting short-term optimization.

Critical weaknesses
  1. Limited methodological novelty and strong dependence on task-specific supervision rules.

The central idea of AS-TRM is to supervise positions that are already correct or relatively easy to derive, while postponing supervision on more difficult positions. This is closely related to existing curriculum learning, self-paced learning, and selective loss masking approaches. The primary distinction is applying these principles dynamically across recursive reasoning steps.

More importantly, the focus-set construction relies heavily on task-specific knowledge. Sudoku uses explicit constraint propagation, Maze-Hard uses shortest-path structure, and ARC-AGI-1 uses puzzle-level loss ranking. These are substantially different selection mechanisms rather than a unified, generally applicable criterion for identifying derivable targets.

This raises a fundamental question about the contribution: whether AS-TRM introduces a general solution to the state-supervision mismatch problem, or mainly demonstrates that carefully designed, task-dependent supervision heuristics can improve TRM training. The lack of evaluation using a unified task-agnostic selection strategy makes the claimed generality difficult to assess.

  1. Insufficient experimental isolation of the proposed mechanism and limited evidence of broader effectiveness.

The experimental improvements are less conclusive than the headline results suggest.

On Maze-Hard, AS-TRM without augmentation actually underperforms standard TRM (81.5% versus 86.8%). The improvement to 93.2% arises only when AS-TRM is combined with augmentation and a model-consistent shortest-path target selection procedure. Therefore, the gain cannot be attributed solely to adaptive supervision. The proposed target-selection mechanism also resolves ambiguity among multiple valid shortest paths, which is a separate issue from the claimed state-supervision mismatch.

On ARC-AGI-1, the final improvement over TRM is negligible (41.4% versus 40.7%), particularly relative to the reported variability across seeds. This weakens the evidence that the method generalizes beyond tasks with explicitly exploitable local structure.

Furthermore, the reported training speedups are defined using the largest ratio over reachable accuracy thresholds, rather than a standardized compute or wall-clock comparison. Since focus-set construction involves additional task-dependent processing, it remains unclear whether the reported improvements translate into equivalent reductions in total training cost.

Finally, the gradient-alignment analysis uses a terminal reference restricted to positions predicted correctly at the final step. As acknowledged in Appendix C.6.6, the claimed alignment advantage largely disappears when this restriction is removed. Consequently, this analysis does not yet convincingly establish that adaptive supervision provides a generally better approximation to the full terminal optimization objective.

Taken together, these issues make it difficult to determine how much of the reported improvement comes from the proposed supervision principle rather than task-specific design choices and evaluation settings.

Questions for authors
  1. Generality of focus-set construction (clarification and potentially decisive additional evidence).

Can the authors demonstrate that AS-TRM remains effective when the focus set is constructed without explicit task-specific solvers or structural knowledge? For example, would a unified confidence-based or loss-based selection mechanism provide comparable improvements across Sudoku, Maze, and ARC-AGI-1?

This would help establish whether the contribution is a general adaptive supervision framework or a collection of task-dependent heuristics.

  1. Attribution of the Maze-Hard improvement (clarification and controlled comparison).

Given that AS-TRM without augmentation performs worse than standard TRM, how much of the final gain is attributable to adaptive masking versus the model-consistent shortest-path target selection?

Could the authors provide a baseline that uses the same dynamically selected valid shortest-path targets and the same augmentation, but retains full supervision? This comparison seems necessary to isolate the contribution of adaptive supervision from the benefit of resolving target ambiguity.

  1. Interpretation of the gradient evidence and training efficiency (clarification).

Why is the terminal reference gradient restricted to positions correctly predicted at the final step, given that this restriction substantially affects the observed alignment? Can the authors provide evidence that the proposed gradient alignment predicts improvements in the actual terminal objective rather than only in this selected subset?

Additionally, can the authors report wall-clock training time or total computational cost, including focus-set construction, at matched accuracy thresholds? This would clarify whether the claimed speedups represent practical computational savings.

Reviewer_GngeOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes AS-TRM, which adapts supervision to the current predictions of tiny recursive models. Task-specific rules select which cells or puzzles to supervise, while other targets are masked or downweighted. Experiments on Sudoku-Extreme, Maze-Hard, and ARC-AGI-1 show improved accuracy or faster convergence, supported by ablations and gradient analysis.

Critical strengths
  1. Natural idea with strong results. Matching supervision to the model’s current reasoning progress is intuitive and simple. AS-TRM consistently improves downstream accuracy across three recursion depths, by up to 7.9 points.
  2. Insightful analysis. The gradient and loss-landscape analyses help explain the benefits of adaptive supervision. Extended rollouts further show that AS-TRM better preserves correct solutions and continues improving with more iterations.
Critical weaknesses

The Maze comparison does not isolate adaptive supervision. AS-TRM also changes target paths and adds augmentation. Without augmentation, it scores 81.5%, below TRM’s 86.8% (Table 13). Although the paper acknowledges this interaction, a baseline (TRM) using the same targets and augmentation with full supervision is missing. (Overall, I think this is a solid paper, adding this baseline would make it more convincing.)

Questions for authors
  1. How does TRM perform on Maze-Hard when using the same model-consistent path-target construction and augmentation as AS-TRM, but with full supervision at every thinking step?
  2. How could the insights from AS-TRM inform the training of standard autoregressive LLMs?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.