acceptodds
Under review as a conference paper at ICLR 2027

Learning Adaptive Reasoning Threads for Mathematical Problem Solving

Abstract

Long chain-of-thought (CoT) has substantially improved mathematical reasoning by giving models more computation to develop and refine their solutions. Yet complex problem solving is not naturally linear: some intermediate steps may require extensive reasoning, while later steps need only their conclusions. Linear CoT keeps all such computation in a single growing context, where obsolete details, checks, and failed attempts can interfere with subsequent reasoning. We therefore ask whether a single LLM can learn to selectively develop intermediate derivations separately and reincorporate their conclusions when needed. We introduce Adaptive Reasoning Threads (ART), an on-demand, state-driven protocol that lets the model selectively extend reasoning from its evolving main-thread state into private local threads. Each extension inherits the problem and reasoning accumulated so far, continues with the same model policy, and returns only its conclusion when needed. We learn this behavior through supervised fine-tuning followed by outcome-based reinforcement learning. Across model scales and competition-mathematics benchmarks, ART consistently outperforms matched linear-CoT baselines, including longer-CoT controls with comparable total generation budgets. Simply exposing the ART interface is insufficient to reproduce these gains, showing that effective extension behavior must itself be learned. Moreover, ART-trained models remain stronger under standard CoT decoding and exhibit substantially fewer corrections and restarts. These results suggest that learning when to extend local reasoning can improve mathematical problem solving beyond simply increasing reasoning length.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Mathematical Reasoning, Large Language Models, Adaptive Reasoning, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Ev4COpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Adaptive Reasoning Threads (ART): a single LLM keeps a main thread as the persistent state of the overall solution and, when a local issue needs deeper work, unfolds it into a private extension that inherits the problem and the reasoning accumulated so far and returns only its conclusion. The behavior is acquired through supervised fine-tuning followed by outcome-based reinforcement learning with main-thread-centered optimization and ART completion shaping. On three competition-mathematics benchmarks with two model sizes, ART outperforms matched linear-CoT baselines, including longer-CoT controls with comparable total generation budgets, and the gains persist under ordinary CoT decoding.

Critical strengths
  1. The motivation is clear and the protocol is specified precisely; the distinction between a persistent main thread and temporary extensions is a natural framing of the problem.
  2. The controls are thorough: cross-evaluating training recipes and inference modes separates the effect of the interface from the learned behavior, and the longer-CoT control with a closely matched total generation budget together with the attention-interaction analysis excludes the "additional computation" explanation.
  3. The observation that the learned organization transfers to ordinary CoT decoding, with fewer correction markers, Wait markers and restarts, is a valuable result.
Critical weaknesses
  1. Sec. 3.4 states three differences from prior work — state inheritance, a persistent main thread, and main-thread-centered optimization — and the all-token GRPO ablation in Tab. 5 supports the third, but none of them is validated against the two closest methods. APR (Pan et al., 2025) already unfolds a child thread that returns only a summary, and Context-Folding (Sun et al., 2025) learns branching and returning with RL alone, its prompted agent already performing branch-and-fold without training; describing this line as taking a "structure-predefined" view, and citing each work in a single clause with no comparison, leaves the claimed difference asserted rather than shown.
  2. The reported effects are of the same order as the measurement uncertainty. The benchmarks are small, the sampling noise of mean@16 alone is comparable to the headline gains, and no confidence intervals or significance tests are given. Evidence also comes from one domain and one model family.
  3. All reported extension-use rates carry the shaping reward: under outcome-only optimization use collapses to 0.2% (Tab. 4), so the model's natural usage at a workable level, and the accuracy it would reach there, remain unobserved. The interaction in Tab. 2 is also unexplained: the Python tool adds about eight to eleven points under ART, while under CoT it is flat to slightly negative.
Questions for authors

Please refer to Weaknesses.

Reviewer_DdSQOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Adaptive Reasoning Threads (ART), a reasoning protocol in which a model maintains a persistent main thread and uses extend and retrieve actions to create and access local reasoning threads on demand (§3.2.2). Each local thread inherits the main-thread context at creation, independently addresses a designated subproblem, and returns only a concise conclusion to the main thread. The authors first use supervised fine-tuning (ART-SFT) to teach the interaction protocol, followed by reinforcement learning (ART-GRPO) to optimize extension and retrieval decisions using task-outcome rewards and protocol-completion shaping rewards (§§3.3.1–3.3.2).

The authors evaluate Qwen3.5-4B and Qwen3.5-9B on AIME 2025, AIME 2026, and HMMT February 2026 (§4.1). The full ART pipeline outperforms matched linear-CoT training baselines (§4.2, Table 1). The paper also presents comparisons across training and inference modes, longer-CoT controls with comparable generation budgets (§4.4, Table 7), and reward and design ablations (§4.3, Tables 4–5). The authors argue that these results demonstrate that models can learn to organize local computation based on their evolving reasoning states.

Critical strengths

S1. Empirical improvements from the complete pipeline. Table 1 (§4.2) shows that ART-SFT + ART-GRPO outperforms the matched CoT-SFT + GRPO pipeline at both 4B and 9B scales. Table 7 (§4.4) further demonstrates that ART maintains an advantage over longer-CoT controls under comparable total generation budgets.

S2. A meaningful research question and informative cross-mode comparisons. The on-demand local reasoning extensions introduced in §3.2 address a worthwhile research direction. Table 1 (§4.2) also shows that ART-trained models perform well under standard CoT inference, suggesting that some benefits of ART training may transfer to conventional decoding.

Critical weaknesses

W1. Insufficient validation of conclusion-only retrieval. The retrieve action (§3.2.2) returns only the local conclusion, potentially omitting important intermediate premises or conditions. However, the design ablations in Table 5 (§4.3) examine state inheritance and the scope of GRPO updates rather than the information preserved during retrieval. Consequently, the experiments do not directly establish whether concise conclusions retain sufficient information or whether isolating local reasoning details reduces downstream interference.

W2. Limited evidence for the independent inference-time benefits and adaptivity of ART. Table 1 (§4.2) shows relatively small and inconsistent incremental gains from ART inference over CoT inference on the same ART-trained models, including decreases in pass@16 in some settings. Furthermore, the shaping rewards in §3.3.2 explicitly encourage completed extension interactions, while Tables 3–4 (§4.3) do not establish whether models learn when extensions are beneficial, rather than merely using extensions more frequently.

Questions for authors

Q1. Why is returning only the conclusion an appropriate information boundary?

Section 3.2.2 separates the local reasoning trace from the returned conclusion, while Appendix C.2 restricts <conclusion> to 1–3 sentences. However, Table 5 (§4.3) compares w/o state inheritance and w/ all-token GRPO without directly examining the content returned by local threads. Moreover, the correction, "Wait," and restart frequencies in Table 6 (§4.4) are measured under standard CoT inference and therefore cannot independently demonstrate that conclusion-only retrieval reduces interference.

Could the authors compare conclusion-only returns, returns preserving key premises and applicability conditions, and full-derivation returns, while reporting their respective context and computational costs? Are there cases where omitted intermediate conditions lead to downstream reasoning errors?

Q2. How do the authors explain the limited and inconsistent incremental gains from ART inference?

For the final Qwen3.5-9B (ART-SFT + ART-GRPO) model, switching from CoT to ART inference increases average mean@16 from 70.6% to 71.7%, but decreases average pass@16 from 90.6% to 88.5% (Table 1, §4.2). On AIME 2026, pass@16 decreases from 96.7% to 93.3%, and on HMMT 2026, from 81.8% to 78.8%.

Furthermore, the incremental pass@16 advantage of ART inference over CoT inference decreases from +7.6 to +3.1 percentage points for the 4B model and from +3.3 to −2.1 points for the 9B model when moving from ART-SFT to ART-GRPO. These differences alone do not establish statistically significant regressions.

Could the authors provide per-problem success rates, repeated evaluations, or uncertainty estimates to assess the robustness of these differences? How do they explain why improvements in mean accuracy do not consistently translate into greater problem coverage?

Q3. Does the model genuinely learn when to extend reasoning?

Equation (8) (§3.3.2) assigns +0.1 for a completed extension–retrieval interaction and −0.2 for a rollout without extensions. After ART-GRPO, extension-use rates reach 77.3% and 75.3% for the 4B and 9B models, respectively (Table 3, §4.3). Table 4 further shows that both extension usage and performance are sensitive to reward shaping.

However, these results do not distinguish adaptive, reasoning-state-dependent extension decisions from increased extension usage driven by reward incentives. Under comparable extension counts, usage rates, and computational budgets, could the authors compare learned extension decisions against fixed-position or random-position extension policies to establish their advantage over simpler alternatives?

Reviewer_Df2qOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Adaptive Reasoning Threads (ART), a framework that enables language models to dynamically organize mathematical reasoning into a main thread and private reasoning extensions. Unlike conventional linear Chain-of-Thought (CoT), ART allows the model to initiate local reasoning threads that inherit the current reasoning state and return only their conclusions to the main thread. The framework is trained through two stages: supervised fine-tuning (ART-SFT) using teacher-generated trajectories, followed by reinforcement learning (ART-GRPO) with outcome-based rewards and protocol-completion shaping. Experiments on Qwen3.5-4B and 9B across AIME 2025, AIME 2026, and HMMT 2026 demonstrate consistent improvements over CoT baselines. The authors further investigate reasoning-budget controls, architectural ablations, reasoning patterns, and integration with Python tools.

Critical strengths
  1. A flexible and intuitive framework for organizing complex reasoning. ART addresses a meaningful limitation of linear CoT by separating detailed intermediate computations from the main reasoning context. Its state-conditioned extension mechanism allows reasoning structures to emerge dynamically, rather than requiring a predefined decomposition. The framework is conceptually straightforward, compatible with existing tool-calling interfaces, and potentially applicable beyond mathematical reasoning.

  2. Consistent empirical improvements supported by multiple analyses. ART improves average mean@16 accuracy from 63.7% to 66.8% for the 4B model and from 68.5% to 71.7% for the 9B model. The improvements persist against longer-CoT controls with comparable generation budgets. Ablations further support the importance of state inheritance and the proposed training objective. The substantial improvements obtained when integrating Python tools also suggest that the framework can accommodate external computation.

Critical weaknesses
  1. Insufficient comparisons to isolate the benefits of adaptive reasoning. Although the paper compares ART against linear CoT, it lacks comparisons with alternative structured reasoning approaches, such as fixed subtask decomposition or context-compression methods under matched training and inference budgets. Furthermore, ART-GRPO explicitly penalizes trajectories without extensions, raising questions about how much of the improvement comes from learning adaptive extension decisions versus encouraging structured reasoning more generally. This limits the evidence supporting the central claim that adaptive reasoning organization is responsible for the observed gains.

  2. Limited evaluation scope and incomplete computational analysis. The evaluation focuses exclusively on competition mathematics, using two model sizes and three benchmarks. The improvements over the strongest CoT baselines are relatively modest, and statistical uncertainty is not reported. Although the paper controls for generated token counts and provides analytical attention-cost estimates, actual inference latency, runtime overhead, and tool-use costs are not systematically evaluated. These omissions make it difficult to assess the robustness and practical efficiency of ART.

Questions for authors
  1. How are ART-SFT and CoT-SFT training trajectories matched in terms of retained examples, total training tokens, and teacher-generated supervision? Could differences in supervision quantity or quality contribute to the observed improvements?

  2. How does ART compare against a fixed-decomposition baseline that uses the same private-thread interface but follows a predetermined extension strategy? This comparison would help establish whether adaptive extension decisions provide benefits beyond structured reasoning itself.

  3. Can the authors provide statistical confidence intervals and empirical inference-cost measurements, including latency, number of model calls, and Python tool usage? These measurements would clarify whether the accuracy improvements remain meaningful under comparable computational resources.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.