acceptodds
Under review as a conference paper at ICLR 2027

Adaptive Soft Thinking for Controlling Candidate Influence Across Reasoning States

Abstract

Chain-of-Thought (CoT) reasoning in large language models (LLMs) proceeds through discrete token generation, requiring a single token to be selected at each step even when multiple alternatives remain plausible. Latent reasoning addresses this limitation through continuous intermediate representations, and Soft Thinking provides a training-free approach by constructing Concept Tokens as probability-weighted mixtures of multiple token candidates. However, our preliminary analysis shows that the effect of candidate integration varies across reasoning states and is partially reflected in the current state, motivating state-dependent control of Soft Thinking. Accordingly, we propose ST-Scheduler, a lightweight Soft Thinking scheduler that controls candidate influence through state-dependent Concept Width selection while keeping the underlying LLM frozen. We optimize ST-Scheduler with reinforcement learning with verifiable rewards, using final reasoning outcomes to learn Concept Width decisions throughout reasoning. Across multiple reasoning models and mathematical and scientific reasoning benchmarks, ST-Scheduler outperforms standard CoT, fixed-width Soft Thinking, and existing training-free Soft Thinking methods, while achieving higher solution coverage with fewer sampled trajectories. Code is available at https://anonymous.4open.science/r/ST-Scheduler-FC3F.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Reasoning, Latent Reasoning, Soft Thinking, RLVR

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_P4TnOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the limitations of using a fixed Concept Width in Soft Thinking. Soft Thinking constructs continuous Concept Tokens as probability-weighted mixtures of candidate token embeddings. Candidate integration can disrupt an otherwise correct trajectory in some states and provide useful information in others. The authors propose ST-Scheduler, a lightweight policy that uses the current hidden state to choose how many candidate tokens to integrate at each step while keeping the underlying LLM frozen. The scheduler is trained with GRPO using verifiable final-answer rewards. Experiments use Qwen3-1.7B, Qwen3-8B, and DeepSeek-R1-Distill-Llama-8B. On Qwen3-1.7B, ST-Scheduler achieves an average accuracy of 62.55% across five benchmarks, compared with 59.55% for CoT, 57.84% for fixed-width Soft Thinking, 60.80% for SwiR, and 60.95% for SeLaR. AIME24 Pass@1 reaches 50.67%, compared with 38.00% for fixed-width Soft Thinking. ST-Scheduler also achieves higher Pass@k under smaller sampling budgets.

Critical strengths

Turning fixed Concept Width into a state-dependent control policy is a clear core contribution. Freezing the LLM and training only the scheduler directly tests when candidate information should be integrated, rather than conflating the gain with retraining the underlying reasoning model.

Why this matters for the overall assessment: if the dynamic policy remains beneficial after controlling candidate count, temperature, and compute, it provides the main reason to accept the paper.

Critical weaknesses
    • The terminal reward provides weak credit assignment for step-level Concept Width decisions. On long reasoning trajectories, final correctness does not identify which states or width choices caused the outcome, and the current analysis does not establish that the scheduler learns a generalizable local rule.

    • Why this matters for the overall assessment: This weakens the mechanistic and reproducibility claims; if the scheduler depends on task-specific terminal rewards, both novelty and generalization may be overstated.

    • Robustness against neighboring soft/latent reasoning methods and decoding choices is under-evaluated. Performance may depend on candidate construction, logit distributions, temperature, candidate-set size, and the verifier, while the main evidence covers mathematical reasoning and three models.

    • Why this matters for the overall assessment: This directly limits the generality claim. Without cross-setting evidence, the result may be a specialized optimization within Soft Thinking rather than a broadly useful reasoning method.

Questions for authors
  1. Does the scheduler need to be retrained for different backbones, sampling temperatures, or candidate-set sizes? Could the authors report transfer across compatible settings and explain architectural constraints on such transfer?

  2. Which gains come from dynamic width selection, and which come from ST-scheduler trained with GRPO?

4 comments
AuthorsOpenReview ↗

Response #1

We thank the reviewer for providing feedback on our work. We address the main concerns through the following issues, supported by additional analyses and experimental results.

Issue 1. Can Terminal Rewards Support Effective State-Dependent Width Control?

We recognize that assigning accurate local credit to individual Concept Width decisions is an important challenge. As discussed in Section 4.3, the current policy exhibits asymmetric learning behavior due to the limitations of terminal-reward supervision. Specifically, the learned scheduler assigns 90.7% probability to n=1 at states where candidate integration changes a correct outcome into an incorrect one, but only 29.8% probability to n>1 at states where broader integration improves correctness.

This indicates that the scheduler learns to avoid harmful candidate integration relatively effectively, while identifying beneficial opportunities for broader integration remains more challenging. Such asymmetry is consistent with the long-tailed uncertainty distribution of reasoning trajectories, where low-entropy states are more frequent than exploratory states. We explicitly acknowledge this limitation in Section 4.3. Developing more informative local credit assignment is an important future direction, although the sequential dependence between reasoning states makes direct step-level supervision challenging.

Nevertheless, the current GRPO objective enables the scheduler to learn useful step-dependent width-selection behavior from final reasoning outcomes. To examine whether this learned policy provides benefits beyond simpler control rules, we additionally compare ST-Scheduler against fixed widths and a training-free Perplexity-based dynamic-width baseline on Qwen3-1.7B, to investigate whether a simple output-uncertainty heuristic is sufficient. At each reasoning step, this baseline estimates the effective candidate count using the perplexity of the current token distribution: $\ n_t=\operatorname{clip}\left(\operatorname{round}(\exp(H(p_t))),1,10\right). $

MethodGSM8KMATH500GPQAAIME24AIME25Avg.
CoT (Sampling)90.3191.0838.3843.3334.6759.55
Soft n=189.9291.6038.3842.6735.3359.58
Soft n=289.3691.6838.6944.6732.0059.28
Soft n=389.6790.0837.6841.3330.6757.89
Soft n=590.1890.2436.2634.0034.6757.07
Soft n=1090.1690.1635.5638.0035.3357.84
PPL-based Dynamic90.2792.2838.8940.0036.0059.49
ST-Scheduler91.2192.4039.9050.6738.6762.57

ST-Scheduler achieves 62.57% average accuracy, outperforming both the best fixed-width configuration (59.58%) and PPL-based dynamic control (59.49%) across all five benchmarks. Since the PPL-based baseline already adjusts Concept Width using the current output distribution, this comparison suggests that learning width-selection decisions from reasoning outcomes and richer internal representations provides benefits beyond a predefined uncertainty-based heuristic. These findings support that terminal-reward optimization can learn a useful state-dependent width-selection policy despite its limitations in local credit assignment.

AuthorsOpenReview ↗

Response #2

Issue 2. Does State-Dependent Width Control Generalize Across Decoding Configurations and Backbones?

Our additional experiments show that ST-Scheduler maintains effective reasoning performance under different decoding configurations without retraining, providing evidence that the learned policy is not restricted to the original token distribution or candidate-set size. To examine this, we conducted sensitivity experiments on Qwen3-1.7B, comparing ST-Scheduler against CoT and fixed-width Soft Thinking under identical variations in temperature and top-k. We kept the trained scheduler checkpoint fixed and evaluated how each method's accuracy changed on AIME24, AIME25, and MATH500.

Temperature robustness. We varied temperature from 0.6 to 1.2 to evaluate sensitivity to changes in the token probability distribution, as shown below:

DatasetMethodT=0.6T=0.8T=1.0T=1.2Range (pp) ↓
AIME24CoT43.3340.0046.6733.3313.34
Soft n=1038.0040.0030.0026.6713.33
ST-Scheduler50.6753.3353.3350.003.33
AIME25CoT34.6733.3336.6726.6710.00
Soft n=1035.3336.6733.3330.006.67
ST-Scheduler38.6740.0036.6736.673.33
MATH500CoT91.0890.0091.0089.601.48
Soft n=1090.1688.8091.4088.203.20
ST-Scheduler92.4091.4091.2091.801.20

ST-Scheduler exhibits the smallest accuracy range across all three benchmarks. In particular, its variation on AIME24 and AIME25 is only 3.33 percentage points, compared with 13.34/10.00 points for CoT and 13.33/6.67 points for fixed-width Soft Thinking. These results suggest that learned state-dependent width selection is less sensitive to changes in the output probability distribution, enabling the scheduler to maintain effective candidate integration without retraining.

Candidate-set robustness To examine the sensitivity to candidate-set size, we varied top-k from 5 to 30 while keeping the trained scheduler fixed without retraining, as shown below:

DatasetMethodTop-k=5Top-k=10Top-k=20Top-k=30Range (pp) ↓
AIME24CoT43.3340.0043.3333.3310.00
Soft n=1040.0036.6738.0033.336.67
ST-Scheduler53.3350.0053.3353.333.33
AIME25CoT40.0036.6734.6733.336.67
Soft n=1036.6730.0035.3333.336.67
ST-Scheduler40.0036.6738.6740.003.33
MATH500CoT91.6091.0091.0889.801.80
Soft n=1091.2090.0090.1689.601.60
ST-Scheduler91.8092.8092.4092.201.00

The accuracy range of ST-Scheduler is consistently smaller than those of CoT and fixed-width Soft Thinking across all three benchmarks. Specifically, ST-Scheduler exhibits ranges of only 3.33, 3.33, and 1.00 percentage points on AIME24, AIME25, and MATH500, respectively, compared with 10.00/6.67, 6.67/6.67, and 1.80/1.60 for CoT and fixed-width Soft Thinking. These results provide additional evidence that the learned state-dependent width-selection policy is less sensitive to changes in candidate-set size and maintains effective candidate integration beyond the original decoding configuration, without retraining.

Backbone compatibility. We train a separate scheduler for each backbone and do not claim cross-backbone transfer of a single checkpoint. Differences in hidden dimensions can prevent direct reuse, while matching dimensions alone does not ensure compatibility between representation spaces. Our three-backbone evaluation therefore establishes the applicability of the scheduling framework across model families, whereas the decoding-configuration experiments evaluate transfer of a fixed scheduler within the same backbone. Overall, the additional sensitivity analyses support that the learned state-dependent width-selection policy remains effective across multiple decoding configurations without retraining. The results provide evidence that adapting candidate integration to the current reasoning state offers greater robustness to changes in the token distribution than applying a fixed Concept Width, while maintaining strong performance across different candidate-set sizes.

AuthorsOpenReview ↗

Response #3

Issue 3. Does the Learned Scheduler Generalize Beyond Mathematical Reasoning?

Our additional cross-domain experiments show that the scheduler trained exclusively on mathematical reasoning remains effective on coding tasks without further training, supporting the generalizability of its state-dependent width-selection policy. To examine whether ST-Scheduler merely learns task-specific width preferences from mathematical rewards, we directly applied the Qwen3-1.7B scheduler trained on DAPO-Math-17K to HumanEval and LeetCode Easy, Medium, and Hard. Neither the backbone nor the scheduler was updated for the coding domain. ST-Scheduler achieves the highest average accuracy across all four coding benchmarks, outperforming the strongest baseline, SwiR, by 0.73, 0.44, 1.10, and 0.45 percentage points, respectively, as shown below:

MethodHumanEvalLeetCode-ELeetCode-MLeetCode-HAvg.
CoT86.34 ± 1.7696.00 ± 4.2763.96 ± 3.5129.09 ± 5.1868.85
Soft Thinking85.00 ± 0.9295.56 ± 0.0057.58 ± 1.2524.09 ± 3.4565.56
SwiR87.44 ± 1.4796.00 ± 3.3065.49 ± 1.2532.73 ± 2.5970.42
SeLaR79.15 ± 2.2288.00 ± 4.0453.41 ± 1.8421.36 ± 1.2460.48
ST-Scheduler88.17 ± 0.3396.44 ± 2.5366.59 ± 3.1733.18 ± 2.5971.10

The math-trained scheduler improves average coding accuracy from 70.42% to 71.10% without any coding-specific training or adaptation. These results provide evidence that ST-Scheduler learns a generalizable policy for controlling candidate integration based on the current reasoning state, rather than merely optimizing width selection for the mathematical tasks used during training.

Thanks for the author's detailed explanation and additional experiments. They address my questions and I therefore raise my score to 2. However, I remain concerned that the main contribution, which is turning a fixed concept width into a state-dependent control policy, does not meet the bar for ICLR. Replacing a fixed strategy with a learned control policy is a well-established approach in LLM research and deep learning more broadly.

Reviewer_gghMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ST-Scheduler, a learned controller for Soft Thinking. At each reasoning step, it uses the current hidden representations and width-specific Concept Tokens to select how many candidate token embeddings to combine. The underlying LLM remains frozen, while the scheduler is trained with GRPO using final-answer rewards on DAPO-Math-17K. Experiments on three backbones and five reasoning benchmarks show consistent improvements over CoT, fixed-width Soft Thinking, SwiR, and SeLaR. The paper also studies the effects of local width changes and the behavior of the learned scheduler.

Critical strengths
  1. The method is simple and provides a practical way to adapt Soft Thinking without updating the underlying LLM. Selecting Concept Width offers a single control over whether to use a discrete token and how many candidates to incorporate.
  2. The experiments provide evidence that the method is effective. Improvements are consistent across the evaluated backbones and benchmarks. The fixed-width comparisons in Appendix A.1 show that the gains cannot be recovered simply by choosing a better fixed width among the tested values. The single-step intervention study also supports the motivation that changing candidate integration can affect the final answer.
Critical weaknesses
  1. Related work and comparisons with trained methods are incomplete. HRPO (Hybrid Latent Reasoning via Reinforcement Learning, 2025) already uses outcome-based RL to learn gates that combine discrete token embeddings with continuous representations, but is not discussed. Soft Concept Mixing (2025) is also relevant. SofT-GRPO and Soft Tokens, Hard Truths are cited but not compared experimentally. These methods differ from the proposed frozen-backbone scheduler, but the paper should explain these differences and establish the benefit of its particular design. All baselines in Table 2 require no additional training, whereas ST-Scheduler is trained on DAPO-Math-17K. Keeping the backbone fixed does not remove this difference in supervision and training cost. Table 4 includes CoT+GRPO, but only for one backbone on two AIME benchmarks, leaving the comparison with trained alternatives limited.
  2. The gains over the stronger baselines are modest, and their practical value is unclear. In Table 2, the average improvements over the strongest baseline for each backbone are 1.60, 0.73, and 1.35 percentage points. Table 4 shows better coverage at smaller sampling budgets, but the Pass@16 results match several baselines. These are positive results, but the paper does not report training cost, actual generation lengths, or inference latency to show what is gained for the additional training and per-step scheduling. Paired uncertainty estimates would also help assess the reliability of the smaller improvements.
Questions for authors
  1. What is the main advantage of learning discrete Concept Width decisions over the learned gating used in HRPO? Please clarify the relationship to HRPO and Soft Concept Mixing.
  2. Can you provide a comparison with a representative trained soft-thinking method under comparable training data and computational budgets? A broader CoT+GRPO comparison would also help establish whether the gains extend beyond the two AIME settings in Table 4.
  3. What are the scheduler's trainable parameter count, training cost, actual generation lengths, and inference latency? Can you provide paired confidence intervals for the performance differences against the strongest baselines?
3 comments
AuthorsOpenReview ↗

Response #1

We thank the reviewer for providing feedback on our work. We address the main concerns through the following issues, supported by additional analyses and experimental results.

Issue 1. What Distinguishes ST-Scheduler from Existing Trained Latent Reasoning Methods?

Our primary distinction from HRPO and Soft Concept Mixing (SCM) lies in the research objective: rather than training LLMs to better perform latent reasoning, we investigate whether their existing reasoning capabilities can be better utilized by dynamically controlling candidate integration. HRPO and SCM leverage additional information from hidden representations and optimize the underlying LLM through reinforcement learning to improve latent reasoning. In contrast, as stated in Lines 89–90 of our Introduction, ST-Scheduler learns how to adapt the use of Soft Thinking across reasoning states, rather than training the LLM to acquire Soft Thinking-specific reasoning patterns.

Our preliminary analysis demonstrates that the effect of candidate integration varies across reasoning states, improving reasoning in some states while disrupting otherwise correct trajectories in others. This motivates our central question: when and how much information from candidate tokens discarded by discrete CoT decoding should be incorporated into reasoning? Unlike HRPO's learned representation gating or SCM's backbone adaptation, ST-Scheduler directly controls the number of candidate embeddings integrated into each Concept Token through Concept Width. Thus, our contribution is to demonstrate that the existing reasoning capabilities of an LLM can be better utilized through state-dependent control of candidate information, without modifying the underlying model.

Issue 2. Are the Improvements Worth the Additional Training and Inference Costs?

We acknowledge that ST-Scheduler requires additional training compared with training-free baselines. To assess whether this additional cost is justified, we further analyze its training and inference efficiency, statistical reliability, and performance relative to GRPO-trained reasoning.

Training efficiency. Unlike GRPO training that updates the underlying LLM, ST-Scheduler freezes the backbone and optimizes only the scheduler's projection layers and MLP. The estimated trainable parameter counts are approximately 1.77M (0.10%) for Qwen3-1.7B and 2.82M (0.035%) for the 8B backbones, representing only a small fraction of the corresponding LLM parameters. Furthermore, the observed saturation of validation accuracy at approximately 100 steps suggests that the scheduler can learn effective width-selection behavior relatively quickly, although rollout generation still incurs additional training cost.

Inference efficiency. ST-Scheduler reuses hidden representations from the backbone and applies lightweight projections and an MLP without requiring additional LLM forward passes. For a Transformer with L layers, hidden dimension d, and generation length T, the backbone computation is approximately O(L(T2d+Td2)), while the scheduler adds O(T[(L+N)dh+L(N+1)h2]), where h=256 and N=10. The additional computation is estimated to be only 0.6–1.1% of the backbone computation for a maximum generation length of 32K tokens, specifically 1.12% for Qwen3-1.7B, 0.60% for Qwen3-8B, and 0.57% for DeepSeek-R1-Distill-Llama-8B. These are architecture-based estimates rather than measured inference latency.

Generation length. We additionally analyzed generation lengths of correct and incorrect trajectories on AIME24. The results show no substantial differences in overall generation length across methods, indicating that the improvements of ST-Scheduler do not require longer reasoning trajectories, as shown below:

MethodCorrect Avg. TokensIncorrect Avg. TokensOverall Avg. Tokens
CoT12,47922,54817,648
Soft Thinking (n=3)13,82421,49518,324
SwiR12,56422,84117,908
SeLaR11,82923,71518,327
PPL-based11,70022,77818,642
ST-Scheduler12,51221,56717,360
AuthorsOpenReview ↗

Response #2

Statistical reliability. To examine the reliability of the modest accuracy improvements, we additionally analyzed the performance differences against the strongest baselines over five matched evaluation runs, including 95% confidence intervals. ST-Scheduler achieves positive mean improvements in all 15 model–benchmark settings, although the intervals reflect variability, particularly on smaller benchmarks such as AIME, as shown below:

BackboneBenchmarkStrongest BaselineΔ Accuracy (pp)95% CI (pp)
Qwen3-1.7BGSM8KSwiR+0.63[+0.02, +1.24]
MATH500SeLaR+0.60[-0.78, +1.98]
GPQASwiR+0.61[-1.30, +2.51]
AIME24SeLaR+3.33[-3.21, +9.88]
AIME25SeLaR+2.00[-5.52, +9.52]
Qwen3-8BGSM8KSoft n=10+0.09[-0.37, +0.55]
MATH500SwiR+0.40[-0.88, +1.68]
GPQASeLaR+0.81[-1.80, +3.42]
AIME24SeLaR+1.33[-0.93, +3.60]
AIME25SeLaR+0.67[-7.29, +8.63]
DeepSeek-R1-8BGSM8KSwiR+0.33[-0.51, +1.18]
MATH500SeLaR+0.80[-0.89, +2.49]
GPQASwiR+0.61[-5.15, +6.36]
AIME24SeLaR+2.00[-6.07, +10.07]
AIME25SwiR+2.00[-4.28, +8.28]

The intervals are calculated from run-level accuracy differences paired by evaluation seed and characterize variability across repeated evaluations.

Comparison with GRPO-trained reasoning. To further address the concern that ST-Scheduler benefits from additional training compared with training-free baselines, we extended the CoT+GRPO comparison to all five reasoning benchmarks on Qwen3-1.7B. Both methods were trained for 1,000 optimization steps, with checkpoints selected based on validation accuracy. While CoT+GRPO directly optimizes the backbone's reasoning policy, ST-Scheduler keeps the backbone frozen and learns only a policy over N=10 Concept Width actions. ST-Scheduler achieves 62.57% average accuracy, compared with 60.84% for CoT+GRPO, outperforming it on all five evaluated benchmarks, as shown below:

MethodGSM8KMATH500GPQAAIME24AIME25Avg.
CoT90.3191.0838.3843.3334.6759.55
CoT+GRPO90.5991.8838.3846.6736.6760.84
ST-Scheduler91.2192.4039.9050.6738.6762.57

Taken together, these analyses show that ST-Scheduler achieves consistent average performance improvements with a small trainable controller, limited additional inference computation, and comparable generation lengths. Furthermore, its competitive performance against a GRPO-trained backbone demonstrates that learning to control how existing candidate information is utilized can provide an effective means of improving reasoning performance without modifying the underlying LLM.

AuthorsOpenReview ↗

Response #3

Issue 3. Does the Learned Scheduler Generalize Beyond Mathematical Reasoning?

Our additional cross-domain experiments show that the scheduler trained exclusively on mathematical reasoning remains effective on coding tasks without further training, supporting the generalizability of its state-dependent width-selection policy. To examine whether ST-Scheduler merely learns task-specific width preferences from mathematical rewards, we directly applied the Qwen3-1.7B scheduler trained on DAPO-Math-17K to HumanEval and LeetCode Easy, Medium, and Hard. Neither the backbone nor the scheduler was updated for the coding domain. ST-Scheduler achieves the highest average accuracy across all four coding benchmarks, outperforming the strongest baseline, SwiR, by 0.73, 0.44, 1.10, and 0.45 percentage points, respectively, as shown below:

MethodHumanEvalLeetCode-ELeetCode-MLeetCode-HAvg.
CoT86.34 ± 1.7696.00 ± 4.2763.96 ± 3.5129.09 ± 5.1868.85
Soft Thinking85.00 ± 0.9295.56 ± 0.0057.58 ± 1.2524.09 ± 3.4565.56
SwiR87.44 ± 1.4796.00 ± 3.3065.49 ± 1.2532.73 ± 2.5970.42
SeLaR79.15 ± 2.2288.00 ± 4.0453.41 ± 1.8421.36 ± 1.2460.48
ST-Scheduler88.17 ± 0.3396.44 ± 2.5366.59 ± 3.1733.18 ± 2.5971.10

The math-trained scheduler improves average coding accuracy from 70.42% to 71.10% without any coding-specific training or adaptation. These results provide evidence that ST-Scheduler learns a generalizable policy for controlling candidate integration based on the current reasoning state, rather than merely optimizing width selection for the mathematical tasks used during training.

Reviewer_A6xcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ST-Scheduler, a method that dynamically controls the Concept Width in Soft Thinking according to the current reasoning state, optimized with GRPO. Experiment result shows consistent improvement over CoT and existing Soft Thinking baselines on mathematical reasoning across Qwen3-1.7B, Qwen3-8B, and DeepSeek-R1-Distill-Llama-8B models. Ablation studies demonstrate that adaptive width selection outperforms different fixed width configurations, and that the learned policy tends to select smaller widths in low-entropy states and larger widths in high-entropy states.

Critical strengths
  1. The motivation is clear and well-supported by preliminary analyses.
  2. The proposed method is simple and intuitive.
  3. ST-Scheduler shows consistent improvements over CoT and existing Soft Thinking methods across benchmarks and models.
  4. The paper provides ablations on different fixed Concept Widths and analyses of the learned policy.
  5. The paper provides discussions on the method's relationship with prediction entropy and candidate probability mass.
Critical weaknesses
  1. Several means in Table 2 do not appear consistent with averages over five runs. MATH500 has 500 problems, so a five-run mean must be a multiple of 0.04, yet ST-Scheduler reports 92.43 and 97.18. Overall, 10 of the 30 MATH500 and GPQA-Diamond means cannot arise from five runs, including 4 of ST-Scheduler's 6. In addition, SwiR's average on Qwen3-8B is printed as 78.4, although its five scores average 78.74. Could the authors check these values and explain how they were computed?
  2. The improvements are small and untested for significance. ST-Scheduler's lead over the best baseline is under one point in 10 of the 15 cells, and only 0.02 on GSM8K with Qwen3-8B. AIME has only 30 problems, so one problem is worth 3.3 points, and the largest lead in Table 2 (3.34) corresponds to about one problem per run. Could the authors report paired significance tests or per-problem results?
  3. The paper attributes the gain to state-dependent width selection, but no experiment keeps the training while removing the dependence on state, for example a trained global width or a simple entropy-threshold rule. The fixed-width comparison in Table 5 covers only widths 3, 5 and 10, and only on Qwen3-1.7B. Could the authors add a trained state-independent baseline, and include widths 1 and 2 across all three models?
Questions for authors

See weakness

3 comments
AuthorsOpenReview ↗

Response #1

We thank the reviewer for providing feedback on our work. We address the main concerns through the following issues, supported by additional analyses and experimental results.

Issue 1. Are the Reported Improvements Numerically Consistent and Reliable?

We addressed the numerical inconsistencies and further examined whether the observed performance improvements remain consistent across different backbones and benchmarks.

Numerical consistency. We identified errors in the reported entries of Table 2 and recalculated the results from the run-level correct-prediction counts. We retain full precision during aggregation and round only the final displayed values. To make these corrections directly verifiable, we provide the counts and corresponding five-run means and standard deviations below. MATH500 (500 problems per run)

BackboneMethodRun 1Run 2Run 3Run 4Run 5Mean ± Std. (%)
Qwen3-1.7BCoT456/500453/500455/500457/500456/50091.08 ± 0.30
Soft 3451/500448/500446/500455/500452/50090.08 ± 0.70
Soft 5448/500446/500455/500452/500455/50090.24 ± 0.82
Soft 10455/500452/500451/500446/500450/50090.16 ± 0.65
SwiR453/500466/500452/500461/500455/50091.48 ± 1.19
SeLaR457/500463/500456/500453/500466/50091.80 ± 1.07
ST-Scheduler464/500463/500466/500455/500462/50092.40 ± 0.84
Qwen3-8BCoT483/500486/500481/500484/500482/50096.64 ± 0.38
Soft 10483/500482/500487/500482/500483/50096.68 ± 0.41
SwiR478/500480/500485/500485/500492/50096.80 ± 1.09
SeLaR485/500492/500478/500480/500483/50096.72 ± 1.08
ST-Scheduler483/500484/500485/500492/500486/50097.20 ± 0.71
DeepSeek-R1-Distill-Llama-8BCoT457/500451/500449/500444/500451/50090.08 ± 0.93
Soft 10419/500424/500457/500461/500461/50088.88 ± 4.21
SwiR461/500419/500457/500460/500457/50090.16 ± 3.57
SeLaR449/500442/500457/500451/500460/50090.36 ± 1.41
ST-Scheduler461/500449/500451/500457/500461/50091.16 ± 1.12

GPQA-Diamond (198 problems per run)

BackboneMethodRun 1Run 2Run 3Run 4Run 5Mean ± Std. (%)
Qwen3-1.7BCoT75/19878/19874/19874/19879/19838.38 ± 1.18
Soft 370/19874/19876/19876/19877/19837.68 ± 1.41
Soft 568/19875/19874/19874/19868/19836.26 ± 1.76
Soft 1067/19874/19875/19868/19868/19835.56 ± 1.91
SwiR80/19880/19875/19880/19874/19839.29 ± 1.53
SeLaR76/19875/19874/19880/19876/19838.48 ± 1.15
ST-Scheduler80/19878/19877/19880/19880/19839.90 ± 0.71
Qwen3-8BCoT119/198117/198119/198121/198115/19859.70 ± 1.15
Soft 10114/198115/198115/198121/198115/19858.59 ± 1.43
SwiR116/198117/198121/198124/198115/19859.90 ± 1.91
SeLaR122/198115/198117/198121/198119/19860.00 ± 1.45
ST-Scheduler124/198122/198121/198119/198116/19860.81 ± 1.54
DeepSeek-R1-Distill-Llama-8BCoT72/19876/19885/19892/19866/19839.49 ± 5.23
Soft 1066/19882/19884/19872/19876/19838.38 ± 3.71
SwiR86/19882/19872/19885/19882/19841.11 ± 2.80
SeLaR65/19886/19886/19884/19882/19840.71 ± 4.48
ST-Scheduler92/19875/19885/19876/19885/19841.72 ± 3.58
AuthorsOpenReview ↗

Response #2

Statistical reliability. To address the concern regarding the modest accuracy gains, we additionally examined performance differences against the strongest baselines using five matched evaluation runs. We computed 95% confidence intervals from run-level accuracy differences paired by evaluation seed, as shown below:

BackboneGSM8KMATH500GPQAAIME24AIME25
Qwen3-1.7B+0.63 [0.02, 1.24]+0.60 [-0.78, 1.98]+0.61 [-1.30, 2.51]+3.33 [-3.21, 9.88]+2.00 [-5.52, 9.52]
Qwen3-8B+0.09 [-0.37, 0.55]+0.40 [-0.88, 1.68]+0.81 [-1.80, 3.42]+1.33 [-0.93, 3.60]+0.67 [-7.29, 8.63]
DeepSeek-R1-8B+0.33 [-0.51, 1.18]+0.80 [-0.89, 2.49]+0.61 [-5.15, 6.36]+2.00 [-6.07, 10.07]+2.00 [-4.28, 8.28]

Values represent mean accuracy improvements in percentage points, with 95% confidence intervals in brackets. The intervals characterize variability across the matched runs. The confidence intervals reflect substantial uncertainty in some settings, particularly on AIME with only 30 problems. Nevertheless, ST-Scheduler achieves positive mean improvements over the strongest baseline in all 15 model–benchmark settings, indicating that the observed benefits are not restricted to a particular backbone or benchmark. Taken together, the corrected results and additional uncertainty analysis strengthen the empirical reliability of our findings: even without updating the underlying LLM, controlling the utilization of existing candidate information consistently improves average reasoning performance across the evaluated settings.

AuthorsOpenReview ↗

Response #3

Issue 2. What Makes State-Dependent Width Control Effective?

Our central claim is that Concept Width should be dynamically controlled because the effect of candidate integration varies across reasoning states. As demonstrated in our preliminary analysis, incorporating additional candidates can improve reasoning in some states but disrupt otherwise correct trajectories in others.

To further substantiate this claim and address the reviewer's concern regarding the source of the improvements, we conducted additional ablation experiments on Qwen3-1.7B to examine two aspects: (1) the importance of dynamic width control and (2) the additional benefit of learned reasoning-state-dependent control.

(1) Dynamic width control. We extended the existing fixed-width comparison to n{1,2,3,5,10}, examining how different Concept Widths affect reasoning performance and whether a single global width is sufficient across reasoning tasks. Here, n=1 reduces to greedy discrete-token selection during reasoning, whereas the CoT baseline uses stochastic sampling.

(2) Reasoning-state-dependent control. We additionally introduced a training-free PPL-based dynamic-width baseline to investigate whether a simple output-uncertainty heuristic is sufficient. At each reasoning step, this baseline estimates the effective candidate count using the perplexity of the current token distribution: $\ n_t=\operatorname{clip}\left(\operatorname{round}(\exp(H(p_t))),1,10\right). $

Unlike ST-Scheduler, this baseline adjusts Concept Width using only the current output distribution, without learning a width-selection policy from reasoning outcomes. The results of these additional comparisons are shown below:

MethodGSM8KMATH500GPQAAIME24AIME25Avg.
CoT (Sampling)90.3191.0838.3843.3334.6759.55
Soft n=189.9291.6038.3842.6735.3359.58
Soft n=289.3691.6838.6944.6732.0059.28
Soft n=389.6790.0837.6841.3330.6757.89
Soft n=590.1890.2436.2634.0034.6757.07
Soft n=1090.1690.1635.5638.0035.3357.84
PPL-based Dynamic90.2792.2838.8940.0036.0059.49
ST-Scheduler91.2192.4039.9050.6738.6762.57

Dynamic width control. The results show that different fixed Concept Widths lead to different performance patterns across benchmarks, with no single width consistently performing best. For instance, n=1 achieves the highest average accuracy among fixed-width configurations (59.58%), while n=2 performs better on MATH500 and AIME24. This variation demonstrates the limitations of selecting a single global Concept Width and supports our key idea that candidate integration should be dynamically adjusted throughout reasoning.

Reasoning-state-dependent control. The PPL-based dynamic baseline achieves 59.49% average accuracy, compared with ST-Scheduler's 62.57%. Although it performs competitively on MATH500 (92.28% vs. 92.40%), a substantial gap remains on AIME24 (40.00% vs. 50.67%). These results highlight the limitations of dynamic width selection based solely on output entropy, which does not necessarily indicate whether integrating additional candidates is beneficial at the current reasoning state. This observation is consistent with our preliminary analysis in Section 3.2, where internal hidden representations provide more informative signals about the effects of candidate integration than output logits alone.

Together, these findings reinforce our central claim: the varying effects of candidate integration require dynamic Concept Width control, while effective width selection benefits from reasoning-state information beyond output uncertainty alone. By learning a state-conditioned width-selection policy, ST-Scheduler enables more effective utilization of existing candidate information without modifying the underlying LLM.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.