- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies the limitations of using a fixed Concept Width in Soft Thinking. Soft Thinking constructs continuous Concept Tokens as probability-weighted mixtures of candidate token embeddings. Candidate integration can disrupt an otherwise correct trajectory in some states and provide useful information in others. The authors propose ST-Scheduler, a lightweight policy that uses the current hidden state to choose how many candidate tokens to integrate at each step while keeping the underlying LLM frozen. The scheduler is trained with GRPO using verifiable final-answer rewards. Experiments use Qwen3-1.7B, Qwen3-8B, and DeepSeek-R1-Distill-Llama-8B. On Qwen3-1.7B, ST-Scheduler achieves an average accuracy of 62.55% across five benchmarks, compared with 59.55% for CoT, 57.84% for fixed-width Soft Thinking, 60.80% for SwiR, and 60.95% for SeLaR. AIME24 Pass@1 reaches 50.67%, compared with 38.00% for fixed-width Soft Thinking. ST-Scheduler also achieves higher Pass@k under smaller sampling budgets.
Critical strengths
Turning fixed Concept Width into a state-dependent control policy is a clear core contribution. Freezing the LLM and training only the scheduler directly tests when candidate information should be integrated, rather than conflating the gain with retraining the underlying reasoning model.
Why this matters for the overall assessment: if the dynamic policy remains beneficial after controlling candidate count, temperature, and compute, it provides the main reason to accept the paper.
Critical weaknesses
-
-
The terminal reward provides weak credit assignment for step-level Concept Width decisions. On long reasoning trajectories, final correctness does not identify which states or width choices caused the outcome, and the current analysis does not establish that the scheduler learns a generalizable local rule.
-
Why this matters for the overall assessment: This weakens the mechanistic and reproducibility claims; if the scheduler depends on task-specific terminal rewards, both novelty and generalization may be overstated.
-
-
-
Robustness against neighboring soft/latent reasoning methods and decoding choices is under-evaluated. Performance may depend on candidate construction, logit distributions, temperature, candidate-set size, and the verifier, while the main evidence covers mathematical reasoning and three models.
-
Why this matters for the overall assessment: This directly limits the generality claim. Without cross-setting evidence, the result may be a specialized optimization within Soft Thinking rather than a broadly useful reasoning method.
-
Questions for authors
-
Does the scheduler need to be retrained for different backbones, sampling temperatures, or candidate-set sizes? Could the authors report transfer across compatible settings and explain architectural constraints on such transfer?
-
Which gains come from dynamic width selection, and which come from ST-scheduler trained with GRPO?
4 comments
Response #2
Issue 2. Does State-Dependent Width Control Generalize Across Decoding Configurations and Backbones?
Our additional experiments show that ST-Scheduler maintains effective reasoning performance under different decoding configurations without retraining, providing evidence that the learned policy is not restricted to the original token distribution or candidate-set size. To examine this, we conducted sensitivity experiments on Qwen3-1.7B, comparing ST-Scheduler against CoT and fixed-width Soft Thinking under identical variations in temperature and top-k. We kept the trained scheduler checkpoint fixed and evaluated how each method's accuracy changed on AIME24, AIME25, and MATH500.
Temperature robustness. We varied temperature from 0.6 to 1.2 to evaluate sensitivity to changes in the token probability distribution, as shown below:
| Dataset | Method | T=0.6 | T=0.8 | T=1.0 | T=1.2 | Range (pp) ↓ |
|---|---|---|---|---|---|---|
| AIME24 | CoT | 43.33 | 40.00 | 46.67 | 33.33 | 13.34 |
| Soft n=10 | 38.00 | 40.00 | 30.00 | 26.67 | 13.33 | |
| ST-Scheduler | 50.67 | 53.33 | 53.33 | 50.00 | 3.33 | |
| AIME25 | CoT | 34.67 | 33.33 | 36.67 | 26.67 | 10.00 |
| Soft n=10 | 35.33 | 36.67 | 33.33 | 30.00 | 6.67 | |
| ST-Scheduler | 38.67 | 40.00 | 36.67 | 36.67 | 3.33 | |
| MATH500 | CoT | 91.08 | 90.00 | 91.00 | 89.60 | 1.48 |
| Soft n=10 | 90.16 | 88.80 | 91.40 | 88.20 | 3.20 | |
| ST-Scheduler | 92.40 | 91.40 | 91.20 | 91.80 | 1.20 |
ST-Scheduler exhibits the smallest accuracy range across all three benchmarks. In particular, its variation on AIME24 and AIME25 is only 3.33 percentage points, compared with 13.34/10.00 points for CoT and 13.33/6.67 points for fixed-width Soft Thinking. These results suggest that learned state-dependent width selection is less sensitive to changes in the output probability distribution, enabling the scheduler to maintain effective candidate integration without retraining.
Candidate-set robustness To examine the sensitivity to candidate-set size, we varied top-k from 5 to 30 while keeping the trained scheduler fixed without retraining, as shown below:
| Dataset | Method | Top-k=5 | Top-k=10 | Top-k=20 | Top-k=30 | Range (pp) ↓ |
|---|---|---|---|---|---|---|
| AIME24 | CoT | 43.33 | 40.00 | 43.33 | 33.33 | 10.00 |
| Soft n=10 | 40.00 | 36.67 | 38.00 | 33.33 | 6.67 | |
| ST-Scheduler | 53.33 | 50.00 | 53.33 | 53.33 | 3.33 | |
| AIME25 | CoT | 40.00 | 36.67 | 34.67 | 33.33 | 6.67 |
| Soft n=10 | 36.67 | 30.00 | 35.33 | 33.33 | 6.67 | |
| ST-Scheduler | 40.00 | 36.67 | 38.67 | 40.00 | 3.33 | |
| MATH500 | CoT | 91.60 | 91.00 | 91.08 | 89.80 | 1.80 |
| Soft n=10 | 91.20 | 90.00 | 90.16 | 89.60 | 1.60 | |
| ST-Scheduler | 91.80 | 92.80 | 92.40 | 92.20 | 1.00 |
The accuracy range of ST-Scheduler is consistently smaller than those of CoT and fixed-width Soft Thinking across all three benchmarks. Specifically, ST-Scheduler exhibits ranges of only 3.33, 3.33, and 1.00 percentage points on AIME24, AIME25, and MATH500, respectively, compared with 10.00/6.67, 6.67/6.67, and 1.80/1.60 for CoT and fixed-width Soft Thinking. These results provide additional evidence that the learned state-dependent width-selection policy is less sensitive to changes in candidate-set size and maintains effective candidate integration beyond the original decoding configuration, without retraining.
Backbone compatibility. We train a separate scheduler for each backbone and do not claim cross-backbone transfer of a single checkpoint. Differences in hidden dimensions can prevent direct reuse, while matching dimensions alone does not ensure compatibility between representation spaces. Our three-backbone evaluation therefore establishes the applicability of the scheduling framework across model families, whereas the decoding-configuration experiments evaluate transfer of a fixed scheduler within the same backbone. Overall, the additional sensitivity analyses support that the learned state-dependent width-selection policy remains effective across multiple decoding configurations without retraining. The results provide evidence that adapting candidate integration to the current reasoning state offers greater robustness to changes in the token distribution than applying a fixed Concept Width, while maintaining strong performance across different candidate-set sizes.
Response #3
Issue 3. Does the Learned Scheduler Generalize Beyond Mathematical Reasoning?
Our additional cross-domain experiments show that the scheduler trained exclusively on mathematical reasoning remains effective on coding tasks without further training, supporting the generalizability of its state-dependent width-selection policy. To examine whether ST-Scheduler merely learns task-specific width preferences from mathematical rewards, we directly applied the Qwen3-1.7B scheduler trained on DAPO-Math-17K to HumanEval and LeetCode Easy, Medium, and Hard. Neither the backbone nor the scheduler was updated for the coding domain. ST-Scheduler achieves the highest average accuracy across all four coding benchmarks, outperforming the strongest baseline, SwiR, by 0.73, 0.44, 1.10, and 0.45 percentage points, respectively, as shown below:
| Method | HumanEval | LeetCode-E | LeetCode-M | LeetCode-H | Avg. |
|---|---|---|---|---|---|
| CoT | 86.34 ± 1.76 | 96.00 ± 4.27 | 63.96 ± 3.51 | 29.09 ± 5.18 | 68.85 |
| Soft Thinking | 85.00 ± 0.92 | 95.56 ± 0.00 | 57.58 ± 1.25 | 24.09 ± 3.45 | 65.56 |
| SwiR | 87.44 ± 1.47 | 96.00 ± 3.30 | 65.49 ± 1.25 | 32.73 ± 2.59 | 70.42 |
| SeLaR | 79.15 ± 2.22 | 88.00 ± 4.04 | 53.41 ± 1.84 | 21.36 ± 1.24 | 60.48 |
| ST-Scheduler | 88.17 ± 0.33 | 96.44 ± 2.53 | 66.59 ± 3.17 | 33.18 ± 2.59 | 71.10 |
The math-trained scheduler improves average coding accuracy from 70.42% to 71.10% without any coding-specific training or adaptation. These results provide evidence that ST-Scheduler learns a generalizable policy for controlling candidate integration based on the current reasoning state, rather than merely optimizing width selection for the mathematical tasks used during training.
Thanks for the author's detailed explanation and additional experiments. They address my questions and I therefore raise my score to 2. However, I remain concerned that the main contribution, which is turning a fixed concept width into a state-dependent control policy, does not meet the bar for ICLR. Replacing a fixed strategy with a learned control policy is a well-established approach in LLM research and deep learning more broadly.
Response #1
We thank the reviewer for providing feedback on our work. We address the main concerns through the following issues, supported by additional analyses and experimental results.
Issue 1. Can Terminal Rewards Support Effective State-Dependent Width Control?
We recognize that assigning accurate local credit to individual Concept Width decisions is an important challenge. As discussed in Section 4.3, the current policy exhibits asymmetric learning behavior due to the limitations of terminal-reward supervision. Specifically, the learned scheduler assigns 90.7% probability to n=1 at states where candidate integration changes a correct outcome into an incorrect one, but only 29.8% probability to n>1 at states where broader integration improves correctness.
This indicates that the scheduler learns to avoid harmful candidate integration relatively effectively, while identifying beneficial opportunities for broader integration remains more challenging. Such asymmetry is consistent with the long-tailed uncertainty distribution of reasoning trajectories, where low-entropy states are more frequent than exploratory states. We explicitly acknowledge this limitation in Section 4.3. Developing more informative local credit assignment is an important future direction, although the sequential dependence between reasoning states makes direct step-level supervision challenging.
Nevertheless, the current GRPO objective enables the scheduler to learn useful step-dependent width-selection behavior from final reasoning outcomes. To examine whether this learned policy provides benefits beyond simpler control rules, we additionally compare ST-Scheduler against fixed widths and a training-free Perplexity-based dynamic-width baseline on Qwen3-1.7B, to investigate whether a simple output-uncertainty heuristic is sufficient. At each reasoning step, this baseline estimates the effective candidate count using the perplexity of the current token distribution: $\ n_t=\operatorname{clip}\left(\operatorname{round}(\exp(H(p_t))),1,10\right). $
ST-Scheduler achieves 62.57% average accuracy, outperforming both the best fixed-width configuration (59.58%) and PPL-based dynamic control (59.49%) across all five benchmarks. Since the PPL-based baseline already adjusts Concept Width using the current output distribution, this comparison suggests that learning width-selection decisions from reasoning outcomes and richer internal representations provides benefits beyond a predefined uncertainty-based heuristic. These findings support that terminal-reward optimization can learn a useful state-dependent width-selection policy despite its limitations in local credit assignment.