- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes AGC-ST, a training-free decoding method that adaptively determines when to apply soft-token mixing and how many candidate tokens to include. The method combines an entropy threshold with the probability gap between the top two candidates, activating soft mixing at positions where the distribution is sufficiently uncertain and no candidate clearly dominates. At selected positions, it uses the Rényi effective number to set a support-size cap, followed by a cumulative-probability rule to determine the final mixing support. Other positions retain standard discrete sampling. The authors evaluate the method on five reasoning models across five benchmarks covering mathematical, scientific, and coding tasks. With a shared hyperparameter configuration, AGC-ST achieves the highest accuracy among the evaluated methods in all 25 model–benchmark settings.
Critical strengths
-
The method achieves consistent accuracy improvements across five models and five benchmarks, obtaining the highest accuracy among the evaluated methods in all 25 model–benchmark settings.
-
The appendix provides statistical support for the reported improvements. On the five-benchmark mean, the 95% paired bootstrap confidence intervals exclude zero in 34 of the 35 model–baseline comparisons, including comparisons with CoT sampling for all five models.
Critical weaknesses
-
Although Table 2 shows that the tightest gate is not the best-performing setting, the ablations do not fully separate the effect of position selection from that of mixing frequency. Activation-rate-matched entropy-only, margin-only, or random controls would help establish the specific benefit of the joint criterion.
-
The comparison with closely related selective Soft Thinking methods is incomplete. Although SeLaR and SwiReasoning are evaluated, ThinkRouter is discussed but not included in the experiments. Since ThinkRouter also routes between soft and discrete decoding based on model confidence, this omission limits the evidence for AGC-ST’s advantage over existing routing strategies.
Questions for authors
-
Appendix E.3 explicitly states that the interventions characterize one-step behavior rather than accuracy. Beyond the aggregate support-rule comparisons in Table 3, is there evidence linking these local prediction changes to improved final-answer correctness?
-
Does the benefit of joint gating come from selecting better positions or simply applying soft mixing less often? Does it still outperform entropy-only gating or random selection when their activation rates are matched?
-
Could the authors include a comparison with ThinkRouter, or explain why it was not included?
-
Table 5 uses top-10 mixing and fixed Cold Stop settings for vanilla Soft Thinking, whereas the original paper reports model-dependent top-n settings and searches over Cold Stop parameters. How were these baseline settings selected, and would model-specific tuning change the comparison?
5 comments
Response to Reviewer XXZQ (2/3)
W2 & Q3. Comparison with ThinkRouter
"Although SeLaR and SwiReasoning are evaluated, ThinkRouter is discussed but not included in the experiments. Since ThinkRouter also routes between soft and discrete decoding based on model confidence, this omission limits the evidence for AGC-ST's advantage over existing routing strategies."
"Could the authors include a comparison with ThinkRouter, or explain why it was not included?"
Thank you for this suggestion. To test the advantage of AGC-ST over existing routing strategies, we added ThinkRouter to all 25 model–benchmark settings. With ThinkRouter included, AGC-ST still achieves the highest accuracy among the evaluated methods in all 25 settings.
ThinkRouter uses top-10 soft mixing when the top-token probability is at least a threshold , and discrete sampling otherwise. We re-implemented it in our framework following its public code, with the decoding settings, token budget and evaluation protocol of Tab. 1. The thresholds follow the values reported in the ThinkRouter paper where available. Otherwise, we transferred them from the Qwen3 model of the closest size. LCB uses the MBPP value. ThinkRouter thus uses a separate threshold for each benchmark and model size. AGC-ST uses one configuration for all 25 settings.
Table XXZQ-2. ThinkRouter and AGC-ST across five models and five benchmarks (Avg@16 on AIME, Avg@4 otherwise).
| Model | Method | AIME2024 | AIME2025 | GPQA-D | MBPP | LCB | Mean |
|---|---|---|---|---|---|---|---|
| DS-Qwen-7B | ThinkRouter | 54.17 | 39.79 | 49.62 | 85.99 | 38.89 | 53.69 |
| AGC-ST | 56.88 | 42.50 | 50.63 | 87.45 | 39.61 | 55.41 | |
| DS-Llama-8B | ThinkRouter | 43.33 | 30.63 | 44.32 | 84.92 | 37.63 | 48.17 |
| AGC-ST | 44.17 | 32.29 | 46.46 | 86.38 | 39.70 | 49.80 | |
| Qwen3-8B | ThinkRouter | 74.38 | 64.38 | 58.33 | 94.07 | 56.27 | 69.48 |
| AGC-ST | 77.50 | 70.00 | 61.11 | 94.84 | 59.95 | 72.68 | |
| QwQ-32B | ThinkRouter | 77.50 | 63.75 | 61.62 | 97.57 | 62.10 | 72.51 |
| AGC-ST | 79.38 | 70.21 | 65.66 | 97.76 | 63.35 | 75.27 | |
| Qwen3-32B | ThinkRouter | 81.25 | 69.38 | 64.39 | 95.91 | 60.22 | 74.23 |
| AGC-ST | 81.88 | 74.38 | 66.92 | 96.50 | 66.04 | 77.14 |
Table XXZQ-3. Paired comparison of AGC-ST and ThinkRouter on the five-benchmark mean. Δ is AGC-ST minus ThinkRouter.
| Model | AGC-ST Mean | ThinkRouter Mean | Δ [95% CI] |
|---|---|---|---|
| DS-Qwen-7B | 55.41 | 53.69 | +1.72 [+0.08, +3.33] |
| DS-Llama-8B | 49.80 | 48.17 | +1.63 [+0.13, +3.16] |
| Qwen3-8B | 72.68 | 69.48 | +3.20 [+1.88, +4.57] |
| QwQ-32B | 75.27 | 72.51 | +2.76 [+1.65, +3.94] |
| Qwen3-32B | 77.14 | 74.23 | +2.91 [+1.65, +4.20] |
AGC-ST is higher than ThinkRouter in all 25 settings. On the five-benchmark mean, it exceeds ThinkRouter by 1.63–3.20 points. The paired bootstrap intervals (same procedure as App. E.1) exclude zero for all five models. So the advantage of AGC-ST also holds against an existing confidence-based routing strategy. We have added ThinkRouter to Tab. 1, its settings to App. D.4 (Tabs. 5 and 6), and its paired comparisons to Tab. 9.
Response to Reviewer XXZQ (3/3)
Q1. Local prediction changes and final-answer correctness
"Appendix E.3 explicitly states that the interventions characterize one-step behavior rather than accuracy. Beyond the aggregate support-rule comparisons in Table 3, is there evidence linking these local prediction changes to improved final-answer correctness?"
Thank you for this insightful question. To examine this link, we applied the substitution from App. E.3 in full rollouts. Gated steps receive the top-1 embedding instead of the mixture. This lowers final-answer accuracy for all three models tested on AIME2024/2025.
In App. E.3, the AGC-ST mixture and the top-1 input give a different next-step argmax at about 47% of the same gated states. Here the joint gate and the sampling at non-gated steps are unchanged. Only the input at gated steps differs: the top-1 embedding, a fixed top-2 mixture, or the AGC-ST mixture. We used DS-Qwen-7B, DS-Llama-8B and QwQ-32B on AIME2024 and AIME2025.
Table XXZQ-4. Final-answer accuracy with different gated-step inputs on AIME2024/2025, with the joint gate fixed (Avg@16). Same runs as Table rd2a-1. CoT is shown for reference.
| Setting | DS-Qwen-7B AIME2024 | DS-Qwen-7B AIME2025 | DS-Llama-8B AIME2024 | DS-Llama-8B AIME2025 | QwQ-32B AIME2024 | QwQ-32B AIME2025 | Mean |
|---|---|---|---|---|---|---|---|
| CoT | 54.79 | 40.63 | 41.88 | 31.04 | 77.50 | 69.58 | 52.57 |
| Fixed size (top-1, argmax) | 55.21 | 39.38 | 43.12 | 30.42 | 77.29 | 67.71 | 52.19 |
| Fixed size (top-2) | 56.25 | 40.62 | 43.75 | 31.46 | 77.50 | 68.54 | 53.02 |
| AGC-ST | 56.88 | 42.50 | 44.17 | 32.29 | 79.38 | 70.21 | 54.24 |
AGC-ST is higher than the top-1 input for all three models tested on AIME2024/2025. The mean gain is 2.05 points, from 52.19 to 54.24. The top-2 mixture lies in between in every case (53.02 on average). The mixture that changes next-step predictions in App. E.3 thus also improves final-answer accuracy over the top-1 input in full rollouts. We have added the top-1 and top-2 variants to Tab. 3(b) and their discussion to App. E.3.
Q4. Soft Thinking baseline settings
"Table 5 uses top-10 mixing and fixed Cold Stop settings for vanilla Soft Thinking, whereas the original paper reports model-dependent top-n settings and searches over Cold Stop parameters. How were these baseline settings selected, and would model-specific tuning change the comparison?"
Thank you for this question. The Soft Thinking settings follow its official release. With top-n or the Cold Stop entropy threshold chosen per model, Soft Thinking is still below AGC-ST on all five models.
How the settings were selected. We used the configuration of the official repository [1] and its released evaluation script [2] (Tab. 5). This configuration uses top-10 mixing, Cold Stop entropy threshold 0.01, and patience 256. This Cold Stop setting is one of the combinations searched in the Soft Thinking paper.
Model-specific tuning. To test whether per-model tuning changes the comparison, we varied top-n (5, 10, 15) and the Cold Stop entropy threshold (0.01, 0.05, 0.10) on AIME2024 for all five models. We changed one parameter at a time and kept the other at its default value.
Table XXZQ-5. Soft Thinking with different top-n and Cold Stop entropy thresholds on AIME2024 (Avg@4).
| Soft Thinking setting | DS-Qwen-7B | DS-Llama-8B | Qwen3-8B | QwQ-32B | Qwen3-32B |
|---|---|---|---|---|---|
| Top-n (entropy threshold 0.01) | |||||
| top-5 | 50.00 | 30.00 | 75.83 | 72.50 | 78.33 |
| top-10 | 50.83 | 23.33 | 75.83 | 75.83 | 79.17 |
| top-15 | 44.17 | 25.00 | 74.17 | 78.33 | 80.00 |
| Entropy threshold (top-10) | |||||
| 0.05 | 47.50 | 27.50 | 75.83 | 76.67 | 80.83 |
| 0.10 | 46.67 | 25.83 | 71.67 | 73.33 | 76.67 |
| AGC-ST | 55.00 | 49.17 | 78.33 | 81.67 | 84.17 |
With its best setting, Soft Thinking is below AGC-ST on all five models. Model-specific tuning of top-n and the Cold Stop entropy threshold therefore does not change the comparison. We have added these results to App. D.4 (Tab. 7).
[1] Soft Thinking official repository
[2] Soft Thinking evaluation script
Thank you again for your concrete suggestions. Following your comments, we conducted activation-rate-matched controls, a comparison with ThinkRouter, the top-1 substitution in full rollouts, and model-specific tuning of Soft Thinking. We have incorporated these results into the revised paper, with all changes highlighted in red. We hope these additional experiments address your concerns. We would be grateful if you would consider raising your score. We would be happy to discuss any remaining questions.
Response to Reviewer XXZQ (1/3)
Thank you for your detailed review and concrete suggestions. We appreciate your recognition of the consistent gains across 25 settings and the supporting paired bootstrap analysis. We addressed each comment with new experiments:
We address W1 & Q2 here, W2 & Q3 in Part 2, and Q1 and Q4 in Part 3.
W1 & Q2. Separating position selection from mixing frequency
Thank you for these questions. At approximately matched activation rates, the joint gate still outperforms entropy-only gating, margin-only gating and random selection for both models tested on AIME2024/2025. Its benefit does not come from applying soft mixing less often.
To establish the specific benefit of the joint criterion, we built three controls with activation rates close to that of AGC-ST. We used DS-Qwen-7B and DS-Llama-8B on AIME2024/2025. Support calibration and decoding are unchanged.
Table XXZQ-1. Gate ablation on AIME2024/2025 at approximately matched activation rates (Avg@16). Same runs as Table Qc6c-2. CoT is shown for reference. Rate is the percentage of gated thinking steps.
AGC-ST is higher than each control for both models tested on AIME2024/2025, with a mean of 43.96 versus 41.72 (entropy only), 41.56 (margin only), and 40.99 (random gate). The entropy-only and random gates closely match the activation rate of AGC-ST. On DS-Llama-8B, the margin-only gate also matches it and is still 1.46 and 2.29 points lower on AIME2024 and AIME2025. On DS-Qwen-7B, its rate stays near 2% even at β=0.002, about 1.8 times that of AGC-ST.
If the benefit came only from mixing less often, a gate with the same activation rate would match AGC-ST. The random gate has this rate but selects steps at random. It is 2.97 points lower at 40.99. Mixing even less does not help either. The tightest gate in Tab. 2 mixes about a quarter to a third as often as AGC-ST and is 0.78 points lower at 43.18. Mixing frequency therefore does not explain the benefit of the joint criterion. The benefit depends on which steps it selects. We have added these comparisons to block (d) of Tab. 2.