- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper trains a small controller that watches a frozen reasoning model and decides how it should continue. Generation pauses at paragraph boundaries ending in ".\n\n" or "?\n\n", and the controller picks one of four actions. Continue adds nothing, while Fast Think, Skip Think and Stop Think each append a fixed sentence to the reasoner's context. The controller is initialized from Qwen3-4B-Instruct-2507 and trained with GRPO on 7,500 OpenR1-Math problems, paired with a greedy DeepSeek-R1-Distill-Qwen-7B. A correct rollout gets 1 plus a bonus of up to 0.5 for being shorter than the other correct rollouts in its group, and a wrong rollout gets -1. No SFT data or token budget is needed. On the training reasoner, average accuracy over MATH-500, AIME 2024, Omni-MATH and GPQA Diamond rises from 53.3 to 59.0 and generation length drops by 47.3%. The same controller is applied without retraining to R1-Distill-Qwen-32B, R1-Distill-Llama-8B, Qwen3-8B and Qwen3-14B, where the reported average length reductions range from 45.6% to 53.7% with similar or higher accuracy. It also outperforms five frontier LLMs used as zero-shot controllers and lowers wall-clock latency on Qwen3.
Critical strengths
1. A plug-in controller that transfers across reasoners. A single 4B controller, trained only with R1-Distill-Qwen-7B on math data, shortens the traces and also works on GPQA and LiveCodeBench, all without changing reasoner weights.
2. A simple recipe with a sensible learned policy. Training needs only outcome rewards, with no expert trajectories and no token budget.
Critical weaknesses
1. The accuracy gains may come from a weak vanilla setup. All reasoners use greedy decoding, and the maximum generation length is not reported. The vanilla scores are far below those in the DeepSeek-R1 report. For example, GPQA Diamond is 32.8 on the 7B model versus 49.1 reported, and 26.3 on Llama-8B, close to chance. MetaCtrl gains most exactly there (+9.6 and +11.1). Since Stop Think tells the reasoner to wrap up and answer, the gain may simply come from rescuing looping or truncated traces, which is a weaker claim than "reason better". The title rests on the accuracy gain, so this is the main reason for my rating.
2. The value of the learned policy is not isolated. There is no random controller with the same action frequencies and no simple rule baseline, so it is unclear whether the gain comes from the policy or from the prompts themselves. The reward coefficients are also chosen on the test benchmarks (Table 4), where AIME 2024 varies from 30.7 to 56.7 across settings, and no variance over runs is reported.
Questions for authors
-
What fraction of vanilla traces hit the length limit or end without an answer, and how much of the gain comes from these problems? Results under the recommended sampling (temperature 0.6) for the 7B model on AIME 2024 and GPQA Diamond would largely resolve Weakness 1.
-
How does a random controller with MetaCtrl's action frequencies perform under the same interface?
-
How many runs are behind each number, and what are the standard deviations?
4 comments
Response to Weakness2, Question2 and Question3
Response to Weakness2 and Question2:
| Method | MATH-500 Acc./Len. | AIME 2024 Acc./Len. | AMC Acc./Len. | GPQA Diamond Acc./Len. |
|---|---|---|---|---|
| Always-Continue (Vanilla) | 85.2 / 2857 | 50.0 / 10570 | 73.5 / 7279 | 32.8 / 5349 |
| Always-Fast | 8.6 / 13417 | 0.0 / 10802 | 0.0 / 13047 | 5.1 / 11180 |
| Always-Skip | 7.0 / 12688 | 0.0 / 13880 | 1.2 / 13592 | 5.1 / 10749 |
| Always-Stop | 69.4 / 1090 | 16.7 / 1702 | 44.6 / 1382 | 28.3 / 1061 |
| Uniform Random | 75.0 / 1213 | 13.3 / 2012 | 54.2 / 1493 | 26.3 / 1224 |
| Frequency-Matched Random | 78.8 / 1594 | 20.0 / 4195 | 65.1 / 3231 | 33.3 / 2464 |
| MetaCtrl (Ours) | 88.0 / 1644 | 56.7 / 5899 | 74.7 / 3036 | 42.4 / 2234 |
We thank the reviewer for this suggestion. To isolate the effect of the learned control policy, we evaluate six additional baselines, as shown in the table above.
MetaCtrl achieves a strong accuracy–efficiency trade-off than all six baselines across four benchmarks. Skip Think is designed to bypass locally redundant reasoning and proceed with the next necessary reasoning steps, rather than skipping the entire reasoning process. Similarly, Fast Think encourages the reasoner to continue reasoning more efficiently. Consequently, repeatedly inserting these prompts into the reasoner's ongoing reasoning trajectory can induce repetitive reasoning loops and prevent the reasoner from concluding and producing a final answer, as observed in the failed Always-Skip and Always-Fast trajectories. In contrast, Always-Stop may terminate deliberation prematurely. These results demonstrate the importance of selecting interventions based on the evolving reasoning trajectory, rather than merely applying state-independent intervention prompts.
Response to Weakness2 and Question3:
We conducted five runs with different random seeds. The corresponding standard deviations for accuracy and generation length are presented in the tables below.
Table 1. DeepSeek-R1-Distill Series
| Reasoner | MATH-500 Acc. SD / Len. SD | AIME 2024 Acc. SD / Len. SD | OmniMath Acc. SD / Len. SD | GPQA Diamond Acc. SD / Len. SD |
|---|---|---|---|---|
| DeepSeek-R1-7B | 0.49 / 90 | 5.96 / 380 | 0.93 / 165 | 0.71 / 58 |
| DeepSeek-R1-32B | 0.46 / 72 | 6.99 / 387 | 0.78 / 145 | 1.88 / 61 |
Table 2. Qwen3 Series
| Reasoner | MATH-500 Acc. SD / Len. SD | AIME 2024 Acc. SD / Len. SD | OlympiadBench Acc. SD / Len. SD | GPQA Diamond Acc. SD / Len. SD |
|---|---|---|---|---|
| Qwen3-8B | 0.46 / 110 | 3.65 / 634 | 0.71 / 180 | 1.43 / 22 |
| Qwen3-14B | 0.36 / 85 | 4.71 / 402 | 0.97 / 150 | 1.15 / 121 |
Table4
| Reward Setting | MATH-500 Acc. SD / Len. SD | AMC Acc. SD / Len. SD | AIME 2024 Acc. SD / Len. SD | OlympiadBench Acc. SD / Len. SD | GPQA Diamond Acc. SD / Len. SD |
|---|---|---|---|---|---|
| 1.5 / -1.0 / 0.5 | 0.63 / 120 | 2.99 / 180 | 5.73 / 420 | 1.72 / 160 | 1.60 / 130 |
| 2.0 / -1.0 / 0.5 | 0.55 / 135 | 3.54 / 300 | 5.73 / 575 | 1.33 / 240 | 1.24 / 270 |
| 1.0 / -0.5 / 0.5 | 0.70 / 105 | 3.05 / 155 | 4.90 / 385 | 0.97 / 145 | 1.06 / 110 |
| 1.0 / -1.0 / 1.0 | 0.67 / 95 | 2.53 / 120 | 3.89 / 335 | 1.16 / 120 | 0.98 / 100 |
| 1.0 / -1.0 / 0.5 (MetaCtrl) | 0.49 / 90 | 1.87 / 170 | 5.96 / 380 | 0.93 / 165 | 0.71 / 58 |
We clarify that the reward coefficients were selected on validation set derived from the Open R1 dataset [3], rather than on the test benchmarks. The selected coefficients were then fixed for evaluation across all test benchmarks. Table 4 reports an additional ablation analysis to examine how different reward coefficients affect the controller's accuracy–efficiency trade-off and whether the trends observed during validation generalize to the test benchmarks. The test results were not used for reward-coefficient selection. The validation set results are in the table below.
Validation set result
| r_correct | r_wrong | lambda_eff | Acc. (%) | Len. |
|---|---|---|---|---|
| — | — | — | 78.0 | 3400 |
| 1.5 | -1.0 | 0.5 | 83.1 | 2350 |
| 2.0 | -1.0 | 0.5 | 83.6 | 2580 |
| 1.0 | -0.5 | 0.5 | 78.7 | 1750 |
| 1.0 | -1.0 | 1.0 | 71.8 | 1410 |
| 1.0 | -1.0 | 0.5 | 86.6 | 2060 |
[3] Hugging Face. Open R1: A Fully Open Reproduction of DeepSeek-R1. 2025.
Thank you for your detailed response. I think most of my concerns have been addressed. Overall, I'll raise my score to 3 (weak acceptance).
We sincerely thank you for your thoughtful feedback and for reconsidering your score. We are glad that our responses have addressed most of your concerns, and we greatly appreciate your constructive suggestions.
Response to Weakness1 and Question1
We thank the reviewer for the comments.
Response to Weakness1 and Question1:
(1) Clarification of evaluation settings. The Vanilla and baseline results in our paper cite the original results reported in two published works, TH2T (ACL 2026) [1] and ASAG (ICML 2026 Spotlight) [2]. As specified in TH2T's Appendix B.3 [1] and ASAG's Section 5.1 [2], both employ greedy decoding. TH2T sets the maximum generation length to 8,192 tokens for MATH-500 and 16,384 tokens for AIME 2024, OmniMath, and GPQA Diamond, while ASAG uses 16,000 tokens. We follow the corresponding greedy decoding and generation length settings for MetaCtrl to ensure fair comparisons. These settings are already documented in Appendix C.4 of our paper. To further address this concern, we independently reproduced Vanilla under the same evaluation settings as TH2T and ASAG. The reproduced results closely match their reported scores.
We additionally evaluated Vanilla and MetaCtrl under the recommended sampling (temperature 0.6) for the 7B model (other settings are kept consistent with the official DeepSeek-R1 report: maximum generation length of 32,768 tokens and top-p 0.95). Results are reported in the table below.
MetaCtrl is designed to reduce reasoning tokens while maintaining or improving accuracy, rather than trading accuracy for shorter reasoning. Its objective is not to simply terminate reasoning as early as possible, but to adaptively regulate computation based on the evolving reasoning trajectory. For challenging problems requiring more computation, MetaCtrl tends to issue more Continue actions and delay Stop Think, allowing the reasoner to think longer rather than prematurely conclude. Consistent with this behavior, under the larger generation limit and sampling setting, both Vanilla and MetaCtrl exhibit increased reasoning lengths and improved accuracy. MetaCtrl can also reach the maximum generation length without producing a final answer, just like Vanilla.
(2) Accuracy gains beyond rescuing truncated, unanswered traces. Paper's Figures 19–24 present concrete cases where Vanilla completes its reasoning but produces an incorrect final answer, whereas MetaCtrl correctly solves the same problem through dynamic intervention. The experimental results (recommended sampling temperature 0.6, maximum generation length of 32,768 tokens and top-p 0.95 for the 7B model) in the table below further support this observation.
Fraction of Vanilla traces
Contribution to accuracy gain
Definitions:
Total Wrong→Rightdenotes the number of problems answered incorrectly by Vanilla but correctly by MetaCtrl.Truncated or No Answer Wrong→Rightcounts cases where Vanilla hit the generation length limit or failed to produce an answer, while MetaCtrl produced the correct answer.Non-truncated Wrong→Rightcounts cases where Vanilla completed its reasoning but produced an incorrect final answer, while MetaCtrl correctly solved the same problem. The percentages indicate each category's share of total Wrong→Right cases.Moreover, MetaCtrl consistently achieves higher or comparable accuracy while using substantially fewer reasoning tokens than TH2T [1] (ACL 2026) and ASAG [2] (ICML 2026 Spotlight), two strong, recently accepted baselines designed to improve LLM reasoning. These improvements in the accuracy–efficiency trade-off demonstrate that MetaCtrl enables better reasoning than these strong baselines.
References
[1] Yongjiang Liu et al., Think How to Think: Mitigating Overthinking with Autonomous Difficulty Cognition in Large Reasoning Models. ACL 2026.
[2] Jiakai Li et al., Stop When Further Reasoning Won't Help: Attention-State Adaptive Generation in Reasoning Models. ICML 2026.