acceptodds
Under review as a conference paper at ICLR 2027

MetaCtrl: Your Large Language Models Can Reason Better and More Concisely with a Metacognitive Controller

Abstract

Large reasoning models improve performance on challenging problems by allocating additional computation before answering, but longer reasoning does not always lead to better results and can introduce substantial redundant reasoning on simple problems. Conversely, aggressively shortening reasoning can degrade performance on difficult ones. Effective reasoning therefore requires dynamically deciding when additional computation is useful based on the reasoner’s capabilities and evolving solution state. Existing approaches often rely on predefined budgets or intervention rules, retrain the target reasoner, or require additional supervision. We introduce MetaCtrl, a lightweight controller that adaptively regulates a frozen reasoner without predefined token budgets or reasoner retraining. We formulate reasoning regulation as a sequential metacognitive control problem: MetaCtrl observes the evolving reasoning trace and decides whether to continue, simplify, skip redundant steps, or conclude reasoning. It is trained directly with reinforcement learning using a reward that prioritizes correctness while favoring shorter trajectories among correct solutions, requiring neither supervised intervention trajectories nor problem-specific budgets. Across seven benchmarks spanning mathematics, science, and code, MetaCtrl consistently improves the accuracy of LRMs while reducing their reasoning length. On DeepSeek-R1-Distill-Qwen-7B, it improves average accuracy by 4.7 points while reducing generation length by 53.3%. Without further training, the same controller transfers to an unseen reasoner (e.g., Qwen3-14B), improving average accuracy by 2.9 points and reducing generation length by 50.3%. These results establish MetaCtrl as a plug-and-play controller for improving reasoning accuracy while substantially reducing inference-time generation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Large Reasoning Model, Efficient Reasoning, Metacognitive Control

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_iJCwOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper trains a small controller that watches a frozen reasoning model and decides how it should continue. Generation pauses at paragraph boundaries ending in ".\n\n" or "?\n\n", and the controller picks one of four actions. Continue adds nothing, while Fast Think, Skip Think and Stop Think each append a fixed sentence to the reasoner's context. The controller is initialized from Qwen3-4B-Instruct-2507 and trained with GRPO on 7,500 OpenR1-Math problems, paired with a greedy DeepSeek-R1-Distill-Qwen-7B. A correct rollout gets 1 plus a bonus of up to 0.5 for being shorter than the other correct rollouts in its group, and a wrong rollout gets -1. No SFT data or token budget is needed. On the training reasoner, average accuracy over MATH-500, AIME 2024, Omni-MATH and GPQA Diamond rises from 53.3 to 59.0 and generation length drops by 47.3%. The same controller is applied without retraining to R1-Distill-Qwen-32B, R1-Distill-Llama-8B, Qwen3-8B and Qwen3-14B, where the reported average length reductions range from 45.6% to 53.7% with similar or higher accuracy. It also outperforms five frontier LLMs used as zero-shot controllers and lowers wall-clock latency on Qwen3.

Critical strengths

1. A plug-in controller that transfers across reasoners. A single 4B controller, trained only with R1-Distill-Qwen-7B on math data, shortens the traces and also works on GPQA and LiveCodeBench, all without changing reasoner weights.

2. A simple recipe with a sensible learned policy. Training needs only outcome rewards, with no expert trajectories and no token budget.

Critical weaknesses

1. The accuracy gains may come from a weak vanilla setup. All reasoners use greedy decoding, and the maximum generation length is not reported. The vanilla scores are far below those in the DeepSeek-R1 report. For example, GPQA Diamond is 32.8 on the 7B model versus 49.1 reported, and 26.3 on Llama-8B, close to chance. MetaCtrl gains most exactly there (+9.6 and +11.1). Since Stop Think tells the reasoner to wrap up and answer, the gain may simply come from rescuing looping or truncated traces, which is a weaker claim than "reason better". The title rests on the accuracy gain, so this is the main reason for my rating.

2. The value of the learned policy is not isolated. There is no random controller with the same action frequencies and no simple rule baseline, so it is unclear whether the gain comes from the policy or from the prompts themselves. The reward coefficients are also chosen on the test benchmarks (Table 4), where AIME 2024 varies from 30.7 to 56.7 across settings, and no variance over runs is reported.

Questions for authors
  1. What fraction of vanilla traces hit the length limit or end without an answer, and how much of the gain comes from these problems? Results under the recommended sampling (temperature 0.6) for the 7B model on AIME 2024 and GPQA Diamond would largely resolve Weakness 1.

  2. How does a random controller with MetaCtrl's action frequencies perform under the same interface?

  3. How many runs are behind each number, and what are the standard deviations?

4 comments
AuthorsOpenReview ↗

Response to Weakness1 and Question1

We thank the reviewer for the comments.

Response to Weakness1 and Question1:

(1) Clarification of evaluation settings. The Vanilla and baseline results in our paper cite the original results reported in two published works, TH2T (ACL 2026) [1] and ASAG (ICML 2026 Spotlight) [2]. As specified in TH2T's Appendix B.3 [1] and ASAG's Section 5.1 [2], both employ greedy decoding. TH2T sets the maximum generation length to 8,192 tokens for MATH-500 and 16,384 tokens for AIME 2024, OmniMath, and GPQA Diamond, while ASAG uses 16,000 tokens. We follow the corresponding greedy decoding and generation length settings for MetaCtrl to ensure fair comparisons. These settings are already documented in Appendix C.4 of our paper. To further address this concern, we independently reproduced Vanilla under the same evaluation settings as TH2T and ASAG. The reproduced results closely match their reported scores.

We additionally evaluated Vanilla and MetaCtrl under the recommended sampling (temperature 0.6) for the 7B model (other settings are kept consistent with the official DeepSeek-R1 report: maximum generation length of 32,768 tokens and top-p 0.95). Results are reported in the table below.

MethodAIME2024 Acc./Len.GPQA Acc./Len.
Vanilla50.0 / 1319447.3 / 8227
MetaCtrl60.0 / 635351.9 / 3894

MetaCtrl is designed to reduce reasoning tokens while maintaining or improving accuracy, rather than trading accuracy for shorter reasoning. Its objective is not to simply terminate reasoning as early as possible, but to adaptively regulate computation based on the evolving reasoning trajectory. For challenging problems requiring more computation, MetaCtrl tends to issue more Continue actions and delay Stop Think, allowing the reasoner to think longer rather than prematurely conclude. Consistent with this behavior, under the larger generation limit and sampling setting, both Vanilla and MetaCtrl exhibit increased reasoning lengths and improved accuracy. MetaCtrl can also reach the maximum generation length without producing a final answer, just like Vanilla.

(2) Accuracy gains beyond rescuing truncated, unanswered traces. Paper's Figures 19–24 present concrete cases where Vanilla completes its reasoning but produces an incorrect final answer, whereas MetaCtrl correctly solves the same problem through dynamic intervention. The experimental results (recommended sampling temperature 0.6, maximum generation length of 32,768 tokens and top-p 0.95 for the 7B model) in the table below further support this observation.

Fraction of Vanilla traces

BenchmarkHit length limit or Without answer
GPQA Diamond4 (2.02%)
AIME20242 (6.67%)

Contribution to accuracy gain

BenchmarkTotal Wrong→RightTruncated or No Answer Wrong→Right (Share of W→R)Non-truncated Wrong→Right (Share of W→R)
GPQA Diamond261 (3.9%)25 (96.1%)
AIME 202451 (20.0%)4 (80%)

Definitions: Total Wrong→Right denotes the number of problems answered incorrectly by Vanilla but correctly by MetaCtrl. Truncated or No Answer Wrong→Right counts cases where Vanilla hit the generation length limit or failed to produce an answer, while MetaCtrl produced the correct answer. Non-truncated Wrong→Right counts cases where Vanilla completed its reasoning but produced an incorrect final answer, while MetaCtrl correctly solved the same problem. The percentages indicate each category's share of total Wrong→Right cases.

Moreover, MetaCtrl consistently achieves higher or comparable accuracy while using substantially fewer reasoning tokens than TH2T [1] (ACL 2026) and ASAG [2] (ICML 2026 Spotlight), two strong, recently accepted baselines designed to improve LLM reasoning. These improvements in the accuracy–efficiency trade-off demonstrate that MetaCtrl enables better reasoning than these strong baselines.

References

[1] Yongjiang Liu et al., Think How to Think: Mitigating Overthinking with Autonomous Difficulty Cognition in Large Reasoning Models. ACL 2026.

[2] Jiakai Li et al., Stop When Further Reasoning Won't Help: Attention-State Adaptive Generation in Reasoning Models. ICML 2026.

AuthorsOpenReview ↗

Response to Weakness2, Question2 and Question3

Response to Weakness2 and Question2:

MethodMATH-500 Acc./Len.AIME 2024 Acc./Len.AMC Acc./Len.GPQA Diamond Acc./Len.
Always-Continue (Vanilla)85.2 / 285750.0 / 1057073.5 / 727932.8 / 5349
Always-Fast8.6 / 134170.0 / 108020.0 / 130475.1 / 11180
Always-Skip7.0 / 126880.0 / 138801.2 / 135925.1 / 10749
Always-Stop69.4 / 109016.7 / 170244.6 / 138228.3 / 1061
Uniform Random75.0 / 121313.3 / 201254.2 / 149326.3 / 1224
Frequency-Matched Random78.8 / 159420.0 / 419565.1 / 323133.3 / 2464
MetaCtrl (Ours)88.0 / 164456.7 / 589974.7 / 303642.4 / 2234

We thank the reviewer for this suggestion. To isolate the effect of the learned control policy, we evaluate six additional baselines, as shown in the table above.

MetaCtrl achieves a strong accuracy–efficiency trade-off than all six baselines across four benchmarks. Skip Think is designed to bypass locally redundant reasoning and proceed with the next necessary reasoning steps, rather than skipping the entire reasoning process. Similarly, Fast Think encourages the reasoner to continue reasoning more efficiently. Consequently, repeatedly inserting these prompts into the reasoner's ongoing reasoning trajectory can induce repetitive reasoning loops and prevent the reasoner from concluding and producing a final answer, as observed in the failed Always-Skip and Always-Fast trajectories. In contrast, Always-Stop may terminate deliberation prematurely. These results demonstrate the importance of selecting interventions based on the evolving reasoning trajectory, rather than merely applying state-independent intervention prompts.

Response to Weakness2 and Question3:

We conducted five runs with different random seeds. The corresponding standard deviations for accuracy and generation length are presented in the tables below.

Table 1. DeepSeek-R1-Distill Series

ReasonerMATH-500 Acc. SD / Len. SDAIME 2024 Acc. SD / Len. SDOmniMath Acc. SD / Len. SDGPQA Diamond Acc. SD / Len. SD
DeepSeek-R1-7B0.49 / 905.96 / 3800.93 / 1650.71 / 58
DeepSeek-R1-32B0.46 / 726.99 / 3870.78 / 1451.88 / 61

Table 2. Qwen3 Series

ReasonerMATH-500 Acc. SD / Len. SDAIME 2024 Acc. SD / Len. SDOlympiadBench Acc. SD / Len. SDGPQA Diamond Acc. SD / Len. SD
Qwen3-8B0.46 / 1103.65 / 6340.71 / 1801.43 / 22
Qwen3-14B0.36 / 854.71 / 4020.97 / 1501.15 / 121

Table4

Reward SettingMATH-500 Acc. SD / Len. SDAMC Acc. SD / Len. SDAIME 2024 Acc. SD / Len. SDOlympiadBench Acc. SD / Len. SDGPQA Diamond Acc. SD / Len. SD
1.5 / -1.0 / 0.50.63 / 1202.99 / 1805.73 / 4201.72 / 1601.60 / 130
2.0 / -1.0 / 0.50.55 / 1353.54 / 3005.73 / 5751.33 / 2401.24 / 270
1.0 / -0.5 / 0.50.70 / 1053.05 / 1554.90 / 3850.97 / 1451.06 / 110
1.0 / -1.0 / 1.00.67 / 952.53 / 1203.89 / 3351.16 / 1200.98 / 100
1.0 / -1.0 / 0.5 (MetaCtrl)0.49 / 901.87 / 1705.96 / 3800.93 / 1650.71 / 58

We clarify that the reward coefficients were selected on validation set derived from the Open R1 dataset [3], rather than on the test benchmarks. The selected coefficients were then fixed for evaluation across all test benchmarks. Table 4 reports an additional ablation analysis to examine how different reward coefficients affect the controller's accuracy–efficiency trade-off and whether the trends observed during validation generalize to the test benchmarks. The test results were not used for reward-coefficient selection. The validation set results are in the table below.

Validation set result

r_correctr_wronglambda_effAcc. (%)Len.
———78.03400
1.5-1.00.583.12350
2.0-1.00.583.62580
1.0-0.50.578.71750
1.0-1.01.071.81410
1.0-1.00.586.62060

[3] Hugging Face. Open R1: A Fully Open Reproduction of DeepSeek-R1. 2025.

Thank you for your detailed response. I think most of my concerns have been addressed. Overall, I'll raise my score to 3 (weak acceptance).

We sincerely thank you for your thoughtful feedback and for reconsidering your score. We are glad that our responses have addressed most of your concerns, and we greatly appreciate your constructive suggestions.

Reviewer_qhXBOpenReview ↗
Rating
3(+1)
Confidence
1(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a lightweight controller that adaptively regulates a frozen reasoner without predefined token budgets or reasoner retraining. The same controller is applied to DeepSeek-R1-Distill 32B and Llama-8B and to Qwen3-8B/14B without further training, seven benchmarks are included to evaluate the proposed method.

Critical strengths
  1. This paper mainly focus on the performance improvement of large reasoning models. Separating a frozen object-level reasoner from a small meta-level policy is a sensible interface for adaptive test-time computation.

  2. The experiments are extensive, this paper covers mathematics, science, and code; compares several efficient-reasoning paradigms; and includes frontier LLMs under the same controller interface.

Critical weaknesses
  1. It is unclear whether the gains come from state-dependent control or simply from inserting intervention prompts. The authors should compare against always-Continue/Fast/Skip/Stop, uniform-random, and frequency-matched shuffled policies.

  2. The used baselines TH2T and ASAG results are taken from prior papers, while ACTS is reproduced under greedy decoding. The authors should verify identical prompts, model versions, answer extraction, generation limits, and evaluation sets—or rerun the strongest baselines under a unified setup.

Questions for authors
  1. On each benchmark, how many items flip from wrong to right versus right to wrong relative to vanilla, and what is the bootstrap interval on AIME?

  2. What fraction of Stop Think and Skip Think calls are followed by the reasoner actually concluding, and what is the invalid-action rate for MetaCtrl and for the API controllers?

6 comments
AuthorsOpenReview ↗

Response to Weakness1, Weakness2 and Question1

We thank the reviewer for the comments.

Response to Weakness1:

MethodMATH-500 Acc./Len.AIME 2024 Acc./Len.AMC Acc./Len.GPQA Diamond Acc./Len.
Always-Continue (Vanilla)85.2 / 285750.0 / 1057073.5 / 727932.8 / 5349
Always-Fast8.6 / 134170.0 / 108020.0 / 130475.1 / 11180
Always-Skip7.0 / 126880.0 / 138801.2 / 135925.1 / 10749
Always-Stop69.4 / 109016.7 / 170244.6 / 138228.3 / 1061
Uniform Random75.0 / 121313.3 / 201254.2 / 149326.3 / 1224
Frequency-Matched Random78.8 / 159420.0 / 419565.1 / 323133.3 / 2464
MetaCtrl (Ours)88.0 / 164456.7 / 589974.7 / 303642.4 / 2234

To isolate the effect of state-dependent control, we evaluate six additional baselines, as shown in the table above. Specifically, Frequency-Matched Random randomly selects each intervention action using its selection frequency under MetaCtrl as the sampling probability, with the probabilities calculated separately for each reasoner–benchmark pair.

MetaCtrl achieves a stronger accuracy–efficiency trade-off than all six baselines across four benchmarks. Skip Think is designed to bypass locally reasoning step and proceed with the next reasoning step, rather than skipping the entire reasoning process. Similarly, Fast Think encourages the reasoner to continue reasoning more efficiently. Consequently, repeatedly inserting these prompts into the reasoner's ongoing reasoning trajectory can induce repetitive reasoning loops and prevent the reasoner from concluding (stop think) and producing a final answer, as observed in the failed Always-Skip and Always-Fast trajectories. In contrast, Always-Stop may terminate deliberation prematurely. These results demonstrate the importance of selecting interventions based on the evolving reasoning trajectory, rather than merely applying state-independent intervention prompts.

Response to Weakness2:

Both TH2T paper's Appendix B.3 [1] and ASAG paper's Section 5.1 [2] employ greedy decoding. Accordingly, we use greedy decoding for both our MetaCtrl and ACTS reproduction (because the results reported in the ACTS paper were not obtained using greedy decoding), which directly uses the officially released ACTS checkpoint. We also verify the prompts, model versions, answer extraction, generation limits and evaluation procedures, and benchmark datasets with the corresponding reference studies. We will clarify these implementation details in the revised manuscript.

References

[1] Yongjiang Liu et al. Think How to Think: Mitigating Overthinking with Autonomous Difficulty Cognition in Large Reasoning Models. ACL 2026.

[2] Jiakai Li et al. Stop When Further Reasoning Won't Help: Attention-State Adaptive Generation in Reasoning Models. ICML 2026.

Response to Question1:

We thank the reviewer for this suggestion. We conduct a paired per-problem analysis comparing MetaCtrl with Vanilla. As shown in the table below, we report the number of problems that change from incorrect to correct (Wrong → Right) and from correct to incorrect (Right → Wrong) across benchmarks.

DeepSeek-R1-7B

BenchmarkWrong → rightRight → wrong
MATH-5002713
AIME 202442
OmniMath298192
GPQA Diamond3112

Qwen3-8B

BenchmarkWrong → rightRight → wrong
MATH-5002213
AIME 202432
OlympiadBench6739
GPQA Diamond2011

To assess statistical uncertainty on AIME 2024, we additionally compute 95% paired bootstrap confidence intervals for the accuracy improvements of MetaCtrl over Vanilla by resampling the 30 evaluation problems with replacement 10000 times. For DeepSeek-R1-7B, MetaCtrl achieves an accuracy improvement of 6.67 percentage points, with a 95% confidence interval of [-1.67, 15.67] percentage points. For Qwen3-8B, the improvement is 3.33 percentage points, with a 95% confidence interval of [-4.33, 11.33] percentage points.

AuthorsOpenReview ↗

Response to Question2

Response to Question2:

We clarify that Skip Think and Stop Think serve distinct purposes, and immediate conclusion is not the intended behavior of skip think actions. Specifically, their intervention prompts are:

  • Skip Think: "Let’s solve this directly with the minimum necessary reasoning and move to the answer."
  • Stop Think: "The reasoning is sufficient; state only the final conclusion briefly and close the reasoning now."

Skip Think aims to bypass locally redundant reasoning steps while preserving necessary subsequent reasoning, rather than forcing an immediate conclusion. Thus, measuring its effectiveness by the fraction of immediate conclusions would be inconsistent with its design. In contrast, Stop Think encourages the reasoner to conclude.

However, strictly determining whether the generation following Stop Think constitutes a semantic conclusion is challenging, we employ an LLM-judge (DeepSeek API) for this assessment. The results show that 95.8% of Stop Think interventions lead the reasoner to begin concluding on DeepSeek-R1-7B.

We additionally report the number of tokens generated after Stop Think but before the </think> token in the table below. The relatively small number of tokens generated between Stop Think and </think> further indicates that the reasoner typically concludes its reasoning shortly after the intervention.

BenchmarkTokens between Stop Think and </think>
MATH-500209
AIME 2024819
AMC291
GPQA Diamond480

To assess whether Stop Think encourages earlier reasoning termination, we conduct a counterfactual replay using MetaCtrl-controlled trajectories. Specifically, we preserve all tokens before Stop Think, replace it with Continue, and let the reasoner generate naturally until completion without further intervention. We then measure the change in total generation length and the percentage of problems with increased token usage after replacing Stop Think.As shown below, Stop Think saves an average of 1426–2710 tokens across four benchmarks, with 93%–100% of problems replays exhibiting increased generation length after replacing Stop Think with Continue, demonstrating its effectiveness in encouraging earlier conclusions.

MetricMATH-500AIME 2024AMCGPQA Diamond
Saved Tokens from Stop Think (Δ)1426271019372308
Percentage of Problems with Increased Token96%93%100%97%

To further disentangle the contributions of different actions (stop think and skip think), we introduce two ablation variants in the Table below: (1) Continue + Fast + Stop, which replaces all Skip Think actions with Continue while retaining Fast Think and Stop Think. (2) Continue + Stop, which replaces all Fast Think and Skip Think actions with Continue.

MethodMATH-500AIME 2024AMCGPQA Diamond
Continue only (Vanilla)28571057072795349
Continue and Stop1920660737023200
Continue, Fast and Stop1879614233153050
Continue, Fast, Skip and Stop (MetaCtrl)1644589930362234

MetaCtrl and API-based controllers employ different mechanisms to ensure action validity. For MetaCtrl, we use regex-constrained decoding in SGLang, restricting the output to four predefined actions, thereby preventing invalid actions at decoding time. In contrast, API-based controllers rely on prompt-based instructions without decoding-level constraints. If no valid action can be identified, we fall back to continue, allowing the reasoner to proceed naturally without additional intervention. The invalid-action rates for MetaCtrl and the API controllers across all benchmarks are reported in the table below.

API ControllerMath500AIME2024AMCGPQA DiamondOlympiadBench
qwen3.8-flash6.31%N/A26.86%14.63%N/A
glm-5.24.88%0.71%3.20%4.67%4.08%
gpt5.6-luna2.63%0.00%0.00%2.88%2.66%
deepseek-v4.1-flash0.12%0.26%0.41%0.06%0.25%
qwen3.8-max0.00%0.00%0.00%0.00%0.00%
MetaCtrl0.00%0.00%0.00%0.00%0.00%

Dear Reviewer,

As the author–reviewer discussion deadline approaches, I would like to remind you to review the authors’ rebuttal and indicate whether your concerns have been adequately addressed.

If you have any remaining concerns or questions, please share them with the authors before the discussion period ends.

Thank you again for your time and valuable feedback.

Reviewer_bJdTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

MetaCtrl trains a small controller with reinforcement learning to steer a frozen reasoning model by picking one of four fixed prompts (Continue, Fast Think, Skip Think, Stop Think) at each paragraph break. It builds on ACTS but drops its token budget and supervised training, rewarding correct answers with a bonus for shorter ones. Across seven math, science, and code benchmarks, it reports large token savings with similar or better accuracy, transfer to unseen reasoners, and lower latency.

Critical strengths
  1. The method is simple yet effective on improving the efficiency of reasoning with matching or even better accuracy.
  2. There is a comprehensive analysis on the latency including the extra factor of the controller, showing that the controller's overhead does not cancel out the benefit.
Critical weaknesses
  1. Missing ablation studies: ACTS is the paper's closest baseline, and the paper credits its improvement to removing the budget and the supervised data. But there is no experiment isolates those changes, or testing the variations of the four fixed prompts.
  2. Potentially unfair comparison. The comparison gives ACTS one budget per benchmark for every reasoner (e.g., 5,000 tokens on AIME 2024). But the ACTS paper used 9,997 tokens (DeepSeek-R1-Distill-Qwen-7B) and 13,755 (Qwen3-8B) on AIME 2024.
  3. The distribution of what the controller actually does is not shown. Figure 6 implies about one stop think per problem in all 12 reasoner–benchmark settings, so probably every attempt ends with stop?
Questions for authors
  1. Does the stop think prompt really force the thinking to end? How many percent of attempts end with it? How much of the token reduction comes from stopping versus the fast/skip prompts?
  2. How many runs or random seeds underlie each number?
  3. Did you use the released ACTS checkpoint or retrain from scratch? And how is the budget chosen to guarantee fair comparison?
8 comments
AuthorsOpenReview ↗

Response to Weakness1

We thank the reviewer for the comments.

Response to Weakness1:

We would like to clarify that budget-free control and the elimination of supervised data are key design advantages of MetaCtrl that address the practical limitations of ACTS, rather than claims that either design choice independently accounts for the observed performance gains.

(1) Budget-free control. Our key motivation is to eliminate the requirement for a predefined budget, since users cannot know in advance the minimum budget required to correctly solve a given problem. An insufficient budget may reduce accuracy, while an excessive budget leads to redundant reasoning and wasted computation (as shown in the acts paper's figure4), substantially limiting the practical applicability of ACTS in real-world scenarios. In contrast, MetaCtrl adaptively regulates reasoning length based on the evolving reasoning trajectory without requiring a predefined budget, achieving a strong accuracy–efficiency trade-off.

Thus, budget-free control is an inherent design property, rather than a removable component whose impact on performance gains. Adding budget conditioning to MetaCtrl or removing it from ACTS would require reformulating their respective control mechanisms rather than conducting a conventional component ablation.

(2) Supervised data. ACTS relies on supervised steering trajectories constructed from DeepSeek-R1 reasoning traces annotated by Qwen3-32B, followed by SFT and RL, incurring substantial data construction and teacher inference costs. In contrast, our key contribution is eliminating the cost of constructing supervised data and dependence on external LLMs distillation, while maintaining a strong accuracy–efficiency trade-off, rather than claiming that removing supervised data inherently improves accuracy or reduces reasoning tokens.

Table R1: Effect of Multi-budget Augmentation Supervised Data Training on ACTS.

BudgetFull ACTS Len./Acc.ACTS w/o Multi-budget Augmentation Supervised Data Training Len./Acc.
24993366 / 29.311411 / 44.7
49985451 / 42.712762 / 46.0
74986711 / 51.311801 / 53.3
99978200 / 60.012294 / 55.3
1249710012 / 58.013014 / 54.0

(3) Evidence from ACTS's own ablation. The result in the Table R1 above (ACTS paper's Table 3) further reveals its strong dependence on specialized SFT data construction. Without multi-budget augmentation, ACTS generates 11411–13014 tokens across five target budgets ranging from 2499 to 12497 tokens, approaching the 13031 tokens used by natural reasoning. In contrast, full ACTS adapts its token usage from 3366 to 10012 tokens. This indicates that ACTS's ability to suppress redundant reasoning effectively collapses without this supervised data augmentation and budget control.

Overall, these results expose a key limitation of ACTS's explicit budget-dependent design, which relies heavily on carefully constructed supervised data for effective reasoning control. MetaCtrl avoids this dependency by eliminating budgets and learning its adaptive control policy directly through RL.

(4) Action prompt robustness. We additionally consider two semantically equivalent variants of the prompts. Since Continue introduces no textual intervention, only the other three prompts are modified. The comparison is structured below.

Action Prompt Variant 1

  • Fast Think: Let's focus on the key derivation, omit repeated steps, and move forward efficiently.
  • Skip Think: Let's take the most direct route, reason only as much as needed, and move to the answer.
  • Stop Think: We have reasoned enough. Briefly give only the final conclusion and end the reasoning here.

Action Prompt Variant 2

  • Fast Think: Carry the derivation forward using only the key steps and avoid repeating earlier work.
  • Skip Think: Skip redundant intermediate steps and take the shortest sound route to the answer.
  • Stop Think: We can conclude now. State just the conclusion briefly, then end the reasoning.
Method (Reasoner)Action PromptMATH500 Acc./Len.AIME2024 Acc./Len.AMC Acc./Len.OlympiadBench Acc./Len.GPQA Acc./Len.
MetaCtrl(DeepSeek-R1-7B)Original88.0 / 164456.7 / 589974.7 / 303652.3 / 296542.4 / 2234
MetaCtrl(DeepSeek-R1-7B)Variant 186.8 / 157453.3 / 563473.5 / 286050.8 / 276843.4 / 2081
MetaCtrl(DeepSeek-R1-7B)Variant 289.0 / 169556.7 /608974.7 / 317851.9 / 315040.9 / 2352
MetaCtrl(Qwen3-8B)Original94.0 / 229466.7 / 864581.9 / 462364.0 / 413457.1 / 3518
MetaCtrl(Qwen3-8B)Variant 193.2 / 246063.3 / 905483.1 / 497364.7 / 448356.6 / 3499
MetaCtrl(Qwen3-8B)Variant 293.8 / 219866.7 / 883280.7 / 445963.6 / 426658.6 / 3348

The results show that MetaCtrl maintains comparable accuracy and token efficiency across alternative prompt formulations, suggesting that its robustness is not strongly dependent on the exact wording of the prompts.

AuthorsOpenReview ↗

Response to Weakness2 and Weakness3

Response to Weakness2:

We clarify that ACTS's reported results rely on benchmark- and reasoner-specific optimal budgets identified through evaluations across multiple budget settings on the test benchmarks (ACTS's paper Figure 4), such as 9,997 and 13,755 tokens on AIME 2024. Comparing MetaCtrl against these testset-selected optimal points introduces an additional budget-tuning advantage, as users cannot know the optimal budget in advance in real-world scenarios. Addressing this practical limitation is precisely the motivation behind MetaCtrl's budget-free design.

To provide a meaningful comparison, we evaluate two settings: (1) matched token consumption, comparing accuracy under similar actual generation lengths, and (2) matched accuracy, comparing token consumption when both methods achieve comparable accuracy.

Matched Token Consumption

ReasonerMethodMATH-500 Acc./Len./BudgetAIME 2024 Acc./Len./BudgetGPQA Diamond Acc./Len./Budget
DeepSeek-R1-7BMetaCtrl88.0 / 1644 / –56.7 / 5899 / –42.4 / 2234 / –
DeepSeek-R1-7BACTS82.8 / 1751 / 150036.7 / 5909 / 500040.4 / 2262 / 2000
Qwen3-8BMetaCtrl94.0 / 2294 / –66.7 / 8645 / –57.1 / 3518 / –
Qwen3-8BACTS88.4 / 2442 / 180060.0 / 8532 / 850055.5 / 3772 / 3000

Matched Accuracy

ReasonerMethodMATH-500 Acc./Len./BudgetAIME 2024 Acc./Len./BudgetGPQA Diamond Acc./Len./Budget
DeepSeek-R1-7BMetaCtrl88.0 / 1644 / –56.7 / 5899 / –42.4 / 2234 / –
DeepSeek-R1-7BACTS87.8 / 2739 / 250056.7 / 7852 / 1000040.9 / 5358 / 4000
Qwen3-8BMetaCtrl94.0 / 2294 / –66.7 / 8645 / –57.1 / 3518 / –
Qwen3-8BACTS93.8 / 3394 / 450063.3 / 9095 / 900057.6 / 5102 / 4000

As shown in the table above, MetaCtrl achieves comparable accuracy at similar total token usage and reduces token consumption at comparable accuracy. However, we emphasize that our contribution is not predicated on outperforming ACTS at budgets retrospectively selected through multiple evaluations on the test benchmarks.

Since MetaCtrl is budget-free whereas ACTS requires a predefined budget, a strictly fair comparison is inherently difficult. We acknowledge that comparing ACTS using only a single predefined budget per benchmark alongside other baselines in Tables 1 and 2 may be misleading. We will therefore move the ACTS results to dedicated comparison tables, evaluating accuracy at similar actual token consumption and token consumption at comparable accuracy levels. This enables a more comprehensive comparison of budget-free MetaCtrl against ACTS across multiple predefined budgets.

Response to Weakness3:

Figure 6 already presents the distribution of the four controller actions. To further clarify the controller's behavior at the problem level, we additionally report the mean action counts and the number of tokens generated after Stop Think but before the </think> token in the table below.

ReasonerBenchmarkContinueFast ThinkSkip ThinkStop ThinkBefore Stop (tokens)After Stop but before </think> (tokens)
DeepSeek-R1-7BMATH-50010.021.161.050.971050209
DeepSeek-R1-7BAIME 202428.402.134.371.004080819
DeepSeek-R1-7BAMC31.951.712.530.991992291
DeepSeek-R1-7BGPQA12.071.482.900.971336480
Qwen3-8BMATH-50011.201.241.690.991261356
Qwen3-8BAIME 202455.801.909.401.007155790
Qwen3-8BAMC34.431.603.720.982968516
Qwen3-8BGPQA14.171.355.010.992043440
Qwen3-14BMATH-5009.681.121.031.001153190
Qwen3-14BAIME 202443.771.833.831.005077907
Qwen3-14BAMC26.351.763.431.002667307
Qwen3-14BGPQA8.481.342.890.981739305

Stop Think is selected in 97%–100.0% of problems. As a terminal controller action, it can occur at most once per problem, but is neither mandatory nor triggered at a fixed position. Instead, the MetaCtrl dynamically determines when sufficient reasoning has been performed to select Stop Think based on the ongoing reasoning trajectory. If the reasoner finishes naturally before Stop Think is selected, no further intervention occurs. Once selected, Stop Think prompts the reasoner to "state only the final conclusion briefly and close the reasoning now," after which the controller ceases intervention and the reasoner continues generating until completion, without forced truncation. The relatively small number of tokens generated between Stop Think and </think> further indicates that the reasoner typically concludes its reasoning shortly after the intervention.

AuthorsOpenReview ↗

Response to Question1, Question2 and Question3

Response to Question1:

We clarify that Stop Think is not designed to forcibly terminate reasoning immediately. Its intervention prompt is: "The reasoning is sufficient, state only the final conclusion briefly and close the reasoning." This prompt is inserted into the reasoner's ongoing reasoning trajectory without forcibly truncating generation or inserting a </think> token to terminate reasoning. The controller then ceases intervention, while the reasoner continues generating naturally until completion (see our response to Weakness 3). Across our evaluated benchmark, Stop Think is selected in 97%–100% of all problems. However, selecting Stop Think does not necessarily cause reasoning to end immediately.

However, strictly determining whether the generation following Stop Think constitutes a semantic conclusion is challenging, we employ an LLM-judge (DeepSeek API) for this assessment. The results show that 95.8% of Stop Think interventions lead the reasoner to begin concluding on DeepSeek-R1-7B.

Additionally, to assess whether Stop Think encourages earlier reasoning termination, we conduct a counterfactual replay using MetaCtrl-controlled trajectories. Specifically, we preserve all tokens before Stop Think, replace it with Continue, and let the reasoner generate naturally until completion without further intervention. We then measure the change in total generation length and the percentage of problems with increased token usage after replacing Stop Think.As shown below, Stop Think saves an average of 1426–2710 tokens across four benchmarks, with 93%–100% of problems replays exhibiting increased generation length after replacing Stop Think with Continue, demonstrating its effectiveness in encouraging earlier conclusions without forcibly terminating reasoning.

MetricMATH-500AIME 2024AMCGPQA Diamond
Saved Tokens (Δ)1426271019372308
Percentage of problems with increased token96%93%100%97%

To further disentangle the contributions of different actions, we introduce two ablation variants: (1) Continue + Fast + Stop, which replaces all Skip Think actions with Continue while retaining Fast Think and Stop Think. (2) Continue + Stop, which replaces all Fast Think and Skip Think actions with Continue.

MethodMATH-500AIME 2024AMCGPQA Diamond
Continue only (Vanilla)28571057072795349
Continue and Stop1920660737023200
Continue, Fast and Stop1879614233153050
Continue, Fast, Skip and Stop (MetaCtrl)1644589930362234

Response to Question2:

We conduct five independent runs with random seeds 1234, 2234, 3234, 4234, and 5234.

Response to Question3:

We use the officially released ACTS checkpoint. We would like to emphasize that MetaCtrl's primary advantages over ACTS lie in its methodological design, which addresses the practical limitations that hinder ACTS's real-world deployment, rather than improvements in accuracy or token efficiency. Since MetaCtrl is budget-free whereas ACTS requires a predefined token budget, achieving a strictly fair performance comparison is inherently difficult. We acknowledge that comparing MetaCtrl against ACTS evaluated with a single budget in Tables 1 and 2 may be inappropriate. We will therefore revise Tables 1 and 2 to exclude the ACTS comparison and present a separate table dedicated to comparing MetaCtrl with ACTS across multiple predefined budgets: (1) comparing accuracy at comparable token consumption and (2) comparing token consumption at comparable accuracy. The experimental results and further discussion are provided in our response to Weakness 1 and 2.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.