- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes BELT, a framework for early stopping in LLM reasoning that learns reusable representations by predicting four reasoning outcomes from hidden states, then fits lightweight linear stopping policies for different token costs. Experiments on mathematical reasoning and code generation demonstrate competitive accuracy–efficiency trade-offs, improved cross-domain transfer, and substantially lower controller training costs compared to existing baselines.
Critical strengths
-
The paper presents a well-motivated approach that separates outcome prediction from stopping policy optimization, enabling the reuse of learned representations across different token costs. This design substantially reduces controller training costs while maintaining competitive accuracy–efficiency trade-offs.
-
The experimental evaluation is comprehensive, covering multiple reasoning models, mathematical benchmarks, and cross-domain transfer to coding tasks. The extensive ablation studies, particularly the comparisons under a unified policy-fitting framework, provide convincing evidence for the effectiveness of outcome-based supervision.
Critical weaknesses
-
The methodological novelty is somewhat limited, as BELT builds on existing probe architectures and utility-based stopping objectives. While the separation of belief learning and policy fitting is useful, the performance gains over the strongest baseline are relatively modest (e.g., 2.2 percentage points in token savings over OSP on Qwen3 OOD at 1 pp accuracy tolerance).
-
The evaluation has some limitations in generalizability and practical applicability. The mathematical benchmarks are restricted to filtered integer-answer subsets, excluding many problems with more complex answer formats. Moreover, the main results rely on test-frontier evaluation, which selects operating points using test-set performance and may not fully reflect realistic deployment settings.
Questions for authors
-
Could the authors further clarify the novelty of the proposed belief representations compared with existing outcome-prediction probes? In particular, how much of the improvement comes from the four prediction targets rather than the additional sequence encoder capacity?
-
How well does BELT perform when stopping policies are selected entirely on the development set rather than using test-frontier operating points? Could the authors report the corresponding accuracy and token savings to clarify the practical performance gains?
-
How sensitive are the results to the filtering of mathematical benchmarks? Would BELT maintain similar advantages on problems requiring fractional, symbolic, or other non-integer answers?
-
The paper reports substantial reductions in controller training costs, but what is the actual end-to-end inference overhead of BELT, including probe execution and hidden-state processing? Does the reduction in generated tokens translate into comparable wall-clock latency improvements?
3 comments
Response to Reviewer 378R (Part 2)
Q4. End-to-end inference overhead and wall-clock latency.
Thanks for the question. We have added the following wall-clock results to the revised version (Qwen3-1.7B with BELT-H):
- Belief modeling adds a small Overhead per token. Decoding time per token increases by less than 1% (16.87 instead of 17.03 ms), consistent with the arithmetic estimate in App. D.1. Half of this time is reading the hidden states and bookkeeping, each probe call at a block boundary takes only 2.7 ms.
- BELT reduces wall-clock significantly End to end. When answering questions end-to-end, BELT saves 23.3% of the wall-clock thanks to generating 24.1% less tokens, without performance drop.
Concluding remarks: We would be grateful if you could let us know if our explanations have been satisfactory. If so, we kindly ask that you consider increasing your rating. We are also open to discussing any other questions you may have.
Gentle reminder and follow-up on our response
Dear reviewer 378R,
Thank you again for your time and insightful comments. We truly appreciate your constructive feedback, which has helped us significantly improve the paper.
As the discussion period is coming to a close soon, we wanted to gently follow up to ensure that our previous responses and additional experiments have adequately addressed your concerns
If you have any remaining questions or need further clarification, we would be more than happy to provide them before the deadline. Thank you very much for your consideration and effort!
Best regards, The Authors
Response to Reviewer 378R (Part 1)
Thank you for the thorough review and the concrete questions. We answer them grouped by topic.
W1 and Q1. The methodological novelty is somewhat limited, as BELT builds on existing probe architectures and utility-based stopping objectives … how much of the improvement comes from the four prediction targets rather than the additional sequence encoder capacity?
Novelty of BELT lies not in the architecture of the probe, but in the belief representation learning and policy-belief disentanglement: one probe learns 4 price independent outcome beliefs (Same, Later, Now, End). Prior work mainly leverages the Now belief (current answer correctness; e.g., DEER, CoDE-Stop and Thought Calibration), which is less expressive than the multi-belief objective. Together, these beliefs give the linear policy a more informative basis for stopping decisions and allow refitting linear policies of different token prices on a token-price-agnostic representation.
targets versus encoder capacity Ablation studies in Section 6 show that The training target is the main driver of the gain. Capacity helps, but is not the main driver. (1) Linear probe trained on the BELT targets beats the OSP-1 features in utility (Table 4); (2) BELT architecture with OSP objective leads to policy collapses (App.E.2); (3) Ablating Later and Same lowers utilities (Table 6a).
W1. The performance gains over the strongest baseline are relatively modest.
While BELT is only slightly better than the OSP baseline in terms of performance, it is much more efficient and flexible.
BELT vs. a single OSP checkpoint: 6.3 times fewer parameters and about 4.1 times fewer inference FLOPs per token, over 20 times less training compute for five prices. Belief training also supervises every boundary equally, whereas utility gradients vanish at boundaries the policy rarely reaches (Eq. 11). For a larger model, the contrast in training and inference efficiency will be more extreme. For OLMo-7B, OSP checkpoint scales from 101M to 405M, while BELT scales from 16M to 19M.
W2 and Q2. the main results rely on test-frontier evaluation, which selects operating points using test-set performance and may not fully reflect realistic deployment settings.
We follow the prior work convention. Nevertheless, to address your concern, we provide additional results.
Prior work convention: Reading a frontier point requires fixing one coordinate (an accuracy tolerance or a token budget), so the test frontier is the standard Pareto comparison. It follows Thought Calibration and OS-Pruner, and is applied identically to every method.
Unseen set selection: In appendix E and Appendix B, we control accuracy/cost and select the member to be compared on half of the test set, and report the actual comparison result on the other half. So this is an unseen selection setting you have requested.
Extra Results: we did another selection on a smaller dev set. At a 1-pp accuracy drop tolerance, BELT-H is comparable to full OSP and saves 6-7 more tokens than OSP-1. We will include this in the final version. But importantly, the test frontier selection is still the convention of this field, and is the only viable approach to compare the algorithms given their multiple family members.
W2 and Q3. The mathematical benchmarks are restricted to filtered integer-answer subsets.
Thanks for the detailed reading! We did the filtering mainly for label reliability, following DAPO-Math and Wang et al. (2024), as explained in App.A. Here we add full results for math OOD test sets without integer filtering:
No-filtering OOD results: For MATH-500, OlympiadBench, and Gaokao, BELT-H outperforms the strongest baseline by +12.8, +23.2, +15.2 of token saving with no tolerance on accuracy drop. The BELT-H is the same one trained on integer-only math problems.
Further results show that this data preprocessing does not hurt generalizability. For example, the math-trained probe transfers to LiveCodeBench graded by unit tests (Sec. 5).
New generalization results: We conduct an additional experiment, where we show that BELT trained on math is applicable for agents in ALFWorld and ScienceWorld. It removes about 85% of thinking tokens at or above the natural score in ScienceWorld, and about 30% within 2 points of natural success in ALFWorld, with large seed variation.