- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper formalizes the distribution of answers obtained by interrupting a reasoning process and applying a fixed answering procedure. It relates changes in that distribution to changes in bounded utility, and derives stopping-regret bounds under cumulative-movement or oracle-proximity assumptions. It also explains why stable token entropy need not certify stable answer quality. An entropy-based controller is evaluated on two Qwen3.5 models and four mathematics benchmarks. It reduces output tokens relative to horizon-limited native termination, but does not establish accuracy preservation and is often worse than calibrated fixed budgets.
Critical strengths
-
The paper makes an important distinction explicit. A distribution can change substantially without changing entropy, and a quiet recent prefix does not exclude useful later computation. Sections 5–6 explain these limitations and the mismatch between full-answer distributions and first-token probes. The question connects to established work on compute allocation (Snell et al., ICLR 2025), budget control (s1, EMNLP 2025), and stability-based stopping (Certaindex, NeurIPS 2025). The submission already cites these directions; distinguishing what their signals do and do not certify is worthwhile.
-
The reporting is candid about unfavorable evidence. The authors include a calibrated fixed-budget baseline, distinguish output tokens from serving latency, and explicitly state that noninferiority was not established. The finding that a simple fixed budget can outperform an adaptive entropy rule could be informative if the underlying comparison were sufficiently well controlled.
Critical weaknesses
-
The theory does not yet demonstrate sufficient insight or decision value beyond its conditional statements. Proposition 4 follows directly from bounded-utility transport and Pinsker's inequality. Theorem 5 can reduce to the trivial horizon bound unless a sufficiently small cumulative movement budget is available. Theorem 6 assumes that stopping is already near the oracle and that intervening movement is small; the window rule establishes neither premise. These are legitimate conditional results, but the paper does not demonstrate a meaningful setting where the assumptions can be checked and the guarantee informs a decision. The implemented entropy rule is expressly outside those guarantees. A comparison with Certaindex's stability assumptions should clarify what additional conclusion this framework enables. This is a concern about demonstrated contribution, not an allegation of incorrect mathematics. A finite-answer example with computed certificates and an informative numerical bound would help establish the theory's value.
-
The empirical study cannot yet explain the negative result reliably. In Table 2, the controller loses 13 and 18.5 accuracy points to the fixed baseline on MATH while using more tokens. Yet Section 7.6 reports unresolved grading for 32% of primary MATH answers, and 233 of 378 competition answers reach their answer cap. These failures can depend on the policy, making it difficult to distinguish a poor stopping signal from answer-format, decoding, or measurement effects. The controller also uses a signed entropy-drop threshold, whereas EAT uses an entropy-variance criterion; failure of this implementation is not evidence against all entropy-based stopping. A matched answer-stability control motivated by Certaindex would help distinguish failure of this proxy from limitations of adaptive stopping more broadly. Negative results need not beat a baseline, but they should isolate what failed and why. Development reuse of the evaluation cohort further limits the conclusions.
Questions for authors
- Theory clarification: Is there a concrete example in which the proposed certificates are computable, their assumptions are verifiable, and Theorem 5 or 6 yields an informative numerical guarantee before seeing the correct answer?
- Measurement clarification: After a blinded audit of unresolved answers and a common extraction procedure, do the MATH and competition comparisons retain their direction and magnitude? Please separate unresolved grading from genuinely incorrect answers without dropping difficult cases.
- Prior-work and mechanism clarification: How do the stopping assumptions differ from Certaindex's, and does an answer-stability control retain the observed failure under matched decoding, calibration, grading, and probe-cost accounting? This would clarify what the present theory and experiment add beyond established stopping approaches.