- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper investigates whether truncated reasoning traces provide reliable feedback for shorter token budgets. The authors find significant excess disagreement at 128 tokens, but not at larger budgets, and examine its impact on compute allocation through offline contextual-bandit experiments. They also propose TRUNC-UCB, which handles potentially biased truncated feedback and provides regret bounds.
Critical strengths
-
The paper investigates whether truncated reasoning traces provide reliable feedback for compute allocation, which is an important issue for methods that reuse intermediate reasoning results.
-
The attempt to connect feedback bias with regret analysis is potentially valuable.
Critical weaknesses
-
Table 1 estimates truncation bias using excess disagreement, whereas Assumption 2 requires a bound on the conditional expected reward difference. These quantities are not equivalent, and the former does not generally bound the latter.
-
The proof of Theorem 8 appears to use assumptions not implied by the context-diversity condition. In particular, the pooled observations are selected adaptively based on context, so their empirical covariance may not remain well-conditioned. The proof also appears to replace time-varying observation counts with final counts when bounding cumulative confidence widths.
Questions for authors
-
How does the excess-disagreement statistic in Table 1 estimate or upper-bound the conditional expected reward bias in Assumption 2?
-
I want to know how Assumption 4 can ensure covariance concentration for the adaptively selected pooled observations (Appendix L.4).