acceptodds
Withdrawn from ICLR 2027

How Faithful Is a Truncated Reasoning Trace? Measuring and Modeling Biased Feedback in Test-Time Compute Allocation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Test-time compute allocation methods must decide how many reasoning tokens to spend on each query. A tempting shortcut is to truncate one long reasoning trace and score each prefix, which yields feedback for every smaller budget without another rollout. Whether such truncated observations faithfully stand in for a native short run has not, to our knowledge, been measured. Comparing them against a same-arm decoding-noise control across budgets from 128 to 2048 output tokens, we find no detectable excess disagreement at budgets of 256 tokens or more, but an excess at 128 tokens (22.5% vs. 11.3% disagreement), the regime where a cost-aware allocator most wants to operate. The excess is bidirectional, so no fixed correction removes it. In an offline replay of an online allocator, reusing truncated feedback helps when tokens are expensive and shows no detectable benefit when they are cheap; removing the 128-token budget from the ladder makes reuse beneficial at the cost weights we tested. Theoretically, we show that pooling biased free observations can be inconsistent, give a regret bound that adapts to which levels’ free observations are trusted, and show that each level’s free observations saturate, so they cannot replace native pulls. Our results come from one model and 75–160 problems, and no learned policy we tested saves tokens at matched top quality.

Primary area:
reinforcement learning
Keywords:
large language models, inference-time scaling, test-time compute, truncated reasoning traces, reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.60 over 5 reviews, above 10% of the others.
Reviewer_vMcEOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates whether truncated reasoning traces provide reliable feedback for shorter token budgets. The authors find significant excess disagreement at 128 tokens, but not at larger budgets, and examine its impact on compute allocation through offline contextual-bandit experiments. They also propose TRUNC-UCB, which handles potentially biased truncated feedback and provides regret bounds.

Critical strengths
  1. The paper investigates whether truncated reasoning traces provide reliable feedback for compute allocation, which is an important issue for methods that reuse intermediate reasoning results.

  2. The attempt to connect feedback bias with regret analysis is potentially valuable.

Critical weaknesses
  1. Table 1 estimates truncation bias using excess disagreement, whereas Assumption 2 requires a bound on the conditional expected reward difference. These quantities are not equivalent, and the former does not generally bound the latter.

  2. The proof of Theorem 8 appears to use assumptions not implied by the context-diversity condition. In particular, the pooled observations are selected adaptively based on context, so their empirical covariance may not remain well-conditioned. The proof also appears to replace time-varying observation counts with final counts when bounding cumulative confidence widths.

Questions for authors
  1. How does the excess-disagreement statistic in Table 1 estimate or upper-bound the conditional expected reward bias in Assumption 2?

  2. I want to know how Assumption 4 can ensure covariance concentration for the adaptively selected pooled observations (Appendix L.4).

Reviewer_vBXgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether scores obtained by truncating a long reasoning trace can serve as reliable feedback for shorter test-time compute budgets. The authors compare truncated and native rollouts against a same-budget decoding-noise control across five token budgets. They then study the effect of reusing truncated feedback through offline simulations of compute allocation policies. The paper also formulates the problem as a bandit with biased nested feedback and develops a two-channel UCB algorithm with regret analysis. The experiments report excess disagreement at the shortest budget and show that the benefit of feedback reuse depends on the token-cost setting.

Critical strengths

(i) The paper examines whether a truncated long reasoning trace can reliably represent a native short-budget rollout, an important issue for test-time compute allocation methods that reuse reasoning traces.

(ii) The experiments distinguish truncation effects from ordinary decoding randomness using a same-budget control, and report both the benefits and failures of feedback reuse under different token-cost settings.

Critical weaknesses

(i) The empirical measurement of truncation bias does not match the bias defined in Assumption 2. Section 4 measures bias as the difference between truncated-versus-native disagreement and native-versus-native disagreement. However, Assumption 2 requires a bound on the difference in conditional expected rewards, which is not equivalent to a difference in disagreement rates. The paper does not establish a relationship between these two quantities, yet directly uses the measured disagreement differences as bias estimates in the theoretical analysis and trust rule.

(ii) The proof of Theorem 8 contains an unjustified step in the pooled-channel regret analysis. In Appendix L.4, Step 3b replaces the time-dependent observation counts (m_k(t)) with the final counts (m_k(T)) when deriving an upper bound. Since (m_k(t) \leq m_k(T)), this replacement makes the reciprocal square-root terms smaller and does not establish the claimed upper bound. The subsequent telescoping argument in Eq. (21) only applies to the final counts. Therefore, the pooled-channel regret bound does not follow from the presented derivation.

Questions for authors

(i) Please explain how the excess disagreement measured in Section 4 can justify the conditional expected-reward bias bound in Assumption 2, given that these are different quantities.

(ii) Please explain how the upper bound in Step 3b of Appendix L.4 is derived when replacing the time-dependent observation counts (m_k(t)) with the final counts (m_k(T)).

Reviewer_3GcgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors study the faithfulness of a shortcut used in test-time compute allocation. An online allocator chooses, per query, how many output tokens to allow, trading correctness against token cost. To learn which budget suits which query, one efficient shortcut is to generate a single long output (reasoning followed by a final answer), truncate it at each smaller budget, and grade the answer extracted from each prefix. This yields feedback for every smaller budget without additional generation. However, it is unclear whether these truncated results represent what the model would actually achieve when run natively under that budget.

The paper analyzes this with a single hosted reasoning model on GSM8K and MATH-500, truncating 2048-token outputs to budgets of 128–1024 tokens and comparing the result against a decoding-noise control. It finds no detectable bias at 256 tokens or more, but a bias at 128 tokens that goes in both directions. It concludes that truncated feedback is sound above the shortest budget, that its value lies in saving tokens rather than in statistical efficiency, and that it cannot replace native runs. An offline bandit replay and a theoretical analysis support this conclusion.

Critical strengths
  1. The question is useful: many allocation and training methods rely on truncated feedback without checking it, and the answer matters whether or not the shortcut turns out to be faithful.
  2. The measurement is well controlled, using a same-budget decoding-noise control, a paired McNemar test, and a 2048-token calibration row (L190–206, Table 1), so the observation is clear within the tested setting.
Critical weaknesses

My main concern is that the experimental scale is too small. For an analysis paper to give the community a useful finding, the experiments should be comprehensive and generalizable, or at least show signs of generalizability. The current setting is limited:

  1. Single model. All measurements come from one hosted model and one answer-extractor model. Without other model sizes or families, it is unclear whether the 128-token bias is a property of this model or of budget-limited reasoning in general.
  2. Narrow budget range. The budget is a hard cap on the whole output (reasoning plus answer), not a thinking budget. It only goes up to 2048 tokens, which is short for current reasoning models, and all truncations start from a single 2048-token output. Whether the bias depends on the gap between the original and the truncated length is untested.
  3. Few tasks. The truncation test covers only GSM8K and MATH-500. The other two benchmarks (MBPP, MMLU-Pro) have no truncation variant.
  4. Small sample. The fidelity test uses about 80 problems per budget, and the headline 128-token result comes from 12 vs. 3 disagreeing problems. With four budgets tested and no multiple-comparison correction, p = 0.035 would not survive a Holm or Bonferroni correction (≈0.14). This makes the statistical significance of the key findings questionable.
Questions for authors

None

Reviewer_yzB9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies whether truncating a long reasoning trace at different token budgets can provide reliable feedback for test-time compute allocation, instead of running the model separately at each budget. The main concern is that a model given a shorter budget might reason differently from one that is initially given a longer budget and then truncated.

The authors first measure how closely truncated traces match native shorter runs, while accounting for randomness in model generation. They find that truncation introduces no statistically significant additional disagreement for budgets of 256 tokens or more, but at 128 tokens, the disagreement increases substantially. They then study how this bias affects compute allocation using contextual bandit algorithms. Their experiments show that reusing truncated feedback can improve cost-adjusted utility when tokens are expensive, but offers little benefit when token costs are low.

On the theoretical side, the paper shows that directly combining truncated and native feedback can lead to incorrect budget selection and linear regret. The authors propose a bias-aware formulation and show that truncated observations eventually reach a point where collecting more of them provides little additional information.

The experiments use GSM8K, MATH-500, MBPP, and MMLU-Pro, with token budgets ranging from 128 to 2048. The authors validate their findings through statistical comparisons, offline allocation experiments across 50 seeds, and theoretical simulations.

Overall, the paper makes a useful contribution by showing that truncated reasoning traces are not always reliable substitutes for shorter runs. However, the evaluation is limited to a single reasoning model and relatively small datasets, and the allocation experiments are conducted offline rather than in a real deployment.

Critical strengths

The main strength of the paper is its empirical finding that truncating a longer reasoning trace at a budget BB is not necessarily equivalent to generating a reasoning trace with budget BB from the beginning. This is an important distinction, as the model may adapt its reasoning strategy depending on the available token budget. The authors validate this by comparing truncated traces against native budgeted runs while accounting for decoding randomness, and find significant additional disagreement at smaller budgets (128 tokens).

I consider this finding the strongest contribution of the paper, as it challenges an implicit assumption behind using truncated reasoning traces as inexpensive feedback for test-time compute allocation. It also highlights why such feedback needs to be evaluated carefully before being used to train budget allocation policies.

Critical weaknesses
  1. Limited experimental validation: The main finding of the paper is supported by experiments on only a single reasoning model and a relatively small number of problems. More importantly, the statistically significant difference between truncated and native reasoning is observed only at the 128-token budget. This raises questions about how well the findings generalize across different reasoning models, tasks, and budget ranges. Evaluating multiple models, particularly those with explicit reasoning-budget controls, would make the central claim much stronger.

  2. Limited practical improvement in compute allocation: Although the paper provides a theoretical framework for handling biased feedback, the experiments do not demonstrate a clear improvement in inference efficiency. In particular, none of the learned allocation policies achieve token savings while maintaining the accuracy of the best fixed-budget baseline. This makes it difficult to assess the practical benefit of the proposed approach beyond identifying the truncation bias itself.

Questions for authors
  1. Generalization across reasoning models: The main finding of the paper is that truncating a longer reasoning trace at budget BB is not necessarily equivalent to generating a trace directly with budget BB. However, this is validated using only a single reasoning model, with statistically significant disagreement observed only at 128 tokens. Do the authors have any additional evidence that this behavior generalizes across different reasoning models? This would help establish whether the observed bias is a general property of budget-conditioned reasoning or specific to the model evaluated.

  2. Impact of the original reasoning budget: The experiments primarily consider truncating a 2048-token trace to smaller budgets. Would the disagreement change if the original trace were generated with a smaller budget, such as truncating 512 tokens to 128 tokens? Understanding whether the bias depends on the difference between the original and target budgets would help assess the reliability of truncated feedback.

  3. Practical benefit of the allocation framework: The paper shows that truncated feedback can improve cost-adjusted utility in certain settings, but none of the learned policies achieve token savings at matched top accuracy. Could the authors clarify under what practical operating conditions their approach would be preferable to simply selecting a fixed token budget? This would help establish the practical significance of the proposed framework.

Reviewer_K6jQOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper investigates if truncated reasoning traces can be used as feedback to predict the reward for an online algorithm that allocates the test time compute budget.

Critical strengths
  • The paper investigates a very relevant topic with potentially big implications
  • The paper is easy to understand
Critical weaknesses
  • The paper's empirical study is extremely limited. I could not find which model was used, the set of reasoning budgets, and tasks are very limited. The tasks itself are rather old and outdated by today's standards.
  • The paper fails to provide meaningful impact to the ML community. As faithfully stated in the paper, many results are negative. Instead of deferring to a future work, I suggest the authors investigate this further until meaningful contributions are made
Questions for authors

What model or models were used in the study?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.