acceptodds
Under review as a conference paper at ICLR 2027

TrajUQ: Predicting LLM Reasoning Failure under Limited Observation Budgets

Abstract

Short continuations offer evidence of failure risk before a large language model (LLM) answers, but observing more branches or following them longer consumes tokens. Can better readout of finite, imperfect probe scores compensate for fewer observations? We address this question with TrajUQ, combining the endpoint mean and the empirical distribution of centered branch-score vectors to predict continuation failure. We evaluate 540 questions across three tasks and thirteen model configurations using question-grouped out-of-fold prediction. With ten-branch, 64-token fits frozen, our TrajUQ readout improves Brier over the learned mean by 3.125% on average across 20 matched combinations of branch count and observation depth, each using at least two branches. In our post-hoc scan, two-branch TrajUQ achieves lower Brier than the ten-branch, 64-token mean readout using 80% fewer observation tokens. At the reference budget, our matched tuning yields Brier 0.1947 versus 0.2011 for the learned mean, also improving on the tested second-order and distribution-only controls. Using these predictions, our offline screening selects prefixes with lower empirical failure at matched coverage. Our offline TrajUQ candidate selection reduces final-answer failure from 56.47% for a single answer chosen uniformly from the same candidate pool to 51.92% without updating LLM parameters. These results connect better use of limited observations to improved risk prediction and more reliable offline answer selection.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Uncertainty Quantification, Failure Risk Estimation, Continuation Sampling

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_QRTWOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes TrajUQ, a method for predicting the probability of failure of an LLM reasoning trajectory from a small number of short continuations. The method combines the mean of branch-level failure predictions with information about their distribution, using a kernel regression model. The authors evaluate the approach on three QA datasets and 13 model configurations. They report consistent improvements in Brier score over a learned mean baseline, as well as some improvements in offline answer selection.

Critical strengths

The problem of predicting reasoning failure before generation is completed is interesting and practically relevant. Such predictions could potentially be used to stop unsuccessful reasoning trajectories early and reduce inference costs.

The paper presents a systematic evaluation across multiple models, with several baselines and ablations. The improvements over the learned mean baseline are relatively small but consistent. I also appreciate the attempt to evaluate the trade-off between prediction quality and observation budget.

Critical weaknesses

My main concern is the limited experimental setting. The evaluation uses only 180 questions from each of three QA datasets. In particular, there are no mathematical reasoning benchmarks, such as GSM8K, MATH, or AIME, or other tasks involving long reasoning trajectories. These seem to be the most natural applications of the proposed method. Without such experiments, it is difficult to assess its practical value.

Another limitation is that the method requires supervised training for each model–task pair. The paper does not investigate transfer to unseen tasks, which limits its applicability to general-purpose LLM inference.

I also find the baseline comparisons not fully convincing. Several existing uncertainty estimation methods are adapted to this specific setting and have different supervision requirements. Although the authors include a baseline based on second-order statistics, I would like to see comparisons with simpler supervised predictors using the same branch scores and training data, especially in the two-branch setting.

Finally, the reported reduction in observation tokens does not directly demonstrate a reduction in end-to-end inference cost. The answer selection experiments are offline and do not establish whether the method can improve practical reasoning efficiency.

Questions for authors
  1. Can the authors evaluate TrajUQ on mathematical reasoning benchmarks such as GSM8K, MATH, or AIME, preferably using models with long reasoning trajectories? This is my main concern. I would particularly like to see whether the predicted failure risk can be used for early stopping or restarting reasoning, and whether this improves accuracy or reduces inference costs. A convincing positive result on mathematical reasoning tasks would be the main factor that could change my assessment of the paper.

  2. How well does the method generalize across tasks? For example, can a probe and readout trained on two datasets be applied to a third without additional training?

  3. Could the authors compare TrajUQ with simpler supervised models, such as an MLP or gradient boosting on the mean, variance, and covariance of branch scores, using the same training data and hyperparameter tuning budget?

  4. Could the authors clarify the selection procedure for the 180 questions per dataset and explain why this sample size was chosen? It would also be useful to discuss how the results might depend on the selection of questions.

5 comments
AuthorsOpenReview ↗

Rebuttal (1/3)

Thank you for emphasizing mathematical reasoning, inference cost, and transfer across tasks. Your questions have helped us improve the paper's clarity and strengthen its empirical evaluation. Below, we address each of your concerns. We present results on mathematical reasoning, online restarting, transfer across tasks, and MLP/GBDT comparisons, and explain how we sampled the benchmark questions.

Critical Weaknesses and Questions For Authors

W1

My main concern is the limited experimental setting. The evaluation uses only 180 questions from each of three QA datasets. In particular, there are no mathematical reasoning benchmarks, such as GSM8K, MATH, or AIME, or other tasks involving long reasoning trajectories. These seem to be the most natural applications of the proposed method. Without such experiments, it is difficult to assess its practical value.

Q1

Can the authors evaluate TrajUQ on mathematical reasoning benchmarks such as GSM8K, MATH, or AIME, preferably using models with long reasoning trajectories? This is my main concern. I would particularly like to see whether the predicted failure risk can be used for early stopping or restarting reasoning, and whether this improves accuracy or reduces inference costs. A convincing positive result on mathematical reasoning tasks would be the main factor that could change my assessment of the paper.

W4

Finally, the reported reduction in observation tokens does not directly demonstrate a reduction in end-to-end inference cost. The answer selection experiments are offline and do not establish whether the method can improve practical reasoning efficiency.

Response. On MATH-500, we improve accuracy by 1.4 percentage points and reduce total generated tokens by 5.4% with TrajUQ-guided restarting.

  1. Mathematical reasoning. We evaluated all 500 MATH-500 questions, fitting on training folds and testing held-out questions using our original question-grouped protocol (p. 5, lines 229–235). We used one prefix per question with H=64H=64 and S=16,64S=\\{16,64\\}. We kept the K=10K=10 probe, readouts, and shrinkage fixed when evaluating K=2K=2. With ten branches, TrajUQ lowers Brier by 0.00590.0059 and 0.00640.0064 on the two models. With two branches, we retain reductions of 0.00220.0022 and 0.00130.0013 against the ten-branch learned mean (Table 1).

  2. Online stopping and restarting. On MATH-500 with Qwen3.8-27B, we increase accuracy from 84.8% to 86.2% and reduce average total generated tokens from 1,697 to 1,605 (Table 2). Compared with mean-guided restarting, we gain 0.8 percentage points in accuracy and use 2.8% fewer tokens. After a 512-token prefix, we observe the live and one auxiliary branch for up to 64 tokens each (K=2K=2, H=64H=64, S=16,64S=\\{16,64\\}). We allow at most one restart and use a common 65,536-token total cap for all four policies.

    On StrategyQA with Qwen2.5-72B-Instruct, we increase accuracy from 70.0% to 72.3% and reduce total generated tokens by 6.9%. We also gain 0.8 percentage points in accuracy and use 3.0% fewer tokens than mean-guided restarting. Here, we observe both branches for up to 16 tokens after a 16-token prefix (K=2K=2, H=16H=16, S=8,16S=\\{8,16\\}), with at most one restart and a 192-token total cap.

    For each task, we share the scorer and observation budget between the two readouts. We fit readouts and select thresholds on training questions. We count every generated token, including auxiliary branches, discarded attempts, and restarts.

  3. Original QA coverage. We evaluated MMLU-Pro, StrategyQA, and TriviaQA (p. 5, lines 252–253), prompting models to provide reasoning, including multi-hop reasoning for StrategyQA (p. 14, lines 705–714). We explain question selection and sample size in Q4.

Table 1. Measured MATH-500 results with task-specific fitting and held-out evaluation.

Model and TrajUQ budgetLearned mean (K=10K=10) Brier ↓\downarrowTrajUQ Brier ↓\downarrowTrajUQ −- learned mean [95% CI]
Qwen3.8-27B (K=10K=10)0.13850.1326−0.0059,[−0.0075,−0.0045]-0.0059\\,[-0.0075,-0.0045]
Qwen3.8-27B (K=2K=2)0.13850.1363−0.0022,[−0.0032,−0.0012]-0.0022\\,[-0.0032,-0.0012]
Gemma 4-E4B-it (K=10K=10)0.17710.1707−0.0064,[−0.0096,−0.0043]-0.0064\\,[-0.0096,-0.0043]
Gemma 4-E4B-it (K=2K=2)0.17710.1758−0.0013,[−0.0028,−0.0006]-0.0013\\,[-0.0028,-0.0006]
AuthorsOpenReview ↗

Rebuttal (2/3)

Table 2. Online stopping and restarting. StrategyQA: Qwen2.5-72B-Instruct, 180 questions averaged over multiple random seeds. MATH-500: Qwen3.8-27B, 500 questions. Tokens denote total generated tokens per question.

PolicyStrategyQA / Accuracy (%) ↑\uparrowStrategyQA / Tokens ↓\downarrowMATH-500 / Accuracy (%) ↑\uparrowMATH-500 / Tokens ↓\downarrow
Single generation70.0132.784.81697
Random restart68.8130.583.21674
Restart using mean71.5127.485.41651
Restart using TrajUQ72.3123.686.21605

W2

Another limitation is that the method requires supervised training for each model–task pair. The paper does not investigate transfer to unseen tasks, which limits its applicability to general-purpose LLM inference.

Q2

How well does the method generalize across tasks? For example, can a probe and readout trained on two datasets be applied to a third without additional training?

Response. We retain the readout advantage on all three held-out QA tasks without target-task fitting.

  1. Source-only fitting. With Qwen2.5-7B-Instruct, we trained on two QA tasks and evaluated the third, rotating the target. We fitted preprocessing, the probe, and the readouts and selected hyperparameters using source tasks, then froze the full pipeline. We used K=10K=10, H=64H=64, and S=16,64S=\\{16,64\\} for both readouts. We used target labels only for evaluation.

  2. Transfer results. We reduce Brier by 0.0013–0.0092 across all three targets compared with the learned mean readout (Table 3).

Table 3. Measured transfer results: training on two tasks and freezing the full pipeline for the third.

TargetLearned mean BrierTrajUQ BrierTrajUQ −- learned mean (95% CI)
MMLU-Pro0.23670.2354$-0.0013[0.0013 [-$0.0031, $-$0.0008]
StrategyQA0.23770.2309$-0.0068[0.0068 [-$0.0101, $-$0.0038]
TriviaQA0.24980.2406$-0.0092[0.0092 [-$0.0137, $-$0.0052]

W3

I also find the baseline comparisons not fully convincing. Several existing uncertainty estimation methods are adapted to this specific setting and have different supervision requirements. Although the authors include a baseline based on second-order statistics, I would like to see comparisons with simpler supervised predictors using the same branch scores and training data, especially in the two-branch setting.

Q3

Could the authors compare TrajUQ with simpler supervised models, such as an MLP or gradient boosting on the mean, variance, and covariance of branch scores, using the same training data and hyperparameter tuning budget?

Response. We obtain lower Brier by jointly reading the mean and branch-score distribution, with the clearest gains at K=10K=10. At K=10K=10, MLP and GBDT improve on the second-order readout, while we further reduce Brier by 0.00300.0030 and 0.00240.0024, respectively (Table 4).

  1. Input representation. We give the first three readouts the same mean, variances, and covariance (O,v_{16},v_{64},v_{16,64}) (Appendix B.7, p. 17, lines 873–878). With TrajUQ, we retain OO and the empirical distribution UU of complete, centered score vectors across depths (p. 4, lines 195–202, Eq. 2). At K=10K=10, we thereby preserve distributional structure beyond second moments. At K=2K=2, second moments determine UU, and the losses are close (p. 23, lines 1239–1240).

  2. Matched comparison. We use Qwen2.5-72B-Instruct / StrategyQA with 180 questions, H=64H=64, and S=16,64S=\\{16,64\\}. We fit and test each readout separately at K=2K=2 and K=10K=10, sharing branch scores, independent continuation labels, and question-grouped folds (p. 5, lines 237–241; p. 7, lines 365–367). We search nine parameter settings per method and select parameters and a blend with the endpoint mean using inner validation folds.

Table 4. Measured supervised readouts on Qwen2.5-72B-Instruct / StrategyQA at matched branch budgets (Brier ↓\downarrow).

PredictorInput featuresK=2K=2K=10K=10
Kernel ridge regressionMean, variance, covariance0.18800.1873
MLPMean, variance, covariance0.18780.1871
GBDTMean, variance, covariance0.18830.1865
Kernel ridge regression (TrajUQ)Mean OO + full empirical distribution UU0.18760.1841
AuthorsOpenReview ↗

Rebuttal (3/3)

Q4

Could the authors clarify the selection procedure for the 180 questions per dataset and explain why this sample size was chosen? It would also be useful to discuss how the results might depend on the selection of questions.

Response to Q4. We randomly sampled 180 questions from each benchmark (Appendix B.1, p. 14, lines 723–728). We chose this size to keep continuation collection manageable across 13 model configurations.

We collected 1,053,000 continuations (180×3×13×5×30\boldsymbol{180\times3\times13\times5\times30}). Across the three benchmarks and 13 model configurations, we used five prefix positions per question and sampled 30 continuations per prefix (p. 5, lines 252–261; p. 14, lines 729–735). We compare readouts on the same questions and use question-grouped splits and bootstrap intervals to assess sampling variability (p. 3, lines 145–148).

We also evaluated all 500 MATH-500 questions, retaining the readout advantage on both models (Table 1).

If we have addressed your concerns, we would be grateful if you could raise your score in light of the additional evidence.

Reviewer_xW5jOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors present TrajUQ, a method for predicting the failure risk of unfinished LLM reasoning trajectories from a limited number of short sampled continuations. Failure risk is defined as the probability that a fresh continuation from a given reasoning prefix produces an incorrect final answer. The method they proposed is motivated by the idea that averaging predicted failure scores across sampled continuations may discard useful information about how those scores vary across branches. To address this, TrajUQ combines the mean failure score at the observation endpoint with the empirical distribution of centered branch score vectors at selected observation depths. The scores come from a trained and frozen hidden state probe, and the combined representation is used in a kernel ridge regression readout trained on failure rates from independently sampled full continuations.

The authors evaluate TrajUQ on 540 questions from three benchmarks (MMLU-Pro, StrategyQA, TriviaQA) across 13 model configurations. At the reference budget (10 branches, 64 tokens), they report a 3.2% reduction in pooled Brier score over a matched learned-mean baseline (0.2011 to 0.1947). In a post-hoc analysis, two-branch TrajUQ outperforms the ten-branch mean baseline with 80% fewer observation tokens. They also report improvement in offline prefix screening and a TriviaQA answer-selection study, where TrajUQ reduce final-answer failure from 56.47% (uniform selection) to 51.92%.

Critical strengths
  1. The paper provides a set of evaluations using question-grouped nested folds, cross-fitted probe scores, and independently sampled continuation outcomes. The mean-only, second-order, and distribution-only readouts are fitted and tuned using matched procedures. This makes the reported improvement more convincing, especially given the small Brier score difference.

  2. In the results, the improvement is not limited to the Brier score. TrajUQ also increases pooled AUROC from 0.616 to 0.685, suggesting better discrimination between higher- and lower-risk prefixes. The Brier improvements are also consistent across all three tasks and 13 model configurations, although the absolute AUROC remains moderate.

Critical weaknesses
  1. The source of the improvement is not fully clear. Although TrajUQ outperforms the simpler readouts, the comparisons change both the representation and the induced kernel. To b more specific, the second-order summary readout provides little improvement over the mean, while TrajUQ performs considerably better, even with only two branches. This makes it difficult to determine whether the gains come from richer distributional information or the particular kernel design, which limits the interpretation of the main contribution.

  2. The evaluation is limited to 540 questions from three QA benchmarks, and the authors mention in Section 6 that the same dataset informed method development. Although the question-grouped nested evaluation reduces concerns about training–test leakage, it does not show whether the improvements hold on completely unseen datasets or reasoning tasks

Questions for authors
  1. In Section 5.4, the comparisons between TrajUQ and the simpler readouts change both the representation and the induced kernel. What evidence shows that the improvement comes from the distributional representation rather than the kernel design? Could the authors compare alternative kernels on the same representation to help separate these effects?

  2. Inn Table 5, the selected bandwidth multiplier is peta = 2, the largest value in the grid, for all three readouts in every fold. Could extending the grid change the relative performance of the methods?

4 comments
AuthorsOpenReview ↗

Rebuttal (1/2)

Thank you for your questions about kernel choice, bandwidth selection, and generalization. Following your constructive suggestions, we have clarified our contribution and strengthened the empirical evaluation. Below, we compare representations under both outer kernels, test a wider bandwidth grid, and report results on MATH-500 and transfer without target-task fitting.

Critical Weaknesses and Questions For Authors

W1

The source of the improvement is not fully clear. Although TrajUQ outperforms the simpler readouts, the comparisons change both the representation and the induced kernel. To b more specific, the second-order summary readout provides little improvement over the mean, while TrajUQ performs considerably better, even with only two branches. This makes it difficult to determine whether the gains come from richer distributional information or the particular kernel design, which limits the interpretation of the main contribution.

Q1

In Section 5.4, the comparisons between TrajUQ and the simpler readouts change both the representation and the induced kernel. What evidence shows that the improvement comes from the distributional representation rather than the kernel design? Could the authors compare alternative kernels on the same representation to help separate these effects?

Response. At K=10K=10, we gain more from the distributional representation than from changing the outer kernel. In Table 1, replacing second-order summaries with distribution embeddings lowers Brier by 0.00620.0062 under either outer kernel. Switching from linear to Gaussian lowers Brier by 0.00050.0005 for either representation. At K=2K=2, we also obtain lower Brier with the distributional representation: reductions of 0.00160.0016 and 0.00240.0024 under Gaussian and linear outer kernels, respectively. Here, UU is determined by second moments (p. 23, lines 1239–1240); the benefit concerns how we read existing branch variation, rather than additional higher-order information.

We match scores, labels, folds, tuning budgets, and the shared mean term, keeping the distribution's point kernel fixed (p. 17, lines 873–878). We fit, tune, and test separately at each KK; our original two-branch result uses frozen K=10K=10 fits (p. 6, lines 304–307).

Table 1. Representation ×\times outer kernel: measured Brier scores.

Input representationOuter kernelK=2K=2 Brier ↓\downarrowK=10K=10 Brier ↓\downarrow
Mean + second-orderGaussian0.20270.2009
Mean + distributionGaussian0.20110.1947
Mean + second-orderLinear0.20450.2014
Mean + distributionLinear0.20210.1952

W2

The evaluation is limited to 540 questions from three QA benchmarks, and the authors mention in Section 6 that the same dataset informed method development. Although the question-grouped nested evaluation reduces concerns about training–test leakage, it does not show whether the improvements hold on completely unseen datasets or reasoning tasks

Response to W2. We retain the readout advantage on MATH-500 with task-specific fitting. We also retain it on all three QA targets without target-task fitting.

  1. New reasoning benchmark. We evaluated all 500 MATH-500 questions on two models, keeping the readout design unchanged; we did not use this benchmark during method development. Within MATH-500, we fitted the probe and readouts on training folds and evaluated held-out questions using our original question-grouped protocol (p. 5, lines 229–235). We used one prefix per question with H=64H=64 and S=16,64S=\\{16,64\\}. We kept the K=10K=10 probe, readouts, and shrinkage fixed when evaluating K=2K=2. On both models, TrajUQ with ten or two branches achieves lower Brier than the learned mean readout with ten branches (Table 2).

  2. Transfer without target-task fitting. With Qwen2.5-7B-Instruct, we trained on two QA tasks and tested on the third, rotating the target. We fitted preprocessing, the probe, and the readout and selected hyperparameters on source tasks, then froze the full pipeline for target-task evaluation. We used K=10K=10, H=64H=64, and S=16,64S=\\{16,64\\} for both readouts. We reduce Brier by 0.00130.0013–0.00920.0092 across the three targets (Table 3).

  3. Original evaluation scale. We collected 1,053,000 continuations (540×13×5×30\boldsymbol{540\times13\times5\times30}). We chose 540 questions to keep collection across 13 model configurations tractable. For each question–model pair, we use five positions along the reasoning trajectory to define the prefix states (p. 5, lines 252–261). From each prefix, we repeat continuation sampling 30 times: 20 feature continuations and ten independent label continuations (p. 14, lines 729–735). We retain original questions as the units for splitting and uncertainty estimation.

AuthorsOpenReview ↗

Rebuttal (2/2)

Table 2. Measured MATH-500 results with task-specific fitting and held-out evaluation.

Model and TrajUQ budgetLearned mean (K=10K=10) Brier ↓\downarrowTrajUQ Brier ↓\downarrowTrajUQ −- learned mean [95% CI]
Qwen3.8-27B (K=10K=10)0.13850.1326−0.0059,[−0.0075,−0.0045]-0.0059\\,[-0.0075,-0.0045]
Qwen3.8-27B (K=2K=2)0.13850.1363−0.0022,[−0.0032,−0.0012]-0.0022\\,[-0.0032,-0.0012]
Gemma 4-E4B-it (K=10K=10)0.17710.1707−0.0064,[−0.0096,−0.0043]-0.0064\\,[-0.0096,-0.0043]
Gemma 4-E4B-it (K=2K=2)0.17710.1758−0.0013,[−0.0028,−0.0006]-0.0013\\,[-0.0028,-0.0006]

Table 3. Measured transfer results: training on two tasks and freezing the full pipeline for the third.

TargetLearned mean BrierTrajUQ BrierTrajUQ −- learned mean (95% CI)
MMLU-Pro0.23670.2354$-0.0013[0.0013 [-$0.0031, $-$0.0008]
StrategyQA0.23770.2309$-0.0068[0.0068 [-$0.0101, $-$0.0038]
TriviaQA0.24980.2406$-0.0092[0.0092 [-$0.0137, $-$0.0052]

Q2

Inn Table 5, the selected bandwidth multiplier is peta = 2, the largest value in the grid, for all three readouts in every fold. Could extending the grid change the relative performance of the methods?

Response to Q2. We obtain the same relative ranking after expanding the bandwidth grid: TrajUQ retains the lowest Brier. We extended the bandwidth-squared multipliers from 0.5,1,2\\{0.5,1,2\\} to 0.5,1,2,4,8\\{0.5,1,2,4,8\\} (Appendix B.7, p. 17, lines 879–885). We kept the scores, labels, question folds, and regularization grid fixed and retuned each readout within training folds. The second-order readout improves from 0.20090.2009 to 0.20040.2004 with β=4\beta=4; the learned mean and TrajUQ retain β=2\beta=2, with unchanged Brier at the reported precision (Table 4).

Table 4. Measured bandwidth search on the original QA setup (Brier ↓\downarrow).

ReadoutOriginal (reported)Expanded (measured)Selected β\beta
M (mean)0.20110.20112
L (second-order)0.20090.20044
T (TrajUQ)0.19470.19472

Original Brier scores: submission p. 21, Table 9.

If we have addressed your concerns, we would be grateful if you could raise your score in light of the additional evidence.

Dear Reviewer xW5j,

We greatly appreciate your time and effort in reviewing our work. We hope that our responses have clarified our work and addressed your concerns. If so, we would sincerely appreciate your consideration of updating your rating accordingly. If you have any further concerns, we would be happy to provide further clarification.

We highly value every exchange during the review process and sincerely hope to receive further feedback from you at your earliest convenience.

Sincerely,

The Authors

AuthorsOpenReview ↗

Follow-up on our rebuttal

Dear Reviewer,

As the discussion deadline approaches, we would appreciate your feedback on our rebuttal and additional results. We would be happy to clarify any remaining concerns. If we have addressed your concerns, please consider updating your rating accordingly.

Thank you for your time and consideration.

Reviewer_h6AKOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents TrajUQ, which are probes trained on LLMs' hidden states in the process-reward-model style to predict LLM reasoning failures. Experiments are conducted on 3 benchmarks (MMLU-Pro, StrategyQA, and TriviaQA) and thirteen models from four families (Qwen2.5, Yi-1.5, Llama-3.1, Gemma3), and experiments show that TrajUQ outperforms some simple baselines (Table 2).

Critical strengths

None.

Critical weaknesses
  • The positioning of this paper is unclear. Based on my understanding, TrajUQ is essentially a process reward model (PRM). Yet throughout the entire paper PRM is only mentioned in a single paragraph in the related work section.

  • The paper claims to predict LLM reasoning failures, but the experiments do not support this claim. Experiments are conducted on datasets rarely used in reasoning-related experiments. Indeed, MMLU-Pro is knowledge-intensive, not reasoning-intensive. The authors should follow the standard practice and use actual reasoning-intensive benchmarks, e.g., MATH and AIME.

  • The models used in experiments are also outdated, especially Qwen2.5.

  • In the experiments, the most commonly used PRMs are missing from the baselines.

  • The problem setup (L120) is strange. Why does Y=1 indicate an incorrect answer instead of a correct one?

  • Some concepts are not properly defined. For example, what's a "branch-score vector"?

Questions for authors

See weakness section. Most of my questions concern soundness and clarity of the paper, and should be considered as weaknesses.

4 comments
AuthorsOpenReview ↗

Rebuttal (1/2)

Thank you for connecting our UQ problem with process reward modeling. Your suggestions have helped us clarify our contribution and strengthen the evaluation. A PRM can score an individual branch. Given KK branches, we study how to jointly read their mean and variation to predict failure risk beyond mean aggregation. Below, we clarify this distinction and report comparisons on newer models, mathematical reasoning, and PRM scorers.

Critical Weaknesses and Questions For Authors

W1

The positioning of this paper is unclear. Based on my understanding, TrajUQ is essentially a process reward model (PRM). Yet throughout the entire paper PRM is only mentioned in a single paragraph in the related work section.

Response to W1. A PRM supplies prefix scores; we develop the downstream UQ readout that jointly uses scores across branches and observation depths to estimate failure risk beyond mean aggregation.

  1. Input evidence. A PRM scorer conditions each score on the supplied reasoning prefix. From the current prefix xx, we sample KK short continuations and observe the generator's hidden states across both branches and depths under a limited token budget (Section 4.1, p. 3, lines 155–159).

  2. Research objective. PRM training and calibration improve how reasoning prefixes are scored. We study budget-constrained UQ: holding the scorer fixed, we learn to combine evidence from short continuations to estimate and rank the current prefix's final-answer failure risk (p. 3, lines 121–124 and 155–159).

  3. What we learn beyond the mean. From these scores, we learn to jointly read the endpoint mean OO and the distribution UU of centered score vectors. We retain branch differences and the pairing of scores across depths, which a mean alone discards (p. 4, lines 195–202, Eq. 2).

  4. PRM upstream, TrajUQ downstream. We can use a calibrated PRM as scorer gg and jointly read its scores with TrajUQ to estimate failure risk at xx (Figure 1, p. 4). In W4, we hold each scorer fixed and compare mean and TrajUQ readouts.

\xrightarrow[\substack{\text{a calibrated }\mathbf{PRM}\\\\\text{could supply this stage}}]{\text{scorer }g} \\{p_{b,t}\\} \xrightarrow[\text{mean }O+\text{centered distribution }U]{\mathbf{TrajUQ}\text{ joint readout}} \widehat\rho(x).$$ $p_{b,t}$ is the score of branch $b$ at depth $t$: $b=1,\ldots,K$ indexes branches, and $t$ denotes token depth within a branch. We already discuss BetaPRM's success-probability beliefs (Li et al., 2026 [3]) and Distributional PRM's calibrated quantiles (Ma et al., 2026 [4]) in Section 2 (p. 2, lines 84–86). We build on representation-based UQ (Srey et al., 2026 [1]; p. 2, lines 104–105). Beyond mean aggregation, such as GUARD's mean token entropy (Zhu et al., 2026 [2]; p. 2, lines 94–95), we jointly read means and centered scores across depths, preserving within-branch pairing (p. 4, lines 195–202, Eq. 2). We improve risk prediction and ranking over a learned mean with the same or fewer observation tokens (p. 7, lines 328–331; p. 21, Table 9). ### W2 > The paper claims to predict LLM reasoning failures, but the experiments do not support this claim. Experiments are conducted on datasets rarely used in reasoning-related experiments. Indeed, MMLU-Pro is knowledge-intensive, not reasoning-intensive. The authors should follow the standard practice and use actual reasoning-intensive benchmarks, e.g., MATH and AIME. ### W3 > The models used in experiments are also outdated, especially Qwen2.5. **Response.** **We retain the risk-readout advantage on MATH-500 with two newer open-weight models, Qwen3.8-27B and Gemma 4-E4B-it.** 1. **Mathematical reasoning.** We evaluated all 500 MATH-500 questions using independent continuation labels and question-grouped folds (p. 5, lines 260–264). We used one prefix per question with $H=64$ and $S=\\{16,64\\}$. We kept the $K=10$ probe, readouts, and shrinkage fixed when evaluating $K=2$. With ten branches, we reduce Brier by $0.0059$ and $0.0064$ on the two models. With two branches, we retain reductions of $0.0022$ and $0.0013$ against the ten-branch learned mean (Table 1). 2. **Updated models.** We develop a method for reading failure risk, rather than a new foundation model, and validate its gains across diverse open-weight LLM families. Our original evaluation covers four families, including Gemma 3 (2025; p. 15, Table 4). We have added Qwen3.8-27B and Gemma 4-E4B-it (2026) and retain the gains on both (Table 1). We use open-weight models because our method requires access to hidden states (p. 2, lines 57–59; p. 3, lines 155–159).
AuthorsOpenReview ↗

Rebuttal (2/2)

Table 1. Measured MATH-500 results with task-specific fitting and held-out evaluation.

Model and TrajUQ budgetLearned mean (K=10K=10) Brier ↓\downarrowTrajUQ Brier ↓\downarrowTrajUQ −- learned mean [95% CI]
Qwen3.8-27B (K=10K=10)0.13850.1326−0.0059,[−0.0075,−0.0045]-0.0059\\,[-0.0075,-0.0045]
Qwen3.8-27B (K=2K=2)0.13850.1363−0.0022,[−0.0032,−0.0012]-0.0022\\,[-0.0032,-0.0012]
Gemma 4-E4B-it (K=10K=10)0.17710.1707−0.0064,[−0.0096,−0.0043]-0.0064\\,[-0.0096,-0.0043]
Gemma 4-E4B-it (K=2K=2)0.17710.1758−0.0013,[−0.0028,−0.0006]-0.0013\\,[-0.0028,-0.0006]

W4

In the experiments, the most commonly used PRMs are missing from the baselines.

Response to W4. We retain the joint-readout advantage with both PRM scorers. Holding each scorer fixed, we reduce Brier by 0.00340.0034–0.00560.0056 over the learned mean across both models (Table 2).

  1. PRM baselines. Our original baselines target UQ (submission Table 2, p. 8). We added Qwen2.5-Math-PRM-7B with scalar calibration (Zhang et al., 2025) [5] and our CondOT-based adaptation of Distributional PRM (Ma et al., 2026) [4], using the same Qwen backbone. We fitted calibration on training-prefix outcomes and converted success estimates to failure risks, integrating the predicted quantiles for Distributional PRM.

  2. Matched comparison. We used the MATH-500 settings in Table 1. For each scorer, we shared pre-answer branches (K=10K=10, H=64H=64, S=16,64S=\\{16,64\\}), independent labels, question folds, and tuning budgets between the two readouts. We retained the original probe (Appendix B.2, p. 14, lines 748–755) as the reference scorer.

Table 2. Measured scorer ×\times readout comparison on MATH-500 (Brier ↓\downarrow).

ScorerQwen3.8-27B / MeanQwen3.8-27B / TrajUQGemma 4-E4B-it / MeanGemma 4-E4B-it / TrajUQ
Probe (PCA + logistic)0.13850.13260.17710.1707
Qwen PRM + scalar calibration0.13960.13480.17830.1732
Distributional PRM0.13910.13570.17960.1740

W5

The problem setup (L120) is strange. Why does Y=1 indicate an incorrect answer instead of a correct one?

Response to W5. We assign Y=1Y=1 to an incorrect answer because we estimate failure probability. We define \rho(x)=\Pr_{\pi}(Y=1\mid x) as the complement of success probability (Section 3.1, p. 3, lines 121–124, Eq. 1). Srey et al. (2026, Sec. 2.3) [1] and Chauhan et al. (2026, Sec. 3.1) [6] also assign label 1 to hallucinated responses in supervised UQ. We already cite both in Section 2 (p. 2, lines 102–105).

W6

Some concepts are not properly defined. For example, what's a "branch-score vector"?

Response to W6. We represent each branch by a vector of its scores across observation depths, centered at each depth. We define u_b=(p_{b,t}-\bar p_t)_{t\in S}, where p_{b,t} is branch bb's score at depth tt, and \bar p_t is the mean across branches at that depth (Section 4.1, p. 4, lines 195–198, Eq. 2).

Q1

See weakness section. Most of my questions concern soundness and clarity of the paper, and should be considered as weaknesses.

Response to Q1. We address your soundness and clarity concerns individually in W1–W6 above.

If we have addressed your concerns, we would be grateful if you could raise your score in light of the additional evidence.

References

[1] Ponhvoan Srey et al. Learning Uncertainty from Sequential Internal Dispersion in Large Language Models. ACL, pp. 40088–40106, 2026.

[2] Wei Zhu et al. Dissecting Failure Dynamics in Large Language Model Reasoning. ACL, pp. 8893–8914, 2026.

[3] Jinyuan Li et al. Process Rewards with Learned Reliability. arXiv:2605.15529, 2026.

[4] Rachel Ma, Dylan Hadfield-Menell, and Kristjan Greenewald. Distributional Process Reward Models: Calibrated Prediction of Future Rewards via Conditional Optimal Transport. arXiv:2605.06785, 2026.

[5] Zhenru Zhang et al. The Lessons of Developing Process Reward Models in Mathematical Reasoning. arXiv:2501.07301, 2025.

[6] Mohit Singh Chauhan, Vipin Gyanchandani, and Dylan Bouchard. When Do Supervised UQ Ensembles Improve LLM Hallucination Detection? A Robustness Study. arXiv:2608.24492, 2026.

Dear Reviewer h6AK,

We greatly appreciate your time and effort in reviewing our work. We hope that our responses have clarified our work and addressed your concerns. If so, we would sincerely appreciate your consideration of updating your rating accordingly. If you have any further concerns, we would be happy to provide further clarification.

We highly value every exchange during the review process and sincerely hope to receive further feedback from you at your earliest convenience.

Sincerely,

The Authors

AuthorsOpenReview ↗

Follow-up on our rebuttal

Dear Reviewer,

As the discussion deadline approaches, we would appreciate your feedback on our rebuttal and additional results. We would be happy to clarify any remaining concerns. If we have addressed your concerns, please consider updating your rating accordingly.

Thank you for your time and consideration.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Contributions and Rebuttal Evidence

Dear Program Chairs, Senior Area Chair, Area Chair, and Reviewers,

Thank you for your time and feedback. We summarize below our core contribution and the additional evidence provided during rebuttal.

  1. Our contribution: risk readout beyond the mean. We study budget-constrained LLM uncertainty quantification. Holding the scorer fixed, we jointly read branch means and distributions across observation depths to improve failure-risk prediction. A PRM can supply the scores within this framework; our contribution concerns how we combine this evidence.

  2. Evidence supporting this contribution. Our original evaluation covers 13 model configurations from four families and 1,053,000 sampled continuations. In matched controls at K=10, replacing second-order summaries with distribution embeddings reduces Brier by 0.0062 under either tested outer kernel, compared with 0.0005 from changing the outer kernel (xW5j, Table 1). We also retain the readout advantage with PRM scorers (h6AK, Table 2). Matched comparisons with MLP and GBDT predictors and an expanded bandwidth search further support the readout advantage (QRTW, Table 4; xW5j, Table 4).

  3. Evidence addressing practical applicability. We added MATH-500 evaluations on two newer models and transfer experiments without target-task fitting. On MATH-500, TrajUQ-guided restarting improves accuracy by 1.4 percentage points and reduces total generated tokens by 5.4% over single generation, counting auxiliary branches and restarts (QRTW, Tables 1–3).

These results support our central finding: we can improve uncertainty estimation by better using limited observations. We would appreciate your consideration of this evidence alongside the original reviews and our detailed responses.

Thank you for your time and consideration.

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.