TrajUQ: Predicting LLM Reasoning Failure under Limited Observation Budgets

Short continuations offer evidence of failure risk before a large language model (LLM) answers, but observing more branches or following them longer consumes tokens. Can better readout of finite, imperfect probe scores compensate for fewer observations? We address this question with TrajUQ, combining the endpoint mean and the empirical distribution of centered branch-score vectors to predict continuation failure. We evaluate 540 questions across three tasks and thirteen model configurations using question-grouped out-of-fold prediction. With ten-branch, 64-token fits frozen, our TrajUQ readout improves Brier over the learned mean by 3.125% on average across 20 matched combinations of branch count and observation depth, each using at least two branches. In our post-hoc scan, two-branch TrajUQ achieves lower Brier than the ten-branch, 64-token mean readout using 80% fewer observation tokens. At the reference budget, our matched tuning yields Brier 0.1947 versus 0.2011 for the learned mean, also improving on the tested second-order and distribution-only controls. Using these predictions, our offline screening selects prefixes with lower empirical failure at matched coverage. Our offline TrajUQ candidate selection reduces final-answer failure from 56.47% for a single answer chosen uniformly from the same candidate pool to 51.92% without updating LLM parameters. These results connect better use of limited observations to improved risk prediction and more reliable offline answer selection.