On the Interplay between Backtracking and Latent Confidence Directions in Reasoning Models

Reasoning models deliberate over multiple answer proposals, often resulting in excessively long traces due to backtracking, even when the correct answer has already been reached (On AIME 2026 with Olmo 3 32B Think, $\sim$87% of rollouts contain 1 unique answer yet still make $\sim$8 answer proposals on average). Given this potentially "decorative" chain of thought (CoT) reasoning, can we look to the model internals to determine if the model will backtrack or commit to an answer proposal? Operating under the hypothesis that models maintain an internal representation of their uncertainty, we seek to extract this representation and characterize its relationship with backtracking behavior at answer proposals. By querying the model's verbalized confidence following answer proposals throughout a response trace, we extract a linear direction for latent confidence that separates high and low verbalized confidence. This direction can steer verbalized confidence as expected, but it surprisingly can also steer the next token probability of backtracking, performing on par with a linear direction extracted directly from backtracking examples. % of backtracking vs. committing to answer. We then test the direction under progressively more realistic interventions. Steering continuously from an answer proposal modulates response length ($\sim$-1k - $\sim$+3k tokens) and the number of answer proposals (-4.52 - +3.26) symmetrically in both directions, with minimal cost to accuracy. Applying the confidence direction from the start of generation, where proposal positions are unknown, still shortens traces at comparable accuracy, and its effect surfaces in the linguistic confidence of proposals. Overall, our study reveals the intricate interplay between a model's internal confidence and its proclivity to backtrack or commit to an answer.