acceptodds
Under review as a conference paper at ICLR 2027

Think Without Thinking: When Empty Reasoning Reveals Answerability

Abstract

Reliable uncertainty before response generation would let a language model decide whether to answer, abstain, retrieve evidence, or spend more computation without first producing a candidate. Existing query-level observers usually read the final query token, implicitly treating observation position as fixed. We show that reasoning-model templates contain a more informative alternative: the first hidden state after the model's native empty-reasoning boundary. We call the resulting observer THInKUQ. It visits this boundary with a fixed teacher-forced suffix, decodes no tokens, and fits a lightweight correctness readout. On Qwen3-4B, THINKUQ improves subject-OOD MMLU AUROC by 9.95 points and transfers without target tuning to MMLU-Pro. Across four Qwen3 scales and tasks spanning factual recall, arithmetic, and code, the gain appears when task-relevant correctness is internally available before generation. Matched ordinary suffixes, official template variants, learned prefixes, activation patching, attention analysis, and a controlled Tagged- CoT training trajectory localize the effect to a learned post-boundary transition. In the positive regime, this transition widens correct-incorrect separation, shortens prequential codelength, and remains useful with few source labels and modest prefill overhead. These results establish observation position as a design variable for pre-answer uncertainty and provide a source-side diagnostic for recognizing when a native reasoning boundary exposes a useful correctness readout.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Uncertainty Quantification, Pre-answer Uncertainty, Selective Prediction, Zero-decoding Observer, Reasoning Models, Hidden States, Large Language Models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_RmUmOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether changing where hidden states are read before generation can improve predictions of whether the model's final answer will be correct. ThinkUQ appends a fixed empty reasoning suffix, reads the hidden state after the boundary, and trains a linear probe to predict correctness. The authors evaluate the method across four Qwen3 scales, multiple tasks, and transfer settings. They also propose a diagnostic that uses source data to assess when the method is applicable.

Critical strengths

The method is simple, and the main experiments provide strong support. Scoring does not require generating reasoning or an answer. In Table 1, compared with Query-end, ThinkUQ improves AUROC by 9.95 and 7.25 percentage points on MMLU subject-OOD testing and MMLU-Pro transfer, respectively. Both methods use matched linear probes, select parameters using only source data, and keep them fixed during transfer. These findings show that changing the readout position before generation can clearly improve correctness prediction. This is a useful empirical contribution.

Critical weaknesses

W1: The additional value of the native reasoning boundary still needs clarification. In §5.3, the main text links the performance gains to computations triggered by the native boundary. The introduction also lists this explanation as a main contribution. Appendix A.2 reports that the full boundary improves AUROC by 0.0124 over the no-tag layout on MMLU-Pro, with a 95% CI of [0.0107, 0.0140]. This supports the transfer value of the tag structure. However, in the B.3 control with 400 source and 200 target examples, user-turn-end and the fixed Answer: prefix also yield point estimates close to those of ThinkUQ when each condition uses its own source-selected probe. These small-sample results do not establish equivalence between the methods, but the additional advantage of the native boundary over other natural positions remains unclear. The main text should therefore distinguish the overall benefit of changing the readout position from the specific contribution of the native boundary.

W2: The proposed source-side diagnostic lacks systematic evaluation. In §6.3, the main text proposes using ThinkUQ when both ranking performance and the separation between correct and incorrect samples improve on source data. The conclusion also presents this diagnostic as a basis for selecting the readout position in practice. However, the paper does not systematically report the choices made using this diagnostic and their corresponding target-domain performance. The prequential codelength results for three configurations in Appendix B.7 provide indirect support, but do not directly evaluate the proposed selection criteria. In particular, for 8B PopQA and 14B GSM8K→MATH, which show negative gains in Table 9, it remains unclear whether the diagnostic would still recommend using ThinkUQ. This limits support for the main text's claim that the source-side diagnostic can guide applicability judgments.

Questions for authors

[Clarification] Can the authors explain, based on A.2 and B.3, what additional advantage the current evidence supports for the native boundary over other natural readout positions? Under which experimental configurations does this conclusion hold?

[Clarification] In the current analysis, how do choices made using only source data and the diagnostic described in §6.3/F.2 correspond to the target-domain results for each configuration in Table 9? In particular, would the diagnostic still recommend ThinkUQ for configurations with negative gains? If no systematic evaluation is available, we suggest framing it as a heuristic supported by preliminary evidence.

3 comments

We sincerely thank you for the thoughtful review and for recognizing the value of pre-generation correctness prediction and our transfer results. Your comments helped us clarify the contribution of the reasoning tags and the choice of readout position.

1. Response to Weakness 1 and Question 1: The Contribution of the Reasoning Boundary

The empty reasoning block improves pre-generation correctness prediction, and its tags add value in the controlled transfer setting. We explain how these findings fit the close Answer: results and examine how the CoT training format relates to readout performance.

1. Changing the readout improves correctness prediction.

In the primary Qwen3-4B MMLU→MMLU-Pro experiment, ThinkUQ improves AUROC from 0.6524 to 0.7249, a gain of 0.0725. Both methods use the same backbone, answer policy, correctness supervision, and probe-fitting procedure.

2. Retaining the reasoning tags provides an additional gain.

The separate Qwen3-4B MMLU→MMLU-Pro experiment in A.2 examines what the tags themselves contribute. It keeps the questions, newline layout, final newline readout, and probe-fitting procedure fixed, and compares:

Input structureTarget AUROC
Empty <think>…</think> block0.7254
Both tags removed; newline layout retained0.7130

Keeping the tags improves AUROC by 0.0124, with 95% CI [0.0107, 0.0140].

3. The Answer: comparison reads after the same reasoning tags.

B.3 evaluates Qwen3-4B on 400 MMLU source and 200 MMLU-Pro target examples. Each position receives its own source-selected probe:

Readout positionTarget AUROC
Query-end0.5754
User-turn-end0.6685
After the empty reasoning block0.6683
After the additional Answer:0.6803

After the question and assistant header, the fixed suffixes are:

ThinkUQ: <think>\n\n</think>\n\n
Answer: <think>\n\n</think>\n\nAnswer:

ThinkUQ reads the final newline token following </think>; the Answer: condition reads the final token of the added prefix. The model has processed the complete empty reasoning block before either readout. All suffix tokens are fixed prefill, and scoring precedes reasoning or answer generation.

The Answer: minus ThinkUQ difference is +0.0120, with paired hierarchical example-bootstrap 95% CI [−0.0406, +0.0621]. User-turn-end also gives a close point estimate in this small panel.

We added the same comparison between the two post-block positions on the official DeepSeek-R1-Distill-Llama-8B checkpoint, with its backbone frozen:

TaskAfter the empty blockAfter the additional Answer:
MMLU→MMLU-Pro0.75170.7493
TriviaQA0.79200.7976

Both paired CIs for the differences include zero. These evaluations consistently show small observed changes when Answer: is appended after the empty block.

4. A training control preserves CoT and removes the reasoning tags.

We further examined how training with reasoning tags relates to the usefulness of the readout. Both training runs start from Qwen3-4B-Instruct-2507 and use rank-16 LoRA and the same 9,984 CoT examples. The new control uses 8,202 steps, matching the step budget of the original 600-minute Tagged-CoT run. Their training sequences are schematically:

Tagged-CoT:
question + <think> reasoning text </think> + Answer: + answer

New control:
question + reasoning text + Answer: + answer

The control retains the reasoning content and Answer: as the delimiter before the final answer, and removes both reasoning tags.

At MMLU→MMLU-Pro evaluation, both models are scored before generating reasoning or an answer. The tagged model receives the question and assistant header followed by the empty reasoning block; its probe reads the final newline. The control receives the question and assistant header followed directly by Answer:; its probe reads the final token of that prefix. Each is compared with its own Query-end baseline:

Training formatEvaluation suffixAUROC gain over Query-end [95% CI]
Tagged-CoT (§7, Appendix D.1)Empty <think>…</think> block+0.1160 [0.0557, 0.1774]
Reasoning tags removedAnswer: alone+0.0070 [−0.0213, +0.0249]

The new control achieves 0.5719 at Query-end and 0.5789 after Answer:.

The experiments support using the empty reasoning block to improve pre-generation correctness prediction: retaining its tags improves transfer, and the model trained with tagged CoT shows a larger gain at this readout. The close Answer: results are consistent with this benefit, since that prefix follows the same empty block. We choose the final newline after </think> because it follows the chat template and requires a shorter suffix than adding Answer:. We will revise the introduction and §5.3 to distinguish the overall readout gain from the contribution of the reasoning tags, and add the supplementary results.

2. Response to Weakness 2 and Question 2: Systematic Evaluation of the Source-Side Diagnostic

Thank you for highlighting the need to evaluate the diagnostic through its actual choices and target-domain outcomes. We added a systematic evaluation of the same 20 model–task configurations reported in Table 9, covering four Qwen3 scales (1.7B, 4B, 8B, and 14B) and five tasks (TriviaQA, PopQA, GSM8K, MATH, and HumanEval), with ten source splits per configuration.

The source-side rule selects Query-end in most splits for both negative-transfer cases highlighted in the review. It substantially reduces their target-domain losses while retaining positive average gains across all 20 configurations.

1. Turning the source-side principle into an explicit selection rule.

For this supplementary evaluation, we operationalize the principle in §6.3 using source AUROC and standardized correct–incorrect score separation. Probes are fitted on source training data, layers and regularization are selected on source validation data, and the diagnostic is computed on an independent source holdout.

For the probe’s pre-sigmoid score zz, we define standardized separation as

{\sqrt{(s_{\mathrm{correct}}^2+s_{\mathrm{wrong}}^2)/2}},$$ where $\mu$ and $s^2$ are the score means and sample variances within each correctness class. Larger $D$ indicates a wider correct–incorrect score gap relative to within-class variation. Let $\Delta A$ and $\Delta D$ denote ThinkUQ minus Query-end in source AUROC and separation. **We select ThinkUQ when the paired bootstrap 95% CI lower bounds for both differences exceed zero; otherwise, we select Query-end.** This joint rule requires both improvements. Source bootstrap resampling is stratified by correctness and uses the same sampled examples for both readouts. The selected readout is frozen before target evaluation. Within each split, all strategies reuse the same Query-end and ThinkUQ probes, their target predictions, and the shared correctness labels. Their target results therefore reflect the source-side selection decisions. **2. Target performance across the 20 configurations.** The table reports equally weighted averages across configurations. Utility assigns +1 to a correct answer, −1 to an incorrect answer, and 0 to abstention, and is reported per 100 test questions using source-selected answering thresholds. The AUROC-only selector chooses ThinkUQ when source $\Delta A>0$. Worst changes refer to the most adverse configuration relative to its Query-end baseline. | Selection strategy | Mean target AUROC | Mean utility/100 | Worst ΔAUROC | Worst Δutility/100 | |---|---:|---:|---:|---:| | Always Query-end | 0.6639 | 21.42 | 0 | 0 | | Always ThinkUQ | 0.6784 | 25.50 | −0.1059 | −5.02 | | Source AUROC only | 0.6835 | 22.88 | −0.0589 | −3.22 | | Joint diagnostic, 95% CI | 0.6779 | 23.55 | −0.0117 | −0.28 | The joint diagnostic improves mean target AUROC and utility relative to always using Query-end. Relative to always using ThinkUQ, it reduces the worst observed AUROC loss from 0.1059 to 0.0117, and the worst utility loss from 5.02 to 0.28 points per 100 questions. Its mean utility of 23.55 lies between the two fixed strategies, reflecting the tradeoff between average benefit and controlling adverse transfer. **3. The choices and outcomes for the two highlighted cases.** The table gives selection counts across ten source splits and the resulting target AUROC. The final column reports the change relative to Query-end and its 95% CI. | Configuration | Query-end / ThinkUQ selections | Always Query-end | Always ThinkUQ | Joint diagnostic | Joint − Query-end [95% CI] | |---|---:|---:|---:|---:|---| | 8B PopQA | 8 / 2 | 0.7851 | 0.6793 | 0.7798 | −0.0053 [−0.0178, 0.0000] | | 14B GSM8K→MATH | 6 / 4 | 0.6436 | 0.5822 | 0.6322 | −0.0114 [−0.0275, −0.0007] | For 8B PopQA, the rule selects Query-end in eight of ten splits and reduces the AUROC loss from 0.1059 to 0.0053. For 14B GSM8K→MATH, it selects Query-end in six splits and reduces the loss from 0.0614 to 0.0114; the remaining AUROC change has a 95% CI of [−0.0275, −0.0007]. **These results directly show how source-side choices translate into target-domain outcomes. Across the evaluated configurations, the joint diagnostic substantially reduces adverse transfer while preserving positive average gains in correctness prediction and decision utility.** This supports its use when controlling degradation is a priority. We will specify the rule in §6.3 and Appendix F.2 and add a compact table in Appendix F.2 reporting, for every configuration in Table 9, the number of ThinkUQ selections, the resulting target AUROC, and its change relative to Query-end.

Thank you for your clarification. I will maintain my original score.

Reviewer_axHQOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes THINK UQ, a method for predicting whether a language model will answer correctly before generating an answer. Instead of probing the final query-token representation, it appends a fixed, empty reasoning block and trains a lightweight logistic probe on the post-boundary hidden state, while keeping the backbone and answer-generation policy unchanged. It argues that where a representation is read can substantially affect how easily correctness can be predicted.

Critical strengths
  • This paper targets at an important yet underexplored problem.
  • Strong empirical results, especially in OOD setting.
  • Extensive controlled results support the importance of the observation boundary.
Critical weaknesses
  • The presentation should be improved, e.g., UQ is used in the paper without definition.
  • All the experiments were conducted on the Qwen model series. Therefore, it is unclear whether the findings will apply to other model families.
  • The training trajectory uses one Tagged-CoT continuation, without a matched untagged-CoT or ordinary fine-tuning control to isolate the effect of reasoning tags.
  • The experiments mainly compare ranking accuracy, but higher ranking accuracy is not equal to better decision making in downstream tasks.
Questions for authors

Please refer to my comments in the weaknes part.

2 comments

We sincerely thank the reviewer for the careful reading and constructive feedback. We appreciate your recognition of the importance of pre-answer uncertainty estimation, the strong OOD results, and the extensive controlled experiments. Your comments have helped us identify ways to strengthen the evidence and improve the presentation. We address each concern below.

1. Response to Weakness 1: Definition of UQ

Thank you for pointing out this omission. We will explicitly introduce uncertainty quantification (UQ) at its first occurrence and use the abbreviation consistently throughout the revised manuscript.

2. Response to Weakness 2: Generalization Across Model Families

Thank you for highlighting the importance of evaluation across model families. We added experiments using the official DeepSeek-R1-Distill-Llama-8B checkpoint with its backbone frozen. This extends our evaluation to a Llama-based reasoning model, complementing the Qwen models studied in the original submission.

We compared Query-end and ThinkUQ on MMLU→MMLU-Pro transfer and TriviaQA:

EvaluationQuery-end AUROCThinkUQ AUROCΔAUROCExample bootstrap 95% CIGroup bootstrap 95% CI
MMLU→MMLU-Pro0.66670.7517+0.0850[+0.0423, +0.1270][+0.0018, +0.1809]
TriviaQA0.74440.7920+0.0476[+0.0273, +0.0677][+0.0329, +0.0626]

ThinkUQ improves AUROC by 8.50 points on MMLU→MMLU-Pro and 4.76 points on TriviaQA. For both evaluations, the gains remain positive under both example-level and group-level bootstrap, with all four 95% confidence intervals above zero.

These results provide direct cross-family evidence that the fixed empty reasoning block improves correctness prediction over Query-end while preserving a frozen backbone and zero decoding for UQ. We will incorporate these results and the experimental protocol into the revised manuscript.

3. Response to Weakness 3: Untagged-CoT Training Control

Thank you for suggesting a matched training control. We added an Untagged-CoT experiment that retains the reasoning content and final-answer supervision while removing both <think> and </think> tags.

Both training runs start from Qwen3-4B-Instruct-2507 and use rank-16 LoRA on the same 9,984 CoT examples. The new control uses 8,202 training steps, matching the step budget of the original 600-minute Tagged-CoT run. Both formats retain the Answer: delimiter before the final answer:

Tagged-CoT:
question + <think> + reasoning text + </think> + Answer: + answer

Untagged-CoT:
question + reasoning text + Answer: + answer

For the Untagged-CoT model, we evaluate the fixed input question + assistant header + Answer: and read the hidden state of the final token in Answer: before generating reasoning or an answer. Query-end and this readout achieve AUROCs of 0.5719 and 0.5789, respectively, giving a gain of 0.0070, with a paired bootstrap 95% CI of [−0.0213, +0.0249].

The original Tagged-CoT experiment (§7, Appendix D.1) evaluates the final newline following the empty <think>\n\n</think>\n\n block. Its reported gain over Query-end is 0.1160, with a 95% CI of [+0.0557, +0.1774].

Training formatFixed prefill readoutGain over Query-end95% CI
Tagged-CoTFinal newline after the empty reasoning block+0.1160[+0.0557, +0.1774]
Untagged-CoTFinal token of Answer: appended directly after the question and assistant header+0.0070[−0.0213, +0.0249]

Each gain is measured relative to Query-end within the corresponding trained model. These endpoint results compare the two training formats through their respective fixed readouts. The tagged setting yields a larger observed readout gain, while the untagged control directly examines the usefulness of an answer delimiter learned through supervision on the same reasoning content.

This addition supplies the requested Untagged-CoT training control, with reasoning content and training steps matched, and complements the original training trajectory with an explicit comparison of training formats and their corresponding zero-decoding readouts. We will include the control, its training protocol, and the readout definitions in the revised manuscript.

4. Response to Weakness 4: Downstream Answer-or-Abstain Decisions

Thank you for emphasizing the importance of downstream decision quality. We added a selective-answering evaluation that directly uses predicted correctness to decide whether to answer or abstain.

The evaluation covers 20 configurations: Qwen3-1.7B, 4B, 8B, and 14B on TriviaQA, PopQA, GSM8K, MATH, and HumanEval. Within each configuration, both methods use the same backbone, answer policy, and per-question answers. A method answers when its predicted correctness reaches the threshold and abstains otherwise.

Correct answers receive +1, incorrect answers −1, and abstentions 0. Net utility per 100 test questions is:

100 × (number answered correctly − number answered incorrectly) / total test questions

This measures the benefit of correct answers and the cost of errors, with abstentions contributing zero.

1. Higher net utility under both decision rules.

We evaluate a fixed threshold of 0.5 and a threshold selected separately for each method to maximize source-validation utility, then frozen for target testing. Results are averaged equally over all 20 configurations, including configurations with zero answered questions.

Decision ruleQuery-end utilityThinkUQ utilityGainRelative gainPaired 95% CI for gain
Fixed threshold .522.0625.23+3.17+14.35%[+2.17, +4.19]
Source-selected threshold21.4225.50+4.07+19.02%[+3.09, +5.05]

Utility and absolute gains are scores per 100 questions. Relative gains divide the utility increase by Query-end utility; all gains are computed from unrounded values. Both rules yield positive gains with 95% confidence intervals above zero. With source-selected thresholds frozen for target testing, ThinkUQ increases mean net utility by 19.02% relative to Query-end.

2. Higher utility across the tested thresholds.

A sensitivity analysis using the same predictions shows positive mean utility gains at all five thresholds from 0.3 to 0.7:

ThresholdQuery-end utilityThinkUQ utilityGainRelative gainPaired 95% CI for gain
.321.7423.80+2.06+9.49%[+1.29, +2.85]
.422.4425.04+2.60+11.57%[+1.70, +3.50]
.522.0625.23+3.17+14.35%[+2.17, +4.19]
.621.2124.17+2.95+13.92%[+1.91, +4.01]
.718.5622.24+3.68+19.81%[+2.56, +4.79]

All five 95% confidence intervals are above zero. For both utility tables, intervals use 5,000 dataset-stratified paired question-bootstrap draws. Each question is resampled jointly across both methods and all four model scales, while probes and thresholds remain frozen. We report percentile intervals for the paired differences.

3. Higher mean answered-question accuracy and coverage.

Coverage is the fraction of questions answered; answered-question accuracy is the fraction of those answers that are correct. To reduce instability at very low coverage, at each threshold we retain configurations where both methods have at least 5% coverage and average each metric equally over the same retained configurations. These results are point estimates; the retained count varies with the threshold.

ThresholdRetained configurationsQuery-end accuracyThinkUQ accuracyQuery-end coverageThinkUQ coverage
.320/2062.47%64.58%58.75%61.58%
.420/2066.77%68.16%51.53%56.73%
.519/2070.21%71.93%47.41%54.53%
.618/2073.15%73.60%42.75%51.74%
.715/2076.28%78.84%41.82%52.89%

At every threshold, ThinkUQ achieves higher mean answered-question accuracy and coverage on the shared retained configurations. The utility analyses above include all 20 configurations.

These results connect correctness prediction to measurable decision benefits: ThinkUQ improves the net utility of answering versus abstaining, with consistent mean gains across decision thresholds and a 19.02% relative improvement under source-selected thresholds transferred to target testing. We will incorporate all three analyses and their protocols into the revised manuscript.

Reviewer_4LErOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies whether changing the token position used for a correctness probe can improve its performance. Rather than probing the final input token, the authors insert an empty reasoning block and probe the hidden state immediately after it, without generating any reasoning tokens. The probe is a linear classifier trained to predict whether the model's eventual answer will be correct.

The authors show improvements on MMLU and MMLU-Pro, with small improvements on factual recall and mathematical reasoning tasks, and no significant improvements on coding tasks. They run several experiments comparing the empty reasoning block to other fixed token sequences, and analyze how the hidden states and attention patterns change at this position. They also study how the improvement develops during chain-of-thought fine-tuning, showing that the post-boundary probe's advantage over the final-input-token probe increases over training. Finally, they evaluate the inference cost and the number of labels required to train the probe.

Critical strengths

The paper includes a fairly thorough set of ablations comparing the empty reasoning block against other fixed token sequences and observation positions. The experiment studying how the relative probe performance changes over chain-of-thought fine-tuning is also interesting, although it is limited to a single training run.

Critical weaknesses

The paper's main finding that the token position used for a probe affects its performance is not novel. Prior work, including Ashok & May (2025), has already studied how probing performance varies across input token positions. Afzal et al. (2025), Zhang et al. (2025), and Boppana et al. (2026) have also studied predicting correctness from activations at different points in a reasoning trace, including before reasoning begins. The paper shows that appending an empty reasoning block improves correctness prediction compared to the final input token, but it is unclear why this particular observation is a substantial contribution beyond prior work.

If correctness information is already present elsewhere in the prompt, then appending an empty reasoning block just makes that information more accessible to a linear probe at the final token. In addition, the mechanistic analysis showing changes in attention patterns and linear separability does not explain how correctness information is being represented. The training experiment similarly only shows that the relative performance of these two probe positions changes during CoT fine-tuning, rather than studying how correctness information itself develops during training.

Ashok & May (2025). Language Models Can Predict Their Own Behavior. arXiv:2502.13329 Afzal et al. (2025). Knowing Before Saying. ACL 2025 Zhang et al. (2025). Reasoning Models Know When They're Right. arXiv:2504.05419 Boppana et al. (2026). Reasoning Theater. ICML 2026

Questions for authors

In Appendix B.3, the authors show that a probe at a fixed "Answer:" prefix token achieves similar performance to the empty reasoning boundary token "<think></think>". Is there evidence that supports the claim that the improvement comes specifically from the model's learned reasoning boundary, rather than a more general transition into answer generation?

5 comments

We thank the reviewer for the careful reading and thoughtful feedback. We appreciate the recognition of our controlled ablations and the interest in the training-trajectory experiment. Below, we first address the question about the reasoning tags and the Answer: readout, and then respond to the concerns about novelty and mechanism.

1. Response to the Question: Reasoning Tags and the Answer: Readout

In Appendix B.3, In Appendix B.3, the Answer: condition is constructed by appending Answer: after the complete empty reasoning block. The two input sequences are:

ThinkUQ:
question + assistant header + <think>\n\n</think>\n\n

Readout after Answer:
question + assistant header + <think>\n\n</think>\n\nAnswer:

ThinkUQ reads the hidden state of the final newline token immediately following </think>. The second condition reads the hidden state of the final token in Answer:. Both conditions process the same empty reasoning block; appending Answer: moves the readout a few tokens later. All appended tokens are supplied through fixed prefill, and the probe predicts eventual answer correctness before reasoning or answer generation begins.

For the Qwen3-4B model evaluated in Appendix B.3, independently source-selected probes at these positions achieve AUROCs of 0.6683 and 0.6803. The difference is +0.0120, with a paired hierarchical example-bootstrap 95% CI of [−0.0406, +0.0621]. The point estimates are close, and the confidence interval includes zero.

To examine this readout choice in another model, we added an experiment using the official DeepSeek-R1-Distill-Llama-8B checkpoint with its backbone frozen:

Model and taskAfter the empty reasoning blockAfter the additional Answer:
DeepSeek-R1-Distill-Llama-8B, MMLU-Pro0.75170.7493
DeepSeek-R1-Distill-Llama-8B, TriviaQA0.79200.7976

On DeepSeek, appending Answer: changes AUROC by −0.0024 and +0.0056 across the two tasks, with both paired confidence intervals including zero. Across these evaluations, the two readout positions yield close AUROC point estimates, and the small changes from appending Answer: vary in direction.

To further investigate the role of reasoning tags behind the B.3 observations, we added a training experiment starting from Qwen3-4B-Instruct-2507. We trained rank-16 LoRA on the same 9,984 CoT examples for 8,202 steps, matching the step budget of our original 600-minute Tagged-CoT training run. We retained the reasoning text, the Answer: delimiter, and the final answer, and removed both <think> and </think> tags. Each training sequence contains:

question + reasoning text + Answer: + answer

This gives Answer: explicit training exposure as the delimiter between reasoning and the final answer. During evaluation, the observer receives:

question + assistant header + Answer:

The probe reads the hidden state of the final token in Answer: before the model generates reasoning or an answer. Query-end and Answer-only achieve AUROCs of 0.5719 and 0.5789, respectively, giving a gain of 0.0070, with a paired hierarchical example-bootstrap 95% CI of [−0.0213, +0.0249].

In the original Tagged-CoT experiment (§7, Appendix D.1), the training sequences enclose the reasoning text within <think> and </think>. Reading immediately after the corresponding empty block yields a gain of 0.1160 over Query-end, with a 95% CI of [0.0557, 0.1774]. This pattern supports the hypothesis that training with explicit reasoning delimiters helps make eventual correctness predictable from the state immediately following </think>.

Given the similar observed prediction performance at the two positions in B.3 and the DeepSeek experiments, we choose the hidden state of the newline immediately following </think>. Its location is specified by the model’s chat template, and it requires a shorter fixed suffix than reading after the additional Answer:. We will make the complete sequences explicit in B.3 and incorporate these supplementary results.

2. Response to Weakness 1: ThinkUQ’s Contribution Beyond Prior Probing Studies

We thank the reviewer for highlighting these studies. ThinkUQ contributes a concrete observation-input design: a fixed empty reasoning structure that improves pre-generation correctness prediction while retaining zero autoregressive decoding. Its value is supported by controlled comparisons, transfer across datasets, and measured inference costs.

1. A dedicated observation input for correctness prediction before generation.

The cited studies establish valuable foundations. Ashok & May (2025) study pre-generation behavior prediction and probe performance across input-token positions. Afzal et al. (2025) predict CoT success before and during generation. Zhang et al. (2025) probe intermediate-answer correctness and use it to guide early exit. Boppana et al. (2026) investigate when the model’s eventual answer choice becomes accessible through activation probes and forced answering.

Building on these findings, ThinkUQ asks: Can a fixed, structured observation input make a frozen model’s correctness information more accessible to a lightweight probe before generation begins?

ThinkUQ processes the question in a separate observation pass, appends an empty block delimited by <think> and </think>, and reads the hidden state at the following newline position. The appended tokens are identical across questions and supplied entirely through prefill. The backbone, original answer policy, and correctness labels remain fixed across observer conditions. This isolates the effect of the observation input on predicting the correctness of the same answers.

The resulting method combines the efficiency of pre-generation probing with a readout constructed using the model’s existing reasoning format. For settings that probe generated CoT, ThinkUQ provides a correctness estimate before that reasoning is generated. For pre-generation probing, its contribution is the improvement obtained through the empty reasoning structure at zero decoding.

2. Controlled experiments quantify the benefit of this design.

Table 1 compares Query-end, a length-matched ordinary suffix, and ThinkUQ using the same backbone, answer policy, correctness supervision, and matched probe-training procedures:

Qwen3-4B evaluationQuery-end AUROCOrdinary suffix AUROCThinkUQ AUROCGain over Query-endGain over ordinary suffix
MMLU subject-OOD0.69000.72640.7895+0.0995+0.0631
MMLU → MMLU-Pro0.65240.65880.7249+0.0725+0.0661

The source-trained probes transfer to MMLU-Pro with all selection confined to the source data. On this transfer task, ThinkUQ’s gain over the length-matched ordinary suffix is +0.0661 AUROC, with paired hierarchical 95% confidence intervals of [0.0371, 0.0963] under example resampling and [0.0314, 0.1010] under subject resampling (Appendix A.1).

These comparisons establish a measurable benefit from choosing the empty reasoning structure beyond the improvement obtained by adding an ordinary suffix. The paper further examines readout positions, randomly sampled ordinary suffixes, official template modes, and a source-trained soft prefix (Appendix B). Together, these experiments turn the observation that probe performance varies by position into a concrete, reproducible design for pre-generation correctness estimation.

3. The improvement requires only a short additional prefill.

All tokens in the empty block are known in advance and processed with the question during prefill. The Qwen3-14B TriviaQA measurements illustrate this quality–cost trade-off (§6.2, Figure 5):

MeasurementQuery-endThinkUQ
Correctness AUROC0.76640.7993
Mean prefill latency107.18 ms115.00 ms

AUROC is evaluated on the 600-example target set, and latency is measured using 64 paired queries on the same hardware. Here, the empty block adds four fixed input tokens. The reported AUROC gain is 0.0330, accompanied by 7.82 ms of additional prefill latency. This provides a concrete improvement in correctness prediction while preserving the latency advantage of obtaining the estimate before autoregressive generation.

Recent systems such as Jev (TypeSafe AI, 2026) underscore the practical importance of obtaining probabilistic decisions directly, with zero autoregressive decoding. ThinkUQ advances this direction through a concrete improvement in correctness estimation on existing frozen reasoning models. Its measured quality–cost gains show why the observation input is a consequential design choice: a few fixed tokens can substantially improve correctness discrimination while preserving zero decoding. This combination offers a practical building block for low-latency reliability assessment, selective answering, and query routing, where an informative correctness estimate is valuable before committing computation to generating a response.

3. Response to Weakness 2: Correctness Representations and Their Development During Training

Thank you for encouraging a closer examination of how correctness information is organized and develops during training. Making information from the prompt accessible to a lightweight probe is central to ThinkUQ. Our account is that processing the fixed empty reasoning block organizes this information into the hidden state immediately following </think>, supporting correctness prediction before decoding. We added three analyses to examine the training dynamics, the transferability of the readout, and the contribution of specific attention heads.

1. Absolute performance improves at the empty reasoning boundary during training.

To complement the training trajectory in §7 and Appendix D.1, we report absolute AUROC at three representative checkpoints: 0, 300, and 600 minutes. At each checkpoint, probes are fitted on MMLU and evaluated on MMLU-Pro, with both readouts fixed at layer 30. We will include the complete results for all 21 checkpoints in the revised manuscript.

Training timeQuery-end AUROCThinkUQ AUROC
0 min0.63500.6431
300 min0.60410.6966
600 min0.58670.7028

ThinkUQ’s absolute AUROC increases by 0.0597, while Query-end decreases by 0.0483. Both changes contribute to the growing relative advantage. The full trajectory shows early fluctuations followed by generally higher ThinkUQ performance during the middle and later stages. These results document an absolute improvement in correctness prediction from the boundary state along this training run.

2. The correctness readout transfers between middle and late training stages.

We fitted probes at 0, 300, and 600 minutes and evaluated each frozen probe on all three model checkpoints, using the same MMLU→MMLU-Pro setting and fixed layer 30. Rows identify the probe-fitting checkpoint; columns identify the model checkpoint used for evaluation.

ReadoutProbe fitted atEvaluate at 0 minEvaluate at 300 minEvaluate at 600 min
Query-end0 min0.63500.57450.5799
Query-end300 min0.59390.60410.5810
Query-end600 min0.58930.58390.5867
ThinkUQ0 min0.64310.62250.6161
ThinkUQ300 min0.61790.69660.7037
ThinkUQ600 min0.62670.69140.7028

The 300-minute ThinkUQ probe achieves 0.7037 on the 600-minute model, close to 0.7028 for the probe fitted at 600 minutes. In the reverse direction, the 600-minute probe achieves 0.6914 on the 300-minute model, compared with 0.6966 for the probe fitted at 300 minutes. On the same 600-minute model, the initial probe achieves 0.6161. Together, these comparisons support a correctness readout that develops during training and remains effective across the middle and late checkpoints.

3. Targeted attention-head interventions affect correctness prediction.

Using out-of-fold probe predictions on 200 MMLU source examples, we selected eight of 160 heads with the largest single-head AUROC drops and froze this set before source validation and MMLU-Pro evaluation. At the final newline token, we replaced their outputs before the attention output projection with those from the same question using an equal-length period suffix; model weights, probes, and standardizers remained frozen. Random controls matched the number of heads per layer; magnitude-matched controls additionally matched the L2 norm of the combined perturbation after output projection for each example and layer.

ConditionAUROCChange from unmodified ThinkUQ
Unmodified ThinkUQ0.7249—
Identity replacement0.72490.0000
Intervention on eight candidate heads0.7047−0.0202
Random heads: mean over 10 groups0.7237−0.0012
Magnitude-matched random heads: mean over 10 groups0.7208−0.0041
Eight heads from the original attention analysis0.7246−0.0003

The candidate-head intervention produces an AUROC reduction of 0.0202, exceeding the mean reductions for random heads and magnitude-matched random heads by 0.0190 and 0.0161, respectively. This pattern supports a functional contribution of the candidate heads to the correctness readout. The small change for the original attention-selected heads further sharpens the distinction between observed routing changes and the effects of intervening on particular heads.

Together, these analyses connect an absolute improvement in the boundary readout, its transferability across training stages, and its sensitivity to specific attention-head interventions. They provide a more concrete account of both the development of the correctness representation during training and the computations supporting its readout. We will incorporate these results and the complete absolute AUROC trajectories into the revised manuscript.

Comments on the paper

1 comment

Dear Area Chairs, Senior Area Chairs, and Program Chairs,

We sincerely thank the reviewers for their careful reading and constructive feedback. We appreciate their recognition of the problem’s importance, the strong OOD results, and the controlled experiments.

ThinkUQ improves correctness prediction before generation with a short fixed prefill and zero decoded tokens. It reads the hidden state after the model’s empty reasoning block, keeping the backbone and answer policy unchanged. On Qwen3-4B, it improves AUROC over Query-end by 9.95 points on subject-OOD MMLU and 7.25 points on MMLU-Pro transfer.

During the rebuttal, we added DeepSeek experiments, a matched training control, probe transfer across training checkpoints, and attention-head interventions. We also evaluated source-based readout selection and selective answering across 20 model–task configurations. Using ThinkUQ scores with source-selected answering thresholds increases mean net utility by 19.02% over Query-end.

We have incorporated these results and clarifications into the revised manuscript, with changes marked in blue. We hope the committee will consider this additional evidence alongside ThinkUQ’s contribution to efficient correctness prediction before generation.

Best regards,
The ThinkUQ Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.