acceptodds
Under review as a conference paper at ICLR 2027

Ahead of the Evidence: Certifying Premature Commitment in Chains of Thought

Abstract

A chain of thought is increasingly read as evidence of how a model reached its answer. The worry that the model already knew before it finished reasoning is easy to state and hard to test, because "already" has no reference point: early relative to what? We supply one. On tasks whose correct procedure can be written as a short program, the program's state after each step fixes how far the answer is pinned down, a prescribed trajectory that no rule reading only that state can exceed. A chain that is really carrying out the procedure is one such reader, so it cannot have settled sooner. Earliness thereby becomes a testable claim rather than a suspicion. We locate a model's commitment causally, by transplanting the hidden state from one step of a different problem into the chain and asking whether the answer follows the transplant. Causal Lookahead Probing (CLP) fits no parameters, so a positive reading cannot be the artifact of a learned mapping that has proved fatal to probe-based causal analyses. On transformers compiled from the program itself, where the computation is known in advance, the method recovers that trajectory to within at every step. Across four such tasks and nine checkpoints up to 8B, reasoning-distilled models commit up to earlier than that state could license on three of them; on the task that licenses commitment latest, the gap to the undistilled parent persists at equal accuracy. Wherever the bound has room to bind, their undistilled parents stay within it, as does every base, instruction-tuned and state-space control. Their chains are, in these cases, transcripts of a decision already made.

Primary area:
interpretability and explainable AI
Keywords:
chain of thought, CoT faithfulness, mechanistic interpretability, causal intervention, activation patching, reasoning models, distillation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_SLLuOpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
3(+1)
Presentation
3(+1)
Soundness
2
Read review
Paper summary

This paper studies “premature commitment” in large language models’ Chain-of-Thought (CoT), asking whether a model’s internal state can effectively lock in the final answer before the explicit reasoning trace has finished. The authors propose Causal Lookahead Probing (CLP), which causally measures commitment depth by transplanting the last-layer residual stream from another problem at a particular reasoning step and testing whether the final answer follows the transplanted source state. In parallel, they construct a prescribed trajectory, CD_P, from an explicit reference program, representing the maximum degree of answer determination that is theoretically justified at each step. Under a licensed-dependence assumption, they prove that a faithful executor should satisfy CD_M(k) <= CD_P(k). Experiments span Dyck-2, finite-group problems, finite-state automata, entity tracking, and nine model checkpoints. CLP closely recovers the known trajectories of compiled reference programs. Most base, instruction-tuned, and control models remain within the prescribed bound, whereas two reasoning-distilled models violate it substantially on several formal tasks. On A5/S5, commitment occurs up to about 2.21 times earlier than the prescribed schedule allows. The authors argue that, in these cases, the emitted CoT often behaves more like a transcript of an already-formed decision than the computation that actually produced it.

Critical strengths
  1. The paper provides a genuinely useful normative reference for “premature” commitment. Prior work can often show that an answer is decodable or behaviorally recoverable early, but this alone does not establish that the timing is anomalous. The prescribed trajectory CD_P turns this qualitative concern into a task-dependent quantitative bound. I view this framing as the strongest conceptual contribution of the paper.

  2. The empirical control suite is unusually thorough. In particular, the random and orthogonalized-state controls convincingly separate generic disruption from source-directed redirection: arbitrary perturbations frequently change the answer, but almost never redirect it to the source answer, whereas a genuine source-state transplant does so at a very high rate. The operation-aligned re-analysis, matched-accuracy comparison, layer-patching controls, and seed/bootstrap checks further address several obvious alternative explanations.

Critical weaknesses
  1. Presentation and version-consistency issues. There is already a clear figure–text inconsistency in the main paper. Figure 4 contains five panels (Dyck-2, A4/S4, A5/S5, FSA, and Entity), whereas the caption refers to A5/S5 as the “right” panel, although the rightmost panel is Entity. This suggests that the figure and its description may not have been fully synchronized. Similar issues appear more extensively in the appendix. Section C.6 reports the same-answer sensitivity analysis over only 35 model-task cells, while the main study contains 9 models × 5 tasks = 45 cells, with no explanation of which ten cells were omitted. Section K.5 also states that four instances were manually audited while describing one instance from every exact-program task plus one entity-tracking instance, which does not appear to match that count. In addition, Figures 9 and 10 are not fully aligned with their captions or surrounding descriptions. Each issue may individually be editorial, but their repeated occurrence across both the main text and appendix makes it difficult to verify that all figures, counts, and analyses correspond to the same final experimental version.

  2. Experimental scope and generality. The experimental setup limits the generality of the main conclusion. The strongest certified results are obtained on highly structured state-tracking tasks for which the authors can explicitly specify a reference program. This is a well-motivated setting for validating CLP and the prescribed trajectory, but it leaves open whether the same phenomenon extends to broader forms of reasoning. The only natural-language task, entity tracking, relies on an approximate prescribed trajectory and is therefore treated as exploratory evidence. In addition, although the two reasoning-distilled checkpoints use different backbones, both are distilled from traces generated by the same DeepSeek-R1 teacher. The current experiments therefore do not cleanly distinguish a general effect of reasoning distillation from a teacher-specific or trace-distribution-specific effect. Finally, the headline CLP results are primarily based on interventions on the last-layer residual stream. The cumulative layer-patching ablations are helpful, but the main conclusions still depend to some extent on this intervention choice.

Questions for authors
  1. The main experimental design contains 45 model-task cells (9 models × 5 tasks), and Section I.11 explicitly reports statistics over all 45 cells. Why does Section C.6 state that the same-answer sensitivity analysis is conducted over only 35 cells? Which ten cells were excluded, what was the exclusion criterion, and do the reported maximum changes in U and k* remain the same if all 45 cells are included?

  2. Several figures appear inconsistent with their captions or surrounding descriptions. For example, Figure 9 is a stacked-bar plot although the caption refers to a “rising curve,” and Figure 10 visually reports prompt-end probe accuracy across models while the text describes an overlay of probe trajectories with CD_M and CD_P on R1-DISTILL. Could the authors clarify whether these are purely manuscript-editing errors and confirm that the numerical results, released result files, and final plotting scripts all correspond to the same experimental snapshot?

  3. Both reasoning-distilled checkpoints use different backbones but are distilled from traces generated by the same DeepSeek-R1 teacher, and the strongest certified evidence comes from structured state-tracking tasks with explicit reference programs. What evidence supports attributing the observed premature commitment to reasoning distillation more generally, rather than to DeepSeek-R1-specific trace properties or to this particular task family? If available, results from a different reasoning teacher or from an additional non-state-tracking task would substantially strengthen this claim.

6 comments
AuthorsOpenReview ↗

Response to Reviewer SLLu (Part 1 of 2)

Thank you for the close reading! Every inconsistency you list is real, and each is corrected in the revised PDF (changes in green). None of them changes a reported result. We answer W1 together with Q1 and Q2, then W2 with Q3, where we add the experiment you suggest.


W1, Q1, Q2. Figure, count and caption consistency.

ItemWhat was wrongCorrection
Figure 4The caption named two panels and placed A5/S5A_5/S_5 on the right; the figure has fiveCaption now describes all five panels, with A5/S5A_5/S_5 at the center
Section C.6 (Q1)The same-answer sensitivity check covered 35 cellsRerun on all 45; see below
Section K.5"one instance from each task with an exact reference program, plus entity tracking" implies fiveFour instances were audited: Dyck-2, A5/S5A_5/S_5, FSA and entity tracking. Text and caption of Table 25 now say so
Figure 9Caption mentioned a "rising curve"; the text said "all five tasks"The figure is a stacked-bar decomposition on A5/S5A_5/S_5 (n=2,400n=2{,}400 per condition), as in Figure 3. Caption and text corrected; the rates (71.471.4%, 1.31.3%, 1.61.6%, 1.41.4%) are unchanged
Figure 10The text described a step-resolved overlay on R1-DistillThe figure reports prompt-end answer-probe accuracy for the nine checkpoints of the main study. The paragraph now describes it and points to Table 17 for the step-resolved comparison

Q1, the ten missing cells. They were the two Qwen checkpoints (Qwen2.5-Math and R1-Distill-Qwen) on the five tasks; the check had been run before that family was added, and there was no exclusion criterion. On all 45 cells the conclusions are the same: the range of UU has median 00 and maximum 0.0300.030, the range of k∗k^{\ast} has median 0.00350.0035 and maximum 0.01160.0116, and on 43 of 45 cells the range is narrower than that cell's own bootstrap interval for k∗k^{\ast} (the two exceptions exceed it by less than 0.0020.002).

Q2, same snapshot. These were manuscript-editing errors: in each case the figure and the released file were correct and the text had not followed. We regenerated every table and figure from the released result files with the single script named in the Reproducibility Statement and checked each caption and count against its output.

AuthorsOpenReview ↗

Response to Reviewer SLLu (Part 2 of 2)

W2, Q3. One teacher, one task family, one layer.

A second teacher (new Table 4 in §6.4; intervals in Table 19, Section I). We added OpenThinker3-7B (Guha et al., 2026), a supervised fine-tune of Qwen2.5-7B-Instruct on 1.2M reasoning traces generated by QwQ-32B, a different reasoning teacher that was itself trained with outcome-based reinforcement (Qwen Team, 2025). Its own parent is the control, and both run under the main protocol:

TaskModelUU [95% CI]k∗k^{\ast}CDP\mathrm{CD}_P crossing
A4/S4A_4/S_4Qwen2.5-7B (inst)0.0020.002 [0.0000.000, 0.0070.007]0.4800.4800.3840.384
A4/S4A_4/S_4OpenThinker3-7B0.3850.385 [0.3770.377, 0.3940.394]0.2280.2280.3840.384
A5/S5A_5/S_5Qwen2.5-7B (inst)0.0000.000 [0.0000.000, 0.0020.002]0.7480.7480.6690.669
A5/S5A_5/S_5OpenThinker3-7B0.5380.538 [0.5310.531, 0.5460.546]0.3710.3710.6690.669
FSAQwen2.5-7B (inst)0.0040.004 [0.0000.000, 0.0170.017]0.5380.5380.3770.377
FSAOpenThinker3-7B0.2520.252 [0.2430.243, 0.2630.263]0.3010.3010.3770.377

The parent stays within the bound on every formal task and the child crosses it on every formal task. The evidence for attributing the effect to reasoning distillation is therefore three students from three different parents under two teachers, each violating the bound while its own parent does not. The violation is smaller for OpenThinker3-7B than for the two DeepSeek-R1 students, so its size varies with teacher and backbone; its presence does not. The same table adds Qwen2.5-Math-7B-Instruct, the base of R1-Distill-Qwen after chain-of-thought SFT and GRPO reinforcement: it stays within the bound on every formal task (U≤0.006U\le0.006), so on the backbone where both routes are measured, distilling a reasoning teacher's traces moves commitment ahead of the evidence and this reinforcement stage at the student does not.

Task family. The certificate requires a reference program, which is what makes the bound a bound rather than a heuristic. We therefore claim certified violations only on the three formal tasks and report entity tracking as an extension with an approximate trajectory (Table 2). Extending the certificate needs tasks whose intermediate state can be written down, not a change to CLP. A concrete next case is PrOntoQA (Saparov & He, 2023), where every example is generated from a first-order ontology with a unique gold proof and each chain sentence corresponds to one proof step.

Last-layer intervention. The cumulative layer patching of Section I.3 addresses this choice. On A5/S5A_5/S_5 at k=0.478k=0.478, widening the patch from the last layer to all layers raises R1-Distill's source-target rate monotonically from 0.8460.846 to 0.9480.948 and the 8B parent's from 0.1110.111 to 0.1590.159. The ordering of the two models is the same on every patch set, so the last-layer reading is the conservative one.


Summary of revisions. Figures 4, 9, 10 and 12: captions. Section C.6: the check on all 45 cells. Section K.5: audit description. Sections I.7 and I.8: text. §6.4, §7 and Section I: Tables 4 and 19, second teacher and direct RL. Section Q: the limitation on distillation sources.

We hope this resolves the consistency concerns, and we would be glad to clarify anything further.


Dear Reviewer SLLu,

Thank you again for the close reading that caught the inconsistencies between figures, counts and text. Each is corrected in the revised PDF, and the second-teacher experiment you suggested is now in Table 4. We would be glad to know whether the corrections resolve your concern about consistency, and whether the new checkpoints address the question of a single teacher. We are happy to clarify anything further in the remaining discussion period.

Best regards,
The Authors

Reviewer_HPxaOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies whether and when LLMs prematurely commit to an answer during Chain-of-Thought (CoT) reasoning. To define what "premature" actually means, the authors introduce a theoretical baseline based on C-RASP programs, which provides an upper bound on how much information is available at each reasoning step (the prescribed trajectory).

They then introduce Causal Lookahead Probing (CLP), a parameter-free method that swaps the last-layer hidden state at step k with that of another instance to test if the final answer follows the swap. Across five state-tracking tasks and several models (up to 8B), they find that non-distilled models stay within the theoretical bound, whereas reasoning-distilled models (like R1-Distill) lock into their final answers significantly earlier than the intermediate reasoning trace justifies.

Critical strengths

1.Well-grounded framework: The main strength is providing a concrete, formal benchmark for "early commitment." Instead of relying on subjective intuition or linear probes (which only show correlation), the paper introduces a Bayes-optimal bound and tests it with a clean, parameter-free intervention. This avoids the common issue where learned alignments can produce misleading positive results on random networks.

2.Solid and extensive empirical validation: The experiments are very thorough. The method is first verified on compiled transformers where the exact ground truth is known (recovering the curve within 0.006). The comparison between distilled models and their direct base/instruct parents at matched accuracy is convincing, showing that early commitment is linked to the distillation of reasoning traces rather than model scale.

Critical weaknesses

1.Limited to simple algorithmic state tracking: The theoretical bound relies on formalizing the task as a C-RASP program. While this makes the math rigorous, it restricts the formal results to stylized tasks like bracket matching and permutation groups. In more open-ended tasks where multiple valid reasoning strategies exist, it is unclear how to establish a similar baseline.

2.Evaluations rely on a single distillation source: Both distilled checkpoints are fine-tuned on traces from DeepSeek-R1. Without testing models trained directly with reinforcement learning (e.g., via RLVR/GRPO), it is hard to tell whether early commitment is an inherent property of RL-trained reasoning or simply an artifact of imitating teacher traces during supervised fine-tuning.

Questions for authors

1.In Algorithm 1, the intervention patches only the last-layer residual stream at step k. In practice, after the patch occurs, does the model actually continue reasoning in a way that reflects the new problem, or does it keep generating the old reasoning and only change the final answer token? A few concrete qualitative examples would be helpful.

2.On the operation-aligned axis (Section I.6), does re-indexing by operations change the conclusions on any of the other tasks besides A5/S5?

  1. It would be interesting to see if this framework can be adapted to tasks like math word problems or code generation, where multiple distinct reasoning paths are possible.
4 comments
AuthorsOpenReview ↗

Response to Reviewer HPxa (Part 1 of 2)

Thank you for the constructive review! We answer the three questions first, since two of them led to additions, and then the two weaknesses. Changes are in green in the revised PDF.


Q1. After the patch, does the model continue reasoning on the new problem, or only change the final answer?

In most cases it continues on the new problem. Under a rule frozen before any outcome was inspected (Section K.5), we classified all 5,4535{,}453 classifiable flips of R1-Distill on A5/S5A_5/S_5 in the window where flips are frequent (k=0.261k=0.261 to 0.5650.565):

Continuation after the patchShare
Names the source instance's elements and computes on them71.9%71.9\% (95% CI [70.7,73.0][70.7, 73.0])
Keeps the base reasoning, changes only the final answer14.9%14.9\%
Mixes both instances7.7%7.7\%
No interpretable step5.5%5.5\%

Two annotators labelled 300300 of these traces independently; they agree with each other on 93.3%93.3\%, and the adjudicated labels agree with the rule on 95.0%95.0\%. As you suggest, Section K.5 now shows a patched continuation: after the transplant the chain states the running product of the source instance, applies the source's last two generators, neither of which appears in the base input, and reaches the source answer.

Q2. Does operation-aligned re-indexing change the conclusion on other tasks?

No. We repeated the re-indexing on the two other formal tasks (new Table 20, Section I.6). No completion is re-run; only the axis changes.

TaskModelToken axis k∗k^*Operation axis o∗o^*UoU_oCDP\mathrm{CD}_P crossing
A4/S4A_4/S_4R1-Distill0.1900.1900.1680.1680.4490.4490.3840.384
A4/S4A_4/S_4Llama-3.1-8B (base)0.4820.4820.4750.4750.0000.0000.3840.384
A5/S5A_5/S_5R1-Distill0.3020.3020.2640.2640.6160.6160.6690.669
A5/S5A_5/S_5Llama-3.1-8B (base)0.7510.7510.7390.7390.0000.0000.6690.669
FSAR1-Distill0.2770.2770.2460.2460.2940.2940.3770.377
FSALlama-3.1-8B (base)0.5420.5420.5310.5310.0020.0020.3770.377

On every formal task R1-Distill commits earlier on the operation axis than on the token axis, and its parent stays past the CDP\mathrm{CD}_P crossing. Correcting for verbosity therefore moves the distilled commitment earlier, not later. Dyck-2 and entity tracking are not re-indexed: the first leaves little room for a violation on either axis, and on the second the operation extractor is systematically generous (Section I.6).


AuthorsOpenReview ↗

Response to Reviewer HPxa (Part 2 of 2)

Q3, W1. Beyond algorithmic state tracking.

This is a boundary of the certificate, and we state it as one. The bound is a bound because the reference program fixes what each step can license; without such a program, early commitment can be measured but not certified as premature. CLP itself needs no program, so CDM\mathrm{CD}_M can be read on any task, as we do on entity tracking with an approximate trajectory (Table 2). For math word problems or code, where several solution paths exist, the natural relaxation is to certify against a family of programs, taking at each step the largest CDP\mathrm{CD}_P any of them licenses, so that a violation holds against every valid strategy. Section Q notes that this optimization is intractable in general and that fixing the family in advance, for instance the proof steps of a formal verifier, is a tractable substitute. A concrete intermediate case is PrOntoQA (Saparov & He, 2023), whose examples are generated from a first-order ontology with a unique gold proof, so each chain sentence maps to one proof step. We leave this to future work and do not claim it here.

W2. A single distillation source; no model trained directly with RL.

We added both (new Table 4 in §6.4; intervals in Table 19, Section I). The first two rows share one backbone and the last three another, parent first.

CheckpointPost-trainingUU: A4/S4A_4/S_4UU: A5/S5A_5/S_5UU: FSAk∗k^* on A5/S5A_5/S_5
Qwen2.5-7B (inst)SFT, DPO, GRPO (parent)0.0020.0020.0000.0000.0040.0040.7480.748
OpenThinker3-7B+ SFT on QwQ-32B traces0.3850.3850.5380.5380.2520.2520.3710.371
Qwen2.5-Mathmath pretraining (parent)0.0000.0000.0010.0010.0000.0000.7440.744
Qwen2.5-Math (inst)+ CoT SFT and GRPO0.0040.0040.0050.0050.0060.0060.7240.724
R1-Distill-Qwen+ SFT on DeepSeek-R1 traces0.4280.4280.5840.5840.2780.2780.3320.332

The CDP\mathrm{CD}_P crossing on A5/S5A_5/S_5 is 0.6690.669.

A second teacher. OpenThinker3-7B (Guha et al., 2026) is a supervised fine-tune of Qwen2.5-7B-Instruct on 1.2M reasoning traces generated by QwQ-32B. It crosses the bound on all three formal tasks and its own parent does not, so the effect is not specific to DeepSeek-R1 traces.

Direct RL. Qwen2.5-Math-7B-Instruct is the Qwen2.5-Math base of R1-Distill-Qwen after chain-of-thought SFT and GRPO, the algorithm you name, with a sequence-level reward from an answer verifier and a reward model (Yang et al., 2024; Shao et al., 2024). It stays within the bound on all three formal tasks. On the backbone where both routes are measured, reinforcement at the student leaves commitment where the task licenses it, while distilling a reasoning teacher's traces moves it ahead. To your question, then, the evidence points to imitation of a reasoning teacher's traces rather than to reinforcement at the student.

We state two limits in the paper. This RL stage is short (66K queries) and its chains are solution-style, so the result shows that RL at the student is not sufficient, not that it could never suffice. And both teachers were themselves trained with outcome-based RL: the reinforcement stage from which DeepSeek-R1's reasoning emerged uses a reward signal "only based on the correctness of final predictions against ground-truth answers, without imposing constraints on the reasoning process itself" (Guo et al., 2025). §7 therefore names the teacher's training as a plausible origin and leaves open, since we do not measure the teacher, whether the timing arises there or in distillation.


Summary of revisions. Section K.5: a patched continuation. Section I.6: Table 20, operation axis on all formal tasks. §6.4, §7 and Section I: Tables 4 and 19, second teacher and direct RL. Section Q: the limitation on distillation sources updated accordingly.

We hope this answers your questions, and we would be glad to discuss further.


Dear Reviewer HPxa,

Thank you again for your constructive questions, which led to three additions: the patched continuation in Section K.5, the operation-axis analysis on all formal tasks (Table 20), and the second teacher and direct-RL checkpoints (Table 4). We would be glad to know whether these address your questions, in particular the one on RL versus distillation, and we are happy to provide anything further in the remaining discussion period.

Best regards,
The Authors

Dear Reviewer HPxa,

As the discussion period closes, we would like to thank you again for your review.

Your Q3 asked whether the framework can be adapted to math word problems or code generation. Reviewer SLLu raised the same question on 18 November. For your convenience we reproduce our answer here. In short: the reference for when commitment is premature, which your review named as the main strength, is also the part that travels.

1. Where the method applies

What the bound needs is "a program whose step-by-step information demand is known" (Section Q). State tracking is one way to supply it, not the only one. Let DtD_t be the intermediate results a chain has stated by step tt. Replacing the program state RkR_k by DtD_t in Definition 2, the proof of Theorem 1 goes through unchanged and gives CDM(t)≤CDP(t)\mathrm{CD}_M(t)\le\mathrm{CD}_P(t). CLP itself needs nothing new: it fits no parameters, so there is no probe to retrain for a new domain. Only the reference changes.

SettingA concrete caseDoes the method apply?
Multi-step calculationAdding two-digit numbers one at a time; evaluating a nested expression from the inside outYes
Templated word problemsGSM-Symbolic (Mirzadeh et al., 2025): a template fixes the solution steps; only names and numbers changeYes
Code executionOutput prediction in the style of CRUXEval (Gu et al., 2024), with several inputs per function, e.g. a function whose loop updates a counterYes
Open-ended code designWriting a function from a docstring, as in HumanEvalA different question: many programs are correct, so there is no single answer to certify

Our hardest tasks are already calculations of the first kind: A5/S5A_5/S_5 asks for a running product of permutations, the same shape as a running sum. Two details decide whether the extension is sound:

  • Use what has been derived, not what is given. The full problem is in the prompt, so the reference counts only the results derived so far. Counting the given numbers would make every answer look settled from the start.
  • Follow the chain's own progress. The model is compared with the results its chain has written so far, whatever their order or pace. This is the operation axis of Section I.6, on which R1-Distill commits even earlier than on the token axis (Table 20).

The reference also shows in advance where a violation is possible: only before the chain itself has settled the answer. In most calculation that happens at the last operation, so most of the chain can be tested. Among our tasks, A5/S5A_5/S_5 settles the answer latest and is where the effect is strongest; where the answer is settled almost at once, little is left to test, as on Dyck-2.

2. Will the same behaviour appear there?

We expect so, for a reason that comes from the data in this paper: whether a model commits prematurely goes with how it was trained; the task only sets how much room there is. On identical prompts, every base, instruction-tuned and state-space control stays within U<0.012U<0.012 on all five tasks, the natural-language one included. The models that cross the bound are those trained on reasoning traces (three students, three parents, two teachers), and reinforcement applied at the student does not produce it (Table 4). Nothing in that pattern depends on state tracking.

3. Why we think this matters

Chains of thought are increasingly read as evidence of how a model reached its answer. Wang & Zhu (2026) already report that answers become recoverable early in the chain on mathematics, science and code. But in their results, how early depends on how hard the problem is: on MATH-500, Qwen3-32B in thinking mode reaches that point at 13% of its chain on the easiest level and at 32% on the hardest. Boppana et al. (2026) see the same contrast between recall-style MMLU questions and multi-hop GPQA-Diamond. An early answer may therefore simply reflect an easy problem. What those domains lack is a way to tell an answer that is early from one that is too early. That is what the reference provides, and it is what let us see that training on reasoning traces is what moves commitment ahead of the chain.

4. What we have done since submission

This is the future work our earlier replies pointed to, and it is well under way. We have built these extensions and run them extensively, on multi-step arithmetic, GSM-Symbolic word problems, CRUXEval-style output prediction and, as the case without a reference, HumanEval. The results so far point the same way as those in this paper. They form a separate study, which we will report in full on its own; for logical reasoning, PrOntoQA remains the next case, as in our earlier reply. We will add the discussion above to Section Q of the final version.

We would be glad to hear if anything remains unclear, and we thank you for your time!

Best regards,
The Authors

Reviewer_CnvvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper focuses on the faithfulness of the CoT and explores whether a reasoning model already has the answer in its hidden state even before it finishes the long chain of reasoning. The author proposes a new framework using CDP(k)CD_P(k) and CDM(k)CD_M(k) to systematically capture the cases where the CDM(k)CD_M(k) breaks the upper-bund of CDP(k)CD_P(k) and thus use additional information than the Bayes-optimal guess. It runs experiments on five state-tracking tasks using multiple open-sourced large reasoning models to capture the violation of the bound.

Critical strengths
  • It focus on the an important problem and propose a new framework to understand and measure the faithfulness of LLM's CoT with CDM(k)CD_M(k) and CDP(k)CD_P(k)
  • It constructs five state-tracking tasks of different difficulties to detect the violation of upperbound.
  • It conducts extensive experiments on distilled models, their parent of distillation and non-distilled models to observe the violation of upper bound.
Critical weaknesses
  • The major weakness is the gap between the upper bound and the faithfulness of CoT. The upper bound CDM(k)<CDP(k)CD_M(k) < CD_P(k) only applies to the case where the language model could only reason based on the status RKR_K and the information at RKR_K. But the language model has the entire information at the beginning of CoT so it doesn't have to following the constraint of RKR_K.
  • Only the task accuracy on (A5/S5)(A_5/S_5) is provided for five models in Table 3 and task accuracy for other tasks are not revealed. If the task is too difficult for the model, the upper bound constraints would be too easy to satisfy.
Questions for authors

Regarding the definition of the faithfulness, your definition within Section 2.3 is that "A CoT is faithful when the steps it emits are the steps the model uses to reach its answer". Does it also includes the case where "the steps it emits are the steps the model used to reach its answer" or it only counts the case where "the steps it emits are the steps the model is using to reach its answer"? Since a model might know the answer immediately after reading the prompt at the beginning of CoT and only use CoT to repeat its reasoning process.

4 comments
AuthorsOpenReview ↗

Response to Reviewer Cnvv (Part 1 of 2)

Thank you for the careful reading! Your question on the tense of "uses" goes to the heart of what the paper measures, so we answer it first; it also settles most of W1. All changes are in green in the revised PDF.


Q. "Used" or "is using"?

Is using. The present tense is not a stylistic choice: it is the only reading under which Theorem 1 bounds faithful chains.

Why the theorem fixes the tense. Take the case you describe: the model has the answer after reading the prompt and uses the chain to repeat its reasoning. At any step kk before the chain itself has computed the answer, the state then already carries it and the output follows that state, so CDM(k)>CDP(k)\mathrm{CD}_M(k)>\mathrm{CD}_P(k) and the bound is violated. If our definition also counted such a chain as faithful (the "used" reading), Theorem 1 would flag chains that are faithful by our own definition, and CDP\mathrm{CD}_P would not bound faithful chains. The bound is consistent only with the present-tense reading, and §2.3 now states it.

What the "used" reading asks. It is a legitimate but different, descriptive question: whether a chain written after the decision accurately retells how the decision was reached, in the sense of Jacovi & Goldberg (2020), "accurately represents the reasoning process behind the model's prediction". Lanham et al. (2023) note that post hoc reasoning does not by itself exclude faithfulness in that sense, while their own measures score less post hoc reasoning as more faithful ("higher is less post-hoc, indicating greater faithfulness", their Table 2). We adopt that operational, present-tense standard and make no claim about whether a post hoc chain retells its computation accurately.

What holds under either reading. The finding itself is about timing: on the three formal tasks the distilled models commit before their chains license it, so their chains are "transcripts of a decision already made" (abstract). This holds whichever tense one prefers; calling it unfaithful follows from the reading of §2.3. CDP\mathrm{CD}_P also separates knowing early from knowing too early: on Dyck-2 the CDP\mathrm{CD}_P crossing is at 0.1290.129, so R1-Distill's commitment at 0.1820.182, early as it is, comes after the point the task licenses.

→ Revision, §2.3:

Faithfulness in this sense concerns the steps the model is using as it emits them, not ones used before: a chain written after the answer is fixed, whose steps restate rather than produce the decision, is post hoc (Lanham et al., 2023) and unfaithful under this reading even if every step is correct.


AuthorsOpenReview ↗

Response to Reviewer Cnvv (Part 2 of 2)

W1. "The language model has the entire information at the beginning of CoT so it doesn't have to follow the constraint of RkR_k."

The premise is correct, and Section I.5 builds on it. What we would like to clarify is the logical role of the bound. Theorem 1 does not assume that a model follows RkR_k; it states what must hold if it does. We use the contrapositive: CDM(k)>CDP(k)\mathrm{CD}_M(k)>\mathrm{CD}_P(k) certifies that the state at step kk carries more about the answer than the first kk steps of the chain have licensed. "The model had the information and did not need the chain" is exactly what a violation concludes.

Two observations show that the test is informative.

(i) Only the transplanted state can carry the source answer. The base instance is drawn independently of the source, so the patched run learns about a(s)a(\mathbf{s}) through that state alone. Having read the whole prompt makes a violation possible; it does not make one inevitable.

(ii) Seeing the input is not sufficient to violate the bound. All nine checkpoints of Table 2 receive the same prompt (identical SHA-256, §6.4). Every non-distilled checkpoint stays within U<0.012U<0.012 on every task, while the two distilled checkpoints reach U=0.641U=0.641 and 0.5840.584 on A5/S5A_5/S_5. If access to the input were enough, all nine would violate.

An analogy: let the chain be a ledger and RkR_k the running total after kk entries. A clerk who saw every receipt in advance may know the final total at entry 3, and nothing forbids that. But if the clerk's notes at entry 3 already fix the final total more tightly than entries 1 to 3 can, the remaining entries are not where the total was computed.

→ Revision, §3.3 (after Theorem 1):

The hypothesis is the claim under test, not an assumption about models: a model reads its whole input before the chain begins and is free to violate it, and a violation is the evidence that it has. Because the base instance is drawn independently of the source, the patched run learns about a(s)a(\mathbf{s}) only through the transplanted state, so the test asks whether that state already carries more about the source answer than the first kk steps of the chain have licensed.


W2. Accuracy on the other tasks; "if the task is too difficult for the model, the upper bound constraints would be too easy to satisfy."

The accuracy of all 45 (model, task) cells is now reported (new Table 24, Section J). It ranges from 0.620.62 to 0.940.94 (chance is 1/601/60 on A5A_5 and 1/1201/120 on S5S_5), and the two distilled checkpoints, the ones that violate the bound, are at least as accurate as every control on every task. Three results address difficulty directly.

1. No control satisfies the bound by failing the task. Sources are instances the model answers correctly (§2, Section F.2), and in all 45 cells CDM\mathrm{CD}_M rises above 0.990.99 by the final step. The controls stay under the bound because they commit late, not because they never commit.

2. Equal accuracy, opposite verdicts. On FSA, Llama-3.1-8B (inst) and R1-Distill-Qwen both reach 0.8790.879, with U=0.000U=0.000 versus 0.2780.278.

3. Accuracy held fixed. §6.5 restricts each comparison to the instances both models answer correctly. We have extended it from A5/S5A_5/S_5 to all three formal tasks and to both families (new Table 5 in §6.5, per-comparison intervals in Table 18, Section I; 11 further comparisons):

TaskComparisonsJointly correct nnDistilled k∗k^*Control k∗k^*CDP\mathrm{CD}_P crossing
A5/S5A_5/S_5 (§6.5)4612612 to 7387380.3060.306 to 0.3130.3130.7380.738 to 0.7440.7440.6690.669
A5/S5A_5/S_5 (new)16826820.3400.3400.7360.7360.6690.669
A4/S4A_4/S_4 (new)5673673 to 7807800.1950.195 to 0.2120.2120.4660.466 to 0.4800.4800.3840.384
FSA (new)5742742 to 8268260.2810.281 to 0.2980.2980.5250.525 to 0.5350.5350.3770.377

In all 15 comparisons the distilled checkpoint commits before the CDP\mathrm{CD}_P crossing and its control after it (1.801.80 to 2.46×2.46\times earlier; every interval on the difference excludes zero).


Summary of revisions. §2.3: tense of "uses". §3.3: the hypothesis is the claim under test. §6.3 and Section J: Table 24. §6.5 and Sections F, I, Q: Tables 5 and 18. Section I.5: one sentence citing Liu et al. (2023) on shortcut solutions.

We hope this clarifies the role of the bound, and we would be glad to discuss further.


Dear Reviewer Cnvv,

Thank you again for your careful reading! Your question on "used" versus "is using" led us to state the definition explicitly in §2.3, and your W1 led to the clarification after Theorem 1 in §3.3. We would be glad to know whether these match the reading you had in mind, or whether any part of the concern about the role of the bound remains. We are happy to clarify further in the remaining discussion period.

Best regards,
The Authors

Dear Reviewer Cnvv,

Over the past six days we ran four further experiments that bear directly on your two concerns. All are on A5/S5A_5/S_5 under the main protocol; the trained models are R1-Distill and its parent, Llama-3.1-8B (base). They are reported in a new Section R of the revised PDF (pages 59 and 60, Tables 31 to 33, in green), and §3.3 (page 5) now points to it.

W1.

"The language model has the entire information at the beginning of CoT so it doesn't have to follow the constraint of RkR_k."

1. Withholding the input (Section R.1, Table 31). Your premise can be made a variable. We reveal each generator only when the chain reaches it, so that at step kk no model has seen more than the first kk generators. For an instance σ1,σ2,σ3,σ4\sigma_1, \sigma_2, \sigma_3, \sigma_4, the model has seen only σ1\sigma_1 and σ2\sigma_2 when it writes the step that applies σ2\sigma_2; σ3\sigma_3 appears only after that step is written.

InputR1-Distill UUR1-Distill k∗k^{\ast}Parent UUParent k∗k^{\ast}
Given in full (Table 2)0.6410.6410.3020.3020.0000.0000.7510.751
Revealed step by step0.0050.0050.6940.6940.0000.0000.7540.754

With the input withheld, the violation disappears and R1-Distill commits after the CDP\mathrm{CD}_P crossing (0.6690.669), as its parent does; R1-Distill-Qwen behaves the same way (U=0.007U=0.007). The parent is unchanged. The violation is therefore the use of input the chain has not yet reached, which is the case you describe, and the bound is what detects it.

2. A model built to do what you describe (Section R.2). We compiled a second executor that computes the product in parallel at the first step and then writes out the same chain as the step-by-step executor of Figure 2. The two emit identical chains. CLP reads U=0.993U=0.993 on the first and a deviation of at most 0.0060.006 on the second. A chain that repeats a finished computation and a chain that carries it out look the same on the page, and the bound separates them.

3. The link to a standard faithfulness test (Section R.3, Table 32). In the adding-mistakes test of Lanham et al. (2023), one step of the chain is replaced by a mistaken version, the model continues from there, and one records how often the final answer changes. We replace one intermediate product; the table gives the fraction of final answers that change.

Mistake inserted atR1-DistillParent
k=0.174k=0.174 (before R1-Distill commits)0.8090.8090.8680.868
k=0.478k=0.478 (after R1-Distill commits, before its parent does)0.1390.1390.8570.857

Before its commitment step (k∗=0.302k^{\ast}=0.302) R1-Distill's answer moves with its chain, as its parent's does; after it, the chain no longer moves the answer. The parent, which commits at 0.7510.751, moves with its chain at both points. Where the bound is violated, the chain has stopped doing the work.

W2.

"If the task is too difficult for the model, the upper bound constraints would be too easy to satisfy."

4. Short against long instances (Section R.4, Table 33). We split A5/S5A_5/S_5 by the number of generators.

InstancesParent accuracyParent UUR1-Distill UU
Short (4 to 5 generators)0.8440.8440.0000.0000.6180.618
Long (7 to 8 generators)0.7310.7310.0010.0010.6590.659

The parent stays within the bound on the short instances, where it is most accurate, exactly as on the long ones, and R1-Distill violates it on both. The controls do not satisfy the bound by finding the task hard.

We hope these experiments make the role of the bound concrete, and we would be glad to hear whether any case remains open for you.

Best regards,
The Authors

Comments on the paper

2 comments

Dear Reviewers,

We are now halfway through the discussion phase. The authors have provided their responses to your reviews. If their responses have not adequately addressed your concerns, please take this opportunity to raise any remaining questions or concerns.

I encourage you to engage with the authors as soon as possible to ensure a productive discussion.

Many thanks,
AC

AuthorsOpenReview ↗

Summary of the discussion

Dear Area Chair and Reviewers,

As the discussion closes, we thank the reviewers and the Area Chair for their time. We summarize what the paper contributes and where each point raised now stands.

The idea. That a model "already knew" its answer before its chain of thought finished is easy to suspect and has been hard to test, because early is not too early: an answer can be available early simply because the problem is easy. What was missing is a reference. On tasks whose procedure can be written as a program, the program's state after each step fixes how far the answer can be pinned down, and Theorem 1 shows that no chain that is carrying out the procedure can commit sooner. Earliness becomes a claim that can be refuted. The reference separates the two in practice: on Dyck-2, R1-Distill commits at 0.1820.182 of its chain, the earliest commitment in the main study, and this still comes after the point the task licenses (0.1290.129).

The measurement. CLP transplants a hidden state from a different problem and asks whether the answer follows it. It fits no parameters, so a positive reading cannot be produced by a learned map, the known weakness of probe-based causal analyses. On transformers compiled from the program itself, it recovers the reference to within 0.0060.006 at every step.

The finding. Whether a model commits prematurely goes with how it was trained. Three students trained on the reasoning traces of two different teachers cross the bound. On identical prompts, each student's own parent stays within it, as does every base, instruction-tuned and state-space control; the gap persists when accuracy is held fixed, and reinforcement applied at the student does not produce it. In these cases the chain is a transcript of a decision already made.

What it opens and where it is useful. For interpretability: a way to certify a claim about a model's internal computation, with a reference fixed by the task and a measurement checked where the ground truth is known, in place of a learned probe. For reasoning models: the pattern points to imitation of reasoning traces, through which a student can learn to write a derivation without its answer depending on it. Whether distillation preserves reasoning thereby becomes a measurable question, including where the effect arises, in the teacher or in distillation. For practice: chains of thought are increasingly read as evidence of how a model reached its answer, and the reference tells when that reading is warranted. It needs only a program whose step-by-step information demand is known, which covers multi-step calculation, templated word problems and code execution; that extension is under way.

Where the reviews agree. All three name the framework as the strength: "a new framework" for "an important problem" (Cnvv), "a concrete, formal benchmark for 'early commitment'" (HPxa), and, of the prescribed trajectory, "the strongest conceptual contribution of the paper" (SLLu). Every point raised is listed below; the last column says whether it was addressed by argument (theory), by experiment, or by both.

Raised byPointHow it was addressedWhereBy
Cnvv (Q)"Used" or "is using"Present-tense reading stated; Theorem 1 bounds faithful chains only under it§2.3Theory
Cnvv (W1)The model sees the whole inputThe hypothesis is the claim under test (contrapositive of Theorem 1); all checkpoints read the same prompt and only the distilled ones violate§3.3, Table 2Both
Cnvv (W2)Accuracy; task difficultyAccuracy for all 45 cells; matched-accuracy comparison on all three formal tasks (15 comparisons)Tables 5, 18, 24Experiment
HPxa (Q1)What follows the patch?5,453 flips classified; a patched continuation shownSection K.5Experiment
HPxa (Q2)Operation axis elsewhereThe two other formal tasks re-indexedTable 20Experiment
HPxa (W2), SLLu (W2, Q3)One teacher; no RLA second teacher and a GRPO-trained checkpoint, each with its parentTables 4, 19Experiment
SLLu (W2)Last-layer patchCumulative layer patching: the same ordering on every patch setSection I.3Experiment
SLLu (W1, Q1, Q2)Figure and count consistencyEvery item corrected; Section C.6 rerun on all 45 cellsFigures 4, 9, 10, 12; Section C.6Rerun
HPxa (Q3, W1), SLLuBeyond state trackingTheorem 1 carries over with derived results in place of the program state; scope statedReplies of 18, 19 Nov; to be added to Section QTheory

No conclusion of the submission has changed. Reviewer SLLu wrote that the responses addressed most concerns, raised the confidence score and, after our follow-up, added: "now I've got a better view for your work".

Thank you again for your effort and the constructive discussion!

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.