acceptodds
Under review as a conference paper at ICLR 2027

How Does Reasoning Become Implicit? Uncovering the Dynamics of Chain-of-Thought Internalization

Abstract

Latent reasoning replaces explicit chain-of-thought (CoT) with continuous states, yet how this transition reorganizes reasoning remains unclear. We study this internalization process through a unified mechanistic framework spanning training-time formation and inference-time function, and apply it to CODI and Coconut as two representative latent-reasoning paradigms. We find that internalization yields a strongly positionally asymmetric latent process, including pronounced even–odd organization and training-time reversals of positional bias. Despite this reorganization, latent reasoning retains task-relevant alignment with explicit CoT while following a distinct intermediate trajectory. At inference, latent-state contributions are recurrently interdependent and strongly conditioned on the surrounding trajectory. Finally, the explicit–latent reliability gap grows sharply with reasoning depth, while failed latent trajectories exhibit weaker stepwise semantic ordering and more diffuse causal organization across states. Together, these results show that reasoning internalization is not simply the direct transfer of explicit CoT into a hidden trajectory. The central mechanistic challenge for better reasoning internalization is therefore how task-relevant computation is organized across latent states and coordinated into reliable reasoning.

Primary area:
interpretability and explainable AI
Keywords:
implicit chain-of-thought, latent reasoning, interpretability, large language models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_DGByOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates how explicit chain-of-thought is internalized into continuous latent computation through a framework connecting training dynamics with inference-time function. The results show positional specialization and training-time parity reversals, partial semantic alignment with explicit reasoning despite distinct intermediate representations, and recurrent dependence between latent states. The explicit–latent reliability gap increases with solution depth, while failed trajectories exhibit weaker semantic ordering and more diffuse positional influence.

Critical strengths
  1. The checkpoint analyses provide consistent evidence of positional specialization and parity reversals across multiple measurements. These findings offer valuable insights into how latent computation is reorganized during training.

  2. Comparing fixed and regenerated continuations, together with restoration experiments, provides convincing evidence of inter-state dependence, and the careful intervention design helps explain how latent states work together during reasoning.

The paper presents a solid empirical study with systematic analyses and carefully designed interventions.

Critical weaknesses

The experiments are restricted to mathematical word problems on GSM8K, leaving the generality of the reported organizational patterns uncertain. Evaluation on non-mathematical benchmarks, such as PrOntoQA for logical reasoning and CommonsenseQA for commonsense reasoning, would help establish whether positional specialization, recurrent dependence, and the relationship between latent organization and reliability extend beyond arithmetic reasoning.

Questions for authors
  1. How can the positional, alignment, and intervention analyses guide practical improvements to existing methods such as CODI and Coconut, for example in latent-state supervision or latent-budget allocation?

  2. Can the authors evaluate whether the main positional patterns and outcome-conditioned organizational differences also appear on non-mathematical tasks such as PrOntoQA and CommonsenseQA?

3 comments

Thank you for the constructive review and positive assessment. We address your questions and concerns below and will incorporate your feedback in the updated version.

W1 + Q2

We extend our experiments to additional benchmarks, including ProsQA, CommonsenseQA, SVAMP, MultiArith and GSM-Hard. The new evidence further strengthens the paper’s central finding: current implicit reasoning paradigms have yet to reliably learn to meaningfully utilize and coordinate latent states into a coherent, well-organized reasoning process. On ProsQA and CommonsenseQA, latent states are unused for computation, and thus no pronounced organization emerges; on SVAMP, MultiArith, and GSM-Hard, where latent computation is actively used, the core organizational patterns reappear, including positional specialization, training-time reversals, and weaker causal organization in failed trajectories.

  • ProsQA and CommonsenseQA: Prior work has shown that latent tokens can be unnecessary for latent computation [1, 2, 3]. Thus, we first directly measure the latent state reliance by training CODI on CommonsenseQA and ProsQA and Coconut on ProsQA with latent budget K=6, and then compare the inference accuracy with and without latent states (K = 6 vs. K = 0).
Accuracy (%)K=6K=0
ProsQA, CODI GPT-281.181.0
ProsQA, Coconut GPT-296.196.1
CommonsenseQA, CODI Llama-1B72.872.9
GSM8K, CODI GPT-243.618.5
GSM8K, CODI Llama-1B56.734.5
GSM8K, Coconut GPT-232.94.9

Removing latent tokens leaves accuracy virtually unchanged on ProsQA and CommonsenseQA, but reduces GSM8K accuracy by 22–28 points. Causal interventions confirm this contrast: donor substitutions change no ProsQA answers and at most 1.2% on CommonsenseQA, versus 94–96% on GSM8K. This negligible causal reliance persists across all analyzed checkpoints.

TaskModelLatent acc. (%)Donor latents change the answerTotal causal load, solved problems (nats per problem)
CODI GPT-243.694%2.47
GSM8KCoconut GPT-232.996%3.61
CommonsenseQACODI GPT-2 / Llama-1B44.0 / 72.8≤1.2%0.071 / 0.018
ProsQACODI GPT-281.10/6490.004
ProsQACoconut GPT-296.10/7690.003

("Donor" replaces the whole latent trajectory with that of a different-answer problem (as in Fig. 5c), and total causal load is the summed effect of erasing each latent input in turn (Σ_i |N_{dir,i}|), in nats of gold-answer log-probability.)

Correspondingly, no pronounced positional specialization emerges on ProsQA or CommonsenseQA. As the model itself does not properly learn to utilize latent states for reasoning, there's naturally not any prominent organization within the latent states emerges.

  • SVAMP, MultiArith and GSM-Hard: To further broaden our evaluation, we applied the same analyses to three additional benchmarks: SVAMP, MultiArith, and GSM-Hard. On these benchmarks, where the models actively utilize latent states for computation, the core organizational and reliability patterns consistently reappear.
Accuracy (%)K=6K=0
SVAMP, CODI GPT-240.225.8
SVAMP, CODI Llama-1B59.453.3
SVAMP, Coconut GPT-235.317.0
MultiArith, CODI GPT-291.831.0
MultiArith, CODI Llama-1B96.867.3
MultiArith, Coconut GPT-283.74.7
GSM-Hard, CODI GPT-210.04.5
GSM-Hard, CODI Llama-1B13.78.6
GSM-Hard, Coconut GPT-27.60.9

Training-time formation and reversals replicate. Across 10 CODI GPT-2 checkpoints, the even–odd selectivity of erasure effects, (even − odd)/(even + odd), follows the same dynamics:

CODI GPT-2 checkpointSVAMPMultiArithGSM-Hard
ep2−0.03−0.05−0.08
ep4+0.02−0.01−0.00
ep6+0.55+0.66+0.63
ep8−0.31−0.47−0.51
ep10−0.30−0.39−0.56
ep12+0.27+0.53+0.54
ep14+0.30+0.58+0.48
ep20+0.18+0.53+0.39
ep30+0.34+0.71+0.69
ep40+0.47+0.91+0.77

Positional and causal specialization persist at convergence. At the final checkpoints:

GSM8K-trained modelLatent / explicit accuracy (%): SVAMP, MultiArith, GSM-HardLayers matching GSM8K parityLayer-profile correlation
CODI GPT-240.2/42.2, 91.8/94.7, 10.0/10.712/12, 11/12, 11/120.96, 0.92, 0.68
CODI Llama-1B59.4/57.5, 96.8/96.5, 13.7/14.311/11, 9/11, 11/110.94, 0.88, 0.90
Coconut GPT-235.3/40.0, 83.7/85.2, 7.6/9.410/10, 10/10, 10/100.90, 0.85, 0.94

Layers: GSM8K parity-index magnitude >0.5. Erasure profiles retain the GSM8K-dominant parity for both CODI models on all benchmarks, and Coconut on MultiArith/GSM-Hard (S=0.45/0.17). GSM-Hard CODI GPT-2 also retains its erasure-profile shape (u0–u5: 1.26/.13/.22/.08/.37/.04 nats; Fig. 1d).

W1+Q2 (continue)

The reliability gap retains its depth dependence (§4.3). On GSM-Hard, which spans multiple reasoning depths, the conditional latent failure rate again increases sharply with solution depth:

Explicit solution depth (GSM-Hard)≤234
CODI GPT-20.110.260.53
CODI Llama-1B0.130.170.36
Coconut GPT-20.230.400.55

Together, these findings highlight a fundamental limitation of current implicit reasoning paradigms: latent computation is not consistently and reliably learned and organized into a coherent reasoning process. Models may either bypass latent states altogether or exhibit pronounced positional biases and insufficient coordination when using them. This motivates stronger relational process supervision to promote meaningful latent computation and coherent coordination across states.

[1] Are Latent Reasoning Models Easily Interpretable? COLM 2026

[2] The Illusion of Superposition? A Principled Analysis of Latent Thinking in Language Models. 2026

[3] Do Latent Tokens Think? A Causal and Adversarial Analysis of Chain-of-Continuous-Thought. EMNLP 2026

Q1

Our mechanistic findings provide concrete directions for improving CODI and Coconut, building on the discussion in §5

  • Relational process supervision. Our analyses highlight the importance of meaningfully utilizing latent states and coordinating their computation into a coherent reasoning process, as reflected in differences in causal reliance across tasks and weaker semantic ordering and more diffuse causal influence in failed trajectories. These findings motivate stronger supervision of intermediate computational content and inter-state coordination, promoting meaningful and well-organized latent computation without imposing rigid step-to-position alignment. This principle could guide improvements to CODI and Coconut, and potentially extend to iterative architectures such as looped models and latent diffusion models.

We hope the above replies meaningfully address your core concerns.

Thank you for the additional cross-task experiments. These results help clarify the scope of the findings, and I encourage you to include them in the revised paper.

Reviewer_jhQFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates how explicit chain-of-thought (CoT) reasoning is internalized into continuous latent states. The authors propose a unified analytical framework that examines both the formation dynamics during training and the functional efficacy of latent reasoning during inference. Through extensive analyses of CODI and Coconut, the paper identifies several findings: (1) latent reasoning exhibits strong positional asymmetry, including even–odd patterns and training-time reversals; (2) latent states preserve task-relevant semantic alignment with explicit CoT while following distinct intermediate trajectories; and (3) reasoning failures are associated with weaker semantic organization and more diffuse causal influence across latent states. These findings provide insights into the internal mechanisms and limitations of latent reasoning.

Critical strengths

1.The proposed analytical framework is interesting and provides a novel perspective on understanding the formation and functionality of latent reasoning. 2.The analysis of positional asymmetry and inter-state interactions is insightful and offers a better understanding of how latent reasoning operates. 3.The experiments include multiple models and various analytical techniques, such as semantic probing and causal interventions, which strengthen the empirical analysis.

Critical weaknesses

1.Some key findings, particularly the even–odd positional patterns and inter-state dependencies, overlap with previous studies. The novelty of the proposed analysis should be better clarified. 2.The connection between latent-state organization and reasoning reliability is mainly supported by correlations. Stronger causal evidence is needed to establish whether structural differences directly contribute to reasoning failures. 3.The experiments focus primarily on GSM8K, limiting the generalizability of the conclusions. Additional reasoning benchmarks would strengthen the evaluation. 4.The increasing performance gap between explicit and latent reasoning may be influenced by the fixed latent-state budget. Experiments with different latent budgets would help clarify this issue. 5.Although the paper identifies interesting structural patterns, it remains unclear whether these findings can be leveraged to improve reasoning performance.

Questions for authors

1.How sensitive are the observed even–odd positional patterns to the number of latent states? 2.Are the reported positional asymmetries consistent across different random seeds?

4 comments

We thank the reviewer for the constructive feedback and address the concerns below.

W2

To establish whether latent organization directly contributes to reasoning reliability, we conducted new inference-time interventions, training-time comparisons, and cross-checkpoint latent-state transplants. These provide direct evidence extending beyond the original aanalysis.

  • Disrupting learned organization affects reasoning outcomes. We rearranged each model's clean latent vectors. Each problem serves as its own control: the question and latent content remain unchanged; only their arrangement differs.
Correct answers retainedCODI GPT-2CODI Llama-1BCoconut GPT-2
Same vectors, uniformly random order0.850.880.58
Same vectors, members of each pair exchanged0.920.900.48
Norm-matched noise (reference)0.260.480.13
  • Pairs are consecutive positions (u0, u1), (u2, u3), and (u4, u5).

Rearranging identical latent vectors turns 12–15% of originally correct CODI predictions and 42% of Coconut predictions into errors. More specifically, disrupting pairings consistently predicts lower answer retention across all three models. Meanwhile, CODI's high retention under permutation relative to noise shows that computation relies primarily on latent content rather than fixed positions. Thus, latent computation is content-driven but relationally organized: the same information can remain useful across positions, yet disrupting its learned organization can undermine reasoning reliability.

  • Latent reliability improves when organization emerges. We retrained CODI GPT-2 without distillation (λ_D = 0 vs. 1). We then compared organizational strength, explicit accuracy, and conditional latent failure across checkpoints:
CODI GPT-2, with / without distillationep4ep6ep12ep20ep32
Layers with parity index above 0.8 in magnitude (of 12)0 / 011 / 05 / 010 / 09 / 0
Answers changed by erasing all latent inputs (% of solved)35 / 792 / 2769 / 3283 / 3357 / 23
Total causal load Σ_i |N_dir,i| (nats)0.45 / 0.171.60 / 0.191.89 / 0.271.44 / 0.201.64 / 0.13
Even–odd selectivity S of erasure effects0.03 / 0.060.55 / 0.100.27 / 0.040.22 / 0.100.34 / 0.07
Explicit accuracy (%)17.2 / 17.723.9 / 25.026.7 / 29.632.9 / 34.943.7 / 46.6
P(latent wrong ∣ explicit correct)0.72 / 0.670.54 / 0.650.40 / 0.610.33 / 0.540.25 / 0.44

At ep6, the distilled model develops strong positional specialization, with substantially greater causal load (1.60 vs. 0.19 nats), selectivity (0.55 vs. 0.10), and latent reliance (92% vs. 27%), alongside lower conditional failure (0.54 vs. 0.65). By ep32, conditional failure further diverges to 0.25 vs. 0.44, despite higher explicit accuracy without distillation (46.6% vs. 43.7%). Thus, the controlled ablation demonstrates systematic changes in organizational formation, the causal contribution of latent states, and latent reasoning reliability. The cross-checkpoint transplant below further establishes that the resulting reliability advantage is carried by the learned latent state.

  • The reliability advantage transfers with the learned latent states. We transplanted six latent inputs across checkpoints for each problem, holding recipient weights and the question fixed.
Weights ← latent states (same problem)Latent accuracy (%)P(latent wrong ∣ explicit correct)
ep6 ← ep6 (organized; original)14.20.54
ep6 ← ep4 (before formation)7.90.80
ep6 ← norm-matched noise5.70.87
ep4, last checkpoint before formation ← its own9.90.72
ep4 ← ep6 (organized)11.80.62
ep20 ← ep24 / ep24 ← ep20 (both organized)28.3 / 27.80.33 / 0.41

The transplant reveals a bidirectional transfer of reliability across organizational formation. With recipient weights fixed, substituting pre-formation ep4 states into ep6 raises conditional failure from 0.54 to 0.80, while supplying organized ep6 states to ep4 reduces failure from 0.72 to 0.62. In contrast, exchanges between already organized checkpoints change accuracy by at most 3.3 points. These results demonstrate that the reliability advantage follows the learned latent representations across training checkpoints, providing direct functional evidence linking organizational formation to reliable inference.

Together, these results provide direct evidence that latent organization contributes to reasoning reliability: disrupting learned organization induces failures; controlled ablation suppresses organizational formation, reduces causal reliance, and increases latent failures; and the reliability advantage transfers with learned latent state. The organization–reliability link is therefore supported by direct causal interventions and converging training-time evidence

W3

We extend our experiments to SVAMP, MultiArith, and GSM-Hard. The core organizational and reliability patterns replicate.

Training-time formation and reversals replicate (§3.1). Across 10 CODI GPT-2 checkpoints, erasure selectivity (even−odd)/(even+odd) follows the same dynamics:

CODI GPT-2 checkpointSVAMPMultiArithGSM-Hard
ep2−0.03−0.05−0.08
ep4+0.02−0.01−0.00
ep6+0.55+0.66+0.63
ep8−0.31−0.47−0.51
ep10−0.30−0.39−0.56
ep12+0.27+0.53+0.54
ep14+0.30+0.58+0.48
ep20+0.18+0.53+0.39
ep30+0.34+0.71+0.69
ep40+0.47+0.91+0.77

Erasure effects are evaluated on the problems solved by the final checkpoint.

Positional and causal specialization persist at convergence (§3.1, §4.1). At the final checkpoints:

GSM8K-trained modelLatent / explicit accuracy (%): SVAMP, MultiArith, GSM-HardLayers matching GSM8K parityLayer-profile correlation
CODI GPT-240.2/42.2, 91.8/94.7, 10.0/10.712/12, 11/12, 11/120.96, 0.92, 0.68
CODI Llama-1B59.4/57.5, 96.8/96.5, 13.7/14.311/11, 9/11, 11/110.94, 0.88, 0.90
Coconut GPT-235.3/40.0, 83.7/85.2, 7.6/9.410/10, 10/10, 10/100.90, 0.85, 0.94

Layers: GSM8K parity-index magnitude >0.5. Erasure profiles retain the GSM8K-dominant parity for both CODI models on all benchmarks, and Coconut on MultiArith/GSM-Hard (S=0.45/0.17). GSM-Hard CODI GPT-2 also retains its erasure-profile shape (u0–u5: 1.26/.13/.22/.08/.37/.04 nats; Fig. 1d).

The reliability gap retains its depth dependence (§4.3). On GSM-Hard, which spans multiple reasoning depths, the conditional latent failure rate again increases with solution depth:

Explicit solution depth (GSM-Hard)≤234
CODI GPT-20.110.260.53
CODI Llama-1B0.130.170.36
Coconut GPT-20.230.400.55

Thus, the formation dynamics, causal specialization, and depth-dependent reliability gap extend beyond GSM8K.

W4+Q1

We varied the latent budget during training and inference to address both concerns.

  • Even–odd specialization persists across trained budgets. We retrained CODI GPT-2 with K=3 (otherwise identical). At convergence, decoded roles and pinned erasure effects are:
Latent inputu0u1u2u3u4u5
K = 6, decoded rolevalueOp./conn.valueOp./conn.valueOp./conn.
K = 6, erasure effect (nats)1.180.130.360.150.620.04
K = 3, decoded roleoperatorvalueoperator–––
K = 3, erasure effect (nats)0.082.560.06–––

Across K=3 training:

K = 3 checkpointInputs decoded as valuesErasure effect u0 / u1 / u2 (nats)Parity index (Fig. 2c)
ep4none0.11 / 0.09 / 0.09+0.13
ep6u20.16 / 0.28 / 0.66+0.91
ep8u0, u20.54 / 0.35 / 0.55+0.99
ep12u10.20 / 1.49 / 0.17−0.99
ep20u10.18 / 2.78 / 0.10−0.98
ep30u10.28 / 2.38 / 0.17−0.99
ep40u10.08 / 2.56 / 0.06−1.00

(Value/Operator denote value/operator&connector-associated positions decoded with the frozen output head. Roles are assigned based on the avg. probability mass on: (i) the problem's intermediate results, defining value-associated positions, and (ii) arithmetic operators and expression delimiters, defining operation/connector-associated positions; erasure effects are pinned single-input effects (Fig. 1d))

The specialization remains strong, while its phase and causal importance shift together. At K = 3, the dominant value position moves from even positions to u1, whose erasure effect exceeds either neighbor by over 30×. During training, decoded roles, erasure effects, and parity indices reverse together between ep8 and ep12. The resulting organization remains stable through ep40, with u1 exhibiting the largest erasure effect in the final phase. Thus, changing the latent budget alters role assignment without weakening positional specialization or its causal function.

  • The depth-dependent reliability gap persists across budgets.
Explicit solution depth≤2345+All
K = 60.100.170.320.670.19
K = 30.160.390.540.710.33

Moreover, extending the inference rollout from K = 6 to 8 barely changes accuracy (CODI GPT-2: 43.6% → 43.2%; Llama-1B: 56.7% → 56.9%; Coconut GPT-2: 32.9% → 34.5%).

Together, these results establish that even–odd specialization remains robust across the tested latent budgets, with causal influence following the reassigned computational roles.

W1

The overlap with prior studies is limited to observing even–odd patterns [1] and inter-state dependencies in trained endpoint [2]. Both are cited in §1 and §6, and we will clarify the distinctions more explicitly in the updated version. To clarify our contributions and distinguish our work from prior studies, our analysis move beyond these findings in two directions: understanding how latent organization emerges and evolves during training, and providing a deeper characterization of its function at trained endpoints.

  • Even-odd pattern: Scratchpad Thinking [1] reports alternating storage and computation roles in a trained CODI model. We establish three properties beyond this observation. (1) The roles are emergent and reassigned during training. Our paper reveals training-time reversals of even–odd organization. New analyses decode intermediate values and operators/connectors across checkpoints, confirming that these reversals correspond to actual exchanges of computational roles. This reassignment is observed across models and different latent budgets. Thus, a trained checkpoint captures only one configuration of a dynamically evolving role structure. (2) Causal influence tracks the reassignment of computational roles rather than fixed positional identities. Training-time reversals provide a means to disentangle computational roles from fixed positional identities, which endpoint-only interventions cannot achieve. We find that causal influence consistently tracks the evolving role assignment. (3) Organizational emergence contributes to reliable latent reasoning. New training-time experiments show that strong positional organization and improved latent reliability emerge together, while removing distillation suppresses specialization, reduces causal reliance, and increases latent failures despite comparable or higher explicit accuracy. Crucially, cross-checkpoint transplants demonstrate that the reliability advantage transfers with the learned latent representations under fixed recipient weights (W2). These findings directly connect the formation of latent organization during training to its functional contribution at inference.

  • Inter-state dependencies: Beyond the stepwise influence and non-local routing reported by Li et al. [2], our framework provides a more detailed causal characterization at trained endpoints. We distinguish direct from recurrently mediated effects through pinned versus regenerated interventions (§4.2, Fig. 6), calibrate latent-state influence against norm-matched noise and donor controls (§4.1, Fig. 5c), and characterize how causal organization differs between successful and failed trajectories (§4.3, Fig. 7). New matched comparisons further confirm that these organizational differences persist after controlling for reasoning depth, intervention magnitude, and answer confidence. Thus, our analysis goes beyond identifying inter-state dependencies to revealing how causal influence is mediated and organized across latent states and reasoning outcomes.

We will make this comparison more explicit in the updated version. Together, these findings advance the understanding of latent reasoning beyond static patterns and dependencies, revealing how computational roles emerge, how causal importance follows their evolving assignment, and how the resulting organization relates to reasoning reliability.

[1] Scratchpad Thinking: Alternation Between Storage and Computation in Latent Reasoning Models, Mechanistic Interpretability Workshop at NeurIPS 2025

[2] Dynamics Within Latent Chain-of-Thought: An Empirical Study of Causal Structure, ICML 2026

W5

While the primary scope of this work is to provide mechanistic understanding on how latent computation forms, functions, and fail, our findings provide concrete, experimentally grounded insights for improving latent reasoning. Our analyses reveal strong inter-state dependencies and coordinated causal organization in successful reasoning, whereas failures exhibit weaker semantic ordering and more diffuse causal influence. Moreover, computational roles are dynamically reassigned, with causal importance following roles rather than fixed positions. This motivates stronger relational process supervision that promotes coherent information flow and coordination across latent states, beyond final-answer supervision.

Q2: seeds

The main findings consistently replicate across seeds. In addition to seed 11 (paper), we trained CODI GPT-2 with seed 42 (new run), keeping the data, training schedule, and analytical procedures identical, and repeated the representation, causal intervention, and reliability analyses.

  • Training-time formation and role reassignment. The following table tracks decoded value positions, single-input erasure effects (mean |N_dir,i|), and the parity index (Fig. 2c) across organized checkpoints of seed 42:
Seed 42 checkpointValue inputs (decoded)Erasure effect u0 / u1 / u2 / u3 / u4 / u5 (nats)Parity index (layers above 0.8)
ep6u0, u2, u40.42 / 0.10 / 0.15 / 0.05 / 0.14 / 0.04+0.97 (100%)
ep8u0, u2, u40.59 / 0.12 / 0.19 / 0.06 / 0.19 / 0.05+0.99 (100%)
ep12u1, u3, u50.10 / 0.38 / 0.09 / 0.10 / 0.06 / 0.11−0.98 (92%)
ep14u0, u2, u40.55 / 0.14 / 0.16 / 0.06 / 0.20 / 0.05+0.99 (92%)
ep16u0, u2, u40.52 / 0.14 / 0.16 / 0.13 / 0.27 / 0.08+0.98 (83%)
ep18u1, u3, u50.06 / 0.58 / 0.15 / 0.14 / 0.17 / 0.28−0.95 (92%)
ep20u1, u3, u50.10 / 0.51 / 0.16 / 0.21 / 0.16 / 0.31−0.99 (92%)
ep22u1, u3, u50.13 / 0.65 / 0.19 / 0.20 / 0.22 / 0.38−0.94 (75%)
ep24u0, u2, u40.74 / 0.18 / 0.24 / 0.18 / 0.60 / 0.07+0.98 (83%)
ep40u0, u2, u41.23 / 0.10 / 0.20 / 0.13 / 0.64 / 0.05+0.97 (100%)

Decoded role assignments, causal erasure profiles, and positional parity evolve in lockstep, reversing together at all four transitions in seed 42 (ep8→12, ep12→14, ep16→18, ep22→24). At every measured checkpoint, the strongest erasure effect falls on a value-associated position. Despite different reversal histories (4 versus 2), both seeds converge to virtually identical value-position profiles, demonstrating consistent role–causality alignment across training runs.

  • Inference-time function and reasoning reliability. The second seed also replicates the paper's causal mechanisms and outcome-conditioned findings:
CODI GPT-2, final checkpointSeed 11 (paper)Seed 42 (new)
Latent / explicit accuracy (1,175 problems)43.6% / 46.8%43.2% / 48.3%
P(latent wrong ∣ explicit correct) at depth ≤2 / 3 / 4 / 5+ (Fig. 7b)0.10 / 0.21 / 0.36 / 0.670.12 / 0.22 / 0.37 / 0.68
Direct effect, mean absolute N_dir,i, u0–u5 (downstream inputs pinned)1.18 / 0.13 / 0.36 / 0.15 / 0.62 / 0.041.23 / 0.10 / 0.20 / 0.13 / 0.64 / 0.05
Total effect, mean absolute N_tot,i, u0–u5 (downstream inputs regenerated; Fig. 6)0.11 / 0.70 / 0.87 / 1.17 / 1.29 / 0.040.07 / 0.69 / 0.70 / 0.80 / 1.15 / 0.05
Solved answers changed by donor latents / norm-matched noise (Fig. 5c)94% / 70%92% / 73%
Reversals of the value positions24
Value and operator positions exact complements15/1617/17
Erasure-profile sign flips: at reversals / at other transitions2/2 and 0/134/4 and 0/5
Total effect A, failed / solved×1.62×1.46
Matched selectivity gap S (vs gap cohort)0.37 [0.27, 0.46]0.33 [0.22, 0.43]

Both seeds reproduce the same functional organization.

Together, these results demonstrate that the main findings are reproducible across seeds.

We hope the above replies could address your main concerns. We will incorporate your feedback into the updated version and would greatly appreciate a reassessment.

Reviewer_z7zKOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates how latent reasoning representations evolve during training and function at inference time. The authors analyze CODI and Coconut using GPT-2 and Llama-3.2 models on GSM8K. Through representation analysis, probing, and causal interventions, they observe positional asymmetry and even–odd patterns that evolve during training. They also find differences between explicit and latent reasoning trajectories, while unsuccessful latent reasoning is associated with weaker semantic ordering and more diffuse causal influence.

Critical strengths
  • The paper examines the evolution of latent representations during training, offering an additional perspective beyond analyses of final checkpoints.

  • The combination of multiple analysis techniques provides useful observations about latent-state organization and its relationship with reasoning outcomes.

Critical weaknesses
  • A central motivation of the paper is to connect how latent reasoning structures emerge during training with how they contribute to reliable reasoning at inference time . However, the current experiments mainly characterize training-time changes and compare successful versus failed trajectories in §4.3, without establishing whether these structural changes actually lead to improved reasoning reliability; could the authors provide more direct evidence linking the two?

  • Regarding fig 5, previous work [1] has already demonstrated alternating storage and computation roles in CODI latent states. While the authors claim that positional asymmetry and parity reversal reflect a substantial reorganization of latent computation during training, it remains unclear what new mechanistic insight this provides beyond prior findings, whether the reversal corresponds to an actual change in computational roles (And if we extend the latent length, will this change?)

  • The comparison between successful and failed latent trajectories (lines 425–431) may be confounded by differences in problem difficulty and answer confidence, while the causal influence profiles in Figure 7(d) are normalized within each outcome group (Appendix C6). Could the authors provide difficulty or confidence matched comparisons and report the absolute intervention effects to verify that the more diffuse profiles reflect differences in causal organization rather than weaker intervention signals?

  • Lack of base model family coverage (Qwen), as well as larger model families (8b or 15b).

  • Duplicated ref: line 536-579 & 566-570

[1] Goyal et al. (2025). Scratchpad Thinking: Alternation Between Storage and Computation in Latent Reasoning Models. NeurIPS 2025 Mechanistic Interpretability Workshop.

Questions for authors

N/A

3 comments

Thank you for your constructive comments. We address your questions and concerns below.

W1

In response we conducted two new experiments: a checkpoint-wise training intervention showing that latent reliability improves when organization emerges, and a cross-checkpoint latent-state transplant testing whether this reliability advantage is carried by the learned latent representations. We measure reliability using the conditional failure rate of Fig. 7b, P(latent wrong | explicit correct), with the explicit pathway providing a same-weights reference for reasoning ability at each checkpoint.

  • Latent reliability improves when organization emerges. We retrained CODI GPT-2 without distillation (λ_D = 0 vs. 1). We then compared organizational strength, explicit accuracy, and conditional latent failure across checkpoints:
CODI GPT-2, with / without distillationep4ep6ep12ep20ep32
Layers with parity index above 0.8 in magnitude (of 12)0 / 011 / 05 / 010 / 09 / 0
Answers changed by erasing all latent inputs (% of solved)35 / 792 / 2769 / 3283 / 3357 / 23
Total causal load Σ_i |N_dir,i| (nats)0.45 / 0.171.60 / 0.191.89 / 0.271.44 / 0.201.64 / 0.13
Even–odd selectivity S of erasure effects0.03 / 0.060.55 / 0.100.27 / 0.040.22 / 0.100.34 / 0.07
Explicit accuracy (%)17.2 / 17.723.9 / 25.026.7 / 29.632.9 / 34.943.7 / 46.6
P(latent wrong ∣ explicit correct)0.72 / 0.670.54 / 0.650.40 / 0.610.33 / 0.540.25 / 0.44

At ep6, the distilled model develops strong positional specialization, with substantially greater causal load (1.60 vs. 0.19 nats), selectivity (0.55 vs. 0.10), and latent reliance (92% vs. 27%), alongside lower conditional failure (0.54 vs. 0.65). By ep32, conditional failure further diverges to 0.25 vs. 0.44, despite higher explicit accuracy without distillation (46.6% vs. 43.7%). Thus, the controlled ablation demonstrates systematic changes in organizational formation, the causal contribution of latent states, and latent reasoning reliability. The cross-checkpoint transplant below further establishes that the resulting reliability advantage is carried by the learned latent state.

  • The reliability advantage transfers with the learned latent states. We directly tested whether the reliability advantage is carried by the learned latent states by substituting the six latent inputs generated for the same problem across checkpoints, while keeping the recipient checkpoint's weights and question fixed.
Weights ← latent states (same problem)Latent accuracy (%)P(latent wrong ∣ explicit correct)
ep6 ← ep6 (organized; original)14.20.54
ep6 ← ep4 (before formation)7.90.80
ep6 ← norm-matched noise5.70.87
ep4, last checkpoint before formation ← its own9.90.72
ep4 ← ep6 (organized)11.80.62
ep20 ← ep24 / ep24 ← ep20 (both organized)28.3 / 27.80.33 / 0.41

The transplant reveals a bidirectional transfer of reliability across organizational formation. With recipient weights fixed, substituting pre-formation ep4 states into ep6 raises conditional failure from 0.54 to 0.80, while supplying organized ep6 states to ep4 reduces failure from 0.72 to 0.62. In contrast, exchanges between already organized checkpoints change accuracy by at most 3.3 points. These results demonstrate that the reliability advantage follows the learned latent representations across training checkpoints, providing direct functional evidence linking organizational formation to reliable inference.

Together, these experiments establish a direct formation–reliability connection: latent reliability improves as organization emerges, and the advantage transfers with the latent states across the formation boundary under fixed recipient weights. Combined with the direct inference-time interventions, this provides evidence that the learned organization contributes to reliable latent reasoning, extending substantially beyond the original outcome-conditioned correlations. We will incorporate these results into the updated version.

W2

Scratchpad Thinking, which we cited in §1 and §6, identifies alternating roles in a trained CODI model. We go further by showing that training-time reversals reassign computational role, causal influence follows the reassigned roles, and this organization persists across latent budgets.

  • Positional reversals reflect actual reassignment of computational roles. At each checkpoint, we apply the frozen output head to the hidden state generating each latent input (App. C.3). We classify positions by their mean decoded probability mass on (i) the problem's intermediate results (value, V) or (ii) operators and expression delimiters (operation/connector, C). For example, at the final CODI GPT-2 checkpoint (K = 6, ep40), decoding the states yields:

W2 (continue)

Latent stateRoleTop-1 token distributionRepresentative top-1 tokens
u0V67% intermediate values6, 20, 10
u1C73% operators, 12% delimiters*, -, /, >>
u2V60% intermediate values10, 8, 6
u3C31% operators, 13% delimiters-, >>, is, was
u4V50% intermediate values2, 8, 30
u5C29% operators, 23% delimiters>>, -, ., /

Crucially, training-time parity reversals correspond to exchanges between these roles. Value- and operation/connector-associated positions occupy complementary parities at 15/16 checkpoints, exchanging assignments at both observed reversals. These exchanges coincide with changes in activation, dimensionality, attention, and causal intervention profiles. Thus, the reversals reflect reorganization of computational content across latent positions.

  • Causal effect of latent states tracks the reassignment of computational roles. We independently measured each latent state's direct causal effect through single-input erasure with downstream inputs pinned (N_{dir,i}, App. C.1). The results closely track the decoded role assignments throughout training. The causal erasure profile follows the decoded value-associated positions at every measured checkpoint, switching at both role reversals and none of the other 13 transitions. This demonstrates that direct causal influence tracks the evolving computational roles rather than fixed positional identities.

Before the first reversal (ep6)

CODI GPT-2, ep6u0u1u2u3u4u5
Decoded roleVCVCVC
Intermediate-value mass0.330.000.200.000.180.01
Donor substitution: answer-margin shift (nats)2.700.070.900.090.800.09
Single-input erasure: drop in gold-answer log-prob (nats)0.950.120.120.070.110.05

After the first reversal (ep8)

CODI GPT-2, ep8u0u1u2u3u4u5
Decoded roleCVCVCV
Intermediate-value mass0.010.370.000.270.000.26
Donor substitution: answer-margin shift (nats)0.012.120.141.380.101.43
Single-input erasure: drop in gold-answer log-prob (nats)0.100.510.120.130.070.15
CODI GPT-2, final checkpoint (ep40)u0u1u2u3u4u5
Decoded roleVCVCVC
Donor substitution: answer-margin shift (nats)3.810.494.100.666.040.29
Single-input erasure: drop in gold-answer log-prob (nats)1.180.130.360.150.620.04
  • Role specialization persists across latent budgets. We vary the latent budget. Retraining CODI GPT-2 with K = 3 preserves the alternation. The decoded roles and direct erasure effects at the final checkpoint are:
Latent inputu0u1u2u3u4u5
K = 6, roleVCVCVC
K = 6, direct effect (nats)1.180.130.360.150.620.04
K = 3, roleCVC———
K = 3, direct effect (nats)0.082.560.06———

and over the training of the K = 3 model:

K = 3 checkpointRole = valueErasure effect u0 / u1 / u2 (nats)Parity index (Fig. 2c)
ep4none0.11 / 0.09 / 0.09+0.13
ep6u20.16 / 0.28 / 0.66+0.91
ep8u0, u20.54 / 0.35 / 0.55+0.99
ep12u10.20 / 1.49 / 0.17−0.99
ep20u10.18 / 2.78 / 0.10−0.98
ep30u10.28 / 2.38 / 0.17−0.99
ep40u10.08 / 2.56 / 0.06−1.00

The specialization remains pronounced, while role assignment and causal importance shift together. At K = 3, u1 becomes the dominant value-associated position, with an erasure effect over 30 times either neighboring input. Decoded roles, erasure profiles, and positional parity also reverse together during training, preserving the same role–causality relationship observed at K = 6.

Together, these results further establish the mechanistic insight of positional reversals: computational roles of latent states are dynamically reassigned during training, causal influence follows the evolving roles, and role specialization persists across different latent budgets. Our contribution extends beyond observing alternation in trained endpoint to explaining its formation, causal organization, and functional significance. We will clarify this distinction more explicitly from prior work in the updated version

W3

We conducted the requested analyses to test whether the more diffuse causal profiles in failed trajectories (Fig. 7d) arise from weaker intervention signals, problem difficulty, or answer confidence. The results consistently confirm that failures exhibit distinct causal organization, even after accounting for these factors.

  • Fig. 7d presents cohort-normalized profiles. We now report the unnormalized total causal effect, A = Σ_i |N_dir,i| (nats; App. C.1), comparing solved problems with the gap cohort (explicit correct, latent wrong):

    Mean total causal effect A (nats)SolvedGap cohortRatio
    CODI GPT-22.474.00×1.62
    CODI GPT-2, second seed2.353.43×1.46
    CODI Llama-1B0.641.87×2.94
    CODI Llama-3B1.262.17×1.73
    Coconut GPT-23.615.19×1.44

    Failed trajectories exhibit 1.4–2.9× larger absolute effects across all settings. Crucially, the increase is concentrated at positions with little influence in successful reasoning. For CODI GPT-2, the effects at u1/u3/u5 rise from 0.13/0.15/0.04 to 0.62/0.58/0.31 nats. Thus, the flatter profiles reflect a redistribution of causal influence across latent positions, not weaker intervention signals or an artifact of normalization.

  • Organizational differences persist after matching on difficulty, confidence, and effect magnitude. We quantify each problem's even–odd causal selectivity as S = (ā_d − ā_o)/(ā_d + ā_o), where ā_d and ā_o are the mean absolute effects on the dominant and opposite parities. Higher S indicates greater concentration of causal influence. We match failed problems to solved problems with the same explicit reasoning-depth bin and similar total causal effect A (five nearest neighbors; 0.25-SD caliper on log A). We report ΔS = S(solved) − S(failed), so positive values indicate more concentrated causal influence in successful reasoning:

ΔS, matched on reasoning depth and AExplicit correct, latent wrong (gap cohort)Both pathways wrong
CODI GPT-20.37 [0.27, 0.46]0.25 [0.17, 0.33]
CODI GPT-2, second seed0.33 [0.22, 0.43]0.27 [0.18, 0.34]
CODI Llama-1B0.46 [0.30, 0.57]0.29 [0.18, 0.41]
Coconut GPT-20.23 [0.13, 0.32]0.20 [0.13, 0.31]

Across all models and the second seed, failed trajectories retain significantly lower causal selectivity even at matched reasoning depth and intervention magnitude.

Since gold-answer confidence nearly separates the outcome groups (AUC 0.98–0.998), we instead match on the model's confidence in its own generated answer, measured without using the gold label. Jointly matching on depth, confidence, and A retains 72–86% of gap-cohort problems, and the difference persists:

ΔS, additionally matched on own-answer confidenceExplicit correct, latent wrong (gap cohort)Both pathways wrong
CODI GPT-20.21 [0.10, 0.36]0.16 [0.09, 0.24]
CODI Llama-1B0.28 [0.14, 0.48]0.18 [0.06, 0.38]
Coconut GPT-20.13 [0.06, 0.26]0.10 [0.02, 0.21]

All 95% confidence intervals exclude zero, confirming that the organizational difference persists under the requested controls.

Together, these results directly address both concerns: failed trajectories exhibit stronger absolute causal effects redistributed across latent positions, while their reduced causal selectivity persists after matching on reasoning depth, answer confidence, and intervention magnitude. This establishes a robust difference in causal organization between successful and failed latent reasoning.

W4

Our experiments already cover two representative backbone families (GPT-2 and Llama), two implicit reasoning paradigms (CODI and Coconut), and model scales from 124M to 3B parameters, encompassing the backbones used in the original CODI and Coconut studies. We agree that extending the analysis to additional families such as Qwen and larger 8B–15B models would further strengthen its generality. We will explicitly acknowledge this scope limitation and highlight broader model-family and scale coverage as future work.

W5

Thank you for pointing out the duplicate reference. We will consolidate the two entries into a single reference in the revised manuscript.

We hope the above replies address the core concerns, and we would welcome a reassessment.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear reviewers,

Thank you again for your thoughtful and constructive feedback.

We would like to briefly follow up on our rebuttal. We have carefully addressed the main concerns raised in the reviews through detailed point-by-point responses and uploaded a revised manuscript with all revisions highlighted in blue.

We would greatly appreciate any feedback on whether our responses adequately address your main concerns. If any questions remain, we would be happy to discuss them further. We hope our responses and revisions will be taken into consideration in your final assessment.

Thank you very much for your time and consideration!

AuthorsOpenReview ↗

Summary of Rebuttal

Dear Area Chairs and Reviewers,

We sincerely thank the reviewers for their thoughtful feedback, and for recognizing the value of our training-time perspectives, unified mechanistic framework, and carefully designed analyses.

To address the main concerns, we provided clarifications in our responses, conducted additional experiments, and uploaded a revised manuscript incorporating the new evidence and clarifications, with all changes highlighted in blue. We summarize the key points below:

  • Stronger controls and robustness (z7zK W3 & jhQF W4/Q1/Q2): Absolute intervention effects and difficulty-, confidence-, and effect-matched comparisons confirm the distinct causal organization of failed reasoning. Additional experiments establish consistency across random seeds and latent budgets, while showing that the depth-dependent reliability gap cannot be explained by the fixed latent budget alone. (Appendices D.2, D.5)

  • Broader evaluation and practical implications (jhQF W3/W5 & DGBy W1/Q1/Q2): Experiments on five additional benchmarks (ProsQA, CommonsenseQA, SVAMP, MultiArith, and GSM-Hard) reinforce our central findings that current implicit reasoning paradigms have yet to reliably learn to meaningfully utilize and coordinate latent states into a coherent, well-organized reasoning process. We also discuss implications for improving latent-state coordination and supervision. (Section 5, Appendix D.6)

  • Direct evidence linking organization to reliability (z7zK W1 & jhQF W2): New inference-time rearrangements, controlled distillation ablation, and cross-checkpoint latent-state transplants provide direct functional evidence that learned latent organization contributes to reasoning reliability, beyond the original analyses. (Appendices D.3–D.4)

  • Clarification of novelty (z7zK W2 & jhQF W1): Content-based decoding and independent causal interventions establish that training-time positional reversals reflect reassignment of computational roles, with causal influence following these roles rather than fixed positions. We also clarify how our formation–function analysis differs from prior endpoint-focused studies. (Appendices D.1–D.2)

We believe these additional experiments and analyses directly and substantially address the reviewers' core concerns. We respectfully hope that our detailed responses and revised manuscript will be taken into consideration in the final assessment.

Thank you again for your time and thoughtful consideration.

Best,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.