acceptodds
Under review as a conference paper at ICLR 2027

Latent Chain-of-Thought as Planning: Decoupling Reasoning from Verbalization

Abstract

Chain-of-Thought (CoT) empowers Large Language Models (LLMs) to tackle complex problems, but remains constrained by the computational cost and early token commitments in discrete reasoning traces. Recent latent reasoning approaches attempt to optimize efficiency by performing reasoning within continuous hidden states. However, many such methods optimize latent states end to end without a trained interface for intermediate textual readout, and several representative configurations use a pre-defined number of latent steps during inference. In this work, we introduce PLaT (Planning with Latent Thoughts), a framework that decouples latent planning from verbalization. The Planner deterministically evolves latent planning states, while an independent Decoder provides textual readouts when needed. Answer-aware textual stopping allows the latent rollout to use a problem-dependent number of groups rather than a pre-specified chain length. PLaT achieves competitive coverage at larger in several mathematical settings, with lower Pass@1: on Llama-1B GSM8K, it reaches 80.59% Pass@128 versus CODI's 72.37%. These results support PLaT as a candidate-generation interface supplying multiple textual readouts for downstream verification or reranking.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Model, Latent Reasoning, Latent Chain-of-Thought

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_bf6GOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes a novel method for latent reasoning (PLaT) that supports inspection of the latent states through a textual representation and a predicting a dynamic number of states dependent on the query. The method approaches latent reasoning through a planner module, generating latent states through full recurrence, paired with a decoder module, decoding chunks of latents states into text. The 2 modules are trained jointly by supervising the decoder to reconstruct chunks of tokens conditioned on the matching chunk of latent states. The method is evaluated on mathematical reasoning in 2 settings and is shown to improve the coverage in some cases (Pass@K) compared to other latent reasoning methods while maintaining lower latency compared to the CoT baseline.

Critical strengths
  1. The method provides a novel architecture supporting joint generation of latent planning states and conditioned text
  2. The paper includes evaluation on realistic mathematical reasoning tasks (MATH-500/AIME), which is sometimes lacking in latent reasoning papers, with an advantage compared to other latent reasoning methods on some of the tasks.
Critical weaknesses
  1. It is unclear what is the added value of the proposed method in terms of performance, efficiency or interpretability. PLaT does not seem to consistently outperform latent reasoning baselines in terms of performance (Pass@K) and leads to higher latency. In terms of interpretability via the decoder module, how reliable are the textual representation if coverage improves with repeated sampling conditioned on the same latent state?
  2. The presentation and writing of the paper is hard to follow, there are many instances in the text where undefined or unrelated terms/concepts are introduced making understanding difficult.
  3. A few important related work are missing, introducing either more modern baselines or methods with similar design. See for example [1],[2] which have similar design of latent states coupled with textual representation and [3][4] for additional baselines.
Questions for authors
  1. How are the CODI and COCONUT baselines trained on the OpenR1-Math-220k dataset, specifically how did you determine the number of latent states and are those ablated? the standard values are likely not tuned to handle CoT traces available in this dataset.
  2. Does training of PLaT rely on back-propagation through time during training? The related work in [2],[3],[4] propose methods to allow parallel training, are those applicable to PLaT?

[1] HYBRIDCOT: INTERLEAVING LATENT AND TEXTCHAIN-OF-THOUGHT FOR EFFICIENT REASONING, Shen et al [2] Latent Reasoning with Supervised Thinking States, Amos et al [3] Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chains Tan et al [4] Parallel Continuous Chain-of-Thought with Jacobi Iteration Wu et al

6 comments
AuthorsOpenReview ↗

Response to Reviewer bf6G (1/2)

Dear Reviewer bf6G, we appreciate your review. Our responses to your questions are as below.

Response to W1

  1. Performance and practical use. PLaT's benefit is stronger coverage at large k: on Llama-1B/GSM8K, Pass@128 is 80.59% versus CODI's 72.37% (Table 1; Section 4.2). On MATH-500, the corrected three-run results are 45.07±1.86% versus 38.60±0.53% (Table 4). With the same DeepSeek answer selector applied to 128 candidates per question, PLaT-16 achieves 44.2% final accuracy versus CODI's 37.4%, solving 34 additional questions (Table R1(a) below). The coverage advantage therefore translates into higher accuracy after actual answer selection.
  2. Efficiency. On the same hardware, GPT-2 greedy decoding takes 237.5 ms/question for PLaT, 240.0 for CODI, 349.6 for CoT and 100.6 for Coconut (Table 2; Section 4.3). We also evaluate time and coverage together for Llama-1B multi-sample inference (Table R1(b) below). At k=128, PLaT takes 7.060 s/question versus CODI's 6.186, while improving coverage from 68.75% to 76.04%. CoT-SFT reaches 87.50% at 14.852 s/question. PLaT offers higher coverage at a recorded time close to CODI's and less than half of CoT-SFT's, while retaining access to intermediate text.
  3. Reliability of textual readouts. The Decoder receives only the current latent group and its within-segment text, so problem-specific information must come through the latent states (Section 3.2). We check whether different readouts preserve valid reasoning at two levels. For intermediate segments, Section 4.4 and Figure 3 report averages of 4.648 distinct readouts and 4.347 locally valid readouts per nonempty state group. For complete solutions, our audit of all 1,319 GSM8K questions finds that increasing sampling from one to 32 candidates raises complete-correct-solution coverage from 282 to 478 questions, with 102 supporting at least two correct solution strategies (Table R1(c) below). Thus, repeated sampling exposes additional complete valid reasoning from the fixed Planner trajectory, giving the textual interface value beyond final-answer coverage alone.

Table R1(a). Common-selector results

MethodQuestions with a correct candidate / coverageFinal correct selections / accuracy
PLaT-16228 / 45.6%221 / 44.2%
PLaT-32175 / 35.0%172 / 34.4%
CODI191 / 38.2%187 / 37.4%
Coconut142 / 28.4%138 / 27.6%
CoT-SFT (explicit reference)387 / 77.4%366 / 73.2%

Table R1(b). Candidate-generation time and coverage

Methodk=1 (s/question)k=8 (s/question)k=32 (s/question)k=128 (s/question)Coverage at k=128 (mean ± sample SD)
PLaT-8 (lazy)0.3970.7521.9377.06076.04±3.61%
CODI0.3610.3941.5456.18668.75±0.00%
CoT-SFT0.7700.9383.77314.85287.50±3.13%

Table R1(c). PLaT full-chain correctness and solution-strategy diversity; counts and percentages of all 1,319 GSM8K questions

MethodAnswer coverage@32Valid-chain coverage@1Valid-chain coverage@32≥2 correct solution strategies@32
PLaT906 (68.69%)282 (21.38%)478 (36.24%)102 (7.73%)

Response to W2

We have clarified the terminology and notation in the marked revision PDF (Table R2 below). In particular, we describe the Planner's state update directly, distinguish tokens from text segments, and explain how Figure 2 corresponds to the equations. We also explain how the same Decoder reads intermediate reasoning and final answers, with its answer signal controlling stopping, and name the Section 4.4 analysis by the quantities measured. The revised passages are marked in red. If any other terms or passages remain unclear, we would appreciate your pointing them out so that we can clarify them further.

Table R2. Terminology and notation corrections in the marked revision

Original PDF locationWording or notationRevision
p. 3, line 128, Section 2.2“path collapse”Explain that selecting a discrete token commits the current trajectory to that textual continuation.
p. 3, lines 151–160, Section 3.1y_k denotes both a token and a text segmentUse u_t for tokens and y_k for segments.
p. 4, Figure 2s_1, s_2 versus S_k, s_(k,i)Define the index mapping and explain Prefill, CE and other figure labels in the caption.
p. 4, line 198, Section 3.2“on the latent manifold”State that the Planner generates each latent vector from the question and preceding latent vectors.
p. 5, lines 225–229, Section 3.3“mode-switching mechanisms”Explain that one Decoder reads both intermediate reasoning and final answers, with its answer signal controlling stopping (Section 3.4).
pp. 7–8, Section 4.4“Exploration vs. Exploitation”Rename the analysis “Readout Diversity and Local Validity across Questions.”
AuthorsOpenReview ↗

Response to Reviewer bf6G (2/2)

Response to W3

Thank you for pointing out these works. We will expand Section 2.2 and the experimental comparisons to address both aspects of your comment:

  1. Related designs [1,2]. HybridCoT interleaves latent states and key text tokens; Thinking States learns supervised thought tokens and feeds their embeddings into subsequent input processing. Both explicitly connect internal computation with text and deserve discussion alongside PLaT. We will add these references and explain the shared design motivation and the different role of text: PLaT's independent Decoder provides textual inspection without feeding decoded text back into planning.
  2. Additional baselines [3,4]. CoLaR learns probabilistic compressed embeddings, making it relevant to sampled-coverage comparisons; PCCoT updates latent positions in parallel, making it relevant to efficiency comparisons. CoLaR is already cited in the Introduction and Section 2.2; we will expand its discussion and add PCCoT. We will also include both baselines in the revised experiments under the same backbone and training-data setting.

Response to Q1

We retain the published latent counts, with six latent states for each baseline. This choice follows the original studies: Coconut reports that adding three continuous thoughts at a curriculum-stage transition causes a sharp training-loss increase and greater instability (Appendix C.1). CODI's GSM8K latent-count ablation peaks at six, with longer latent chains introducing optimization difficulties (Section 4.2, Figure 5). We therefore use these reference settings on OpenR1. We will add an OpenR1 latent-count ablation to the revised experiments, reporting accuracy and training stability; retraining these configurations cannot be completed within the rebuttal period.


Response to Q2

  1. Does PLaT use BPTT? Yes. Each Planner state depends on earlier generated states, with no gradient detachment between steps. A later segment's reconstruction loss therefore propagates through earlier state-generation steps to update the Planner's trainable parameters (Section 3.3).
  2. Are the parallel training methods in [2,3] directly applicable? No. Thinking States constructs training states from annotated thought tokens; CoLaR uses compressed text embeddings. Their parallel training relies on these known state inputs. PLaT learns its latent states through text reconstruction and feeds predicted states into subsequent Planner steps, so those known inputs are unavailable. Teacher forcing within the Decoder does not remove this Planner dependency.
  3. Is PCCoT [4] applicable? Yes. During training, the number of latent groups is known from the supervised text segmentation. Jacobi iteration can therefore update these positions in parallel within each iteration while retaining PLaT's Decoder reconstruction objective. Gradients would propagate through the iterations, so this approach parallelizes latent positions while preserving dependencies between successive iterations. Using fewer iterations than required to recover the sequential rollout would approximate the original Planner computation.

I thank the reviewer for the thoughtful and detailed response which partially addresses my concerns, I am detailing the main remaining items below.

My main concern is on how to interpret the relative advantage of the method compared to the baselines. If I understand the authors correctly, the claim is that PLaT shows gains in pass@k as k increases and specifically at larger values of k. this still leaves the following concerns unresolved:

  1. in Table 1 considering it seems the advantage of PLaT is consistent only at k=128 but no larger values of k are considered - considering either larger values and/or plotting the curve of pass@k against k could strengthen the results.
  2. in Table 4 it seems that at k=128 CODI performs better on AIME 2024, although PLaT uses different hyperparameters than in table 1.

Can you please elaborate on your response about interpretability (possibly unclear due to poor phrasing of the question on my end). My question is: if the results suggests that the pass@K performance of PLaT improves with increased k, i.e. repeated sampling of different text sequences conditioned on the same latent improves coverage and therefore leads to meaningfully different outputs, how can one:

  1. decide which text sequence represents the latent most faithfully allowing to interpret it
  2. infer which part of the reasoning is encoded in the latent representation and which is executed by the decoder module

due to these remaining concerns I am keeping my score.

Reviewer_yPRhOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduce Planning with Latent Thoughts, which decouples latent reasoning from verbalization, ensuring diverse and interpretable latent tokens and higher pass@128 accuracy on math tasks.

Critical strengths
  1. Latent tokens are well known to improve efficiency, while keeping that, this paper also improve the interpretability by end2end training the planner and decoder.
  2. It shows higher diversity when decode the latent token compared with CoT-SFT, and also achieve higher pass@128 even if lower pass@1 on math tasks.
Critical weaknesses
  1. The positioning relative to related latent-reasoning approaches is underdeveloped. Relevant directions include probabilistic latent trajectories and decodable latent representations (e.g., NF-CoT [1] and LaDiR [2]). A clearer comparison would help establish the specific novelty of combining a deterministic Planner, independent textual readout, and answer-aware stopping.
  2. The paper does not deeply analyze failure sources (e.g., cross-segment inconsistency, calculation errors, stopping errors), so the gap between low Pass@1 and higher Pass@k remains unexplained. Text-level diversity and locally valid segments also do not imply distinct strategies or correct full reasoning chains.
  3. Accuracy is reported with full decoding while efficiency uses lazy readout. Moreover, timing is limited to GPT-2 with greedy decoding and does not cover the multi-sample setting used to demonstrate PLaT’s Pass@k advantages.

[1] Tu, Guancheng, et al. "Latent Reasoning with Normalizing Flows." arXiv preprint arXiv:2606.06447 (2026).

[2] Kang, Haoqiang, et al. "Ladir: Latent diffusion enhances llms for text reasoning." International Conference on Learning Representations. Vol. 2026. 2026.

Questions for authors

See weakness

Reviewer_iXgGOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes PLaT, which separates latent reasoning from text generation. A planner produces a sequence of latent states, and a separate decoder turns these states into short pieces of reasoning text. The two are trained together to reconstruct reasoning traces, but the generated text is not fed back into the planner. The model uses the decoded answer text to decide when to stop.

The authors evaluate on GSM8K and related arithmetic datasets, with additional experiments on MATH-500 and AIME. The main finding is that PLaT can outperform some latent baselines when many answers are sampled, although its single-answer accuracy is lower. Explicit CoT remains stronger in the main comparison. The paper also includes ablations, an analysis of intermediate readouts, and a limited latency comparison.

Critical strengths
  1. The architecture provides a straightforward way to read intermediate latent states without feeding the decoded text back into the planner (section 3). This is a useful feature for studying what these states contain.
  2. The gains over latent baselines are meaningful in some settings. On Llama-1B GSM8K, PLaT reaches 80.59% Pass@128 versus CODI’s 72.37% (Table 1). This supports its potential use for generating candidate answers.
Critical weaknesses
  1. The practical benefit is still unclear. On the same GSM8K comparison, Pass@1 drops from CODI’s 54.55% to 36.20%, and CoT-SFT is stronger at both Pass@1 and Pass@128 throughout Table 1. There is no evaluation showing that an answer selector can turn the coverage gains into better final accuracy. The timing results do not resolve this concern: they measure lazy, greedy GPT-2 inference, while the accuracy results use full decoding (Section 4.3; Appendix B.1).
  2. Table 4 states that results average three runs over 30 AIME questions, but includes means such as 1.67% Pass@1. Under the stated metric, these averages should occur in increments of approximately 1.11 percentage points. The number of runs and the reported uncertainty therefore need clarification.
Questions for authors
  1. Clarification: How many runs were used for each Table 4 entry? Could you provide the per-run solved-question counts and explain the AIME averages?
  2. Additional evaluation: With a common answer selector and matched total inference cost, does PLaT improve final-answer accuracy? Accuracy and timing should use the same decoding setting.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Author Responses

We thank the Area Chair and reviewers for their careful assessment and constructive feedback. We summarize the main clarifications and additional evidence provided during the discussion.

1. Clarified contribution and positioning. PLaT jointly trains a deterministic latent Planner and an independent Decoder through reasoning reconstruction. The Decoder provides intermediate textual readouts and determines stopping through its answer readout, while decoded text never feeds back into the Planner. Sampling occurs in the Decoder. Our responses clarify this combination of latent planning, independent verbalization, and answer-aware stopping relative to related approaches.

2. Coverage gains translate into improved selected-answer accuracy. On MATH-500, the corrected three-run Pass@128 is 45.07±1.86% for PLaT-16, compared with 38.60±0.53% for CODI. In the additional selection experiment, applying the same DeepSeek-V4.1-Flash selector to 128 candidates per question yields 44.2% final accuracy for PLaT-16 versus 37.4% for CODI, solving 34 additional questions. This provides evidence that the coverage gain can benefit actual answer selection. Lower Pass@1 and the remaining gap to explicit CoT are acknowledged limitations.

3. Additional analysis addresses complete-chain correctness and strategy diversity. Our audit across all 1,319 GSM8K questions finds that increasing sampling from one to 32 candidates raises the number of questions with a complete correct solution from 282 to 478. Moreover, 102 questions admit at least two distinct strategies yielding complete correct solutions. We also analyzed visible failure sources, with arithmetic/algebra and modeling/deduction errors being the most frequent categories. These analyses extend the evidence beyond textual diversity and locally valid segments.

4. Multi-sample timing clarifies the coverage–latency trade-off. In the additional Llama-1B timing evaluation at k=128k=128, PLaT achieves 76.04% coverage at 7.060 s/question, compared with CODI’s **68.75% at 6.186 s/question and CoT-SFT’s **87.50% at 14.852 s/question. These measurements concern candidate generation. The selector comparison uses equal candidate counts; final selected accuracy at matched total inference cost remains to be established. Separately, paired checks on 1,319 greedy-decoded questions found identical final answers and stopping positions under full and lazy readout.

5. Reporting corrections and revision commitments are explicit. We corrected Table 4 in the rebuttal and supplied per-run solved-question counts, clarifying that the reported uncertainty is the sample standard deviation across three sampling runs of the same trained model. The marked revision clarifies terminology and notation, and our responses explain the Planner’s use of backpropagation through time. Expanded related-work discussion, additional CoLaR/PCCoT comparisons, and an OpenR1 latent-count ablation are planned for the revision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.