acceptodds
Under review as a conference paper at ICLR 2027

Latent Reasoning with Normalizing Flows

Abstract

Large language models often improve reasoning by generating explicit chain-of-thought (CoT), demonstrating the importance of intermediate computation. However, textual CoT forces this computation through a discrete, serial, and communication-oriented token stream: each reasoning step must be verbalized before the model can proceed, even when the underlying update is semantic, uncertain, or only partially formed. Latent reasoning offers a higher-bandwidth alternative by performing intermediate computation in compact continuous states before committing to text. Yet existing latent-reasoning methods often sacrifice key advantages that make CoT effective in autoregressive language models, including native left-to-right generation, probabilistic sampling, compatibility with KV-cache decoding, and tractable likelihood estimation. We propose NF-CoT, a latent reasoning framework that preserves these advantages by modeling continuous thoughts with normalizing flows. NF-CoT instantiates a TarFlow-style normalizing flow inside the LLM backbone, defining a tractable probability model over compact continuous thoughts distilled from explicit CoT. Continuous-thought positions are generated by an NF head, while text positions are generated by the standard LM head within the same causal stream. This design provides exact likelihoods for latent thoughts, enables KV-cache-compatible probabilistic left-to-right decoding, and supports direct policy-gradient optimization in the latent reasoning space. On code-generation benchmarks, NF-CoT improves pass rates over explicit-CoT and prior latent-reasoning baselines while substantially reducing intermediate-reasoning cost, and a complementary study on six math benchmarks shows that NF-CoT transfers beyond code.

Primary area:
generative models
Keywords:
Generative Models, Representation Learning, LLM Reasoning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_LNCsOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3(+1)
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper propose NF-CoT, a latent CoT method that models continuous reasoning with an autoregressive normalizing flow inside the LLM. This gives latent thoughts the same interface as text tokens, allow the latent thoughts embeddings to be sampled left-to-right, scored with an exact likelihood, and can be optimized with RL. The architecture first compress the textual CoTs into fixed-length continuous latents by a frozen VAE, and shallow invertible flow blocks map these latents into a space that is easier to model. The LLM backbone then acts as a deep autoregressive flow, generating latent thoughts through a Gaussian NF head and answer text through the standard LM head within a single causal stream.

Critical strengths

Using jointly trained invertible flow blocks to learn a target space that suits autoregressive Gaussian modeling, which provides a principled answer to how a continuous latent CoT can be fit by exact maximum likelihood inside a single LLM causal stream.

Critical weaknesses
  1. The paper claims that it develops "a likelihood-based formulation of continuous CoT that preserves the sampling, scoring, and decoding interface of explicit CoT.", so that it allows to perform sft and rl on the latent thinking tokens. However, some existing work already did this, could you justify the difference and compare with them? (E.g. Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chains NeurIPS 2025)

  2. The necessity of NF module is not empirically justified. It seems that the purpose for the flow module is to produce an embedding space that is easier for the LLMs to learn. But what if the LLM directly models the VAE latents autoregressively (i.e., removing the flow module, but using an NLL objective instead of the L2 objective used in LaVAE)?

Questions for authors
  1. How does your method compares to the deterministic hidden states like Coconut and its variations?
  2. Why LaVAE performs so badly (Table 1)?
  3. The gain of NF-CoT over Standard SFT (69.1 vs. 57.8 on average) is surprisingly large, given that the latent thoughts are distilled from the same explicit CoT data. Could the authors provide the training and evaluation details of the Standard SFT baseline, and justify where the gain comes from?
6 comments
AuthorsOpenReview ↗

Clarifications on Methodological Novelty, Baseline Comparisons, and Additional Ablations

We sincerely thank the reviewer for the thoughtful questions. We clarify the distinctions from prior methods and provide additional controlled ablations below.

W1: Differences from Prior Likelihood-Based Latent Reasoning Methods

We thank the reviewer for highlighting this relevant work. Our key distinctions lie in more expressive latent distribution modeling through normalizing flows and semantic reasoning representations learned through VAE encoding.

Distribution modeling. CoLaR directly models each latent step using a Gaussian distribution whose mean and variance are predicted autoregressively by the LLM. However, latent CoT distributions can be complex and potentially multimodal, since one problem may admit multiple reasoning strategies. NF-CoT instead learns an invertible nonlinear transformation using normalizing flows, allowing complex distributions in the original latent space to be modeled through simpler Gaussians in a transformed space, while preserving exact likelihood computation and autoregressive sampling.

Latent representations. CoLaR compresses local groups of token embeddings through scaled summation, whereas NF-CoT uses a VAE to learn compact semantic representations of entire reasoning traces. Thus, NF-CoT models learned CoT representations rather than directly compressed token embeddings.

W2: Necessity of the Normalizing Flow Module

We thank the reviewer for this suggestion. We conducted a controlled ablation removing the shallow flow blocks while retaining autoregressive Gaussian modeling of the same VAE latents with the same NLL objective. All other training and evaluation settings remain unchanged.

Latent density modeling. We compute exact NLL on 128 real VAE latent samples and compare it with Gaussian noise matched to their per-dimension mean and variance.

Metric (bits/dim)NF-CoTw/o Shallow
NLL on real latents ↓0.5730.864
NLL gap (Gaussian − Real) ↑2.931.62

Removing the shallow blocks increases NLL by approximately 50%, with worse likelihood on all 128 samples. NF-CoT also better distinguishes real latents from moment-matched Gaussian noise (2.93 vs. 1.62 bits/dim), indicating that the flow captures complex, non-Gaussian structure in the latent CoT distribution.

Downstream reasoning performance. Under the same evaluation protocol, we compare against the original NF-CoT results:

BenchmarkNF-CoTw/o Shallow
HumanEval85.876.9
HumanEval+78.370.8
MBPP83.978.7
MBPP+72.167.9

Performance drops substantially on all four benchmarks without the shallow blocks. Autoregressive Gaussian NLL modeling alone is therefore insufficient to match NF-CoT; the flow improves both density modeling and downstream reasoning performance. We will include this ablation in the revised paper.

Q1: Comparison with Deterministic Latent Reasoning (e.g., Coconut)

We thank the reviewer for raising this question. Unlike methods that propagate deterministic latent states without an explicit latent likelihood, NF-CoT models a probability distribution over latent trajectories and samples from it autoregressively. This has two advantages:

  1. Avoiding mode averaging. A problem may admit multiple valid reasoning trajectories. L2 regression tends to predict their conditional mean, which may fall between distinct reasoning modes and fail to represent a coherent solution path. Probabilistic modeling instead allows NF-CoT to capture and sample different reasoning trajectories rather than collapsing them into a single averaged prediction.
  2. Enabling latent-space RL. Exact likelihoods of sampled latent trajectories allow direct policy-gradient optimization of latent generation using downstream rewards, rather than optimizing only answer tokens.

Controlled deterministic ablation. We implement a deterministic counterpart with the same backbone, training data, latent layout, and training steps. It directly predicts each continuous latent token using L2 loss instead of a Gaussian distribution; at inference, each prediction is fed back autoregressively as the next input embedding, without sampling.

Continuous thoughtsHumanEval / HE+MBPP / MBPP+
Deterministic (L2 regression)75.3 / 69.178.3 / 65.4
Probabilistic (NF-CoT)85.8 / 78.383.9 / 72.1

NF-CoT consistently outperforms deterministic regression, supporting the benefit of probabilistic latent modeling. We will include this ablation in the revised paper.

Q2: Why Does LaVAE Perform Poorly?

We thank the reviewer for this question. LaVAE's performance highlights two limitations of directly modeling complex latent CoT distributions.

First, LaVAE relies on a simple Gaussian assumption for the original latent CoT distribution, using an L2 regression objective to predict VAE latents. However, latent CoT distributions can be highly complex and multimodal, as a single problem may admit multiple distinct reasoning strategies. A simple Gaussian may fail to capture such distributions, leading to mode averaging and less meaningful latent representations. In contrast, NF-CoT uses normalizing flows to learn an invertible transformation, allowing more complex distributions to be modeled in a transformed space with a simpler Gaussian density.

Second, LaVAE generates the entire latent block in a single step, without explicitly modeling dependencies between latent tokens. In contrast, NF-CoT autoregressively factorizes the joint distribution into 64 conditional distributions, each conditioned on previously generated latents. This enables more expressive joint distribution modeling, capturing complex dependencies between reasoning steps and allowing the reasoning trajectory to evolve progressively.

Q3: Standard SFT Baseline and the Source of NF-CoT's Gains

We thank the reviewer for this insightful question. Although NF-CoT and Standard SFT learn from the same explicit CoT data, we attribute NF-CoT's performance gains to two key advantages of continuous latent reasoning: smoother semantic transitions and the ability to represent diverse reasoning possibilities in continuous latent space.

Standard SFT supervises both explicit CoT traces and final answers, not answers alone, using next-token cross-entropy. Its configuration is:

SettingStandard SFT
Backbone / DataQwen3-8B-Base / 733K Ling-Coder Python samples
TrainingFull fine-tuning, 2 epochs (22,902 steps), global batch 64
OptimizerAdamW, β=(0.9, 0.95), weight decay 0.01
Implementation8×B200, bf16, FSDP, no packing
EvaluationvLLM, temperature 0.6, top-p 1.0, 16 samples/problem, average pass@1

These settings match NF-CoT's training setup and evaluation protocol.

Why can latent reasoning outperform explicit CoT? Explicit CoT serializes reasoning into discrete language tokens, progressively committing to particular textual expressions. Continuous latents can encode richer semantic information and preserve multiple plausible reasoning paths, allowing more flexible and potentially parallel reasoning. Appendix A provides qualitative evidence: different latent samples for the same problem decode into distinct strategies (e.g., dynamic programming, tabulation, and memoization), each producing correct solutions.

Continuous latents also allow smoother semantic transitions. Rather than repeatedly expressing intermediate steps in discrete natural-language tokens, the model operates directly on continuous representations, enabling gradual and flexible updates of reasoning states, including intermediate computations difficult to verbalize. By reducing the constraints of discrete tokenization, NF-CoT can support more natural and effective latent-space reasoning.

AuthorsOpenReview ↗

Follow-up on Our Response

Dear Reviewer,

Thank you again for your positive assessment of our work and your constructive feedback.

We hope our responses have satisfactorily addressed your questions and suggestions. Please feel free to let us know if any further clarification would be helpful.

Best regards,

The Authors

Reviewer_CtU2OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes NF-CoT, which models continuous reasoning states with an autoregressive normalizing flow within an LLM. The model learns from compressed explicit CoT traces and generates latent thoughts before producing an answer. The tractable latent likelihood also enables policy-gradient training. Experiments show improved code-generation performance over SFT and latent-reasoning baselines, with faster inference than latent diffusion, alongside additional results on math reasoning.

Critical strengths
  1. I like the formulation of latent reasoning as an autoregressive probability model. It provides a coherent way to sample continuous thoughts, compute their likelihoods, and optimize them with RL within the same backbone. This is a useful contribution beyond simply compressing CoT into fewer tokens.

  2. The code-generation results are convincing, with substantial gains over SFT and LaDiR using the same backbone and training data, alongside measured speedups over LaDiR. The cross-question latent intervention also strengthens the evidence: performance drops when the model receives another problem’s latents, suggesting that the latent content is task-specific.

Critical weaknesses

Overall, I like the paper's idea of combining flow model and the latent reasoning. My concerns are mainly about ablations:

  1. While the overall formulation is appealing, the ablations do not fully isolate the contribution of the flow transformation. A matched autoregressive Gaussian baseline over the same VAE latents would help establish whether the gains come from the learned transformation or from the broader latent-supervision and training setup.

  2. Might miss one ablation: an answer-only RL control would clarify whether updating the latent distribution contributes to the improvement.

Questions for authors

See weakness for the 2 ablation requests.

5 comments
AuthorsOpenReview ↗

Additional Ablations on Flow Transformation and Latent Policy Optimization

We sincerely thank the reviewer for the valuable suggestions regarding ablation studies. We provide additional experimental evidence to address both concerns.

W1: Contribution of the Flow Transformation

We thank the reviewer for this valuable suggestion. To isolate the contribution of the flow transformation, we conducted an additional ablation that removes the shallow flow blocks while retaining the same VAE latents, autoregressive Gaussian modeling, and NLL objective. All other training and evaluation settings remain identical to the original NF-CoT setup, allowing us to examine whether the learned flow transformation provides benefits beyond the latent-supervision and training framework.

First, we evaluate the quality of latent density modeling. We compute the exact NLL of 128 real VAE latent samples and compare it with Gaussian noise matched to the per-dimension mean and variance of the real latents.

Metric (bits/dim)NF-CoTw/o Shallow
NLL on real latents ↓0.5730.864
NLL gap (Gaussian − Real) ↑2.931.62

Removing the shallow blocks increases NLL by approximately 50%, with worse likelihood on all 128 samples. Moreover, NF-CoT distinguishes real latents from moment-matched Gaussian noise more effectively, indicating that the learned flow transformation helps capture complex, non-Gaussian structures in the latent CoT distribution.

Second, we evaluate downstream reasoning performance using the same evaluation protocol.

BenchmarkNF-CoTw/o Shallow
HumanEval85.876.9
HumanEval+78.370.8
MBPP83.978.7
MBPP+72.167.9

Removing the shallow blocks leads to substantial performance degradation across all four benchmarks. These results demonstrate that the gains cannot be attributed solely to autoregressive latent supervision or the broader training framework. The learned flow transformation contributes to both more accurate latent density modeling and stronger downstream reasoning performance. We will include this ablation in the revised paper.

W2: Contribution of Latent Policy Optimization in RL

We thank the reviewer for this valuable suggestion. We conducted an additional answer-only RL ablation, removing the latent GRPO loss and optimizing only the answer-token GRPO objective. All other training and evaluation settings are identical to our original RL experiment.

BenchmarkAnswer-only RLJoint RL (Ours)
MBPP84.385.4
MBPP+72.373.3
HumanEval85.986.7
HumanEval+79.480.2
LCB v624.226.1
Average69.270.3

Joint RL consistently outperforms answer-only RL across all five benchmarks. These results demonstrate that explicitly optimizing the latent reasoning policy with reward signals is important for achieving the observed RL gains.

AuthorsOpenReview ↗

Follow-up on Our Response

Dear Reviewer,

Thank you again for your positive assessment of our work and your constructive feedback.

We hope our responses have satisfactorily addressed your questions and suggestions. Please feel free to let us know if any further clarification would be helpful.

Best regards,

The Authors

AuthorsOpenReview ↗

Follow-up on Rebuttal and Discussion

Dear Reviewer,

As the discussion period is approaching its end, we would like to kindly follow up on our rebuttal. If any questions or concerns remain, we would be happy to provide further clarification.

Best regards,
The Authors

Reviewer_aAjsOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes NF-CoT, a latent reasoning framework that models continuous chain-of-thought with normalizing flows inside an LLM’s causal stream. Explicit CoT is first encoded into continuous targets by a frozen VAE; shallow flow blocks reparameterize those targets into an LLM-facing thought space, and an NF head then defines an autoregressive Gaussian density over continuous thoughts while a standard LM head generates answer tokens in the same backbone. The resulting exact likelihood supports supervised training and policy-gradient refinement directly in the continuous reasoning space, with left-to-right sampling at inference. On five code-generation benchmarks with a Qwen3-8B-Base backbone, unified NF-CoT improves average pass@1 from 55.8 to 69.1 over the base model; a complementary math study on six benchmarks shows transfer beyond code.

Critical strengths
  1. The modeling goal is clear and well motivated. Placing a tractable autoregressive density over continuous thoughts restores sampling, scoring, and KV-cache-compatible left-to-right decoding that Coconut-style deterministic latents and diffusion-based latents largely give up, and the unified NF/LM-head design makes that interface concrete.
  2. The main code results are strong on aggregate. Unified NF-CoT raises average pass@1 by 13.3 points over the base model and by 7.5 points over LaDiR under the same backbone (Table 1), while Table 3 shows faster latent generation and higher training throughput than LaDiR’s iterative denoising.
  3. The analyses are useful. Inference-time interventions show that zeroing or replacing the latent prefix hurts pass@1 in a content-sensitive way, the LaVAE control collapses badly, and the dual-path versus unified comparison helps isolate the benefit of scoring and decoding from the same continuous prefix.
Critical weaknesses

1. The practical benefit of the exact likelihood is not clearly demonstrated

The main motivation of NF-CoT is that a tractable likelihood over continuous thoughts enables sampling, scoring, and policy-gradient refinement. However, the likelihood appears to help only to a limited extent in practice: selecting samples by NF density is close to random selection (Table 9), and the likelihood-based RL stage brings only a modest gain (Table 1). It is therefore unclear how much the exact likelihood, which distinguishes NF-CoT from prior latent reasoning methods, contributes to the final performance.

2. The evidence that the latents perform reasoning is limited

The paper frames the continuous thoughts as latent reasoning, but the intervention results point to a weaker role. Removing the latent prefix hurts pass@1 sharply while leaving pass@10 nearly unchanged, which the authors interpret as improved single-shot reliability rather than new capability (Table 4). In addition, a pause-token model without latent content comes close to NF-CoT on MBPP and MBPP+. The current evidence therefore seems more consistent with the latents acting as a helpful conditioning prefix than with them carrying the reasoning process.

Questions for authors
  1. Since the latent density is not used for scoring, could you explain where the exact likelihood brings a clear practical benefit beyond the modest RL gain?
  2. The authors note that the latent prefix mainly improves single-shot reliability rather than adding new capability. Could you explain how this observation fits the claim that NF-CoT performs reasoning in the continuous latent space?
  3. Could you explain why the gains are much larger on MBPP and MBPP+ than on HumanEval, where NF-CoT is on par with TaH+ and below LaDiR?
4 comments
AuthorsOpenReview ↗

Clarifications on Exact Likelihood, Latent Reasoning, and Benchmark Performance

We sincerely thank the reviewer for the thoughtful feedback and insightful questions. We provide further clarification and additional experimental evidence below.

W1/Q1: Practical Benefit of Exact Likelihood

We thank the reviewer for this important question. The practical value of exact likelihood lies primarily in enabling probabilistic latent modeling and reward-driven policy optimization, rather than directly ranking solutions by correctness.

Regarding likelihood-based selection (Table 9), the latent density learned during SFT reflects how likely a reasoning trajectory is under the learned CoT distribution, rather than its probability of producing a correct answer. Since SFT likelihood is not directly aligned with task rewards, higher density does not necessarily imply higher correctness. Thus, the limited effectiveness of density-based reranking is expected and does not diminish the value of exact likelihood for optimization.

Regarding the modest RL gain in Table 1, our original experiment was a small-scale proof of concept, involving only 150 training steps on a subset of the RL dataset. To further demonstrate the practical benefit of exact likelihood, we extended RL training to 2 epochs (1,590 steps) on the full dataset of approximately 50K samples, obtaining substantially stronger results:

BenchmarkNF-CoT (SFT)NF-CoT (+ Full RL)Δ
MBPP83.989.3+5.4
MBPP+72.176.5+4.4
HumanEval85.891.2+5.4
HumanEval+78.384.6+6.3
LCB v625.432.8+7.4
Average69.174.9+5.8

These results demonstrate the practical value of exact likelihood for latent reasoning optimization. By providing tractable log-probabilities for sampled latent trajectories, NF-CoT enables policy-gradient updates directly in the continuous reasoning space. With sufficient RL training, this translates into substantial and consistent improvements across all five benchmarks. We will incorporate these results into the paper.

W2/Q2: Evidence of Reasoning in Continuous Latents

We thank the reviewer for raising this question. We provide two complementary analyses to clarify how latent reasoning influences solution generation.

(1) Qualitative evidence: Decoded algorithmic strategies. Appendix A demonstrates that the latents encode meaningful reasoning information. For the same fibfib problem, different latent samples decode into distinct algorithmic strategies, including rolling-state dynamic programming, tabulation, and recursive memoization, with the generated programs following the corresponding strategies. This suggests that latent content captures task-specific reasoning choices rather than merely serving as a generic conditioning prefix.

(2) Quantitative evidence: Fixed vs. independently sampled latents. We further investigate whether sampling different latent trajectories improves solution diversity. Under the same 16-answer budget, we compare (i) sampling one fixed latent and decoding 16 answers from it, and (ii) independently sampling 16 latents and decoding one answer from each. We report pass@16:

BenchmarkFixed latent × 16 answers16 latents × 1 answer
HumanEval90.995.1
HumanEval+85.491.5
MBPP90.593.1
MBPP+79.183.6

Independently sampled latents consistently improve pass@16, showing that exploring different latent trajectories provides better solution coverage than repeatedly decoding from a fixed latent. Together with the decoded algorithmic strategies in Appendix A, these results support that NF-CoT latents carry task-specific reasoning information that meaningfully guides solution generation, rather than merely acting as a generic conditioning prefix.

Q3: Why Are Gains Larger on MBPP Than HumanEval?

We thank the reviewer for this insightful question. NF-CoT achieves statistically significant improvements over the base model on all four benchmarks, despite differences in gain magnitude.

We conducted two-sided significance tests with cluster-robust standard errors based on per-problem pass@1 across 16 samples:

BenchmarkGain (95% CI)p-value
MBPP+23.4 [+18.9, +27.9]4.4e-24
MBPP++18.3 [+12.8, +23.8]7.7e-11
HumanEval+7.6 [+1.5, +13.7]0.014
HumanEval++9.7 [+2.4, +17.0]0.009

The larger gains on MBPP can be explained in part by benchmark difficulty. We used an independent Qwen3-4B-Base model to categorize problem difficulty, avoiding bias from grouping by the evaluated model's own performance. Defining hard problems as those with pass rates below 1/3, 18.5% of MBPP problems are hard, compared to only 11.6% for HumanEval. Moreover, the base model already achieves 78.2% on HumanEval versus 60.5% on MBPP, leaving less room for improvement. These findings help explain the smaller aggregate gains on HumanEval.

Regarding comparisons with TaH+ and LaDiR, NF-CoT matches TaH+ (85.8) and trails LaDiR by only 1.6 points (85.8 vs. 87.4) on HumanEval. However, on the more stringent HumanEval+, NF-CoT outperforms both TaH+ (78.3 vs. 74.3) and LaDiR (78.3 vs. 73.2). Thus, the small HumanEval difference does not reflect a consistent disadvantage, while NF-CoT delivers significant gains over the base model across all four benchmarks.

AuthorsOpenReview ↗

Follow-up on Our Response

Dear Reviewer,

Thank you again for your positive assessment of our work and your constructive feedback.

We hope our responses have satisfactorily addressed your questions and suggestions. Please feel free to let us know if any further clarification would be helpful.

Best regards,

The Authors

AuthorsOpenReview ↗

Follow-up on Rebuttal and Discussion

Dear Reviewer,

As the discussion period is approaching its end, we would like to kindly follow up on our rebuttal. If any questions or concerns remain, we would be happy to provide further clarification.

Best regards,
The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Common Concerns: Necessity of the Flow, Practical Value of Exact Likelihood, and What the Latents Carry

We thank all reviewers for their thoughtful and constructive feedback. Several reviewers raised related questions about the necessity of the flow transformation, the practical value of exact likelihood, and what the sampled latents represent. We summarize our new experimental evidence below, with full results and reviewer-specific details provided in our individual responses.

Q1: Is the flow transformation necessary? (Reviewer CtU2 and Reviewer LNCs) Both reviewers asked for a matched baseline in which the LLM models the same VAE latents with an autoregressive Gaussian and the NLL objective, without the flow. We ran this ablation by removing the shallow flow blocks and keeping everything else fixed. The NLL on real latents rises from 0.573 to 0.864 bits/dim, and pass@1 falls on all four benchmarks (HumanEval 85.8 → 76.9, HumanEval+ 78.3 → 70.8, MBPP 83.9 → 78.7, MBPP+ 72.1 → 67.9). The gains therefore do not come from latent supervision or the training setup alone; the learned transformation improves both the density fit and downstream accuracy.

Q2: What does the exact likelihood buy in practice? (Reviewer aAjs and Reviewer CtU2) Exact, tractable likelihood enables direct policy-gradient optimization over continuous latent trajectories. Two new results demonstrate its practical value. First, the RL stage in the submission was a 150-step proof of concept on a subset of the RL data. Trained for 2 epochs (1,590 steps) on the full set of about 50K samples, it raises average pass@1 over the five benchmarks from 69.1 to 74.9, with gains of 4.4 to 7.4 points on each. Second, we ran the answer-only RL control in the original small-scale setting. Joint RL outperforms it on all five benchmarks (70.3 vs. 69.2 on average), so optimizing the latent policy contributes to the RL gain beyond what updating the answer tokens alone achieves. On density-based reranking (Table 9), the SFT density measures how typical a latent trajectory is under the CoT distribution, not whether it leads to a correct program, so we do not expect it to rank samples by correctness.

Q3: Do the sampled latents carry reasoning, and why model them probabilistically? (Reviewer aAjs and Reviewer LNCs) Two new experiments address this. Under a fixed budget of 16 answers, drawing 16 independent latents and decoding one answer from each gives higher pass@16 than decoding 16 answers from a single latent on all four benchmarks (HumanEval 90.9 → 95.1, HumanEval+ 85.4 → 91.5, MBPP 90.5 → 93.1, MBPP+ 79.1 → 83.6), so sampling different latents widens solution coverage beyond what resampling the answer achieves. In addition, a deterministic counterpart that regresses each latent with an L2 loss, under the same backbone, data, latent layout and training steps, reaches 75.3 / 69.1 on HumanEval / HumanEval+ and 78.3 / 65.4 on MBPP / MBPP+, against 85.8 / 78.3 and 83.9 / 72.1 for NF-CoT. Together with the decoded strategies in Appendix A, where latent samples for one problem decode into rolling-state DP, tabulation and memoization, these results indicate that the latents hold problem-specific choices of strategy, and that a distribution over them keeps these choices distinct instead of averaging them.

We will include all of the above in the final revised paper. We would be grateful if the reviewers could let us know whether these results resolve their concerns, and we are happy to provide further clarification before the discussion period closes.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.