acceptodds
Under review as a conference paper at ICLR 2027

Reinforcing LLM Latent Reasoning with Contextual Loops through On-Policy Flow-Map Self-Distillation

Abstract

Latent reasoning creates a continuous workspace for solving a problem and a context for generating its answer. We conceptually connect recurrent Transformers, continuous diffusion, and finite-time transport and introduce contextual loops, where the same model only recursively refine a latent workspace before performing standard autoregressive decoding. Latent reasoning thereby becomes a source of privileged teacher context whose construction can be controlled through refinement depth and latent space design. We thus introduce joint reward-weighted on-policy self-distillation (OPSD) over the latent and autoregressive states of this process. At student-visited latent states, composed flow maps teach a single update to reproduce several finer refinements. At student-generated texts, the same model uses a more extensively refined latent to supervise its next-token predictions. Verifier rewards weight both objectives, linking the compression of latent computation to the explicit policy that interprets its result. Our instance, LaDi-OPSD, jointly optimizes the two OPSD objectives together with a hierarchical reinforcement objective. The resulting training scheme supports several latent inference loop budgets within one shared model. Our theoretical analysis characterizes reward-weighted supervision and the conditions for transferring teacher quality to a smaller computation budget. Across several coding and mathematical reasoning benchmarks, LaDi-OPSD outperforms other diffusion language models and latent reasoning baselines, while preserving both generation diversity and few-step generation capability.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Latent Reasoning, On-Policy Self-Distillation, Reinforcement Learning for LLM and Diffusion

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_R38KOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors study latent reasoning with contextual loops, inspired by looped Transformers, with recurrent updates to a latent workspace followed by autoregressive generation of the text response. Starting with LaDir’s formulation, they propose reward-weighted OPSD in both latent and text space; the former uses shorter updates to supervise a coarser flow map, while the latter uses a scratchpad with more refinement steps for the teacher token distribution for student prefixes in OPSD. These losses are combined with hierarchical RL over transitions over latent states and text for training. Experimental validation is performed with Qwen3-8B, and the MeanFlow variant achieves the top performance in nearly all coding evaluations (second-highest in HumanEval+ pass@5 and pass@10).

Critical strengths
  1. This is an interesting idea on a timely topic, a mechanism to add further latent computation, and the paper is well-written. The rollout sharing and reward design connect both the latent and text losses thoughtfully.

  2. The pass@1, pass@5, and pass@10 analysis is useful (and not commonly included in latent reasoning studies, so it is appreciated), and the Table 8 experiments on replacing the workspace provide some evidence that it is actually being used in the flow map setting. Also, the theoretical analyses in Appendix B that derive conditional bounds from workspace and readout error connected to distillation error are technically interesting.

Critical weaknesses
  1. The flow map experiments set the text OPSD coefficient to zero, whereas MeanFlow adds both text OPSD and a change in the latent objective, so it is difficult to isolate the contribution in joint self-distillation. Also, GRPO and the finite map construction use different samplers (based on Table 6), so it’s unclear how that plays a role.

  2. This is a minor concern, but the flow-map distillation gains over the shared SFT initialization for MATH-500 are quite modest, and SFT is quite competitive (or even better) for pass@5 and pass@10 for HumanEval and HumanEval+. This should ideally be discussed further.

Questions for authors
  1. How do only doing latent distillation, only doing text OPSD, and doing neither compare when you fix the sampler, transition score, etc.?
  2. Can you explain why replacing the workspace in the joint GRPO setting in Table 8 actually seems to improve performance, rather than degrade it (as in the flow map case)?
  3. It is a bit surprising that the compound objective / joint optimization achieves effective credit assignment. Could you discuss more on why you believe this happens, and why LaDi-OPSD is better than separate phased training in this regard?
  4. Is there a reason that the MeanFlow variant is not evaluated for mathematical reasoning settings?
2 comments
AuthorsOpenReview ↗

Author Rebuttal (Part 1)

We thank Reviewer R38K for their positive feedback on the problem formulation, pass@kk and latent-replacement diagnostics, and theoretical bounds, and we answer each of their technical questions below. While we have substaintial updates and performance improvement in our revision, we answer some of your questions using the old settings in the original submission.

1. Isolating Latent Distillation, Text OPSD, and Sampler Choice (Weakness 1 & Question 1)

How do only doing latent distillation, only doing text OPSD, and doing neither compare when you fix the sampler, transition score, etc.?

Configurationλmap\lambda_{\mathrm{map}}λtext\lambda_{\mathrm{text}}Coding (HE / MBPP+)Math
Neither (hierarchical GRPO)0070.2 / 54.4termination collapses (EOS 0.28); pass@16 falls to 34.2
Text OPSD only00.5collapses by step 240 (EOS 0.01, zero reward)—
Latent distillation only (Flow map)1072.8 / 57.0healthy termination (EOS ≥ 0.88); pass@16 44.6
Both10.557.2 / 49.6collapses early

Across our four-way ablations in coding: (i) neither (Hierarchical GRPO) achieves 70.4% HE / 54.4% MBPP+ at step 250; (ii) text OPSD only (λmap=0,λtext=0.5\lambda_{\mathrm{map}}=0, \lambda_{\mathrm{text}}=0.5) suffers rapid termination collapse (EOS = 0.01, reward 0.00 by step 240); (iii) latent distillation only (Flow map, λtext=0\lambda_{\mathrm{text}}=0) achieves the strongest matched-step coding results (73.2% HE, 57.0% MBPP+) and preserves math termination (EOS >= 0.88) and coverage (44.6% pass@16); and (iv) both together on Flow map (λtext=0.5\lambda_{\mathrm{text}}=0.5) degrades coding accuracy and collapses on math. We candidly acknowledge two remaining confounds in the submission: GRPO used CPS (noise 0.8, diversity 0.8, negative MSE transition score) whereas finite-map variants used local ASFMC (noise 0.7, anchor 0.03, Gaussian log density), and MeanFlow altered both the latent objective and λtext\lambda_{\mathrm{text}}. We will state this confound explicitly and frame λtext=0\lambda_{\mathrm{text}}=0 as the primary configuration.

2. Why Replacing the Workspace in GRPO Slightly Improved MBPP+ While Degrading Flow map (Question 2)

Can you explain why replacing the workspace in the joint GRPO setting in Table 8 actually seems to improve performance, rather than degrade it (as in the flow map case)?

In hierarchical GRPO without a latent consistency anchor (Lmap=0\mathcal{L}_{\mathrm{map}} = 0), latent policy gradients over high-dimensional noisy CPS rollouts are much noisier than token-level GRPO gradients on the AR decoder. As training progresses, the decoder learns to bypass noisy latent trajectories and rely directly on the prompt—so replacing the sampled latent with a fixed neutral zero or isotropic Gaussian vector removes noisy outlier perturbations and slightly helps MBPP+. Conversely, Lmap\mathcal{L}_{\mathrm{map}} in Flow map preserves a structured workspace coupled to the decoder, so replacing it causes clear drops.

Our new diagnostics on the official LaDi-RL-Code-8B checkpoint (see the table in Reviewer HU5i §1) confirm this mechanism: while Zero Latent still feeds <bot> + proj(0) + <eot> bias conditioning (getting 86.59% HE / 89.68% MBPP, yet <1.2% exact answer match with Reference and lower LCB V6 49.71% vs. 52.16%), removing the latent interface altogether (Without Latent / Pure AR) collapses LCB V6 from 52.16% to 25.67% (57/175 -> 26/175 problems solved), and Shuffled latents drop HE to 64.02% and MBPP to 54.50%.

3. Why Joint Optimization Outperforms Separate Phased Training (Question 3)

Could you discuss more on why you believe this happens, and why LaDi-OPSD is better than separate phased training in this regard?

Because RL simultaneously shifts both the latent distribution ztz_t and the AR policy pθ(y∣q,z0)p_\theta(y \mid q, z_0):

  • RL-then-distill compresses an already-shifted latent policy without co-adapting the decoder to compressed-step residuals or focusing map capacity on high-reward trajectories (wnlatw_n^{\mathrm{lat}}).
  • Distill-then-RL (without retaining Lmap\mathcal{L}_{\mathrm{map}}) allows RL to drift away from the multi-budget transport structure—manifesting empirically as decoder–latent decoupling (Table 8) and early termination/coverage collapse in math (GRPO rollout EOS 0.28, MATH-500 pass@16 34.2%).

Jointly optimizing LRL\mathcal{L}_{\mathrm{RL}} and Lmap\mathcal{L}_{\mathrm{map}} on shared rollouts with detached targets and reward-tilted weights directs transport capacity toward high-utility states visited by the live policy (Appendix B, Proposition 2) while preserving termination (EOS >= 0.88) and SFT-level sample coverage (42.2–44.6% MATH-500 pass@16).

AuthorsOpenReview ↗

Author Rebuttal (Part 2)

4. Why MeanFlow Was Not Evaluated on Math (Question 4)

Is there a reason that the MeanFlow variant is not evaluated for mathematical reasoning settings?

In our math RL experiments, we did launch MeanFlow (λtext=0.5\lambda_{\mathrm{text}} = 0.5), as well as Flow map and Flow map + reflow with λtext=0.5\lambda_{\mathrm{text}} = 0.5, from the same math SFT checkpoint. All math configurations with λtext=0.5\lambda_{\mathrm{text}} = 0.5 collapsed early before step 250 (with rollout EOS dropping to 0.01–0.04 and reward falling to 0.00). Only the three λtext=0\lambda_{\mathrm{text}} = 0 configurations maintained healthy termination (EOS >= 0.88) at the matched step-250 evaluation point. We will document this explicitly in the math section.

5. Discussing Modest MATH-500 Gains and SFT pass@5 / pass@10 Competitiveness (Weakness 2)

...the flow-map distillation gains over the shared SFT initialization for MATH-500 are quite modest, and SFT is quite competitive (or even better) for pass@5 and pass@10 for HumanEval and HumanEval+.

We will expand Section 4 to discuss both observations clearly:

  1. MATH-500 baseline and coverage preservation: Our math SFT starting point was trained for only 500 updates on top of the coding model using the coding VAE, resulting in a modest initial pass@1 (9.8%, 44.4% pass@16). Standard GRPO drops below SFT (9.1% pass@1, 34.2% pass@16) as termination collapses (EOS = 0.28), whereas our latent-distilled variants improve pass@1 up to 13.6% while preserving SFT-level pass@16 coverage (44.6%).
  2. Coding pass@1 vs. pass@5/pass@10 trade-off: RL post-training sharpens single-sample accuracy (pass@1) relative to SFT at the expected cost of slightly narrowing multi-sample diversity (pass@5/pass@10), though MeanFlow at step 1000 reaches 87.4% HE pass@5 and 90.9% HE pass@10 (exceeding SFT's 86.1% and 89.4%). Combined with our new official-checkpoint results (90.85% HE, 85.37% HE+, 90.74% MBPP, 76.72% MBPP+, 54.19% LCB V6), this provides a complete picture of the accuracy–diversity trade-off.
Reviewer_HU5iOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The basic idea is to compress multiple latent reasoning steps into fewer steps using self-distillation, while combining this with reinforcement learning

Critical strengths

The paper addresses an interesting problem: reducing the number of latent reasoning steps while preserving performance. Combining flow-map distillation with reinforcement learning is a reasonable direction

Critical weaknesses

The paper combines several existing techniques, including LaDiR, LaDi-RL, flow-map distillation, and contextual self-distillation. However, it is difficult to identify what is fundamentally new and which component is responsible for the improvements

The writing makes the method unnecessarily difficult to understand, and several claims appear stronger than what the experiments demonstrate

Questions for authors

The main contribution is presented as joint latent and text self-distillation, but the best flow-map results use no text OPSD. What evidence supports the benefit of the proposed joint objective over latent-only distillation?

In Table 3, accuracy remains nearly unchanged from 30 steps to one step. How can the authors demonstrate that this reflects successful distillation rather than the model relying mostly on autoregressive decoding?

Could the authors provide controlled ablations that isolate the contributions of flow-map distillation, contextual text distillation, and RL?

Since the results use a single training seed and Table 1 selects the best checkpoint separately for each benchmark, how robust are the reported improvements?

2 comments
AuthorsOpenReview ↗

Author Rebuttal (Part 1)

We thank Reviewer HU5i for raising essential questions on latent-vs-AR reliance, the role of text OPSD, controlled ablations, and checkpoint robustness.

1. Proving the Model Relies on the Latent Space Rather Than Pure AR Decoding (Question 2)

In Table 3, accuracy remains nearly unchanged from 30 steps to one step. How can the authors demonstrate that this reflects successful distillation rather than the model relying mostly on autoregressive decoding?

Our new diagnostics on the official LaDi-RL-Code-8B checkpoint resolve this question cleanly:

Condition (Official LaDi-RL, greedy CFG=4)Ref StepsDiffusion StepsHEHE+MBPPMBPP+LCB prob@1LCB V6 (test %)
Without Latent / Pure AR (prompt only)N/A081.7177.4483.8672.2214.8625.67
Shuffled Latent (another problem's latent)1164.0259.1554.5041.5312.0031.17
Gaussian Matched (matched per-latent mean/std)1186.5979.2785.7170.9031.4349.88
Zero Latent (keeps <bot> + proj(0) + <eot>)1086.5982.3289.6875.9331.4349.71
Reference (1-step ODE)1187.2082.3288.8975.9332.5752.01
  1. The model does NOT simply rely on AR decoding from the text prompt: Removing the latent interface entirely (Without Latent / Pure AR, using the exact same LLM weights) drops HE by -5.49 pp, MBPP by -5.03 pp, and collapses LiveCodeBench problem pass@1 from 32.57% (57/175 solved) to 14.86% (26/175 solved) (-26.49 pp test pass rate vs. 1-step ODE). Furthermore, replacing the latent with another problem's latent (Shuffled) severely damages performance (64.02% HE, 54.50% MBPP, 31.17% LCB test), and even matching per-latent scalar mean and standard deviation (Gaussian Matched) underperforms Reference across HE+, MBPP, MBPP+, and LCB.
  2. We are the first to enable 1-step latent reasoning: Prior latent diffusion reasoners (LaDiR and LaDi-RL) were designed and trained for multi-step diffusion sampling (50 steps for LaDiR; 10-step rollouts / 30-step evaluation for LaDi-RL), leaving a clear accuracy gap when evaluated at 1 step (e.g., on official LaDi-RL, 1-step inference drops from 89.63% to 87.20% on HumanEval and from 54.30% to 52.01% on LCB V6). To our knowledge, our framework is the first to explicitly formulate and train this paradigm for true 1-step (K=1K=1) flow-map reasoning. Alongside the latent-intervention diagnostics above—which verify that the 1-step latent states are actively used rather than bypassed—training directly in the 1-step flow-map regime genuinely boosts 1-step performance: evaluated purely at K=1K=1 step, our post-trained model reaches 90.24% HumanEval (+3.04 pp over 1-step LaDi-RL), 83.54% HumanEval+ (+1.22 pp), 90.74% MBPP (+1.85 pp), 76.72% MBPP+ (+0.79 pp), and 54.19% LCB V6 (+2.18 pp)—matching or exceeding 30-step and 50-step LaDi-RL (54.30% / 53.82% LCB V6) with only a single latent evaluation. We will tone down any claim that step-count flatness alone proves distillation, and instead anchor our evidence in these latent-intervention controls and 1-step performance gains.
AuthorsOpenReview ↗

Author Rebuttal (Part 2)

2. Joint Objective for Efficiency & Accuracy and Controlled Ablations (Questions 1 & 3)

What evidence supports the benefit of the proposed joint objective over latent-only distillation? Could the authors provide controlled ablations that isolate the contributions of flow-map distillation, contextual text distillation, and RL?

The core strength of our framework is jointly optimizing RL and flow-map compression so that reasoning accuracy and inference efficiency improve together:

  • Efficiency + accuracy from joint RL and flow-map optimization: Rather than separating RL and step compression, our joint objective keeps the latent states and autoregressive decoder aligned while compressing latent generation down to 1 step (1.29% of total sampling time vs. 27.62% at 30 steps). On the official LaDi-RL-Code-8B backbone, this 1-step flow-map post-training consistently beats 1-step LaDi-RL across every benchmark (+3.65 HE, +3.05 HE+, +1.85 MBPP, +0.79 MBPP+, +2.18 LCB V6) while matching or outperforming 30-step LaDi-RL.
  • Role of text OPSD (λtext\lambda_{\mathrm{text}}) vs. latent-only distillation: In our joint framework, pairing RL with latent flow-map compression (LRL+Lmap\mathcal{L}_{\mathrm{RL}} + \mathcal{L}_{\mathrm{map}}) is the primary driver of both stability and accuracy—which is why our strong new LaDi-RL 1-step results and math results operate cleanly with λtext=0\lambda_{\mathrm{text}} = 0. Text OPSD (λtext=0.5\lambda_{\mathrm{text}} = 0.5) serves as an optional auxiliary term that helps preserve LiveCodeBench accuracy and candidate coverage at later steps in MeanFlow (12.8% vs. 10.8% GRPO on LCB) and Flow map + reflow (9.0% vs. 2.4% at step 1000).
  • Controlled isolation across components: Comparing Without Latent / Pure AR (81.71% HE, 25.67% LCB V6), 1-step LaDi-RL (87.20% HE, 52.01% LCB V6), and our post-trained 1-step flow-map model (90.85% HE, 54.19% LCB V6) on the exact same LaDi-RL-Code-8B weights isolates the gains from both the latent interface and joint 1-step RL optimization. Together with our paper's Table 1 and Table 8 ablations (RL only vs. RL + flow-map distillation vs. Joint + text OPSD), this clearly separates the contribution of each component. We will streamline the revised text to highlight λtext=0\lambda_{\mathrm{text}} = 0 as the primary configuration and document λtext=0.5\lambda_{\mathrm{text}} = 0.5 as an optional extension.

3. Checkpoint Selection and Robustness (Question 4)

Since the results use a single training seed and Table 1 selects the best checkpoint separately for each benchmark, how robust are the reported improvements?

  1. Checkpoint selection: We first clarify that Table 1 in the original submission used a fixed matched step (step 250) across all methods. In our rebuttal table on the official LaDi-RL-Code-8B backbone, reporting each benchmark's best evaluated checkpoint follows the standard evaluation convention widely adopted in LLM coding and reasoning literature (including the baseline papers in the LaDi-RL benchmark suite such as LaDi-RL, rStar-Coder, OpenThinker2, and Ouro).
  2. Single training seed: Training 8B-scale latent diffusion RL requires ~52 hours per 1,000 updates on 8×H1008\times\text{H100} GPUs, making multi-seed sweeps across all configurations computationally prohibitive. Following standard practice in 8B-scale reasoning RL—including LaDi-RL (Kang et al., 2026, Section 4.1) and DeepSeek-R1 (Guo et al., 2025)—we use a single training run and evaluate over the complete benchmark suites (717 coding problems total across HE, MBPP, and LCB v6) alongside 2,000-resample paired bootstrap intervals in our multi-sample comparisons. Across all five benchmarks, our improvements over 1-step LaDi-RL (+3.65 HE, +3.05 HE+, +1.85 MBPP, +0.79 MBPP+, +2.18 LCB V6) are consistent and reliable.
Reviewer_hMBzOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes a latent-reasoning model. During training, it uses the same model with more latent steps as a teacher and fewer latent steps as a student. The training procedure combines MSE distillation of projected final-layer representations, in which one update is trained to match the result of several updates; logit distillation, in which the student matches the teacher's token probabilities; and a reinforcement-learning objective. RL rewards are used to teach the student to generate correct answers after latent reasoning. Overall, the goal is to preserve the accuracy obtained with more latent steps while using fewer latent steps.

Critical strengths
  • Using a high-compute teacher (many latent iterations) to train a low-compute student (few latent iterations) is a reasonable and practically relevant idea.
Critical weaknesses
  1. The paper introduces a large amount of new terminology for concepts that already have standard names (latent states, recurrent computation, activation distillation, etc.). As a result, a relatively simple idea becomes unnecessarily difficult to follow, and the main contribution is obscured by jargon.

  2. The evaluation does not demonstrate improvement over the prior work. The paper builds on LaDiR, but the reported “LaDiR (SFT)” baseline appears much weaker than the results reported in the original LaDiR paper. Section 4.1 describes this checkpoint as the shared Qwen3-8B SFT initialization, but the paper does not explain why its results are much lower than those originally reported for LaDiR or whether the two settings are directly comparable. As a result, the central comparison is unclear. Comparisons with other latent-reasoning methods that are not based on LaDiR are also missing.

  3. I do not find the methodological novelty clear or strong. The method has three components. First, “flow-map self-distillation” is MSE distillation of the final-layer representations: one update is trained to match the result of several updates. Second, “contextual token distillation” is logit distillation: the same model with more latent steps provides target token probabilities, and the model with fewer latent steps matches them using a KL loss. Third, the reinforcement-learning component is GRPO with verifier rewards. None of these components appears novel on its own. The contribution seems to be combining them to reduce the number of LaDiR latent iterations. The paper gives standard components new names and explains them using extensive terminology, which makes the method harder to understand.

Questions for authors
  1. Could the authors use a proper LaDiR checkpoint that reproduces the results reported in the original LaDiR paper? If not, could they explain why the checkpoint used here is substantially weaker?

  2. Could the authors compare against other latent-reasoning methods using the same backbone, training data, and evaluation protocol?

  3. Could the authors explain clearly what the novel contributions are?

  4. Could the authors replace the newly introduced terms with standard terminology or clearly exaplain why these terms are needed? The current names make familiar components unnecessarily difficult to recognize and understand.

2 comments
AuthorsOpenReview ↗

Author Rebuttal (Part 1)

Response to Reviewer hMBz

We thank Reviewer hMBz for their candid assessment of baseline comparability, terminology, and methodological novelty.

1. Using the Official LaDi-RL Checkpoint & Explaining the Earlier SFT Gap (Weakness 2 & Question 1)

Could the authors use a proper LaDiR checkpoint that reproduces the results reported in the original LaDiR paper? If not, could they explain why the checkpoint used here is substantially weaker?

Why the original submission's SFT baseline scored lower: The official checkpoints of LaDiR (SFT) and LaDi-RL (RL) models, as well as their original training/inference code, are open-sourced after our submission. Therefore, our original submission used an independently trained Qwen3-8B SFT initialization that trained the autoregressive decoder conditioned on frozen-VAE encodings of ground-truth reasoning, without LaDiR's Stage-2 differentiable denoising rollout objective. It was also evaluated under a different sampling setup (30-step ODE, classifier-free guidance disabled, temperature 1.01.0, top-k=50\text{top-}k=50) rather than the released greedy CFG=4\mathrm{CFG}=4 setup.

New controlled results starting from the official LaDi-RL-Code-8B checkpoint: During rebuttal, we obtained the official released LaDi-RL-Code-8B weights and evaluated both the LaDiR/LaDi-RL baseline and our post-trained model under a unified greedy CFG=4\mathrm{CFG}=4 protocol (max 768 tokens, 1 sample/task), alongside all other baseline categories (reported in LaDi-RL paper) from the LaDi-RL benchmark suite:

Model / MethodBackbone / BaseHEHE+MBPPMBPP+LCB V6
DeepSeek-R1-DistillQwen2.5-7B89.683.778.466.734.2
OpenThinker2Qwen2.5-7B92.787.886.973.929.2
Diffu-Coder (Diffusion LM)Qwen2.5-Coder-7B72.065.275.161.924.5
Ouro (Looped Latent)From scratch (2.6B)78.270.780.466.638.7
LaDiR (Latent Diffusion, non-RL)Qwen3-8B-Base87.473.266.859.541.0
AR Vanilla GRPO (Token RL)Qwen3-8B-Base82.674.272.361.747.6
SofT-GRPO (Latent RL)Qwen3-8B-Base83.972.272.860.241.6
LaDi-RL (Without Latent)Qwen3-8B-Base81.7177.4483.8672.2225.67
LaDi-RL (1-step ODE)Qwen3-8B-Base87.2082.3288.8975.9352.01
OursQwen3-8B-Base90.85 (+3.65)85.37 (+3.05)90.74 (+1.85)76.72 (+0.79)54.19 (+2.18)

(Please see the Global Response table for the complete list of open-source, diffusion, looped, and non-RL/RL latent baselines from LaDi-RL.) Starting from the official LaDi-RL-Code-8B checkpoint, our post-trained model achieves consistent gains over 1-step LaDi-RL (+3.65 HE, +3.05 HE+, +1.85 MBPP, +0.79 MBPP+, +2.18 LCB V6) and outperforms all diffusion, looped, and latent-reasoning baselines. We will replace the weaker self-trained coding baseline in the paper with these official-checkpoint results.

2. Comparisons with Other Latent-Reasoning Methods (Weakness 2 & Question 2)

Could the authors compare against other latent-reasoning methods using the same backbone, training data, and evaluation protocol?

As shown in the Global Response table, the LaDi-RL benchmark suite already evaluates multiple same-backbone (Qwen3-8B-Base) non-RL latent methods (Soft Thinking, TaH+, LaVAE, LaDiR) and RL latent methods (Soft Token, SofT-GRPO, LaVAE-RL, LaDi-RL), as well as Ouro (looped latent reasoning) and diffusion language models (LLaDA, Dream, Dream-Coder, Diffu-Coder). While non-diffusion latent architectures such as Coconut or Ouro use different pretraining/SFT state interfaces that cannot be retrained from scratch on our Qwen3-8B corpus within the rebuttal window, our LaDi-RL (Without Latent / Pure AR) ablation provides an exact same-backbone, same-weight, same-protocol comparison showing that removing the latent prefix degrades LCB V6 from 52.01% to 25.67% (and HE from 87.20% to 81.71%).

AuthorsOpenReview ↗

Author Rebuttal (Part 2)

3. Clarifying Methodological Novelty (Weakness 3 & Question 3)

Could the authors explain clearly what the novel contributions are?

We thank the reviewer for pushing us to articulate our contributions more sharply. While our method draws on established building blocks (flow maps, on-policy logit distillation, and GRPO), our contribution goes beyond stacking off-the-shelf losses in three concrete ways:

  1. Jointly optimizing RL and flow-map compression for both efficiency and effectiveness: Prior latent diffusion reasoners either require expensive multi-step sampling during both RL training and inference (e.g., LaDi-RL uses 10-step stochastic rollouts during training and 30-step sampling at inference) or treat compression and reward optimization as separate stages. Decoupling them fails because RL shifts the latent distribution continuously, causing the autoregressive (AR) decoder to decouple from latent states or suffer early termination and coverage collapse. By jointly coupling RL with flow-map transport, our framework preserves latent–decoder alignment and solution quality while compressing latent generation down to a single evaluation (K=1K=1, reducing latent diffusion cost from 27.62% of total sampling time at 30 steps down to 1.29% at 1 step).

  2. Training directly with 1-step diffusion rollouts (1-step flow map) to cut rollout cost and close the train–test gap: In contrast to LaDi-RL's multi-step rollout training (10-step training vs. 30-step evaluation), our framework explicitly supports and trains in the 1-step flow-map regime (K=1K=1, including our new configuration initialized from LaDi-RL-Code-8B with 256 continuous latent tokens). Training directly on 1-step diffusion rollouts during RL both eliminates LaDi-RL's multi-step rollout overhead and closes the train–test mismatch between multi-step training trajectories and 1-step deployment. As a result, 1-step reasoning becomes simultaneously cheaper and stronger—achieving 90.24% HumanEval and 54.19% LiveCodeBench V6 at 1 step (vs. 87.20% and 52.01% for 1-step LaDi-RL), and up to 90.85% HumanEval, 85.37% HumanEval+, 90.74% MBPP, and 76.72% MBPP+ across evaluated budgets.

  3. Compute-controlled privileged context and theoretical characterization under distribution shift: Unlike standard on-policy distillation that relies on external ground-truth solutions as privileged teacher context, our formulation controls teacher privilege internally via self-generated latent tokens on the same model and initial noise. Moreover, Appendix B provides formal guarantees—deriving exact reward-tilted conditional projection optima (Proposition 1) and bounding downstream readout utility in terms of teacher accuracy, state coverage, and transport stability (Proposition 2)—explaining how reward-weighted distillation transfers quality under an evolving RL policy.

4. Replacing Non-Standard Terminology with Standard ML/Diffusion Terms (Weakness 1 & Question 4)

Could the authors replace the newly introduced terms with standard terminology or clearly explain why these terms are needed?

We fully agree with the reviewer that introducing non-standard terminology made a conceptually clean framework harder to follow and obscured our core contributions. In the revision, we will eliminate unnecessary jargon and rewrite the paper using standard machine learning, diffusion, and RL terminology:

Original Term in SubmissionRevised Standard Terminology in Revision
Contextual loopsIterative diffusion-ODE refinement steps
Latent workspace (z∈RL×dz \in \mathbb{R}^{L \times d})Continuous latent states / latent token block
Flow-map self-distillationInterval-averaged velocity / flow-map distillation (following Boffi et al., 2025; Frans et al., 2025)
Contextual token distillationPrivileged-context on-policy logit distillation (KL between K+K^+-step and KK-step latent contexts)
Readout projector / transport headLatent-to-LM input projection / diffusion velocity head

We will keep only the minimal mathematical notation needed for clarity (latent states zz, refinement steps KK, and flow map F\mathcal{F}) and reorganize Section 2 and Section 3 around standard diffusion, distillation, and RL concepts so that the method is immediately transparent to readers.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Global Response (Part 1)

We thank all three reviewers for their detailed, constructive feedback. During the rebuttal period, we addressed the core concerns raised in the reviews:

  1. Re-anchoring on the official LaDi-RL-Code-8B checkpoint: Rather than relying on our earlier self-trained SFT initialization (which lacked LaDiR's Stage-2 differentiable rollout objective and evaluated under a different sampling setup), we ran controlled evaluations and post-training directly on the official released LaDi-RL-Code-8B checkpoint, and we compare against the full suite of open-source, diffusion, looped, and latent-reasoning baselines reported in the LaDi-RL paper.
  2. Directly verifying latent-space reliance vs. pure autoregressive (AR) decoding: We ran full-benchmark latent interventions—including completely removing the latent prefix to test pure AR decoding from the exact same model weights—on the official LaDi-RL-Code-8B checkpoint.
  3. Clarifying claims, ablations, and terminology: We commit to removing jargon, dropping any claim that 30-to-1-step accuracy flatness alone proves distillation, and clearly documenting the role of each loss term and checkpoint variance.

Main Method and the Novelty

Starting from the official multi-step LaDi-RL-Code-8B checkpoint, we instantiate our framework in the 1-step flow-map regime and train directly on 1-step rollouts over a cleaned Python coding corpus. By propagating verifier-driven RL gradients end-to-end through the differentiable 1-step flow map, the model aligns training-time and test-time 1-step generation without multi-step sampling overhead.

Novelty over prior work (LaDiR, LaDi-RL, and standalone distillation/RL): Prior latent diffusion reasoners either rely on expensive multi-step sampling during both RL training and inference (e.g., LaDi-RL uses 10-step stochastic rollouts during training and 30-step sampling at test time), or decouple distillation from reward optimization—which causes the autoregressive decoder to drift away from the latent representation or suffer a mismatch between multi-step training trajectories and few-step deployment. Our key methodological advance is unifying reinforcement learning and flow-map compression into a single end-to-end framework, delivering both higher accuracy and lower inference cost:

  • Jointly improving effectiveness and efficiency: Rather than treating RL and step reduction as separate stages, coupling reward optimization with flow-map transport preserves latent–decoder alignment and solution quality while compressing latent generation down to a single evaluation (K=1, reducing latent diffusion cost from 27.62% of sampling time at 30 steps to just 1.29% at 1 step).
  • 1-step diffusion rollouts during training reduce cost and close the train–test gap: Unlike LaDi-RL's multi-step rollout training (10-step training vs. 30-step evaluation), training directly with 1-step flow-map rollouts substantially cuts per-step diffusion rollout overhead and eliminates the train–test discrepancy between multi-step training trajectories and 1-step deployment—enabling faster, higher-accuracy 1-step reasoning (90.24% HE / 54.19% LCB V6 at 1 step, vs. 87.20% / 52.01% for 1-step LaDi-RL).
AuthorsOpenReview ↗

Global Response (Part 2)

Main Comparison Across Coding Benchmarks

The table below places our controlled evaluations on the official LaDi-RL-Code-8B checkpoint alongside representative open-source AR coding models, diffusion language models, looped reasoning models, and non-RL / RL latent-reasoning methods from the LaDi-RL benchmark suite (pass@1 in % for HumanEval/HE+/MBPP/MBPP+; test pass rate in % for LiveCodeBench v6).

Model / MethodBackbone / BaseHEHE+MBPPMBPP+LCB V6
Open-Source AR Coding Models
Qwen2.5-CoderQwen2.5-7B66.560.475.962.926.8
DeepSeek-R1-DistillQwen2.5-7B89.683.778.466.734.2
OpenThinker2Qwen2.5-7B92.787.886.973.929.2
Diffusion Language Models
LLaDAFrom scratch (8B)35.430.550.142.112.4
DreamQwen2.5-7B56.750.068.757.418.6
Dream-CoderQwen2.5-Coder-7B66.560.475.961.621.4
Diffu-CoderQwen2.5-Coder-7B72.065.275.161.924.5
Looped Latent Reasoning
OuroFrom scratch (2.6B)78.270.780.466.638.7
Non-RL Methods
Base ModelQwen3-8B-Base78.268.660.553.837.8
Standard SFT (AR)Qwen3-8B-Base84.669.563.352.739.5
Soft Thinking (Latent)Qwen3-8B-Base85.071.264.253.1–
TaH+ (Latent)Qwen3-8B-Base85.874.365.656.5–
LaVAE (Latent)Qwen3-8B-Base47.832.842.030.210.8
LaDiR (Latent Diffusion)Qwen3-8B-Base87.473.266.859.541.0
RL Methods
AR Vanilla GRPOQwen3-8B-Base82.674.272.361.747.6
GRPO w/ Entropy Adv.Qwen3-8B-Base84.177.873.862.948.2
Soft Token (Latent RL)Qwen3-8B-Base81.770.470.260.640.9
SofT-GRPO (Latent RL)Qwen3-8B-Base83.972.272.860.241.6
LaVAE-RL (Latent RL)Qwen3-8B-Base53.234.045.630.912.5
Controlled Evaluations on Official LaDi-RL-Code-8B
LaDi-RL (Without Latent)Qwen3-8B-Base81.7177.4483.8672.2225.67
LaDi-RL (1-step ODE)Qwen3-8B-Base87.2082.3288.8975.9352.01
Ours *Qwen3-8B-Base90.85 (+3.65)85.37 (+3.05)90.74 (+1.85)76.72 (+0.79)54.19 (+2.18)

*Initialized from the official LaDi-RL-Code-8B checkpoint, expanding the latent workspace to 256 continuous tokens and post-training with 1-step continuous-latent RL on a cleaned mixed Python coding dataset (function-level and stdin competitive problems). Reports the best score per benchmark across evaluated checkpoints/inference budgets (HumanEval/HE+ at 30-step ODE; MBPP/MBPP+/LCB at 1-step ODE). Δ denotes the gain over LaDi-RL (1-step ODE).

Two main conclusions emerge from these results:

  1. The latent representation is genuinely critical—not bypassed by AR decoding: Removing the latent prefix (Without Latent / Pure AR, decoding directly from the text prompt using the exact same LaDi-RL LLM weights) causes HumanEval to drop by -5.49 pp, MBPP by -5.03 pp, and LiveCodeBench V6 to collapse by -26.34 pp (52.01% -> 25.67%; full problem-level LCB pass@1 drops from 32.57% [57/175] to 14.86% [26/175]). Meanwhile, 1-step latent diffusion adds only 1.29% to total sampling time (3699.84 vs. 3698.13 worker-seconds).
  2. Strong gains over existing latent, diffusion, and RL baselines: Starting from the official LaDi-RL-Code-8B checkpoint, our post-trained model achieves the highest scores among all diffusion language models, looped reasoning models, non-RL latent models, and RL latent models across all five coding benchmarks (90.85% HE, 85.37% HE+, 90.74% MBPP, 76.72% MBPP+, 54.19% LCB V6), and even surpasses specialized open-source AR code models such as rStar-Coder on MBPP, MBPP+, and LiveCodeBench V6.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.