- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors study latent reasoning with contextual loops, inspired by looped Transformers, with recurrent updates to a latent workspace followed by autoregressive generation of the text response. Starting with LaDir’s formulation, they propose reward-weighted OPSD in both latent and text space; the former uses shorter updates to supervise a coarser flow map, while the latter uses a scratchpad with more refinement steps for the teacher token distribution for student prefixes in OPSD. These losses are combined with hierarchical RL over transitions over latent states and text for training. Experimental validation is performed with Qwen3-8B, and the MeanFlow variant achieves the top performance in nearly all coding evaluations (second-highest in HumanEval+ pass@5 and pass@10).
Critical strengths
-
This is an interesting idea on a timely topic, a mechanism to add further latent computation, and the paper is well-written. The rollout sharing and reward design connect both the latent and text losses thoughtfully.
-
The pass@1, pass@5, and pass@10 analysis is useful (and not commonly included in latent reasoning studies, so it is appreciated), and the Table 8 experiments on replacing the workspace provide some evidence that it is actually being used in the flow map setting. Also, the theoretical analyses in Appendix B that derive conditional bounds from workspace and readout error connected to distillation error are technically interesting.
Critical weaknesses
-
The flow map experiments set the text OPSD coefficient to zero, whereas MeanFlow adds both text OPSD and a change in the latent objective, so it is difficult to isolate the contribution in joint self-distillation. Also, GRPO and the finite map construction use different samplers (based on Table 6), so it’s unclear how that plays a role.
-
This is a minor concern, but the flow-map distillation gains over the shared SFT initialization for MATH-500 are quite modest, and SFT is quite competitive (or even better) for pass@5 and pass@10 for HumanEval and HumanEval+. This should ideally be discussed further.
Questions for authors
- How do only doing latent distillation, only doing text OPSD, and doing neither compare when you fix the sampler, transition score, etc.?
- Can you explain why replacing the workspace in the joint GRPO setting in Table 8 actually seems to improve performance, rather than degrade it (as in the flow map case)?
- It is a bit surprising that the compound objective / joint optimization achieves effective credit assignment. Could you discuss more on why you believe this happens, and why LaDi-OPSD is better than separate phased training in this regard?
- Is there a reason that the MeanFlow variant is not evaluated for mathematical reasoning settings?
2 comments
Author Rebuttal (Part 2)
4. Why MeanFlow Was Not Evaluated on Math (Question 4)
Is there a reason that the MeanFlow variant is not evaluated for mathematical reasoning settings?
In our math RL experiments, we did launch MeanFlow (), as well as Flow map and Flow map + reflow with , from the same math SFT checkpoint. All math configurations with collapsed early before step 250 (with rollout EOS dropping to 0.01–0.04 and reward falling to 0.00). Only the three configurations maintained healthy termination (EOS >= 0.88) at the matched step-250 evaluation point. We will document this explicitly in the math section.
5. Discussing Modest MATH-500 Gains and SFT pass@5 / pass@10 Competitiveness (Weakness 2)
...the flow-map distillation gains over the shared SFT initialization for MATH-500 are quite modest, and SFT is quite competitive (or even better) for pass@5 and pass@10 for HumanEval and HumanEval+.
We will expand Section 4 to discuss both observations clearly:
- MATH-500 baseline and coverage preservation: Our math SFT starting point was trained for only 500 updates on top of the coding model using the coding VAE, resulting in a modest initial pass@1 (
9.8%,44.4%pass@16). Standard GRPO drops below SFT (9.1%pass@1,34.2%pass@16) as termination collapses (EOS = 0.28), whereas our latent-distilled variants improve pass@1 up to13.6%while preserving SFT-level pass@16 coverage (44.6%). - Coding pass@1 vs. pass@5/pass@10 trade-off: RL post-training sharpens single-sample accuracy (
pass@1) relative to SFT at the expected cost of slightly narrowing multi-sample diversity (pass@5/pass@10), thoughMeanFlowat step 1000 reaches87.4%HE pass@5 and90.9%HE pass@10 (exceeding SFT's86.1%and89.4%). Combined with our new official-checkpoint results (90.85%HE,85.37%HE+,90.74%MBPP,76.72%MBPP+,54.19%LCB V6), this provides a complete picture of the accuracy–diversity trade-off.
Author Rebuttal (Part 1)
We thank Reviewer R38K for their positive feedback on the problem formulation, pass@k and latent-replacement diagnostics, and theoretical bounds, and we answer each of their technical questions below. While we have substaintial updates and performance improvement in our revision, we answer some of your questions using the old settings in the original submission.
1. Isolating Latent Distillation, Text OPSD, and Sampler Choice (Weakness 1 & Question 1)
Across our four-way ablations in coding: (i) neither (
Hierarchical GRPO) achieves70.4%HE /54.4%MBPP+ at step 250; (ii) text OPSD only (λmap=0,λtext=0.5) suffers rapid termination collapse (EOS = 0.01, reward0.00by step 240); (iii) latent distillation only (Flow map, λtext=0) achieves the strongest matched-step coding results (73.2%HE,57.0%MBPP+) and preserves math termination (EOS >= 0.88) and coverage (44.6%pass@16); and (iv) both together onFlow map(λtext=0.5) degrades coding accuracy and collapses on math. We candidly acknowledge two remaining confounds in the submission:GRPOused CPS (noise 0.8, diversity 0.8, negative MSE transition score) whereas finite-map variants used local ASFMC (noise 0.7, anchor 0.03, Gaussian log density), andMeanFlowaltered both the latent objective and λtext. We will state this confound explicitly and frame λtext=0 as the primary configuration.2. Why Replacing the Workspace in
GRPOSlightly Improved MBPP+ While DegradingFlow map(Question 2)In hierarchical
GRPOwithout a latent consistency anchor (Lmap=0), latent policy gradients over high-dimensional noisy CPS rollouts are much noisier than token-level GRPO gradients on the AR decoder. As training progresses, the decoder learns to bypass noisy latent trajectories and rely directly on the prompt—so replacing the sampled latent with a fixed neutralzeroor isotropicGaussianvector removes noisy outlier perturbations and slightly helps MBPP+. Conversely, Lmap inFlow mappreserves a structured workspace coupled to the decoder, so replacing it causes clear drops.Our new diagnostics on the official
LaDi-RL-Code-8Bcheckpoint (see the table in Reviewer HU5i §1) confirm this mechanism: whileZero Latentstill feeds<bot> + proj(0) + <eot>bias conditioning (getting86.59%HE /89.68%MBPP, yet<1.2%exact answer match withReferenceand lower LCB V649.71%vs.52.16%), removing the latent interface altogether (Without Latent / Pure AR) collapses LCB V6 from52.16%to25.67%(57/175 -> 26/175problems solved), andShuffledlatents drop HE to64.02%and MBPP to54.50%.3. Why Joint Optimization Outperforms Separate Phased Training (Question 3)
Because RL simultaneously shifts both the latent distribution zt and the AR policy pθ(y∣q,z0):
GRPOrollout EOS0.28, MATH-500 pass@1634.2%).Jointly optimizing LRL and Lmap on shared rollouts with detached targets and reward-tilted weights directs transport capacity toward high-utility states visited by the live policy (Appendix B, Proposition 2) while preserving termination (
EOS >= 0.88) and SFT-level sample coverage (42.2–44.6%MATH-500 pass@16).