acceptodds
Under review as a conference paper at ICLR 2027

How to Train Your Latent Flow Map for Few-Step Text Generation

Abstract

Diffusion language models generate tokens in parallel, but they need many denoising steps, which offsets their speed advantage. Flow maps reduce the number of steps by learning to jump between noise levels in a single network evaluation. Prior flow-map language models build these maps over token embeddings or frozen encoder states, and each makes its own design choices, so it is unclear which choices matter. We study them in a latent space learned jointly with the denoiser: we distill a latent diffusion language model into a flow map within its own latent space and analyze each component of the distillation, namely the regression target, self-conditioning, rollout depth, decoder alignment and the treatment of the latent space. Refining the target with the teacher matters most for open-ended text, while aligning the decoder to the flow map's outputs is essential on GSM8K, where a single wrong token breaks a program. The pre-trained latent space can stay frozen, and adapting it during distillation, together with the teacher, can improve coverage (pass@ at large ) but not single-sample accuracy, which drops on Sudoku. With the resulting recipe, distillation on GSM8K takes k iterations, and our model with steps exceeds its teacher with up to steps and matches the best baseline, which needs steps. On OpenWebText, with steps, it has % lower generative perplexity than the strongest flow-map baseline at matched entropy.

Primary area:
generative models
Keywords:
diffusion language models, flow maps, few-step generation, latent space co-training, distillation, latent diffusion

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_XtzWOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper extends LDLM (a latent language diffusion model that jointly trains the latent space and the diffusion process) to a flow map. Overall, the paper explores and analyses multiple axes such as self-conditioning, the "rollout depth", and alignment between the latent space and the flow map. Novelty is limited, with the primary contributions being around bringing together aspects from multiple papers and analysis of their interplay. Experiment results are strong.

Critical strengths

Strong empirical numbers on OWT and GSM8K.

Critical weaknesses

Writing is very poor. It is hard to make sense of what the paper is trying to convey and how the experiments tie to this claim. The central novelty is not well defined. Converting an existing flow into its associated flow map with other techniques such as self-conditioning that have already been explored for language diffusion is limited novelty at best. Experiments and results are inconsistent. The takeaways are different from dataset to dataset, and some settings that have positive impact for one dataset have negative ones for others. Citations, backgrounds and conclusions are weak/missing.

Questions for authors

What is the final conclusion from the paper? Is there a one-sentence takeaway for readers?

3 comments
AuthorsOpenReview ↗

Response to Reviewer XtzW

We appreciate the reviewer's time and honest feedback on clarity. It helped us see where the paper's message was not coming through, and we try to state it plainly below. Points shared with other reviewers are detailed in the general response (GR).

One-sentence takeaway. Our experiments suggest a recipe for training latent language flow maps: refine the target with the teacher, align the decoder to the map's outputs where exact tokens matter, and keep the latent space frozen by default, though adapting it jointly with the flow map raises pass@k on GSM8K and MAUVE on OWT — decoder alignment and joint space adaptation were not studied in prior flow-map LMs.

In more detail. A latent flow map generates text in a few steps. It works in the latent space of a latent diffusion LM, its teacher, and a decoder turns its output into tokens. We split the training of such a model into five components:

  1. The diagonal target
  2. Self-conditioning
  3. The semigroup loss and its rollout depth
  4. Decoder alignment: the decoder is trained on the flow map's outputs, so it learns to read what the map actually produces.
  5. Joint adaptation of the latent space: the latent space and the teacher keep training together with the flow map.

We ablate each component and pick its best setting (Section 5, Tables 1 and 2). In the default, frozen-space model, the same component settings work on OWT and GSM8K, except decoder alignment (below).

With this recipe, our model with 8 steps matches the 64-step GSM8K accuracy of ELF* and FMLM+, at a tenth of FMLM+'s FLOPs (GR-2, GR-4). On OWT at 8 steps, it has 40% lower gPPL than FMLM* at matched entropy (Table 3, GR-5).

"Some settings that have positive impact for one dataset have negative ones for others." In the default model, this holds for one component, decoder alignment, and it likely follows from the task: on GSM8K, one wrong token breaks a program, so alignment matters a lot (Table 2: +24 points at 8 steps). On OWT, a substituted token is mostly tolerated, so alignment does not help and we leave it off. All other components use the same setting on OWT and GSM8K, and it is the best or within run-to-run variation of the best once diversity is taken into account (Tables 1, 2 and 6). Adapting the latent space raises pass@k on GSM8K and MAUVE on OWT (Table 3: 0.92 vs 0.86 at 8 steps), which suggests better mode coverage on both tasks. On GSM8K, teacher-only training does not explain this gain: at N=2, pass@128 rises 6.0 points with adaptation but only 1.3 with teacher-only training (Table 9).

Background and citations. We reviewed the literature carefully and cited the work closest to ours:

  • flow-map foundations
  • flow-map LMs (FMLM, FMLM+, FMLM*)
  • latent and continuous diffusion LMs
  • discrete diffusion LMs
  • other distillation methods for both continuous and discrete diffusion LMs

The field moves fast, and we could not cover every related paper. If there are specific works whose absence changes the picture of our contribution, we would be grateful if you could point us to them.

Reviewer_XtzWOpenReview ↗

Response to author rebuttal

Dear authors, thank you for the detailed rebuttal.

  1. The one-sentence takeaway is still not clear. There are multiple different statements in the provided one-sentence takeaway. I believe the paper would benefit a great deal from a rewrite that paints a simple picture and it is obvious to readers what that picture is. Enlisting all results and then providing post-hoc explanations makes it hard to draw a conclusion from the paper. The overall writing is also very hard to follow.
  2. I maintain that novelty is limited. As the authors point out, 1/2/3 already exist in the literature, 5 does not add significant value and 4 is important only in certain settings. Moreover, I believe decoder alignment (4) and latent space adaptation (5) are relatively simple ideas. I would suggest the authors add more value for readers through either non-trivial ideas that lead to performance improvements or show why their proposed improvements and experiments are important for the community. An interesting direction here would be examining why ELF* -> FMLM* fails to train but LDLM-FM does not, given these are very tightly related methods.

While the rebuttal adds more clarity to what the paper does, I don't think it resolves my concerns about writing quality and novelty. Thus, I choose to maintain my rating.

AuthorsOpenReview ↗

Response to Reviewer XtzW (2)

We thank the reviewer again for their time, the quick response and the close engagement with our rebuttal. We would like to highlight two aspects of the work.

  • Our work aims to answer open questions about latent language flow maps, rather than to propose a conceptually new method. Each of these questions (the components of our study) arises naturally as soon as one starts building a flow map. Our work answers exactly these questions and shows the best setting for each. We believe these results are helpful for anyone building a latent language flow map.

  • We also want to emphasize the importance of the joint space adaptation question. The latent space is a core part of the method, so which space is best to use? The trade-off between diffusability and decodability is still an active research question (Zhang et al., 2026). LDLM finds this trade-off automatically by training the space jointly with the denoiser. It is natural to ask the same for flow maps: would joint training reshape the space so that it becomes easier to denoise in few steps? This is the question we address in the paper: the jointly trained teacher’s space is already good enough for few-step generation, and further adapting it is worth it when higher pass@k matters, which may be useful, for example, for RL.

As for why ELF* -> FMLM* fails to train on GSM8K while LDLM-FM does not, we do not yet know. We find it an interesting direction and thank the reviewer for it.

References:

  • Zhang et al. "Scaling and Distilling Text Embeddings for Better Diffusibility", arXiv:2610.01016 (2026)
Reviewer_ecUxOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes to distill a latent diffusion language model (LDLM) into a flow map so that text can be generated in a few steps. The flow map is trained inside LDLM's latent space with the diagonal and semigroup losses of prior flow-map language models (FMLM, FMLM*). Empirical results on GSM8K show that the 8-step model performs on par with FMLM+ at 64 steps and better than its teacher, while on OpenWebText (OWT) it achieves 40% lower generative perplexity (gPPL) than FMLM* at 8 steps at matched entropy. Additionally, ablation studies are conducted on each component, including the regression target, self-conditioning, rollout depth, decoder alignment, and a frozen versus adapted latent space.

Critical strengths
  1. The ablations are well conducted, and run-to-run variation is properly measured and used as a tie threshold.
  2. The proposed decoder alignment trick is simple and effective, especially on tasks that require exact token matches such as GSM8K.
Critical weaknesses
  1. The novelty is limited. The work is incremental over LDLM and the FMLM line of work, and the main new component is decoder alignment. The paper is mainly an ablation study of combining two existing techniques.

  2. The evaluation does not support the main claims:

    i. OWT is at best a sanity check for language modeling today, and the reported win is in gPPL, which the paper itself notes can be gamed even at data-level entropy. On MAUVE, the distributional metric added for that reason, LDLM-FM and FMLM* are tied by the paper's own 0.05 tolerance (0.83 vs 0.79 at 4 steps, 0.86 vs 0.82 at 8 steps), and FMLM* has lower gPPL at 2 steps (Table 3). The samples do not match the claimed quality either: the 8-step sample in Figure 5 (gPPL 44.1) is locally fluent but globally incoherent, and it contradicts itself throughout. Real text has a gPPL of 15.0.

    ii. On GSM8K, "exceeds its teacher" is confounded: the student uses a decoder aligned to its own outputs while the teacher keeps its original decoder, and the paper reports that alignment alone is worth about 24 points at 8 steps. The baselines are weak or incomplete: FMLM+ numbers are copied from its paper with a different initialization, FMLM* is missing, ELF and ELF-PD are trained by the authors with an unreported budget, and no autoregressive baseline is included. This matters because accuracy plateaus at about 35% from 16 to 64 steps (Figure 1), while FMLM+ (its Table 7), whose TinyGSM setup the paper follows, reports an autoregressive baseline in the same setting at 53.9% with sampling and 63.3% with greedy decoding.

    iii. The scope is narrow: two main benchmarks plus Sudoku, a single 136M flow map, one teacher family and one latent space, as the limitations section acknowledges.

Questions for authors
  1. What is the GSM8K accuracy of the teacher when its decoder is aligned to the teacher's own outputs in the same way?
  2. What is the round-trip (encode then decode) execution accuracy of the autoencoder on gold GSM8K solutions? Several failure cases in the appendix are corrupted text rather than wrong reasoning.
  3. (Suggestion) How does the method compare in accuracy and wall-clock time with an autoregressive model of similar size trained on TinyGSM?
  4. (Suggestion) Given the incoherence of the samples, can the authors add a coherence evaluation beyond gPPL, entropy and n-gram diversity, such as human or LLM-judge ratings?
3 comments
AuthorsOpenReview ↗

Response to Reviewer ecUx

We are grateful to the reviewer for the thorough review and the sharp questions on evaluation. They led us to add the LLM-judge study, the autoencoder round-trip and the AR comparison. Shared points are in the general response (GR).

W1. Novelty. Please see GR-1. Two components of the recipe are new to flow-map LMs. Decoder alignment trains the decoder on the flow map's outputs, so it learns to read what the map actually produces. It is essential where exact tokens matter (Table 2: +24 points of GSM8K pass@1 at 8 steps). Joint space adaptation keeps training the latent space together with the flow map, and it raises pass@k, which suggests better mode coverage (Table 9: at N=2, pass@128 rises 6.0 points, against 1.3 with teacher-only training). Beyond these two, the paper tests every component under one protocol, so a reader knows which choices matter and where.

W2.i OpenWebText. Please see GR-5. We use OWT because every flow-map baseline reports it, so all of them can be compared from released checkpoints. You are right on MAUVE: LDLM-FM only ties FMLM* within our tolerance, at every N. At N=2, where every method is above 90 gPPL, FMLM* is preferred by the judges. But LDLM-FM-A has, in fact, higher MAUVE at N=8 (0.92 vs 0.82), and the new LLM-judge evaluation clearly prefers our models at N=8 (Q4).

At N=4, the judges rate LDLM-FM as tied with FMLM* or slightly behind, and LDLM-FM-A as tied or ahead, but our models match the corpus-level n-gram diversity of real text (0.34 and 0.33, real 0.33), while FMLM* is lower (0.28). A judge compares single samples, so it cannot see this.

W2.ii GSM8K.

  • Decoder confound: we aligned the teacher's decoders on its own outputs, as requested. At 8 steps, the teacher then reaches 2.1%, against 33.7% for LDLM-FM. At 64 steps it reaches 31.9%, which LDLM-FM matches with 8 steps, please see GR-3.
  • FMLM+: re-evaluated from its released checkpoint, with CIs (GR-2).
  • FMLM*: we trained its first two stages (ELF and ELF*) on TinyGSM (ELF*, now in Figure 1: 33.6% at 64 steps, below LDLM-FM), but the final distillation into FMLM* did not converge.
  • ELF and ELF-PD: we add their training details to Appendix A. ELF is ELF-B on GPT-2 Small hidden states, trained on TinyGSM with ELF's released code for 500k iterations, with all other settings as in the original ELF. ELF-PD distills it in 6 stages, from 64 steps down to 1, one epoch per stage. Both use a shifted time schedule (shift 32 for ELF, 128 for ELF-PD), which improves them at every step budget.
  • AR: we evaluated the AR model FMLM+ cites (160.5M) on our bench: 55.2 ± 1.6% with sampling and 63.5% greedy. Every diffusion model here stays well below it, including ELF* and FMLM+ at 64 steps and our teacher, so the gap is not specific to few-step distillation. LDLM-FM at N=8 is 12x faster per answer (115 vs 1369 ms, GR-4). We add AR to the appendix (Table 10).

W2.iii Scope. Agreed, we test one model scale, one teacher family and one latent space, as the limitations say. But within this scope, we consider several types of tasks: unconditional text (OWT), conditional math with executable answers (GSM8K) and Sudoku puzzles.

Q1. We ran this experiment, the teacher with decoders aligned on its own outputs. Please see GR-3.

Q2. Autoencoder round-trip. GSM8K has no Python gold solutions, so we use 10,240 held-out TinyGSM programs, encoded and decoded without diffusion (Table 11, %):

AutoencoderTokens matchedProgram runsCorrect end to end
Teacher's97.071.266.9
LDLM-FM98.992.887.8
LDLM-FM-A99.293.488.7

With LDLM-FM's decoders, 92.8% of reconstructed programs run, against 71.2% with the teacher's. End to end, 88% reproduce the gold result.

Q3. Please see the AR bullet above and GR-4.

Q4. Coherence. Two LLM judges, 500 random pairs per matchup, both orders (GR-5). Against FMLM* at N=8, LDLM-FM wins 69–72% of pairs and LDLM-FM-A 72–75%. At N=2, FMLM* is preferred. Real text wins over every model, including the baselines, in 99.5%+ of pairs.

Dear authors, thank you for the detailed response and the new experiments. My main concerns have been adequately addressed by the additional results, including the aligned-decoder teacher, the autoencoder round-trip, the AR comparison and the LLM-judge evaluation. While my concern on novelty remains, I will raise my rating to 3.

AuthorsOpenReview ↗

Response to Reviewer ecUx (2)

We thank the reviewer for engaging with the new results and for raising the score. We are glad the additional experiments addressed the main concerns, and we appreciate the careful questions on evaluation, which made the paper stronger.

Reviewer_CouzOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper distills a latent diffusion LM (LDLM) into a flow map inside LDLM’s own jointly learned latent space, giving a few-step generator (LDLM-FM). It ablates each distillation component:

  • the diagonal target (an iterated, guided teacher)
  • self-conditioning
  • rollout depth
  • decoder alignment (the decoder is trained on flow-map outputs)
  • frozen versus adapted latent space
Critical strengths

1- Systematic component-wise ablations. The paper isolates each design choice and finds clear, usable effects:

  • Decoder alignment moves GSM8K at N=8 from 14.8 to 39.1.
  • The refined target gives 44.3 gPPL against 112.5 for the clean latent.
  • Deeper rollouts for all jumps ([4,4]) hurt.

This is useful for anyone building few-step latent LMs.

2- Strong and careful few-step results. LDLM-FM is competitive or better than all baselines on GSM8K and Sudoku. On OpenWebText it leads at N=4 and N=8. The evaluation is more careful than usual:

  • It compares at matched entropy.
  • It reports MAUVE and seed variance.
  • It states its limitations.

Distillation is also cheap, at 5k to 20k iterations on 2 GPUs.

Critical weaknesses

1- Confounded headline comparison on GSM8K. The claim that 8 steps beat the teacher at 64 steps depends on decoder alignment, which the teacher does not get. The paper says removing alignment costs about 24 points. There is no teacher run with an aligned decoder, so the gain from distillation cannot be separated from the decoder fix. Other settings are also tuned per task:

  • Alignment is used only on GSM8K and Sudoku.
  • The anchor, anneal, and J differ between datasets.
  • Sampling uses W=4, outside the training range.

2 - Weak statistical support and unfair baselines.

  • GSM8K has single runs, with run-to-run variation of about 1.2 to 3 points. Several ablation claims sit at that edge, for example self-conditioning (2.7 to 4.6 points) and frozen versus adapted.
  • The FMLM+ numbers are copied from its paper, with a different teacher, data pipeline, and model size. FMLM⋆ is missing on GSM8K.
  • Compute is not matched. LDLM-FM runs 258M parameters at inference versus 157M for FMLM⋆, and the paper reports no wall-clock latency.

3 - Limited novelty and weak generality. Most pieces come from prior work: the losses and target iteration from FMLM and FMLM⋆, the latent space from LDLM. The new parts are decoder alignment and the frozen-versus-adapted study. The adapted-space coverage claim cannot be attributed to the space, because the teacher also trains. Evidence covers one scale, one teacher family, and one latent space. Absolute text quality at N≤2 is still poor (gPPL above 90 versus 15 for real text), and the N=4 and N=8 samples shown are still incoherent.

Questions for authors
  • Clarification: what does the teacher score on GSM8K at N=64 if its decoder is also aligned (or fine-tuned) on its own outputs? This decides whether “beats teacher” holds.
  • Clarification: was FMLM+ retrained on your GSM8K data and compute budget? Appendix B only does this for Sudoku.
  • Clarification: in LDLM-FM-A, how much of the pass@k gain comes from the teacher training versus the encoder adapting? A teacher-only-trained ablation would answer this.
  • Clarification: how sensitive are results to W=4 at inference, and why was a value outside the training range chosen?
  • Clarification: can you report seeds or confidence intervals for the main GSM8K numbers in Figure 1?
  • Suggestion (future work): report wall-clock latency and throughput, and test a larger scale.
5 comments
AuthorsOpenReview ↗

Response to Reviewer Couz

We thank the reviewer for the careful and detailed review and for the concrete suggestions. The requested experiments made the paper stronger, and we're glad to address all of the concerns below. Shared points are in the general response (GR).

W1 / Q1. Teacher with an aligned decoder. We aligned the teacher's decoders on its own outputs, as you suggested. At the same 8-step budget, the teacher then reaches 2.1%, against 33.7% for LDLM-FM. At 64 steps it reaches 31.9%, which LDLM-FM matches with 8 steps and exceeds with 16 (GR-3).

W1. Settings that differ between tasks.

  • Decoder alignment is used only on GSM8K and Sudoku. These are conditional tasks with one correct answer, where a single wrong token breaks a program or a grid, so decoding the exact token matters. Unconditional OWT generation is open-ended and mostly tolerates a substituted token, such as a synonym, so alignment does not help there (Table 1) and we leave it off.
  • J is not tuned per task: we use J=5 on all datasets (Table 6).
  • The anchor and the anneal are used only in LDLM-FM-A, not in the default model. The anneal sets how long the encoder adapts before its learning rate reaches zero. On OWT, the anchor is essential (Table 8: gPPL at N=8 is 249.9 without it, 43.6 with it), and the best anneal is 3000 steps. On GSM8K, the anchor makes no difference, and a short anneal of 300 steps works best: longer anneals cost 3.8 and 9.5 points at N=8 (Table 7).

W1 / Q4. W=4 outside the training range. W is used only on OWT. GSM8K uses Wc=2.5, inside its range [1, 3]. W is an input of the map, so it works as a free sampling parameter, like a guidance scale. LDLM-FM's Pareto-optimal point closest to real-text entropy for each W (Table 15), as gPPL / entropy (real text: 15.0 / 5.44):

WN=2N=4N=8
1155.5 / 5.4588.1 / 5.4145.2 / 5.32
2122.1 / 5.4492.9 / 5.5364.8 / 5.51
3113.2 / 5.4974.0 / 5.4447.0 / 5.48
4106.5 / 5.4668.9 / 5.4244.3 / 5.45
5103.6 / 5.4469.0 / 5.4243.6 / 5.45

Quality saturates at W=4: going to W=5 changes gPPL by 0.1–2.9, within seed deviation. The map keeps working beyond its training range.

W2. Statistics. The main GSM8K results now have 95% CIs (GR-2). These cover sampling variation. Across training runs, accuracy varies with a standard deviation of about 1.2 points (Section 5). On GSM8K the self-conditioning gap is at the edge of run-to-run variation, as Section 5.1 already states. On OWT the effect is clear (gPPL 44.3 vs 70.0–72.5 at N=8).

W2 / Q2. Baselines.

  • FMLM+: we did not retrain it on GSM8K, but we now evaluate its released checkpoint on our bench (GR-2). Our models are trained with FMLM+'s own TinyGSM data pipeline. The teacher cannot be shared, since FMLM+ distills a discrete MDLM. We compare with their strongest variant, initialized from MDLM. It trains much longer than our model: 250k steps at batch 512 on top of the MDLM initialization, against 5k steps at batch 64 for LDLM-FM (GR-4). On Sudoku we retrained FMLM+ ourselves (Appendix B), because our models needed more data and we wanted to keep the comparison fair.
  • FMLM* is trained in stages: ELF -> ELF* -> FMLM*. We trained ELF and ELF* on TinyGSM (ELF*, now in Figure 1: 25.3% at 8 steps, 33.6% at 64, both below LDLM-FM), but the final distillation from ELF* into FMLM* did not converge, so there is no FMLM* model to evaluate.
  • Compute: please see GR-4. Of the 258M inference parameters, only the flow map (136M) runs at every step, and the decoders (122M) run once. That is why LDLM-FM needs fewer FLOPs than FMLM* at every N on OWT, and fewer than FMLM+ from N=8 on (Table 12). ELF* has exactly the same architecture as FMLM*, so on GSM8K, LDLM-FM with 8 steps also costs far less than ELF* with 64 for the same accuracy.

W3. Novelty, generality, N <= 2. For the contributions, please see GR-1. We agree on scale and say so in the limitations. At N <= 2, every method, FMLM* included, stays above 90 gPPL (GR-5).

Q3. Teacher training vs space adaptation. We added the ablation you suggest: the teacher keeps training while the space stays frozen. Change in pass@128 relative to LDLM-FM (GSM8K test):

NTeacher trains, space frozenSpace adapted (LDLM-FM-A)
2+1.3+6.0
8+0.4+2.0
32+0.5+3.5

Across k = 32–128, training the teacher alone changes pass@k by at most 1.3 points, so teacher-only training does not explain the gain. We add it as Table 9 and update Section 5.2.

Q5. Please see GR-2. Q6. For wall-clock, please see GR-4. A larger scale is in the limitations.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer Couz,

Thank you again for your thoughtful review and suggestions. With about two days left in the discussion period, we wanted to gently follow up on our response, which includes the aligned-decoder teacher experiment, the teacher-only training ablation, the additional statistical and efficiency results, among other additions.

If your schedule allows, we would appreciate hearing whether these additions address your concerns. We would be happy to clarify any remaining points. Thank you for your time!

Dear Reviewer Couz,

With less than 24 hours remaining in the discussion period, we wanted to briefly follow up once more. We would be grateful for any feedback on our rebuttal, particularly whether the additional experiments address your main concerns or leave any points we could still clarify.

We appreciate your time and understand that this is a busy period.

Best regards, The authors

Comments on the paper

3 comments
AuthorsOpenReview ↗

General response (1/2)

We sincerely thank all reviewers for their time and thoughtful feedback. Below are the concerns shared by several reviewers, with new results. Each reviewer's thread covers the rest. Changes in the revised PDF are in blue. Tables and sections refer to the revised PDF, and "GR-n" refers to point n of this general response.

1. What the paper is about

We study and propose a recipe for training latent language flow maps. We identify the main components of that pipeline:

  1. The diagonal target
  2. Self-conditioning
  3. The semigroup loss and its rollout depth
  4. Decoder alignment
  5. The latent space

Components 4 and 5, decoder alignment and joint space adaptation, were not studied in prior flow-map LMs. For each component, we ran a thorough ablation showing which settings to use. On OWT and GSM8K, the best settings for most components turned out to be the same. In the default, frozen-space model, the only component that differs is decoder alignment: it helps a lot on the conditional GSM8K task, where predicting the exact token matters, whereas on the unconditional OWT task, which likely tolerates synonyms, it is better left off.

On GSM8K, LDLM-FM in 8 steps is as accurate as ELF* and FMLM+ in 64, and uses 10x fewer FLOPs than FMLM+ (GR-2, GR-4). On OWT, it has 40% lower gPPL than FMLM* at 8 steps and matched entropy (Table 3).

Jointly adapting the latent space while training a flow map raises pass@k on GSM8K (Figure 2) and MAUVE on OWT (Table 3: 0.92 vs 0.86 at 8 steps), which suggests better mode coverage.

2. GSM8K with confidence intervals (Couz, ecUx)

We now evaluate the released FMLM+ checkpoint on our bench instead of copying its numbers. We also add ELF*, the first stage of FMLM*, which replaces ELF in Figure 1 because it is stronger at every N. Pass@1 (%) on the GSM8K test set, mean ± 95% CI over 5 sampling seeds:

NLDLM-FMLDLM-FM-AFMLM+ELF*
15.82 ± 0.547.20 ± 0.952.27 ± 0.720.60 ± 0.10
215.15 ± 0.5314.50 ± 0.712.68 ± 0.254.14 ± 0.43
427.29 ± 1.3025.75 ± 0.857.79 ± 1.0612.69 ± 0.29
833.69 ± 0.6532.25 ± 1.1017.65 ± 1.2925.31 ± 0.43
1636.09 ± 1.3633.83 ± 1.1426.91 ± 0.6130.25 ± 0.59
3235.77 ± 0.9734.94 ± 0.3330.31 ± 0.9132.40 ± 0.55
6435.15 ± 0.9235.41 ± 0.8632.62 ± 0.8433.59 ± 0.59

FMLM+ here is its strongest variant, initialized from MDLM. Its other variants score lower from N=2 on (Table 10). With 8 steps, LDLM-FM matches both baselines with 64 steps (33.7 vs 33.6 and 32.6). With 16 steps, it exceeds FMLM+ and is 2.5 points above ELF* (36.1 vs 32.6 and 33.6). Our numbers are close to the published ones (FMLM+ at 64 steps: 32.6 vs 33.6, AR: 55.2 and 63.5 vs 53.9 and 63.3), so our bench does not handicap the baselines.

3. Teacher with aligned decoders (Couz Q1, ecUx Q1)

We aligned the teacher's decoders on its own outputs, as in Eq. 9 with the teacher's prediction in place of the flow map's (5k steps, teacher and encoder frozen). As controls, we also fine-tuned the decoders with the reconstruction-only objective, without alignment and tried decoding the teacher with LDLM-FM's decoders. Pass@1 (%):

NTeacherFine-tuned, no alignmentSelf-alignedLDLM-FM decodersLDLM-FM
81.70 ± 0.461.85 ± 0.422.08 ± 0.531.96 ± 0.2033.69 ± 0.65
1624.64 ± 1.1222.84 ± 0.6724.93 ± 1.1526.66 ± 0.7236.09 ± 1.36
6429.45 ± 1.5330.14 ± 1.2231.90 ± 1.3932.04 ± 1.0335.15 ± 0.92

Alignment helps the teacher a little at 64 steps (31.9% against 29.5%), slightly more than plain fine-tuning (30.1%). At 8 steps, the teacher stays near 2% with every decoder. So even with self-aligned decoders, the teacher with 64 steps only matches LDLM-FM with 8 steps, and LDLM-FM with 16 steps exceeds it (Table 10).

AuthorsOpenReview ↗

General response (2/2)

4. Compute (Couz, ecUx)

GSM8K, one H100, bf16, full pipeline, each method with its authors' sampler, median of 10 runs:

MethodNpass@1 (%)TFLOPsLatency, batch 1 (ms)Throughput, batch 16 (samples/s)
LDLM-FM833.71.1811596
FMLM+6432.611.626856.0
AR, T=1–55.20.05613697.7

At matched accuracy, LDLM-FM needs 10x fewer FLOPs than FMLM+, with 6x lower latency and 16x higher throughput. It also needs 8x less memory at batch 64 (7.1 vs 57.6 GB). AR needs the fewest FLOPs, but generates about 150 tokens one after another, so its latency is 12x higher.

On OWT LDLM-FM is also cheaper than FMLM* at every N (TFLOPs per sample):

NFMLM*LDLM-FM
10.590.43
20.810.60
41.260.96
82.171.67

Distillation of LDLM-FM takes 4.3 h on 2xH100 on OWT and 48 min on GSM8K, while FMLM*'s best OWT model trains for 82 h on 8xB200 (Yoo et al., 2026). On GSM8K, LDLM-FM distills in 5k steps at batch 64 (about 0.3M sequences), while FMLM+ trains for 250k steps at batch 512 (about 128M) on top of its MDM initialization (Agarwal et al., 2026, their Table 11). Full tables are in Appendix B (Tables 12 and 13).

5. OpenWebText beyond gPPL (ecUx, Couz)

Below N=4, all methods have gPPL above 90 (real text: 15), so we compare at N=4 and 8. On MAUVE, LDLM-FM ties FMLM* within our 0.05 tolerance (0.83 vs 0.79 and 0.86 vs 0.82), and LDLM-FM-A beats it (0.87 and 0.92). At N=8, LDLM-FM is also close to its teacher with 128 steps (0.86 vs 0.89).

We also ran two LLM judges (Gemini 3.8 Flash, GPT-6.1-Terra) on 500 random pairs per matchup, each judged in both orders. Share of pairs won against FMLM* at the same N, range over the two judges (Table 14):

NLDLM-FMLDLM-FM-A
20.27–0.300.22–0.26
40.42–0.490.49–0.57
80.69–0.720.72–0.75

At N=8 both our models are clearly preferred over FMLM*. Real text wins over every model in 99.5%+ of pairs.

A judge sees one sample per model at a time, so it cannot measure corpus-level diversity. It also tracks gPPL, which can be lowered by giving up entropy. This is why we compare models at matched entropy. At N=4, the judges rate LDLM-FM as tied with FMLM* or slightly behind, and LDLM-FM-A as tied or ahead, but our models are more diverse than FMLM* (0.34 and 0.33 vs 0.28, real text 0.33) and have higher MAUVE (0.83 and 0.87 vs 0.79).

6. Changes to the paper

  • Figure 1 with re-evaluated FMLM+, and ELF* in place of ELF, since ELF* is stronger at every N (ELF stays in Table 10). Full GSM8K tables with CIs, including the teacher with aligned decoders and the AR reference, are in Table 10.
  • New in Appendix B: autoencoder round-trip (Table 11), FLOPs (Table 12), wall-clock (Table 13), LLM judge (Table 14), W sweep (Table 15).
  • Section 5.2 and Table 9: the teacher-only ablation for the adapted space.
  • Appendix A: training details and budgets of ELF, ELF-PD and ELF*.

Dear reviewers,

The author-reviewer discussion period is ending soon. If you have not done so yet, please read the authors’ rebuttal and reply to it. Tell the authors whether it addresses your concerns, and update your score if your assessment has changed.

The ICLR 2027 Reviewer Guidelines list « discussing with the authors to clarify and improve the paper » as one of the core duties you accepted as a reviewer. With the high number of submissions this year, your input matters more than ever. Recall that reviewers that do not fulfill their duties may also face penalties on their own submissions.

Thank you for your time and help.

Best regards,


Your Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.