acceptodds
Under review as a conference paper at ICLR 2027

Gumbel Straight Flow: Distilling Autoregressive Models into One-step Flow Maps

Abstract

We present Gumbel Straight Flow (GSF), a continuous flow map language model that leverages the noise-data coupling of a pretrained autoregressive language (AR) model. We theoretically demonstrate that the coupling between Gumbel noise and one-hot token sequences induced by an autoregressive model yields non-intersecting linear paths connecting the noise to the sequence representations. To further enhance high-quality few-step path sampling, we use a flow map semigroup objective where the tangent (velocity) condition is guided directly by the AR teacher. Across various benchmarks, including pretraining and downstream tasks, GSF can outperform current few-step language generation baselines.

Primary area:
generative models
Keywords:
flow matching, diffusion, language models, autoregressive models, parallel text generation, flow maps, step distillation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_M6abOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes Gumbel Straight Flow (GSF), a method for distilling autoregressive language models into few-step parallel flow-map generators. The key idea is to construct noise–data pairs using the Gumbel coupling induced by an autoregressive teacher, rather than independently pairing noise and text. The authors theoretically show that this coupling produces non-intersecting straight trajectories under ideal conditions, and combine it with a flow-map consistency objective for efficient generation. Experiments on OpenWebText, LM1B, TinyStories, conditional text completion, and XSum summarization demonstrate improvements over several existing few-step generation and distillation baselines.

Critical strengths
  1. Strong theoretical motivation and novelty: The paper provides an interesting connection between autoregressive Gumbel sampling and straight trajectories in continuous flow matching. The non-intersection analysis offers a principled explanation for why the proposed noise–data coupling may make few-step generation easier to learn.

  2. Strong few-step results and evaluation: GSF demonstrates substantial improvements over FMLM and Gumbel Distillation, particularly at one sampling step. The paper includes multiple datasets, conditional generation tasks, distributional metrics, LLM-judge evaluations, and ablations, providing meaningful empirical support for the approach.

Critical weaknesses
  1. Gap between theoretical straightness and practical generation quality: The paper establishes non-intersecting straight trajectories for the ideal AR-induced coupling, supported by empirical straightness measurements in Figure 5. However, the final one-step flow-map model still produces text with limited coherence, as illustrated in Appendix H. It remains unclear how closely the trained student approximates the ideal flow and what primarily limits its ability to retain the AR teacher's generation quality.

  2. Limited breadth of downstream evaluation: While the paper demonstrates strong improvements on generative perplexity, distributional metrics, conditional text completion, and XSum summarization, the downstream evaluation is relatively narrow. In particular, it remains unclear whether the improvements in few-step generation translate to broader language-model capabilities such as question answering, reasoning, or instruction following. Additional task-level evaluation would strengthen the practical significance and generalizability of the proposed approach.

Questions for authors
  1. Figure 5 demonstrates improved empirical straightness for GSF-FM, while the final GSF model additionally uses a flow-map consistency objective. Can the authors clarify whether similar trajectory-straightness diagnostics hold for the final trained flow-map model?

  2. The paper demonstrates promising results on XSum summarization and conditional text completion. Could the authors provide additional evaluation or discussion of generalization to other conditional generation tasks, such as question answering, to better establish the broader applicability of the method?

  3. Appendix D discusses the remaining generation-quality gap relative to the AR teacher, including the challenges of high-dimensional one-hot representations and sensitivity to token prediction errors. Could the authors provide further empirical analysis to identify which factors most strongly contribute to this gap, and whether increasing student capacity helps mitigate it?

3 comments
AuthorsOpenReview ↗

Official Comment by Authors for Reviewer M6ab (1/3)

We thank Reviewer M6ab for your careful evaluation and constructive feedback. We particularly appreciate the reviewer's recognition of the following strengths:

  • Strong theoretical motivation
  • Novelty of our approach
  • Strong few-step generation performance
  • Comprehensive empirical evaluation

We address the reviewer's concerns as follows.


W1 & Q3 [Teacher-student performance gap]:

(Weakness 1): Gap between theoretical straightness and practical generation quality: The paper establishes non-intersecting straight trajectories for the ideal AR-induced coupling, supported by empirical straightness measurements in Figure 5. However, the final one-step flow-map model still produces text with limited coherence, as illustrated in Appendix H. It remains unclear how closely the trained student approximates the ideal flow and what primarily limits its ability to retain the AR teacher's generation quality.

(Question 3) Appendix D discusses the remaining generation-quality gap relative to the AR teacher, including the challenges of high-dimensional one-hot representations and sensitivity to token prediction errors. Could the authors provide further empirical analysis to identify which factors most strongly contribute to this gap, and whether increasing student capacity helps mitigate it?

[Theory on practical straightness] Theorem 1 characterizes the optimally trained model, our learning target. For imperfectly trained models, the new Proposition 2 bounds learned trajectory straightness by the FM loss. The non-intersecting property enables a lower FM-loss upper bound, consistent with the reduced empirical training loss (Fig. 5a) and improved empirical straightness (Fig. 5b).

Proposition 2 (Straightness and flow matching loss). Let (x0,x1)∼π(\mathbf{x} _0,\mathbf{x} _1)\sim\pi for any coupling π\pi with finite second moments, let vθ(⋅,t)\mathbf{v} _\theta(\cdot,t) be KK-Lipschitz for all t∈[0,1]t\in[0,1] (K>0K>0), and let {xtθ}\{\mathbf{x}^\theta _t\} solve dxtθ=vθ(xtθ,t)dtd\mathbf{x}^\theta _t=\mathbf{v} _\theta(\mathbf{x}^\theta _t,t) dt with x0θ=x0\mathbf{x}^\theta _0=\mathbf{x} _0. Then

\le C _K \mathcal{L} _{\mathrm{FM}}(\theta;\pi), \qquad C _K := 2+\frac{1}{2}(e^{2K}-1-2K).$$ **[Finite model capacity]** Since Proposition 2 bounds trajectory straightness by the FM loss, reducing this loss in practice is crucial, with model capacity being a key factor. The student uses 1,024× fewer sequential network evaluations than the AR teacher, limiting its computational expressivity and potentially contributing to the performance gap. **[Analysis on teacher-student capacity gap]** To empirically verify that capacity explains the gap, we fixed the teacher at size S (Gen-PPL=46.5) and varied only the student size (initialized from scratch). As student model capacity (parameters × steps) increased, performance gradually approached that of the teacher. | Gen-PPL ↓ | 1-step | 2-step | 4-step | |-|:-:|:-:|:-:| | S → S (12 layers) | 171.5 | 177.3 | 159.4 | | S → M (24 layers) | 139.6 | 119.3 | 114.8 | | S → L (36 layers) | 114.1 | 109.9 | 103.3 | | S → L (36 layers, more training) | 86.2 | 78.5| 75.7 | We also fixed the student size (AR-init.) and varied the teacher size (S: 46.5, M: 32.05, L: 27.81 Gen-PPL): | Gen-PPL ↓ | 1-step | 2-step | 4-step | |-|:-:|:-:|:-:| | S → S | 114.9 | 105.6 | 99.8 | | M → S | 119.7 | 112.3 | 104.6 | | L → S | 114.7 | 106.7 | 101.2 | At 4 steps, student scaling (S → L) improved Gen-PPL (159.4 → 75.7), while teacher scaling produced almost identical performance. Larger teachers yielded no significant gains, further supporting student capacity as the main bottleneck. **[Clarification on the goal]** The primary goal of distillation is not to match the teacher's absolute performance, but to improve the efficiency–performance frontier, as shown in Figure 1. Figure 1 shows our main empirical contributions: (1) improving distillation over Gumbel-Distill (Zhang et al., 2026) using the AR teacher, and (2) outperforming FMLM (Lee et al., 2026) through AR teacher supervision with the same student parametrization. We also outperform compute-matched AR models. **[Comparison to compute-matched AR and coherence]** We evaluated the wall-clock latency–performance trade-off with KV caching and sufficiently large batch sizes. As shown in Figure 11 of the revised manuscript, GSF achieves a better latency–performance trade-off than AR models, confirming the Pareto frontier advantage illustrated in Figure 1. In particular, compared with the most efficient AR (3.1M, 1024 steps) model in Figure 1, our 2-step and 4-step GSF models achieve higher efficiency while outperforming it in both coherence and overall winning rate according to LLM-judge evaluations: | Method | Params × Step Efficiency | Wall-clock Speed-up | Coherence | Overall | |-|:-:|:-:|:-:|:-:| | GSF-2 | **9.45×** | **2.41×** | **71.3%** | **64.3%** | | GSF-4 | **4.72×** | **1.53×** | **81.0%** | **84.4%** |
AuthorsOpenReview ↗

Official Comment by Authors for Reviewer M6ab (2/3)

Q1 [Straightness of FlowMap]:

(Question 1): Figure 5 demonstrates improved empirical straightness for GSF-FM, while the final GSF model additionally uses a flow-map consistency objective. Can the authors clarify whether similar trajectory-straightness diagnostics hold for the final trained flow-map model?

[Straightness measure] We also measured the trajectory straightness of GSF and FMLM flow maps under the same linear schedule. Both methods show improved straightness over their flow-matching counterparts (at temperature 1, with optimal-temperature results in parentheses), and GSF remains straighter than FMLM. This suggests that the underlying straight-path property carries over to the learned flow maps.

MethodStraightness ↓
FLM4.80 (1.82)
GSF-FM3.22 (0.70)
FMLM0.74
GSF-FlowMap0.48

W2 & Q2 [Limited Downstream task]:

(Weakness 2): Limited breadth of downstream evaluation: While the paper demonstrates strong improvements on generative perplexity, distributional metrics, conditional text completion, and XSum summarization, the downstream evaluation is relatively narrow. In particular, it remains unclear whether the improvements in few-step generation translate to broader language-model capabilities such as question answering, reasoning, or instruction following. Additional task-level evaluation would strengthen the practical significance and generalizability of the proposed approach.

(Question 2): The paper demonstrates promising results on XSum summarization and conditional text completion. Could the authors provide additional evaluation or discussion of generalization to other conditional generation tasks, such as question answering, to better establish the broader applicability of the method?

[Instruction-Following Task] We chose the instruction following as a comprehensive evaluation covering diverse tasks. Following CSD (Kim et al., 2026), we distill a 124M AR student and a 170M GSF student from the same 1.5B AR teacher on Dolly-15K, which includes eight categories: open_qa, general_qa, classification, closed_qa, brainstorming, information_extraction, summarization, and creative_writing. We evaluate both models on the Dolly test split and four separate instruction-following benchmarks (Self-Inst, Vicuna, Sinst, UnNI) to assess generalization beyond the training data.

At 32 steps, GSF achieves an average ROUGE-L of 22.26, outperforming the AR (CSD) student (21.34) and approaching the teacher (23.33), while being 1.22× faster than the AR student (184.0 vs. 223.7 ms) and 36× faster than the teacher (6681 ms). At 16 steps, GSF is 2.43× faster than the AR student (92.0 ms) and still achieves higher ROUGE-L (21.87), and it outperforms MDLM at 16 steps in both latency (92.0 vs. 148.3 ms) and ROUGE-L (21.87 vs. 19.06) with 3.3× fewer parameters. At 8 steps, GSF is 4.86× faster (46.0 ms) with only a small drop in ROUGE-L (20.78). GSF also outperforms the AR student on four of the five benchmarks at 32 steps. These results show that GSF applies to diverse conditional generation tasks beyond summarization and text completion while keeping efficient few-step generation. They also suggest that the teacher–student gap may be smaller when distillation targets downstream tasks, which may require less student capacity than general pre-training.

ModelParamsStepsLatency (ms) ↓DollySelf-InstVicunaSinstUnNIAvg.
Teacher1.5B256668126.5814.2016.5027.6131.7723.33
MDLM554M16148.319.9511.0616.1323.7024.4419.06
AR (KL)124M256223.723.6910.1814.9916.7119.4917.01
AR (CSD)124M256223.725.1613.1815.5725.5127.2921.34
GSF (Ours)170M846.024.9412.1017.0623.9825.8020.78
GSF (Ours)170M1692.025.6912.5417.2826.1327.7021.87
GSF (Ours)170M32184.026.0012.7616.9227.0628.5722.26

We have included these results in Table 7 of the revised manuscript, provided further details on instruction following in Appendix C, and reported the wall-clock latency–performance trade-off in Figure 12. Qualitative samples are provided in Appendix J (Table 23).

AuthorsOpenReview ↗

Official Comment by Authors for Reviewer M6ab (3/3)

Reference (Cited in the Rebuttal)

  • Zhang, C., Hu, X., Liu, B., & Liu, Q. (2026). Gumbel Distillation for Parallel Text Generation. International Conference on Learning Representations (ICLR 2026).

  • Lee, C., Yoo, J., Agarwal, M., Shah, S., Huang, J., Raghunathan, A., ... & Kim, J. (2026). Flow Map Language Models: One-Step Language Modeling via Continuous Denoising. Advances in Neural Information Processing Systems (NeurIPS 2026).

  • Kim, Y., Shin, D., Kang, M., Na, B., & Moon, I.-C. (2026). Distillation of Large Language Models via Concrete Score Matching. International Conference on Learning Representations (ICLR 2026).


Thank you again for your valuable feedback. Please feel free to ask any further questions. Note that we have highlighted all changes in the manuscript in purple\textcolor{purple}{\text{purple}}.

Reviewer_JgHKOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose a method for distilling an autoregressive model into a flow map that generates the entire text in parallel in one or a few steps. The method builds on the observation that sampling from an AR model via the Gumbel-max trick defines a deterministic correspondence between Gumbel noise and the generated token sequence. The authors show that when flow matching in one-hot space is trained on such noise–text pairs instead of random ones, the straight lines between noise and text do not intersect, so the optimal trajectories are straight. For a single token, this correspondence coincides with optimal transport. The noise is mapped to a Gaussian by a monotone transformation that preserves straightness, and the pairs are constructed from real texts with a single forward pass of the AR model. A flow map is then trained on these pairs: at each point of the trajectory, the model learns to predict the corresponding text, so no intermediate flow model is needed as a teacher.

Experiments are conducted on OpenWebText, LM1B, and TinyStories, as well as on prefix completion and XSum summarization. The metrics are generative perplexity, unigram entropy, Gradient Moment, and pairwise comparisons with an LLM judge. The trajectories are straighter than with random pairing, and with 1–4 sampling steps the method outperforms FMLM of the same architecture in quality at comparable or higher diversity. At the same time, a large gap to the AR teacher remains, and quality plateaus as the number of steps increases.

Critical strengths
  1. Simple and original idea. Using the Gumbel-max sampling of an AR model as a source of deterministic noise–text pairs yields straight flow trajectories without reflow or optimal transport solvers.
  2. Sound theoretical grounding. The authors prove that the induced linear paths do not intersect, so the optimal flow is straight, and give a clear optimal transport interpretation.
  3. Clear empirical benefit in the few-step regime. The AR-induced pairing produces straighter paths than random pairing (Fig. 5). At 1–4 steps, GSF outperforms FMLM of the same architecture in generative perplexity and Gradient Moment at higher entropy (Table 2), so the gain does not come from reduced diversity. It also substantially outperforms distilled discrete diffusion models and Gumbel-Distill.
Critical weaknesses
  1. Low absolute generation quality. The student remains far from its AR teacher: at 4 steps, generative perplexity is 99.7 vs. 46.5 and Gradient Moment is 0.73 vs. 0.24 (Table 2), even though the teacher also has higher entropy (5.59 vs. 5.32). The LLM judge itself describes the samples at 1–4 steps as largely incoherent (Appendix H).
  2. Missing comparison with ELF. ELF with progressive distillation (Hu et al., 2026a, Appendix B, Table 4) reports generative perplexity of 136.1 / 68.3 / 34.3 at 1 / 2 / 4 steps under the same evaluation protocol (OpenWebText, L = 1024, GPT-2 Large), with a smaller model (105M vs. 168M parameters). GSF obtains 114.9 / 105.6 / 99.7. Although the two methods use different representation spaces and ELF samples with guidance, these numbers suggest that GSF is not competitive beyond one step.
  3. The many-step regime is not evaluated. The flow map is evaluated quantitatively only up to 4 steps on OpenWebText (8 on TinyStories); for 8–64 steps, only qualitative samples are shown (Tables 14–15). Fig. 5c covers up to 1024 steps, but only for the flow-matching variant and only in terms of Gradient Moment. It therefore remains unclear whether GSF continues to improve with more steps and whether it can approach the quality of its AR teacher. Generative perplexity and entropy for GSF at 8–64 or more steps would clarify this.
  4. Only the Euler solver is considered. The evidence that straight paths improve few-step generation (Fig. 5c) is obtained with the Euler solver only. Higher-order solvers reduce discretization error on curved paths, so the advantage over random pairing may shrink. It is also not stated which time schedule is used for FLM in Fig. 5c.
  5. Limited evaluation on TinyStories. Table 4 reports only generative perplexity and entropy, without the GPT-based evaluation of grammar, creativity, and consistency used in the original TinyStories paper (Eldan and Li, 2023)
Questions for authors
  1. Many-step regime. How does GSF behave with a large number of sampling steps, e.g., 64, 128, 256 or more? Could you report generative perplexity, entropy, and Gradient Moment in this regime, and does the proposed method improve quality there compared with FMLM and the standard FLM?
  2. Other ODE solvers. Does the advantage of the AR-induced pairing persist with ODE solvers other than Euler, e.g., Heun or DPM-Solver?
  3. TinyStories evaluation. Could you provide a GPT-based evaluation of the generated stories, rating grammar, creativity, and consistency, as in the original TinyStories paper (Eldan and Li, 2023)?
  4. LLM-judge comparison with AR. The LLM judge prefers GSF over FMLM in 71–80% of comparisons at 1–2 steps, yet its own comments describe both texts as largely incoherent. Could you report pairwise comparisons of GSF against its AR teacher, or against an AR model of comparable inference cost, to show how the quality of GSF compares with autoregressive generation?
8 comments
AuthorsOpenReview ↗

Official Comment by Authors for Reviewer JgHK (1/6)

We sincerely appreciate Reviewer JgHK's thoughtful assessment and constructive comments. We are especially grateful for the reviewer's acknowledgment of the following strengths of our work:

  • Simple and original idea
  • Sound theoretical grounding
  • Clear empirical benefits in few-step generation

We address the reviewer's concerns as follows (We have highlighted all changes in the manuscript in purple\textcolor{purple}{\text{purple}}.).


W1 & Q4 [Comparison to AR]:

(Weakness 1): Low absolute generation quality. The student remains far from its AR teacher: at 4 steps, generative perplexity is 99.7 vs. 46.5 and Gradient Moment is 0.73 vs. 0.24 (Table 2), even though the teacher also has higher entropy (5.59 vs. 5.32). The LLM judge itself describes the samples at 1–4 steps as largely incoherent (Appendix H).

(Question 4) LLM-judge comparison with AR. The LLM judge prefers GSF over FMLM in 71–80% of comparisons at 1–2 steps, yet its own comments describe both texts as largely incoherent. Could you report pairwise comparisons of GSF against its AR teacher, or against an AR model of comparable inference cost, to show how the quality of GSF compares with autoregressive generation?

[Clarification on the goal] The primary goal of distillation is not to exactly match the teacher's absolute performance, but to improve the efficiency–performance frontier, as shown in Figure 1. Our main empirical contributions are (1) improving upon the Gumbel-Distill (Zhang et al., 2026) baseline, which also uses an AR teacher, and (2) outperforming FMLM (Lee et al., 2026) through AR teacher supervision with the same student parametrization. Moreover, GSF outperforms compute-matched AR baselines as shown in Figure 1 & Figure 11.

[Pairwise comparison against AR model] Using the same Gemini evaluation as before, we compare GSF with 2 and 4 steps against the lightest AR model in Figure 1 (2 layers, 60 hidden dimensions, 3.1M parameters, 1024 steps). GSF is 9.45× and 4.72× more efficient in parameters × NFE, respectively. However, as noted by Reviewer xpCM, this metric does not account for AR-specific inference optimizations such as KV caching. We therefore also measured wall-clock latency on TPU at batch size 16, a setting favorable to KV-cached AR models, which are memory-bandwidth-bound and benefit from larger batches (for reference, SDTT (Deschenaux and Gulcehre, 2025) used batch size 8 for AR–diffusion comparisons). Even in this AR-favorable setting, GSF achieves 2.41× and 1.53× wall-clock speedups at 2 and 4 steps. GSF-2 achieves a 64.3% overall and 71.3% coherence win rate, and GSF-4 achieves 84.4% and 81.0%. Thus, at lower inference cost, GSF produces samples that the judge rates as more coherent than those of the compute-matched AR model.

MethodParams × Step EfficiencyWall-clock Speed-upCoherenceOverall
GSF-29.45×2.41×71.3%64.3%
GSF-44.72×1.53×81.0%84.4%
AuthorsOpenReview ↗

Official Comment by Authors for Reviewer JgHK (2/6)

W1 & Q4 [Comparison to AR] (continued):

[Comparison to AR under a shared-teacher distillation setting] Beyond comparisons with AR models trained directly on data, we consider a controlled setting in which AR and GSF students are distilled from the same 1.5B AR teacher. This lets us compare the efficiency–performance trade-off with the teacher supervision held fixed. Following CSD (Kim et al., 2026), we distill a 124M AR student and a 170M GSF student on task-agnostic instruction-following data (Dolly-15K), and evaluate them on the Dolly test split and four additional instruction-following benchmarks (Self-Inst, Vicuna, Sinst, UnNI). Latency is measured on a single A100 40GB GPU with batch size 8, using KV caching for the AR models. At 16 steps, GSF is 2.43× faster than the AR (CSD) student (92.0 vs. 223.7 ms) while achieving higher average ROUGE-L (21.87 vs. 21.34). At 32 steps, GSF remains 1.22× faster (184.0 ms) and further improves to 22.26, approaching the teacher's 23.33 and outperforming the AR student on four of the five benchmarks. These results show an improved latency–quality trade-off under a shared-teacher distillation setting.

ModelParamsStepsLatency (ms) ↓DollySelf-InstVicunaSinstUnNIAvg.
Teacher1.5B256668126.5814.2016.5027.6131.7723.33
AR (KL)124M256223.723.6910.1814.9916.7119.4917.01
AR (CSD)124M256223.725.1613.1815.5725.5127.2921.34
GSF170M846.024.9412.1017.0623.9825.8020.78
GSF170M1692.025.6912.5417.2826.1327.7021.87
GSF170M32184.026.0012.7616.9227.0628.5722.26

[Gap to the teacher] Our scaling analysis (response to Reviewer M6ab, W1 & Q3) shows that, for pre-training on OWT, the gap to the teacher is mainly due to student capacity: increasing student size reduces 4-step Gen-PPL from 159.4 to 103.3 (75.7 with longer training), whereas larger teachers give no gain. We hypothesize that the gap is smaller on downstream tasks, which are the common setting in distillation, because the student can focus its limited capacity on a narrower task distribution. Consistent with this, in the instruction-following setting above, 32-step GSF reaches 95% of the teacher's average ROUGE-L (22.26 vs. 23.33).

We have included these results in Table 7 of the revised manuscript, provided further details on instruction following in Appendix C, and reported the wall-clock latency–performance trade-off in Figure 12. Qualitative samples are provided in Appendix J (Table 23).

AuthorsOpenReview ↗

Official Comment by Authors for Reviewer JgHK (3/6)

W2 [Comparison to ELF+PD]:

(Weakness 2): Missing comparison with ELF. ELF with progressive distillation (Hu et al., 2026a, Appendix B, Table 4) reports generative perplexity of 136.1 / 68.3 / 34.3 at 1 / 2 / 4 steps under the same evaluation protocol (OpenWebText, L = 1024, GPT-2 Large), with a smaller model (105M vs. 168M parameters). GSF obtains 114.9 / 105.6 / 99.7. Although the two methods use different representation spaces and ELF samples with guidance, these numbers suggest that GSF is not competitive beyond one step.

We would first like to clarify two points regarding the comparison with ELF (Hu et al., 2026).

[Fairness of training data] In Tables 2–5, we compare methods trained entirely on the same datasets (OWT, LM1B, TinyStories, and OWT, respectively), for both teacher and student. In contrast, ELF uses a pretrained T5 encoder (Raffel et al., 2020), trained primarily on the Colossal Clean Crawled Corpus (C4, around 154B tokens), which is over 17× larger than OWT (around 9B tokens). T5 also benefits from supervised datasets such as GLUE and SuperGLUE. Indeed, the encoder ablation in ELF (Fig. 5a) shows that the pretrained T5 embedding space improves performance over an encoder trained on OWT. Thus, the comparison is not controlled for training data. In addition, ELF samples with self-conditioning guidance, which lowers Gen. PPL at the cost of entropy (5.16 vs. 5.32 for GSF at 4 steps), whereas standalone GSF uses no guidance.

[Concurrent work policy] Although ELF (NeurIPS 2026) is considered concurrent work under the ICLR review policy, we agree that comparing with recent advances is important. We therefore conducted additional experiments integrating GSF into ELF+PD.

[Combining GSF with ELF+PD] GSF is complementary to the design choices of ELF. Specifically, GSF modifies the noise–data coupling during training, whereas ELF changes the representation space. To verify this complementarity, we integrated GSF into ELF-B while retaining the same 105M architecture, frozen T5-small encoder, optimizer, and base-model training budget as ELF, followed by progressive distillation.

Concretely:

  • ELF training: We replace the independent Gaussian noise in zt=tx+(1−t)σϵz_t = tx + (1-t)\sigma\epsilon with noise sampled from the GSF posterior conditioned on data tokens. This posterior is computed using a frozen AR teacher trained on the same data and tokenizer. A two-level Gumbel race maps the coupling to the 512-dimensional latent space while preserving the exact standard Gaussian noise marginal, N(0,I)\mathcal{N}(0,I). The same coupled noise is used for both denoising and decoding branches.

  • Progressive distillation: We construct the noisy latents ztz_t using the same coupled noise.

  • Inference: The sampling procedure remains unchanged, starting from N(0,I)\mathcal{N}(0,I) with the original SDE sampler and self-conditioning guidance. The AR teacher is not required at inference time.

[Additional results] We report generative perplexity (Gen. PPL) and entropy on OpenWebText (L = 1024), evaluated using GPT-2 Large:

StepsELF + PDGSF + ELF + PDGSF + ELF + PD (Entropy-matched)
1136.1 / 5.26100.3 / 5.18120.0 / 5.26
268.3 / 5.2452.8 / 5.1959.1 / 5.24
434.3 / 5.1628.4 / 5.0734.2 / 5.16

The GSF+ELF+PD results use ELF's original sampling settings (γ=1.5\gamma=1.5, self-conditioning CFG = 2.5). Since GSF yields lower entropy under these settings, we additionally match the entropy of ELF+PD by sweeping γ\gamma and the guidance scale. At matched entropy, GSF+ELF+PD improves Gen. PPL by 16.1 at 1 step and 9.2 at 2 steps, while matching ELF+PD at 4 steps, with the same model size (105M). These results demonstrate that GSF can complement ELF and PD without increasing inference-time model size or requiring an AR teacher during sampling. We provide more detailed explanations and results in Appendix G.

Reviewer_xpCMOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper converts an AR language model into a few-step flow map applied to one-hot sequences. The essential insight is that ancestral Gumbel-max sampling associates each Gumbel noise sequence with exactly one token sequence, and the linear interpolants between these pairs do not cross (Theorem 1, Proposition 3). It follows that the population-optimal flow under this coupling is straight. The noise is transformed into a Gaussian distribution using a monotone map (Proposition 2), the pairs are constructed by performing Gumbel posterior sampling on actual data, and these coupled pairs are used to guide the diagonal FMLM loss. On OpenWebText GSF outperforms a matched reimplementation of FMLM at 1, 2 and 4 steps (one-step GM 0.94 versus 1.30), together with further results on LM1B, TinyStories, prefix completion and XSum.

Critical strengths
  1. The comparison with FMLM-re is well controlled since it employs the same backbone, has almost the same number of parameters (167.8M compared to 169.7M) and the same training budget. For every step count shown in Table 2, GSF achieves better results on both GM and Gen PPL, and the PPL-entropy curve for GSF in Figure 6 lies above that of FMLM-re.

  2. The appendix is open about a number of points that many papers would omit: the direct noise-to-data model, the gap between the student and the teacher, the result that D-MMD outperformed GSF after four steps, and the direct verification of the mismatch between the prior and posterior.

Critical weaknesses
  1. Nothing isolates the coupling from all the other things that have changed. Moving from FMLM-re to GSF changes at least four things at the same time:
    • the noise is coupled to the data;
    • the diagonal target is the data token, not an FLM teacher's denoiser;
    • the student starts from the AR teacher's weights;
    • the timetable is different.

The paper gives the straight path a mention and "a better teacher" is listed in section 3.2 as well. There's no run with the exact GSF setup and independent Gaussian noise. Table 8 seems to indicate that this is important. Without AR init the one step PPL shoots up to 145.4 (worse than FMLM-re's 140.0) though GM does improve to 0.81. Figure 5 is as close as we get to a clean test and I still couldn't tell whether GSF-FM was initialised from the AR teacher whereas FLM wasn't.

  1. The theory talks about the target coupling, not about the model that's being trained. Theorem 1 states that with sufficient capacity the optimal velocity under the Gumbel coupling is straight. That's just an existence result. It tells you nothing about whether that map is in fact learnable and in this case it pretty much isn't. If I've got Appendix D correct then one step GSF only matches the coupled target on 6.68% of tokens. The trained model is not in any real sense following the AR induced path. It's producing different text from the same noise.

The only indication that the learned paths are any straighter is in Figure 5b. That figure uses a CE-scale proxy at softmax temperature 0.3 (this was selected as it gives the best straightness - see Figure 9). Even then S=0.70 not 0. Straight doesn't necessarily mean easy to learn either as at small t the target velocity is jumping across Laguerre cell boundaries in RL×∣V∣\mathbb{R}^{L\times|V|}.

  1. The assertion that GSF outperforms AR models in terms of efficiency (Figs 1 and 10) is contingent upon how the AR cost is defined. The NFE x Params figure charges a 1024-step AR model 1024 full forward passes (i.e. one pass for each step), which is how Figure 10 gets to 79.1x. With a KV cache though, each AR step only processes one new token so to generate 1024 tokens you're only doing roughly one forward pass over the full sequence. Using the standard 2 x params x tokens estimate, the AR teacher (same 12-layer backbone as the student) to generate a full sequence would cost about the same as a single GSF step (Figure 8a: 351.4 GFLOPs). That teacher achieves GM 0.24 and PPL 46.5 versus GM 0.94 and PPL 114.9 for one step GSF. GSF's real advantage is sequential latency and that is never measured in the paper at all (no wall-clock times anywhere). For reference the judge themselves describe the one step samples in Table 11 as "largely incoherent".

  2. The main table omits the most compelling baseline. At four steps D-MMD achieves a PPL of 66.1 versus GSF's 99.7 (higher entropy, nearly identical GM) and it's only included in Appendix D (Table 9). The stated reason for its exclusion is the 318.1M parameter count. If however the auxiliary network is only used during training (as is the case with DMD style methods) then the inference cost is the same and D-MMD should be in Table 2. The abstract's assertion that GSF "can outperform current few-step language generation baselines" needs to be qualified to be accurate. On a related point the LLM judge win rate against FMLM at four steps is 52.7% overall and 46.3% on coherence which is a tie.

  3. The ablations quote whichever metric is relevant to the story. In Table 8 coupled off-diagonal sampling is better than the default on both GM (0.89 vs 0.94) and PPL (108.4 vs 114.9). Random initialisation gives the best GM in that table, the case for AR initialisation is entirely based on PPL though. In Table 7 Gaussianisation improves the GM but makes the PPL worse (133.5 to 150.7). That's all perfectly valid in itself, just that the GM and PPL are quite at odds in those tables. There are no error bars and each evaluation uses 1024 samples so I don't know whether a difference of 0.94 to 0.89 is actually significant.

  4. The conditional results are very difficult to interpret. On XSum the reimplemented AR teacher is significantly stronger than the AR baseline from Hu et al. (34.1 vs 30.5 ROUGE-1). GSF therefore starts from a better teacher than the teacher used in the ELF and E2D2 baselines that it's being compared against. FMLM-re gets a ROUGE-2 of 1.2 which looks like a complete failure rather than a baseline.

There's also a fairly minor presentation issue. Equations 6 and 8 sample x1 from p_AR, but Algorithm 1 is trained on real data with posterior noise. Table 10 shows this makes very little difference in practice, but the main text should make it clear what is actually being done.

Questions for authors
  1. Can you apply GSF using independent Gaussian noise while keeping everything else the same (including the AR initialization, the data targets in L_diag, the schedule, and the budget)? Also, was GSF-FM in Figure 5 initialized using the AR teacher?

  2. Since there is 6.68% token agreement as stated in Appendix D, what is your view of the connection between Theorem 1 and the trained model?

  3. What are the wall-clock latencies and the FLOPs associated with KV-cached AR generation when compared to GSF on the same hardware?

  4. Is the auxiliary network used by D-MMD when carrying out inference? If not, then there is no reason to exclude it from Table 2.

11 comments
AuthorsOpenReview ↗

Official Comment by Authors for Reviewer xpCM (1/8)

We are grateful to Reviewer xpCM for your thoughtful evaluation and constructive feedback. We especially appreciate the reviewer’s acknowledgment of the following strengths:

  • Well-controlled and fair comparison with FMLM
  • Consistent empirical improvements across sampling steps
  • Transparent and comprehensive reporting of experimental findings and limitations

We address the reviewer's concerns as follows (We have highlighted all changes in the manuscript in purple\textcolor{purple}{\text{purple}}.).


W1 & Q1 [Isolating the Effect of Coupling]:

(Weakness 1): Nothing isolates the coupling from all the other things that have changed. Moving from FMLM-re to GSF changes at least four things at the same time:

  • the noise is coupled to the data;
  • the diagonal target is the data token, not an FLM teacher's denoiser;
  • the student starts from the AR teacher's weights;
  • the timetable is different. The paper gives the straight path a mention and "a better teacher" is listed in section 3.2 as well. There's no run with the exact GSF setup and independent Gaussian noise. Table 8 seems to indicate that this is important. Without AR init the one step PPL shoots up to 145.4 (worse than FMLM-re's 140.0) though GM does improve to 0.81. Figure 5 is as close as we get to a clean test and I still couldn't tell whether GSF-FM was initialised from the AR teacher whereas FLM wasn't.

(Question 1) Can you apply GSF using independent Gaussian noise while keeping everything else the same (including the AR initialization, the data targets in L_diag, the schedule, and the budget)? Also, was GSF-FM in Figure 5 initialized using the AR teacher?

[Isolating the Effect of Coupling] First, we clarify that the original FMLM recipe initializes the student from the FLM teacher and zero-initializes the new time-conditioning parameters, thereby exactly matching the diagonal target at initialization. In contrast, our AR initialization does not provide this advantage.

To isolate the effect of coupling, we conducted a controlled ablation across four configurations, combining weight initialization {scratch, AR} and time schedule {linear, coupled decoding error rate (DER)}. Within each configuration, we kept all other settings fixed, including the data-token target in diagonal loss and the training budget (500K steps), varying only the noise sampling strategy: independent Gaussian noise or AR-Gumbel posterior sampling.

Weight InitTime ScheduleNoise Coupling1-step GM ↓2-step GM ↓4-step GM ↓
ScratchLinearIndependent2.251.701.89
ScratchLinearAR-Gumbel Posterior1.530.960.90
ScratchCoupled DERIndependent2.902.302.52
ScratchCoupled DERAR-Gumbel Posterior1.231.111.33
AR-initLinearIndependent2.331.741.99
AR-initLinearAR-Gumbel Posterior1.050.740.73
AR-initCoupled DERIndependent3.242.672.90
AR-initCoupled DERAR-Gumbel Posterior0.940.740.73

Across all configurations, our method achieves substantially better gradient moment (GM) scores. Note that the last row corresponds to the default GSF setting in Table 2. These controlled results identify AR-induced noise–data coupling as a key contributor to the performance improvement.

[Clarification on Figure 5] GSF-FM in Figure 5 was initialized from the AR teacher. We additionally trained both GSF-FM and FLM from scratch using the same linear time schedule, and GSF-FM still outperformed FLM:

Gradient Moment (GM ↓, 1000K training steps)

ModelN=4N=16N=64N=256N=1024
FLM (Linear, Scratch)6.171.981.661.501.47
GSF-FM (Linear, Scratch)1.751.581.421.351.31

[Correction to the initialization ablation] While preparing these experiments, we identified implementation errors in the weight initialization and time scheduling of the original initialization ablation. We corrected the affected results in Table 9 of the revised manuscript (Table 8 in the original submission). Specifically, the previously reported 1-step Gen-PPL of 145.4 and GM of 0.81 for the scratch-initialized configuration have been updated to 171.5 and 1.23, respectively. Finally, we clarify that FMLM-re benefits from exact diagonal-target matching at initialization through its FLM teacher, whereas our AR initialization does not provide this advantage. Thus, the controlled ablations above isolate the effect of coupling, while the main table provides a complementary comparison under each method's respective teacher-initialized setting.

AuthorsOpenReview ↗

Official Comment by Authors for Reviewer xpCM (2/8)

W2&Q2 [Optimal Straightness vs. Learned Model]:

(Weakness 2): The theory talks about the target coupling, not about the model that's being trained. Theorem 1 states that with sufficient capacity the optimal velocity under the Gumbel coupling is straight. That's just an existence result. It tells you nothing about whether that map is in fact learnable and in this case it pretty much isn't. If I've got Appendix D correct then one step GSF only matches the coupled target on 6.68% of tokens. The trained model is not in any real sense following the AR induced path. It's producing different text from the same noise. The only indication that the learned paths are any straighter is in Figure 5b. That figure uses a CE-scale proxy at softmax temperature 0.3 (this was selected as it gives the best straightness - see Figure 9). Even then S=0.70 not 0. Straight doesn't necessarily mean easy to learn either as at small t the target velocity is jumping across Laguerre cell boundaries in RL×∣V∣\mathbb{R}^{L\times|V|} .

(Question 2): Since there is 6.68% token agreement as stated in Appendix D, what is your view of the connection between Theorem 1 and the trained model?

[Clarification on Theorem 1] Theorem 1 is not merely an existence result. A typical existence result shows that an optimal model exists without specifying how training reaches it. For example, Theorem 4.2 of IMDM [Yoo et al., 2026] constructs parameters that attain zero factorization error, and the authors note that this does not imply that the trained model realizes them; the construction is not the minimizer of the training objective they optimize. In contrast, Theorem 1 characterizes the minimizer of the objective we actually optimize during training (Eq. 6). We do not claim that our trained model produces perfectly straight trajectories. The purpose of Theorem 1 is to characterize the straightness of our target coupling paths.

[Comparison with theoretical results from prior work on straight flows] Even at the level of the optimum, we believe that obtaining straight paths from a single training objective is an important contribution to learning straight flows. Rectified Flow [Liu et al., 2023, Theorem D.7] shows that, when each reflow round is trained optimally on the coupling generated by simulating the previous learned flow, the minimum straightness over K rounds decays as O(1/K), reaching zero only in the limit. In contrast, Theorem 1 shows that, under the same assumption of optimal training, the minimizer of a single flow matching objective is already straight, without any iterative reflow. This is possible because the AR teacher provides a non-intersecting coupling in closed form through Gumbel-max sampling.

[New theoretical results for imperfect optimization] To characterize the trained model, we add Proposition 2 to the revised manuscript. Assuming only that the learned velocity field is K-Lipschitz, as is standard in ODE stability analysis [Bansal et al., 2026], we prove the following bound for any coupling π:

Proposition 2 (Straightness and flow matching loss). Let (x0,x1)∼π(\mathbf{x} _0,\mathbf{x} _1)\sim\pi for any coupling π\pi with finite second moments, let vθ(⋅,t)\mathbf{v} _\theta(\cdot,t) be KK-Lipschitz for all t∈[0,1]t\in[0,1] (K>0K>0), and let {xtθ}\{\mathbf{x}^\theta _t\} solve dxtθ=vθ(xtθ,t)dtd\mathbf{x}^\theta _t=\mathbf{v} _\theta(\mathbf{x}^\theta _t,t)dt with x0θ=x0\mathbf{x}^\theta _0=\mathbf{x} _0. Then

\le C _K\mathcal{L} _{\mathrm{FM}}(\theta;\pi), \qquad C _K := 2+\frac{1}{2}(e^{2K}-1-2K).$$ The bound applies to any trained model without assuming exact optimization. Combined with Theorem 1, it recovers zero straightness at the optimum. The coupling determines the minimum attainable FM loss, $\int \mathbb{E}[\mathrm{Var}(x_1-x_0\mid x_t)] dt$. Our non-intersecting coupling achieves zero minimum loss, whereas intersecting couplings, such as the independent coupling used by FLM, have a positive loss floor. Thus, our coupling allows the straightness bound to approach zero as the training loss decreases. This also connects the theory to the learned model with $S=0.70$, rather than only the ideal case of $S=0$. Empirically, GSF-FM achieves lower training loss (Fig. 5a) and straighter trajectories than FLM across all evaluated softmax temperatures (Figs. 5b and 9). FLM has a straightness value of $S=1.82$ with the best temperature. As clarified in Fig. 9, we report temperature 0.3 because it yields the best straightness for both methods.
AuthorsOpenReview ↗

Official Comment by Authors for Reviewer xpCM (3/8)

W2&Q2 [Optimal Straightness vs. Learned Model] (continued):

[Benefits of a straight underlying flow]

OT-CFM (Tong et al., 2024) defines the objective variance:

\mathbb{E} _{t \sim U(0,1), (x _0,x _1) \sim \pi} \left\|\left\| (x _1-x _0)-v^*(x _t,t) \right\|\right\|^2.$$ and observes that couplings with lower OV converge faster. Since $v^*$ is the conditional expectation of $x_1-x_0$ given $x_t$, the FM loss decomposes as: $$\mathcal{L} _{FM}(\theta;\pi) = E[\\|v _\theta(x _t,t)-v^*(x _t,t)\\|^2]+OV(\pi)$$ Under the independent coupling, $OV(\pi)>0$ because multiple pairs pass through the same $x_t$, so each sampled target deviates from $v*$ by zero-mean noise. Under our non-intersecting coupling (Theorem 1), $x_1 - x_0$ is $v*$ almost surely, so $OV (p_{\text{AR}})=0$. Each sequence is then supervised directly by the marginal velocity, and its loss reflects only the approximation error $|v_\theta - v^*|^2$. Because the convergence of stochastic gradient methods depends on loss variance (Bottou et al., 2018), we also track the variance of the per-sequence FM loss within each batch (Figure 13): GSF-FM stays lower than FLM after the first ~5k steps (≈3.8 vs. ≈5.3 at the end of training). Details are in Appendix H. Regarding the 6.68% token agreement (Q2), reproducing the exact coupled sample from a given noise is not what drives generation quality. The direct one-step mapping in Appendix D achieves higher agreement (9.92%) but a GM above 30. We therefore view the coupling as providing straight trajectories and zero conditional target variance, which may facilitate optimization, rather than a pointwise target to be memorized. In the semi-discrete setting, a nonconstant deterministic mapping from continuous noise to discrete tokens must be discontinuous, regardless of the coupling. For GSF, the target velocity can change sharply near Laguerre cell boundaries at small $t$. However, our empirical results show that GSF-FM achieves lower training loss (Fig. 5a) and better GM (Fig. 5c) than FLM, suggesting that these discontinuities do not prevent effective learning.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the reviewers for their constructive feedback. Reviewers M6ab and JgHK highlighted the originality of the core idea: Gumbel-max sampling from a pretrained autoregressive (AR) model gives a closed-form noise-text coupling whose straight paths do not intersect, so a single flow-map training stage suffices without reflow or optimal-transport solvers. Reviewer xpCM highlighted our controlled comparison with FMLM and our transparent reporting of limitations.

We have uploaded a revised manuscript (changes in purple\textcolor{purple}{\text{purple}}). The main additions are:

1. Isolating the coupling (xpCM, JgHK): With initialization, time schedule, training targets, and budget held fixed, replacing independent noise with our AR-Gumbel coupling improves few-step generation in every configuration, including training from scratch. The advantage persists with a higher-order (Heun) solver.

2. Theory for the trained model (M6ab, xpCM): A new proposition bounds the straightness of the learned trajectories by the training loss. Our coupling removes the irreducible part of this loss, so lowering the loss directly yields straighter paths. Scaling experiments show that student capacity, not teacher quality, is the main cause of the remaining gap to the teacher.

3. Wall-clock latency against KV-cached AR decoding (JgHK, xpCM): We agree that our original cost metric overstated the speedup over KV-cached AR models, so we now measure wall-clock latency on the same hardware. Few-step GSF reaches a latency range that no AR model in our sweep reaches; in that range it is faster than even our smallest (3M-parameter) AR model, and an LLM judge rates its samples as more coherent. The full AR teacher remains higher quality but is much slower.

4. Instruction following (M6ab, JgHK, xpCM): We distill the same 1.5B AR teacher into a GSF student and an AR student and evaluate both on five instruction-following benchmarks. GSF slightly exceeds the AR student's average score while running more than twice as fast, and with more steps it comes within about 5% of the teacher.

5. Concurrent methods (JgHK, xpCM): Our coupling only changes how training pairs are built, so it plugs directly into ELF with progressive distillation and improves its 1- and 2-step generation at matched diversity, with no extra inference cost. Against D-MMD, temperature-adjusted GSF performs better at equal or lower inference compute.

We also narrowed the abstract's claim to the 1-2-step regime, corrected an implementation error in one ablation (the corrected result further supports AR initialization), and added evaluation-seed error bars.

We hope these additions address your concerns, and we would be grateful if you would take them into account in your final evaluation. We are happy to answer any remaining questions during the discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.