acceptodds
Under review as a conference paper at ICLR 2027

Self-Conditioned Flow Map Language Models via Fixed-Point Flows

Abstract

Self-conditioning is a core technique that enhances continuous flow-based language models, where the model learns to denoise generated text by conditioning on its own denoising estimate. While empirically successful, its performance improvements are poorly understood. Moreover, there is growing interest in the use of few-step generators based on flow maps, for which how to leverage self-conditioning is unclear. Here, we show that flow language models with self-conditioning perform a fixed-point iteration that improves generation through iterative refinement. We use this viewpoint to formulate fixed-point flows, a two-dimensional class of selfconditioned flows, where the first dimension represents the flow process and the second represents the fixed-point iteration. We show that fixed-point flows define valid flow maps, and show that they can be distilled from self-conditioned flow models by compressing both fixed-point iterations and the flow process, the former with fixed-point distillation and the latter with flow map distillation. Our resulting flow map language model, FMLM*, outperforms state-of-the-art self-conditioned models and few-step models in one- and few-step generation on OpenWebText.

Primary area:
generative models
Keywords:
Few-step language models, self-conditioning, flow language models

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.60 over 5 reviews, above 99% of the others.
Reviewer_s1dhOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper interprets self-conditioning in continuous flow language models as a fixed-point iteration of the denoiser over its own prediction. Replacing the self-conditioning state with its fixed point yields “fixed-point flows”: an ordinary ODE whose velocity depends only on (t, x), which admits a well-defined flow map. Under this view, conventional self-conditioned sampling is a warm-started, single-iteration approximation of the fixed point. The authors show that ELF and LangFlow approximately learn fixed-point iterations, and distill ELF into a self-conditioning-free flow (ELF*) and a flow map (FMLM*), using either a two-stage (offline) or one-stage (online) recipe. FMLM* achieves strong few-step results on OpenWebText.

Critical strengths
  1. Treating self-conditioning as a fixed-point iteration turns it into an ordinary flow, which opens the door to flow-map distillation. FMLM* shows this leads to strong few-step quality.
  2. The fixed-point behavior isn’t specific to one model: the authors show it on LangFlow as well as ELF.
  3. Strong few-step results on OpenWebText.
  4. I appreciate offering both online and offline distillation, a practical choice between training cost and quality.
  5. The paper is clean, well structured, and coherent.
Critical weaknesses
  1. Table 3 (and online vs. offline in Table 1) compares FMLM* variants at different entropy levels, e.g., 73.86 / 5.35 vs. 80.26 / 5.44 at 4 steps. Since a small entropy drop can easily buy lower gPPL, these comparisons aren’t conclusive. Fig. 3 points the same way: the many-FPI frontier nearly matches warm-started single-FPI sampling, just at lower entropy. Frontier or matched-entropy comparisons would settle this.
  2. Per-sample entropy can’t detect reduced diversity across samples, which is a natural concern for fixed-point iteration and distillation. An inter-sample diversity metric (e.g., Self-BLEU) and a distributional metric (e.g., MAUVE) would help here.
  3. All experiments are unconditional. A conditional task, e.g. training on TinyGSM and evaluating on GSM8K, would show the approach goes beyond open-ended generation.
Questions for authors
  1. Does the benefit of more FPIs hold at matched entropy?
  2. Could you show FMLM* frontiers for different numbers of FPIs?
4 comments
AuthorsOpenReview ↗

Response to Reviewer s1dh (1/2)

W1-1. Table 3 (and online vs. offline in Table 1) compares FMLM$^\star$ variants at different entropy levels, e.g., 73.86 / 5.35 vs. 80.26 / 5.44 at 4 steps. Since a small entropy drop can easily buy lower gPPL, these comparisons aren’t conclusive. … Frontier or matched-entropy comparisons would settle this.

Q2. Could you show FMLM$^\star$ frontiers for different numbers of FPIs?

We thank the reviewer for this comment and agree that matched-entropy comparisons are more conclusive. Following the reviewer's suggestion, we plot the gPPL-entropy frontiers of online FMLM$^\star$ with different numbers of FPIs and of offline FMLM$^\star$ at one, two, and four steps, sweeping the self-conditioning guidance weight at inference. Where the frontiers overlap, they allow a comparison at matched entropy. The number of FPIs refers to the teacher evaluations in online distillation, and FMLM$^\star$ uses no FPIs at inference (Appendix C.3).

As shown in Figure 12 in the revised Appendix D.3, with four steps, where the frontiers overlap over a wide range of entropy, more FPIs give lower gPPL at the same entropy. Offline FMLM$^\star$ achieves the lowest gPPL, and the frontier of online FMLM$^\star$ approaches it as the number of FPIs grows. With one and two steps, the guidance sweep changes entropy only slightly, so the frontiers overlap less, but they show the same trend.

In the revised manuscript, we have added these frontiers to Appendix D.3 (Figure 12) and will revise the comparison between online and offline FMLM$^\star$ in Section 4.2 to refer to them.

W1-2. Fig. 3 points the same way: the many-FPI frontier nearly matches warm-started single-FPI sampling, just at lower entropy.

Q1. Does the benefit of more FPIs hold at matched entropy?

As the reviewer suggested, we compare generation quality across numbers of FPIs at a matched entropy of 5.30 (Table A). We use cold start, where the iteration at every step starts from zero, so that the number of FPIs counts all iterations at each step, as in our distillation (Equation (18)). We also include conventional self-conditioned sampling, which uses warm start with a single FPI, as a reference.

As shown in Table A, gPPL with cold start decreases consistently as the number of FPIs increases for both models, so the benefit of more FPIs holds at a matched entropy. Conventional sampling reaches a gPPL close to that of cold start with three FPIs, consistent with the reviewer's observation on Figure 3, while many FPIs still achieve a lower gPPL. We attribute this to the initialization provided by warm start (Section 3.3): the estimate from the previous flow step initializes the iteration close to the fixed point, so that fewer FPIs are needed (Appendix B.5).

We have added this comparison to Appendix D.2 (Table 4) of the revised manuscript and will clarify in Section 4.1 that Figure 3 compares the two initializations.

Table A. gPPL at a matched entropy of 5.30 (32 steps for ELF and 64 for LangFlow, without damping). Warm start with a single FPI corresponds to conventional self-conditioned sampling.

Warm start, 1 FPICold start, 1 FPI2 FPIs3 FPIs4 FPIs5 FPIs
ELF36.5778.5041.9135.4933.9833.48
LangFlow52.6180.4562.2354.4849.4647.02
AuthorsOpenReview ↗

Response to Reviewer s1dh (2/2)

W2. Per-sample entropy can’t detect reduced diversity across samples, which is a natural concern for fixed-point iteration and distillation. An inter-sample diversity metric (e.g., Self-BLEU) and a distributional metric (e.g., MAUVE) would help here.

We agree with the reviewer that per-sample entropy cannot detect reduced diversity across samples. Following the reviewer's suggestion, we measure MAUVE and Self-BLEU for the teacher across FPIs and for the distilled models.

As shown in Table B, MAUVE increases consistently with the number of FPIs (from 0.19 to 0.81 for ELF), while Self-BLEU moves from below that of the data to slightly above it. The change in Self-BLEU is small compared to the gain in MAUVE, suggesting that the gain from more FPIs does not come from a loss of diversity.

As shown in Table C, FMLM$^\star$ achieves a much higher MAUVE than all baselines at every step count, with Self-BLEU slightly above that of the data and well below that of FMLM. Some baselines have Self-BLEU below that of the data but low MAUVE, as does ELF with a single FPI in Table B. This suggests that their samples share fewer n-grams because they are less coherent rather than more diverse. Overall, fixed-point iteration and distillation bring the samples substantially closer to the data distribution in MAUVE while keeping Self-BLEU near that of the data.

We will add these results to Appendix D in the next revision.

Table B. MAUVE and Self-BLEU across FPIs with cold-start sampling (1024 samples; 32 steps for ELF and 64 for LangFlow).

ELFFPI=1FPI=2FPI=3FPI=4FPI=5Data
MAUVE0.1920.4670.6910.7640.8071.000
Self-BLEU0.2450.2670.2900.3050.3150.274
LangFlowFPI=1FPI=2FPI=3FPI=4FPI=5Data
MAUVE0.8510.9190.9400.9530.9561.000
Self-BLEU0.2700.2810.2940.3000.3060.282

Table C. MAUVE and Self-BLEU at 1 / 2 / 4 sampling steps for the models in Table 1 (1024 samples).

1 / 2 / 4 stepsMAUVESelf-BLEU
Duo + DCD0.055 / 0.063 / 0.1500.060 / 0.156 / 0.271
Duo + Di4C0.054 / 0.075 / 0.1660.406 / 0.470 / 0.418
MDLM + SDTT0.053 / 0.057 / 0.0900.200 / 0.191 / 0.265
MDLM + Di4C0.053 / 0.057 / 0.1070.196 / 0.205 / 0.292
FMLM0.078 / 0.165 / 0.3510.401 / 0.397 / 0.399
FMLM$^\star$ (offline)0.692 / 0.884 / 0.8650.322 / 0.324 / 0.349
FMLM$^\star$ (online)0.679 / 0.883 / 0.8820.312 / 0.316 / 0.339
Data1.0000.282

W3. All experiments are unconditional. A conditional task, e.g. training on TinyGSM and evaluating on GSM8K, would show the approach goes beyond open-ended generation.

As the reviewer suggested, we evaluate our method on a conditional generation task. We train an ELF teacher on TinyGSM, distill it into FMLM$^\star$ with online distillation (8 FPIs), and evaluate both zero-shot on GSM8K. As shown in Table D, FMLM$^\star$ outperforms the teacher and baselines at every matched step count. This suggests that our distillation extends to conditional generation.

We have added this result to Appendix D.4 (Table 7) of the revised manuscript.

Table D. GSM8K accuracy (%) of models trained on TinyGSM, following the setting of [1, 2, 3, 4]. The teacher uses warm-start sampling. †^\dagger denotes results taken from the original papers.

Sampling steps124
FMLM+ (Init)†^\dagger [3]0.30.75.1
MDLM$^\dagger$ [3]0.00.00.1
ELF (teacher)0.080.151.82
FMLM$^\star$ (ours)1.142.966.22

[1] Kim et al. "Stop Training for the Worst: Progressive Unmasking Accelerates Masked Diffusion Training." 2026.

[2] Deschenaux and Gulcehre. "Language Modeling with Hyperspherical Flows." 2026.

[3] Agarwal et al. "Posterior Refinement: Fast Language Generation via Any-Order Flow Maps." 2026.

[4] Hu et al. "ELF: Embedded Language Flows." 2026.

Reviewer_s1dhOpenReview ↗

Thank you for the detailed response and for running the additional experiments.

Thank you for the detailed response and for running the additional experiments. I especially appreciate the GSM8K evaluation, which directly addresses my concern about conditional generation.

I still have a question about the matched-entropy comparison in Table 4. More fixed-point iterations lower gPPL at an entropy of 5.30. Could the authors explain why they chose this value and whether the improvement holds closer to the OpenWebText data entropy of 5.44?

I would also qualify the claim about diversity. MAUVE improves with more iterations, but the samples also share more phrases. For ELF, Self-BLEU rises from 0.245 to 0.315, above the real-data value of 0.274. This suggests that the improvement may come with some loss of lexical diversity across samples. Overall, I’ve decided to keep my original rating.

We thank the reviewer for the follow-up, and we are glad that the GSM8K evaluation addressed the concern about conditional generation.

Matched entropy. The entropy value 5.30 came from one of our sampling configurations used during development, which we set as a reference with no particular intent.

Following the reviewer's request, we repeat the comparison at the data entropy of 5.44 (Table A). We find that the trend in Table 4 holds at entropy 5.44, where the gPPL improves with every additional FPI for both models.

Lexical diversity. We agree that more FPIs may come with some loss of lexical diversity, as Self-BLEU increases beyond that of the data. Yet, the MAUVE score, which measures distributional diversity with respect to true data, improves consistently with every additional FPI, in line with the trend in Table A. As such, we believe that although some lexical phrases are shared, the diversity of the sample distribution is not substantially reduced.

In the final version, we will add the comparison at 5.44 to Appendix D.2 and distinguish lexical diversity across samples (Self-BLEU) from diversity measured by MAUVE in our claim on diversity.

Table A. gPPL at a matched entropy of 5.44 (the data entropy). The samples are generated with 32 steps for ELF and 64 for LangFlow, without damping. Warm start with a single FPI corresponds to conventional self-conditioned sampling.

Warm start, 1 FPICold start, 1 FPI2 FPIs3 FPIs4 FPIs5 FPIs
ELF67.5099.7767.8664.0163.8063.36
LangFlow69.97108.6983.2174.2668.2564.71
Reviewer_8WcaOpenReview ↗
Rating
4
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper analyses the self-conditioning for continuous diffusion language models.

The authors show that iterative application of self-conditioning converges, and the resulting score estimate produces text with a lower gPPL.

Moreover, they show that the score obtained by this iterative process can be learned by a student model, as well as the distilled to the flow model.

Critical strengths

Easy to read and follow.

Surprisingly, self-conditioning might be removed.

Critical weaknesses

See questions

Questions for authors

It would be interesting to see how the denoising score matching loss changes. This will answer the question of whether the fixed-point score is indeed closer to the true one; the current paper only estimates gPPL.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 8Wca

Q1. It would be interesting to see how the denoising score matching loss changes. This will answer the question of whether the fixed-point score is indeed closer to the true one; the current paper only estimates gPPL.

We thank the reviewer for this suggestion. As the reviewer suggested, we measure how close the score at each FPI is to the true score on held-out examples. Since the score is an affine function of the denoiser [3], this distance equals, up to a time-dependent weighting, the MSE between the denoising prediction and the ideal denoiser, the posterior mean E[x1∣It]\mathbb{E}[x_1 \mid I_t] (Proposition 3.1). This MSE equals the denoising score matching loss [4] up to an additive constant.

On general datasets, the posterior mean is a weighted average over all training examples [2], which is costly to evaluate for every noisy input. We therefore use Sudoku-Hard [1], where each puzzle has a unique solution. Given the puzzle, the posterior concentrates on this solution, so the posterior mean equals the ground-truth solution at every tt and can be obtained exactly. We train FLM [5] with self-conditioning following [1] and measure this MSE at each FPI on 256 held-out puzzles, together with the exact-match accuracy of the decoded prediction.

As shown in Figure 10 in the revised Appendix D.1.3, the MSE drops sharply over the first few FPIs and stays low over 200 iterations. The exact-match accuracy rises over the same iterations. In this setting, the fixed-point score is therefore closer to the true score than the initial prediction without self-conditioning.

We have added this analysis to Appendix D.1.3 (Figure 10) of the revised manuscript.

[1] Deschenaux and Gulcehre. "Language Modeling with Hyperspherical Flows." 2026.

[2] Scarvelis et al. "Closed-Form Diffusion Models." 2025.

[3] Karras et al. "Elucidating the Design Space of Diffusion-Based Generative Models." 2022.

[4] Vincent. "A Connection Between Score Matching and Denoising Autoencoders." 2011.

[5] Lee et al. "Flow Map Language Models: One-step Language Modeling via Continuous Denoising." 2026.

Reviewer_8WcaOpenReview ↗

LangFlow Evaluations

Below, I clarify my question. Could authors do the following:

  1. Take the LangFlow or EFL model denoisers DkD^k achieved by kk rounds of self-conditioning.
  2. Compute the denoising score matching loss corresponding to iteration kk as ∑Nwtn∣∣Dk(tn,xtnn)−x1n∣∣2\sum_{N} w_{t^n} ||D^k(t^n, x^n_{t^n}) - x^n_1||^2?

This does not require computing the dataset-wide mean and is a quite cheap monte-carlo approximation of the score matching loss.

We thank the reviewer for the clarification. As suggested, we compute the denoising score matching loss ∑nwtn∥Dk(tn,xtnn)−x1n∥2\sum_n w_{t^n}\|D^k(t^n, x^n_{t^n}) - x^n_1\|^2 for LangFlow on OpenWebText (Table A) with wtn:=SNR(tn)w_{t^n} := SNR(t^n), and take expectation over tnt^n with respect to the training distribution. This gives us the DSM loss over the number of FPIs k.

The result is in Table A. For every k ≥ 2, the loss is lower than that of the initial prediction without self-conditioning (k = 1), so the score is closer to the true score than the initial one. The loss decreases mostly in the first FPIs.

After the first FPIs, the loss plateaus, while the quality of samples (Pred. gPPL, MAUVE) keeps improving, whereas the entropy of the prediction decreases. We conjecture that FPIs steer the prediction towards one realistic text in this setting, whereas the loss favors a posterior mean over plausible texts.

We supplement this interpretation with the example below. At k = 1, the prediction contains fragments that blend plausible tokens, such as "Teslaasonic" from "Tesla" and "Panasonic". At k = 5, it commits to "Tesla" and continues as a more coherent text.


Example (t = 0.647)

Ground truth: "Tesla/Panasonic continue to exhibit a clear advantage on cell and pack technology compared to all peers, on chemistry, cooling and cost. Clear visibility about high-volume cell-sourcing strategies continue to elude traditional manufacturers …"

k = 1: "Tesla (Teslaasonic continue to conduct cell data equipped with Tesla andk technology available to all and, on research, including and manufacturing. Solar vehicles with high-performance data-transition, will to do with traditional cars …"

k = 5: "While [Tesla] continue to exhibit new storage associated with electric and pack cell devices, it is focus on storage, efficiency and manufacturing. Solar vehicles with high-volume battery-transure devices continues to compete with traditional manufacturers …"


We will add these results to Appendix D.1.3 in the final version.

Table A. Denoising score matching loss and prediction statistics across FPIs for LangFlow on the OpenWebText validation set. Predictive entropy is the entropy of the predicted token distribution averaged over positions. Pred. gPPL is measured on the decoded predictions, and MAUVE separately on samples generated with 64 steps from a cold start.

kDSM loss ↓Pred. gPPL ↓Predictive entropyMAUVE ↑
10.337158.873.6830.851
20.321750.013.5960.919
30.319847.893.5090.940
40.320046.103.4490.953
50.321145.263.4050.956
Reviewer_2TWEOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper focuses on self-conditioning in continuous flow language models by adopting a fixed-point angle. The main idea developed here is that self-conditioned denoisers are secretly learning to an iterative refinement process. Each time the model self-condition, the output can move toward a fixed point.

Using this interpretation, they propose fixed-point flows, which separate flow evolution from the fixed-point refinement step (having 2 loops, one inner and one outer loop). Then they distill both into a single flow map model, FMLM*, so you can generate in one or a few steps without needing self-conditioning at inference time.

The paper also provides some theoretical results about the fixed-point angle. It shows fixed-point convergence under contractivity, explain how to build fixed-point flows and works out the flows maps that go with them. Their iterative-refinement interpretation is further supported by experiment on ELF and LangFlow.

The distilled FMLM* model achieves strong few step generation performance on OWT against existing discrete and continuous language models.

Critical strengths
  • A fresh take on self-conditioning: Framing and interpreting self-conditioning as a hidden fixed-point interation is a nice way to think about it. I particularly like the distinction between warm start vs cold start sampling. The authors show that with enough refinement, both end up at the same quality, which suggest that reusing the last prediction mostly just gives you a better starting point rather than being what actually gets you to the fixed point.
  • A well found link to flow-map distillation: Swapping the self-conditioning state for its fixed point gives you a flow that only depends on the current state and time, so standard flow map distillation objectives apply well. I find this a particularly interesting contribution, as it connects fixed point computation to fast few-step generation
  • Promising experimental results: The paper tests its method on two different self-conditioned flow language models and extra fixed-point iterations help consistently in both. The authors also have strong few step results and the online distillation version gets close to the offline one, while decreasing the reported training time.
Critical weaknesses
  • The theory doesn’t quite explain why refinement helps. Prop 3.3 shows convergence under contractivity, but that doesn’t mean the fixed point is closer to the Bayes-optimal denoiser, and the bound in Equation 11 doesn’t guarantee the error drops at each step. The contractivity result in Appendix B.2 is also weaker than what Prop 3.3 assumes. And Figure 2(a) measures distance to a fixed point found by running the same model, so it doesn’t independently show global contractivity or uniqueness.

  • The experiments could better pin down quality and compute trade-offs. The evaluation relies mostly on GPT-2 Large perplexity. Although Table 1 compares against several discrete and continuous baselines at matched sampling steps, the paper does not provide full Gen PPL - entropy frontiers for these baselines. This makes it harder to assess whether the reported results and ordering is robust to different sampling configurations. There are also recent results, see [1], that suggests to use other metrics like MAUVE as Generative PPL can be gamed. It’s also unclear to me whether extra fixed-point iterations are a better use of compute than more flow steps or distillation updates. The paper shows that additional fixed-point iterations improve generation quality, but it would be interesting to understand whether iterative refinement is a more effective use of computation than increasing model capacity.

[1] Franca, A., & Tong, A. (2026). Hacking Generative Perplexity: Why Unconditional Text Evaluation Needs Distributional Metrics. arXiv preprint arXiv:2606.08417.

Questions for authors

1 - Proposition 3.3 guarantees convergence under contractivity, but not that denoising actually gets more accurate. Could you measure the denoising error against the ground-truth target over fixed-point iterations on held-out examples? That would show whether refinement really improves the denoising prediction, and not just the final generation metrics.

2 - Appendix B.2 only shows approximate contractivity for inputs that are far enough apart, while Proposition 3.3 assumes a full contraction. Also, Figure 2(a) uses damped Picard iterations to show convergence, but the generation-quality experiments in Figure 2(b) use undamped ones. Could the authors check convergence under the sampling scheme they actually use, for example by tracking residuals and convergence rates from different starting points?

3 - Table 3 shows clear gains from more fixed-point iterations in online distillation. Would those gains still show up at matched training compute, for example comparing extra fixed-point teacher evaluations against extra student updates? Similarly, at a fixed inference budget, how do more inner fixed-point iterations compare with more flow steps?

4 - Since GPT-2 Large provides the teacher’s pretrained representations and also computes generative perplexity, do the gains hold with a different evaluation model? Also what are the results if you compute MAUVE for your model and baselines?

5 - How would a smaller model with multiple fixed-point iterations compare against a larger Transformer with fewer iterations under matched inference FLOPs? Similarly, could the authors compare FMLM* against a larger standard FMLM under comparable training and inference compute budgets

7 comments
AuthorsOpenReview ↗

Response to Reviewer 2TWE (1/5)

W1-1. The theory doesn’t quite explain why refinement helps. Prop 3.3 shows convergence under contractivity, but that doesn’t mean the fixed point is closer to the Bayes-optimal denoiser, and the bound in Equation 11 doesn’t guarantee the error drops at each step.

Q1. Proposition 3.3 guarantees convergence under contractivity, but not that denoising actually gets more accurate. Could you measure the denoising error against the ground-truth target over fixed-point iterations on held-out examples? That would show whether refinement really improves the denoising prediction, and not just the final generation metrics.

We agree with the reviewer that Proposition 3.3 guarantees convergence under contractivity but not more accurate denoising. As the reviewer suggested, we measure the denoising error on held-out examples. Since the ideal target of the denoiser is the posterior mean E[x1∣It]\mathbb{E}[x_1 \mid I_t] (Proposition 3.1), we measure the error with respect to this posterior mean.

On general datasets, the posterior mean is a weighted average over all training examples, each weighted by how likely it is to produce the noisy input [2], which is costly to evaluate for every noisy input. We therefore use Sudoku-Hard [1], where each puzzle has a unique solution. Given the puzzle, the posterior concentrates on this solution, so the posterior mean equals the ground-truth solution at every tt and can be obtained exactly. We train FLM [3] with self-conditioning following [1] and analyze it with 256 held-out puzzles, measuring at each FPI the MSE between the denoising prediction and the posterior mean, together with the exact-match accuracy of the decoded prediction.

As shown in Figure 10 in the revised Appendix D.1.3, the MSE drops sharply over the first few FPIs and stays low over 200 iterations. The exact-match accuracy rises over the same iterations. In this setting, refinement therefore improves the denoising prediction itself, and not only the final generation metrics.

Regarding why refinement helps, we offer a conjecture based on related work on looped and equilibrium models. In these models, the benefit of additional iterations has been empirically associated with path independence, the tendency of the iteration to reach the same steady-state behavior regardless of its initialization [4]. Consistent with this association, the distance between the iterates initialized from zero and from Gaussian noise decreases substantially on OpenWebText and nearly vanishes on Sudoku-Hard (see our response to W1-2 + Q2).

We have added this analysis to Appendix D.1.3 (Figure 10) of the revised manuscript. We will also revise Sections 3.1 and 4.1, which describe the refinement as moving toward the Bayes-optimal prediction, so that the theoretical guarantee and the empirical evidence are stated separately.

[1] Deschenaux and Gulcehre. "Language Modeling with Hyperspherical Flows." 2026.

[2] Scarvelis et al. "Closed-Form Diffusion Models." 2025.

[3] Lee et al. "Flow Map Language Models: One-step Language Modeling via Continuous Denoising." 2026.

[4] Anil et al. "Path Independent Equilibrium Models Can Better Exploit Test-Time Computation." 2022.

AuthorsOpenReview ↗

Response to Reviewer 2TWE (2/5)

W1-2. The contractivity result in Appendix B.2 is also weaker than what Prop 3.3 assumes. And Figure 2(a) measures distance to a fixed point found by running the same model, so it doesn’t independently show global contractivity or uniqueness.

Q2. Appendix B.2 only shows approximate contractivity for inputs that are far enough apart, while Proposition 3.3 assumes a full contraction. Also, Figure 2(a) uses damped Picard iterations to show convergence, but the generation-quality experiments in Figure 2(b) use undamped ones. Could the authors check convergence under the sampling scheme they actually use, for example by tracking residuals and convergence rates from different starting points?

We agree with the reviewer that Appendix B.2 establishes a weaker property than the full contraction assumed in Proposition 3.3, and that Figure 2(a) alone does not show global contractivity or uniqueness. However, the two results play different roles. Proposition 3.3 states the idealized condition under which our construction is well defined: the fixed point exists, is unique, and is reached from any initialization. Appendix B.2 shows what the training objective provides: on a high-probability set, a small self-conditioned loss maps every iterate close to the target and implies contraction while consecutive iterates are far apart. This guarantees a common neighborhood rather than a unique fixed point.

This gap between a sufficient condition and trained models also appears in related work. Analyses of plug-and-play denoising and implicit networks assume contraction-type conditions [1, 2], while equilibrium models are commonly trained without enforcing them, so that a unique fixed point is not guaranteed [3, 4, 5]. We therefore measure the iteration of the trained denoisers directly.

Following the reviewer's suggestion, we track the residual and the convergence rate of the undamped iteration from zero, which is the initialization used in our distillation and in the revised Figure 2(b, c), and from Gaussian noise. We also measure the distance between the iterates from the two starting points. These quantities are computed from the iterates alone, without a reference fixed point.

As shown in Figures 8, 9, and 11 in the revised Appendix D.1, the residual falls to a few percent of the first update within about ten FPIs from both starting points, and the convergence rate approaches one afterwards. The distance between the iterates from zero and from noise shrinks sharply in the first FPIs and then stays at a small value, larger for LangFlow than for ELF. On Sudoku-Hard, where the target is unique, the iterates from the two starting points nearly coincide. These results suggest that the trained denoisers contract strongly while the updates are large and bring different initializations into a common neighborhood, as Appendix B.2 predicts.

Our method does not rely on uniqueness in practice. As described in Section 3.3, the distillation target is computed by iterating from zero (Equation (18)), and cold-start sampling uses the same initialization. We will revise Section 3.2 to present Proposition 3.3 as an idealized assumption and Appendix B.2 as guaranteeing a neighborhood rather than a unique fixed point, and note that the zero-initialized iteration used in practice does not require uniqueness. We have added these measurements to Appendix D.1 (Figures 8, 9, and 11).

[1] Ryu et al. "Plug-and-Play Methods Provably Converge with Properly Trained Denoisers." 2019.

[2] Fung et al. "JFB: Jacobian-Free Backpropagation for Implicit Networks." 2022.

[3] Bai et al. "Deep Equilibrium Models." 2019.

[4] Anil et al. "Path Independent Equilibrium Models Can Better Exploit Test-Time Computation." 2022.

[5] Bai et al. "Stabilizing Equilibrium Models by Jacobian Regularization." 2021.

AuthorsOpenReview ↗

Response to Reviewer 2TWE (3/5)

W2-1. Although Table 1 compares against several discrete and continuous baselines at matched sampling steps, the paper does not provide full Gen PPL - entropy frontiers for these baselines. This makes it harder to assess whether the reported results and ordering is robust to different sampling configurations.

We thank the reviewer for the thoughtful and constructive comments. Following the reviewer's suggestion, we plot the gPPL-entropy frontiers of the baselines together with FMLM$^\star$ at one, two, and four steps (Figure 13 in the revised Appendix D.3). The frontiers are obtained by sweeping the sampling temperature for Duo and MDLM, and γ\gamma of the γ\gamma-sampling (Appendix C.3) for FMLM and FMLM$^\star$. Since γ\gamma-sampling has no effect with a single step, FMLM and FMLM$^\star$ appear as single points at one step.

As shown in Figure 13, the ordering in Table 1 holds across these frontiers: in the entropy range around that of the data (5.44), FMLM$^\star$ achieves the lowest gPPL at every step count. The baselines reach a gPPL comparable to that of FMLM$^\star$ only when their entropy is reduced well below that of the data, to about 5.0 or lower at four steps, and do not reach it at one step within the swept range. This suggests that the advantage of FMLM$^\star$ in Table 1 is robust to the sampling configuration.

DFM is not included, since neither its code nor its checkpoints are publicly available. Its results in Table 1 are taken from its paper. We have added these frontiers to Appendix D.3 (Figure 13) of the revised manuscript.

W2-2. It’s also unclear to me whether extra fixed-point iterations are a better use of compute than more flow steps or distillation updates.

Q3 (a). Table 3 shows clear gains from more fixed-point iterations in online distillation. Would those gains still show up at matched training compute, for example comparing extra fixed-point teacher evaluations against extra student updates? [...]

As the reviewer suggested, we examine the gains from more FPIs in Table 3 (Table 5 in the revised manuscript) at matched training compute. To this end, we train online FMLM$^\star$ with one, two, and four FPIs for longer, so that their training FLOPs match those of the model with eight FPIs. The models with fewer FPIs thus spend the remaining compute on additional student updates.

As shown in Table A, the gains from more FPIs remain at matched training compute: gPPL decreases with the number of FPIs at every step count, while the entropy changes by less than 0.1 nats. This suggests that teacher FPIs are a more effective use of a fixed training budget than additional student updates. A plausible reason is that FPIs refine the distillation target with only forward passes of the teacher, and this stronger target benefits the student more than additional updates toward a weaker one.

We have added this comparison to Appendix D.5 (Table 8) of the revised manuscript.

Table A. Online FMLM$^\star$ with different numbers of FPIs at matched training compute. Models with fewer FPIs are trained longer, so that all models use the same training FLOPs as the model with eight FPIs (1.47×10201.47 \times 10^{20} FLOPs, excluding teacher training). We report gPPL / entropy.

# FPIs1 step2 steps4 steps
1173.23 / 5.46146.72 / 5.52119.76 / 5.46
2140.44 / 5.43113.87 / 5.5088.77 / 5.44
4123.67 / 5.41101.71 / 5.4981.81 / 5.45
8117.89 / 5.3998.96 / 5.4780.26 / 5.44
Reviewer_ZXf2OpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper investigates the effect of self-conditioning in flow-based language models. They cast the neural network denoiser equipped with self-conditioning as a contractive map, which therefore converges to a fixed-point upon Picard iterations. They demonstrate that more iterations of refinement of the self-conditioning lead to improved generative quality and that the usual inference algorithm with self-conditioning can be interpreted as a (locally, in time) warm-started version of their fixed-point formulation. Based on this observation, the authors formulate fixed-point flows, where the self-conditioning state is replaced with its fixed-point. Using this formulation, the authors demonstrate that it is possible to obtain self-conditioning-free models from self-conditioned counterparts, by distilling a self-conditioned denoiser into its fixed-point one. Additionally, while it is unclear how to distill general self-conditioned flow models into their flow maps, fixed-point flow models are amenable to flow map formulations, unlocking effective few-step generation from (originally) self-conditioned models.

The paper provides substantial implementation and hyperparameter details, which should facilitate reproduction; the authors also state that they plan to release code.

Critical strengths
  1. The paper provides insight into a relevant but understudied component of current diffusion/flow language models: self-conditioning. The authors provide an explanation for why this technique is effective, unify usual practice on self-conditioning with the whole framework, and use this framework to further push model performance (gPPL improves with number of fixed-point iterations) and unlock few-step generation with self-conditioned models (enable flow map formulation for self-conditioned models).
  2. The paper is cleanly written, is easy to follow, and overall has a clear message. Although it still has some points that can be improved (see "Questions For Authors"), the paper is in a mature state.
Critical weaknesses
  1. My main concern with this paper is its empirical scope: the empirical evaluation is restricted to a single dataset and generation setting. Although testing two model families provides some diversity, it remains unclear whether the proposed fixed-point interpretation and distillation benefits extend to other datasets, conditional generation tasks (e.g., TinyGSM/GSM8K setting [1,2]), or model scales (tested models are small: in the original papers, 105M parameters for ELF; 130M for LangFlow). Additionally, while the theoretical support of the paper relies on contractivity of self-conditioned models, the provided empirical evidence is limited (again, single setting, averaging over all tt, and with a damping factor in Picard iterations that can be a confounder. Please check "Questions for Authors" for more details). Overall, broader empirical evaluation would significantly strengthen the generality of the conclusions.
Questions for authors
  1. Figure 1 is not easy to interpret. What do the yellow arrows mean? Also, from there, it's not obvious that the self-conditioning state is being used for the prediction of the next timestep.
  2. It is unclear to me what Proposition 3.3 adds to the paper. Where do the properties presented differ from typical properties of contractive maps in complete spaces? Is there something specific to self-conditioning there? Can you clarify their role in the narrative?
  3. Fig 2.a.: in this figure the authors include a damping factor in the Picard iterations without due motivation. Why is this factor included? This factor may make the damped map contractive even if the original map is not, thus being a confounder in the shown plot (in other words, the damping factor could be masking the behavior of the self-conditioned denoiser, making it difficult to assess whether the denoiser itself is contractive). Could the authors provide the same plot but without the damping factor included? Additionally, would it be also possible to provide the results isolated by tt, i.e., the curves for each tt? Such more fine-grained analysis could provide more insight into which regions the contractivity of the self-conditioning is more or less pronounced (or even if it's contractive everywhere).
  4. Fig. 2.c.: is it possible to narrow the y-axis range, for example to the range used for Fig 3 (e.g., 4.75-6)? While I do not think that the message on the results would change significantly, it would be more informative.

[1] - Stop Training for the Worst: Progressive Unmasking Accelerates Masked Diffusion Training, Kim et al., ICML 2026

[2] - Language Modeling with Hyperspherical Flows, Deschenaux & Gulcehre, TMLR 2026

5 comments
AuthorsOpenReview ↗

Response to Reviewer ZXf2 (1/3)

W1-1. My main concern with this paper is its empirical scope: the empirical evaluation is restricted to a single dataset and generation setting. [...], it remains unclear whether the proposed fixed-point interpretation and distillation benefits extend to other datasets, conditional generation tasks (e.g., TinyGSM/GSM8K setting [1,2]), [...]

We thank the reviewer for the valuable comments. As the reviewer suggested, we extend our experiments to a different dataset and a conditional generation task, following the setting in [1, 2]. We train an ELF teacher on TinyGSM, distill it into FMLM$^\star$ with online distillation (8 FPIs), and evaluate both on GSM8K.

As shown in Table A (a), the benefit of fixed-point iterations carries over to this setting: the accuracy of the teacher increases from 6.37% to 20.55% as the number of FPIs increases from one to five. Table A (b) shows that FMLM$^\star$ outperforms the teacher and baselines at every matched step count. These results suggest that the benefits of fixed-point iterations and of our distillation extend to conditional generation.

We have added these results to Appendix D.4 (Tables 6 and 7) of the revised manuscript.

Table A. GSM8K accuracy (%) of models trained on TinyGSM. (a) ELF teacher with different numbers of FPIs (1024 sampling steps, cold start). (b) FMLM$^\star$ and the ELF teacher at matched sampling steps; the teacher uses warm-start sampling. †^\dagger denotes results taken from the original papers.

(a)

# FPIs12345
ELF (teacher)6.3711.4516.5317.6620.55

(b)

Sampling steps124
FMLM+ (Init)†^\dagger [3]0.30.75.1
MDLM$^\dagger$ [3]0.00.00.1
ELF (teacher)0.080.151.82
FMLM$^\star$ (ours)1.142.966.22

[1] Kim et al. "Stop Training for the Worst: Progressive Unmasking Accelerates Masked Diffusion Training." 2026.

[2] Deschenaux and Gulcehre. "Language Modeling with Hyperspherical Flows." 2026.

[3] Agarwal et al. "Posterior Refinement: Fast Language Generation via Any-Order Flow Maps." 2026.

W1-2. [...] or model scales (tested models are small: in the original papers, 105M parameters for ELF; 130M for LangFlow).

As the reviewer suggested, we examine whether our approach extends to a larger scale. We train a medium-size ELF teacher (ELF-M) [1] and distill it into FMLM$^\star$ with the same online distillation as in Section 4.2 (8 FPIs). This increases the size of FMLM$^\star$ from 157M to 395M parameters.

As shown in Table B, scaling up the model improves FMLM$^\star$ at every step count: with the same online distillation, gPPL decreases while the entropy remains close to that of the data (5.44). FMLM$^\star$ distilled from ELF-M also outperforms offline FMLM$^\star$ distilled from ELF-B, which requires an additional stage to train a separate fixed-point denoiser. These results suggest that our distillation extends to larger models and benefits from scaling.

We also clarify the parameter count of our base-size model. It has more parameters than the original ELF-B (105M) because we replace its T5 encoder with GPT-2 Large. This matches the tokenizer with the baselines for a fair gPPL comparison (Section 4.2), since gPPL depends on the tokenizer. The larger embedding dimension and vocabulary of GPT-2 Large enlarge the decoder head to about 65M parameters.

We have added this result to Appendix D.6 (Table 9) of the revised manuscript.

Table B. Few-step generation of FMLM$^\star$ distilled from base-size (B) and medium-size (M) ELF teachers on OpenWebText (gPPL / entropy, 1024 samples).

TeacherDistillation1 step2 steps4 steps
ELF-BOffline113.91 / 5.3793.66 / 5.4673.86 / 5.35
ELF-BOnline (8 FPIs)117.89 / 5.3998.96 / 5.4780.26 / 5.44
ELF-MOnline (8 FPIs)103.66 / 5.4479.03 / 5.4965.84 / 5.43

[1] Hu et al. "ELF: Embedded Language Flows." 2026.

AuthorsOpenReview ↗

Response to Reviewer ZXf2 (2/3)

W1-3. [...] Additionally, while the theoretical support of the paper relies on contractivity of self-conditioned models, the provided empirical evidence is limited (again, single setting, averaging over all tt, and with a damping factor in Picard iterations that can be a confounder. Please check "Questions for Authors" for more details). Overall, broader empirical evaluation would significantly strengthen the generality of the conclusions.

Q3. Fig 2.a.: in this figure the authors include a damping factor in the Picard iterations without due motivation. Why is this factor included? [...] Could the authors provide the same plot but without the damping factor included? Additionally, would it be also possible to provide the results isolated by tt, i.e., the curves for each tt? [...]

We agree with the reviewer that damping can be a confounder, since a damped iteration may converge even when the original map is not contractive. Following the reviewer's suggestion, we repeat the analysis of Figure 2(a) without damping and separately for each tt. We also repeat it in an additional setting, the ELF teacher trained on TinyGSM (see our response to W1-1).

We included damping only as a solver to compute the reference fixed point. The damped iteration zj+1=αzj+(1−α)D^t(x,zj)z^{j+1} = \alpha z^j + (1-\alpha)\hat{D}_t(x, z^j) has the same fixed points as the original one for α<1\alpha < 1, and damped or accelerated solvers are commonly used to compute such fixed points [1, 2, 3]. The generation experiments in Figure 2(b, c) use undamped iterations (Appendix C.1).

As shown in Figure 6 in the revised Appendix D.1.1, the undamped iteration converges in almost the same way as the damped one. For both ELF and LangFlow, the relative distance drops sharply in the first few iterations and keeps decreasing afterwards. The only visible difference is a small oscillation of LangFlow in later iterations.

As shown in Figure 7 in the revised Appendix D.1.1, the relative distance decreases at every tt, with and without damping. The rate differs across tt: convergence is fast at large tt and slowest around tt = 0.2 to 0.3. Without damping, LangFlow at large tt settles into a small oscillation of about 5% in relative distance. This is consistent with Appendix B.2, which guarantees contraction only while consecutive iterates are separated by a certain distance (Lemma B.3).

The ELF teacher trained on TinyGSM shows the same behavior in Figures 14 and 15 in the revised Appendix D.4: with and without damping, the relative distance falls close to zero within about 20 iterations at every tt. These results suggest that damping is not the source of the convergence in Figure 2(a), in all settings we examine.

We have added the undamped and time-resolved results to Appendices D.1.1 and D.4 of the revised manuscript, and will state that damping is used only to compute the reference fixed point.

[1] Lauriere. "Numerical Methods for Mean Field Games and Mean Field Type Control." 2021.

[2] Bai et al. "Deep Equilibrium Models." 2019.

[3] Movahedi et al. "Fixed-Point Reasoners: Stable and Adaptive Deep Looped Transformers." 2026.

Q1. Figure 1 is not easy to interpret. What do the yellow arrows mean? Also, from there, it's not obvious that the self-conditioning state is being used for the prediction of the next timestep.

We thank the reviewer for pointing this out. The yellow arrows denote the flow update, the Euler step from x^ti\hat{x}_{t_i} to \hat{x\}_{t_{i+1}} (Equations (9) and (16)). The red arrows denote warm start: the estimate from the previous flow step is used as the self-conditioning input at the next step (Equation (9)). We have added both arrows to the legend of Figure 1 as "flow update" and "warm start."

AuthorsOpenReview ↗

Response to Reviewer ZXf2 (3/3)

Q2. It is unclear to me what Proposition 3.3 adds to the paper. Where do the properties presented differ from typical properties of contractive maps in complete spaces? Is there something specific to self-conditioning there? Can you clarify their role in the narrative?

We agree with the reviewer that Proposition 3.3 does not introduce a new fixed-point theorem: (i) is the Banach fixed-point theorem applied to the self-conditioned denoiser, and (ii) and (iii) follow from standard estimates. Instead, its role is to state the conditions under which our construction is well defined. Under contractivity, the fixed-point denoiser D⋆D^\star (Equation (13)) exists and does not depend on the initialization. This justifies the cold-start view of self-conditioned sampling (Section 3.3) and the computation of the distillation target from zero initialization (Equation (18)).

The part specific to self-conditioning is the interpretation of the initial error in (iii). With cold start, the first iterate is the prediction of the denoiser without self-conditioning, whose error is minimized by the first term of Equation (7). Thus, (iii) bounds the error of the fixed point by the error of this standard prediction and the size of the first update.

In the next revision, we will state explicitly that the proposition follows from standard results and clarify its role in Section 3.2.

Q4. Fig. 2.c.: is it possible to narrow the y-axis range, for example to the range used for Fig 3 (e.g., 4.75-6)? While I do not think that the message on the results would change significantly, it would be more informative.

Following the reviewer's suggestion, we have narrowed the vertical range of Figure 2cc to 4.75–6.0, as shown in the revised manuscript. In the revised figure, we also report Figure 2(b, c) with cold-start sampling, which starts every flow step from the same zero initialization as Figure 2(a) and our distillation. The warm-start results remain in Appendix D.2 (Table 3).

As shown in the revised Figure 2cc, the narrower range makes the change in entropy visible: entropy decreases with more FPIs and approaches the data entropy (5.44) for both models. Since lower entropy can also lower gPPL, we additionally compare gPPL at a matched entropy of 5.30. As shown in Table C, gPPL still decreases with more FPIs, and ELF with five FPIs reaches less than half the gPPL of one FPI. This suggests that the gain from FPIs is not only due to the change in entropy.

We have updated Figure 2(b, c) and Section 4.1 accordingly, and added this comparison to Appendix D.2 (Table 4).

Table C. gPPL at a matched entropy of 5.30 (32 steps for ELF and 64 for LangFlow, without damping). Warm start with a single FPI corresponds to conventional self-conditioned sampling.

Warm start, 1 FPICold start, 1 FPI2 FPIs3 FPIs4 FPIs5 FPIs
ELF36.5778.5041.9135.4933.9833.48
LangFlow52.6180.4562.2354.4849.4647.02
Reviewer_JBgYOpenReview ↗
Rating
3(+1)
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper interprets self-conditioning in flow language models as fixed-point iteration and establishes convergence under a contractivity assumption. Building on this view, it introduces fixed-point flows and distills both prediction refinement and flow evolution into FMLM⋆ for one- and few-step generation. Experiments on OpenWebText show improved generation quality while maintaining diversity.

Critical strengths
  1. Clear and actionable framework: The fixed-point perspective connects self-conditioning to flow-map learning and motivates a practical method for compressing both prediction refinement and flow evolution.
  2. Coherent empirical evidence: Experiments examine iterative refinement, warm versus cold starts, and distillation, showing improved few-step generation while largely preserving diversity.
Critical weaknesses
  1. Limited evaluation: Results focus on OpenWebText and gPPL–entropy metrics. Broader tasks and equal-compute comparisons would strengthen the claims.
  2. Incremental conceptual novelty: Fixed-point interpretations of self-conditioning already exist; the main contribution lies in their integration with flow-map distillation.
Questions for authors

See above

3 comments
AuthorsOpenReview ↗

Response to Reviewer JBgY (1/2)

W1-1. Limited evaluation: Results focus on OpenWebText […] Broader tasks and […] would strengthen the claims.

We thank the reviewer for the valuable comments. Our main experiments follow the established benchmarking setting of unconditional generation on OpenWebText [1, 2, 3]. To examine whether our method extends beyond this setting, we additionally evaluated it on a conditional generation task. Specifically, we trained an ELF teacher on TinyGSM, distilled it into FMLM$^\star$, and evaluated both zero-shot on GSM8K. As shown in Table A, FMLM$^\star$ outperforms the teacher and baselines at every matched step count. This suggests that our distillation remains effective for conditional generation. We have added this result to Appendix D.4 (Table 7) of the revised manuscript.

Table A. GSM8K accuracy (%) of models trained on TinyGSM, following the setting of [4, 5, 6, 7]. FMLM$^\star$ is distilled online with eight FPIs, and the teacher uses warm-start sampling. †^\dagger denotes results taken from the original paper [6].

Sampling steps124
FMLM+ (Init)†^\dagger [6]0.30.75.1
MDLM$^\dagger$ [6]0.00.00.1
ELF (teacher)0.080.151.82
FMLM$^\star$ (ours)1.142.966.22

[1] Sahoo et al. "The Diffusion Duality." 2025.

[2] Lee et al. "Flow Map Language Models: One-step Language Modeling via Continuous Denoising." 2026.

[3] Pynadath et al. "CANDI: Hybrid Discrete-Continuous Diffusion Models." 2025.

[4] Kim et al. "Stop Training for the Worst: Progressive Unmasking Accelerates Masked Diffusion Training." 2026.

[5] Deschenaux and Gulcehre. "Language Modeling with Hyperspherical Flows." 2026.

[6] Agarwal et al. "Posterior Refinement: Fast Language Generation via Any-Order Flow Maps." 2026.

[7] Hu et al. "ELF: Embedded Language Flows." 2026.

W1-2. Limited evaluation: Results focus on […] and gPPL-entropy metrics. […]

We would like to clarify that gPPL-entropy on OpenWebText is the standard protocol for unconditional few-step language generation [1, 2, 3], and we followed it to allow a direct comparison with prior work. However, we agree with the reviewer that relying on these two metrics alone gives a limited view of generation quality. To address this, we evaluated our models and the baselines with MAUVE, Self-BLEU, and gPPL under a second evaluator (Llama 3.1-8B) [4]. As shown in Table B, FMLM$^\star$ achieves substantially higher MAUVE than all baselines and the lowest Llama 3.1 gPPL at every step count, while its Self-BLEU stays closer to that of the data than that of FMLM. This shows that the advantage of FMLM$^\star$ is not specific to GPT-2 Large gPPL.

Table B. Additional metrics at 1 / 2 / 4 sampling steps (1024 samples).

1 / 2 / 4 stepMAUVE ↑\uparrowgPPL (Llama 3.1) ↓\downarrowSelf-BLEU
Duo + DCD0.055 / 0.063 / 0.1503291.27 / 905.76 / 289.820.060 / 0.156 / 0.271
Duo + Di4C0.054 / 0.075 / 0.166185.73 / 190.45 / 153.000.406 / 0.470 / 0.418
MDLM + SDTT0.053 / 0.057 / 0.090940.89 / 804.17 / 332.850.200 / 0.191 / 0.265
MDLM + Di4C0.053 / 0.057 / 0.107916.11 / 667.31 / 232.050.196 / 0.205 / 0.292
FMLM0.078 / 0.165 / 0.351174.62 / 139.92 / 111.520.401 / 0.397 / 0.399
FMLM$^\star$ (offline)0.692 / 0.884 / 0.865119.83 / 95.16 / 68.750.322 / 0.324 / 0.349
FMLM$^\star$ (online)0.679 / 0.883 / 0.882121.69 / 98.77 / 77.370.312 / 0.316 / 0.339
Data1.00010.130.282

[1] Sahoo et al. "The Diffusion Duality." 2025.

[2] Lee et al. "Flow Map Language Models: One-step Language Modeling via Continuous Denoising." 2026.

[3] Pynadath et al. "CANDI: Hybrid Discrete-Continuous Diffusion Models." 2025.

[4] Grattafiori et al. "The Llama 3 Herd of Models." 2024.

AuthorsOpenReview ↗

Response to Reviewer JBgY (2/2)

W1-3. Limited evaluation: […] and equal-compute comparisons would strengthen the claims.

As the reviewer suggested, we compare FMLM$^\star$ with the baselines at matched inference compute. We measure compute as the inference FLOPs per sample, including the decoder of FMLM$^\star$. For each setting of FMLM$^\star$, each baseline uses the fewest steps whose compute is comparable to or larger than that of FMLM$^\star$ (Table C).

As shown in Table C, one-step FMLM$^\star$ outperforms all two-step baselines when gPPL and entropy are considered together. DFM-ESD achieves a slightly lower gPPL, but at a substantially lower entropy (5.15, compared to 5.37 for FMLM$^\star$ and 5.44 for the data). With two and four steps, FMLM$^\star$ outperforms all baselines with available results while using the fewest FLOPs, as the fixed cost of its decoder becomes smaller relative to the total cost. These results show that FMLM$^\star$ remains competitive with or outperforms the baselines at matched inference compute.

Table C. Comparison at matched inference compute. GFLOPs are per sample and include the decoder of FMLM$^\star$. Each baseline uses the fewest steps whose compute is comparable to or larger than that of FMLM$^\star$; in (a), the two-step Duo and MDLM use 584 GFLOPs, compared to 585 for FMLM$^\star$. DFM is not available in (b), since neither its code nor its checkpoints are released and only its one-, two-, and four-step results are reported.

GroupModelStepsGFLOPsgPPL ↓\downarrowEntropy
(a)FMLM$^\star$ (ours)1585113.915.37
Duo-DCD2584891.165.41
Duo-Di4C2584210.224.63
MDLM-SDTT2584877.225.34
MDLM-Di4C2584758.235.35
DFM-PSD2696152.835.03
DFM-ESD2696108.915.15
FMLM2742133.295.25
(b)FMLM$^\star$ (ours)281193.665.46
Duo-DCD3876425.245.39
Duo-Di4C3876233.874.85
MDLM-SDTT3876552.995.37
MDLM-Di4C3876419.785.36
DFM-PSD31044N/AN/A
DFM-ESD31044N/AN/A
FMLM31113120.975.34
ccFMLM$^\star$ (ours)4126373.865.35
Duo-DCD51460189.055.40
Duo-Di4C51460124.374.88
MDLM-SDTT51460236.595.39
MDLM-Di4C51460165.75.40
DFM-PSD41392122.325.10
DFM-ESD4139277.085.27
FMLM41484111.315.26

We will add this comparison to Appendix D.3 in the next revision.

W2. Incremental conceptual novelty: Fixed-point interpretations of self-conditioning already exist; the main contribution lies in their integration with flow-map distillation.

We agree that the fixed-point interpretation of self-conditioning itself is not new. Gulrajani & Hashimoto [1] present the self-conditioned denoiser as the fixed point of a recurrence, and we credit this work in Section 3.2 and Appendix A. However, our contribution lies not in the interpretation itself but in its formalization. We develop the interpretation into fixed-point flows, which make flow map distillation possible, as we explain below.

Fixed-point flows turn the interpretation into a generative process to which flow map learning applies. In conventional self-conditioned sampling (Equation (9)), the velocity depends on both the flow state and the prediction carried over from the previous step. As discussed in Section 3.1, the process is therefore not an ODE in the flow state alone, and the flow map of Equation (5) does not directly apply. Fixed-point flows resolve this by replacing the self-conditioning state with its fixed point, which gives a velocity that depends only on the time and the flow state (Proposition 3.4). We analyze the convergence of this iteration under a contractivity assumption (Proposition 3.3) and show that conventional sampling is a warm-started, single-iteration approximation of this flow. On ELF and LangFlow, cold-start sampling with enough iterations matches warm-start sampling (Section 4.1).

This formalization is what makes flow map distillation possible. Since a fixed-point flow is an ordinary ODE, it admits a valid flow map (Propositions 3.5–3.7). This allows us to remove self-conditioning through fixed-point distillation and to distill self-conditioned models into few-step flow maps (FMLM$^\star$). As the reviewer noted, this integration is our main contribution, and we are not aware of prior work that distills self-conditioned flow language models into flow maps.

[1] Gulrajani and Hashimoto. "Likelihood-Based Diffusion Language Models." 2023.

Thank you for the detailed reply; I will improve my score.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

We thank all reviewers for their thoughtful and constructive feedback. We have uploaded a revised manuscript, in which all changes are marked in blue. Most additions are in the new Appendix D, which covers the fixed-point behavior of self-conditioned denoisers, comparisons at matched entropy and matched compute, conditional generation on GSM8K, and scaling to a larger model. We respond to each reviewer's comments in the individual threads.

Dear reviewers,

The author-reviewer discussion period is ending soon. If you have not done so yet, please read the authors’ rebuttal and reply to it. Tell the authors whether it addresses your concerns, and update your score if your assessment has changed.

The ICLR 2027 Reviewer Guidelines list « discussing with the authors to clarify and improve the paper » as one of the core duties you accepted as a reviewer. With the high number of submissions this year, your input matters more than ever. Recall that reviewers that do not fulfill their duties may also face penalties on their own submissions.

Thank you for your time and help.

Best regards,


Your Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.