acceptodds
Under review as a conference paper at ICLR 2027

Few-Step Generation via Data-Space Iteration

Abstract

Flow matching has emerged as a scalable paradigm for training high-quality generative models, but sampling from the learned probability flow requires many network evaluations. Distillation can reduce this cost to one or a few evaluations; however, one-step generation often sacrifices quality, making few-step generation the practical operating regime. Existing few-step methods perform their iterative computation along the probability flow and therefore require a fixed, manually chosen timestep discretization. This discretization is often chosen heuristically and is expensive to tune; it may also be restrictive when refinement difficulty differs across samples or spatial locations. We introduce data-space iteration, a few-step generation framework that removes flow discretization altogether. Starting from noise, a shared generator directly refines its prediction in data space, with every iteration trained to produce the best sample permitted by its capacity. Our formulation integrates with distribution matching distillation (DMD) with minimal changes, enabling a controlled comparison between iteration methods under matched training settings. On class-conditional ImageNet 256x256, data-space iteration outperforms standard discretization baselines and matches or improves upon variants selected through schedule search, without requiring schedule-specific training. These results show that data-space iteration provides a simple and effective alternative to discretized probability-flow iteration for fast generation.

Primary area:
generative models
Keywords:
generative models, efficiency, few-step generation, distillation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_pJiSOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a few-step generation method based on data-space iteration. The method moves the rollout from probability-flow space to data space while retaining the remaining DMD training components. The paper validates the method mainly on ImageNet 256×256.

Critical strengths
  1. The paper is well-written and easy to follow.
  2. The paper achieves performance improvements with a relatively small modification.
Critical weaknesses
  1. The work is relatively incremental. The core change is only switching the DMD rollout from probability-flow space to data space. The authors should more clearly articulate the essential mechanistic difference beyond a different iteration space.
  2. Comparison with other few-step generation methods is missing. Comparison against non-DMD few-step methods such as iMeanFlow is recommended.
  3. The work is validated mainly on ImageNet 256×256 and a single architecture (DiT-XL/2).
Questions for authors
  1. Please see weaknesses.
  2. It is recommended to report metrics beyond FID in the main text, such as sFID, IS, precision, and recall.
3 comments
AuthorsOpenReview ↗

Response to Reviewer pJiS

We thank the reviewer for their feedback and address their questions and concerns below.

1. Importance of our work

The key contribution is not merely changing the iteration space, but introducing a new schedule-free paradigm for few-step generation.

Existing flow-space methods pass schedule-specific noisy states between steps, requiring the schedule to be fixed before self-rollout training; testing another schedule may therefore require training a separate model. This is an increasingly important bottleneck in image and causal streaming-video distillation. Our method instead passes the current data prediction back to the generator and directly refines it toward the data distribution. This removes costly schedule search and consistently improves FID over flow-renoise and flow-map baselines under matched training settings. Importantly, this new paradigm can be adopted in existing DMD pipelines with minimal changes and no additional sampling cost.

2. Comparison to MeanFlow

As the reviewer suggested, we include MeanFlow [1] as an additional baseline.

We selected MeanFlow [1] rather than iMeanFlow [2] because their objectives are identical in the distillation setting. Moreover, MeanFlow uses the SiT architecture, allowing us to adopt its hyperparameters directly, whereas iMeanFlow uses a different custom architecture.

Comparison with MeanFlow1NFE2NFE3NFE4NFE
MeanFlow Distillation (No CFG)14.8211.1810.3810.42
Data-space DMD (No CFG)3.852.682.472.46
MeanFlow Distillation (CFG)5.692.962.572.51
Data-space DMD (CFG)3.312.472.332.29

Our MeanFlow distillation model was trained using velocity predictions from the SiT teacher and the same hyperparameter configuration as the XL/2+ model in the MeanFlow paper, except that we changed the adaptive loss-weighting parameter pp from 1.0 to 0.5. We made this change because the model failed to learn with p=1.0p=1.0, an issue also reported in FACM [3]. Our reproduced MeanFlow distillation model performs worse than a MeanFlow model trained from scratch, for which the original paper reports FIDs of 3.43 at 1 NFE and 2.20 at 2 NFEs. This discrepancy may be due to the SiT teacher, and further hyperparameter tuning could potentially improve the results. We report our best-effort reproduction of MeanFlow distillation solely as an additional baseline.

3. Evaluation on additional datasets and architectures

Our original submission includes an additional evaluation on CIFAR-10 using the EDM U-Net architecture in the appendix, following the evaluation protocol of MeanFlow [1].

Because Reviewer 1Yce also raised concerns about the generality of our method, we conducted an additional experiment using the LightningDiT [4] architecture for this rebuttal. Specifically, we used the pretrained LightningDiT on ImageNet-256 (FID = 2.17, without CFG) as the teacher and adopted the same hyperparameter configuration as in our main SiT experiment. The results show that data-space iteration consistently outperforms the flow-renoise and flow-map methods across all evaluated NFE settings.

LightningDiT (no CFG)1NFE2NFE3NFE4NFE
Flow-renoise3.192.582.602.78
Flow-map2.60
Data-space (Ours)2.922.442.382.19

Thus far, we have evaluated our method on two datasets (ImageNet and CIFAR-10) and three architectures (SiT, LightningDiT, and EDM U-Net). We believe that these results demonstrate the generality of our method.

4. Evaluation metrics

We report the full set of metrics (sFID, IS, precision, and recall) in Table 5 of the appendix. Following the convention of recent papers and because of space constraints, the main text reports only FID. We will take the reviewer's suggestion into account when preparing the camera-ready version.

Conclusion

In this rebuttal, we clarified the importance and mechanistic novelty of data-space iteration, provided an additional comparison with MeanFlow distillation, evaluated our method on the LightningDiT architecture, and pointed to the full evaluation metrics reported in the appendix. These additions address the reviewer's concerns about novelty, baseline comparisons, generality, and evaluation breadth.

Citations

  1. Mean Flows for One-step Generative Modeling (NeurIPS 2025) (https://arxiv.org/abs/2505.13447)
  2. Improved Mean Flows: On the Challenges of Fastforward Generative Models (CVPR 2026) (https://arxiv.org/abs/2512.02012)
  3. FACM: Flow-Anchored Consistency Models (ICLR 2026) (https://arxiv.org/abs/2507.03738)
  4. Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models (CVPR 2025) (https://arxiv.org/abs/2501.01423)

Thank you for your responses.

I am sorry for not stating my main concern clearly. My primary worry is about dataset coverage. The only large-scale dataset is ImageNet, and CIFAR-10 is relatively small in resolution. I therefore still have some concerns about whether the empirical evaluation covers a sufficiently diverse set of datasets.

However, I am not fully confident in my judgment. After considering the comments of other reviewers and the initial response, I have decided to raise my rating to 3. I will revisit the discussion before it closes and confirm my final rating.

We thank the reviewer for updating their rating.

We would also like to clarify that evaluating on ImageNet-256 and CIFAR-10 is a well-established protocol in recent work on few-step generative models. The table below summarizes the evaluation datasets used by representative methods. Most of these works evaluate on ImageNet together with one smaller dataset, while several report results on ImageNet alone. Notably, MeanFlow, iMM, and FACM use exactly the same ImageNet-256 and CIFAR-10 setting as ours. We therefore believe that our experimental scope is consistent with the evaluation standards established by the recent literature and provides a sufficiently rigorous basis for comparison.

PaperEvaluation datasetsVenue and citation
MeanFlowImageNet-256 + CIFAR-10NeurIPS 2025
iMeanFlowImageNet-256CVPR 2026
pMeanFlowImageNet-256 + ImageNet-512ICML 2026
ShortcutImageNet-256 + CelebA-HQICLR 2025
CMImageNet-64 + CIFAR-10 + LSUNICML 2023
iCMImageNet-64 + CIFAR-10ICLR 2024
sCMImageNet-64 + ImageNet-512 + CIFAR-10ICLR 2025
iMMImageNet-256 + CIFAR-10ICML 2025
AlphaFlowImageNet-256ICLR 2026
AFMImageNet-256ICML 2026
FACMImageNet-256 + CIFAR-10ICLR 2026

We hope this clarification addresses the reviewer’s remaining concern. If there are any outstanding questions or reservations that prevent a stronger assessment, we would be grateful if the reviewer could let us know. We would be happy to provide any additional clarification, analysis, or experimental evidence that would be helpful.

Reviewer_1YceOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes data-space iteration for few-step generation. Instead of iterating through states defined by a fixed discretization, the method repeatedly refines a clean-data prediction. The method is trained with distribution matching distillation (DMD) and requires minor modifications to a pretrained model. The proposed method consistently outperforms the standard DMD baseline on ImageNet 256x256. The paper also provides useful ablations on the DMD training recipe, timestep discretization, and generator conditioning. However, key experiments needed to isolate the mechanism of the proposed method remain missing, and the evaluation is limited in terms of datasets and model settings.

Critical strengths
  • The paper introduces a simple and practical alternative to time discretization. The method removes schedule-specific training with minor architectural modifications.

  • The ImageNet results are consistent across one to four NFEs. The method improves over the standard DMD and Phased DMD baselines, and the paper provides detailed implementation information and several useful ablations.

Critical weaknesses
  1. The main comparison does not isolate the effect of the iteration space. The data-space generator receives the previous clean prediction, the original noise zz, and the refinement index rr at every step. In contrast, the DMD baseline receives only a noisy mixed state and a timestep, and uses newly sampled noise after the first step. Therefore, the improvement may partly result from persistent access to zz, explicit self-conditioning, or different input information, rather than data-space iteration itself. Matched hybrid baselines are necessary to support the paper's central claim.

  2. The empirical validation is limited. The main results are obtained with a single ImageNet teacher and backbone, leaving it unclear whether the proposed advantage generalizes across datasets, architectures, and teacher models. The only additional dataset, CIFAR-10, is evaluated briefly in the appendix, where all distilled models remain far behind the teacher. In contrast, recent non-adversarial distillation methods have achieved substantially stronger CIFAR-10 results [1]. More comprehensive evaluation beyond the primary ImageNet setting is needed.

Reference

[1] Luo W, Huang Z, Geng Z, et al. One-step diffusion distillation through score implicit matching[J]. Advances in Neural Information Processing Systems, 2024, 37: 115377-115408.

Questions for authors
  1. Can the authors provide matched baselines that isolate the effect of the iteration space? For example, how does flow-renoise DMD perform when it reuses the same noise zz across steps or is additionally conditioned on zz, rr, and the previous clean prediction? Conversely, how does data-space DMD perform when some of these inputs are removed? (Table. 1b provides preliminary evidence, which is good)

  2. How well does the method generalize beyond the primary ImageNet setting? Could the authors provide results with additional datasets, teachers, or backbones?

  3. The paper motivates data-space iteration by arguing that different samples and spatial regions may benefit from different effective refinement behaviors. However, it seems that no direct evidence is provided that the learned updates are actually sample- or spatially adaptive.

  4. Tab. 2a applies Beta(2.5,1) sampling with w(τ)=1w(\tau)=1 to the flow-renoise baseline, but does not compare it against the original DMD weighting. Does the proposed sampling and weighting strategy also outperform the original DMD weighting for flow-renoise? This comparison is needed to establish the strength of the reported DMD baseline.

  5. The proposed rollout strategy may affect convergence speed. How does data-space DMD compare with the DMD baseline in terms of convergence speed?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 1Yce

We thank the reviewer for the detailed review of our work. Below, we provide additional experimental results to address the questions and concerns.

1. Additional ablation studies on the flow-renoise method

As the reviewer suggested, we performed additional ablation studies on the flow-renoise method. Specifically, we trained a flow-renoise model that reuses the same noise zz across iterations. We also trained variants conditioned on zz or x^0\hat{x}_0. Our data-space formulation consistently outperforms all flow-renoise variants across all NFE settings.

Additional Ablation1NFE2NFE3NFE4NFE
Flow-renoise4.453.222.672.75
Flow-renoise fixed noise4.202.742.662.78
Flow-renoise fixed noise + z cond4.492.862.732.79
Flow-renoise fixed noise + x cond4.172.672.622.81
Data-space (Ours)3.852.682.472.46

2. Evaluation on additional datasets and architectures

Following MeanFlow [1], we evaluate SiT on ImageNet-256 and EDM U-Net on CIFAR-10. Our CIFAR-10 results trail the best recent distillation results because we directly transferred the ImageNet weighting configuration without a new hyperparameter search. This zero-shot transfer enables a fair comparison among the three iteration methods.

To further test generality, we also evaluate LightningDiT [2], using its pretrained ImageNet-256 model (FID = 2.17, without CFG) as the teacher and the same hyperparameters as in our SiT experiment. Without additional tuning, data-space iteration outperforms the compared methods at all reported NFE settings.

LightningDiT NoCFG1NFE2NFE3NFE4NFE
Flow-renoise3.192.582.602.78
Flow-map2.60
Data-space (Ours)2.922.442.382.19

Together, these experiments cover two datasets and three architectures.

3. Clarifying sample adaptivity and spatial adaptivity as motivations

Prior work demonstrates the effectiveness of sample-adaptive (LD3 [3] and Instance-Aware Discretization [4]) and spatially adaptive (Patch-Forcing [5]) denoising schedules. However, self-rollout requires a schedule to be fixed before training, motivating our schedule-free data-space iteration. We cite adaptivity only as a possible explanation for its gains, not as a definitive mechanism.

4. Comparison of flow-renoise and data-space using the original DMD weighting

As the reviewer suggested, we trained a flow-renoise model using the original DMD dynamic weighting for comparison. The results show that our data-space method still outperforms flow-renoise under the original DMD weighting. Together with the existing comparisons in Table 2a, these results show that data-space iteration consistently outperforms flow-renoise across all weighting configurations.

Using DMD dynamic weighting1NFE2NFE3NFE4NFE
Flow-renoise4.804.304.023.83
Data-space (Ours)4.243.243.223.22

5. Convergence speed

Below, we report 4-NFE FID at 10-epoch intervals using EMA decay 0.9999. Flow-map converges faster initially, but data-space surpasses both baselines by 40 epochs and achieves the best FID on both architectures.

Training Speed - SiT 4NFE10ep20ep30ep40ep (selected)50ep
Flow-renoise4.782.802.762.75 (best)2.78
Flow-map4.142.602.562.55 (best)2.61
Data-space (Ours)54.173.512.612.46 (best)2.49
Training Speed - LightningDiT 4NFE10ep20ep30ep40ep50ep60ep (selected)70ep
Flow-renoise4.583.363.032.912.882.78 (best)2.80
Flow-map3.162.702.782.632.652.60 (best)2.64
Data-space (Ours)32.724.582.842.462.302.19 (best)2.24

Conclusion

We provide new flow-renoise ablations, LightningDiT results, a DMD-weighting comparison, and a convergence analysis, while clarifying the CIFAR-10 protocol and adaptivity motivation. Across these evaluations, data-space iteration continues to outperform the compared methods.

Citations

  1. Mean Flows for One-step Generative Modeling (NeurIPS 2025) (https://arxiv.org/abs/2505.13447)

  2. Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models (CVPR 2025) (https://arxiv.org/abs/2501.01423)

  3. Learning to Discretize Denoising Diffusion ODEs (ICLR 2025) (https://arxiv.org/abs/2405.15506)

  4. Few-Step Diffusion Sampling Through Instance-Aware Discretizations (CVPR 2026) (https://arxiv.org/abs/2603.17671)

  5. Denoising, Fast and Slow: Difficulty-Aware Adaptive Sampling for Image Generation (CVPR 2026) (https://arxiv.org/pdf/2604.19141)

Thank you for your effort in providing these new results. The matched flow-renoise ablations largely address my concern about isolating the effect of data-space iteration. The convergence analysis also improves the understanding of the method, while the LightningDiT results provide encouraging evidence for new backbones.

My assessment has therefore become more positive, although I am not yet ready to recommend acceptance. My remaining concern is the limited dataset scope. The current CIFAR-10 results are still preliminary. While ImageNet-256 and CIFAR-10 are common benchmarks for several methods cited in the response to Reviewer pJiS, DMD differs from these methods in that it is already a well-established and extensively validated framework for large-scale text-to-image distillation. Since DMD2, subsequent DMD-based methods have been extensively evaluated on large-scale text-to-image models such as SDXL and SD3, and more recently FLUX and Qwen-Image. I understand that text-to-image experiments require substantial computational resources, but preliminary results with matched baselines would strengthen the empirical evidence and could further change my assessment.

We thank the reviewer for the more positive assessment of our work.

Regarding the choice of datasets, we believe that ImageNet and CIFAR-10 are appropriate for the scope and objectives of this study and are consistent with established evaluation practices in the relevant literature.

As the reviewer noted, our response to Reviewer pJiS lists several recently accepted works on few-step generative modeling, demonstrating that evaluation on ImageNet-256 and CIFAR-10 is a well-established protocol. In particular, influential work such as MeanFlow [1] evaluates exclusively in this setting, even though methods prior to MeanFlow had already expanded to large-scale text-to-image generation. For example, Latent Consistency Models [2] and DMD2 [3], published in 2023 and 2024, respectively, both predate MeanFlow [1] (2025). Thus, the appropriate evaluation setting depends primarily on the specific contribution and scope of a study, rather than on a categorical distinction between DMD-based and flow-based methods.

First, our work studies iteration paradigms for generative models as a general machine-learning problem, rather than focusing exclusively on a particular computer-vision application. This differs from large-scale distillation studies such as DMD2 [3] and rCM [6], which evaluate in text-to-image and/or text-to-video settings and focus on application-specific challenges at scale. Second, our evaluation requires controlled comparisons among three iteration paradigms, together with extensive ablations over numerous hyperparameters. The current dataset scale makes it possible to conduct these comparisons systematically and rigorously within our available computational resources.

We agree that large-scale experiments could provide an additional and valuable perspective. However, given our limited computational resources, we believe it is appropriate for the present work to establish the methodological foundation and to leave a systematic investigation of scaling to future work. This staged progression also has precedent in the literature: CM [4] was evaluated on academic datasets before LCM [2] extended consistency modeling to text-to-image generation, while sCM [5] was evaluated on ImageNet and CIFAR-10 before rCM [6] scaled continuous-time consistency distillation to large text-to-image and text-to-video models. A similar progression can be seen in distribution-matching distillation: Diff-Instruct [7] was evaluated on academic-scale datasets before DMD [8] extended distribution-matching distillation to large-scale text-to-image generation.

We hope this explanation clarifies the rationale behind our experimental scope. Given the established evaluation protocol, the breadth of our controlled comparisons, and the extensive ablation studies included in the paper, we believe that the current work provides a sufficiently rigorous and complete empirical evaluation to meet the standard for acceptance. If any questions or reservations remain, we would be grateful if the reviewer could let us know, and we would be happy to provide any additional clarification or analysis that would be helpful.

Citation

  1. Mean Flows for One-step Generative Modeling (NeurIPS 2025) (https://arxiv.org/abs/2505.13447)
  2. Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference (https://arxiv.org/abs/2310.04378)
  3. Improved Distribution Matching Distillation for Fast Image Synthesis (NeurIPS 2024) (https://arxiv.org/abs/2405.14867)
  4. Consistency Models (ICML 2023) (https://arxiv.org/abs/2303.01469)
  5. Simplifying, Stabilizing and Scaling Continuous-Time Consistency Models (ICLR 2025) (https://arxiv.org/abs/2410.11081)
  6. Large Scale Diffusion Distillation via Score-Regularized Continuous-Time Consistency (ICLR 2026) (https://arxiv.org/abs/2510.08431)
  7. Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models (NeurIPS 2023) (https://arxiv.org/pdf/2305.18455)
  8. One-step Diffusion with Distribution Matching Distillation (CVPR 2024) (https://arxiv.org/abs/2311.18828)
Reviewer_giqgOpenReview ↗
Rating
4
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper attempts to improve DMD by changing how a few-step generator produces samples, moving away from the existing flow-renoise and flow-map decoding approaches toward a method in which samples are iteratively refined in data space. The authors argue that the sampling schedule used during training is a sensitive parameter that is expensive to tune, and propose data-space iteration as a simpler, schedule-free alternative. Specifically, this is done by augmenting the generator to take in the previous sample x^0\hat{x}_0, the initial noise zz, and the iteration counter rr, resulting in x0=G(x^0,z,r)x_0 = G(\hat{x}_0, z, r). Starting from x^0=0\hat{x}_0 = 0, the method repeatedly applies the generator to iteratively refine the sample. The proposed method can be easily integrated into existing DMD pipeline. In the experiments, the authors compare their approach against standard flow-renoise and flow-map rollouts, showing that data-space iteration consistently outperforms both baselines.

Critical strengths

I really enjoyed reading this paper. The paper itself is extremely well written, with a clear motivation, a well-justified approach, and a final method that is simple and straightforward. The method requires only a few small changes to the model and can be easily integrated into existing DMD-based distillation pipelines. The authors also provide clear pseudocode in the appendix which helps further clarify the approach. Finally, the authors conduct a comprehensive set of experiments and include a large number of ablations. In almost all settings, data-space iteration consistently outperforms the baselines across the board, making the proposed approach very promising.

Critical weaknesses

As stated above, the paper is a strong one and the following is more like small nitpick rather than a true weakness.

  • The experimental study is strong, but its mostly limited to ImageNet-256 and CIFAR-10. Therefore, it’s unclear how well it scales to larger settings such as text-to-image and text-to-video generation.
  • The paper also explicitly states in lines 194-200 that they forego additional auxiliary objectives such as the commonly used GAN loss in DMD2. Therefore, it’s not immediately clear how the improvements gained from using data-space iteration vs. standard DMD transfer to the settings with these additional losses in play. This becomes a greater concern for larger scales where such adversarial losses make a big difference in final quality.
Questions for authors
  • One difference between data-space iteration and standard flow-renoise DMD beyond how rollouts are generated is the noise used at different steps. In data-space iteration, it is kept constant whereas in flow-renoise we resample new noise at every intermediate step for renoising. Have the authors tested either flow-renoise with fixed noise or data-space with randomized zz in their experiments?
  • The paper points out the similarity between data-space iteration and fixed-point models. This makes me curious what happens when you fix rr and treat the model like a fixed point one. Have the authors tried doing this and seeing whether the images keep improving and hitting a plateau or if it starts to diverge?
  • Does performing data-space iteration have any impact on the output diversity after distillation? Intuitively, before any training is done, all the 1-step outputs of the generator would point towards the data-mean, and I wonder whether the lack of renoise will make it easier for the model to ignore zz and focus on just refining x0^\hat{x_0} into a single good sample.
1 comment
AuthorsOpenReview ↗

Response to Reviewer giqg

We thank the reviewer for the positive assessment of our work. We address the remaining questions below.

1. Flow-renoise with fixed noise

The original DMD work uses flow-renoise with freshly resampled noise. As the reviewer suggested, we additionally trained a flow-renoise model under the same setting while keeping the noise fixed. The results show that fixing the noise slightly improves FID at 1 and 2 NFEs, but not at 3 or 4 NFEs. Data-space iteration still outperforms flow-renoise with fixed noise across all numbers of function evaluations.

1NFE2NFE3NFE4NFE
Flow-renoise4.453.222.672.75
Flow-renoise fixed noise4.202.742.662.78
Data-space (Ours)3.852.682.472.46

2. Fixed-point iteration beyond the trained maximum

As the reviewer suggested, we experimented with performing additional fixed-point iterations beyond the maximum used during training. For iterations beyond 4 NFEs, we fixed G(x,z,r=3)G(x, z, r=3) (r is trained for 0,1,2,3). The results show no consistent improvement beyond the trained limit; overall, FID begins to degrade.

1NFE2NFE3NFE4NFE5NFE6NFE7NFE8NFE
Data-space (No CFG)3.852.682.472.462.482.452.522.51
Data-space (CFG)3.312.472.332.292.392.342.362.43

3. Data-space iteration preserves diversity

We would like to emphasize that all three iterative methods (data-space, flow-renoise, and flow-map) use a velocity-prediction initialization, in which the initial step points toward the data mean. Under ideal assumptions, DMD—or, more precisely, variational score distillation (VSD)—ensures that the model converges toward the full data distribution. Prior work in 3D generation (ProlificDreamer [1]) has even demonstrated VSD-based distillation into randomly initialized student models, suggesting that the initialization should not be an issue. Empirically, Table 5 shows that the data-space iteration method achieves both the best recall and the best FID compared with the flow-renoise and flow-map iteration methods.

4. Addressing the additional concerns

We chose not to include a GAN loss in order to provide a simpler and more reproducible setting. In fact, recent methods (rCM [2] and Self-Forcing [3]) have achieved state-of-the-art performance on large-scale models without using GAN losses. We believe that our setting may therefore be more aligned with recent trends.

Conclusion

In this rebuttal, we provided an additional flow-renoise ablation with fixed noise, evaluated fixed-point iteration beyond the trained maximum, clarified why data-space iteration preserves diversity, and explained our decision not to use a GAN loss. These results and clarifications address the reviewer's remaining questions while further supporting the effectiveness of data-space iteration.

Citations

  1. ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation (NeurIPS 2023) (https://arxiv.org/abs/2305.16213)

  2. Large Scale Diffusion Distillation via Score-Regularized Continuous-Time Consistency (ICLR 2026) (https://arxiv.org/abs/2510.08431)

  3. Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion (NeurIPS 2025) (https://arxiv.org/abs/2506.08009)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.