- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper addresses two practical limitations of Distributional Diffusion Models: the cost of processing multiple stochastic predictions for each noisy example, and the difficulty of using one fixed scoring-rule configuration across different sampling budgets. It proposes sharing early transformer computation across particles and injecting particle-specific noise only in late layers. It combines this architecture with adaptive noise conditioning, time-dependent generalized energy-score parameters and , and a modified training-time distribution.
The resulting iDDM model is trained without a diffusion teacher or self-distillation. ImageNet experiments include architectural and objective ablations, independent component additions, compute-matched comparisons, additional quality and coverage metrics, and a conditional-diversity diagnostic. At XL scale, the paper reports FID values of 4.48, 3.04, 3.03, and 2.38 at 4, 8, 16, and 50 sampling steps, respectively. A text-to-image experiment also shows improved performance at small sampling budgets, with some important compositional limitations.
The contribution is principally a computational and empirical improvement to an existing distributional-training framework, supported by a limited theoretical rationale for the direction of its schedules.
Critical strengths
-
A consequential computational modification, with evidence about its representational trade-off. Deferred expansion separates feature extraction shared by all particles from the computation that must depend on auxiliary noise. This changes the placement of stochastic computation while preserving shared parameters across particles; it is not simply a renamed loss or an independently parameterized multi-head model. For , , and , the idealized block-count cost is of naïve DDM, or approximately 1.43 times single-particle FM. Table 12's measured XL training-step ratio is approximately 1.41 relative to FM, supporting the practical significance of the saving.
Table 2(a) and Figure 4 also test the important boundary: too little particle-specific computation can hurt quality, and unguided generation prefers more distributional layers, whereas guidance can reverse that preference. Selecting a short distributional tail is therefore supported by comparisons, not merely asserted as an efficiency heuristic. This offers a useful direction for future stochastic generative architectures.
-
A substantial, reasonably controlled body of validation supports a complete empirical contribution. The paper includes cumulative ablations in Table 1, conditioning and depth comparisons in Table 2, schedule and time-sampling comparisons in Table 3, independent additions in Table 8, and multiple quality and coverage metrics in Table 9. Most importantly, Table 10 addresses the extra training cost: at the larger matched budget, four-step FID improves from 26.36 for FM to 13.13 for iDDM, while fifty-step FID is 4.21 versus 4.61. Thus, the few-step benefit survives compute matching, while the many-step trade-off is visible.
The final XL checkpoint also improves monotonically across the reported step grid, and Table 13 extends the method to text-to-image with improvements in four-step FID, GenEval, human-preference score, and most compositional categories. Together, these results establish more than a parameter-tuning anecdote. The authors also acknowledge stronger alternatives at very small budgets and do not need to solve posterior calibration or every application to make this a complete empirical study.
Critical weaknesses
-
The full-posterior interpretation is stronger than the default objective and diagnostics justify. Strict propriety of the global energy score applies at and . The default schedule deliberately uses away from the noise endpoint, introducing a bias whose Gaussian-surrogate interpretation is discussed in Appendix D. In addition, the default local score in Appendix F identifies per-position marginals in the proper-score regime, not the full joint law across positions. For example, equal mixtures on and have identical coordinate marginals and identical expected local scores, despite opposite dependence structures.
Appendix F explicitly acknowledges the loss of cross-spatial constraints, and Appendix B correctly states that Table 11 measures conditional diversity rather than posterior accuracy. These are appropriate caveats, but they should also qualify the abstract and the main-text characterization of the model as learning . The evidence establishes a useful stochastic, time-dependent, biased distributional predictor; it does not establish calibrated full-posterior sampling.
The regime connection is similarly a rationale for schedule direction, not a demonstrated necessity of the estimated transition times. Appendix D's rearrangement argument supports assigning stronger shrinkage to higher SNR under its covariance criterion. The selected linear schedule does not require those thresholds, and Table 3 contains alternatives with comparable or better FID in some settings. This limits the conceptual claim, although it does not undermine the practical recipe.
-
The strength and boundaries of the claimed performance frontier need tighter characterization. The large improvement over naïve DDM and the matched-compute FM result are convincing. However, the tables do not report uncertainty across independent training runs. Smaller choices, such as linear versus dynamical-regime schedules or isolated conditioning refinements, are therefore not established with the same confidence. Table 1 also shows that adding the schedules improves four-step FID from 15.74 to 14.33 while worsening fifty-step FID from 5.47 to 5.69; the final improvement at both budgets follows the subsequent change to time sampling. The evidence should be described as a tested combination with component interactions, rather than assigning every gain to the schedule alone.
The transfer result has a meaningful exception. In Table 13, iDDM's spatial-relation score is below FM at every evaluated budget: 0.1178 versus 0.2901 at sixteen steps, and 0.0232 versus 0.0443 at four steps. Smaller degradation from an already lower reference score does not imply better absolute compositional performance. The text-to-image comparison also uses equal update counts rather than an established matched-compute budget.
Finally, Tables 4 and 12 should be read with their stated architectural and training differences. The paper already separates controlled experiments from system-level context, which is appropriate. Its stronger total-training-cost comparisons nevertheless depend on the backbone and implementation used for benchmarking, and its monotone-FID result depends on separately selected CFG scales at the tested budgets. Clarifying these conditions would improve confidence in the claimed efficiency frontier.
Questions for authors
-
How should the full-posterior language be interpreted given the default local score and ? Please distinguish stochastic conditional prediction, identification of local marginals, and calibration of the joint posterior. Can you provide a tractable conditional-distribution check, or explicitly narrow the claim to what the objective and Table 11 establish? This could change my assessment of the paper's conceptual contribution; I am not requesting large-scale posterior-reference experiments.
-
Does the selected time-dependent schedule outperform a jointly retuned constant configuration under the same late-expansion architecture, time sampler, compute budget, and per-budget guidance selection? Please identify the relevant comparison and explain which result depends on the regime anchors rather than on generic time-dependent weighting. Appendix D justifies direction under a surrogate, but Table 3 does not establish a uniquely best profile; this distinction affects the claimed novelty and mechanism.
-
How many independent training runs and evaluation draws support the main results and the smaller ablation differences? Please give uncertainty for the decisive schedule comparisons and state whether CFG and EMA choices were selected on a separate tuning set or the reported evaluation samples. Also, does the final checkpoint retain its qualitative improvement with one fixed CFG across budgets, or only after selecting CFG separately? These answers would affect confidence in the reported frontier.
-
Is the spatial-relation deficit in Table 13 reproducible across evaluation draws and reasonable guidance choices, and what evidence helps explain it? Is the FM/iDDM transfer comparison matched in total training compute, or only in training steps? Please distinguish the supported few-step improvements from compositional capabilities that worsen. This could change my assessment of the breadth of transfer; it is not a request for additional application domains.
-
Which depth and implementation were used for each Table 12 benchmark, especially iMF, whose published Table 4 checkpoint has 48 layers? Do the total-cost ratios combine compatible measured step costs and training lengths? Please also state inference latency or FLOPs under matched guidance settings, and clarify whether the reported step/NFE counts include conditional and unconditional CFG computation. These details could materially affect the practical efficiency comparison.
4 comments
Response 2
[Q2] Does the selected time-dependent schedule outperform a jointly retuned constant configuration under the same late-expansion architecture, time sampler, compute budget, and per-budget guidance selection?
We thank the reviewer for this suggestion and have run the requested control. We trained a 5×5 grid of constant configurations under the same late-expansion architecture, time sampler, compute budget, and per-budget guidance selection as Config D (which corresponds to the cell). The resulting heatmap is included as Figure 5 in Appendix B.1.
The grid confirms that the preferred constant depends on the inference budget, consistent with the original DDM paper (De Bortoli et al., 2025, Appendix K.2): the best 4-step constant is = (1, 1.95), with FID 14.93 at 4 steps and 5.27 at 50 steps, whereas the best 50-step constant is , (which is very close to a standard diffusion model with )), with 4.97 at 50 steps but 25.71 at 4 steps. Time-dependent schedules improve on this reference in the same setting (Table 3b). A -only schedule with an early switch at t=0.1 reaches 13.06 at 4 steps and 5.10 at 50 steps, improving over the best 4-step constant by 1.87 FID while remaining within single-seed noise of the best 50-step constant (5.10 vs. 4.97; [1]). Adapting the scoring rule across diffusion time therefore outperforms every constant configuration we tested at 4 steps, without a measurable cost at 50 steps.
Our default linear schedule (Config E) yields more modest improvements: relative to the best 4-step constant, it improves 4-step FID from 14.93 to 14.33, at the cost of 50-step FID (5.27 to 5.69). We retain this simple linear parameterization for our main experiments, while the early-switch results demonstrate that alternative time-dependent schedules can achieve further improvements over jointly retuned constants.
[1] Dufour et al., 2026, The FID Lottery
[W1] The regime connection is similarly a rationale for schedule direction, not a demonstrated necessity of the estimated transition times. Appendix D's rearrangement argument supports assigning stronger shrinkage to higher SNR under its covariance criterion. The selected linear schedule does not require those thresholds, and Table 3 contains alternatives with comparable or better FID in some settings. [Q2] Please identify the relevant comparison and explain which result depends on the regime anchors rather than on generic time-dependent weighting [W2] Table 1 also shows that adding the schedules improves four-step FID from 15.74 to 14.33 while worsening fifty-step FID from 5.47 to 5.69; the final improvement at both budgets follows the subsequent change to time sampling. The evidence should be described as a tested combination with component interactions, rather than assigning every gain to the schedule alone.
The dynamical regimes motivate the direction and transition regions of the schedules, not uniquely optimal thresholds. As explicitly stated in Section 3.2, “The regime thresholds interpret the schedule direction but are not required in practice.” Appendix D further supports the orientation analytically. Unlike generic time-dependent weighting, scheduling and changes the scoring rule itself. Importantly, Table 1 (D E) and Table 8 isolate its contribution: 4 step FID improves from 15.74 to 14.33, with only a minor change at 50 steps (5.47 to 5.69). The subsequent improvement from t-sampling is complementary and separately ablated.
[Q3] state whether CFG and EMA choices were selected on a separate tuning set or the reported evaluation samples. Does the final checkpoint retain its improvement with fixed CFG across budgets?
We use fixed EMA length of [4], we don’t sweep this value. CFG was selected by minimizing FID under the standard 50k-sample evaluation protocol, consistently applied across all methods. Selecting CFG independently for each sampling budget is standard practice [1,2,3] and essential for a fair comparison, since optimal guidance generally varies with the number of steps [5].
FID typically follows a U-shaped curve with CFG, with fewer steps favoring stronger guidance. Fixing a low CFG yields monotonically improving FID, but sacrifices optimal quality at individual budgets. Even with CFG optimized separately at each budget (common practice [1, 2, 3]), some competing methods degrade as the step count increases, whereas iDDM consistently improves.
[1] Jonathan Ho et al., 2022, Classifier-Free Diffusion Guidance.
[2] Prafulla Dhariwal et al., 2021, Diffusion Models Beat GANs on Image Synthesis.
[3] Zhengyang Geng et al., 2026, Improved mean flows: On the challenges of fastforward generative models.
[4] Tero Karras et al., 2024, Analyzing and Improving the Training Dynamics of Diffusion Models.
[5] Dufour et al., 2026, The FID Lottery
Response 3
[W2] However, the tables do not report uncertainty across independent training runs. [Q3] How many independent training runs and evaluation draws support the main results and the smaller ablation differences?
All models are trained with a single seed per configuration, consistent with common practice [1] in computationally intensive generative modeling. Our main conclusions are supported by large differences: under controlled DiT-B settings, iDDM reduces 4 step FID from 43.68 to 13.13 and 50 step FID from 10.04 to 4.56 relative to naïve DDM, with large gains also at matched compute.
For perspective, The FID Lottery (Dufour et al., 2026) reports a training-seed FID coefficient of variation of approximately 1-2% on ImageNet-256 using SiT. Although this is not a direct variance estimate for iDDM, the magnitude of our improvements is far beyond the variability observed in that study. Moreover, our improvements persist across model scales, sampling budgets, and complementary evaluation metrics.
[1] Dufour et al., 2026, The FID Lottery
[W2] In Table 13, iDDM's spatial-relation score is below FM at every evaluated budget: 0.1178 versus 0.2901 at sixteen steps, and 0.0232 versus 0.0443 at four steps. Smaller degradation from an already lower reference score does not imply better absolute compositional performance. The text-to-image comparison also uses equal update counts rather than an established matched-compute budget. [Q4] Is the spatial-relation deficit in Table 13 reproducible across evaluation draws and reasonable guidance choices, and what evidence helps explain it? Is the FM/iDDM transfer comparison matched in total training compute, or only in training steps? Please distinguish the supported few-step improvements from compositional capabilities that worsen.
We agree that smaller degradation does not imply better absolute compositional performance. Our claim concerns degradation as the step count is reduced, and we now state this more explicitly. We have added a compute-matched comparison (Table 15), in which iDDM is trained for fewer iterations so that total training compute equals FM’s. There, iDDM still degrades less than FM from 16 to 4 steps on every metric, and at 4 steps it is better in absolute terms on all metrics except spatial relations. On spatial relations iDDM is below FM at every budget, which we now report in the text-to-image section (Appendix B.2).
[W2] The paper already separates controlled experiments from system-level context, which is appropriate. Its stronger total-training-cost comparisons nevertheless depend on the backbone and implementation used for benchmarking [Q5] Which depth and implementation were used for each Table 12 benchmark, especially iMF, whose published Table 4 checkpoint has 48 layers? Do the total-cost ratios combine compatible measured step costs and training lengths? Please also state inference latency or FLOPs under matched guidance settings, and clarify whether the reported step/NFE counts include conditional and unconditional CFG computation. These details could materially affect the practical efficiency comparison.
Table 13 benchmarks all methods within the official meanflow/iMF JAX codebase, using identical hardware and batch settings. FM, meanflow, and iDDM use the same 28-layer DiT backbone, while iMF uses its published 48-layer architecture. Total training-cost ratios combine measured per-step runtimes with official training lengths, yielding approximately 1.67x for meanflow and 17.6x for iMF relative to iDDM. These comparisons reflect the published training recipes, including their architectural differences.
Reported step counts refer to denoising steps, not CFG-adjusted NFEs. Explicit CFG requires two model evaluations per step (e.g., 4 steps correspond to 8 forward evaluations). We updated Table 8 and 9 headers and the abstract from “NFE” (previously incorrectly used, the values reported are already denoising steps) to “steps” to match the convention held throughout the paper.
Regarding inference cost, all iDDM evaluations sample only one particle per denoising step, therefore there is no multi-particle overhead at inference time. Combined with the chosen ξ conditioning mechanism (fixed-inner-width) that preserves the attention and FFN dimensions, the results in near-identical inference cost compared to FM under matched sampling steps and CFG settings (~0.96x FM throughput, measured on DiT-B/2) .
Thank you for the additional experiments and clarifications. The retuned constant-parameter comparisons and compute-matched transfer results strengthen the empirical contribution. I also appreciate the clearer distinction between stochastic conditional prediction and calibrated posterior recovery.
My remaining concerns mainly concern uncertainty around smaller ablation differences and the scope of the schedule and transfer claims, including the spatial-relation deficit. These limitations do not outweigh the demonstrated computational contribution and substantial few-step improvements.
Overall, I maintain my original rating of weak acceptance.
Response
Dear reviewer QdnQ,
Thank you for highlighting the importance of our contributions as "a consequential computational modification", highlighting our novelty "it is not simply a renamed loss or an independently parameterized multi-head model" supported by "substantial, reasonably controlled body of validation supports a complete empirical contribution"
We address your questions below.
We agree that stochastic conditional prediction and accurate joint-posterior recovery are distinct objectives. Importantly, none of our central empirical claims requires posterior calibration, as our contributions concern training efficiency, few-step generation quality, and conditional diversity.
We characterize the theoretical scope of these guarantees:
To make these distinctions more prominent, we have further refined the introduction to distinguish conditional distribution learning from posterior fidelity and explain the deliberate trade-off with prediction concentration. We have also lightly edited Section 4.2 (positioning paragraph) to emphasize stochastic conditional prediction and diversity rather than suggesting accurate posterior recovery. All changes are reported below and available in the pdf and highlighted in blue.
These refinements make explicit that our empirical conclusions do not rely on calibrated joint-posterior sampling, while retaining the distributional motivation underlying iDDM.
lines 71-72 “DDMs thus offer a non-adversarial alternative to learn conditional distributions beyond their means, while potentially allowing the learned predictions to be more concentrated than the true posterior”
lines 88-90 “Removing both bottlenecks makes distributional denoising practical with large transformer backbones while retaining the goal of approximating p(x1∣xt) rather than only its conditional mean, and permitting deliberate variance shrinkage at high SNR”
lines 412-413 “(i)~It is stochastic: it learns a stochastic conditional predictor trained to sample from an approximate p(x1∣xt), whereas MeanFlow, iMF, and IMM learn deterministic maps.”