- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes second-order updates for the beginning and end of diffusion sampling, together with schedules that allocate more of the trajectory to these updates. It combines them with existing solvers and uses a mean estimate, or a model prediction in the SDXL experiments, to initialize sampling. Results across several models show improved sample quality at low NFE.
Critical strengths
-
The method works with existing solvers and pretrained models. When a mean estimate is available, the second-order first step requires only one evaluation of the main model. This is useful when the sampling budget is small.
-
The experiments show gains across multiple models and solvers. Improvements with the analytic Extend schedule (Tables 3–6) are particularly reassuring, since they do not rely on the modified GITS search.
Critical weaknesses
-
The methodological novelty is less clear to me. LS-1 is the usual final denoising step, while FS-2 and LS-2 have a two-stage Runge–Kutta form after the variable changes in Appendix A. The paper could explain more clearly what the endpoint analysis and initialization add to these existing ideas.
-
The gain from the boundary updates is difficult to separate from that of the schedule. Figure 14 compares FS-only and FSLS at equal NFE, but gives them different freedom to optimize the final interval. The additional LS gains are also mixed, leaving uncertainty about the benefit of the last-step update itself.
Questions for authors
-
Could you explain the relationship between FS-2/LS-2 and existing second-order integration schemes, and identify the contribution of the endpoint formulation and initialization?
-
At equal NFE, could you isolate the gains from LS-2 and from FS-2 without a precomputed mean, using baselines with comparable timestep optimization?
3 comments
W2. The Contribution of LS2 under a Fixed Schedule
The additional LS gains are also mixed, leaving uncertainty about the benefit of the last-step update itself.
We have added ablations using the same timestep nodes without a precomputed mean and find that adding the LS module can affect sample quality in certain settings. In models such as RAE and PixelDiT, this issue is caused by the “curse of dimensionality” encountered when directly predicting noise in very high-dimensional spaces at high SNR [1]. To investigate this issue, we additionally conducted experiments with JiT-L/32 and PixelDiT2 [2]. Both models use the v-loss discussed in our paper, but their networks predict the signal. Consequently, they can accurately estimate high-dimensional noise even at high SNR, allowing the LS module to improve sample quality on these models. We will include a more detailed discussion of this behavior in the paper.
The four values in each cell correspond to DDIM / DPM-Solver++2M / UniPC / iPNDM-V, respectively. FSLS uses FS cache where applicable, and the total budget is 5 NFE. For the initial signal estimate, we use one network evaluation to obtain the signal anchor, following an approach similar to Table 2 in the paper by choosing a sufficiently small . This evaluation is included in the NFE budget.
| FS + LS | FS | LS | |
|---|---|---|---|
| CIFAR-10 | 9.84/10.80/7.67/9.74 | 20.16/14.29/9.85/14.68 | 12.94/15.44/10.98/12.84 |
| AFHQv2 | 6.49/5.68/5.19/6.22 | 9.88/8.34/8.43/8.14 | 11.72/10.05/9.73/10.89 |
| FFHQ | 12.12/11.05/15.49/12.60 | 18.45/12.03/15.93/14.64 | 22.07/20.56/23.03/23.41 |
| EDM2-S | 10.46/8.52/8.15/9.66 | 18.56/9.81/9.30/10.91 | 18.14/13.59/11.79/16.41 |
| RAE | 12.02/13.68/12.05/12.06 | 9.53/12.47/11.38/9.36 | 17.79/20.25/19.72/17.29 |
| PixelDiT | 13.43/11.70/12.51/12.43 | 12.12/9.99/9.98/10.08 | 16.77/15.93/16.19/15.60 |
| Additional Models | |||
| JiT-L/32 | 11.29/10.29/10.00/10.71 | 15.67/12.45/12.16/12.88 | 12.51/11.66/11.37/12.31 |
| PixelDiT2-H/16 | 10.16/9.20/8.63/10.86 | 13.30/11.54/10.65/12.43 | 11.03/10.11/9.37/11.40 |
Q2. Ablations with Independently Optimized Schedules
At equal NFE, could you isolate the gains from LS-2 and from FS-2 without a precomputed mean, using baselines with comparable timestep optimization?
We have added ablations with independently optimized timestep nodes and without a precomputed mean.
| FS + LS w/ EFS | FS + LS | FS | LS | |
|---|---|---|---|---|
| CIFAR-10 | 10.53/8.37/4.68/8.65 | 9.84/10.80/7.67/9.74 | 15.73/9.66/10.06/11.65 | 11.39/10.37/9.90/11.06 |
| AFHQv2 | 7.86/6.61/5.27/5.37 | 6.49/5.68/5.19/6.22 | 9.11/7.45/7.39/7.45 | 10.06/7.96/8.69/9.10 |
| FFHQ | 13.58/11.26/8.91/11.98 | 12.12/11.05/15.49/12.60 | 17.39/20.78/17.36/15.34 | 19.56/17.10/16.72/18.62 |
| EDM2-S | 7.94/5.65/7.31/5.87 | 10.46/8.52/8.15/9.66 | 18.16/8.11/7.08/13.05 | 14.24/11.27/10.40/15.45 |
| RAE | 10.02/10.03/11.05/10.99 | 12.02/13.68/12.05/12.06 | 11.54/18.42/11.89/11.74 | 19.94/17.02/16.88/15.09 |
| PixelDiT | 9.39/7.95/7.36/8.77 | 13.43/11.70/12.51/12.43 | 11.51/10.21/11.05/10.53 | 17.80/16.23/15.71/16.13 |
| Additional Models | ||||
| JiT-L/32 | 9.56/7.50/7.00/8.25 | 11.29/10.29/10.00/10.71 | 14.47/11.32/10.93/11.61 | 12.62/10.63/10.99/11.44 |
| PixelDiT2-H/16 | 7.23/6.46/6.01/7.31 | 10.16/9.20/8.63/10.86 | 13.61/11.58/10.38/11.16 | 11.10/9.53/9.87/10.22 |
Without a precomputed mean, at matched NFE and with comparable timestep optimization, FS+LS outperforms LS-only in 30/32 configurations, demonstrating the additional benefit of FS2.
Across all 8 configurations of JiT-L/32 and PixelDiT2, FS+LS outperforms FS-only. This holds under both fixed and independently optimized schedules, further demonstrating the benefit of LS2.
References
[1] Li, T., et al. Back to Basics: Let Denoising Generative Models Denoise. CVPR 2026.
[2] Yu, Y., et al. PixelDiT2: Representation-Grounded Pixel Diffusion Transformers. arXiv 2026.
Thank you for the additional explanations and comparisons at the same sampling budget. They address my main empirical concern by showing that FS-2 generally improves FID even without a precomputed mean, while the gains from LS-2 vary across models. My overall assessment remains positive, and I will keep my score at 3.
We thank the reviewer for the careful assessment and the specific questions about the relationship between our endpoint formulas and existing second-order methods, as well as the individual contributions of FS2 and LS2. We further clarify the roles of endpoint analysis and initialization and provide ablations without a precomputed mean at matched NFE and with comparable timestep optimization.
W1 & Q1. Relationship to Second-Order Integration and the Contribution of Endpoint Initialization
FS2 and LS2 have a two-stage RK2 structure under appropriate changes of variables. Our endpoint analysis identifies the coordinates and prediction forms for this structure:
=D_\theta(z_\lambda,\lambda),\qquad \frac{d(z_\lambda/\alpha_\lambda)}{d(e^{-\lambda})} =\epsilon_\theta(z_\lambda,\lambda).$$ The former provides a signal-form closure at the low-SNR endpoint, while the latter provides a noise-form closure at the high-SNR endpoint. **This asymmetry determines which quantities to integrate, which boundary conditions to use, and how to avoid singular normalization at the two endpoints.** Proposition 1 further establishes the corresponding higher-order endpoint expansions and error orders. LS1 corresponds to conventional terminal denoising as the first-order case of this framework. For the first step, the analysis identifies the required signal anchor, whose ideal zero-SNR limit is the conditional mean. EFS provides this initialization without an additional main-network evaluation; alternatively, it can be replaced with a network prediction whose evaluation is counted toward the NFE budget. The ablations below use the latter approach to directly examine the benefits of FS2/LS2 without a precomputed mean.