- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The goal of the paper is to accelerate sampling of diffusion and flow models. Their approach does not change the model weights but rather optimize the sampler of the model.
A planned sampler for a fixed budget of NFE is a time schedule and a solver schedule , where is a step of an existing numerical solver e.g., Euler, Heun, DPM2++, and more, such that the sum of the NFE of the total number of steps steps is equal to the NFE budget. In practice their approach to accelerate the sampling is via a dynamic programing algorithm that plan solvers which they propose.
The main contribution of the paper as I understand them are:
-
Given a noise which the high accuracy solution maps to a sample (i.e., image), they propose new way to measure local error. Instead of measuring error w.r.t. to the solution the ground truth solution of the ODE, they measure is w.r.t. to the interpolant of and : Then they provide a bound of the global error using the proposed new local error. This bound is minimized to optimize the planned sampler.
-
They provide empirical analysis of the local error along the trajectory.
-
They provide empirical validation of their algorithm on SDXL, Qwen-Image, Qwen-Image-Edit, and Z-Image-Turbo. Notably, the reported results actually shows that the best performance can always be achieved via constate solver schedule (i.e., single solver type).
Critical strengths
Accelerating samplers of diffusion and flow model is a relevant critical problem for many application. And since the approach they propose essentially squeeze the efficiency of the model using existing solvers, without the need for data or to change the model's weights. Hence it could potentially be useful to many users.
Additionally, they provide a principled derivation of the global error bound they optimize, and they provide nice empirical analysis of local error of different solvers on multiple models.
Critical weaknesses
The two main weaknesses which I identify in the paper are significant. First, the proposed local error w.r.t. to the interpolant path is not motivated well and the authors do not discuss any geometrical interpretation of such definition of error. Second, is weak empirical results. In practice the reported results shows that the best performance can always be achieved via a single solver, and the method mostly optimize the time schedule.
Error w.r.t. to the interpolant: I am concerned that the definition of the error is not well justified in the paper. I might be missing something in my understanding, but I will present my current understanding.
Obviously the interpolant is not a solution to the ODE. Hence, the fact that the author defines the error w.r.t. to this path without any motivation, geometrical discussion, or what is the meaning of this error in limit of high accuracy solver (clearly it does not approach zero) makes the paper hard to interpret. Note, that even the connection to global truncation error via the bound does not help so much as there is no discussion about how tight the bound is. Even the section "DOES THE LOWEST PATH COST PICK A GOOD PATH?" seems to compare ImageReward without any mention the global error as defined in Proposition 5. Hence, its not clear that minimizing the bound in Proposition 5 actually reduce the global error.
Additionally, since the reference interpolant path is already defined via the map between noise to image of the model, it is already required to solve the ODE with high accuracy (50 steps in the paper). So I would expect the authors to motivate their choice of reference path over defining the reference path to be the peace-wise linear interpolation between every point on the trajectory of the high accuracy solver (This is instead of the linear interpolation between the endpoints as currently in the paper). The latter, appears to be the naive approach to approximate the exact solution of the ODE and it does not require more compute.
Lastly, it seems to me that minimization of the bound proposed by the authors essentially tries to make the trajectories straight. As implied by Proposition 5, If succeeded, then indeed global error should be minimized as well. However, not every map between noise to image allows such a flow. This is discussed in rectified flows[1], they also straighten the trajectory by making the model follow the straight paths connecting and . Hence, I think it might be appropriate add a discussion about the connection to rectified flows[1].
Weak empirical results: Tables 1 and 2 shows that the mixed solver schedule is not needed, the proposed method mostly optimize the time schedule. First, it seems that proposed algorithm for finding the best solver schedule is an over kill, and simple grid search over three solvers is enough. Second, it would be appropriate as well as useful to the reader to compare with the popular "time shift"[2] reparametrization on a grid of shift parameter values. It is safe to say that this time transformation is already standard and it is used as the default time schedule for many open source models (including Qwen-Image).
[1] Liu, et al. "Flow straight and fast: Learning to generate and transfer data with rectified flow." [2] Esser, et al. "Scaling rectified flow transformers for high-resolution image synthesis."
Questions for authors
Can the authors please address my concerns regarding the definition of the error w.r.t. to the interpolant? Can the authors motivate why this definition is superior to error w.r.t. the ground truth solution of the ODE which can be approximated by peace-wise linear interpolation between the points on the trajectory of the high accuracy solver?
Can the authors provide empirical evidence that minimizing the proposed global bounded indeed reduce global error?
Perhaps, consider to provide empirical evidence that the proposed bound is better than the naive approach (as I described it)?
To Reviewer uqDx
Dear Reviewer uqDx,
As the discussion period is closing, we would like to check whether our response has addressed your questions. The revised PDF adds the window against a high-accuracy ODE solution, paired 95% intervals, the Euler-ending and guidance experiments, and blind preference tests (Appendix R, changes in blue). The revised abstract and introduction also state the question the paper answers, and Section 5.3 states that every row of Table 1 is planned by the graph, each solver on its own schedule at equal NFE, which bears on your Q4 and W2(iii). We would be glad to clarify anything that remains open. Thank you for your time.