- Rating
- 2
- Confidence
- 4(+1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper addersses the problem of how to further improve the same image with a small amount of additional computation after sampling has finished. It proposes to select the steps with the largest errors on a completed Euler trajectory, upgrade them to Heun, and propagate the corrections to subsequent steps using a linear approximation derived from the trajectory. On SD3, SD3.5, and FLUX, it shows that under the same total compute, ImageReward is generally higher than simply taking more Euler steps. However, it lacks comparisons with multistep solvers, and the theoretical goal of getting close to the exact solution is not directly verified.
Critical strengths
(1) The problem is clearly defined and the method is cleverly designed: the correction direction coincides with a direction that can be estimated directly from the trajectory, so propagating correction to subsequent steps requires no additional model evaluations, and the first Heun correction also reuses an existing evaluation.
(2) The experiments are relatively large-scale, which gives the results some credibility.
Critical weaknesses
(1) A large single-step error does not necessarily mean a large impact on the final image. An error at an early step may be amplified or smoothed out by many subsequent steps, so errors of the same size can have different consequences at different times. FixUp ranks steps only by single-step error and does not consider how sensitive the endpoint is to error propagation.
(2) Where the error is large, Heun does not necessarily correct it well. FixUp may well select the intervals where Heun is least reliable, and further experiments are needed to rule this out.
(3) The theory and the experiments do not match. The theory guarantees a solution closer to the exact ODE solution, but the experiments only report ImageReward and CLIP, without the error with respect to a high-accuracy reference solution.
(4) How common is the premise of “run Euler first and fix it afterwards” in practical pipelines? If a better sampler is used from the start, is this method still needed? Hence, the most direct baselines are missing. The improvement obtained without any additional model evaluations indicates that part of the gain comes from recombining existing velocities, which multistep solvers such as DPM-Solver++(2M) and UniPC already do at zero cost.
Questions for authors
Pls see weaknesses
8 comments
What does the theory guarantee, and does reference error improve?
FixUp is a post-sampling method, not a standalone solver. Corollary 1 and Eq. (16) bound the gap to an ideal selective rerun; they do not establish a smaller error relative to the exact ODE solution than a fresh NFE-matched solver. The ideal rerun uses the same grid and selected corrections but re-evaluates the true velocity at every step. Under the assumptions and a fixed budget, the gap is second order in the maximum step size, preserving the rerun's leading first-order accuracy while avoiding downstream model calls. We apologize for the ambiguity and have clarified the comparison target after Eq. (16).
To assess reference accuracy directly, we measured latent MSE against a 1,000-step Euler reference on 1,024 SD3 prompts, using the three main-paper operating points.
| Original Euler sample | |||
| FixUp-FM | |||
| Restart | |||
| SDEdit | |||
| Fresh NFE-matched Euler |
Entries are latent MSE (↓). Superscripts denote total NFE minus ; fresh Euler is rerun from noise.
FixUp is the only evaluated post-sampling method that reduces reference MSE at all three points, while remaining close to fresh NFE-matched Euler. A practical distinction is that FixUp achieves this accuracy without restarting sampling. Improving an existing 20-step sample takes one additional evaluation, whereas fresh 21-step Euler requires 21 new evaluations.
Is Euler-first refinement useful when stronger samplers are available?
FixUp is an optional post-sampling method, not a standalone solver, allowing users to improve a sample they already have. We develop this approach on Euler trajectories for their simplicity and broad applicability, with practical gains measured from the existing -step sample.
Our contribution is to turn a completed trajectory into a reusable resource, revisiting selected intervals and propagating corrections to the output without re-evaluating every downstream velocity. We hope this trajectory-reuse perspective can inspire future work on post-sampling extensions to higher-order samplers, for example by using cached Runge–Kutta stages and embedded defect estimates to guide retrospective corrections.
We additionally evaluated fresh Euler, Heun, DPM-Solver++(2M), and UniPC on FLUX.1-dev with 1,024 prompts, using the three main-paper operating points and matched initial noise. FixUp remains competitive with strong fresh solvers.
| FixUp-FM | |||
| Euler | |||
| Heun | |||
| DPM-Solver++(2M) | |||
| UniPC |
Entries are ImageReward. Superscripts denote total NFE minus ; fresh solvers are rerun from noise. Heun uses 15 NFEs in the first column to accommodate its two-stage update.
FixUp achieves the highest mean ImageReward among all compared methods at both and , and exceeds fresh Euler and Heun at all three operating points. Users can thus choose to improve a sample they already have with a small additional budget, without committing to a full sampling rerun.
We hope these clarifications and additional experiments address your concerns.
Thanks for the rebuttal and additional experiments. I still have a few concerns:
(1) Evaluation: The results rely heavily on ImageReward. I also ran tests using Diffusers on FLUX.1-dev: at 10 NFE, Euler achieved 0.63 GenEval, compared with 0.58 for Heun (0.62 with an Euler ending). Although this is not a direct test of FixUp, it suggests that higher-order corrections do not necessarily improve compositional quality. A GenEval evaluation would be helpful.
(2) Reference accuracy: FLUX.1-dev typically use around 28–50 sampling steps, as in the official Diffusers documentation and published studies. Some studies also report no quality improvement beyond 50 steps. The authors use 1000-step Euler as a reference without showing convergence. Does lower MSE against this reference actually correlate with better image quality?
(3) Generality: The method mainly focsues on Euler trajectories. Its effectiveness across different schedules, few-step budgets, and stronger solvers remains unclear.
Overall, I'm still not fully convinced about the practical benefits and generality. I'll keep my original score for now.
Thank you for the thoughtful review and for recognizing our correction mechanism and large-scale evaluation.
Does local-error ranking help control endpoint error?
Our top-K rule exactly minimizes the remaining-defect term in the terminal-error bound of Theorem 3, Eq. (14). The bound accounts for downstream amplification and includes correction-related remainders.
We additionally evaluated two endpoint-aware alternatives. Terminal-norm propagates each local defect to the endpoint using the cached linear propagation model and ranks the resulting magnitudes. Greedy-terminal selects intervals that most reduce the squared norm of the remaining aggregate endpoint defect.
On SD3 Medium with 512 prompts, a 21-step trajectory and three corrections, all methods use 23 total NFEs.
Our score achieves the highest mean ImageReward with the simplest selection rule.
Is Heun reliable on the selected high-error intervals?
We tested 64 SD3 prompts. Euler and Heun start from the same left-end state and take one step across each selected interval. We measure local MSE against an RK4 reference computed with 32 substeps.
Heun improves local accuracy on 317 of 320 selected intervals, reducing mean local MSE by 79.2%, 85.1%, and 87.3% at base steps 12, 20, and 32, respectively.
Repeating the comparison with 8- and 16-substep RK4 references gives exactly the same interval-wise winners. These checks support applying Heun precisely where FixUp selects corrections, addressing the concern that selection targets unreliable intervals. We have included the full results and reference-refinement checks in Appendix N (Table 15).