- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes TraceSampler to improve low-step sampling. Using a higher-step integration over the same interval as the reference, the method decomposes the endpoint error into three components:
- A shared residual: all errors at a timestep t share this term.
- Directions: a linear combination of different directions (2 for predictor and 4 for corrector), where the directions are computed from the trajectory so far and the coefficients are outputs of a learned network taking different features computed from the trajectory.
- A low-rank remainder: to model the remaining error, where the low-rank basis is computed offline and the coefficients are output by the same network as the direction coefficients.
The paper also proposes a way to optimize the timestep schedule. The method is evaluated with Rectified Flow on CIFAR-10, SiT-XL/2 on ImageNet-256, and SD3.5 Medium for text-to-image generation, and compared against Euler, IIA-Euler, and UniPC (Euler only on SD3.5), and outperforms the baselines on most metrics.
Critical strengths
- The experiments show the improvement over baselines across three generative models, different numbers of steps, and different baseline solvers. The inference overhead is small. The training overhead is a few hours, but it's a one-time cost per step budget.
- The ablation study shows the contribution of each component and confirms that using all components with the optimized schedule has the best performance.
Critical weaknesses
- The motivation for several design choices remains unclear. The directions and features (Sec. 3.2, App. A.1) are quite heuristic, for example, the target blocks, how many target blocks, etc. It is unclear whether other feature engineering would work equally well or whether, as long as the features are sufficiently rich, the method would still work. The writing is also hard to follow and not consistent. For example, since each column of and $ \tilde {C} _ k $ has the same dimensionality as the sampling state, readers need to assume that $ \hat {r} _ {p,k} $ and $ \hat {r} _ {c,k} $ in Eq. (1) and (2) are vectorized. The notation in (line 182, 183) is used before its definition in line 243. In equation (3), the paper introduces , while in Appendix B it does not appear, but .
- Positioning against related works: Compared to IIA, TraceSampler uses a model with more capacity to model the error, where multiple directions are considered, and the coefficients are not learnable scalars but outputs of a learnable neural network. AMED-Solver[1] also predicts the parameters (intermediate timestep , scaling factor ) with a neural network from timesteps , and the bottleneck feature . Furthermore, methods such as LD3[2] and GITS[3] are also relevant to optimize the timestep schedule. These need discussion to clarify the contribution.
[1] Zhou, Z., Chen, D., Wang, C., & Chen, C. (2024). Fast ODE-based Sampling for Diffusion Models in Around 5 Steps. CVPR 2024, 7777–7786.
[2] Tong, V., Hoang, T.-D., Liu, A., Van den Broeck, G., & Niepert, M. (2025). Learning to Discretize Denoising Diffusion ODEs. ICLR 2025.
[3] Chen, D., Zhou, Z., Wang, C., Shen, C., & Lyu, S. (2024). On the Trajectory Regularity of ODE-based Diffusion Sampling. ICML 2024, PMLR 235, 7905–7934.
Questions for authors
- Can you elaborate on the design of the predictor and corrector directions, for example, why is the initial noise involved in the corrector directions but not in the predictor directions?
- How does TraceSampler differ from AMED-Solver?
- Optional: Ablation studies to show which directions are the most important and how the coefficients vary among samples.