- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper presents a method for approximating the optimal guidance scaling in classifier-free guidance for every time along the sampling trajectory. Since flow-based generative models obey the continuity equation, they use its weak form to formulate an optimization of the guidance scaling for the next subinterval of the sampling time grid. The optimization problem involves two integrals. These integrals are approximated using Monte Carlo with an ensemble of samples, and the resulting approximate optimization problem is solved exactly. The guidance scaling may be computed online or offline and reused. The method is evaluated with flow matching models for a Gaussian-mixture model and MNIST with KL divergence, Wasserstein-2 distance, and FID metrics. The paper also has ablations for the design decisions when learning the guidance schedule, such as which test functions to use.
Critical strengths
- The need for learning/approximating the guidance schedule is well-motivated. The paper addresses a common issue in test-time guidance of flow-based generative models.
- The optimization problem, the derivation of its solution, and its approximation are presented clearly, and the method integrates well with the sampling procedure of flow-based generative models.
- The paper has thorough ablations of each aspect of the method's implementation (e.g., choice of test functions) and use (sampling with coarser time grids than the guidance was fitted with).
- The learned guidance schedule can be computed offline for some fixed condition label , and reused. Being just a scalar-valued function, storing the guidance schedule costs little memory.
Critical weaknesses
- The method requires approximating multidimensional integrals. The evaluation on MNIST already uses particles. The method is not evaluated on real-world classifier-free guidance tasks (e.g., [1]).
- The method fixes a condition label before learning the guidance schedule. The ability of the guidance schedule learned offline to generalize to different conditional labels is not evaluated. Generalization would be needed when the set of is large (e.g., in text-to-image applications [1, 2]) and learning the guidance schedule online is too computationally costly.
[1] Rombach, Robin. "High-Resolution Image Synthesis with Latent Diffusion Models." arXiv preprint arXiv:2112.10752 (2022).
[2] Saharia, Chitwan, et al. "Photorealistic text-to-image diffusion models with deep language understanding." Advances in neural information processing systems 35 (2022): 36479-36494.
Questions for authors
- How well does a guidance schedule trained offline generalize to unseen conditioning labels ? Can the method support fitting a guidance schedule to multiple conditioning labels to improve generalization?