- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The manuscript proposed to use the gauge symmetry in flow-based models to correct leading-order numerical discretization error when deploying a learned model. The authors investigated idealized Gaussian to Gaussian transport models, and one large numerical experiment on CIFAR-10 dataset. Additional details are presented below in Strengths and Weaknesses sections.
Critical strengths
-
[S1] The manuscript proposes an interesting idea of using gauge freedom to compensate higher-order numerical errors.
-
[S2] Using gauge freedom-inspired corrector for CIFAR-10 dataset is an interesting attempt of post-training fine-tuning or distillation for diffusion or flow-based models.
Critical weaknesses
-
[W1. Limited generalizability.] The basic idea of the proposed method is to inject an additional velocity to transport samples to "make up" the leading-order numerical errors. The reviewer finds this idea interesting, but found the manuscript incomplete to recognize the difficulty of such an approach when the transport lacks of a certain invariant structure.
- For example, suppose the initial distribution is Gaussian but the radial expansion rate depends on the angle, i.e., for . The terminal distribution can be a non-Gaussian one, which is much more relevant setting for generative models. The reviewer has a hard time rationalizing how the proposed method can be achieved globally.
- The blockwise construction does not solve this problem
- A relevant problem is if there's any odd transport left in the system, e.g., even for anisotropic Gaussian transport in with three distinct growth rates, the method cannot be used.
- The authors also pointed out it's not always possible to use the added field for making correction, even under the extremely restrictive isotropic case: in Proposition 2, "If , no real rotational gauge can cancel the term."
-
[W2. Lack of comparison with existing training or distillation methods.] The reviewer finds the idea interesting but as the manuscript currently stands, it lacks discussions why this method is more advantageous than existing methods for improving inference quality of a trained diffusion model.
- The most straightforward idea is to use a higher-order ODE solver. Higher order methods were presented in the manuscript, but accuracy v. computational cost tradeoff is not.
- The underlying problem is some form of curvature or nonlinearity of the ODE solver leads to numerical error. However, built in the diffusion model is the choice of noise schedule, which is tightly connected to a nonlinear transformation of time, in either the typical zero-drift Wiener process (Variance Exploding) or Ornstein-Uhlenbeck process (Variance Preserving); see "Using Ornstein-Uhlenbeck Process to understand Denoising Diffusion Probabilistic Model and its Noise Schedules". For DDPM, the "velocity" is not estimated but instead the conditional mean displacement between discrete times, and a suitable choice of noise scheduling could lead to significantly reduce "discretization error" of the discrete-time ssytem.
- [W2.3] Using a curved trajectory to "add up" higher order error is very contradictory to existing movements of attempting to learn straight-line flows, such as Rectified Flow. The distinction of the proposed method and the pursue of sraight-line flow models is a deep discrepancy in the modeling philosophy: the proposed model seeks literal accuracy of the numerical methods for integrating a prescribed, but learned, velocity field, at every point of time, while the straight-line flows recognize the there exist multiple intermediate trajectories connecting the initial and terminal distributions. During inference time, accurately simulate a poorly learned, curved (in space and in time) trajectory, may not be as effective as a less accurate simulation of a more straight trajectory. Note that in the idealized where a suitable Monge transport map exists, the endpoint transport can be represented through straight-line displacement interpolation, potentially eliminating trajectory discretization errors.
-
[W3. Disconnection between the idealized Gaussian transport and the CIFAR-10 numerical results.] The reviewer finds a large gap between the theoretical exhibition and the numerical experiments on CIFAR-10.
- This ties to [W2.3], that the reviewers found the concept of a theoretical transport velocity that satisfies the Liouville equation, that transports the data distribution and the prior distribution, and a learned transport velocity, mingled in this manuscript. The learned velocity does not have to satisfy the boundary conditions but still satisfies the Liouville equation, see for example, Liouville Flow Importance Sampler, in which the gauge freedom and Helmholtz-Hodge decomposition was extensively discussed. Accurately simulating a learned but inaccurate transport velocity does not ensure good sampling property.
- At a more nuanced level, the gauge freedom can be considered as a vast null space of the Liouville operator, see Vuong et al. 2026. The study established that the learned velocity field may have already had some component in the null kernel space. Using the motivating example of the manuscript, let us consider the true transport as the isotropic and radially expanding Gaussian, let . Let us assume a learned model somehow picked up a spiral signal, which leads to the coupled ODE \dot{y} = y + \varepsilon x$$ or equivalently in a vector form $\dot{\mathbf{x}} = c \mathbf{x} + \varepsilon \mathbf{J}\cdot \mathbf{x}$. Now, applying the proposed method to this *learned* velocity field leads to $c \mathbf{x} + \varepsilon \mathbf{J}\cdot \mathbf{x} + \mathbf{J}\cdot \left[ c \mathbf{x} + \varepsilon J\cdot \mathbf{x}\right]$. While the part $\varepsilon \mathbf{J} \cdot \mathbf{x} + c \mathbf{J}\cdot \mathbf{x}$ lives in the null space, the residual term $\varepsilon J^2 \mathbf{x}=- \varepsilon \mathbf{x}$ does not ensure the marginal distribution. - Noted, in CIFAR-10 the authors operates on the score function, but a similar argument can be established. With this illustrative example in mind, the reviewer has a hard time processing the CIFAR-10 numerical results. In particular, the manuscript does not establish that the proposed correction remains within the Liouville null space when constructed from a learned score, even if the learned score already generates the correct marginal evolution. As such, one cannot rule out the hypothesiis that the reported improvements are due to changes to the underlying continuous-time probability distribution, instead of the cancellation of numerical discretization errors.
- The reported improvements are significant only for high-FID models; For the much better Heun baseline, FID improves only from 3.050 to 2.996. Moreover, the five-run projected-FID evaluation reported in Appendix E does not establish a statistically significant improvement for Heun, as its 95% interval includes zero. This raises questions about the practical significance of the correction when applied to an already accurate sampler. FID is extremely sensitive to number of samples and training checkpoints: in a sense, an additional "training" was performed on the baseline, pre-trained model, by fitting the correction coefficients. The cost should be considered in assessing the overall accuracy–efficiency tradeoff.
Questions for authors
-
[Q1] Outside the highly symmetric analytic cases, what principle determines the gauge family? In particular, why should the chosen 18-dimensional channel/spatial operator bank contain useful solver-error-reducing directions, and how would the authors construct such a family for a new architecture or data distribution?
-
[Q2] The non-Gaussian order-lifting theorem assumes invariance under the full continuous action for every , not merely a discrete rotational symmetry. Can the authors clarify this restriction in the high-level claims and explain whether any analogous result survives under only finite or approximate symmetries?
-
[Q3] Appendix D acknowledged that the learned score correction need not exactly preserve the marginal distributions. Although the authors presented empirical diagnostics, they involve projected features and finite resolutions. How did the authors determined that the observed FID improvement is due to reduced discretization error or a change in the fundamental transport?
-
[Q4] How do the authors view the practical regime in which GaugeFix is preferable to simply using a stronger solver, and is there evidence that the method remains useful once the base discretization is already competitive?
-
[Q5] How were development, validation, and final evaluation separated across the reported experiments? In particular, were the operator bank, schedule, candidate-search procedure, and advancement criteria fixed before inspecting the corresponding evaluation metrics, and how many configurations were considered before the final reported settings were selected?
-