- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper asks whether rectified-flow couplings can be built from the closed-form empirical probability-flow field instead of a pretrained teacher. They cannot: students trained on them perform close to random-coupling students. The authors show the flow returns the initialisation noise plus a small drift (x₁ = z + Δ), whose share scales as log N / d. Lowering the dimension helps, but closed-form couplings remain well behind teacher couplings.
Critical strengths
- The question is relevant, and the negative answer is clean and useful.
- The diagnostics in Section 4 and Appendix B are well designed.
- The decomposition with a noise-independent drift, gives an intuitive explanation of the failure and is well supported.
- Propositions 5.1 and 5.2 are exact and make no distributional assumptions.
Critical weaknesses
- Only one closed-form construction
Couplings are built with a single setting (). The paper does not compare against other couplings: correlated-Gaussian/whitening coupling, minibatch or semi-discrete OT, and smoothed closed-form fields
- Capacity and memorization are not checked
At CIFAR-10 16px, the network has more parameters (58M) than there are pixel values in the training set (38M). Table 7 shows that random students reproduce their training pairs, and FID rewards memorization. If the teachers are close to memorizing in the low-dimensional regime and as the closed-form is a memorizer, this would also partly explain the 99.9% basin agreement.
- The dimension experiment is confounded by network capacity
The network is held fixed as d decreases, so fitting capacity per dimension rises sharply. The experiments does support the claim that the coupling becomes more informative. But the claim that this produces better students is not well supported (for example scaling network width with d)
Questions for authors
-
How does the simple closed-form compare with other (teacher-free) coupling? (See W1)
-
How sensitive are the results to ? Prop. 5.1 suggests z survives for any , but the drift may change. On , values below 1 break the Gaussian marginal. Have you considered any marginal-preserving way of strengthening the drift?
-
Why did you build teacher couplings by inversion (data -> noise) rather than standard reflow sampling (noise -> data)? Do the conclusions hold for the standard construction?
4 comments
Sensitivity to $\epsilon$: The drift does not change with \epsilon, as expected from our analysis in Figure 8. Using $\lambda = 1$, every start $x_\epsilon$ lies on the line $(1 − t) x_0 + t z$, which the closed-form flow follows closely until the collapse time $\tau$. For the same , the endpoints at $\epsilon = 10^{-2}, 10^{-1}$ and 0.3 agree with those at $\epsilon = 10^{-3}$ to a relative difference of $2 \times 10^{-5}$, and the students are equal up to seed variation (see Table R3, general response). Hence, increasing $\epsilon$ does not strengthen the drift. It only matters once it approaches the collapse time $\tau$, where the trajectory stops tracking its conditional path.
Marginal-preserving drift strengthening: The marginal-preserving option we tested is the smoothed closed-form field of Scarvelis et al. (see “Comparison to other teacher-free couplings”). It increases the drift share and improves the one-step FID by 6, but stronger smoothing breaks the Gaussian marginal, as $\lambda < 1$ does. Within the closed-form family, the drift can therefore be strengthened only slightly before the endpoint marginal is lost.
Coupling direction: We use data -> noise couplings to keep the data marginal ($x_0$) identical across coupling sets, such that the performance differences reflect only how the endpoints ($x_1$) are assigned. With noise -> data couplings, the FM teacher would generate new images, while the closed-form flow could assign multiple noise samples to the same training image, resulting in a different data marginal for each coupling.
However, we also tested noise -> data couplings. For the closed-form flow, these cover only 63.1% of the training set, with the remaining pairs repeating images. Nevertheless, the student performs similarly to the data -> noise construction, reaching a one-step FID of 68.8 (vs. 69.2). Noise -> data couplings from the FM teacher improve performance (one-step 9.0 vs. 14.7), which we attribute to the reduced prior mismatch, as the endpoints are then exact Gaussian samples. The gap between closed-form and teacher couplings is therefore unchanged, and our conclusions hold for the standard reflow construction as well.
We hope these additions resolve the reviewer's concerns. If so, we would be grateful if they would consider raising their score, and we are happy to discuss any remaining questions.
Christopher Scarvelis, Haitz Sáez de Ocariz Borde, and Justin Solomon. Closed-form diffusion models. Transactions on Machine Learning Research, 2025.
Thank you for the detailed answer.
The new experiements address my concerns. I encourage you to include these results in the revised version.
I maintain my recommendation for acceptance.
We thank the reviewer for the constructive feedback and suggestions. We will include all new results in the final version. If you feel that our revision has addressed your concerns and strengthened the case for acceptance, we would be grateful if you would consider reflecting this in your score.
We thank the reviewer for their thoughtful feedback and are glad they found the question relevant and the diagnostics and theoretical analysis well designed. Below, we address the comparison to other teacher-free couplings, memorisation, network capacity, and the reviewer’s questions. Results shared across reviewers are collected in the General Response.
Memorisation: We agree that teacher memorisation would be an important confound, and we now tested for it in two ways. First, we compared 5,000 samples from fresh noise with their nearest training images and computed the ratio between the distance to the nearest and to the second-nearest training image. On CIFAR-10 16px, where we would expect memorisation the most, this ratio is 0.98 for the teacher, similar to the value for held-out images. Second, a memorising teacher would have to spread the small perturbations around each training image over its entire basin, making it as noise-sensitive as the closed-form flow. We repeated the noise-sensitivity analysis of Table 1 on all datasets and resolutions. At every resolution, teacher endpoints remain almost noise-independent (noise share at most 8%), whereas the closed-form endpoints are dominated by the injected noise (at least 97.7%). Hence, the teachers do not memorise, and the 99.9% basin agreement reflects consistency with the closed-form partition rather than memorisation. For the students, fitting the training pairs is expected, as these define the vector field the model is supposed to learn. Beyond the training pairs, the nearest-neighbour ratio is 0.97 for random and closed-form students and 0.98 for FM students, and closed-form and FM students recover 33–61% and ≥92% of the variance of held-out images from unseen endpoints (Table 7). Therefore, the students do not reproduce training images either.
Network capacity: To ensure a fair comparison, we followed the reviewer’s suggestion and reduced the width of the network at resolutions 24 and 16, so that each resolution now has 18.4k parameters per data dimension, as in the 32px base model. The smaller networks leave the FM students essentially unaffected, but make the random and closed-form students worse in absolute FID, most strongly the random student at 16px, which relies on fitting its training pairs (Appendix H). Nevertheless, the gap score G changes only slightly compared to Table 4 and still doubles from resolution 32 to 16. Therefore, lowering the dimension improves the closed-form students also at equal capacity per dimension.
Table R8: Students at equal capacity per dimension (CIFAR-10)
Comparison to other teacher-free couplings: We trained students on three further teacher-free couplings on CIFAR-10 at 32px, with the same noise and training seed as our closed-form student and kept the number of couplings at 50k for fair comparison.
Optimal transport: We pair randomly sampled noise points with the data using exact OT (squared Euclidean cost). It lowers the transport cost by only 5% compared to random pairing, and its endpoints carry as little information about the image as the closed-form ones ($\cos(x_0, x_1) = 0.062$ for both), so the OT student performs on par with the closed-form student.
Whitening: Mapping each image to its whitened principal coordinates gives fully informative couplings (linear reconstruction $R^2 = 1$) but breaks the Gaussian marginal: $\frac{\|x_1\|^2}{d}$ varies 35 times more than under the standard normal. The student does not generalise to fresh noise and only produces the Gaussian fit of the data (FID 218).
Smoothed closed-form field: Following Scarvelis et al., we average the posterior mean over input perturbations of scale $\sigma$, which keeps the endpoint decomposition of Prop. 5.1 valid. Endpoints stay approximately Gaussian up to $\sigma = 1$, where the drift share increases. The resulting student improves on the closed-form student by 6 FID points at one step, but remains far from the FM student.
In summary, no teacher-free coupling can close the gap. The marginal-preserving couplings (OT, smoothing) stay close to the closed-form student, while the informative whitening coupling breaks the marginal.
Table R9: Teacher-free couplings (CIFAR-10, 32px)