acceptodds
Under review as a conference paper at ICLR 2027

The Limits of Closed-Form Couplings for Few-Step Generation

Abstract

Few-step generation remains an open challenge for flow models. Rectified flows address it by distilling a pretrained teacher into a few-step student using the noise–data couplings induced by the teacher. Obtaining the pretrained model and sampling the couplings is expensive. However, for a finite dataset and a Gaussian probability path, the probability-flow vector field of the empirical distribution is available in closed form, so one could use this field to construct the couplings directly and skip pretraining altogether. Surprisingly, this fails, even though the pretrained coupling is almost always compatible with the data-point assignment induced by the closed-form flow. Students trained on closed-form couplings perform much closer to those trained on random noise–data pairings than to those trained on teacher-generated couplings. We trace this failure to an exact endpoint identity for the closed-form flow. The integration returns the noise vector drawn at initialisation, plus a drift that is empirically a faint copy of the data point, almost independent of the noise. The coupling therefore differs from an independent pairing only through this drift. By bounding the drift and combining the bound with the known collapse time of empirical diffusion dynamics, we show that the informative share of the coupling grows with the dataset size and shrinks with the data dimension , scaling as . Growing the dataset therefore helps only logarithmically, while lowering the dimension helps linearly. We confirm this empirically by decreasing the dimension of the data itself, downsampling images to a range of resolutions. The drift share grows as , and students trained on closed-form couplings recover an increasing fraction of the gap between random and teacher couplings. However, even at the lowest resolutions the recovery remains incomplete, suggesting that closed-form couplings become competitive teachers only at values of far beyond those seen in natural image data.

Primary area:
generative models
Keywords:
rectified flows, flow matching, closed-form couplings, few-step generation, generative modelling

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_QWNDOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks whether rectified-flow couplings can be built from the closed-form empirical probability-flow field instead of a pretrained teacher. They cannot: students trained on them perform close to random-coupling students. The authors show the flow returns the initialisation noise plus a small drift (x₁ = z + Δ), whose share scales as log N / d. Lowering the dimension helps, but closed-form couplings remain well behind teacher couplings.

Critical strengths
  • The question is relevant, and the negative answer is clean and useful.
  • The diagnostics in Section 4 and Appendix B are well designed.
  • The decomposition x1=z+Δx_1 = z + \Delta with a noise-independent drift, gives an intuitive explanation of the failure and is well supported.
  • Propositions 5.1 and 5.2 are exact and make no distributional assumptions.
Critical weaknesses
  1. Only one closed-form construction

Couplings are built with a single setting (ϵ=10−3,λ=1\epsilon = 10^{-3}, \lambda = 1). The paper does not compare against other couplings: correlated-Gaussian/whitening coupling, minibatch or semi-discrete OT, and smoothed closed-form fields

  1. Capacity and memorization are not checked

At CIFAR-10 16px, the network has more parameters (58M) than there are pixel values in the training set (38M). Table 7 shows that random students reproduce their training pairs, and FID rewards memorization. If the teachers are close to memorizing in the low-dimensional regime and as the closed-form is a memorizer, this would also partly explain the 99.9% basin agreement.

  1. The dimension experiment is confounded by network capacity

The network is held fixed as d decreases, so fitting capacity per dimension rises sharply. The experiments does support the claim that the coupling becomes more informative. But the claim that this produces better students is not well supported (for example scaling network width with d)

Questions for authors
  1. How does the simple closed-form compare with other (teacher-free) coupling? (See W1)

  2. How sensitive are the results to ϵ\epsilon ? Prop. 5.1 suggests z survives for any ϵ\epsilon, but the drift may change. On λ\lambda, values below 1 break the Gaussian marginal. Have you considered any marginal-preserving way of strengthening the drift?

  3. Why did you build teacher couplings by inversion (data -> noise) rather than standard reflow sampling (noise -> data)? Do the conclusions hold for the standard construction?

4 comments

We thank the reviewer for their thoughtful feedback and are glad they found the question relevant and the diagnostics and theoretical analysis well designed. Below, we address the comparison to other teacher-free couplings, memorisation, network capacity, and the reviewer’s questions. Results shared across reviewers are collected in the General Response.

Memorisation: We agree that teacher memorisation would be an important confound, and we now tested for it in two ways. First, we compared 5,000 samples from fresh noise with their nearest training images and computed the ratio between the distance to the nearest and to the second-nearest training image. On CIFAR-10 16px, where we would expect memorisation the most, this ratio is 0.98 for the teacher, similar to the value for held-out images. Second, a memorising teacher would have to spread the small perturbations around each training image over its entire basin, making it as noise-sensitive as the closed-form flow. We repeated the noise-sensitivity analysis of Table 1 on all datasets and resolutions. At every resolution, teacher endpoints remain almost noise-independent (noise share at most 8%), whereas the closed-form endpoints are dominated by the injected noise (at least 97.7%). Hence, the teachers do not memorise, and the 99.9% basin agreement reflects consistency with the closed-form partition rather than memorisation. For the students, fitting the training pairs is expected, as these define the vector field the model is supposed to learn. Beyond the training pairs, the nearest-neighbour ratio is 0.97 for random and closed-form students and 0.98 for FM students, and closed-form and FM students recover 33–61% and ≥92% of the variance of held-out images from unseen endpoints (Table 7). Therefore, the students do not reproduce training images either.

Network capacity: To ensure a fair comparison, we followed the reviewer’s suggestion and reduced the width of the network at resolutions 24 and 16, so that each resolution now has 18.4k parameters per data dimension, as in the 32px base model. The smaller networks leave the FM students essentially unaffected, but make the random and closed-form students worse in absolute FID, most strongly the random student at 16px, which relies on fitting its training pairs (Appendix H). Nevertheless, the gap score G changes only slightly compared to Table 4 and still doubles from resolution 32 to 16. Therefore, lowering the dimension improves the closed-form students also at equal capacity per dimension.
Table R8: Students at equal capacity per dimension (CIFAR-10)

pxwidthparametersparameters per dimensionrandomFID ($\pi^{PF})$FID ($\pi^{FM})$G
3212856.4M18.4k85.169.214.70.23
249631.7M18.4k84.858.59.10.35
166414.1M18.4k93.952.14.70.47

Comparison to other teacher-free couplings: We trained students on three further teacher-free couplings on CIFAR-10 at 32px, with the same noise and training seed as our closed-form student and kept the number of couplings at 50k for fair comparison.

Optimal transport: We pair randomly sampled noise points with the data using exact OT (squared Euclidean cost). It lowers the transport cost by only 5% compared to random pairing, and its endpoints carry as little information about the image as the closed-form ones ($\cos(x_0, x_1) = 0.062$ for both), so the OT student performs on par with the closed-form student.

Whitening: Mapping each image to its whitened principal coordinates gives fully informative couplings (linear reconstruction $R^2 = 1$) but breaks the Gaussian marginal: $\frac{\|x_1\|^2}{d}$ varies 35 times more than under the standard normal. The student does not generalise to fresh noise and only produces the Gaussian fit of the data (FID 218).

Smoothed closed-form field: Following Scarvelis et al., we average the posterior mean over input perturbations of scale $\sigma$, which keeps the endpoint decomposition of Prop. 5.1 valid. Endpoints stay approximately Gaussian up to $\sigma = 1$, where the drift share increases. The resulting student improves on the closed-form student by 6 FID points at one step, but remains far from the FM student.

In summary, no teacher-free coupling can close the gap. The marginal-preserving couplings (OT, smoothing) stay close to the closed-form student, while the informative whitening coupling breaks the marginal.

Table R9: Teacher-free couplings (CIFAR-10, 32px)

coupling$\cos(x_0, x_1)$FID 1 stepFID 2 steps
random0.00085.165.5
closed-form0.06269.252.5
exact OT, 50k × 50k0.06267.451.9
smoothed closed-form field, $\sigma = 1$0.08062.847.8
whitening0.330217.7219.2
FM teacher0.23014.712.1

Sensitivity to $\epsilon$: The drift does not change with \epsilon, as expected from our analysis in Figure 8. Using $\lambda = 1$, every start $x_\epsilon$ lies on the line $(1 − t) x_0 + t z$, which the closed-form flow follows closely until the collapse time $\tau$. For the same zz, the endpoints at $\epsilon = 10^{-2}, 10^{-1}$ and 0.3 agree with those at $\epsilon = 10^{-3}$ to a relative difference of $2 \times 10^{-5}$, and the students are equal up to seed variation (see Table R3, general response). Hence, increasing $\epsilon$ does not strengthen the drift. It only matters once it approaches the collapse time $\tau$, where the trajectory stops tracking its conditional path.

Marginal-preserving drift strengthening: The marginal-preserving option we tested is the smoothed closed-form field of Scarvelis et al. (see “Comparison to other teacher-free couplings”). It increases the drift share and improves the one-step FID by 6, but stronger smoothing breaks the Gaussian marginal, as $\lambda < 1$ does. Within the closed-form family, the drift can therefore be strengthened only slightly before the endpoint marginal is lost.

Coupling direction: We use data -> noise couplings to keep the data marginal ($x_0$) identical across coupling sets, such that the performance differences reflect only how the endpoints ($x_1$) are assigned. With noise -> data couplings, the FM teacher would generate new images, while the closed-form flow could assign multiple noise samples to the same training image, resulting in a different data marginal for each coupling.
However, we also tested noise -> data couplings. For the closed-form flow, these cover only 63.1% of the training set, with the remaining pairs repeating images. Nevertheless, the student performs similarly to the data -> noise construction, reaching a one-step FID of 68.8 (vs. 69.2). Noise -> data couplings from the FM teacher improve performance (one-step 9.0 vs. 14.7), which we attribute to the reduced prior mismatch, as the endpoints are then exact Gaussian samples. The gap between closed-form and teacher couplings is therefore unchanged, and our conclusions hold for the standard reflow construction as well.

We hope these additions resolve the reviewer's concerns. If so, we would be grateful if they would consider raising their score, and we are happy to discuss any remaining questions.

Christopher Scarvelis, Haitz Sáez de Ocariz Borde, and Justin Solomon. Closed-form diffusion models. Transactions on Machine Learning Research, 2025.

Thank you for the detailed answer.

The new experiements address my concerns. I encourage you to include these results in the revised version.

I maintain my recommendation for acceptance.

We thank the reviewer for the constructive feedback and suggestions. We will include all new results in the final version. If you feel that our revision has addressed your concerns and strengthened the case for acceptance, we would be grateful if you would consider reflecting this in your score.

Reviewer_GqyoOpenReview ↗
Rating
2
Confidence
3(−1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies why noise–data pairs from the exact empirical probability-flow field lead to worse few-step generation than pairs from a learned teacher. It writes the noise endpoint as the initial noise draw plus a drift term, then combines a bound on this drift with known collapse-time scaling to predict an O(log⁡N/d)O(\log N / d) scaling for normalized drift energy. Experiments on CIFAR-10, CelebA and ImageNet compare generation and reconstruction across resolutions.

Critical strengths
  • Connecting the drift bound to known collapse-time scaling gives an O(log⁡N/d)O(\log N / d) prediction for the drift’s share of endpoint energy. This is a useful quantitative step beyond the observation of diminishing gains in PairFlow [1], making the roles of dataset size and dimension explicit.

  • The experiments show a large advantage for learned-teacher pairings on three natural-image datasets, with the same student architecture and number of training pairs in each comparison. The closed-form field is evaluated exactly over the full training sets, and repeated seeds and held-out reconstructions (Table 7) show the difference beyond FID.

[1] Park et al., “PairFlow: Closed-Form Source-Target Coupling for Few-Step Generation in Discrete Flow Models”, ICLR 2026.

Critical weaknesses
  • The main contribution is the explanation. PairFlow [1] already constructs continuous empirical-flow couplings in its appendix, evaluates them on MNIST, and reports diminishing gains with increasing dimension in synthetic experiments. The present paper adds broader natural-image comparisons and a theoretical account of noise retention. My concern is how far that explanation goes, especially in accounting for the learned teacher’s advantage.

  • The learned teacher’s advantage remains largely unexplained. The drift analysis explains why the empirical flow retains noise, but says much less about why a learned approximation to this field produces much better couplings. The question left open in Section 7 is, in my view, the most important and interesting one raised by this comparison. I see a connection to a fundamental question in machine learning: how neural networks learn smooth functions from finite samples and generalize between and beyond those samples. Here, the teacher may capture an underlying continuous data manifold, so that its velocity reflects this manifold structure rather than just the individual training samples. This could help explain why its couplings are stable, a possibility that also relates to work on diffusion generalization [2, 3]. As Section 5.2 points out, the log⁡N/d\log N / d parameter behind the drift scaling also controls when the empirical flow starts to memorize, and Biroli et al. [4] note that trained models avoid this through a score that is smoother than the exact one. Concrete progress on this connection, or a way to obtain similarly useful couplings without training a teacher, would be a substantial advance. I do not think the current analysis goes far enough in answering this central question to justify acceptance.

  • I do not see a substantive error in Propositions 5.1–5.2 under the assumptions in Appendix C, but the drift-energy scaling alone does not explain why the coupling is difficult to learn. Eq. (13) measures drift energy relative to the total noise energy in the ambient space. Adding independent noise coordinates orthogonal to the data decreases this fraction without changing the best achievable reconstruction from the informative coordinates. The image reconstruction results in Tables 6–7 support the observed trend, but the analysis does not explain how the geometry of the drift affects student performance.

[2] Song et al., “Selective Underfitting in Diffusion Models”, arXiv 2025.

[3] Kamb and Ganguli, “An Analytic Theory of Creativity in Convolutional Diffusion Models”, ICML 2025.

[4] Biroli et al., “Dynamical Regimes of Diffusion Models”, Nature Communications 2024.

Questions for authors
  • Beyond the endpoint measurements in Table 1, is there a property of the learned field that the authors think explains its stable, useful couplings? For instance, could the authors apply the endpoint decomposition and the tracking-error analysis of Figure 7 to the teacher’s trajectories? This would show at which noise levels the teacher departs from the empirical flow.

  • The drift scaling raises a natural question about the data needed for few-step generation: would increasing NN to keep log⁡N/d\log N / d fixed as dd grows also preserve student performance? Could measuring student performance along the existing NN sweep in Figure 8, while keeping the coupled training images fixed, provide an initial test of this connection?

  • All experiments use natural images. Under the same Gaussian probability path, would the authors expect a similar performance gap between closed-form and learned-teacher couplings on other types of data? Which properties of the data might change that comparison?

5 comments

We thank the reviewer for the detailed feedback. We appreciate that they see the scaling as a useful quantitative step, and we agree that the teacher’s advantage is an interesting question raised by our comparison. We address it below by applying our endpoint decomposition to the teacher (see also the General Response), and answer the reviewer’s remaining questions. The new Appendix G in the revised manuscript covers the link between drift geometry and learnability.

Drift scaling and learnability: We agree that the drift-energy share is not a direct measure of learnability, but it is closely connected to it, as we show in the new Appendix G. There we use linear readability, the fraction $R^2$ of image variance recovered by the best linear predictor of $x_0$ from $x_1$, as a proxy for learnability, and show that it is exactly

$$R^2 = \frac{|\mathrm{Cov}(x_0, \Delta(x_\epsilon))|_F^2}{V}$$

The geometry of the drift therefore enters through its covariance with the image, i.e. how well it aligns with the image and its high-variance directions. For closed-form couplings, this alignment is strong, since the drift is a faint copy of its data point. This identity also resolves the reviewer’s counterexample. Added coordinates that carry no image variance change neither the covariance nor $V$, so $R^2$ is unchanged while the drift-energy share decreases. In turn, the drift energy bounds how much signal is available. Summed over the principal directions of the data, the readable fractions of the components add up to $\eta \mathbb E|\Delta(x_\epsilon)|^2$, where $\eta$ is the fraction of drift energy linearly explained by the image. For isotropic data, this gives $R^2 = \eta \frac{\mathbb E|\Delta(x_\epsilon)|^2}{\mathbb E|x_1|^2}$, so the drift-energy share directly sets readability. For images, readability additionally depends on how much image variance the readable directions carry. Table 8 shows that strong readability is concentrated in a few leading directions, which hold a larger share of image variance at lower resolution. Within each dataset, the drift-energy share therefore tracks the improvement in linear readout and student reconstruction (Tables 6–7), which is how it should be seen: as a bound on the available signal and as a proxy within a dataset, not as a standalone measure of learnability.

Student performance with varying N: We ran this test on CIFAR-10 at 16 px resolution. To create the different coupling sets, we always integrate the same 10k data points (with the same initial noise), but use N = 10k, 20k, 50k, 60k (train + test) as the reference set for computing the closed-form vector field. Then, we train with the same setup on the corresponding 10k closed-form couplings for each N. As shown in the table below, the drift share grows with N almost exactly as the predicted ratio $\frac{\log N}{\log 10\text{k}}$. The linear readout improves only slightly, and the student FID does not change beyond the seed variation we observed. Hence, increasing N does not measurably improve the student. As predicted by Equation 13, keeping $\frac{\log N}{d}$ fixed while d grows requires N to grow exponentially in d. Going from 16 px to 32 px (d from 768 to 3072) at fixed ratio would require $N = (10^4)^4 = 10^{16}$ images, far beyond any dataset.

Table R7: Closed-form couplings with growing reference set N (CIFAR-10, 16px).

Ndrift shareratio to N = 10kpredicted $\frac{\log N}{\log 10\text{k}}$readout $R^2$FID 1 / 2 steps
10k1.74%1.001.000.4632.6 / 25.0
20k1.88%1.081.080.4832.5 / 25.6
50k2.06%1.181.170.5032.1 / 25.4
60k2.10%1.211.190.5032.3 / 26.0

Other data types: The endpoint decomposition does apply beyond natural images, but we would not necessarily expect the same performance gap for every data distribution. In Appendix G we show one relevant property. Reconstruction depends on whether the endpoint preserves information in directions which carry substantial data variance. A small drift can therefore support good reconstruction if its signal is concentrated in the important directions. Furthermore, the scaling argument depends on the spacing of the data points. It assumes that trajectories initially track their originating point and uses concentration of pairwise distances to estimate when this tracking ends. Strong clustering, a low effective dimension, or points that are only slightly separated relative to the initial noise can change this behaviour. We therefore expect the mechanism to extend to other data satisfying these conditions, while the size of the performance gap depends on their geometry and the learned teacher.

Why are FM couplings good teachers: We agree that the teacher’s advantage is an interesting question raised by our comparison, and our new results make concrete progress on it. PairFlow observes diminishing gains with dimension on synthetic data. We explain why this happens, confirm it on natural images, and now also quantify how the teacher departs from this behaviour.

Applying the endpoint decomposition to the teacher shows that its drift does not add a small correction to the injected noise. It removes most of the noise and replaces it with a component determined by the image, so that the endpoint becomes an almost deterministic and easily readable function of the data (Tables R1 and R2). This is consistent with the energy bounding the available signal (see answer to “Drift scaling and learnability” and o4PX). The small closed-form drift limits readability, whereas the teacher’s larger drift is not just a copy of the image (Table R1) but cancels the noise so that the endpoint is strongly readable. As a result, a change in the initial noise moves the teacher’s endpoint 4–10 times less than the closed-form endpoint, which passes the noise on unchanged. Following the reviewer’s suggestion, we also tracked the drift along the trajectories, i.e. the integral in Eq. (6) up to time t, as a fraction of the full drift, to see where the teacher departs from the empirical flow (CIFAR-32, Table R5). The teacher builds half of its drift below $t=0.04$, where the empirical score collapses onto individual training points, and a trained network does not. Swapping the two fields shows that this early regime decides the coupling (Table R6, metrics as in Table 1). Using the teacher only below $t = 0.1$ already yields a nearly deterministic coupling, while using the closed-form field only below $t = 0.03$ makes 65% of the endpoint variance noise. This suggests that the smoother score, described by Biroli et al., is what makes the teacher’s couplings informative. Reproducing this without a teacher remains difficult. Smoothing the closed-form field improves the student by 6 FID points, but stronger smoothing breaks the Gaussian marginal (see our response to Reviewer QWND).

Why the network learns this smoother field and how neural networks generalise is an interesting question, but it is a major open problem in machine learning. We therefore respectfully suggest that resolving it should not be a requirement for this paper.

Table R5: Drift accumulated along the trajectory (CIFAR-10, 32px)

t0.0040.0170.0360.0720.310.621
$‖\Delta(x_t)‖$000000.0100%
$‖\Delta_{\theta}(x_t)‖$10%35%47%60%84%93%100%

Table R6: Swapping teacher and closed-form fields along the trajectory (CIFAR-10, 32px)

fieldnoise shareNN match
teacher7%100%
teacher for $t < 0.1$, closed-form above10%99%
closed-form for $t < 0.03$, teacher above65%100%
closed-form for $t < 0.3$, teacher above94%37%
closed-form99.7%0.1%

We hope these results address the reviewer's main concern about the teacher's advantage. If so, we would be grateful if they would consider updating their assessment, and we are happy to discuss any remaining questions.

We thank the reviewer again for the constructive feedback and suggestions. We hope the additional results and analysis address your concerns. Specifically,

  • teacher analysis explaining the benefit of FM couplings (Tables R1, R2, R5, R6)
  • new Appendix G linking drift energy to learnability
  • student performance with increasing N (Table R7).

We are happy to address any additional concerns.

Reviewer_o4PXOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper tries to explain why using the exact closed-form PF coupling does not yield good pairings for training a few-step diffusion student in their experiments, while a pretrained teacher model gives much better couplings. To show that, the paper first checked the consistency of the pairings by running the inverse of a pretrained teacher on images and checking whether the exact PF flow could return the original images from the resulting noise endpoints.

Given this analysis, the paper continues by checking the characteristics of the noise. They prove that the PF coupling gives you noise equal to z (the initial random noise added to the images) plus a correction term. They further show empirically that this correction term has very little energy (0.14%–0.52% of the endpoint energy), but this term is where the image-dependent information is contained. This helps explain why the PF coupling still gives mostly random noise and leads to results similar to those of random pairing.

Furthermore, the paper also argues that the relative energy of the correction scales as log N / d, where N is the dataset size and d is the dimension, and shows that reducing the dimension gives better results.

Critical strengths

The exact coupling is appealing because such pairs can be constructed directly from a dataset without pretraining a teacher. If this worked well, it could provide a much cheaper way to obtain training pairs for few-step generation. This makes it an interesting direction to investigate, even though the results here are mostly negative. However, explaining why it does not work sometimes is much more valuable.

I also find the analysis reasonable and well organized. The paper goes step by step, first checking the consistency between the teacher and PF couplings, then examining what the PF endpoints contain and why the image-dependent correction remains small.

Critical weaknesses

My main concern is the connection between drift energy and learnability. In my view, a correction with small energy may still contain a lot of structured information that a model can extract. For example, Table 6 shows that a linear readout can recover information from PF endpoints. This may explain why PF coupling is still better than random coupling, but the connection is difficult to quantify and compare across settings. Can residual energy really serve as a measure for comparing different couplings? Please also see my question below.

Also, the dimension experiments support the proposed explanation, but downsampling changes image content and learning difficulty as well as dimension. Could another controlled experiment separate these effects?

One related work that should also be discussed is

Berthelot et al., “The Coupling Within: Flow Matching via Distilled Normalizing Flows” (2026) (https://arxiv.org/abs/2603.09014),

which uses couplings from a pretrained normalizing flow to train an FM student. Their formulation uses a similar procedure for constructing the noise endpoints: adding noise to an image and then passing it through the NF to obtain the final noise representation. A comparison would help clarify how the observations here relate to that work, particularly regarding what makes a learned coupling effective.

Questions for authors
  1. The initial noise is necessary to select a path because the exact empirical PF is singular at t = 0. Does the initial noise level, controlled by the initialization time ε, affect the resulting coupling or student performance?

  2. Can the same analysis be applied to text-to-image generation, where each condition may have only a very limited number of associated images?

  3. Can the same residual-energy analysis be applied to pretrained teacher couplings, beyond the noise-sensitivity comparison in Table 1? In particular, how does their residual energy vary with N and d, and can this comparison help explain why teacher couplings lead to better students?

3 comments

We thank the reviewer for their constructive feedback and are glad they found the question worth investigating and the analysis well organised. Below, we address the connection between drift energy and learnability, the downsampling confounder, and the reviewer’s questions. Results shared across reviewers are collected in the General Response. The revised manuscript includes a new Appendix G on drift energy and learnability.

The connection between drift energy and learnability: We agree that a small correction can contain substantial structured information (Table 6), so the drift-energy share is not a direct measure of learnability. It is, however, closely connected to it through the question of how much of this structure remains accessible once the correction is mixed with noise. In the new Appendix G, we quantify this accessibility using “linear readability”, the fraction R2R^2 of image variance recovered by the best linear predictor of x0x_0 from x1x_1, and show that it is exactly

R^2 \= \frac{\|\mathrm{Cov}(x_0, \Delta(x_\epsilon))\|_F^2}{V}

Accessibility therefore depends on how the drift co-varies with the image, in particular along its high-variance directions, and not on its energy alone. This is why the closed-form drift remains readable along the leading image directions, despite its small energy (Table 8). In turn, the drift energy bounds the available signal and, for isotropic data, sets readability exactly: R^2 \= \eta \frac{\mathbb E\|\Delta(x_\epsilon)\|^2}{\mathbb E\|x_1\|^2}, where η\eta is the fraction of drift energy linearly explained by the image. For images, where it matters which directions are readable, we therefore use the drift-energy share only as a proxy within each dataset. There, it tracks improvements in linear readout and student reconstruction (Tables 6–7).

Downsampling introduces a confounder: We agree that downsampling changes the image content in addition to the dimension. Our evaluation, however, already accounts for changes in difficulty, since the gap score G compares the closed-form student with the random and teacher students trained on the same data, so changes in overall learning difficulty would affect all three. Separating dimension and content completely is difficult. Changing d while leaving the data itself unchanged is only possible in a trivial way, by padding with constant coordinates. These carry no image variance, so the coupling and its readability stay unchanged (Appendix G). Alternatively, one could place the data on a low-dimensional manifold embedded in a higher-dimensional space, but this would significantly affect the learning difficulty by adding a manifold-learning task to the generative modelling. To show that our findings do not depend on downsampling, we therefore reduce dimensionality with a learned autoencoder instead. It preserves the image content with high fidelity (rFID ≤ 0.95) but alters the data in a very different way than downsampling. On CIFAR-10, the prediction holds as well: G grows from 0.14 to 0.49 as d decreases from 1024 to 256.

Table R4: Dimension reduction with a learned autoencoder (CIFAR-10).

dAE rFIDrandomclosed-formFMG
2560.9569.343.817.70.49
5120.5571.149.420.80.43
10240.3771.264.824.10.14

Related Work: We thank the reviewer for pointing us to that work and have included it in our related work section for the revised version. While the construction is similar, our propositions 5.1 and 5.2 show that the endpoint is almost exactly the injected noise, whereas there is no such identity for the Normalising Flow. While the paper reports relatively high noise sensitivity of the latents, it is still a learned nonlinear map whose latent still encodes the image. In our understanding, the NF coupling therefore sits between our FM teacher and the closed-form coupling. Hence, the noisy initialisation is not the problem in itself. What matters is whether the endpoint carries the information of the data in an accessible form, which is in line with our conclusion that the benefit of a learned teacher lies in how it selects endpoints within each region.

Sensitivity to $\epsilon$: We have run additional experiments with $\epsilon \in \{10^{-2}, 10^{-1}, 0.3\}$ and found that the student performance does not change (Table R3, general response). The drift is also nearly invariant to $\epsilon$, so the endpoints $x_1$ are very similar across values of $\epsilon$. This follows from our analysis in Figure 8. For most of the trajectory, the nonlinear part of the ODE, the posterior mean, is concentrated around the data point, hence the drift is close to 0 on these intervals. $\epsilon$ can only have a significant effect once set to a time where trajectories start overlapping. For the FM coupling, however, this behaviour is different. As it builds its drift very early in the trajectory (see response to reviewer Gqyo), the noise share increases and performance shrinks. When choosing $\epsilon = 10^{-1}$, we find that the 1-step FID worsens to 19.0.

Closed-form for Text-to-Image generation: We understand this question as applying the closed-form construction separately for each condition c, using only the $N_c$ images associated with it. Our analysis can be directly applied to this setting, and it predicts that closed-form couplings become even less informative. The drift share scales as $\frac{\log N}{d}$, so a small number of images per condition shrinks the data-dependent part of the coupling further. In the extreme case of a single image per condition, which is common for text captions, the posterior mean always equals that image and the drift vanishes. Therefore, the flow returns exactly the injected noise (Proposition C.1). The closed-form coupling is then a random coupling. A pretrained text-to-image teacher does not suffer from this, since it can share and exploit structure across conditions, whereas the per-condition closed-form field cannot.

Residual Energy analysis applied to pretrained teacher: Proposition 5.1 only requires writing the field as $v = (x − m_t(x))/t$, which any learned field admits with $m_{\theta,t}(x) = x − t·v_{\theta}(x)$. Teacher endpoints therefore decompose as $x_1 = z + \Delta_{\theta}(x_{\epsilon})$, and we can compare the two drifts directly. The two drifts behave in opposite ways. The closed-form drift carries at most 2.7% of the endpoint energy, decaying as 1/d. The teacher’s drift carries 1.6–2.1 times the endpoint energy at every resolution (Table R2), because it points against the injected noise ($\cos \approx −0.65$, Table R1). Rather than adding a small correction, it removes most of the noise and replaces it with a component determined by the image. The same holds across N. For teachers trained on 5k, 20k, and 50k CIFAR-10 images (16 px), the drift share remains around 190%, and the endpoint remains nearly noise-independent (noise share 2–5%), whereas the closed-form drift increases only from 1.6% to 2.1% of the endpoint energy. The teacher’s drift is therefore not governed by the $\frac{\log N}{d}$ scaling of the closed-form flow. What depends on N is whether the endpoint encodes the image. Its alignment with the data point is 0.24–0.34 for 20k and 50k images, but only 0.10–0.11 for 5k. This is consistent with more data leading to a smoother field, which maps similar images to similar endpoints and thereby aligns endpoints and data (see also our answer to Reviewer Gqyo). Together with the alignment with the data (Table 5), the noise sensitivity (Table 1) and the readable directions (Table 8), this noise cancellation shows that, with enough data, the image information is revealed rather than buried. This explains why teacher couplings make better students. The teacher pairs each image with an almost deterministic, easily readable endpoint, so the student learns a smooth noise-to-data map. The closed-form flow instead pairs each image with its initialisation noise plus a faint copy of itself, which is close to an independent pairing.

We hope these additions resolve the reviewer's concerns. If so, we would be grateful if they would consider raising their score, and we are happy to discuss any remaining questions.

We thank the reviewer again for the constructive feedback and suggestions. We hope the additional results and analysis address your concerns. Specifically,

  • new Appendix G linking drift energy to learnability
  • a dimension sweep with a learned autoencoder (Table R4)
  • the drift analysis of FM couplings (Tables R1, R2)
  • ϵ\epsilon sensitivity (Table R3).

We are happy to address any additional concerns.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response

We thank all reviewers for their constructive feedback. Two questions came up in several reviews, so we answer them here: how pretrained teacher couplings behave under our endpoint decomposition (o4PX, Gqyo), and how sensitive the closed-form coupling is to the initialisation time $\epsilon$ (o4PX, QWND).

All new experiments follow the setup of the paper (Appendix E) with the linear schedule $\alpha_t = 1 - t$, $\sigma_t = t$. For a fair comparison, all new CIFAR-10 32px experiments use the same seed as a single reference run of the base model in Table 4.

Table R1: Properties of the teacher drift.

CIFAR-32CelebA-64ImageNet-64
$\cos(z, \Delta_\theta(x_\epsilon))$−0.67−0.63−0.67
$\cos(x_0, \Delta_\theta(x_\epsilon))$0.170.090.15
$\Vert\Delta_\theta(\alpha_\epsilon x_0)\Vert / \sqrt{d}$0.940.980.96
$\cos(\Delta_\theta(x_\epsilon), \Delta_\theta(x_\epsilon'))$0.540.590.53

Table R2: Drift share $\mathbb{E}\Vert\Delta(x_\epsilon)\Vert^2 / \mathbb{E}\Vert x_1\Vert^2$ of FM-teacher and closed-form couplings.

datasetpxdteacherclosed-form
CIFAR-1016768189%2.1%
CIFAR-10241,728196%0.93%
CIFAR-10323,072175%0.52%
CelebA16768205%2.2%
CelebA241,728207%0.99%
CelebA323,072189%0.57%
CelebA6412,288157%0.14%
ImageNet16768189%2.5%
ImageNet323,072182%0.69%
ImageNet6412,288180%0.18%

Table R3: Sensitivity of the closed-form coupling to $\epsilon$ (CIFAR-10, 32px).

$\epsilon$drift sharenoise shareFID 1 / 2 steps
$10^{-3}$0.52%99.6%69.2 / 52.5
$10^{-2}$0.52%99.6%68.2 / 52.4
$10^{-1}$0.52%99.6%69.1 / 52.6
0.30.52%99.6%68.0 / 51.9
AuthorsOpenReview ↗

Final Summary

We thank all three reviewers and the Area Chair for the time and effort they invested in evaluating our work. We are glad that the reviewers found the question relevant, the negative answer clean and useful, the theoretical analysis exact and well organised, and the log⁡Nd\frac{\log N}{d} scaling a useful quantitative step.

This study asks why couplings built from the closed-form probability-flow field, the vector field a Flow Matching with enough capacity should converge to, fail as teachers for few-step generation. We show that the closed-form endpoint is exactly the injected initialisation noise plus a small data-dependent drift, so that the information about the data point is buried in this noise. We bound the drift and show that its share of the endpoint energy scales as log⁡Nd\frac{\log N}{d}. Closed-form couplings should therefore become better teachers in lower dimensions, as we empirically confirm by downsampling images across a range of resolutions.

During the rebuttal, we addressed the reviewers’ concerns through additional experiments and clarifications, with the main updates summarised below.

  • Drift share and learnability. In the new Appendix G, we show how the drift share connects to learnability. It bounds how much of the data can be linearly recovered from the endpoint, which justifies its use as a proxy. We revised the abstract and main text to make this explicit.
  • Robustness of the dimension result. We replace downsampling with a learned autoencoder and show that our scaling prediction holds. Additionally, we increase the number of data points and show that student performance does not increase, which is also consistent with the exponential data requirement we derive.
  • Comparison to alternative teacher-free couplings. We test three additional teacher-free couplings: exact OT, smoothed closed-form fields and whitening. None closes the gap to teacher couplings. Those that preserve the Gaussian marginal remain about as uninformative as the closed-form coupling, while the informative whitening coupling breaks the marginal.
  • Additional insight into teacher couplings. Beyond the scope of the original submission, we applied our endpoint decomposition to the teacher. Its drift cancels most of the injected noise, making the endpoint a nearly deterministic function of the image. The teacher builds most of this drift early in the trajectory, where the empirical score collapses onto individual training points.

We appreciate the reviewers’ constructive feedback and are encouraged that two reviewers recommend acceptance, with Reviewer QWND confirming that the new experiments address their concerns. The remaining disagreement with Reviewer Gqyo concerns significance, which they described as a judgement call, and we thank them for their engagement.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.