- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper proposes Quantile AlignTree (QAT), a coupling for Flow Matching that splits the data with a KD-tree and the Gaussian at matching quantiles, so each training image is paired with its own region of the noise; the tree is built once offline and also supports conditional generation. The paper proves that the resulting paths do not cross and are more separated than under the independent coupling, and argues that this reduces velocity ambiguity. Experiments cover unconditional CIFAR-10, class-conditional ImageNet-32 and ImageNet-256 (latent), and text-conditional CelebA-64.
Critical strengths
- A cheap coupling over the whole dataset. The tree is built once, offline, in O(Nd log N) time, and the noise for a given image is drawn in O(d) time, so drawing a training pair costs about as much as with I-FM. Unlike mini-batch OT, the pairing is defined over the whole dataset rather than within a batch, and the construction scales to more than 2M samples on ImageNet and to VAE latents on ImageNet-256.
- Exact Gaussian prior, also per class. The quantile-aligned splits keep the noise distribution exactly N(0, I) by construction. For class-conditional generation, each class gets its own tree, so the noise paired with each class is also exactly N(0, I), which avoids the train-test gap of condition-agnostic OT (Cheng & Schwing, 2025) without solving any OT problem.
- Broad evaluation with gains over I-FM. The method is tested on unconditional, class-conditional, text-conditional, and latent-space generation, changing only the coupling. On all four benchmarks, QAT-FM improves over I-FM both with the fewest steps and with the most accurate solver (e.g., Euler-2 on CIFAR-10: 98.68 vs. 164.18; ImageNet-256 with 250 steps: 1.99 vs. 2.08).
Critical weaknesses
-
The theory does not support the paper's central claim. The paper claims that non-crossing paths (Theorem 1) and larger path separation (Theorem 2) alleviate local velocity ambiguity and make the velocity field easier to fit. Both theorems are correct, but they do not establish this:
-
Theorem 2 is a statement about transport cost. For a given coupling, let be the path separation for two independent pairs and from this coupling, and the transport cost. Since , The first two terms do not depend on the coupling: are two independent noise samples, and are two independent data samples. In the last term, , and the products across pairs vanish because the pairs are independent and the noise has zero mean (e.g. ), so this term equals . Hence both quantities depend on the coupling only through : where denotes the first two terms. QAT and the independent coupling have the same marginals, so and the norm terms are the same for both, and under the independent coupling. Hence So in Theorem 2 equals , and the gain is , i.e. the theorem only says that QAT has a lower transport cost than the independent coupling. Any coupling with a lower cost has larger separation, and full OT maximizes it.
This also contradicts Table 1, which marks mini-batch OT and RecFlow as not improving separation, although both lower the transport cost (for RecFlow, see Liu et al., 2023).
-
Path separation does not measure ambiguity. The paper says that ambiguity "arises locally", where paths cross or come close with different velocities, whereas path separation averages over all pairs of paths, most of which are far apart. So a coupling can remove all ambiguity and still have lower separation than the independent one. For example, let the data also be a 2D standard Gaussian and set , with a rotation by . Then with and , so determines : paths never cross and the velocity is unambiguous. Yet , while it is for the independent coupling, so by the identity above the separation is lower.
-
Non-crossing removes ambiguity only by making the target discontinuous. With one sample per leaf, the coupling is deterministic, so the ideal flow is exactly straight and one Euler step would suffice. But its velocity is piecewise, with one piece per training image and jumps at every split, so a network can only approximate it. The paper does not analyze whether this target is easier to learn than the smooth, averaged velocity of I-FM. In practice, the trained model is far from the ideal flow: the Euler-2 FID on CIFAR-10 is 98.68 (Table 2).
-
-
The experiments only partially show the geometric benefit that motivates the method. Non-crossing and better-separated paths should make the ODE easier to integrate, i.e. require fewer function evaluations (NFE) and give better samples with few steps. The evidence for this is mixed:
- NFE is not lower. With Dopri5, QAT-FM uses the most NFE on CIFAR-10 (145.6 vs. 137.3 for I-FM, Table 2), slightly more on ImageNet-32 (143.4 vs. 140.3, Table 4), and only slightly fewer on CelebA (144.1 vs. 150.0, Fig. 4).
- The few-step gain is inconsistent. QAT-FM improves over I-FM at Euler-2 (e.g., 105.16 vs. 122.40 on ImageNet-32) and at 10 steps on ImageNet-256 (10.67 vs. 11.83, Table 5). However, on ImageNet-32 the gain shrinks to at most 0.16 FID at Euler-5 to Euler-20 and reverses at Euler-10 (Euler-5/10/20: 22.88/10.30/6.40 vs. 23.04/10.12/6.48); a gain of about 0.25 FID appears again only at Euler-50+ and Dopri5.
- On CIFAR-10 and ImageNet-256, the gains with many steps are small. On CIFAR-10, QAT-FM is better than I-FM with Dopri5 (3.26 vs. 3.38) but worse at Euler-100 (3.91 vs. 3.79). On ImageNet-256, the gain is about 0.1 FID at 50 and 250 steps (Table 5, no std reported).
Overall, the results show a modest quality improvement at low coupling cost, but do not show that it comes from better path geometry; the paper reports neither transport cost nor path straightness.
-
The conditional extension is justified for classes, but not for continuous conditions such as text.
- Classes work. With w = infinity, each class gets its own tree that covers the whole Gaussian, so the noise paired with each class is N(0, I), the same as at inference.
- Continuous conditions break this. In CQAT, a split on a condition coordinate leaves the noise region unchanged, while a split on an image coordinate also splits the noise region in two. With finite w, as used for text, image splits occur before the captions are fully separated, and nodes with fewer than 128 samples are split only on image coordinates; the paper chooses this rule to increase the number of noise splits and thus the gain of Theorem 2 (Section 3.4). As a result, during training the noise paired with a given caption covers only a small part of the Gaussian, while at inference it is drawn from the full N(0, I). Cheng & Schwing (2025) identify the same train-test gap as the cause of the poor results of condition-agnostic OT, also visible here on ImageNet-32 (FID 6.65 for OT-FM vs. 3.82 for I-FM, Table 4). The paper cites this work but does not discuss the gap for CQAT.
- The theoretical claims do not cover this case. Table 1 states that Gaussian alignment and path separation are "also preserved in conditional generation". For finite w, Gaussian alignment holds only for the noise averaged over all conditions, not for a fixed condition. Proposition 2 averages path separation over independent pairs of samples, which mostly have different conditions; what matters is ambiguity for a fixed condition, since the network receives the condition as input.
- For text, the ambiguity that CQAT targets is already absent. In real text-image data, almost every caption belongs to a single image. Then all paths with a given caption go from different noise samples to the same image and meet only at that image, so the target velocity is unambiguous even under the independent coupling. A gain from CQAT would therefore have to come from how the network shares information across similar captions, which the paper does not analyze.
- The CelebA experiment does not test this setting. CelebA-Dialog captions are generated from templates over five facial attributes with six levels each (Jiang et al., 2021), i.e. at most 15552 attribute combinations for about 200K images, so the task is close to class-conditional generation. In addition, caption alignment is measured only by a CLIP score reported to two decimals (0.10 for both QAT-FM and I-FM), which is too coarse to detect small changes.
Questions for authors
-
Mini-batch OT with multiple GPUs. Appendix C lists the batch size per GPU (128 for ImageNet-32, 64 for CelebA) and uses 4 GPUs, i.e. an effective batch of 512 and 256. Is the OT-FM coupling computed on each GPU's local batch or on the effective batch? For C2OT, the paper states that OT is solved per GPU, but for OT-FM this is not specified. This matters because a smaller OT batch gives a coupling closer to the independent one.
-
Coupling cost of C2OT. Appendix C states that the oversampling in C2OT "substantially increases the coupling cost" and refers to Appendix D.1, but the runtime comparison there (Fig. 6) does not include C2OT. Since low coupling cost is the main practical advantage of QAT over C2OT, could the authors report the total training time of I-FM, C2OT, and QAT-FM in their setup, including the tree construction for QAT?
-
Memorization. Each training image is paired with a fixed region of the noise, so copying may be easier to learn than with I-FM, and FID against training statistics does not penalize it. Could the authors compare nearest-neighbor distances to the training set for QAT-FM and I-FM, and report FID on held-out data (e.g., the CIFAR-10 test set)?
7 comments
Response to Reviewer A12M (Part 2 / 3)
-
On W3.1 & W3.2: Train–Inference Consistency for Continuous Conditions (Part 2/2)
To further validate this strategy, we compare the FID and CLIP scores obtained using the standard Gaussian prior and the conditional prior .
-
CelebA Caption-Conditional Generation (Euler-100):
Prior FID (Train) FID (Test) CLIP (Train) CLIP (Test) 3.17 4.28 0.104 0.104 3.14 4.26 0.105 0.105 -
ImageNet-256 Caption-Conditional Generation (MeanFlow, NFE = 1):
Prior FID (Train) FID (Test) CLIP (Train) CLIP (Test) 24.13 22.40 0.313 0.313 23.65 21.85 0.314 0.314
These results show that the conditional prior further improves generation quality and text–image alignment on both training and held-out data. Additional results are provided in Figure 4 (P9, Lines 466-474) and Section D.3 (P25, Lines 1329-1403).
-
-
On W3.3: Theoretical Claims
-
Gaussian Alignment. We would like to clarify that CQAT constructs a valid coupling between the continuous Gaussian and the joint data-condition distribution , i.e., . Thus, our claim of Gaussian Alignment refers to global marginal consistency, rather than conditional Gaussian consistency for each fixed condition.
-
Proposition 2. We would like to emphasize that Proposition 2 extends Theorem 2 to conditional generation, aiming to quantify the global Path Separation Gain of CQAT. More importantly, it provides an explicit node-wise expression showing that further non-degenerate splits along data coordinates increase the global Path Separation Gain, thereby providing theoretical guidance for constructing deeper trees in conditional generation.
-
-
On W3.4: The Condition–Image Pairs
We would like to clarify that one-to-one text-image pairs in the training set do not imply a one-to-one mapping in conditional generation. In fact, Flow Matching learns to transport a noise distribution to the conditional image distribution, rather than learning a deterministic mapping from text to a single image. A well-trained conditional generative model should generate diverse images consistent with the same caption using different initial noise samples. Therefore, one-to-one text–Image pairs in a finite training set neither imply a deterministic underlying conditional distribution nor invalidate the motivation for optimizing conditional couplings.
We also provide CelebA generation examples in Figure 12 (P30). For each fixed caption, our model generates diverse images consistent with the condition from different initial noise samples, even though each caption corresponds to only one image in the training set.
-
On W3.5: The CelebA Settings
We would like to clarify that, in the actual caption-generation code of CelebA-Dialog, the same attribute combination can correspond to multiple different captions. Specifically, among approximately 200K samples, we identify 181,956 unique captions, substantially exceeding 15,552. Moreover, we use continuous text embeddings as conditioning inputs, making it appropriate to treat this task as continuous-condition generation.
Regarding the CLIP scores, we have updated the reported values to three decimal places in the revised manuscript (P9, Lines 466-474).
-
On Q1: Mini-batch OT with Multiple GPUs
We thank the reviewer for pointing this out. We clarify that OT-FM computes the coupling independently on each GPU's local batch, with batch sizes of 128 for ImageNet-32 and 64 for CelebA-64, rather than using the aggregated batch across four GPUs. We have clarified this implementation detail in Appendix C.
-
On Q2: C2OT Runtime Comparison
We additionally report the training throughput and end-to-end runtime of I-FM, OT-FM, C2OT, and QAT-FM, including offline tree construction (with PCA) for QAT-FM.
- ImageNet-32:
Method Throughput (img/s) Offline Build (h) E2E (h) I-FM 1733.3 — 24.745 OT-FM 1727.0 Online 24.802 C2OT 1696.3 Online 25.251 QAT-FM 1730.7 0.068 24.850 - CelebA-64:
Method Throughput (img/s) Offline Build (h) E2E (h) I-FM 624.0 — 11.525 OT-FM 620.3 Online 11.592 C2OT 590.2 Online 12.189 QAT-FM 621.0 0.014 11.593 These results show that QAT-FM achieves higher training throughput and lower end-to-end runtime than C2OT, with minimal offline construction overhead.
Response to Reviewer A12M (Part 3 / 3)
-
On Q3: Memorization
We additionally evaluate the 1-NN MSE between generated samples and the training set, as well as FID against the held-out CIFAR-10 test set. The results are reported below.
-
1-NN MSE:
Method Euler-2 Euler-100 Dopri5 Test Set 0.0292 0.0292 0.0292 I-FM 0.0034 0.0297 0.0314 OT-FM 0.0093 0.0302 0.0314 SD-FM 0.0035 0.0285 0.0300 QAT-FM 0.0107 0.0294 0.0306 -
CIFAR-10 Test-Set FID:
Method Euler-2 Euler-100 Dopri5 I-FM 167.97±0.66 7.83±0.07 7.37±0.07 OT-FM 81.71±0.44 7.99±0.07 7.49±0.05 SD-FM 166.37±0.77 8.17±0.11 7.69±0.08 QAT-FM 103.06±0.53 7.94±0.08 7.28±0.05
QAT-FM achieves the highest 1-NN MSE among generative methods at Euler-2. Notably, at Euler-100 and Dopri5, its 1-NN MSE is even higher than that of real test samples. Moreover, it maintains competitive test-set FID, achieving the best result under Dopri5. These results suggest no clear evidence of excessive training-set memorization. Full experimental details are provided in Appendix D.3 (P25, Lines 1329-1347).
-
In summary, we hope the above response resolves your concerns and helps you re-evaluate our work. Feel free to contact us if you have any other questions.
Reference
[1] The Curse of Conditions: Analyzing and Improving Optimal Transport for Conditional Flow-Based Generation. ICCV 2025.
Response to Reviewer A12M (Part 1 / 3)
Thank you for your thoughtful feedback. Below, we provide responses to your comments in the following section and we have modified the manuscript to address these issues. (Changes are highlighted in blue in the updated PDF.)
On W1.1: Theorem 2 and Transport Cost
First, we would like to clarify that Theorem 2 is not intended to show that QAT outperforms OT-based methods. More importantly, it provides an explicit node-wise expression for the path-separation gain, showing that further non-degenerate splits along data coordinates continuously increase this gain. This provides theoretical guidance for constructing deeper QAT trees in practice.
We acknowledge that exact Semi-Discrete OT between the full data distribution and the standard Gaussian provides an upper bound on path separation gain. We thank the reviewer for pointing this out and have added this observation to the revised manuscript (P16, Lines 825-831). However, solving the full OT problem is computationally challenging. As shown in Figure 6, Full OT incurs substantial coupling construction and sampling costs, limiting its practical applicability.
Regarding Table 1, we thank the reviewer for pointing out this issue and have corrected it in the revised manuscript.
On W1.2: Path Separation and Velocity Ambiguity
We thank the reviewer for providing this counterexample. We would like to clarify that we do not regard Path Separation as a strict measure of Velocity Ambiguity, but rather as a way to improve path geometry.
Specifically, larger Path Separation means that intermediate states Xt are more spatially dispersed, potentially reducing the likelihood that paths with different target velocities overlap or come too close in local regions, thereby alleviating Velocity Ambiguity.
We acknowledge that there are special cases where Velocity Ambiguity is absent despite smaller Path Separation, as illustrated by the reviewer's counterexample. However, this does not invalidate our motivation to alleviate local Velocity Ambiguity by increasing Path Separation.
On W1.3: Learnability of the Velocity Field
In general, it is difficult to directly determine whether the velocity field induced by a given coupling is easier for a neural network to learn. Nevertheless, QAT-FM achieves a substantially better Euler-2 FID than I-FM (98.68 vs. 164.18), suggesting that its piecewise structure does not prevent effective learning in practice.
On W2.1: NFE and FID Results
Dopri5 uses adaptive step sizes, so NFE depends on the learned velocity field and solver tolerances. Although QAT-FM requires slightly more NFE, it achieves the best FID. Moreover, under the same Euler-100 sampling budget, QAT-FM outperforms OT-FM and SD-FM on CIFAR-10, and all baselines on ImageNet-32. These results support the effectiveness of our method.
On W2.2: Euler-10 Performance on ImageNet-32 On ImageNet-32, although we observe a slight fluctuation at Euler-10, QAT-FM outperforms I-FM under all other settings (Euler-2/5/20/50/100 and Dopri5), providing strong empirical evidence of its effectiveness.
On W2.3: The FID Gains
On CIFAR-10, many coupling-based methods yield limited improvements. For instance, OT-FM achieves an FID of 3.49, while QAT-FM improves it to 3.26, demonstrating the effectiveness of our method.
For ImageNet-256, we repeated the 250-step evaluation across three random seeds, obtaining FIDs of
2.088 ± 0.009for I-FM and2.005 ± 0.023for QAT-FM. These results demonstrate that our improvement is consistent across different seeds.On W3.1 & W3.2: Train–Inference Consistency for Continuous Conditions (Part 1/2)
We thank the reviewer for highlighting this issue. First, train–inference mismatch is not unique to CQAT but is a general concern in conditional OT-based methods (e.g., C2OT [1]). We further provide a rigorous theoretical analysis showing that, for augmented OT with a finite condition weight, conditional Gaussian consistency holds if and only if the source and target conditions match almost surely (cs=ct, π⋆-a.s.). This highlights a general challenge in continuous conditional coupling.
An advantage of CQAT is that it can explicitly characterize and sample from the conditional source distribution. Specifically, CQAT represents this distribution as γc=(projx0)♯πCQAT(⋅∣C=c). Since γc is a mixture of truncated Gaussians, we can directly sample from it at inference for conditions observed during training, ensuring consistency with the training source distribution.