- Rating
- 2(−1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper develops a two-sample test from a time-reversal symmetry of flow-matching probability flux. It extends the zero-flow characterization to distributions with finite first moments and establishes a Wasserstein bound for the resulting Zero-Flow Discrepancy (ZFD). A variational representation motivates learning a vector-valued witness on a training split and evaluating its directional alignment on independent held-out pairs. Witnesses are trained either by regression or by maximizing a regularized signal-to-noise ratio. Swap antisymmetry permits finite-sample sign-flip calibration, and branch/time averaging reduces auxiliary variance. Experiments cover synthetic distributions, HIGGS, natural-image distribution shifts, and real-versus-generated CIFAR-10 images. The reported image results are promising, particularly for ZF-SNR, although important parts of the experimental protocol are not yet reproducible from the supplied package.
Critical strengths
-
A coherent connection between the population characterization and a valid testing procedure. Theorem 3.1 and Proposition 3.1 give a useful characterization and quantitative Wasserstein control without requiring densities. More importantly for the implemented test, sample splitting and score antisymmetry provide finite-sample null calibration without requiring accurate estimation of the entire residual field (Appendix A.9). Reusing precomputed scores for sign flips makes this statistically principled construction computationally attractive.
-
Promising empirical power with a small image witness. The reported gains on ImageNet versus ImageNetV2 and RF-generated CIFAR-10 are substantial. Table 1b also includes several generator families and reports a meaningful exception: MMD-FUSE and MMDagg outperform ZF-SNR on MeanFlow. The variance-reduction studies and backbone ablation provide useful supporting evidence. These results suggest a practical advantage worth investigating, although the backbone experiment does not fully isolate the contribution of architecture, objective, and optimization.
Critical weaknesses
-
The validation protocol and total data budget are unresolved. Appendix C.4, lines 1392–1393, states that a validation set as large as the training set was used to select hyperparameters and regularization. However, inspection of all seven main experiment scripts finds 18 calls to the ZF trainers and no call supplying
val_set; the trainers' validation branches only log diagnostics. This does not establish that tuning never occurred elsewhere, but the supplied code does not reproduce it. Please specify the search space, selection criterion, tuning frequency, and source of validation samples, including whether their cost is included in and matched across methods. This is especially important for CIFAR-10.1: the reported 1,000 training and 1,021 testing images already exhaust its 2,021 images. A further disjoint validation set cannot come from that same collection. Clarifying this is necessary to assess both the sample-efficiency comparison and independence of the final test from tuning. -
The central generated-image experiments lack necessary sample-generation artifacts. The supplied
sampler_genmodel.pyloads four precomputed tensors for ScoreSDE, RF, MeanFlow, and StyleGAN-XL. None is present in the supplieddata/directory, and the README does not explain how to obtain or construct them. The RF Euler experiments similarly expect per-NFE tensors. Appendix C.2 provides upstream repositories, checkpoints, and some sampling information, but this does not reproduce the actual tensors, including their conversion to the required uint8 representation. The provideddata/fid.pyconsumes existing tensors rather than generating them. Please supply the tensors or an executable generation pipeline with exact configurations, seeds, preprocessing/quantization, and artifact identifiers. Until then, Table 1b and the RF experiments cannot be independently regenerated from this package. -
Some implementation and evaluation details require reconciliation before interpreting the reported precision. Section 4.3, Eq. (18), and Algorithm 1 specify i.i.d. uniform time samples, whereas
one_step_snr_lossand the defaultdraw_test_timessample one time from each of equal strata. This still preserves the mean alignment and swap antisymmetry, but the displayed i.i.d. variance equality is not the exact variance of the implementation. Please describe stratification and give the corresponding expression. Separately, the CIFAR-10.1 alternative sampler reuses the entire fixed held-out subset within each training run while resampling . Consequently, these repeats are not fresh draws from both populations as the protocol suggests. A conditional finite-corpus evaluation is legitimate; please state that estimand and clarify what the reported standard errors include. Neither observation by itself establishes invalid Type-I calibration.
Questions for authors
- Scope of consistency. Propositions 4.3 and A.1 explicitly require a fixed witness with positive population alignment. These are useful conditional results, but definiteness of ZFD does not establish that neural training yields such a witness for every alternative. Please preserve this distinction in the interpretation of the method. Failure to reject is not evidence that ZFD or Wasserstein distance is small.
- Moment assumptions. The regression identity in Lemma 3.1/Eq. (9) should explicitly state the second-moment condition used in Appendix A.4. The first-moment population characterization should be distinguished from the integrability assumptions needed for regression and asymptotic testing.
6 comments
Clarification of the Questions
Question 1: Scope of Consistency
but definiteness of ZFD does not establish that neural training yields such a witness for every alternative
We agree with the reviewer’s observation. To address this concern, we have added high-probability bounds for ZF-Reg and ZF-SNR in the revised Appendix E. Under the stated assumptions, both learned witnesses achieve positive alignment with probability at least whenever
{\rm ZFD}(p,q) &> \varepsilon_{\rm app}^{\rm reg} +\varepsilon_{\rm stat}^{\rm reg} +\varepsilon_{\rm opt}^{\rm reg},\\\\ {\rm SNR}_\lambda^* &> \varepsilon_{\rm app}^{\rm snr} +\varepsilon_{\rm stat}^{\rm snr} +\varepsilon_{\rm opt}^{\rm snr}. \end{aligned}$$ Here, ${\rm ZFD}(p,q)$ and ${\rm SNR}_\lambda^*$ denote the respective population signals for ZF-Reg and ZF-SNR. The errors account for approximation, finite-sample estimation, and optimization, respectively. Detailed bounds and proofs are provided in Appendix E.1. As the statistical errors vanish as $n_{\rm tr}\to\infty$, positive alignment holds with probability tending to one when the population signals exceed the approximation and optimization errors by fixed positive margins. Combined with Proposition 4.3, this provides sufficient conditions for conditional testing consistency, under the required non-degeneracy condition. These conditions are **sufficient, not necessary**. As the reviewer noted, failure to reject may reflect insufficient witness alignment rather than closeness between distributions. We have clarified this point in the revision. ## Question 2: Moment assumptions We thank the reviewer for pointing out this distinction. We have explicitly added the finite second-moment assumption $\mathbb E\\|Y-X\\|^2 < \infty$ to Lemma 3.1 in the revised version. **References (Primarily Cited in [Clarifications on Weakness](https://openreview.net/forum?id=E3VOWpZeF6¬eId=C1BjjjwZKH))** [1] Learning Deep Kernels for Non-Parametric Two-Sample Tests. Liu et al. ICML2020.Thanks, I have read the comments.
Clarifications on Weakness: Validation, Reproducibility and Implementation
We sincerely thank the reviewer for the careful assessment of our paper, particularly for the effort devoted to examining our implementation details. We appreciate the constructive comments and address them below.
Weakness 1: Validation Protocol and Total Data Budget
We thank the reviewer for raising this important concern. We acknowledge that the tuning procedure was not clearly described in Appendix C.4, which we have now clarified in the revision.
For HIGGS and ImageNet vs. ImageNetV2, preliminary tuning samples may reappear in subsequent randomized train/test splits. These validation samples were not included in the reported n. All formal evaluations used fixed hyperparameters after tuning.
Since our experiments largely follow [1], we adopted its published baseline hyperparameters without retuning. For the additional ImageNet and real-versus-generated CIFAR-10 experiments, we used the CIFAR-10 vs. CIFAR-10.1 settings from [1]. Thus, tuning budgets were not explicitly matched: ZF2ST was tuned once, whereas the MMD-D and C2ST were not retuned.
Weakness 2: The Lack of Sample-generation Artifacts
We thank the reviewer for pointing out the missing artifacts. We have updated the supplementary materials. Please refer to the last part of
readme.mdfor the tensors used in the experiments. The original PNG files can be reconstructed usingdata/fid.py.The reviewer can reproduce our results using the provided tensors or independently generate images from the corresponding official model repos and convert them using
data/compress.py. We expect the resulting testing power to be close to our reported values.Weakness 3: Implementation and Evaluation Details Requiring Reconciliation
We thank the reviewer for carefully examining our implementation and evaluation protocol. We address both concerns below.
Stratified Time Sampling
Eq. (18) (now Eq. (17) after revision) considers i.i.d. uniform time sampling, whereas our implementation uses stratified sampling for further variance reduction. We partition [0,1/2] into K equal strata I1,…,IK and independently draw tk∼Unif(Ik). Denoting the resulting score by sh,Kstra, in contrast to the i.i.d. score sh,K, we obtain the exact variance decomposition
=\sigma^2_{X,Y}+\frac1K\mathbb E_{X,Y}[\sigma^2_{t\mid X,Y}]-\underbrace{\frac1{K^2}\mathbb E_{X,Y} \left[\sum_{k=1}^K(\mu_k-\bar\mu)^2\right]}_{\text{Between-stratum variance reduction}}\leq \sigma_K^2(\mathbf{h}),$$ where $\mu_k:=\mathbb E_{t\sim\mathrm{Unif}(I_k)}[\bar{\mathfrak{s}}_{\mathbf{h}}(t)\mid X,Y]$ and $\bar\mu:=K^{-1}\sum_{k=1}^K\mu_k$, with the remaining notation defined as in the revised Eq. (17). Thus, the i.i.d. variance expression serves as an upper bound for the variance of the stratified estimator. We have clarified this in a new remark in Appendix B.1. ### CIFAR-10.1 Estimand and Standard Errors > please state that estimand and clarify what the reported standard errors include. Our CIFAR-10 vs. CIFAR-10.1 evaluation follows a protocol similar to [1]. The **estimand** is the rejection probability conditional on the available CIFAR-10.1 corpus, averaged over randomized data splits, witness training, and repeated testing. The reported mean $\pm$ SE is computed across the 10 trial-level rejection rates, reflecting variability across training runs and repeated testing, but not uncertainty from newly collected CIFAR-10.1 samples. We have clarified this in the revised Appendix C.1.