- Rating
- 4(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes the "Zero Flux" criterion/discrepancy that quantifies the "difference" between two (potentially) high-dimensional discrete distributions. The paper's main contribution is an extension of the recent "Zero Flow" Criterion using theory from discrete flow matching resulting in the Zero Flux criterion. The paper then defines a sample-based discrepancy from this criterion and shows that the discrepancy is invariant to independent irrelevant variables. Another theoretical contribution of the paper is a theoretical analysis of the Zero Flux discrepancy with regards to its error and how large the empirical discrepancy can be when the two distributions are the same. The empirical experiments of the paper also corroborates Zero Flux's invariance to independent irrelevant variables and shows that, under certain conditions, Zero Flux discrepancy is able to track distribution shifts better than other approaches. Lastly, the paper ends with a small case study by tracing the partisan separation of the U.S. over time to showcase a practical usecase of the Zero Flux discrepancy.
Critical strengths
The paper nicely links the recent Zero Flow Criterion with Discrete Flow Matching resulting in a discrete analogue of the Zero Flow Criterion (which they call the Zero Flux Criterion) in Theorem 3.4. Also, as far as I can tell, the proofs for the theory needed for the construction of the Zero Flux Criterion and Discrepancy (Propositions 3.2, 3.3, 4.1, 4.2 and Theorem 3.4) don't contain any obvious errors as well, so the theoretical contributions of the paper is motivated and situated well within the existing literature and is sound; thus making these contributions an interesting result for researchers familiar with the prior work.
Furthermore, the some of the experiments in the paper are well motivated as well. In particular, the experiment in Section 7.1 empirically demonstrates the result in Proposition 4.2 that shows that Zero Flux in invariant to independent irrelevant features/coordinates, a property that is vital when dealing with high-dimensional distributions in general. Furthermore, the experiments in Section 7.4 provides a nice casestudy into situations where having a discrepancy with properties like Zero Flux can be useful.
Critical weaknesses
The main weakness of the paper is that the main text lack discussions as to the method's limitations or situations where the method might not perform as well as other methods. For instance, in Table 1, we can see that Zero Flux is outperformed by 3 other methods (in terms of AUROC) for the Markov 8d dataset, but there doesn't seem to be any discussion about that. Furthermore, the only experiment that tests how Zero Flux responds when the order of the shifting dependence relation increases is relegated to Appendix F.1 and in this experiment Zero Flux seems to lose power when the dependence order increases. It would be nice if such a crucial limitation be mentioned somewhere in the main text or even just in the conclusion. Overall, the paper should be more forthcoming about its limitations.
Questions for authors
Section 7.2 compares Zero Flux's ability to detect distribution shifts against other methods, but without directly defining a threshold to use. This is somewhat fine for the experiment since it showcases compares detection ability in the average or best case. But how would someone who would like to use the Zero Flux discrepancy for distribution shift detection set the threshold in practice? I mean eq 20 might be a good start, but that bound contains the terms which involves a KL divergence with the exact posteriors.
Lines 417-418 (and later in line 1619 in the appendix) say that MNIST is binarised to , but how exactly was it binarized? And why not discretizing the MNIST pixel values into more levels than just binary?
Also in the set of methods being compared against in Section 7.2 and 7.3 seem to be different. Most notably is the absence of Zero Flow since the one-hot construction in Section 7.1 seems to be extendable to any alphabet. Is there a reason why Zero Flow was left out in Section 7.3?
In Table 1 and Figure 7 in the Appendix, the AUROC of a couple of methods like Classifier-JS, VAE-ELBO, and VAE-IWAE seem to decrease as the contamination levels increases, especially on the Markov d=128 dataset. This seems somewhat un-intuitive for me since, I would expect that the problem of detecting distributional shift becomes easier as the second distribution contains less and less samples from the "null" distribution . So despite the problem (in principle) becoming easier as increases, the AUROC decreases for these methods. Do you know the reason for this counterintuitive result?
Minor Comments
- line 068: maybe state what is here.
- eq 1: maybe define t here as well instead of just in eq 3
- Line 114: "Since..." did you mean assume here? I might be misunderstanding this line .
- line 264: maybe state where the evaluation midpoints come from.
- Figure 3: seems vertically squished
4 comments
Question 4. Analysis of the counter intuitive AUROC behavior of Classifier-JS and VAE methods
We thank the reviewer for pointing out this interesting observation. We have added training trajectories, retrospective checkpoint comparisons, and a likelihood approximation error analysis in Appendix F.6 (Figures 13–14 and Tables 7–8).
Classifier-JS: Overfitting and early stopping
For Classifier-JS, the training trajectories indicate that continued optimization can lead to in-sample overfitting and deteriorating AUROC, while earlier checkpoints can achieve substantially stronger discrimination. Even with retrospectively AUROC-optimal checkpoints, Classifier-JS achieves AUROCs of 0.7028 and 0.9928 at and , respectively, compared with 1.0000 for Zero Flux at both levels using its standard checkpoint (Table 7).
In our experiments, early stopping is based on the convergence of the respective training objectives, using training cross-entropy for Classifier-JS and ELBO-based likelihood estimates for VAE methods. However, there is no universal early-stopping criterion across different discrepancy estimators, as their estimation objectives and convergence behaviors differ. In particular, AUROC is a downstream evaluation metric that measures discrimination between null and alternative distributions rather than the convergence of the estimated discrepancy, and therefore should not serve as a common early-stopping criterion.
VAE-ELBO and VAE-IWAE: Relative likelihood approximation errors
For VAE-based methods, we train three VAEs: two separately on and , and one on their mixture . The discrepancy is estimated through their ELBO or IWAE likelihood estimates:
where is the empirical discrepancy computed using the exact likelihoods, and , , and are the corresponding likelihood approximation errors.
Our analysis shows that the relative likelihood approximation errors among the three VAEs can dominate the underlying distributional separation, distorting the estimated discrepancy. For example, at , the empirical separation relative to the null is , while the relative approximation-error correction changes by for VAE-ELBO, reversing the ordering of the estimated discrepancies. This explains the observed AUROC degradation, as further supported by the training trajectories and checkpoint comparisons in Appendix F.6.
We thank the reviewer again for motivating this additional analysis.
Due to the detailed response by the authors, especially regarding the addition of the discussion on the dependence-sensitive behavior of Zero Flux and its limitations, as well as a procedure for threshold selection, I will raise my rating from 3 to 4.
Thank you for your time and careful consideration of our responses. We remain available should you have any further questions or concerns.
We Have Added a Limitations Discussion, Permutation-testing for Threshold Selection, and Expanded the Experimental Evaluation
We thank the reviewer for the constructive comments.
Weakness 1. Discussion of limitations
We have added a discussion of the limitations and dependence-sensitive behavior of Zero Flux in Sections 7.2 and 8, including the results of the higher-order interaction experiment in Appendix F.1.
We have expanded the discussion of Table 1 in Section 7.2. On Markov 8d, LRD (0.982), Classifier-JS (0.977), and Zero Flow (0.958) all achieve higher AUROC than Zero Flux (0.874). LRD fits a first-order Markov likelihood that directly captures the change in transition probabilities between the reference and contaminating chains. Classifier-JS and Zero Flow also perform well on these short sequences without specifying a transition model.
On higher dimensional data like Markov 128d, however, Zero Flux achieves the highest AUROC among all methods except LRD (0.920 for Zero Flux and 1.000 for LRD). Zero Flux outperforms both Zero Flow (0.861) and Classifier-JS (0.424), demonstrating its ability to detect high-dimensional distributional differences directly from samples without specifying a Markov transition model or estimating the joint likelihood.
The higher-order interaction experiment in Appendix F.1 characterizes the dependency-sensitive inductive geometry of Zero Flux. As the interaction order increases, the population discrepancy can decrease even when total variation and KL divergence remain fixed. This reflects the intrinsic sensitivity of Zero Flux to different dependency structures, while preserving population identifiability. A practical implication is that, as the population signal becomes smaller at higher interaction orders, its estimation can become more susceptible to posterior estimation errors and optimization noise, as discussed in Section 8.
Question 1. Practical threshold selection
In practice, the detection threshold for Zero Flux can be determined using a permutation test, a standard nonparametric approach under exchangeability [1, 2]. Given two independently sampled datasets from P0 and P1, we first compute the observed Zero Flux discrepancy. We then pool the observations and randomly permute their group labels while preserving the original sample sizes. For each permutation, we reinitialize and fully retrain the posterior networks to obtain a null statistic.
The detection threshold is then selected from the resulting permutation null distribution according to a prespecified significance level α. Specifically, we reject the null hypothesis when the observed discrepancy exceeds the corresponding threshold, or equivalently, when the permutation p-value satisfies p≤α. This allows the threshold to be determined directly from the observed data without requiring knowledge of the exact posterior distributions or their approximation errors.
We have added this threshold selection procedure and its empirical evaluation in Appendix F.2. On MM128, using 100 permutations per trial across 50 independent trials at α=0.05, the selected thresholds successfully detect distributional differences in 98% and 100% of trials at ε=0.05 and ε=0.1, respectively (Table 5).
Question 2. MNIST preprocessing
We clarified the binarization procedure in Appendix E.2. MNIST images primarily consist of digit strokes against a background, making a binary foreground-background representation a natural choice. Moreover, binarized MNIST is an established benchmark in discrete generative modeling, as demonstrated by recent works [3, 4]. We therefore follow this commonly adopted setting, using a threshold of 0.5 after rescaling pixel intensities to [0,1].
Question 3. Additional UCI baseline
Response. We have added Zero Flow to the UCI comparison in Section 7.3 (Table 2). Zero Flux achieves a C-index of 0.994 and a Pearson correlation of 0.953, improving upon Zero Flow in both metrics.
Minor Comments
All minor comments have been addressed in the revised manuscript.
We thank the reviewer again for the helpful suggestions.
References
[1] Jesse Hemerik and Jelle Goeman. Exact testing with random permutations. TEST, 27(4):811–825, 2018.
[2] Markus Ojala and Gemma C. Garriga. Permutation Tests for Studying Classifier Performance. Journal of Machine Learning Research, 11(62):1833–1863, 2010.
[3] Mingue Park, Jisung Hwang, Seungwoo Yoo, Kyeongmin Yeo, and Minhyuk Sung. PairFlow: Closed-Form Source-Target Coupling for Few-Step Generation in Discrete Flow Models. ICLR 2026.
[4] Zijing Ou, Chinmay Pani and Yingzhen Li. Test-Time Alignment of Discrete Diffusion Models with Sequential Monte Carlo. ICLR 2026.