acceptodds
Under review as a conference paper at ICLR 2027

Predictable Wasserstein Critics for Two-Sample Testing

Abstract

A Wasserstein critic learns distributional differences that respect a chosen metric. Reusing past observations to train successive critics preserves out-of-sample evaluation, but leaves a random variance scale that can invalidate Gaussian studentization. We resolve this difficulty for a finite Lipschitz dictionary by letting entropy regularization vanish more slowly than training noise. The learned critic then has vanishing null variance, while retaining the dictionary's population discrepancy under alternatives. Adding independent random signs with positive amplitude gives a studentized Gaussian test under a -moment null. The same fresh-pair scores also admit a closed-form half-normal e-process, valid at arbitrary stopping times for any predictable critic without moment assumptions. Approximate empirical Kantorovich-Rubinstein optimization recovers full or restricted population drift under first moments and fixed-alternative consistency under second moments, without requiring the fitted functions to converge. Simulations show earlier detection than online Newton-step betting in the studied alternatives, and a stratified penguin analysis illustrates learned projection critics.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Martingale, Gaussianization, Two-sample inference, Wasserstein distance

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_m5A2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies sequential two-sample testing with a critic learned only from past observations and evaluated on a fresh pair (Xi,Yi)(X_i, Y_i). Each score is the critic's mean difference plus two zero-mean perturbations: a Rademacher-signed pair distance and an independent Rademacher jitter. Under the null, the scores are conditionally symmetric martingale differences.

The paper has three main contributions:

  1. Variance stabilization for a regularized dictionary. Adaptive critic learning can leave a random predictable-variance scale, so studentization alone does not guarantee a Gaussian limit (counterexample in C.4). For a fixed finite Lipschitz dictionary with entropic (softmax) regularization and temperature ϑi=ri−1/4\vartheta_i = r_i^{-1/4}, the critic's null variance vanishes while its drift converges to the largest dictionary contrast. This yields a studentized Gaussian test under a (2+δ)(2+\delta)-moment null, including point-mass nulls (Prop. 3.2).

  2. Anytime-valid inference. A closed-form half-normal mixture e-process on the same scores gives anytime-valid inference without moment assumptions (Thm. 3.3).

  3. Drift convergence for general critics. Exact or approximate empirical Kantorovich–Rubinstein (KR) critics, and restricted critic classes, have population drift converging to W1W_1 or to the restricted discrepancy (Thms. 3.4–3.5).

The validation includes:

  • an 8-D dictionary simulation, covering variance concentration and a comparison of half-normal vs. online Newton step (ONS) betting on identical scores;

  • a temperature and perturbation sensitivity study;

  • a 1-D exact-KR comparison against permutation W1W_1 and energy tests;

  • a stratified analysis of Palmer penguin morphology.

Critical strengths
  • It identifies and formalizes a genuine calibration pitfall for learned critics. Past-only learning preserves null centering but can make the variance scale random. The counterexample in C.4 shows that conditional symmetry plus observed studentization cannot replace a deterministic variance-scale condition. This matters because learned-witness and martingale two-sample tests are increasingly common, and the clarification applies beyond Wasserstein critics.

  • It gives a concrete, rigorously analyzed construction that satisfies this condition. For the regularized dictionary, bounds (11)–(12) make the trade-off transparent: the temperature suppresses the null critic variance at the training-noise scale, while costing at most ϑlog⁡(2K)\vartheta \log(2K) in drift. Point-mass nulls are handled through external jitter, and measurability issues are treated carefully (e.g., Lemma D.2). I found no errors in the proofs I checked (Prop. 3.1, Prop. 3.2, Thms. C.1, C.2, 3.3). This technical care is the main reason the paper is not a clear rejection.

Critical weaknesses
  1. The main Gaussian guarantee covers a much narrower object than the title and abstract suggest, and its power cost is not quantified.

    • Narrow scope. Gaussian calibration is established only for a fixed finite dictionary, which in the experiments is xj,∣xj∣\\{x_j, |x_j|\\}. Its target, max⁡j∣(P−Q)hj∣\max_j |(P-Q)h_j|, is neither W1W_1 nor consistent against general alternatives.

    • Unverified condition for Wasserstein critics. For actual Wasserstein critics (full KR, restricted, or neural), condition (20) is assumed rather than verified. The 1-D study reports a predictable-variance coefficient of variation of about 0.35 at n=1000n = 1000, so users who want a Wasserstein critic in practice obtain only the e-process.

    • Likely loss of local power. Heuristically, for a contrast Δ≪ϑi\Delta \ll \vartheta_i the critic is approximately (Δ/ϑi)h(\Delta/\vartheta_i)h, so the drift is about Δ2ri1/4\Delta^2 r_i^{1/4}. Summed over n pairs against n\sqrt{n} fluctuations, this suggests a detection boundary near n−3/8n^{-3/8} rather than n−1/2n^{-1/2}. Theorem C.5 could address this question but is not applied to the proposed test, so the price paid for calibration remains unclear.

  2. The novelty relative to prior work is not clearly delineated, and the empirical evidence is too limited to establish practical value.

    • Overlap with prior work. The past-witness/fresh-pair design with studentized Gaussian calibration already appears in the martingale kernel test (Chatterjee & Ramdas, 2026). Perturbation of degenerate studentized statistics follows Corradi et al. (2024). The half-normal mixture of exp⁡(λSn−λ2Vn/2)\exp(\lambda S_n - \lambda^2 V_n/2) under conditional symmetry is a classical self-normalized construction (de la Peña et al., 2004).

    • Limited experimental scope. All experiments are low-dimensional: d≤8d \le 8, d=1d = 1 for KR, and a 4-D application with n=34n = 34 per group. There are no standard ML benchmarks and no comparisons to the witness test, cross-MMD, deep-kernel MMD, C2ST, or the martingale kernel test. Neural critics are discussed but never run.

    • Weak comparative results. Where comparisons exist, the method often loses. Fixed-KR is below permutation W1W_1 under location, mixture, and contamination alternatives at small n. In the penguin application it gives p≈0.020p \approx 0.020, versus 0.002 for both permutation baselines. In addition, the half-normal rule's advantage over ONS coincides with a higher null crossing rate (e.g., 0.030 vs. 0.016 under t5).

Questions for authors
  • (Weakness 1) Beyond the finite dictionary, under what verifiable conditions does the learned KR critic satisfy condition (20), and what local-power cost does the proposed regularization impose?

  • (Weakness 2) What is the key technical advance over existing martingale two-sample tests, and what evidence supports a practical advantage over established learned-witness baselines?

7 comments
AuthorsOpenReview ↗

Response to Reviewer m5A2 (1/5)

Thank you for your time and valuable comments.

W1. We distinguish three inferential targets. The regularized dictionary has Gaussian calibration and fixed-alternative consistency for DH=max⁡j∣(P−Q)hj∣>0D_{\mathcal H}=\max_j|(P-Q)h_j| > 0. Approximate full/restricted KR optimization recovers the corresponding population drift. Conditional symmetry supports anytime-valid monitoring for arbitrary predictable critics. For example, P=(δ−1+δ1)/2P=(\delta_{-1}+\delta_1)/2 and Q=δ−2/4+δ0/2+δ2/4Q=\delta_{-2}/4+\delta_0/2+\delta_2/4 have identical xx and ∣x∣|x| means but W1=1W_1=1, explicitly identifying the finite dictionary's restricted target.

The local-power result is as follows. With bounded update gaps and ϑi=cri−β\vartheta_i=c r_i^{-\beta}, 0<β<1/20 < \beta < 1/2, the learner's separation rate is n−1/4−β/2n^{-1/4-\beta/2}. For independent row-wise pairs, fixed anchored one-Lipschitz features, a fixed finite zero-critic initialization, fixed τ≥0\tau\ge0, ρ>0\rho > 0, uniform local-array (2+δ)(2+\delta) distance moments and κn=EPn⊗Qnd(X,Y)2→κ0\kappa_n=E_{P_n\otimes Q_n}d(X,Y)^2\to\kappa_0,

n1/4+β/2Δn→b⟹Tn⇒N(∣b∣2Kc(1+β)σ0,1),σ02=τ2κ0+ρ2.n^{1/4+\beta/2}\boldsymbol\Delta_n\to b\quad\Longrightarrow\quad T_n\Rightarrow N\left(\frac{\\|b\\|^2}{Kc(1+\beta)\sigma_0},1\right),\qquad \sigma_0^2=\tau^2\kappa_0+\rho^2.

The default gives n−3/8n^{-3/8}. Every single deterministic temperature schedule shared across the rows and satisfying ri<ir_i < i, ri→∞r_i\to\infty, ϑi→0\vartheta_i\to0 and riϑi2→∞r_i\vartheta_i^2\to\infty has limiting rejection α\alpha at root-nn feature contrasts under these local-array and fixed-perturbation assumptions. In the additional local-alternative experiment, rejection at shift 3n−3/83n^{-3/8} is 0.331, 0.354, 0.330 for n=250,1000,4000n=250,1000,4000, versus 0.094, 0.090, 0.065 at shift 3n−1/23n^{-1/2}.

For the continuous KR and application projection fits, Gaussian calibration is conditional on predictable-variance stabilization. We therefore describe these measurements as normal-reference rejection frequencies. The exact KR cases in Q1 supply a concrete verification and a concrete obstruction to treating this condition as automatic.

Put pβ=1/4+β/2p_\beta=1/4+\beta/2. The local-power calculation uses the signed-softmax map gj(z)=sinh⁡(zj)/∑kcosh⁡(zk)g_j(z)=\sinh(z_j)/\sum_k\cosh(z_k). Let ξni\xi_{ni} denote the fresh-pair score and Gn,i−1\mathcal G_{n,i-1} the past observations and signs. Put Dn=∣Δn∣2D_n=\\|\boldsymbol\Delta_n\\|_2, B0=sup⁡n∑jVar⁡Pnhj+Var⁡Qnhj<∞B_0=\sup_n\sum_j\\{\operatorname{Var}_{P_n}h_j+\operatorname{Var}_{Q_n}h_j\\}<\infty, and let εni\varepsilon_{ni} denote the training error. Then E∣εni∣2≤B0/ri\mathbb E\\|\varepsilon_{ni}\\|^2\le B_0/r_i, ∣g(z)∣≤min⁡(1,∣z∣)\\|g(z)\\|\le\min(1,\\|z\\|) and ∣g(z)−z/K∣≤4∣z∣2\\|g(z)-z/K\\|\le4\\|z\\|^2. For the conditional score mean mni=Δn⊤g((Δn+εni)/ϑi)m_{ni}=\boldsymbol\Delta_n^\top g((\boldsymbol\Delta_n+\varepsilon_{ni})/\vartheta_i), the triangle inequality gives

\mathbb E\left|\frac{\sum_i m_{ni}}{\sqrt n} -\frac{D_n^2}{K\sqrt n}\sum_i\vartheta_i^{-1}\right| &\le\frac{D_n\sqrt{B_0}}{K\sqrt n}\sum_i\frac{1}{\sqrt{r_i}\vartheta_i}\\\\ &\quad+\frac{4D_n}{\sqrt n}\sum_i\frac{D_n^2+B_0/r_i}{\vartheta_i^2}. \end{aligned}$$ This bound applies directly to the overlapping prefixes. At $D_n=O(n^{-p_\beta})$, its three contributions have orders $n^{\beta/2-1/4}$, $n^{\beta/2-1/4}$ and $n^{3\beta/2-3/4}$, all tending to zero. Since $\sum_i\vartheta_i^{-1}\sim n^{1+\beta}/\\{c(1+\beta)\\}$, the normalized drift tends to $\\|b\\|^2/\\{Kc(1+\beta)\\}$. For the critic coefficients $a_{ni}=g((\boldsymbol\Delta_n+\varepsilon_{ni})/\vartheta_i)$, $$\frac1n\sum_i\mathbb E\\|a_{ni}\\|^2 \le\frac1n\sum_i\min\left\\{1,\frac{D_n^2+B_0/r_i}{\vartheta_i^2}\right\\}=o(1).$$ Thus the average conditional variance of the critic component vanishes, and the independent signs give $n^{-1}\sum_i\mathbb E[(\xi_{ni}-m_{ni})^2\mid\mathcal G_{n,i-1}]\to\sigma_0^2$. The uniform $(2+\delta)$ moment bound gives conditional Lindeberg. For $s=1+\min(\delta,2)/2$, the martingale moment bound for centered squared increments is $O(n^{1-s})$; together with $n^{-1}\sum_i m_{ni}^2\le D_n^2\to0$, this yields $n^{-1}\sum_i\xi_{ni}^2\to\sigma_0^2$. The martingale central limit theorem and the drift limit give the displayed noncentral normal law. Revised Appendix G.1 (Proposition G.1) gives this result, and Appendix I the KR variance examples.
AuthorsOpenReview ↗

Response to Reviewer m5A2 (2/5)

W2. We credit past-witness/fresh-pair Gaussian testing to Chatterjee and Ramdas (2026), perturbation for degenerate studentized statistics to Corradi et al. (2024), and the conditional-symmetry exponential and one-sided normal-mixture constructions to de la Peña et al. (2004) and Howard et al. (2021). The contribution specific to this paper is the entropy-regularized one-Lipschitz learner's simultaneous null-variance and fixed-alternative drift bounds, together with KR population-drift recovery without fitted-function convergence. The new local-power result makes the associated regularization cost explicit.

The added comparisons include Hotelling, a held-out empirical linear witness, linear and RBF martingale MMD, and RBF cross-MMD. The held-out linear comparator uses the training mean difference on the fixed feature vector; it is a simple sample-split witness. The numerical comparisons cover these fixed-feature and RBF procedures. Deep-kernel MMD, C2ST, neural critics and standard ML datasets remain outside the evaluated benchmark set.

In the original one-dimensional n=100n=100 study, the full-KR normal-reference frequency is 0.302 under scale change versus 0.268 for permutation W1W_1 and 0.184 for energy, whereas under location change it is 0.684 versus 0.768 and 0.766. The former rows illustrate a response to scale changes; their Gaussian calibration requires An2/an2→1A_n^2/a_n^2\to1 in probability for a deterministic an→∞a_n\to\infty, where An2A_n^2 is the accumulated predictable variance. For sequential detection, an independent paired rerun (2000 null and 1000 alternative streams) raises location crossing at n=1000n=1000 to 1.000 for half-normal and 0.999 for ONS, from 0.862 and 0.778 with noise; the associated noise-free normal-null rates are 0.0105 and 0.0010. With perturbations, the rerun gives 0.0190/0.0115 under the t5t_5 null and 0.862/0.778 under location change; the submitted batch gives 0.030/0.016 and 0.882/0.790. Half-normal has higher null crossing as well as higher detection, with both null rates below 0.05; the power comparison is at the common nominal threshold rather than equal attained null crossing. The practical evidence is therefore most useful for understanding the variance/detection tradeoff and noise-free monitoring, with the comparator results in Q2 giving the corresponding fixed-horizon cost.

For the original one-dimensional n=100n=100 comparison, the normal-reference rejection frequency of Fixed-KR is 0.642 under the mixture-weight 0.7 alternative, compared with 0.702 for permutation W1W_1 and 0.720 for energy. Under 0.05 contamination, the corresponding values are 0.274, 0.320 and 0.206. Permutation W1W_1 has higher rejection frequency than Fixed-KR in both of these settings, while Fixed-KR exceeds energy under contamination. The Fixed-KR frequencies use a normal reference whose validity depends on predictable-variance stabilization; the permutation columns use randomization calibration. These comparisons are in revised Section 4.3 (Table 4), Appendix B.3 (Table 9) and Appendix H (Tables 11–12).

AuthorsOpenReview ↗

Response to Reviewer m5A2 (3/5)

Q1. A sufficient condition for broader anchored one-Lipschitz critics is convergence in L2(π)L^2(\pi), up to a predictable sign, to a deterministic anchored one-Lipschitz function f⋆f_\star, with π\pi having a finite (2+δ)(2+\delta) distance moment. The common Lipschitz envelope bounds their L2L^2 norms, and

∣Var⁡π(fi)−Var⁡π(f⋆)∣≤2∣fi−εif⋆∣2(∣fi∣2+∣f⋆∣2)\left|\operatorname{Var}_\pi(f_i)-\operatorname{Var}_\pi(f_\star)\right|\le 2\\|f_i-\varepsilon_i f_\star\\|_2(\\|f_i\\|_2+\\|f_\star\\|_2)

gives variance convergence in L1L^1 when the displayed function discrepancy tends to zero in probability. Averaging then yields An2/n→2Var⁡π(f⋆)+τ2κπD+ρ2A_n^2/n\to 2\operatorname{Var}_\pi(f_\star)+\tau^2\kappa_\pi^D+\rho^2 in probability for fixed perturbations. This gives an explicit sufficient condition on the learned critics.

An exact verifiable KR case is a null supported on two points a,ba,b with masses 1−p,p1-p,p, 0<p<10 < p < 1, and separation L=d(a,b)>0L=d(a,b) > 0. For any measurable exact empirical optimizer, deterministic training sizes ri→∞r_i\to\infty, fixed τ≥0\tau\ge0, ρ>0\rho > 0, and the distance stabilizer,

An2n⟶2p(1−p)L2(1+τ2)+ρ2in L1.\frac{A_n^2}{n}\longrightarrow 2p(1-p)L^2(1+\tau^2)+\rho^2\quad\text{in }L^1.

Whenever the empirical proportions differ, exact optimality forces ∣fi(a)−fi(b)∣=L|f_i(a)-f_i(b)|=L. The tie probability is at most π/[4rip(1−p)]\sqrt\pi/[4\sqrt{r_i p(1-p)}], so its averaged contribution vanishes. The bounded scores and positive variance limit give An2/(nσ02)→1A_n^2/(n\sigma_0^2)\to1 with σ02=2p(1−p)L2(1+τ2)+ρ2>0\sigma_0^2=2p(1-p)L^2(1+\tau^2)+\rho^2 > 0, and hence Gaussian calibration.

The three-point law π=(δ0+2δ1+δ2)/4\pi=(\delta_0+2\delta_1+\delta_2)/4 shows the additional issue for richer KR fits. With slopes u,v∈−1,1u,v\in\\{-1,1\\} selected by the empirical cumulative-count signs, Var⁡π(f)=3u2/16+uv/8+3v2/16\operatorname{Var}_\pi(f)=3u^2/16+uv/8+3v^2/16: it equals 1/21/2 for equal signs and 1/41/4 for opposite signs. With bounded update gaps, each sign pattern persists over a substantial final fraction of the stream with probability bounded away from zero. Consequently An2/nA_n^2/n retains separated ranges and fails deterministic variance stabilization. To see the separation, let ZkZ_k be the two cumulative-count differences at k=⌊n/4⌋k=\lfloor n/4\rfloor. Its increment covariance is (3/81/81/83/8)\left(\begin{smallmatrix}3/8&1/8\\\\1/8&3/8\end{smallmatrix}\right). For C=4C=4, both events Zk/n∈[C,C+1]2Z_k/\sqrt n\in[C,C+1]^2 and Zk/n∈[C,C+1]×[−C−1,−C]Z_k/\sqrt n\in[C,C+1]\times[-C-1,-C] have positive limiting probability by the two-dimensional central limit theorem. Independently, the future increments stay within Cn/2C\sqrt n/2 in each coordinate with probability at least 1−3/C21-3/C^2. On the resulting events, the respective signs persist throughout the final three quarters, giving n−1∑iVar⁡π(fi)≥3/8−o(1)n^{-1}\sum_i\operatorname{Var}_\pi(f_i)\ge3/8-o(1) and ≤5/16+o(1)\le5/16+o(1). Since the distance second moment equals one, An2/nA_n^2/n equals twice this average plus τ2+ρ2\tau^2+\rho^2, which has two separated ranges of positive probability. The continuous-KR experiments retain their conditional Gaussian interpretation, while their symmetry-based monitoring remains valid.

For the regularized dictionary, the local-power result gives the boundary and noncentrality displayed in W1. The drift and studentization calculation in W1 handles the overlapping prefixes directly; the result also gives consistency above the test-specific boundary. Revised Appendix I gives the two-point case (Proposition I.1) and the three-point example, and Appendix G.1 the local-power result.

Q2. The technical advance is an explicit one-Lipschitz learning construction with both null-variance control and fixed-alternative drift recovery, plus the separation of Gaussian calibration from conditional-symmetry monitoring. The new local-power theorem identifies the price of that construction.

The fixed-horizon comparison below uses the common input Φ(x)=(x1,…,x8,∣x1∣,…,∣x8∣)\Phi(x)=(x_1,\ldots,x_8,|x_1|,\ldots,|x_8|). HT is Hotelling; LM is linear martingale MMD; HW is the held-out empirical linear witness; RM is the RBF martingale MMD test of Chatterjee and Ramdas (2026), and RC is RBF cross-MMD. HT/LM use within-pair randomization, HW uses independent evaluation data, and the RBF methods use their native normal reference on a richer nonlinear class. The table uses fresh streams, independent of the submitted experiment: 2000 per null and 1000 per alternative.

SettingnnDefaultHTLMHWRMRC
Normal null10000.05350.05200.05550.03950.04400.0520
Standardized t5t_5 null10000.04450.04900.04950.05400.03600.0470
Location shift 0.42500.3660.7840.7500.7330.6870.712
Location shift 0.410000.9701.0000.9991.0000.9991.000
Scale factor 1.32500.7941.0001.0001.0001.0001.000
Scale factor 1.310001.0001.0001.0001.0001.0001.000
Reviewer_FGdwOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes a two-sample testing framework that learns Wasserstein critics from past observations and evaluates them on fresh samples to determine whether two distributions differ.

Critical strengths

None

Critical weaknesses

The primary issue of this work is presentation and the lack of an understandable storyline. I would go one step further: this work is one of the most intricate examples of LLM-written submissions I have ever seen. From a distance, this work does not have the usual tells of LLM writing, but trying to understand even the beginning shows there are many presentation issues that a human would have likely caught.

  • The abstract is almost completely unintelligible. It was not written for humans (and by humans): no linking words, use of uncommon (or nonexistent) idioms.
  • The introduction is not a proper introduction: it mixes contextualization, what seems to be a motivation, contributions and related works. It is extremely difficult to decipher. It feels like a collection of LLM outputs stitched together. There is no red line, no connection between the ideas: just a collection of facts without clear intent.
  • Given it is impossible for me to understand what this work is about after a few hours (beyond the obvious of course), I cannot comment further on its scientific content.

In my opinion, this work needs a complete (human-made) rewrite.

Questions for authors

I am taking the perspective of an average reader reading the paper in order:

  • What is "Transport-specific"? I do not think it is a commonly used idiom. Why not express this more commonly if you meant something standard?
  • What is a "variance-stabilizing critic learner"? It has not been introduced.
  • What is a "fixed Lipschitz dictionary"? If it's a standard concept, why not use a reference at its first mention?
  • What is a "Gaussian null limit under a (2 + delta) moment"? Why express it this way at this point in the introduction?
  • Is "KR optimization" a common idiom? If yes, why is there no reference when it is first mentioned?
  • What is the average reader supposed to make out of "An eight-dimensional learned dictionary links the variance theorem to observable behavior and compares two valid sequential tests on identical scores"? The introduction is mostly composed of this type of statement.
3 comments
AuthorsOpenReview ↗

Response to Reviewer FGdw

Thank you for your time and valuable comments.

W1. We replace the opening of the abstract with:

A Wasserstein critic is a scalar function whose changes are bounded by the distance between observations. Reusing past observations to train successive critics preserves out-of-sample evaluation, but leaves a random variance scale that can invalidate Gaussian studentization. For a fixed finite set of one-Lipschitz functions, called a dictionary, we resolve this difficulty by letting entropy regularization vanish more slowly than training noise. The learned critic then has vanishing null variance, while retaining the dictionary's population discrepancy under fixed alternatives.

The next sentences state the calibration guarantee:

Under a null law with a finite (2+δ)(2+\delta) moment of distance to a fixed point, for some δ>0\delta>0, independent random-sign perturbations yield an asymptotically standard normal studentized statistic. The same fresh-pair scores also admit the classical half-normal mixture e-process, valid at arbitrary stopping times for any predictable critic without moment assumptions.

We replace the introductory explanation with:

Wasserstein–1 distance measures transport cost; its dual learns a scalar critic whose variation is bounded by the metric. This interpretation underlies Wasserstein generative adversarial networks (Arjovsky et al., 2017). Testing additionally requires distinguishing population separation from sampling noise.

We then present the finite-dictionary Gaussian guarantee, population-drift recovery for approximate Kantorovich–Rubinstein optimization, and symmetry-based monitoring as separate results. These changes appear in the revised Abstract and Section 1.

Q1. We replace “Transport-specific methods” with “Methods based on transport discrepancies.” This wording introduces projected and sliced transport methods before explaining the accumulated scalar-score construction. The new wording appears in revised Section 1.

Q2. Entropy regularization makes the learned critic's null variance vanish, while fixed independent-sign perturbations retain a positive deterministic limiting variance. This identifies the role of each component in Gaussian calibration; the population-drift bound explains the behavior under fixed alternatives. Revised Proposition 3.2 in Section 3 states this result.

Q3. A dictionary is a finite set of functions fixed before observing the data, each satisfying ∣hj(x)−hj(y)∣≤d(x,y)|h_j(x)-h_j(y)|\le d(x,y). Learning selects signed-softmax weights on these functions. The fixed-alternative target is their largest absolute population mean contrast, max⁡j∣(P−Q)hj∣\max_j|(P-Q)h_j|. Revised Section 2.2 gives this definition.

Q4. For a null law P=Q=πP=Q=\pi, a fixed reference point x0x_0 and some δ>0\delta>0, the condition Eπd(X,x0)2+δ<∞\mathbb E_\pi d(X,x_0)^{2+\delta}<\infty gives Tn⇒N(0,1)T_n\Rightarrow N(0,1) for the specified regularized dictionary and fixed perturbations. This states the moment assumption and the asymptotic calibration conclusion explicitly. The conditional-symmetry result separately provides anytime-valid monitoring without a moment assumption. The revised Abstract and Section 3 state this condition.

Q5. Kantorovich–Rubinstein (KR) duality (Villani, 2009) expresses W1(P,Q)=sup⁡Lip⁡(f)≤1Pf−QfW_1(P,Q)=\sup_{\operatorname{Lip}(f)\le1}\\{Pf-Qf\\}, where Pf=EPf(X)Pf=\mathbb E_Pf(X) and ∣f(x)−f(y)∣≤d(x,y)|f(x)-f(y)|\le d(x,y). The empirical optimizer maximizes the past-sample mean difference over this class up to its numerical tolerance. The drift bound separates optimization error, statistical error and any restriction of the critic class. Revised Section 2.2 gives the definition and Section 3.3 the drift bound.

Q6. We replace the quoted sentence with: “The eight-dimensional study checks null variance concentration and fixed-horizon rejection frequencies, then compares half-normal and ONS monitoring on identical perturbed scores.” The paired noise-removal comparison measures how the perturbations affect both monitoring rules, and the temperature comparison measures sensitivity to regularization. With no perturbation, temperature multipliers 0.5, 1 and 2 give location-change mean stopping times 210.8, 250.7 and 312.2 for half-normal and 265.7, 382.0 and 604.4 for ONS, capped at 1000. These results are in revised Section 4.1 and Appendix H (Tables 12–13).

Thank you again for your valuable feedback. We hope our responses address your concerns and welcome any further questions or discussion.

I thank the authors for their reply.

The edits are going in the right direction in my opinion but I maintain that this paper needs a rewrite that goes beyond a single round. I also maintain the rewrites should be human-made. I had a look at some of the changes in blue by the end of the main part. Again, I do not blame LLM usage in writting but most of the new / changed text is still very difficult to parse (by a human). You should be careful with 1) long sentences 2) stacking of qualifiers (adjectives) 3) weird, uncommon or never introduced idioms. 4) lack of connecting words.

However, I will maintain a low confidence score, the presentation preventing me from understanding most of the contribution.

Thank you for rereading the paper and for your specific suggestions on sentence structure, terminology, and the connections between ideas. In plain terms, the paper asks how to learn differences between two distributions from past observations and then test those differences on new observations. Its central finding is that this separation alone does not ensure a valid Gaussian test; the analysis identifies the variance problem and provides a construction that resolves it under the stated conditions. We hope this underlying statistical point retains value as we improve its presentation.

We respect your assessment and your decision. Your detailed comments provide practical guidance for making the exposition easier to follow, and we appreciate the time you have devoted to the review.

Reviewer_32vCOpenReview ↗
Rating
2
Confidence
2(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper builds a two-sample test from 1-Lipschitz (Wasserstein) critics in a sequential way. Each critic is trained only on past pairs and evaluated once on the next fresh pair, so under the null every score is centered and symmetric. The summed scores are used in two ways, as a studentized statistic compared to a normal quantile at a fixed sample size, and as a closed-form half-normal mixture e-process that is valid at any stopping time.

The authors observe that fresh evaluation fixes the mean but not the variance, since a critic that keeps learning can leave a random accumulated variance, and they give an example where the studentized sum is not normal. Their fix is an entropy-regularized softmax over a fixed finite dictionary of Lipschitz functions, with a temperature that decays more slowly than the training noise, plus a small injected noise term. Under the null the learned critic shrinks to zero and the injected noise gives a deterministic variance, so the statistic is asymptotically normal, while under alternatives the drift tends to the largest dictionary contrast. The paper also shows that approximate Kantorovich-Rubinstein critics recover the population distance in value. The validation consists of simulations with an 8-dimensional dictionary, exact one-dimensional transport critics compared with permutation tests, and a stratified analysis of penguin measurements.

Critical strengths
  1. The paper makes a clear and useful point about learned critics. It is tempting to think that training on the past and scoring the fresh pair is enough for a Gaussian test, and the example in Appendix C.4 shows that it is not, even when the observed and predictable variances coincide. This counts in my assessment because it applies well beyond Wasserstein critics, and it is the main thing I learned from the paper.
  2. For the regularized dictionary the paper gives a complete and correct answer. The two bounds, one for the null variance of the critic and one for its optimization gap, make the choice of temperature transparent. The proofs I checked (Propositions 3.1 and 3.2 and Theorem 3.3) are correct.
Critical weaknesses
  1. My main concern is the price paid for the Gaussian calibration. The temperature is chosen so that the learned critic becomes small, and the added noise then controls the variance. As far as I can tell, this makes the test weak against small differences. With the default temperature it cannot detect alternatives that shrink at the usual root-n rate, even when the dictionary already contains the best witness. The paper also does not compare with simple tests on the same features, such as Hotelling's T2T^2, or with the martingale kernel test of Chatterjee and Ramdas (2026), which also learns from past pairs and keeps a stable variance without adding any noise. Without these comparisons I do not see what the proposed construction gains, and this is the main reason for my score.

  2. Outside the finite dictionary, I do not think the claims are new or fully supported. The Gaussian result is only proved for a fixed dictionary, which is not a metric. Even there, the dictionary is only tested in dimension 8, where the added noise is large because it grows with the pair distance, so it is unclear how well the normal approximation holds in lower dimension. For the full Wasserstein critics of Section 4.3, the paper itself says the variance condition is not verified, yet Tables 3 and 8 report power with normal critical values. Finally, the anytime-valid test is a known construction (the one-sided normal mixture of Howard et al., 2021), and the comparison with ONS in Table 1 is not quite fair, because both rules run on scores with added noise that the e-process does not need. So the parts that would make the method broadly useful are either not new or not justified in my opinion.

Questions for authors
  1. Do you agree that with the default temperature the test cannot detect alternatives that shrink at the root-n rate? Is there a temperature schedule allowed by Proposition 3.2 that keeps root-n power?
  2. What does the tempered dictionary offer compared with the construction of Chatterjee and Ramdas, or with dividing each score by a standard deviation estimated from past pairs? Both give a stable variance without added noise.
  3. If you can, please compare with Hotelling's T2T^2 and the martingale kernel test on the same features in the 8-dimensional study, check the dictionary's calibration in lower dimension, and report the sequential results (Tables 1 and 2 and the penguin study) without the added noise.
7 comments
AuthorsOpenReview ↗

Response to Reviewer 32vC (1/5)

Thank you for your time and valuable comments.

W1. Our added local-power result quantifies the tradeoff between calibration and local power for the actual signed-softmax learner. Consider independent row-wise pairs and a fixed anchored one-Lipschitz dictionary of size KK, with a fixed finite zero-critic initialization, bounded update gaps, temperature ϑi=cri−β\vartheta_i=c r_i^{-\beta} for 0<β<1/20 < \beta < 1/2, fixed perturbation scales τ≥0\tau\ge0, ρ>0\rho > 0, and uniformly bounded (2+δ)(2+\delta) distance moments along the local alternatives. Put pβ=1/4+β/2p_\beta=1/4+\beta/2. If npβΔn→bn^{p_\beta}\boldsymbol\Delta_n\to b and κn→κ0\kappa_n\to\kappa_0, then

Tn⇒N(∣b∣2Kc(1+β)σ0,1),σ02=τ2κ0+ρ2.T_n\Rightarrow N\left(\frac{\\|b\\|^2}{Kc(1+\beta)\sigma_0},1\right),\qquad \sigma_0^2=\tau^2\kappa_0+\rho^2.

The default β=1/4\beta=1/4 therefore gives the test-specific separation rate n−3/8n^{-3/8}. At shift 3n−3/83n^{-3/8}, rejection frequencies at n=250,1000,4000n=250,1000,4000 are 0.331, 0.354, 0.330; at shift 3n−1/23n^{-1/2} they are 0.094, 0.090, 0.065. This identifies the cost of obtaining the explicit variance guarantee with this learner and fixed perturbations.

We also added the requested same-feature Hotelling and martingale-MMD comparisons, together with a held-out linear witness and RBF cross-MMD. The representative results in Q3 show that the dictionary reaches 0.970 location power at n=1000n=1000, while these comparators reach 0.999–1.000. Its practical attraction in this comparison is a single accumulating score with explicit variance/drift control and an anytime-valid monitoring option; its local-power cost is now explicit. The computational comparison evaluates horizons 250, 500 and 1000 on the same feature input: scoring with Gaussian, half-normal and ONS diagnostics takes 2.720 ms, versus 14.004 ms for Hotelling and 72.823 ms for linear martingale MMD with 499 within-pair swaps. These standalone implementation times include calibration and swap generation and exclude shared feature construction and warmup.

With fixed scalar jitter (τ,ρ)=(0,0.05)(\tau,\rho)=(0,0.05), the higher-dimensional comparison in Q3 gives power 0.706 at a sparse shift 0.6, versus 0.494–0.652 for the same-data baselines, with null frequency 0.0475. The original distance-perturbed default and a diffuse-signal control are included alongside it.

The local-power calculation uses the signed-softmax map gj(z)=sinh⁡(zj)/∑kcosh⁡(zk)g_j(z)=\sinh(z_j)/\sum_k\cosh(z_k). Let ξni\xi_{ni} denote the fresh-pair score and Gn,i−1\mathcal G_{n,i-1} the past observations and signs. Put Dn=∣Δn∣2D_n=\\|\boldsymbol\Delta_n\\|_2, B0=sup⁡n∑jVar⁡Pnhj+Var⁡Qnhj<∞B_0=\sup_n\sum_j\\{\operatorname{Var}_{P_n}h_j+\operatorname{Var}_{Q_n}h_j\\}<\infty, and let εni\varepsilon_{ni} denote the training error. Then E∣εni∣2≤B0/ri\mathbb E\\|\varepsilon_{ni}\\|^2\le B_0/r_i, ∣g(z)∣≤min⁡(1,∣z∣)\\|g(z)\\|\le\min(1,\\|z\\|) and ∣g(z)−z/K∣≤4∣z∣2\\|g(z)-z/K\\|\le4\\|z\\|^2. For the conditional score mean mni=Δn⊤g((Δn+εni)/ϑi)m_{ni}=\boldsymbol\Delta_n^\top g((\boldsymbol\Delta_n+\varepsilon_{ni})/\vartheta_i), the triangle inequality gives

\mathbb E\left|\frac{\sum_i m_{ni}}{\sqrt n} -\frac{D_n^2}{K\sqrt n}\sum_i\vartheta_i^{-1}\right| &\le\frac{D_n\sqrt{B_0}}{K\sqrt n}\sum_i\frac{1}{\sqrt{r_i}\vartheta_i}\\\\ &\quad+\frac{4D_n}{\sqrt n}\sum_i\frac{D_n^2+B_0/r_i}{\vartheta_i^2}. \end{aligned}$$ This bound applies directly to the overlapping prefixes. At $D_n=O(n^{-p_\beta})$, its three contributions have orders $n^{\beta/2-1/4}$, $n^{\beta/2-1/4}$ and $n^{3\beta/2-3/4}$, all tending to zero. Since $\sum_i\vartheta_i^{-1}\sim n^{1+\beta}/\\{c(1+\beta)\\}$, the normalized drift tends to $\\|b\\|^2/\\{Kc(1+\beta)\\}$. For the critic coefficients $a_{ni}=g((\boldsymbol\Delta_n+\varepsilon_{ni})/\vartheta_i)$, $$\frac1n\sum_i\mathbb E\\|a_{ni}\\|^2 \le\frac1n\sum_i\min\left\\{1,\frac{D_n^2+B_0/r_i}{\vartheta_i^2}\right\\}=o(1).$$ Thus the average conditional variance of the critic component vanishes, and the independent signs give $n^{-1}\sum_i\mathbb E[(\xi_{ni}-m_{ni})^2\mid\mathcal G_{n,i-1}]\to\sigma_0^2$. The uniform $(2+\delta)$ moment bound gives conditional Lindeberg. For $s=1+\min(\delta,2)/2$, the martingale moment bound for centered squared increments is $O(n^{1-s})$; together with $n^{-1}\sum_i m_{ni}^2\le D_n^2\to0$, this yields $n^{-1}\sum_i\xi_{ni}^2\to\sigma_0^2$. The martingale central limit theorem and the drift limit give the displayed noncentral normal law. Revised Proposition G.1 states this result; the local-alternative and comparator experiments are in Appendix H, and the recommended configuration in Section 4.2 (Table 3).
AuthorsOpenReview ↗

Response to Reviewer 32vC (2/5)

W2. We distinguish three guarantees: Gaussian calibration for the regularized finite dictionary, population-drift recovery for approximate full or restricted Kantorovich–Rubinstein (KR) optimization, and anytime validity from conditional symmetry. The dictionary target is DH=max⁡j∣(P−Q)hj∣D_{\mathcal H}=\max_j|(P-Q)h_j|, and its fixed-alternative consistency applies when DH>0D_{\mathcal H} > 0. For x,∣x∣\\{x,|x|\\}, the laws P=(δ−1+δ1)/2P=(\delta_{-1}+\delta_1)/2 and Q=δ−2/4+δ0/2+δ2/4Q=\delta_{-2}/4+\delta_0/2+\delta_2/4 have DH=0D_{\mathcal H}=0 and W1(P,Q)=1W_1(P,Q)=1, making the distinction concrete.

We checked lower-dimensional calibration and removed the perturbations in the sequential comparisons; the results appear directly in Q3. We describe the continuous-KR measurements as normal-reference rejection frequencies: Gaussian validity for those fits requires a deterministic sequence an→∞a_n\to\infty such that An2/an2→1A_n^2/a_n^2\to1 in probability, where An2=∑i≤nE(ξi2∣Gi−1)A_n^2=\sum_{i\le n}E(\xi_i^2\mid\mathcal G_{i-1}) is the accumulated predictable variance; this remains a condition for that experiment. For a two-point null with masses 1−p,p1-p,p, 0<p<10 < p < 1, and separation LL, exact empirical KR learning with diverging deterministic prefixes and fixed τ≥0\tau\ge0, ρ>0\rho > 0 gives An2/n→2p(1−p)L2(1+τ2)+ρ2A_n^2/n\to2p(1-p)L^2(1+\tau^2)+\rho^2 in L1L^1. This supplies a directly verifiable KR case; the continuous fits retain the stated variance condition.

The monitoring formula is explicitly credited to the classical conditionally symmetric exponential-supermartingale and one-sided normal-mixture constructions of de la Peña et al. (2004) and Howard et al. (2021). Both half-normal and ONS monitoring are evaluated on the same unperturbed scores. At n=1000n=1000, removing noise raises location crossing from 0.862 to 1.000 for half-normal and from 0.778 to 0.999 for ONS; their attained null crossing rates are reported alongside them in Q3. The revised paper states the three guarantees in Section 3, gives the KR variance examples in Appendix I and the noise-removal results in Appendix H (Table 12).

Q1. Yes. Consider independent row-wise pairs with law Pn⊗QnP_n\otimes Q_n, the fixed anchored one-Lipschitz dictionary, uniformly bounded (2+δ)(2+\delta) distance moments and κn=EPn⊗Qnd(X,Y)2→κ0\kappa_n=E_{P_n\otimes Q_n}d(X,Y)^2\to\kappa_0. Use the distance stabilizer with fixed τ≥0\tau\ge0, ρ>0\rho > 0, a fixed finite zero-critic initialization, and then the first rir_i pairs for training. Under these conditions, every single deterministic temperature schedule, shared across the rows, satisfying ri<ir_i < i, ri→∞r_i\to\infty, ϑi→0\vartheta_i\to0, and riϑi2→∞r_i\vartheta_i^2\to\infty has the following implication with fixed perturbations:

∣Δn∣=O(n−1/2)⟹Tn⇒N(0,1).\\|\boldsymbol\Delta_n\\|=O(n^{-1/2})\quad\Longrightarrow\quad T_n\Rightarrow N(0,1).

Indeed, for the critic coefficient vector ania_{ni}, the signed-softmax bound gives

\frac1n\sum_i\mathbb E\\|a_{ni}\\|^2\le \frac1n\sum_i\min\left\\{1,\frac{C}{r_i\vartheta_i^2}\right\\}\to0,

so the cumulative drift divided by n\sqrt n tends to zero, while the fixed perturbations retain positive limiting variance. A Gaussian translation with Pn=N(0,1)P_n=N(0,1), Qn=N(a/n,1)Q_n=N(a/\sqrt n,1) and dictionary x\\{x\\} illustrates the consequence even with the optimal signed witness available: the learned test has limiting rejection α\alpha, whereas the known-witness statistic ∑i(Yi−Xi)/2n\sum_i(Y_i-X_i)/\sqrt{2n} has distribution N(a/2,1)N(a/\sqrt2,1). The quantified rate in W1 describes the resulting range of local alternatives detected by the regularized learner. Revised Appendix G.1 gives this result, and Appendix I the Gaussian translation example.

AuthorsOpenReview ↗

Response to Reviewer 32vC (3/5)

Q2. Chatterjee and Ramdas (2026) already use past witnesses and fresh-pair increments for Gaussian testing. Our learner-specific result gives explicit control of two competing quantities for one-Lipschitz critics: entropy regularization makes the null critic variance vanish, while its optimization error vanishes under fixed alternatives, retaining the largest dictionary mean contrast. Fixed random-sign perturbations then provide a positive deterministic variance scale, including for point-mass nulls. Approximate KR optimization is handled separately through a training/optimization/class-approximation error decomposition for population drift.

Predictable normalization is another valid route when its variance estimates satisfy the appropriate relative condition. If UiU_i is a null martingale difference, vi=E(Ui2∣Gi−1)v_i=\mathbb E(U_i^2\mid\mathcal G_{i-1}), and v^i>0\widehat v_i > 0 is predictable, then the conditions

1n∑iviv^i→1,sup⁡iE[∣Uiv^i∣2+δ∣Gi−1]≤C\frac1n\sum_i\frac{v_i}{\widehat v_i}\to1,\qquad \sup_i\mathbb E\left[\left|\frac{U_i}{\sqrt{\widehat v_i}}\right|^{2+\delta}\mid\mathcal G_{i-1}\right]\le C

with the first convergence in probability, give Gaussian studentization of the normalized scores. For a concrete case, fix finitely many features with finite (2+δ)(2+\delta) moments under the null and sj2=Var⁡πhj>ϵ>0s_j^2=\operatorname{Var}_\pi h_j > \epsilon > 0. Select any signed single feature using past data and set v^i=2max⁡s^Ji,i,2,ϵ\widehat v_i=2\max\\{\widehat s_{J_i,i}^{\\,2},\epsilon\\}, where s^j,i,2\widehat s_{j,i}^{\\,2} is the pooled past sample variance. Uniform consistency over the finite dictionary gives vi/v^i→1v_i/\widehat v_i\to1 almost surely even under adaptive selection; the fixed floor gives the conditional moment bound. This yields Gaussian inference without added noise for null laws satisfying the stated positive-variance restriction. Our jitter construction additionally covers point-mass nulls without estimating a feature variance. Positive predictable scaling also preserves conditional symmetry, supporting the mixture e-process without these moment conditions. Thus this alternative transfers the Gaussian-calibration requirement to relative variance estimation; the entropy construction supplies a directly verified variance limit for its specified learner. Revised Lemma I.2 in Appendix I states this normalization result.

Q3. We evaluated all methods on Φ(x)=(x1,…,x8,∣x1∣,…,∣x8∣)\Phi(x)=(x_1,\ldots,x_8,|x_1|,\ldots,|x_8|). HT is Hotelling, LM is linear martingale MMD, HW is the held-out empirical linear witness, RM is the RBF martingale MMD test of Chatterjee and Ramdas (2026), and RC is RBF cross-MMD. HT and LM use within-pair swap calibration; HW trains on one half and calibrates on the independent evaluation half. RM and RC use their published Gaussian calibration with γ=1/(4d)\gamma=1/(4d) on the common input. The RBF methods use a richer nonlinear function class. The comparison uses fresh streams, independent of the submitted experiment: 2000 per null and 1000 per alternative, at nominal 0.05:

SettingnnDefaultHTLMHWRMRC
Normal null10000.05350.05200.05550.03950.04400.0520
Standardized t5t_5 null10000.04450.04900.04950.05400.03600.0470
Location shift 0.42500.3660.7840.7500.7330.6870.712
Location shift 0.410000.9701.0000.9991.0000.9991.000
Scale factor 1.32500.7941.0001.0001.0001.0001.000
Scale factor 1.310001.0001.0001.0001.0001.0001.000

Setting τ=0\tau=0 and retaining the original scalar perturbation ρ=0.05\rho=0.05 also improves the eight-dimensional comparison. On the same observations and signs as the default-configuration experiment:

SettingnnDefaultScalar jitter
Normal null2500.04550.0385
Standardized t5t_5 null2500.04300.0465
Normal null10000.05350.0435
Standardized t5t_5 null10000.04450.0455
Location shift 0.42500.3660.849
Scale factor 1.32500.7941.000
Location shift 0.410000.9701.000

Comments on the paper

1 comment

Dear Reviewers,

Thank you for your thoughtful reviews. The authors have provided detailed responses to the concerns raised.

I would appreciate it if you could revisit the author responses and clarify whether your main concerns have been adequately addressed, particularly which issues remain critical to your overall assessment.

Please feel free to update your ratings if your assessments have changed following the discussion.

Thank you for your continued contributions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.