acceptodds
Under review as a conference paper at ICLR 2027

KREDIT: High-Dimensional Conditional Independence Testing via Kernel-Residualized Distances

Abstract

Conditional independence testing underpins graphical model learning and causal discovery, yet high-dimensional inference with estimated conditional distance features remains challenging. To this end, we introduce KREDIT (Kernel-Residualized Distance Testing), a nonparametric framework combining two-sided distance residualization, independent conditional-mean fitting, and off-diagonal joint studentization. An exact pair-kernel analysis separates sampling variation from directional regression error. Under explicit moment, spectral, and prediction conditions, we prove feasible Gaussian limits, finite-sample approximation bounds, and dimension-dependent convergence rates. We further establish shrinking-signal power guarantees, empirical-variance inference under nondegenerate alternatives, and testing and estimation lower bounds in a rectangular Gaussian subexperiment. A specified nonlinear family remains detectable despite zero linear covariances. Simulation studies verify linear and nonlinear settings under which stratum-calibrated KREDIT has higher empirical power than the benchmark methods, with size near the nominal level. Wine chemistry and rolling gold–inflation analyses further illustrate its practical utility.

Primary area:
learning theory
Keywords:
Conditional dependence, distance correlation, kernel regression, conditional independence testing

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_YxFvOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes KREDIT, a conditional independence test based on kernel-residualized distances. The method estimates conditional feature means using an independent training sample, residualizes distance-induced features on both sides, and constructs an off-diagonal, jointly studentized pair statistic. The theoretical analysis studies both null calibration and power in growing-dimensional settings. In particular, the paper separates sampling variation from errors introduced by conditional-mean estimation, establishes feasible Gaussian limits and finite-sample approximation bounds, and derives explicit dimension-dependent rates in several structured models. It also provides signal-relative power guarantees for shrinking alternatives and an information-theoretic lower bound in a Gaussian subexperiment.

Critical strengths

(1) The theoretical analysis is well motivated by the characteristics of the high-dimensional setting. The paper explicitly considers how spectral behavior, nuisance estimation, and signal strength change with dimension, and derives corresponding null calibration and dimension-dependent rates in several concrete regimes.

(2) The treatment of nuisance-estimation error is relatively refined. The analysis separates mean shift, directional first-projection error, and second-order pair-kernel perturbation. Compared with prior CME-based error analyses, this appears to provide a finer characterization of how conditional-mean estimation affects the test statistic. I did not identify any substantive theoretical issues in the derivations or main results.

Critical weaknesses

(1) The theoretical positioning of the paper is somewhat unclear to me. Most of the analysis is developed for the specific KREDIT statistic and for several structured model families, while parts of the perturbation framework appear substantially more general and may extend to other residual-covariance tests discussed in Table 1. At the same time, the empirical claims emphasize the practical advantage of the particular KREDIT construction. I think the paper would benefit from a clearer separation between what is genuinely specific to KREDIT and what should be viewed as a more general high-dimensional inference principle.

(2) The empirical evidence for the claimed power advantage is somewhat narrow relative to the breadth of the methodological claims. The main comparison considers only the two structured alternatives in Eq. (16), where KREDIT achieves very large power gains over all baselines. However, the paper provides limited analysis of why these particular alternatives lead to such strong power, or how they relate to the signal-relative conditions underlying the power-one result in Theorem 2. I think the paper should more explicitly characterize the regimes in which KREDIT is expected to be particularly powerful, and also provide boundary or counterexample settings where its global residual-covariance signal becomes weak or vanishes. Since the paper itself notes that this target can cancel across conditioning values, such cases would help clarify the actual scope and limitations of the proposed test.

Questions for authors

(1) Would it be possible to clarify which parts of the theoretical analysis are specific to the KREDIT statistic and which may extend to related residual-covariance tests such as GHCM or GKCM? I understand that a complete extension may be technically difficult, but even some discussion of this distinction would help clarify the scope of the theoretical contribution.

(2) Could the authors provide more intuition for the particularly large power gains in Eq. (16), and discuss settings in which this advantage is expected to weaken?

Reviewer_kacKOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes KREDIT, a high-dimensional conditional independence test. It combines distance-kernel residualization, conditional feature-mean estimation using independent training data, and joint studentization. The theoretical analysis accounts for estimation error, establishes asymptotic normality under the null, and analyzes power against shrinking alternatives. Simulations examine both linear and nonlinear settings.

Critical strengths

The paper uses a refined bound on the first projection to relax the dimension requirements for consistency while accounting for estimation error. In the nonlinear example with estimated global means, Corollary 4 establishes power tending to one for d=o(n3/4)d=o(n^{3/4}).

Critical weaknesses

The paper presents many theoretical results (3 theorems, 12 propositions, 4 lemmas, and 5 corollaries across the main text and appendices) but does not clearly highlight its main contribution, making it hard to follow. The connections between the general theorems, the specific examples, and the improvements over prior work need clearer explanation.

  • The theoretical contribution beyond GHCM [1], GKCM [2], and SplitKCI [3] remains unclear. The paper lists differences from these methods but does not clearly explain which require new analysis or what statistical improvements follow.

  • The paper introduces 14 assumptions across different results, some of which are high-level. In particular, the restrictiveness of the spectral and estimation error conditions in Assumptions 2-3 remains unclear beyond the worked examples.

[1] Lundborg, A. R., Shah, R. D., and Peters, J. (2022). Conditional Independence Testing in Hilbert Spaces with Applications to Functional Data Analysis. Journal of the Royal Statistical Society Series B: Statistical Methodology, 84(5), 1821–1850.

[2] Bergen, L., Sejdinovic, D., and Didelez, V. (2026). The Generalised Kernel Covariance Measure. Proceedings of the Fifth Conference on Causal Learning and Reasoning, PMLR 323, 1198–1220.

[3] Pogodin, R., Schrab, A., Li, Y., Sutherland, D. J., and Gretton, A. (2025). Practical Kernel Tests of Conditional Independence. arXiv preprint arXiv:2402.13196v2.

Questions for authors
  • Appendix E.3 uses d=⌊n0.4⌋d=\lfloor n^{0.4}\rfloor, which does not explore the additional regime n≪d≪n3/4\sqrt n\ll d\ll n^{3/4} covered by Corollary 4. Could the authors add experiments in this regime?

  • Proposition 12 addresses individual tests, but Section 5.2 applies BH adjustment across overlapping windows. What supports the marginal calibration and joint dependence conditions needed for FDR control?

  • High dimensionality and prediction consistency alone do not ensure Assumptions 2-3. Beyond the specified models, what diagnostics or practical guidance can help assess the spectral and estimation-error conditions?

Reviewer_i82VOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

Paper proposes KREDIT, a conditional independence test for X⊥Y∣ZX\perp Y\mid Z based on Euclidean distances. The method is based on distance-induced kernel k(x,x’)=12(∣x∣+∣x’∣−∣x−x’∣)k(x,x’)=\frac12(|x|+|x’|-|x-x’|). It splits the sample into training and evaluation sets, using Nadaraya–Watson smoothing on the training set to estimate conditional kernel means (given ZZ). Then these estimates are used to residualize the pairwise Euclidean distances of XX and YY on the evaluation set. The method multiplies the two residualized distances and averages over distinct observation pairs, producing an off-diagonal U-statistic. It then studentizes this statistic using the empirical second moment of the pair products, accounting for their dependence through ZZ even under conditional independence. Under suitable conditions, the resulting statistic has an asymptotic standard normal null distribution. The paper also provides multiplier and within-stratum permutation calibration.

Critically, KREDIT does not characterize conditional independence: its population target is ∥EZ[CXY∣Z]∥2\|\mathbb E_Z[C_{XY\mid Z}]\|^2, which vanishes under conditional independence but can also vanish under alternatives when conditional dependence cancels across values of Z.

Critical strengths
  • Clarity and theory. The method is clearly presented and the proofs appear correct to me. But the assumptions of the main results should appear in main body.

  • Simple and computationally efficient method. The statistic is straightforward to implement using distance matrices, without explicit feature representations.

Critical weaknesses
  1. Limited methodological novelty. KREDIT builds on established ideas in residual-based conditional independence testing (Patra et al., 2016), distance–kernel conditional dependence measures (Sheng & Sriperumbudur, 2023), and conditional mean embedding methods (Huang et al., 2022). The proposed combination of sample splitting, NW estimation, off-diagonal evaluation, and joint studentization is not accompanied by an ablation or a convincing comparison isolating its benefits over existing approaches. Consequently, the paper does not clearly establish the added value of its methodological construction. References: [1] Huang, Z., Deb, N., & Sen, B. (2022). Kernel partial correlation coefficient — a measure of conditional dependence. JMLR; [2] Patra, R. K., Sen, B., & Székely, G. J. (2016). On a nonparametric notion of residual and its applications. Statistics & Probability Letters; [3] Sheng, T., & Sriperumbudur, B. K. (2023). On distance and kernel measures of conditional dependence. JMLR

  2. The population target does not characterize conditional independence. KREDIT measures the squared norm of the average conditional cross-covariance, which vanishes under conditional independence but may also vanish under alternatives due to cancellation across values of (Z). Unlike approaches that incorporate a conditioning kernel, it cannot detect such alternatives, even asymptotically. Although the authors acknowledge this limitation, the abstract and introduction present KREDIT as a general conditional independence test, overstating its scope.

  3. The benchmark does not convincingly establish empirical superiority. The experiments rely on overly simple conditioning structures, particularly binary (Z), for which regression reduces to estimating group means and exact permutation calibration is available. Nonlinear alternatives retain substantial linear dependence, while the zero-linear-covariance example is evaluated without comparators. Moreover, some baseline methods use less samples than KREDIT, and the Gaussian-kernel baselines potentially use unfavorable bandwidths. Without ablations separating the effects of kernel choice, sample splitting, and studentization, the reported power gains cannot be attributed to the proposed construction.

Questions for authors
  1. Continuous Z. Power comparison uses a binary Z, where regression is two group means and exact permutation is available. Can you add a design with a smooth, continuous scalar Z (nonlinear conditional mean and variance), and run all comparators on it with a calibration that does not need discrete strata? Can you please report size and power? In that setting regression error matters, and it is the regime your theory addresses.

  2. High-dimensional evaluation. Despite the title, experiments only consider d=8,16d=8,16 with n=100n=100, while Corollary 3 requires d=o(n1/3)d=o(n^{1/3}) for nonconstant conditional means. Could the authors compare size and power against baselines as dd increases, including sparse alternatives and nonconstant conditional means? This would clarify the method’s practical high-dimensional validity.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.