High-Dimensional Conditional Independence Testing via Random Projection Aggregation

Conditional independence (CI) testing is a fundamental problem in statistics. However, classical nonparametric CI tests suffer from the curse of dimensionality, as nonparametric estimation degrades rapidly in high dimensions. This paper proposes RP-CIT, a random-projection-aggregated CI test that remains effective in the high-dimensional regime. The central idea is to project $X$ and $Y$ onto random one-dimensional subspaces before applying a univariate base CI test. This projection (i) preserves conditional independence under the null hypothesis and (ii) reduces the original high-dimensional problem to a collection of univariate CI subproblems, thereby avoiding high-dimensional nonparametric estimation. Since a single projection may miss the dependence signal, we aggregate the $p$-values from $k$ independent projections via a Bonferroni-max rule, which asymptotically controls the Type I error at the nominal level and yields a limiting Type II error bound that decreases exponentially in $k$ under a detectable-projection condition. The paper also develops a top-$r$ aggregation extension for alternatives whose signal is spread across multiple projection directions. More broadly, the random-projection aggregation framework can be combined with other scalar base tests; as one example, we use a projected GCM base test to accommodate higher-dimensional conditioning variables. Experiments on synthetic benchmarks, image-valued CI tasks, and Norman Perturb-seq module skeleton learning illustrate the calibration, power, and practical utility of the framework.