acceptodds
Under review as a conference paper at ICLR 2027

Uncertainty-Aware Rank Test with Missing Data and Application in Causal Discovery

Abstract

Covariance-rank constraints are widely used in causal discovery, including settings with latent variables, but missing observations complicate their statistical testing. Pairwise covariance estimates use different, overlapping sets of rows; yet, plugging these estimates into a classical rank test with an effective sample size correction need not properly control Type-I error. In this paper, we propose the Missing-value uncertainty-Aware Rank Test (MART), which uses partially observed rows beyond those retained by test-wise deletion. MART consistently estimates the joint uncertainty of the pairwise covariance estimates and measures their weighted distance to the rank-constrained set. A Kronecker approximation makes this distance solvable by a whitened singular value decomposition, while the full estimated uncertainty calibrates the statistic through a weighted chi-square mixture as the asymptotic null distribution. For causal discovery with many rank test queries, the covariance and uncertainty estimates can be computed once and reused to reduce the computational cost. Experiments on both synthetic and real-world data validate our theoretical claims and the effectiveness of our method.

Primary area:
causal reasoning
Keywords:
Rank Test, Missing Value

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_Xe4EOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes a statistical test for the rank of a covariance matrix when observations of some variables are missing within a sample. A typical naive approach would estimate each covariance entry from the rows where two variables are observed and plugs these estimates into a classical rank test. However, each entry then comes from a different, overlapping subset of rows, so a single effective sample size cannot capture their joint uncertainty. The proposed method in this work addresses this by estimating the joint sampling uncertainty of the pairwise estimates and using it to calibrate a rank-test statistic. The method is evaluated on synthetic and real-world data with an application to causal discovery.

Critical strengths

The paper clearly describes the issue and fairly discusses why 'naive' approaches (e.g., imputation or pairwise estimation with single effect sample size) do not ensure proper type-1 error control. The authors provide theoretical guarantees for their approach and show promising experimental results.

Critical weaknesses

Majority of the experiments use synthetic data and even the real-world experiment requires artificially masking observations. The O(M^4) memory requirement for caching the full joint covariance estimate seems quite large and not suitable for larger variable sets. The work by Dong et al. (2025) also uses a covariance rank test for causal discovery and, while the paper mentions it in the related work, a direct comparison in the experiments is missing.

Questions for authors

The work addresses a fairly common issue in practice and, overall, this is promising work. A few clarifying questions:

  • What happens in practice when the matrices are singular? The paper mentions a singular uncertainty matrix is not an issue but the exact stabilization rule is not given as far as I see. How should one choose it and how sensitive are the results with respect to it?
  • You mention the paper by Dong et al. (2025) "Permutation-Based Rank Test in the Presence of Discretization and Application in Causal Discovery with Mixed Data", which also uses a covariance rank test for causal discovery. It targets discretized rather than missing data, but real data often have both problems. Could the two approaches be combined to handle data that are both discretized and incomplete? A short discussion, or a small experiment if feasible, would be really insightful.
  • It is not entirely clear why the pairwise baselines (pairwise covariances in the classical rank test with a single effective sample size) have Type-I error rates so far above 5%. You mention the uncertainty mismatch, but the reported numbers do not seem to fully explain it. Is there a clearer explanation? I might be missing some details here.
3 comments

Thank you for your insightful comments and valuable feedback. We appreciate your positive assessment of the paper's clear motivation, practical relevance, and theoretical contributions, particularly your recognition of the importance of accounting for joint sampling uncertainty in pairwise covariance estimation and our theoretical guarantees for valid statistical testing. We address your concerns and suggestions in detail below.


Q1: The O(M4)O(M^4) memory for caching seems quite large?

A1: Thank you for raising this concern. Although the full cache has a storage complexity of O(M4)O(M^4), its absolute memory requirement remains manageable for moderately sized problems. Specifically, we cache the joint uncertainty matrix of the k=M(M+1)/2k=M(M+1)/2 unique covariance entries, requiring 8k28k^2 bytes with float64 precision:

Number of variables MMCache storage (float64)
5013.0 MB
100204.0 MB
2003.23 GB
30016.31 GB

As shown above, full caching remains feasible for moderate dimensions, although memory requirements become substantial as MM increases.

Importantly, this storage cost should be considered alongside the computational demands of causal discovery. Constraint-based causal discovery algorithms can have exponential worst-case search complexity, making the search procedure the major bottleneck in larger problems. In this context, caching avoids repeated covariance and uncertainty estimation across rank-test queries, improving computational efficiency at the expense of memory.

We thank the reviewer again for highlighting this concern and have added a corresponding discussion to the revised manuscript (Appendix I.6).


Q2: Comparison with MPRT (Dong et al., 2025) and the possibility of combining the two methods to handle discretized and incomplete data?

A2: Thank you for this insightful question. MPRT addresses rank test with discretized observations, assuming complete data, whereas MART focuses on rank test with missing values. Since the two methods address different observation mechanisms, a direct experimental comparison under our current setting would not be appropriate. However, combining the two methods to handle data that are both discretized and incomplete is feasible. We outline two possible strategies below.

(i) Available-pair estimation with permutation calibration. We estimate each polychoric correlation using all observations available for the corresponding variable pair. Under MCAR, positive observation probabilities, and a correctly specified latent Gaussian threshold model, these estimators are consistent under standard identifiability and regularity conditions. We then compute the CCA projections from the estimated correlation matrix and follow Equation (8) of Dong et al. (2025): after each permutation, we re-estimate the cross-block correlations while keeping the original CCA projections and within-block normalizations fixed to construct a permutation reference distribution for the test statistic.

(ii) Available-pair estimation with MART-style asymptotic calibration. Alternatively, MART's sandwich-based uncertainty estimation can be extended to polychoric correlations estimated from incomplete observations. Specifically, the pairwise likelihood score equations, together with the estimating equations for ordinal thresholds, provide a basis for estimating their joint asymptotic covariance. This would account for both threshold-estimation uncertainty and dependence induced by overlapping subsets of observed data. The resulting covariance estimate could then support a weighted chi-square calibration of the rank statistic. While theoretically motivated and methodologically feasible, this approach requires additional derivations and regularity conditions.

To demonstrate empirical feasibility, we conducted new experiments following (i). Specifically, we set ∣A0∣=∣B0∣=∣C∣=5|A_0|=|B_0|=|C|=5 and generate latent Gaussian observations and discretize each variable into three ordered categories. Missingness is then introduced independently of the data, with variable-specific missingness probabilities drawn from Unif⁡(0.09,0.11)\operatorname{Unif}(0.09,0.11). We test H0:rank⁡(ΣA,B)≤6H_0:\operatorname{rank}(\Sigma_{A,B})\leq6 for the underlying continuous covariance matrix, with true population ranks of 6 and 7 for evaluating Type-I error and power, respectively.

The results are shown in the newly added Table 6. The combined method effectively controls Type-I error near α=0.05\alpha=0.05, while statistical power increases from 16.0% to 99.7% as the sample size grows from 500500 to 50005000. These results provide empirical evidence of the feasibility and effectiveness of combining the two methods for covariance-rank testing with both discretized and missing observations.

We thank the reviewer again for this valuable suggestion. We have incorporated the additional experiments in Table 6 and expanded the related discussion in Appendix I.1.


Q3: How are the Kronecker factors stabilized?

A3: Thank you for pointing out that the stabilization procedure was insufficiently specified. Stabilization is necessary to ensure that the fitted Kronecker factors admit numerically stable inverse square roots. Specifically, nonpositive eigenvalues make the inverse square roots undefined, while very small eigenvalues may amplify numerical errors.

For each fitted factor KK, we first symmetrize it as Ks=K+K⊤2,K_s=\frac{K+K^\top}{2}, which preserves its asymptotic limit under the assumed convergence conditions. We then apply a diagonal shift:

\qquad \delta=\max(\epsilon-\lambda_{\min}(K_s),\epsilon),$$ where $\epsilon=10^{-6}$. This procedure ensures that all eigenvalues of the stabilized factors are bounded below by $\epsilon$. As an additional numerical safeguard, we also floor eigenvalues at $\epsilon$ when computing the inverse square roots. We further conducted a sensitivity analysis with $\epsilon$ ranging from $10^{-8}$ to $10^{-5}$ and observed nearly identical test results, demonstrating that MART is robust to the choice of this stabilization parameter within the evaluated range. We thank the reviewer again for this helpful comment and have updated Appendix C of the revised manuscript to include these implementation details. --- **Q4: Why do the pairwise baselines (pairwise covariance estimates plugged into a classical rank test with a single effective sample size) exhibit Type-I error rates substantially above 5\%?** **A4:** Thank you for this insightful question. **The key reason is the mismatch between the actual asymptotic null distribution and the assumed chi-square reference distribution.** We provide a four-variable example below with minimum effective sample size to illustrate how this mismatch can lead to substantial Type-I error inflation. **Example setup.** Let $Z\sim N(0,1)$, $X_i=a_iZ+\varepsilon_i$ for $i=1,2$, and $Y=Z+\varepsilon_Y$, where $a_1=1$, $a_2=1/2$, and the noise terms are mutually independent $N(0,v)$ variables with $v=0.1$, independent of $Z$. Then $(X_1,X_2)\perp Y\mid Z$, and the covariance block between $A=(X_1,X_2,Z)$ and $B=(Y,Z)$ has rank exactly one. We keep $Z$ fully observed and independently observe each remaining variable with probability $s=0.9$ (i.e., 0.1 missing rate), independently of the data. Under this MCAR mechanism, the pairwise covariance estimates remain consistent, and the minimum pair count satisfies $N_{\min}/N\to s^2$. **Actual asymptotic null distribution.** Define the estimated partial covariances $\widehat g_i=\widehat\sigma_{X_iY}-\widehat\sigma_{X_iZ}\widehat\sigma_{ZY}/\widehat\sigma_{ZZ}$, whose population values are zero. Applying the delta method to the joint asymptotic distribution of the pairwise covariance estimates yields $$\frac{\sqrt{N_{\min}}}{v}\widehat{\boldsymbol g} \xrightarrow{d} N\left(0,\operatorname{diag}(\kappa_1,\kappa_2)\right),$$ where $$\kappa_i= 1+\frac{(1-s)(1+a_i^2)}{v} +\frac{2a_i^2(1-s)^2}{v^2}.$$ Substituting the specified parameters gives $\kappa_1=5$ and $\kappa_2=2.75$. The additional variance terms arise because different covariance entries are estimated from different, overlapping subsets of observations, disrupting the cancellation of estimation errors that occurs with complete data. Consequently, the classical plug-in statistic satisfies $$T_{\mathrm{pairwise}} =\frac{N_{\min}}{v^2} (\widehat g_1^2+\widehat g_2^2)+o_p(1),$$ and **its actual asymptotic null distribution is** $$T_{\mathrm{pairwise}} \xrightarrow{d} 5G_1^2+2.75G_2^2, \qquad G_1,G_2\overset{\mathrm{iid}}{\sim}N(0,1),$$ **rather than the conventional $\chi_2^2$ distribution.** **Type-I error inflation.** Using the nominal 5\% cutoff of $5.99$ from the $\chi_2^2$ distribution yields an asymptotic rejection probability of approximately 45.2\%, whereas the correct weighted chi-square mixture has a 5\% upper-tail cutoff of approximately 23.74. This example demonstrates that **even though the pairwise covariance estimates are consistent, using a single effective sample size can severely miscalibrate the null distribution and inflate Type-I error.** As the sample size increases, the rejection rate under mismatch can further increases, which is aligned with the substantial Type-I error inflation observed in Section 4.1.1. --- **Thank you again for your valuable comments. We have uploaded our revised manuscript, with all changes highlighted in blue for ease of reference. Please feel free to let us know if you have any further questions or comments.**

I want to thank the authors for their clarifications and addressing my questions (in particular on Q2), I keep my recommendation for acceptance.

Reviewer_7vNFOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes MART, a covariance-rank test for incomplete data. The authors first states the relationship between conditional independence (CI) and rank test in Th. 1. The estimator of each covariance element is given in Eq. (3), which leverages the jointly observed sample. Then, the asymptotic distribution and the variance of the estimator is given in Th. 2. The estimator of the variance is given in Eq. (9), and its asymptotic distribution is given in Th. 3. The test statistics is constructed by the squared mahalanobis distance of the covariance estimator in Eq. (10). However, the test statistics is hard to compute directly. As a result, a feasible computation way is given in Eq. (11) - (14). Further, the asymptotic distribution of the test statistics under H0:rank=rH_0: rank = r is given in Th. 5, which leads to a valid statistical test (please read weakness 2). The test statistics is shown to have higher power (Th. 6).

Critical strengths
  1. The paper presents a clear framework, with good theoretical analysis, and empirical evaluation. Overall, it is a nice statistical paper.

  2. Causal discovery in the presence of missing data is an important and practically relevant problem. This paper provides a potential approach to addressing this challenge.

  3. The authors theoretically prove that the proposed test achieves strictly higher asymptotic local power than the classical CCA rank test based on test-wise deletion under a specific setting (Th. 6).

Critical weaknesses
  1. The MCAR assumption (Assumption 1) is quite restrictive. Missing pattern may depend on observed or unobserved variables, especially in causal discovery. This is why I said this is a 'potential approach' in Strength 2.

  2. Th. 5 only indicates the type I error of the proposed test under H0:rank=rH_0: rank=r, rather than H0:rank≤rH_0: rank\leq r. Although sequentially testing (r=0,1,…r=0,1,\ldots) can solve this issue in practice, I would prefer to see a complete Type-I error guarantee.

Questions for authors

Please read the weakness.

3 comments

Thank you for your insightful comments and valuable feedback. We appreciate your positive assessment of the paper's clear framework, theoretical rigor, and empirical evaluation, particularly your recognition of the practical importance of handling missing data in causal discovery and our theoretical guarantee of strictly higher asymptotic local power over test-wise deletion (Theorem 6). We address your concerns and suggestions in detail below.


Q1: Scope of the method and applicability under MAR and MNAR?

A1:

1. Scope of MART and its relationship to MVPC [1]

We would like to clarify that MART is a statistical rank test designed to improve upon classical test-wise deletion, whereas MVPC is a causal discovery framework that addresses more general missingness mechanisms.

Specifically, conditional independence testing via partial correlation is a special case of covariance-rank testing. Compared with test-wise deletion, MART utilizes additional incomplete observations and achieves strictly higher asymptotic local power under the setting of Theorem 6.

Therefore, MART and MVPC address different but complementary aspects of causal discovery with missing data: MART improves the underlying statistical test, while MVPC incorporates graphical assumptions and missingness correction to handle certain MAR and MNAR mechanisms.

2. Why MCAR is straightforward, while MAR and MNAR require additional assumptions

Under MCAR, both test-wise deletion and MART are valid because their covariance estimators consistently target the corresponding population covariances.

For example, suppose YY contains missing values, with missingness indicator RYR_Y. Under MCAR, RYR_Y is independent of the data, implying

Cov⁡(X,Y∣Z,RY)=Cov⁡(X,Y∣Z).\operatorname{Cov}(X,Y\mid Z, R_Y) = \operatorname{Cov}(X,Y\mid Z).

Thus, covariance estimates based on observed cases remain consistent for the population target.

However, under MAR or MNAR, this equality does not generally hold. Consider the following example adapted from Figure 1 of MVPC [1]:

X & \longrightarrow & Z & \longrightarrow & Y \\\\ & \searrow & & \swarrow & \\\\ & & W & & \\\\ & & \downarrow & & \\\\ & & R_Y & & \end{array}$$ Here, $W$ is a collider on the path $X\to W\leftarrow Y$ and a parent of $R_Y$. Conditioning on $R_Y$ can induce dependence through this collider, even after conditioning on $Z$. Consequently,

\operatorname{Cov}(X,Y\mid Z, R_Y) \neq \operatorname{Cov}(X,Y\mid Z)

in general. Therefore, directly applying either test-wise deletion or MART may lead to biased covariance estimation. **3. How MVPC addresses MAR and MNAR** Importantly, this limitation can be addressed by incorporating graphical assumptions and identifying the parents of missingness indicators, as demonstrated by MVPC [1]. Specifically, after identifying $W$ as the parent of $R_Y$, the population distribution can be recovered through

p(X,Y,Z) =\int p(X,Y,Z\mid W)p(W),dW =\int p(X,Y,Z\mid W,R_Y)p(W)dW,

where the second equality follows from the conditional independence of $(X,Y,Z)$ and $R_Y$ given $W$, as $W$ is the parent of $R_Y$. This allows consistent recovery of the target population covariances and the corresponding conditional independence relations. However, MVPC still relies on test-wise deletion for its underlying conditional independence tests, which can substantially reduce the available sample size, especially when the conditioning set $Z$ is high-dimensional.

4. Extending our method MART to MAR and MNAR

A natural question is whether MART can be extended to MAR and MNAR by incorporating the strategy of MVPC. The answer is yes.

Once the relevant parents of missingness indicators are identified, the population distribution can be recovered using the same graphical correction strategy as MVPC, as p(X,Y,Z)=∫p(X,Y,Z∣W,RY)p(W)dWp(X,Y,Z)=\int p(X,Y,Z\mid W,R_Y)p(W)dW. The key difference is that now MART serves as the underlying statistical test, so we can make use of all available rows even they are incomplete for X,Y,ZX,Y,Z. Such an integration would combine MVPC's strategy to address broader missingness mechanisms with MART's improved utilization of incomplete data, offering particular advantages in higher-dimensional settings.

We thank the reviewer for highlighting this important direction. While the extension is feasible under appropriate identification assumptions, establishing rigorous theoretical guarantees would require extending MART's uncertainty-aware calibration to account for the additional uncertainty introduced by missingness correction. We view the theoretical development and empirical evaluation of this integrated framework as a promising direction for independent future work, beyond the scope of the present paper.

5. Summary of the comparison

Statistical test|Causal discovery frameworks
Test-wise deletionMART (Ours)|MVPCFuture work
Data utilizationOnly test-wise complete rowsAll available rows|Underlying testTest-wise deletionMART
MCARYesYes|MCARYesYes
MARNoNo|MARYes, under graphical assumptionsYes, by incorporating the strategy in MVPC
MNARNoNo|MNARYes, under graphical assumptionsYes, by incorporating the strategy in MVPC

Thank you again for the insightful question. We have also revised the manuscript to incorporate these discussions and clarify the scope of MART and its potential integration with existing missingness-aware causal discovery frameworks (Appendix I.2).


Q2: Theorem 5 establishes Type-I error control under H0:rank⁡=rH_0:\operatorname{rank}=r. Although sequential testing addresses this issue in practice, a complete Type-I error guarantee would be preferable.

A2: Thank you for raising this important theoretical point. We agree that extending the Type-I error guarantee to the full composite null H0:rank⁡≤rH_0:\operatorname{rank}\leq r would strengthen our theoretical results. We note, however, that classical CCA-based rank tests face a similar theoretical limitation, as their standard asymptotic null distributions are also derived under the regular case where the true population rank is exactly rr. Nevertheless, this limitation has limited practical implications for the following reasons.

  1. Practical applicability through sequential testing. You are totally right. In practice, most rank-based causal discovery methods determine the rank sequentially, testing r=0,1,2,…r=0,1,2,\ldots until the first non-rejection. Therefore, the lack of a complete composite-null guarantee has limited practical impact on sequential rank estimation.

  2. Empirical Type-I error control under the composite null. We have explicitly investigated this issue in Section 4.1.1 by considering cases where the true population rank is strictly smaller than the hypothesized rank bound. Our experiments show that both MART and the classical CCA-based rank test effectively control Type-I error at α=0.05\alpha=0.05 across the evaluated settings. These results provide empirical evidence that MART remains well calibrated even beyond the regular case.

We agree that establishing a guarantee over the entire composite null is an important theoretical direction. We believe such an extension is feasible for both classical CCA-based rank tests and MART. However, it is nontrivial because lower-rank points introduce singularities in the geometry of the rank-constrained set, potentially requiring a different asymptotic analysis. We therefore leave this broader theoretical investigation to future work. We have added this discussion to Appendix I.5 in our revised manuscript.


Thank you again for your valuable comments. We have uploaded our revised manuscript, with all changes highlighted in blue for ease of reference. Please feel free to let us know if you have any further questions or comments.


[1] Tu, Ruibo, et al. "Causal discovery in the presence of missing data." The 22nd International Conference on Artificial Intelligence and Statistics. Pmlr, 2019.

Reviewer_7vNFOpenReview ↗

Thanks for the rebuttal

Thanks for the detailed rebuttal. My concerns about missing pattern and type I error have been adequately addressed.

Reviewer_XUUzOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2(+1)
Soundness
2
Read review
Paper summary

This is an emergency review completed on short notice. I am reading as an informed outsider rather than as a domain expert, so my assessment focuses on clarity, soundness of the core argument, and the strength of the evidence as presented. I have not independently verified proofs, re-run experiments, or checked the related work in depth, and the area chair should weight this review accordingly, particularly on questions of novelty relative to prior work.

Summary

This paper proposes MART, a novel covariance-rank test that is applicable when missing values are present completely at random (MCAR). Interestingly, despite this test is based on pairwise covariance estimates, the authors provide a thorough theory with asymptotic guarantees. Supporting experiments are provided.

Critical strengths
  • [S1: Practical Usefulness] The problem being tackled is useful, as most real-world scenarios have some form of missing values, and using each sample is often strictly necessary.
  • [S2: Supporting theory] The authors provide a theoretical analysis of MART, including asymptotic guarantees. As an emergency reviewer, I couldn't check all the nuances and proofs in detail in a short time, but the ones I checked seemed correct.
Critical weaknesses
  • [W1: Presentation] Although the introduction and abstract are clear, the writing quality significantly degrades in the remaining part of the manuscript. The major issue with this paper is the presentation and the order in which the results (theoretical and non) are presented, which is quite unorganized and makes it challenging to parse. A few examples:
    • In section 2.3, for example, the sampling uncertainty is stated to be approximately V/N, although V is only defined later in the paper. Further, the proof that it is approximately V/N appears even later in Sec.3..
    • Many results or assumptions appear without a proper explanation. One example is Assumption 2.
    • Section 2.2: Please specify what L and E are.
  • [W2: Experiments and Reproducibility] Although some information for reproducibility is provided, other details are still missing. For example the paper mentions multiple times about stabilizing numerically the Kronecker factors, but no further information is provided on how this is done. Further, no code is available at the current time.

Other typos:

  • Line 170, "generally".
  • Line 178: "are".
Questions for authors
  • Sec. 4.1.1, we have that positive definiteness implies (rank(ΣA,B)≥∣C∣rank(\Sigma_{A,B})\ge |C|), since A and B share C and the tested block therefore contains the nonsingular submatrix (ΣC,C\Sigma_{C,C}). However, Section 4.1.1 states that population ranks are sampled uniformly from (r,r−1,r−2{r,r-1,r-2}), with (r=∣C∣+kr=|C|+k), which includes ranks below this bound when (k=0) or (k=1). Could the authors verify and clarify which rank combinations were actually used, what was run, and how those infeasible cases were handled?
5 comments

Thank you for your insightful comments and valuable feedback. We appreciate your positive assessment of the paper's practical relevance and theoretical contribution, particularly your recognition of the importance of effectively utilizing incomplete observations, as well as the thorough theory with asymptotic guarantees of our proposed MART method. We address your concerns and suggestions in detail below.


Q1: Suggestions for Clarity.

A1:

1. The approximation V/NV/N for the sampling uncertainty is introduced before VV is formally defined or the approximation is justified, making the presentation difficult to follow.

Thank you for highlighting this presentation issue. Section 2.3 is intended to motivate the method before introducing the formal theory. There, we identify VV as the asymptotic covariance of the scaled estimation error and give its entrywise expression in Equation (5). Presenting this expression early highlights the central difficulty: the co-observation factor (qij,kℓ/(qijqkℓ)(q_{ij,k\ell}/(q_{ij}q_{k\ell}) varies across pairs of covariance entries, so a single effective sample size generally cannot capture their joint sampling uncertainty.

We agree that this progression should be signposted more clearly. We have revised Section 2.3 to explain that Equation (5) is introduced for motivation, with its derivation provided in Appendix D.1 and the corresponding joint asymptotic normality result established in Theorem 2 of Section 3.1.

2. Explanation of Assumption 2

Thank you for pointing out that Assumption 2 requires further explanation. The finite-fourth-moment assumption, E∣X∣4<∞\mathbb{E}|\mathbf{X}|^4<\infty, ensures that the influence functions of the pairwise covariance estimators have finite second moments, thereby supporting their joint central limit theorem and consistent estimation of their sampling uncertainty. These results form the theoretical foundation for MART's weighted chi-square calibration.

This assumption accommodates both Gaussian and non-Gaussian distributions, including Laplace and Student-tνt_\nu distributions with ν>4\nu>4. For 2<ν≤42<\nu\leq4, the population covariance exists, but the fourth moment is infinite, so our theoretical guarantees do not apply. For ν≤2\nu\leq2, the population covariance is not finite, making covariance-rank testing ill-defined.

To further investigate MART's robustness beyond this assumption, we have conducted additional experiments with Student-t3t_3 noise in Section 4.1.4. Despite the infinite population fourth moment, MART maintains Type-I error rates near the nominal level across all tested sample sizes and achieves higher statistical power than test-wise deletion, pairwise minimum and average corrections, and mean imputation. These results provide empirical evidence of MART's robustness under this form of model misspecification.

Thank you again for this helpful suggestion and have added a detailed explanation of Assumption 2 to the revised manuscript (Appendix I.4).

3. Section 2.2: Please specify what L and E are.

Here, L\mathsf{L} denotes a latent variable, and Ei\mathsf{E}_i denotes mutually independent noise terms. We have revised Section 2.2 to clarify these definitions.


Q2: Details about stabilizing Kronecker factors?

A2: Thank you for pointing out that the stabilization procedure was insufficiently specified. Stabilization is necessary to ensure that the fitted Kronecker factors admit numerically stable inverse square roots. Specifically, nonpositive eigenvalues make the inverse square roots undefined, while very small eigenvalues may amplify numerical errors.

For each fitted factor KK, we first symmetrize it, by $ K_s=\frac{K+K^\top}{2}, $ which preserves its asymptotic limit under the assumed convergence conditions. We then apply a diagonal shift:

\qquad \delta = \max\\{\epsilon-\lambda_{\min}(K_s),\epsilon\\}.$$ where $\epsilon=10^{-6}$. This procedure ensures that all eigenvalues of the stabilized factors are bounded below by $\epsilon$. As an additional numerical safeguard, we also floor eigenvalues at $\epsilon$ when computing the inverse square roots. Thank you again for your feedback. We have updated Appendix C of our manuscript to include these implementation details. ---

Q3: Other implementation details and code availability?

A3: Additional implementation details are provided in Appendix I. We will publicly release our code upon acceptance.


Q4: Which rank combinations were actually tested, and how were lower than ∣C∣|C| cases handled?

A4: Thank you for raising this question. Our experimental design considered nine combinations of the hypothesized rank bound and population rank, as summarized below.

kkNull hypothesisPopulation rank rrPopulation rank r−1r-1Population rank r−2r-2
00rank⁡≤r=∣C∣\operatorname{rank}\leq r=\lvert C\rvert∣C∣\lvert C\rvert∣C∣−1\lvert C\rvert-1∣C∣−2\lvert C\rvert-2
11rank⁡≤r=∣C∣+1\operatorname{rank}\leq r=\lvert C\rvert+1∣C∣+1\lvert C\rvert+1∣C∣\lvert C\rvert∣C∣−1\lvert C\rvert-1
22rank⁡≤r=∣C∣+2\operatorname{rank}\leq r=\lvert C\rvert+2∣C∣+2\lvert C\rvert+2∣C∣+1\lvert C\rvert+1∣C∣\lvert C\rvert

Regarding feasibility, we agree that rank⁡(ΣA∪C,B∪C)≥∣C∣\operatorname{rank}({\Sigma}_{A\cup C,B\cup C})\geq |C| whenever ΣC,C{\Sigma}_{C,C} is positive definite. For the cases in our table where the target population rank is below ∣C∣|C|, we instead use linearly dependent variables in CC, so that ΣC,C{\Sigma}_{C,C} is singular.

Specifically, for a target population rank q<∣C∣q<|C|, let H∼N(0,Iq)H\sim N(0,I_q) and construct

A=M_AH+\varepsilon_A,\quad B=M_BH+\varepsilon_B,$$ where $L\in\mathbb{R}^{|C|\times q}$ has full column rank, and $\varepsilon_A$, $\varepsilon_B$, and $H$ are mutually independent. The noise vectors have zero means and positive-definite covariance matrices. Consequently, the population rank $\operatorname{rank}({\Sigma}_{A\cup C,B\cup C})$ is exactly $q$. Thus, although $C$ contains $|C|$ coordinates, it spans only $q$ independent directions, allowing the population cross-covariance rank to be smaller than $|C|$. At the same time, **specifying a rank-test null hypothesis with a bound below $|C|$ is always mathematically allowed.** For the case that the true rank is smaller than $|C|$, no special treatment is required to perform our MART. **Our motivation for considering population ranks strictly below the hypothesized rank bound** was to investigate Type-I error control in the interior of the composite null, rather than only at its boundary. This is particularly relevant because the asymptotic calibration guarantees of classical CCA-based rank tests and MART are established at the regular boundary case where the true population rank equals the hypothesized rank. We appreciate your helpful comment, which has improved the clarity of our paper. To address the concern, we have added the table summarizing all tested rank combinations, along with an explanation, to Section I.3 of our revised manuscript. --- **Thank you again for your valuable comments. Regarding other typos, we have revised our paper accordingly. We have uploaded our revised manuscript, with all changes highlighted in blue for ease of reference. Please feel free to let us know if you have any further questions or comments.**

Dear Reviewer XUUz,

Thank you again for your constructive feedback. We have carefully addressed your questions in our rebuttal and revised the manuscript to include additional clarifications and experimental results. We hope these updates will be helpful in your final assessment.

As the discussion period draws to a close, we would be grateful if you could review our responses and let us know whether they address your concerns. We would be happy to clarify any remaining points.

Thank you again for your time and thoughtful consideration.

Reviewer_y9YwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies covariance-rank testing in the presence of missing data and proposes the Missing-value uncertainty-Aware Rank Test (MART). The method consistently estimates the joint uncertainty of pairwise covariance estimates and computes their weighted distance to the rank-constrained set. A Kronecker approximation allows this distance to be computed through a whitened singular value decomposition. Meanwhile, the full joint uncertainty estimate is used to construct a weighted chi-square mixture as the asymptotic null distribution, thereby calibrating the test statistic. The paper demonstrates the effectiveness of the proposed method on synthetic and real-world data.

Critical strengths
  1. Different variable pairs use different, overlapping subsets of observations to estimate covariances, and their joint uncertainty generally cannot be represented by a single effective sample size. The paper provides a clear explanation of this issue. The problem is well defined and practically relevant.
  2. The Kronecker approximation is used to obtain a computationally tractable low-rank distance, while calibration still uses the full covariance estimate. This avoids treating the computational approximation as the true sampling covariance.
  3. The effectiveness of the proposed method is demonstrated on synthetic and real-world data.
Critical weaknesses
  1. The paper does not sufficiently evaluate the method’s performance in higher-dimensional settings. Could the authors add experiments with 50 and 70 nodes?
  2. The paper considers only the MCAR setting. Could the authors include additional discussion and experiments under MAR and MNAR [1] mechanisms? This would help clarify the method’s scope of applicability.
  3. The current synthetic experiments mainly use linear structural equation models. Could the authors add experiments involving nonlinear causal mechanisms?

[1] Tu, R., Zhang, C., Ackermann, P., Mohan, K., Kjellström, H., & Zhang, K. (2019, April). Causal discovery in the presence of missing data. In The 22nd International Conference on Artificial Intelligence and Statistics (pp. 1762-1770). Pmlr.

Questions for authors

See Weakness

3 comments

Thank you for your insightful comments and valuable feedback. We appreciate your positive assessment of the paper's practical relevance, particularly your recognition of the importance of accounting for joint sampling uncertainty in pairwise covariance estimation, the computational efficiency and statistical validity of our proposed method, and its effectiveness on synthetic and real-world data. We address your concerns and suggestions in detail below.


Q1: Experiments with 50 and 70 nodes?

A1: Thank you for your suggestion. Our original experiments focused on 30 variables to enable comparisons with test-wise deletion, as this baseline typically leaves no usable observations with 50 variables at a missing rate of 0.1. Following your suggestion, we have conducted additional experiments with 50 and 70 variables, with results presented in the newly added Section 4.1.5. For example, with 50 variables, MART effectively controls the Type-I error at the nominal level of α=0.05\alpha=0.05 across all sample sizes (N=500,1000,2000,5000N=500, 1000, 2000, 5000). Moreover, MART demonstrates strong statistical power, with calibrated power increasing from 0.55 to 0.98 as the sample size grows from 2,000 to 5,000, whereas the pairwise-average baseline achieves only around 0.80 at N=5000N=5000. These additional results further demonstrate the effectiveness of MART in higher-dimensional settings.


Q2: Scope of the method and applicability under MAR and MNAR?

A2:

1. Scope of MART and its relationship to MVPC [1]

We would like to clarify that MART is a statistical rank test designed to improve upon classical test-wise deletion, whereas MVPC is a causal discovery framework that addresses more general missingness mechanisms.

Specifically, conditional independence testing via partial correlation is a special case of covariance-rank testing. Compared with test-wise deletion, MART utilizes additional incomplete observations and achieves strictly higher asymptotic local power under the setting of Theorem 6.

Therefore, MART and MVPC address different but complementary aspects of causal discovery with missing data: MART improves the underlying statistical test, while MVPC incorporates graphical assumptions and missingness correction to handle certain MAR and MNAR mechanisms.

2. Why MCAR is straightforward, while MAR and MNAR require additional assumptions

Under MCAR, both test-wise deletion and MART are valid because their covariance estimators consistently target the corresponding population covariances.

For example, suppose YY contains missing values, with missingness indicator RYR_Y. Under MCAR, RYR_Y is independent of the data, implying

Cov⁡(X,Y∣Z,RY)=Cov⁡(X,Y∣Z).\operatorname{Cov}(X,Y\mid Z, R_Y) = \operatorname{Cov}(X,Y\mid Z).

Thus, covariance estimates based on observed cases remain consistent for the population target.

However, under MAR or MNAR, this equality does not generally hold. Consider the following example adapted from Figure 1 of MVPC [1]:

X & \longrightarrow & Z & \longrightarrow & Y \\\\ & \searrow & & \swarrow & \\\\ & & W & & \\\\ & & \downarrow & & \\\\ & & R_Y & & \end{array}$$ Here, $W$ is a collider on the path $X\to W\leftarrow Y$ and a parent of $R_Y$. Conditioning on $R_Y$ can induce dependence through this collider, even after conditioning on $Z$. Consequently,

\operatorname{Cov}(X,Y\mid Z, R_Y) \neq \operatorname{Cov}(X,Y\mid Z)

ingeneral.Therefore,directlyapplyingeithertest−wisedeletionorMARTmayleadtobiasedcovarianceestimation. in general. Therefore, directly applying either test-wise deletion or MART may lead to biased covariance estimation.

3. How MVPC addresses MAR and MNAR

Importantly, this limitation can be addressed by incorporating graphical assumptions and identifying the parents of missingness indicators, as demonstrated by MVPC [1].

Specifically, after identifying WW as the parent of RYR_Y, the population distribution can be recovered through

p(X,Y,Z)=∫p(X,Y,Z∣W)p(W),dW=∫p(X,Y,Z∣W,RY)p(W)dW,p(X,Y,Z) =\int p(X,Y,Z\mid W)p(W),dW =\int p(X,Y,Z\mid W,R_Y)p(W)dW,

where the second equality follows from the conditional independence of (X,Y,Z)(X,Y,Z) and RYR_Y given WW, as WW is the parent of RYR_Y. This allows consistent recovery of the target population covariances and the corresponding conditional independence relations.

However, MVPC still relies on test-wise deletion for its underlying conditional independence tests, which can substantially reduce the available sample size, especially when the conditioning set ZZ is high-dimensional.

4. Extending our method MART to MAR and MNAR

A natural question is whether MART can be extended to MAR and MNAR by incorporating the strategy of MVPC. The answer is yes.

Once the relevant parents of missingness indicators are identified, the population distribution can be recovered using the same graphical correction strategy as MVPC, as p(X,Y,Z)=∫p(X,Y,Z∣W,RY)p(W)dWp(X,Y,Z)=\int p(X,Y,Z\mid W,R_Y)p(W)dW. The key difference is that now MART serves as the underlying statistical test, so we can make use of all available rows even they are incomplete for X,Y,ZX,Y,Z. Such an integration would combine MVPC's strategy to address broader missingness mechanisms with MART's improved utilization of incomplete data, offering particular advantages in higher-dimensional settings.

We thank the reviewer for highlighting this important direction. While the extension is feasible under appropriate identification assumptions, establishing rigorous theoretical guarantees would require extending MART's uncertainty-aware calibration to account for the additional uncertainty introduced by missingness correction. We view the theoretical development and empirical evaluation of this integrated framework as a promising direction for independent future work, beyond the scope of the present paper.

5. Summary of the comparison

Statistical test|Causal discovery frameworks
Test-wise deletionMART (Ours)|MVPCFuture work
Data utilizationOnly test-wise complete rowsAll available rows|Underlying testTest-wise deletionMART
MCARYesYes|MCARYesYes
MARNoNo|MARYes, under graphical assumptionsYes, by incorporating the strategy in MVPC
MNARNoNo|MNARYes, under graphical assumptionsYes, by incorporating the strategy in MVPC

Thank you again for the insightful question. We have also revised the manuscript to incorporate these discussions and clarify the scope of MART and its potential integration with existing missingness-aware causal discovery frameworks (Appendix I.2).


Q3: Experiments with nonlinear causal mechanism?

A3: Thank you for raising this point. Our original experiments followed previous studies on causal discovery with missing data [1] and covariance-rank testing [2], using linear causal models for data generation. We agree that evaluating MART under nonlinear causal mechanisms would provide valuable additional insights. Following your suggestion, we conducted new experiments in which each variable is generated by applying a nonlinear transformation to a linear combination of its parents and a noise term. Specifically, we consider two nonlinear functions: leaky ReLU and hyperbolic tangent (tanh). The results are presented in the newly added Section 4.1.6. Under both nonlinear mechanisms, MART consistently controls the Type-I error near the nominal level of α=0.05\alpha=0.05 across all four sample sizes. Among the compared methods, only MART and the complete-data test adequately control Type-I error, while the other baselines fail to do so. These additional results further demonstrate the robustness of MART under nonlinear causal mechanisms, highlighting its applicability beyond linear causal models.


Thank you again for your valuable comments. We have uploaded our revised manuscript, with all changes highlighted in blue for ease of reference. Please feel free to let us know if you have any further questions or comments.


[1] Tu, Ruibo, et al. "Causal discovery in the presence of missing data." The 22nd International Conference on Artificial Intelligence and Statistics. Pmlr, 2019.

[2] Dong, Xinshuai, et al. "Permutation-based rank test in the presence of discretization and application in causal discovery with mixed data." International Conference on Machine Learning. 2025.

Dear Reviewer y9Yw,

Thank you again for your constructive feedback. We have carefully addressed your questions in our rebuttal and revised the manuscript to include additional clarifications and experimental results. We hope these updates will be helpful in your final assessment.

As the discussion period draws to a close, we would be grateful if you could review our responses and let us know whether they address your concerns. We would be happy to clarify any remaining points.

Thank you again for your time and thoughtful consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.