- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper proposes a statistical test for the rank of a covariance matrix when observations of some variables are missing within a sample. A typical naive approach would estimate each covariance entry from the rows where two variables are observed and plugs these estimates into a classical rank test. However, each entry then comes from a different, overlapping subset of rows, so a single effective sample size cannot capture their joint uncertainty. The proposed method in this work addresses this by estimating the joint sampling uncertainty of the pairwise estimates and using it to calibrate a rank-test statistic. The method is evaluated on synthetic and real-world data with an application to causal discovery.
Critical strengths
The paper clearly describes the issue and fairly discusses why 'naive' approaches (e.g., imputation or pairwise estimation with single effect sample size) do not ensure proper type-1 error control. The authors provide theoretical guarantees for their approach and show promising experimental results.
Critical weaknesses
Majority of the experiments use synthetic data and even the real-world experiment requires artificially masking observations. The O(M^4) memory requirement for caching the full joint covariance estimate seems quite large and not suitable for larger variable sets. The work by Dong et al. (2025) also uses a covariance rank test for causal discovery and, while the paper mentions it in the related work, a direct comparison in the experiments is missing.
Questions for authors
The work addresses a fairly common issue in practice and, overall, this is promising work. A few clarifying questions:
- What happens in practice when the matrices are singular? The paper mentions a singular uncertainty matrix is not an issue but the exact stabilization rule is not given as far as I see. How should one choose it and how sensitive are the results with respect to it?
- You mention the paper by Dong et al. (2025) "Permutation-Based Rank Test in the Presence of Discretization and Application in Causal Discovery with Mixed Data", which also uses a covariance rank test for causal discovery. It targets discretized rather than missing data, but real data often have both problems. Could the two approaches be combined to handle data that are both discretized and incomplete? A short discussion, or a small experiment if feasible, would be really insightful.
- It is not entirely clear why the pairwise baselines (pairwise covariances in the classical rank test with a single effective sample size) have Type-I error rates so far above 5%. You mention the uncertainty mismatch, but the reported numbers do not seem to fully explain it. Is there a clearer explanation? I might be missing some details here.
3 comments
Q3: How are the Kronecker factors stabilized?
A3: Thank you for pointing out that the stabilization procedure was insufficiently specified. Stabilization is necessary to ensure that the fitted Kronecker factors admit numerically stable inverse square roots. Specifically, nonpositive eigenvalues make the inverse square roots undefined, while very small eigenvalues may amplify numerical errors.
For each fitted factor , we first symmetrize it as which preserves its asymptotic limit under the assumed convergence conditions. We then apply a diagonal shift:
\qquad \delta=\max(\epsilon-\lambda_{\min}(K_s),\epsilon),$$ where $\epsilon=10^{-6}$. This procedure ensures that all eigenvalues of the stabilized factors are bounded below by $\epsilon$. As an additional numerical safeguard, we also floor eigenvalues at $\epsilon$ when computing the inverse square roots. We further conducted a sensitivity analysis with $\epsilon$ ranging from $10^{-8}$ to $10^{-5}$ and observed nearly identical test results, demonstrating that MART is robust to the choice of this stabilization parameter within the evaluated range. We thank the reviewer again for this helpful comment and have updated Appendix C of the revised manuscript to include these implementation details. --- **Q4: Why do the pairwise baselines (pairwise covariance estimates plugged into a classical rank test with a single effective sample size) exhibit Type-I error rates substantially above 5\%?** **A4:** Thank you for this insightful question. **The key reason is the mismatch between the actual asymptotic null distribution and the assumed chi-square reference distribution.** We provide a four-variable example below with minimum effective sample size to illustrate how this mismatch can lead to substantial Type-I error inflation. **Example setup.** Let $Z\sim N(0,1)$, $X_i=a_iZ+\varepsilon_i$ for $i=1,2$, and $Y=Z+\varepsilon_Y$, where $a_1=1$, $a_2=1/2$, and the noise terms are mutually independent $N(0,v)$ variables with $v=0.1$, independent of $Z$. Then $(X_1,X_2)\perp Y\mid Z$, and the covariance block between $A=(X_1,X_2,Z)$ and $B=(Y,Z)$ has rank exactly one. We keep $Z$ fully observed and independently observe each remaining variable with probability $s=0.9$ (i.e., 0.1 missing rate), independently of the data. Under this MCAR mechanism, the pairwise covariance estimates remain consistent, and the minimum pair count satisfies $N_{\min}/N\to s^2$. **Actual asymptotic null distribution.** Define the estimated partial covariances $\widehat g_i=\widehat\sigma_{X_iY}-\widehat\sigma_{X_iZ}\widehat\sigma_{ZY}/\widehat\sigma_{ZZ}$, whose population values are zero. Applying the delta method to the joint asymptotic distribution of the pairwise covariance estimates yields $$\frac{\sqrt{N_{\min}}}{v}\widehat{\boldsymbol g} \xrightarrow{d} N\left(0,\operatorname{diag}(\kappa_1,\kappa_2)\right),$$ where $$\kappa_i= 1+\frac{(1-s)(1+a_i^2)}{v} +\frac{2a_i^2(1-s)^2}{v^2}.$$ Substituting the specified parameters gives $\kappa_1=5$ and $\kappa_2=2.75$. The additional variance terms arise because different covariance entries are estimated from different, overlapping subsets of observations, disrupting the cancellation of estimation errors that occurs with complete data. Consequently, the classical plug-in statistic satisfies $$T_{\mathrm{pairwise}} =\frac{N_{\min}}{v^2} (\widehat g_1^2+\widehat g_2^2)+o_p(1),$$ and **its actual asymptotic null distribution is** $$T_{\mathrm{pairwise}} \xrightarrow{d} 5G_1^2+2.75G_2^2, \qquad G_1,G_2\overset{\mathrm{iid}}{\sim}N(0,1),$$ **rather than the conventional $\chi_2^2$ distribution.** **Type-I error inflation.** Using the nominal 5\% cutoff of $5.99$ from the $\chi_2^2$ distribution yields an asymptotic rejection probability of approximately 45.2\%, whereas the correct weighted chi-square mixture has a 5\% upper-tail cutoff of approximately 23.74. This example demonstrates that **even though the pairwise covariance estimates are consistent, using a single effective sample size can severely miscalibrate the null distribution and inflate Type-I error.** As the sample size increases, the rejection rate under mismatch can further increases, which is aligned with the substantial Type-I error inflation observed in Section 4.1.1. --- **Thank you again for your valuable comments. We have uploaded our revised manuscript, with all changes highlighted in blue for ease of reference. Please feel free to let us know if you have any further questions or comments.**I want to thank the authors for their clarifications and addressing my questions (in particular on Q2), I keep my recommendation for acceptance.
Thank you for your insightful comments and valuable feedback. We appreciate your positive assessment of the paper's clear motivation, practical relevance, and theoretical contributions, particularly your recognition of the importance of accounting for joint sampling uncertainty in pairwise covariance estimation and our theoretical guarantees for valid statistical testing. We address your concerns and suggestions in detail below.
Q1: The O(M4) memory for caching seems quite large?
A1: Thank you for raising this concern. Although the full cache has a storage complexity of O(M4), its absolute memory requirement remains manageable for moderately sized problems. Specifically, we cache the joint uncertainty matrix of the k=M(M+1)/2 unique covariance entries, requiring 8k2 bytes with float64 precision:
As shown above, full caching remains feasible for moderate dimensions, although memory requirements become substantial as M increases.
Importantly, this storage cost should be considered alongside the computational demands of causal discovery. Constraint-based causal discovery algorithms can have exponential worst-case search complexity, making the search procedure the major bottleneck in larger problems. In this context, caching avoids repeated covariance and uncertainty estimation across rank-test queries, improving computational efficiency at the expense of memory.
We thank the reviewer again for highlighting this concern and have added a corresponding discussion to the revised manuscript (Appendix I.6).
Q2: Comparison with MPRT (Dong et al., 2025) and the possibility of combining the two methods to handle discretized and incomplete data?
A2: Thank you for this insightful question. MPRT addresses rank test with discretized observations, assuming complete data, whereas MART focuses on rank test with missing values. Since the two methods address different observation mechanisms, a direct experimental comparison under our current setting would not be appropriate. However, combining the two methods to handle data that are both discretized and incomplete is feasible. We outline two possible strategies below.
(i) Available-pair estimation with permutation calibration. We estimate each polychoric correlation using all observations available for the corresponding variable pair. Under MCAR, positive observation probabilities, and a correctly specified latent Gaussian threshold model, these estimators are consistent under standard identifiability and regularity conditions. We then compute the CCA projections from the estimated correlation matrix and follow Equation (8) of Dong et al. (2025): after each permutation, we re-estimate the cross-block correlations while keeping the original CCA projections and within-block normalizations fixed to construct a permutation reference distribution for the test statistic.
(ii) Available-pair estimation with MART-style asymptotic calibration. Alternatively, MART's sandwich-based uncertainty estimation can be extended to polychoric correlations estimated from incomplete observations. Specifically, the pairwise likelihood score equations, together with the estimating equations for ordinal thresholds, provide a basis for estimating their joint asymptotic covariance. This would account for both threshold-estimation uncertainty and dependence induced by overlapping subsets of observed data. The resulting covariance estimate could then support a weighted chi-square calibration of the rank statistic. While theoretically motivated and methodologically feasible, this approach requires additional derivations and regularity conditions.
To demonstrate empirical feasibility, we conducted new experiments following (i). Specifically, we set ∣A0∣=∣B0∣=∣C∣=5 and generate latent Gaussian observations and discretize each variable into three ordered categories. Missingness is then introduced independently of the data, with variable-specific missingness probabilities drawn from Unif(0.09,0.11). We test H0:rank(ΣA,B)≤6 for the underlying continuous covariance matrix, with true population ranks of 6 and 7 for evaluating Type-I error and power, respectively.
The results are shown in the newly added Table 6. The combined method effectively controls Type-I error near α=0.05, while statistical power increases from 16.0% to 99.7% as the sample size grows from 500 to 5000. These results provide empirical evidence of the feasibility and effectiveness of combining the two methods for covariance-rank testing with both discretized and missing observations.
We thank the reviewer again for this valuable suggestion. We have incorporated the additional experiments in Table 6 and expanded the related discussion in Appendix I.1.