- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper investigate and contribute to conditional independence testing with dependent time-series observations, with applications to causal discovery. The authors propose PC-OP, a modification of PCMCI algorithm that conditions on the parents of the candidate effect rather than both variables. They establish asymptotic validity of the chi-square test under specific assumptions and evaluate the resulting algorithm on synthetic and real-world data.
Critical strengths
The paper targets a relevant statistical problem by proposing a simple modification to an established algorithm. The theoretical analysis is extensive, and the synthetic experiments report substantial improvements over PCMCI. The oracle-parent ablation is useful for isolating the effect of the conditioning strategy.
Critical weaknesses
W1: empirical results do not sufficiently support the proposed motivation. The method is motivated by improved statistical power through smaller conditioning sets, yet Tables 2–3 show nearly identical power for the two strategies. In contrast, the downstream synthetic experiments show large improvements, without sufficiently explaining when or why these arise. The improvements on CausalRivers are relatively modest, making it difficult to assess the practical significance of the method.
W2: scope and applicability of the theoretical results. The analysis focuses on a specific class of stationary, rapidly mixing processes with independent innovations, rather than general dependent observations. Furthermore, the main theorem assumes a fixed parental conditioning set, whereas the proposed algorithm estimates this set from the same data. The authors acknowledge that calibration after selection is not established. It remains unclear how much of the theoretical analysis carries over to the implemented procedure.
W3: problem formulation and presentation could be clearer. The assumptions underlying the causal model are introduced across different sections, without clearly distinguishing those required for the statistical test from those needed for causal discovery. The extensive theoretical development also makes it difficult to identify the central new insights and their relationship to the empirical findings.
Questions for authors
Q1: What explains the substantial downstream improvements when the isolated CI tests show nearly identical power? Can the authors better characterize the settings in which PC-OP is expected to outperform PCMCI?
Q2: How sensitive is the method to violations of the structural assumptions and errors in the estimated parent sets? To what extent do the theoretical guarantees support the practical algorithm?
Q3: Can the authors clarify the novelty of the theoretical results relative to existing martingale-based inference, and distinguish the main new arguments from standard tools?
Q4: In Section 4, why does PC-OP+ use a threshold of 1/M, rather than \alpha/M, for its Bonferroni-style correction?
2 comments
The additional experiments and analysis substantially address W1, clarifying when and why the proposed approach improves over PCMCI. I also appreciate the improved exposition and clarification of the contributions (W3).
My main remaining concern is W2: the theoretical guarantees still do not cover the data-dependent parent selection used by the practical algorithm. Together with the relatively modest real-world improvements, this leaves me inclined to maintain my original assessment.
We would like to thank the reviewer 5bZC for their time and detailed, constructive feedback. Please find our replies to your questions below. We also updated the manuscript to address comments from all reviewers.
Q1. Power of the test.
Tables 2 and 3 focus on a single pairwise test conditioned on one variable, generally demonstrating equal or superior performance for our methods when the sample size is n<1000 or the effect size is small (β≤0.1). When applied to synthetic graph discovery problems, however, both the number of variables and the size of the conditioning sets can be substantially larger. The table below evaluates the performance of PC-OP variants (PC-OP, PC-OP+, PC-OP-M) against PCMCI across different graph dimensions (d) and maximum in-degrees, reporting the F1 score, mean conditioning set size (mean ∣Z∣), d-separation accuracy, and final test power.
For this evaluation, we use primary datasets (5 seeds each) executed with our updated implementation (incorporating lag fixes and a Step 2 threshold of α=0.05). To establish ground truth, we generate an independent reference dataset of 107 samples simulated directly from the true conditional probability tables (CPTs) and compute conditional mutual information (CMI) and chi-square tests as our reference benchmark. Verification on a d=5, L=1 system confirms that this reference dataset matches the exact stationary Markov chain to within 0.0002 on the marginals and 0.0001 on the mutual information of a true edge. A test is counted as correct if a "dependent" decision aligns with CMI >ϵ, and test power/accuracy is measured by whether decisions match the most likely outcome of a chi-square test at particular sample and alpha values, derived from the true distribution.
As shown in the table below, PC-OP and its variants consistently achieve high performance—maintaining roughly 94% to 96% power across settings and superior F1 scores—while PCMCI's power drops significantly in denser graphs (ranging from 36.9% to 81.1% depending on the setting) and tends to select much larger conditioning sets (mean ∣Z∣). This showcases the effectiveness and power gain of the methods over PCMCI.
Q2. Wrong parent set.
We don't have any formal guarantees when even the parents of the target variable is false. Note that a conservative estimate, a superset of parents of Y, does not affect soundness, and the PC stage is purposefully using a loose alpha value precisely to deal with this in practice.
Q3. Novelty.
As far as we know, this is the first rigorous result for independence testing under a causal time series assumption. While what translates to practice is not settled, we feel theory is important here. In regard to technical novelty: Our proof combines several insights we came up with for the first time: The shortfall concept, viewing the data generative process as sampling rows first, then columns randomly, carefully positioning and handling the conditional sampling process as a martingale, and so on. The existing martingale-based inference results cannot be directly applied in this setting.
Q4. PC-OP+ Threshold
1/M was a typo and has now been corrected. Algorithm 1 showed the correct computation.
We hope that our rebuttal addressed the reviewer's concerns. Thank you again for your time.