- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
COSMIC accelerates causal ordering and parent screening through Nyström approximation and grouped removal of variables, followed by regression pruning to recover the final graph. The paper presents computational analysis, component ablations, and diagnostics of approximation and ordering errors.
Critical strengths
- Nyström approximation and grouped removal address distinct computational costs, and the ablations demonstrate their complementary benefits.
- The paper clearly documents the scope of its theoretical results and the limitations introduced by approximation and downstream pruning.
Critical weaknesses
- The comparison omits SciNO [1], which directly addresses the same Hessian estimation bottleneck and can replace the estimator used in CaPS. Comparisons with SciNO and representative alternatives such as DrBO [2] and observational SEA [3] are needed to establish COSMIC’s contribution relative to current approaches to efficient causal discovery.
- The experiments do not systematically test denser graphs such as SynER5 or vary graph depth and the proportion of linear mechanisms. These conditions are relevant to the reliability and computational benefit of grouped removal, while the existing noise and hyperparameter diagnostics provide limited evidence about complete graph recovery across such changes.
- Theorem 5.1 does not establish that the heuristic selects valid terminal blocks or orders connected variables correctly within each block. Moreover, assigning zero scores to relations within a group can exclude true edges during screening when other candidate parents have sufficiently positive scores, creating an accuracy risk that is not adequately characterized across graph structures.
- The largest experiments demonstrate scalability of ordering and screening, while GAM pruning restricts complete graph recovery to substantially smaller problems. The paper does not clearly establish the conditions under which the computational savings justify the observed costs in recovery accuracy and provide a compelling overall advantage.
- Sachs is the only dataset based on real measurements, while the fMRI benchmark is simulated. This leaves limited evidence that the proposed scaling strategy produces useful causal graphs on larger and more varied real datasets.
Reference
[1] Kang et al., "Score-informed Neural Operator for Enhancing Ordering-based Causal Discovery," NeurIPS, 2025.
[2] Duong et al., "Causal discovery via Bayesian optimization," ICLR, 2025.
[3] Wu et al., "Sample, estimate, aggregate: A recipe for causal discovery foundation models," TMLR, 2025.
Questions for authors
- How does COSMIC compare with SciNO and representative recent alternatives under matched observational settings, considering total computation and final graph accuracy, with pretraining costs reported separately where applicable?
- How do graph density, depth, and the number of available sinks affect the accuracy and computational benefit of grouped removal?
2 comments
Thank you for the detailed response and clarifications. After carefully reading the rebuttal, my overall assessment remains unchanged, so I will maintain my current score.
Response to Reviewer SCEk
We thank Reviewer SCEk for the constructive review and for highlighting the value of our Nyström and grouped-removal ablations. Because end-to-end pruning scalability and heuristic macro-tier batching were raised across reviews, we address both in the general comment above and focus here on the rest.
1. SciNO, DrBO, and SEA
Matched empirical comparisons with SciNO, DrBO, and observational SEA are not yet available for this response. We are planning five-seed comparisons; here, we clarify two distinctions regarding training paradigms and scaling axes:
2. Graph Density, Depth, Sink Availability, and Co-Batching
We ran a matched study at d=50,N=1,000 (3 seeds, 50% nominal linear mechanisms) comparing grouped vs. singleton (50 tiers) COSMIC (speedups are ratios of mean matched times):
SynER1/2/5at d=50): Speedups persist with density, while batch validity and recovery deteriorate:SynER1SynER2SynER5The heuristic does not shrink average batch sizes as density increases: initial true sinks fall from 20.7 (
SynER1) to 5.0 (SynER5), yet tier counts stay near 12 and valid co-sink batches fall from 66.2% to 44.4%. Thus, speedups persist despite departures from ideal co-sink batching.Deeper configurations show lower recovery for both grouped and singleton COSMIC (grouped F1 is slightly higher at L=3 and lower at L=5,10), while co-sink batch validity drops (69.9%→49.3%) and tier counts remain nearly unchanged (12.56→12.30).
Sink Availability: Increasing initial sinks from 1 to 13 reduces Ttiers from 12.70 to 12.00 and raises ordering speedup from 6.36× to 7.17×, while grouped F1 shifts from 0.567 to 0.537, giving modest speedup without better recovery.
Intra-Group Diagnostic & Linearity Scope:
SynER1), 8.31% (SynER2), and 12.50% (SynER5), and rise across depths 3/5/10 from 9.28% to 12.00% to 13.50%. These quantify exposure to intra-tier zero scoring, rather than measured screening deletions: saved outputs lack screening masks or childwise maxima. This study uses 50% nominal linear mechanisms; a multi-level linearity sweep against CaPS/SCORE remains outstanding.3. Real-World vs. Semi-Synthetic Benchmarks
Verified ground-truth DAGs for continuous observational data at d≥50 are scarce. Contemporary ordering-based methods likewise combine real measurements with synthetic or semi-synthetic benchmarks. We originally attempted to include DREAM5, but end-to-end discovery at that scale proved infeasible under our GAM pruning bottleneck; useful recovery on larger real datasets beyond Sachs therefore remains unverified here.
We thank Reviewer SCEk for the constructive feedback, which helped us quantify the trade-offs of grouped removal and refine our evaluation.