- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents SCORPIO, a method for multivariate post-nonlinear causal discovery under causal sufficiency. It combines monotone-flow parameterization of inverse outer transformations with CAM-style greedy edge addition based on local likelihood improvements, followed by residual-based independence testing for pruning. Preliminary neighborhood selection is used for larger systems. Experiments cover synthetic data with different noise distributions, mechanisms, graph sizes, and densities, together with a semi-synthetic manufacturing benchmark and the Sachs dataset. The main empirical contribution is improved computational feasibility relative to the evaluated multivariate PNL methods, alongside favorable graph-recovery results in many synthetic settings.
Critical strengths
Meaningful computational improvement within the multivariate PNL setting. SCORPIO substantially reduces the reported wall-clock runtime relative to existing PNL baselines. At 100 variables, it takes 21.62 minutes compared with 822.57 minutes for CAF-PoNo under the same CPU-core allocation. The screened pipeline also completes experiments with 500 variables, making larger-scale PNL discovery more computationally feasible. A reasonably extensive and transparent empirical study. The evaluation varies several important data-generating factors and includes diagnostics for score directionality, pruning, noise-family selection, and preliminary neighborhood selection. The paper explicitly acknowledges the heuristic nature of the procedure and the absence of an order-recovery guarantee.
Critical weaknesses
The significance of the specific problem addressed and its practical payoff are not sufficiently established. Causal discovery is clearly important, but the paper needs a stronger argument for why scaling this particular PNL pipeline addresses a consequential unmet need. The strongest large-scale results are obtained on synthetic PNL data with uniform noise and preliminary neighborhood selection. These results demonstrate performance within the chosen model class, but provide less evidence that the additional PNL flexibility yields a meaningful advantage in realistic applications. On Sachs, CAM achieves better SHD, SID, and F1 than SCORPIO; on the manufacturing benchmark, SCORPIO improves order divergence and SID but does not obtain the best SHD or F1. At 500 variables, the reported F1 scores are 0.45 and 0.21 for the two graph densities. Thus, completing larger problems is a useful computational result, but the paper does not yet convincingly establish when this translates into sufficiently accurate and practically useful causal discovery.
The methodological novelty appears limited relative to the existing components. The greedy edge-addition principle is explicitly adapted from CAM, normalizing-flow-based PNL modeling is already present in CAF-PoNo, and residual/latent independence testing has established precedents discussed in the paper. The positive-linear-term sigmoid flow provides a convenient invertible parameterization, but its role appears primarily architectural. The resulting method is a coherent combination and adaptation of existing ideas; what remains unclear is which new insight produces a substantial advance beyond this integration. The existing ablations are helpful, but do not directly separate the effects of the search strategy, flow parameterization, pruning rule, and screening under matched noise assumptions and training budgets. Since the paper is primarily an empirical algorithmic contribution, stronger evidence that its distinctive design choices are responsible for the gains would be important to justify acceptance.
Questions for authors
- Practical significance: Can the authors identify a concrete application regime in which the PNL assumptions are substantively justified and SCORPIO’s additional flexibility provides an important advantage over scalable alternatives? How should the mixed DAG-recovery results on Sachs and the manufacturing benchmark inform the claimed practical value?
- Distinct methodological contribution: What is the principal new methodological insight beyond combining CAM-style greedy search, flow-based PNL modeling, and residual-based pruning? Please distinguish the new algorithmic elements from adaptations of existing techniques and explain why the new elements are consequential.
- Attribution of empirical gains: Can the authors provide a controlled comparison isolating the order-learning strategy from the flow parameterization, noise-family choice, and screening procedure? A representative experiment with matched components and computational budgets would help establish the source of the gains.
- Pruning reliability: Table 3 specifies a polynomial kernel for independence testing. What degree and normalization are used, and how sensitive are the pruning results to replacing it with a Gaussian/RBF kernel? Please also clarify whether the observations used for testing are independent of those used for early stopping and model selection.
3 comments
Please note that we have updated the Figure and Table numbers in the above response to match the updated paper submission.
Thank you for the additional experiments and updates. The reported comparison between the oracle noise model and the selection protocol addresses my concern about relying on a known noise family. The RESIT ablation also makes the source of the speedup clearer. I give the paper credit for these additions.
I am still leaning against acceptance, mainly for the two reasons raised in my original review. First, the method remains largely a combination of established ideas: CAM-style edge ranking and screening, flow-based PNL modeling, and residual independence testing. The RESIT comparison supports the efficiency of this combination, but the similar structural errors suggest that the demonstrated gain is mainly computational. This is useful, though I do not find the methodological advance substantial enough to change my recommendation.
Second, the application results still leave the practical benefit unclear. Better order divergence is encouraging, but it does not consistently carry through to better DAG recovery on Sachs or causalAssembly. I would also retain CAM-ext as a relevant baseline; the need for preprocessing does not diminish the value of that comparison. My concern is not that PNL discovery is unimportant, but that the paper has yet to show a sufficiently strong benefit from using this more flexible model at the scales it now makes feasible.
The rebuttal has strengthened the evidence for the runtime improvement. It has not changed my overall assessment of the contribution, so I will keep my score at 2.
Response to Reviewer dyGk
Thank you very much for your time and effort to review our submission. We take your suggestions seriously and address them below:
Practical significance:
Topological order estimation: The practical significance of SCORPIO's PNL assumption and likelihood framework, compared with methods that use additive noise models or linear assumptions, is evident in the order divergence metric. SCORPIO achieves the lowest mean error in the estimated order in 18 of 21 reported settings and, in particular, the lowest mean order divergence in the semi-synthetic and real-world benchmarks.
Mixed data support: Indeed, our CAM variant achieved the lowest SHD on the semi-synthetic causalAssembly data. We failed to point out, however, that unmodified CAM's spline fitting can fail on the low-cardinality or near-constant variables present in these mixed categorical and continuous data. In our benchmark, we extended CAM with extra preprocessing to handle categorical variables. We now label this variant CAM-ext and retain it as a baseline. For a fair comparison, excluding CAM would result in SCORPIO having the lowest SHD prediction.
Scalability: Real-world data are often characterized by high dimensionality and complex causal mechanisms. Our experiments in Section 5.1.3 and, in particular, the runtime comparison in Section 5.1.4 show that the other evaluated multivariate PNL methods are limited in scalability. SCORPIO is the only evaluated multivariate PNL method that completes experiments with p≥150 variables within the 24-hour limit and, in particular, scales up to p=500 variables. This scalability improvement comes primarily from our new order-selection module, which uses local greedy updates. Furthermore, the newly added RESIT ablation, see Figure 14 and Table 11, supports the computational advantage of greedy edge ranking with our sigmoid-flow PNL likelihood over iterative sink identification, with broadly similar structural errors.
SCORPIO's main contribution is a scalable implementation of likelihood-based order learning for multivariate PNL models. Previous PNL works required repeated all-variable fits within their sink-identification subroutines, whereas our new approaches require only local updates and enable parallel candidate evaluation. Additional innovations are the flexible noise family and avoiding surrogate models in residual pruning. Our algorithm design not only drastically improves computational feasibility in high-dimensional problems; our results show that SCORPIO achieves the lowest mean error in order estimation across all but three settings.
We provide ablations that isolate different components of SCORPIO. Appendix E.6 shows the effect of applying PNS as a screening procedure, with errors shown in Figure 21 and a runtime study in Table 13. The contribution of greedy edge ranking is reported in Appendix E.2 by comparison with a version of SCORPIO that instead implements RESIT with PNL loss for order identification. See Figure 14 for the error comparison and Table 11 for the runtime report. While order divergence, SHD and SID are broadly similar in the available plotted settings up to p=100 variables, the RESIT approach yields approximately 855% more runtime at p=100, q=4. Results for p≥150 are still pending due to the increased runtime. An ablation of the significance level and kernel for independence testing in the pruning phase is shown in Appendix E.4, Figure 16 and Figure 17, respectively.
We are happy to provide further details on the pruning phase. All reported experiments used a degree-two polynomial kernel, with data standardized to zero mean and unit sample variance prior to the test. Our configuration allows the use of training samples, validation samples, or both. The validation samples comprise a held-out 30% of the observations and are used for score computation and early stopping during order learning, as well as noise-family selection when enabled. For all reported experiments, we sampled from the combined training and validation data for independence testing in pruning. The additional simulations in Appendix E.4, specifically Figure 17, show small differences in SHD, SID, and F1 score between polynomial and Gaussian RBF kernels, suggesting limited sensitivity to kernel choice.