acceptodds
Under review as a conference paper at ICLR 2027

Scalable Post-Nonlinear Causal Discovery via Likelihood-Based Order Learning

Abstract

Post-nonlinear (PNL) structural causal models allow for flexible nonlinear mechanisms but remain difficult to learn in multivariate and high-dimensional settings. The few existing multivariate PNL methods primarily estimate a causal order through recursive sink identification and scale poorly. We present SCORPIO, a method for learning multivariate PNL DAGs under causal sufficiency. SCORPIO parameterizes each inverse outer transformation as a strictly monotone sigmoid normalizing flow, yielding a tractable node-wise likelihood that supports several noise distribution families. We estimate a topological order by greedily adding edges based on local likelihood improvements, which requires updating only the affected node model, avoiding retraining of the full system model. Candidate edges are subsequently pruned via marginal independence tests between scalar residuals. Across synthetic experiments varying the noise distribution, parental-mechanism complexity, graph size, and graph density, SCORPIO achieves the lowest order divergence and structural Hamming distance in most settings. Under a 24-hour time limit, SCORPIO is the only evaluated multivariate PNL method that completed experiments beyond variables, with results reported for up to variables. It also achieves the lowest order divergence and competitive DAG-recovery performance on a semi-synthetic manufacturing benchmark and the Sachs protein dataset.

Primary area:
causal reasoning
Keywords:
causal+discovery, post-nonlinear, structure+learning, order, likelihood, normalizing+flow

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_dyGkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents SCORPIO, a method for multivariate post-nonlinear causal discovery under causal sufficiency. It combines monotone-flow parameterization of inverse outer transformations with CAM-style greedy edge addition based on local likelihood improvements, followed by residual-based independence testing for pruning. Preliminary neighborhood selection is used for larger systems. Experiments cover synthetic data with different noise distributions, mechanisms, graph sizes, and densities, together with a semi-synthetic manufacturing benchmark and the Sachs dataset. The main empirical contribution is improved computational feasibility relative to the evaluated multivariate PNL methods, alongside favorable graph-recovery results in many synthetic settings.

Critical strengths

Meaningful computational improvement within the multivariate PNL setting. SCORPIO substantially reduces the reported wall-clock runtime relative to existing PNL baselines. At 100 variables, it takes 21.62 minutes compared with 822.57 minutes for CAF-PoNo under the same CPU-core allocation. The screened pipeline also completes experiments with 500 variables, making larger-scale PNL discovery more computationally feasible. A reasonably extensive and transparent empirical study. The evaluation varies several important data-generating factors and includes diagnostics for score directionality, pruning, noise-family selection, and preliminary neighborhood selection. The paper explicitly acknowledges the heuristic nature of the procedure and the absence of an order-recovery guarantee.

Critical weaknesses

The significance of the specific problem addressed and its practical payoff are not sufficiently established. Causal discovery is clearly important, but the paper needs a stronger argument for why scaling this particular PNL pipeline addresses a consequential unmet need. The strongest large-scale results are obtained on synthetic PNL data with uniform noise and preliminary neighborhood selection. These results demonstrate performance within the chosen model class, but provide less evidence that the additional PNL flexibility yields a meaningful advantage in realistic applications. On Sachs, CAM achieves better SHD, SID, and F1 than SCORPIO; on the manufacturing benchmark, SCORPIO improves order divergence and SID but does not obtain the best SHD or F1. At 500 variables, the reported F1 scores are 0.45 and 0.21 for the two graph densities. Thus, completing larger problems is a useful computational result, but the paper does not yet convincingly establish when this translates into sufficiently accurate and practically useful causal discovery.

The methodological novelty appears limited relative to the existing components. The greedy edge-addition principle is explicitly adapted from CAM, normalizing-flow-based PNL modeling is already present in CAF-PoNo, and residual/latent independence testing has established precedents discussed in the paper. The positive-linear-term sigmoid flow provides a convenient invertible parameterization, but its role appears primarily architectural. The resulting method is a coherent combination and adaptation of existing ideas; what remains unclear is which new insight produces a substantial advance beyond this integration. The existing ablations are helpful, but do not directly separate the effects of the search strategy, flow parameterization, pruning rule, and screening under matched noise assumptions and training budgets. Since the paper is primarily an empirical algorithmic contribution, stronger evidence that its distinctive design choices are responsible for the gains would be important to justify acceptance.

Questions for authors
  1. Practical significance: Can the authors identify a concrete application regime in which the PNL assumptions are substantively justified and SCORPIO’s additional flexibility provides an important advantage over scalable alternatives? How should the mixed DAG-recovery results on Sachs and the manufacturing benchmark inform the claimed practical value?
  2. Distinct methodological contribution: What is the principal new methodological insight beyond combining CAM-style greedy search, flow-based PNL modeling, and residual-based pruning? Please distinguish the new algorithmic elements from adaptations of existing techniques and explain why the new elements are consequential.
  3. Attribution of empirical gains: Can the authors provide a controlled comparison isolating the order-learning strategy from the flow parameterization, noise-family choice, and screening procedure? A representative experiment with matched components and computational budgets would help establish the source of the gains.
  4. Pruning reliability: Table 3 specifies a polynomial kernel for independence testing. What degree and normalization are used, and how sensitive are the pruning results to replacing it with a Gaussian/RBF kernel? Please also clarify whether the observations used for testing are independent of those used for early stopping and model selection.
3 comments
AuthorsOpenReview ↗

Response to Reviewer dyGk

Thank you very much for your time and effort to review our submission. We take your suggestions seriously and address them below:

  1. Practical significance:

    • Topological order estimation: The practical significance of SCORPIO's PNL assumption and likelihood framework, compared with methods that use additive noise models or linear assumptions, is evident in the order divergence metric. SCORPIO achieves the lowest mean error in the estimated order in 18 of 21 reported settings and, in particular, the lowest mean order divergence in the semi-synthetic and real-world benchmarks.

    • Mixed data support: Indeed, our CAM variant achieved the lowest SHD on the semi-synthetic causalAssembly data. We failed to point out, however, that unmodified CAM's spline fitting can fail on the low-cardinality or near-constant variables present in these mixed categorical and continuous data. In our benchmark, we extended CAM with extra preprocessing to handle categorical variables. We now label this variant CAM-ext and retain it as a baseline. For a fair comparison, excluding CAM would result in SCORPIO having the lowest SHD prediction.

    • Scalability: Real-world data are often characterized by high dimensionality and complex causal mechanisms. Our experiments in Section 5.1.3 and, in particular, the runtime comparison in Section 5.1.4 show that the other evaluated multivariate PNL methods are limited in scalability. SCORPIO is the only evaluated multivariate PNL method that completes experiments with p≥150p\geq150 variables within the 24-hour limit and, in particular, scales up to p=500p=500 variables. This scalability improvement comes primarily from our new order-selection module, which uses local greedy updates. Furthermore, the newly added RESIT ablation, see Figure 14 and Table 11, supports the computational advantage of greedy edge ranking with our sigmoid-flow PNL likelihood over iterative sink identification, with broadly similar structural errors.

  2. SCORPIO's main contribution is a scalable implementation of likelihood-based order learning for multivariate PNL models. Previous PNL works required repeated all-variable fits within their sink-identification subroutines, whereas our new approaches require only local updates and enable parallel candidate evaluation. Additional innovations are the flexible noise family and avoiding surrogate models in residual pruning. Our algorithm design not only drastically improves computational feasibility in high-dimensional problems; our results show that SCORPIO achieves the lowest mean error in order estimation across all but three settings.

  3. We provide ablations that isolate different components of SCORPIO. Appendix E.6 shows the effect of applying PNS as a screening procedure, with errors shown in Figure 21 and a runtime study in Table 13. The contribution of greedy edge ranking is reported in Appendix E.2 by comparison with a version of SCORPIO that instead implements RESIT with PNL loss for order identification. See Figure 14 for the error comparison and Table 11 for the runtime report. While order divergence, SHD and SID are broadly similar in the available plotted settings up to p=100p=100 variables, the RESIT approach yields approximately 855%855\% more runtime at p=100p=100, q=4q=4. Results for p≥150p\geq150 are still pending due to the increased runtime. An ablation of the significance level and kernel for independence testing in the pruning phase is shown in Appendix E.4, Figure 16 and Figure 17, respectively.

  4. We are happy to provide further details on the pruning phase. All reported experiments used a degree-two polynomial kernel, with data standardized to zero mean and unit sample variance prior to the test. Our configuration allows the use of training samples, validation samples, or both. The validation samples comprise a held-out 30%30\% of the observations and are used for score computation and early stopping during order learning, as well as noise-family selection when enabled. For all reported experiments, we sampled from the combined training and validation data for independence testing in pruning. The additional simulations in Appendix E.4, specifically Figure 17, show small differences in SHD, SID, and F1F_1 score between polynomial and Gaussian RBF kernels, suggesting limited sensitivity to kernel choice.

Please note that we have updated the Figure and Table numbers in the above response to match the updated paper submission.

Thank you for the additional experiments and updates. The reported comparison between the oracle noise model and the selection protocol addresses my concern about relying on a known noise family. The RESIT ablation also makes the source of the speedup clearer. I give the paper credit for these additions.

I am still leaning against acceptance, mainly for the two reasons raised in my original review. First, the method remains largely a combination of established ideas: CAM-style edge ranking and screening, flow-based PNL modeling, and residual independence testing. The RESIT comparison supports the efficiency of this combination, but the similar structural errors suggest that the demonstrated gain is mainly computational. This is useful, though I do not find the methodological advance substantial enough to change my recommendation.

Second, the application results still leave the practical benefit unclear. Better order divergence is encouraging, but it does not consistently carry through to better DAG recovery on Sachs or causalAssembly. I would also retain CAM-ext as a relevant baseline; the need for preprocessing does not diminish the value of that comparison. My concern is not that PNL discovery is unimportant, but that the paper has yet to show a sufficiently strong benefit from using this more flexible model at the scales it now makes feasible.

The rebuttal has strengthened the evidence for the runtime improvement. It has not changed my overall assessment of the contribution, so I will keep my score at 2.

Reviewer_ScfQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SCORPIO learns multivariate PNL DAGs. The inverse outer functions are modeled as monotone sigmoid flows, which gives a node-wise likelihood. Instead of recursive sink identification, the order is learned by greedily adding edges according to their local likelihood gain, an idea adapted from CAM's incEdge. Edges are then pruned with an HSIC test between estimated noises and residuals. The method is much faster than existing PNL methods and runs on graphs with up to 500 nodes.

Critical strengths

The scalability gap for PNL models is real, and SCORPIO clearly closes it: at p=100 it takes 21.6 min, versus 13.7 h for CAF-PoNo with the same number of cores. The experimental study is careful and includes useful diagnostics. The limitations are stated honestly, and code is provided.

Critical weaknesses
  1. The ordering step is the main contribution, yet it comes with no guarantee. Even a population-level result would help a lot: does the true order maximize the expected PNL likelihood, and when does the greedy search find it? Fig. 11 suggests the per-edge signal is weak, with the correct direction preferred for only 57–78% of edges.
  2. The method mostly combines known pieces (incEdge/PNS, sigmoid flows, latent residual tests), and I could not tell which of them matters. An ablation that keeps the same flow likelihood but uses RESIT-style sink identification would settle this.
  3. The larger benchmarks (5.1.2, 5.1.3 and the 500-node runs) all use uniform noise. That is exactly where SCORPIO's advantage is largest in Fig. 2. These runs appear to use the matched noise family. The end-to-end family-selection protocol is evaluated only on the p=10 benchmark of 5.1.1, and it costs 2.45–3.13× the matched-family runtime. The headline runtimes therefore seem to exclude family selection. 4. Please show scaling results under Gaussian or Student-t noise, using the selection protocol.
  4. There is no real-data evidence of better DAG recovery. On Sachs, CAM beats SCORPIO on SHD, SID and F1. On causalAssembly, every method has F1 < 0.2, and SCORPIO's F1 (0.157) is below both CAF-PoNo (0.182) and CAM (0.171), while CAM attains the lowest SHD.
Questions for authors
  1. Which noise family did SCORPIO use in 5.1.2, 5.1.3 and the 500-node experiments, and how do the results and runtimes change under the validation-NLL selection protocol?
  2. Could you run the sink-identification ablation described in Weakness 2?
  3. Is there any population-level argument for the greedy edge ranking?
  4. The related work presents multivariate PNL ordering only through RESIT-style sink identification, and SCORE is cited only for the order-divergence metric. Score-matching ordering methods (SCORE, DAS, DiffAN, LINMix, CaPS) are the main scalable alternatives for order learning. Since ANMs are a special case of PNL, they can be run directly on the ANM-like settings. The paper should discuss them and compare against at least SCORE/DAS in accuracy and runtime.

Scalable causal discovery with score matching. CLeaR, 2023.(DAS) Diffusion models for causal discovery via topological ordering. ICLR, 2023.(DiffAN) Causal discovery with mixed linear and nonlinear additive noise models: A scalable approach. CLeaR, 2024. (LINMix) Ordering-based causal discovery for linear and nonlinear relations. NeurIPS, 2024.(CaPS)

3 comments
AuthorsOpenReview ↗

Response to Reviewer ScfQ

We thank you for your helpful suggestions and criticism. We address the points raised below:

  1. It is correct that SCORPIO used a uniform noise likelihood in the experiments in Sections 5.1.2 and 5.1.3 and the 500-node experiments. Parameters of the distribution, in this case the lower and upper bounds, were not provided and had to be estimated by SCORPIO. We performed the suggested experiment of evaluating SCORPIO with full selection protocol on a version of the scalability benchmark of Section 5.1.3 with Gaussian data-generating noise, without providing the likelihood family. The results are now added to Appendix E.5 and visualized in Figure 20. Note that this figure currently compares SCORPIO with the full selection protocol on Gaussian-noise data to SCORPIO with a fixed uniform noise likelihood on uniform-noise data. Therefore, the two effects of 1) using the full selection protocol and 2) evaluating on data with Gaussian noise are currently combined in the results. We provide absolute errors of SCORPIO on data with different noise distributions, including Gaussian and Uniform, in Appendix D.1, Figure 7. Simulations of SCORPIO with a fixed Gaussian likelihood are currently being performed and will enable isolation of the effect of the full selection protocol in the comparison.

    We observe an average runtime increase of approximately 3.283.28 across different pp (number of variables) and qq (expected in-degree) settings, see Table 12. For example, the selection protocol requires 176.92176.92 minutes per trial on average for graphs with 200200 variables and expected in-degree 44, while PNL methods CAF-PoNo, AbPNL and RankPNL did not complete systems of this size within 24 hours.

  2. We implemented a RESIT ablation of SCORPIO by substituting greedy edge-ranking order learning with iterative sink identification that uses our sigmoid flow models and PNL loss function during model fitting. The HSIC independence tests are kept as in the original RESIT, as they are model- and loss-function independent. PNS and pruning of SCORPIO is not adapted. Partial results are shown in Appendix E.2, Figure 14 and Table 11. Due to the prolonged runtime of the RESIT approach, see Table 11, we currently report the available SCORPIO-RESIT settings up to p=100p=100 and will update the missing entries as soon as simulations finish. While order divergence (OD), SHD and SID are broadly similar in the plotted settings, the increase in runtime is substantial: for q=4q=4, the RESIT ablation requires approximately 3.8x runtime for p=50p=50 and 8.5x for p=100p=100, supporting SCORPIO's better computational feasibility for high-dimensional data.

  3. We are currently not aware of a population-level consistency guarantee for greedy edge ranking (including CAM's incEdge). This is an open research question that requires further investigation.

  4. Thank you for pointing out another valuable order learning ansatz that was previously missing from our discussion. We provide a new section, Appendix D.6 ``Comparison with Score-Matching'', that evaluates SCORE on the parental-mechanism benchmark of Section 5.1.2. SCORE requires about 0.03970.0397 minutes per trial compared to SCORPIO's 1.53331.5333 minutes, averaged equally across the four classes with p=10p=10 variables. This runtime difference may could possibly be attributed to SCORE's simpler ANM assumption. SCORPIO achieves lower mean order divergence, SHD and SID in all four settings as well as higher mean precision, recall and F1F_1 score (Figure 12). For example, SCORPIO's mean order divergence ranges from 1.611.61 to 5.705.70 across parental-mechanism classes, compared to 7.547.54 to 9.669.66 for SCORE.

Update regarding (1): Figure 20 now shows the requested comparison of SCORPIO with an oracle (data-generating) noise likelihood vs. SCORPIO with the full selection protocol. Performance in (a) order divergence, (b) SHD, (c) SID, (d) precision, (e) recall and (f) F1F_1 score is very similar with a slight decrease in precision under the selection protocol. The runtimes are updated and shown in Table 12. Averaged equally across all fully evaluated (p,q)(p,q) settings, the full selection protocol takes 3.053.05 times as long as the oracle configuration. As highlighted previously, this results in, for instance, 176.92176.92 minutes per trials on average for graphs with 200200 variables and expected in-degree 44 under SCORPIO's full selection protocol, whereas PNL methods CAF-PoNo, AbPNL and RankPNL did not complete evaluations on systems of this size in 2424 hours.

Update regarding (2): We added further datapoints to the RESIT ablation of SCORPIO in Figure 14. Due to the prolonged runtime of RESIT, not all 5050 trials (55 trials for each of the 1010 graphs) per setting of (p,q)(p,q) were completed. We provide data up to p=150p=150 and q=2q=2 where all trials were evaluated. At (p,q)=(50,2)(p,q)=(50,2), the RESIT variant takes approximately 4.78×4.78\times as long as original SCORPIO. At (p,q)=(150,2)(p,q)=(150,2), the RESIT variant requires 494.79494.79 minutes per trial on average, compared with 34.8834.88 minutes for standard SCORPIO, corresponding to approximately 14.19×14.19\times the runtime.

Thank you for your response. What I'm thinking is that the components come from prior work: greedy edge ranking (incEdge) and PNS are taken from CAM; the flow-based inverse outer transformation follows CAF-PoNo (Hoang et al., 2024) and uses a standard monotone sigmoid-sum flow; the latent residual independence test follows Duong & Nguyen (2022) and Hoang et al. (2024). The worst-case complexity is cubic in p, which is the same as for the other PNL methods (Section 4.4). Even though the order learning is faster than sink identification by a constant factor, but it does not recover the order more accurately. And accuracy at scale is close to trivial baselines. Without new identifiability results or guarantees, however, the case for the method rests on the experiments, but still the paper does not yet show that PNL discovery at this scale pays off. I keep my score.

Reviewer_vKDWOpenReview ↗
Rating
4(+2)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The paper presents a new causal discovery method for the quite general class of post-nonlinear models, and the method is more accurate and much more scalable than other existing ones. The method is not proven to be consistent, but it is evaluated on a range of synthetic and semi-synthetic data as well as the Sachs et al. data.

Critical strengths
  • The method performs better than competitors and does quickly on large datasets, for a very general class of causal DAG models
Critical weaknesses
  • Very sparse graphs are used in the experiments, without sufficient discussion of performance compared to chance
  • No consistency guarantees.
Questions for authors
  1. Without doing the actual computation, the results in 4(b) look hardly better to me than what a method would do if it always returned the empty graph. What's the actual chance performance here compared to the proposed method's performance? (I'm thinking, e.g., of something like this paper). The paper is a clear accept to me, if this point is adequately addressed.
5 comments
AuthorsOpenReview ↗

Response to Reviewer vKDW

Thank you, Reviewer vKDW, for your time and effort in reviewing our submission. Your suggestions are very appreciated. The note about a negative control with an empty and random baseline is a valid point. We performed the requested experiments and added the results to a new Section E.1. "Comparison with Negative Controls" on page 25 in an updated, uploaded version. The plots are contained in Figures 11 and 12 on page 26. We believe that this adds further support for our method and value to the paper.

We included a baseline returning an empty graph and a random baseline. The random baseline constructs random ER DAGs as proposed in the negative-control approach of Petersen (2024). Details are given in the updated paper. SCORPIO achieves lower mean SHD and SID than both controls across the ten evaluated settings of the scalability benchmark and lower mean order divergence than the random control in nine settings, with the exception of (p,q)=(10,4)(p,q)=(10,4), where pp denotes the number of variables and qq the expected in-degree (Figure 11). Note that, order divergence is not applicable to the empty baseline. Other methods, like CAM, NOTEARS-linear and DAGMA are close to the SHD and SID of the empty control.

Furthermore, the quality of SCORPIO's prediction over the negative controls, is presented in the mean F1F_1 scores, with SCORPIO achieving a visibly higher mean F1F_1 score in all evaluated settings. The empty control has an F1F_1 score of zero, while the random control approaches zero as graph size increases. Despite their similar predicted edge counts, SCORPIO recovers substantially more true positive edges than the random control. For instance, at p=200p=200 and q=2q=2, SCORPIO recovers 128128 true positive edges on average, compared with 1.41.4 true positive edges on average for the random control.

This helps address my concern, but I still think it's worth adding the random- and empty-graph baselines also to other plots besides only those using the F-score. It should be quick/easy to add it to Figure 4 (b), no? I'm worried the SHD and SID plots without such baselines along with the F-score with the baselines could still unintentionally hide something about the performance. I suspect the baselines perform comparatively better on SHD than F-score, considering the sparsity of the graphs.

I have updated my rating and would consider updating it again if the remaining concerns are addressed.

Thank you very much for your time to answer. The requested Figure showing SHD and SID of empty and random negative controls on the scalability benchmark is Figure 13 on page 27 in the Appendix. We had updated the paper with the suggestions of the other reviewers which had led to a renumbering of the Figures. We apologize for the confusion and now placed the Figure right below the text for clearer visibility. Figure 14 on page 28 shows the F1F_1 score.

We will also update the paper to show the negative controls directly in the main text as they are important baselines that add comparativeness and strength to the study. We will make another comment once updated to the main text.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Comment to all Reviewers and AC

We thank the reviewers for their time, effort and helpful feedback. For the AC's convenience, we provide a brief summary of the discussions:

  • Reviewer vKDW raised their rating to clear accept after we provided negative control baselines (empty and random graphs) that substantiated our experiments and addressed vKDW's main concern.
  • Reviewers ScfQ and dyGK provided lengthier lists of small issues and requests for additional experiments. Although they have not followed up, we addressed their questions with
    • an ablation study replacing our new order learning module with RESIT-style iterative sink identification with PNL likelihood (including order divergence, SHD, SID and runtime),
    • direct comparison of oracle noise against noise selection protocol under Gaussian noise with respect to scalability (reporting order divergence, SHD, SID, precision, recall, F1F_1 score and runtimes)
    • comparison against a score-matching algorithm (SCORE),
    • sensitivity analysis for the kernel in the pruning test. These additions further corroborate the good empirical performance and computational advantage of the proposed algorithm.
  • The main remaining weakness, a lack of formal consistency guaranteed, is acknowledged (and shared by most competing methods).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.