acceptodds
Under review as a conference paper at ICLR 2027

What Joint Score Information Retains for Causal Ordering

Abstract

Score-based methods can identify leaves in nonlinear additive noise models, but recovering a full causal order requires repeating this decision after each removal. Can the expected joint score-Jacobian matrix alone support this recursion? We show that it cannot in general, even when the matrix is known exactly. The proof uses two three-variable nonlinear models with equal-variance Gaussian noise. Their joint matrices are identical and identify X3 as a leaf in both models. After X3 is removed, one model requires X1 to precede X2, while the other requires X2 to precede X1. Since the matrix cannot distinguish these orders, no rule using it alone can guarantee a valid order over the model class. This limitation also applies to Schur elimination, although it is exact in linear Gaussian models. An exact identity shows that the Schur update differs from the true marginal matrix by a scaled covariance of the removed leaf's mechanism gradient. This term vanishes when the gradient is constant. When nonzero, it need not change the selected leaf, so experiments with Score-Schur Topological Sort (SSTS) test when the uncorrected update still yields valid orders. Across five paired synthetic datasets per dimension, SSTS gives zero edge violations at 50 variables but 4.6 +/- 2.4 at 100 variables, compared with 0.2 +/- 0.4 for public SCORE/DAS ordering. Separate paired diagnostics show that lower matrix-estimation error can accompany more order violations. A guarantee over the stated nonlinear model class still requires information beyond this joint expectation or further model restrictions.

Primary area:
causal reasoning
Keywords:
causal discovery, causal ordering, score matching, Schur complement, identifiability

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_GhNXOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies whether the location Fisher information matrix of the joint observational distribution suffices to recover the causal order. It constructs two nonlinear models with identical matrices but incompatible orders, and derives an exact gradient-covariance expression for the discrepancy between Schur elimination and true marginalization. It also evaluates Score-Schur Topological Sort, which estimates the matrix once and recursively applies Schur updates. Validation includes algebraic checks, synthetic benchmark comparisons, paired matrix diagnostics, and the Sachs dataset, showing successful recovery in some synthetic settings but limited accuracy and inconsistent overall efficiency gains elsewhere.

Critical strengths
  1. A clear counterexample. This establishes a precise distinction between identifying a leaf and recovering a complete order, ruling out universal recovery by any ordering rule based solely on this matrix even with perfect estimation.

  2. An interpretable characterization of elimination error. The exact gradient-covariance identity identifies what a Schur update omits when removing a true leaf and explains why the discrepancy vanishes for linear mechanisms. This provides a useful analytical basis for understanding the limits of extending linear precision-based elimination to nonlinear models.

Critical weaknesses
  1. Limited significance of the theoretical contribution. The paper does not sufficiently establish why restricting causal discovery to a single joint expected score-Jacobian matrix is an important practical problem, particularly when existing methods already avoid retraining while retaining sample-level score access. The counterexample establishes a specific information limitation, but the covariance identity is a short consequence of standard score decompositions, and the recovery conditions do not yield an informative nonlinear recoverable class or characterize partially identifiable structure. Together, these results provide insufficient theoretical advancement to support acceptance.
  2. The experiments establish neither a compelling practical benefit nor a convincing explanation of failure. The main public comparison covers a narrow synthetic family, with weaker ordering accuracy than established baselines at larger scales and no consistent end-to-end runtime advantage in the matched comparison. The experiments also do not isolate whether the gradient-covariance term explains the observed sorting errors, and the proposed sufficient condition does not explain successful leaf selections in the reported diagnostics. These limitations leave the paper without sufficiently strong empirical evidence of either practical utility or mechanistic insight to support acceptance.
Questions for authors
  1. What concrete practical benefit does restricting access to the fixed joint SJIM provide over methods that already avoid retraining?
  2. What evidence distinguishes sorting errors caused by the gradient-covariance gap from those caused by score estimation error?
Reviewer_6GkGOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether the expected joint score-Jacobian information matrix (SJIM) is sufficient to recover a complete causal ordering in nonlinear additive noise models (ANMs) with equal-variance Gaussian noise. The key contributions are 1. An impossibility result: Theorem 3.1 constructs two three-variable nonlinear ANMs with different causal orderings but identical joint SJIMs. 2. A characterization of the nonlinear marginalization error: Proposition 3.2 shows that the discrepancy between Schur elimination of the joint SJIM and the true marginal SJIM is given by the covariance of the removed leaf's mechanism gradient. 3. An empirical investigation: The authors introduce Score-Schur Topological Sort (SSTS), which estimates the joint SJIM once and recursively performs Schur elimination to recover a causal ordering.

Critical strengths
  1. A concrete impossibility result for causal ordering from joint score information. Theorem 3.1 provides an explicit counterexample showing that two nonlinear causal models with incompatible causal orderings can have the same expected joint SJIM.

  2. An exact characterization of the error introduced by Schur elimination. Proposition 3.2 characterizes the discrepancy between the Schur-updated SJIM and the true marginal SJIM as a positive-semidefinite covariance term involving the removed leaf's mechanism gradient.

Critical weaknesses
  1. The theoretical contribution concerns a rather restrictive information setting, limiting its broader significance. The main impossibility result assumes that the ordering procedure has access only to the expected joint SJIM, with no additional access to the observational data, sample-dependent scores, or marginal score information. This is a strong restriction on the information available to the algorithm. Existing score-based causal discovery approaches, such as SCORE and DAS, are not subject to this restriction, since they can use additional score information during recursive ordering. Consequently, the impossibility result does not establish a fundamental limitation of score-based causal discovery in general. Moreover, the covariance correction in Proposition 3.2 follows relatively directly from the score-Hessian identities under additive Gaussian noise. Thus I am not fully convinced that the contribution is significant enough for a ICLR paper.

  2. The proposed algorithm does not demonstrate a consistent empirical advantage over existing approaches. Furthermore, the matched-schedule experiment in Table 2 does not establish a consistent advantage in either ordering accuracy or end-to-end runtime. On the Sachs benchmark, SSTS performs poorly, with more edge violations than the expected value for a random ordering.

Questions for authors
  1. Theorem 3.1 establishes impossibility when only the expected joint SJIM is available. Could the authors clarify why this particular information restriction is important in practical causal discovery applications?

  2. Proposition 3.2 shows that the nonlinear marginalization discrepancy depends on the covariance of the removed leaf's mechanism gradient. Can the authors characterize a nontrivial nonlinear model subclass for which the joint SJIM alone is sufficient for complete ordering?

3.What is the principal practical advantage of SSTS relative to existing score-based ordering algorithms? The reported experiments do not establish a consistent improvement in accuracy or total runtime.

Reviewer_4qrdOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies whether the expected joint score-Jacobian matrix contains sufficient information to recover a complete causal order in nonlinear additive Gaussian noise models. It constructs two models with identical matrices but incompatible causal orders, and derives a covariance expression for the discrepancy between Schur elimination and exact marginalization. The paper also provides conditional stability results and evaluates Score-Schur Topological Sort (SSTS), which estimates a joint score model once and applies repeated Schur updates to its averaged Jacobian.

Critical strengths
  • The explicit three-variable counterexample illustrates the distinction between identifying the first leaf and retaining enough information to recover a complete causal order.
  • The paper organizes the relevant assumptions and supporting results.
Critical weaknesses
  1. The methodological novelty of SSTS is very limited. Its main components are established techniques: DSM, the expected diagonal leaf criterion, and minimum-diagonal selection followed by Schur elimination. The block selection and ridge regularization do not appear to provide a substantial new algorithmic contribution. SSTS therefore appears primarily to be an implementation and empirical evaluation of a nonlinear extension of an existing elimination procedure.
  2. The significance of the central contribution is insufficiently established. Theorem 3.1 provides a precise limitation of using only the expected joint score-Jacobian matrix. However, established methods such as SCORE already recompute information for the reduced system, and DiffAN retains access to its learned score representation. The counterexample does not invalidate these approaches. The manuscript should explain more clearly which existing claim, unresolved question, or practically important design choice this impossibility result addresses, beyond ruling out a straightforward extension of linear Gaussian elimination. Although the individual results are identifiable, I remain unconvinced that their combined significance constitutes a substantial advance.
  3. The nonlinear Schur discrepancy appears to require a relatively straightforward derivation, and the paper does not develop it into a corrective method. In the stated additive Gaussian setting, the covariance expression follows by differentiating the leaf conditional, averaging the resulting blocks, and substituting them into the Schur complement. The exact expression is informative, but the failure of a Gaussian precision-marginalization rule to remain exact for general nonlinear mechanisms is not itself surprising. If this characterization is intended as a major contribution, its additional implications need to be stronger. For example, a practical correction using additional sample-level information, or a concrete and useful nonlinear model class with a recovery guarantee, would make the result more consequential. SSTS instead leaves the identified discrepancy uncorrected.
Questions for authors

Please see Weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.