acceptodds
Under review as a conference paper at ICLR 2027

Causal Discovery from Diverse Data

Abstract

Causal discovery is increasingly applied across heterogeneous environments whose data-generating processes may differ substantially. Unlike standard multi-domain formulations based on hard or soft interventions, we do not assume that the domains arise from interventions on a common baseline graph: their causal graphs may differ structurally, including through edge additions, deletions, or reversals. We ask a fundamental question: how much causal knowledge can be learned by jointly exploiting such heterogeneous datasets? We show that mechanisms that remain stable across domains can induce cross-domain distributional invariances, even when other mechanisms and the underlying causal graphs differ. By systematically leveraging these invariances together with within-domain conditional independences, we identify the distributional constraints that such invariances impose on tuples of domain-specific causal graphs and characterize the resulting inv-Markov equivalence class. In particular, cross-domain invariances can reveal causal orientations that are not identifiable from any domain in isolation. Our algorithm can leverage cross-domain invariances to learn cause-effect relations beyond what is learnable in each domain, even when the underlying causal graphs are arbitrarily different, such as having reversed edges. Building on this characterization, we develop Diverse-FCI, a constraint-based algorithm that learns domain-specific partially oriented graphs while exploiting pairwise cross-domain invariances. Under a faithfulness condition ensuring that the relevant cross-domain dependencies remain detectable after latent projection, we prove its soundness. Synthetic experiments show improved orientation recovery over learning each domain independently, with larger gains as structural heterogeneity increases.

Primary area:
causal reasoning
Keywords:
Causal Discovery, Markov Equivalence, Heterogeneous Data

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_Fw9sOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies causal discovery across heterogeneous domains whose causal graphs may differ structurally, including edge reversals and latent confounding, without assuming a common baseline graph or intervention semantics. It introduces cross-domain invariance constraints (inv-constraints), characterizes inv-Markov equivalence using augmented maximal ancestral graphs (MAGs), and proposes Diverse-FCI, which combines within-domain conditional independence tests with cross-domain invariance tests. The algorithm is proven sound under inv-faithfulness but is not complete in general. Synthetic and real-data experiments evaluate its ability to recover domain-specific causal structures.

Critical strengths

Novel and well-motivated setting. The framework accommodates structurally different domain-specific graphs, including reversed causal directions and latent confounding, without requiring a common acyclic causal template. Figure 1 illustrates how cross-domain invariances can identify causal orientations that individual domains cannot.

Potentially significant theoretical contribution. Reducing inv-Markov equivalence to standard MAG equivalence via twin-augmented MAGs (Theorem 4.6), together with the domain-centric representation (Proposition 5.2), is an elegant approach. Diverse-FCI also provides a soundness guarantee under inv-faithfulness, although the equivalence characterization has a correctness issue discussed below.

Relevant comparisons and transparent limitations. The synthetic experiments demonstrate improved orientation recovery under substantial structural heterogeneity. The authors explicitly acknowledge the algorithm's incompleteness under latent confounding (Appendix E.4) and report settings where CD-NOD outperforms Diverse-FCI (Table 3).

Critical weaknesses

A counterexample invalidates Lemma B.5 as stated. Section 3 defines the mechanism-shift set partly through observed distributional invariance. However, Lemma B.5 uses such invariance to infer graphical (F)-separation, which Proposition 4.4 does not justify. A linear-Gaussian counterexample (given in Q1) satisfies the paper's definition of I(i,j)I^{(i,j)} but violates the claimed symmetry of (F)-adjacencies across domain copies. Since Lemma B.4 and the necessity direction of Theorem 4.6 rely on Lemma B.5, the central characterization is not established as written. Defining stability witnesses graphically may repair this issue, but would require revising the formulation. Separately, Lemma B.6 incorrectly asserts that adjacent-pair dependence along an active path implies endpoint conditional dependence; a generic-parameter argument is needed in Lemma B.7.

Questions for authors

Q1. Correctness of Lemma B.5 and Theorem 4.6.

Consider independent standard Gaussian variables U,ϵX,ϵYU,\epsilon_X,\epsilon_Y and two domains:

  • Domain 0: X=ϵX,Y=X+U+ϵYX=\epsilon_X,\quad Y=X+U+\epsilon_Y.
  • Domain 1: X=−2U+ϵX,Y=X+U+ϵYX=-2U+\epsilon_X,\quad Y=X+U+\epsilon_Y.

The mechanism of YY is unchanged, and both domains satisfy P(Y)=N(0,3)P(Y)=\mathcal N(0,3). Thus, under Section 3's definition, I(0,1)={X}I^{(0,1)}=\{X\}, with W=∅W=\varnothing witnessing the stability of YY. Hard intervention on XX also yields a common bridge model. Nevertheless, in the augmented domain-0 graph, FF and YY are separated by XX, while in domain 1 they are inseparable over the observed variables because of the paths F→X→YF\to X\to Y and F→X←U→YF\to X\leftarrow U\to Y.

This contradicts Lemma B.5's claimed symmetry of FF-adjacencies. The invariance results from parameter cancellation: replacing −2-2 with aa in the domain-1 equation for XX gives Var⁡(Y)=(a+1)2+2=3\operatorname{Var}(Y)=(a+1)^2+2=3 only when a∈{0,−2}a\in\{0,-2\}. Thus, the example violates inv-faithfulness and does not contradict Theorem 6.3, which assumes it; however, Theorem 4.6 does not impose this assumption.

Could the authors clarify whether stability witnesses should instead be defined graphically, as suggested by Figure 2, and provide a corrected statement and proof of Theorem 4.6? Alternatively, does the theorem require inv-faithfulness? These remedies are not equivalent, since imposing faithfulness would change the class of distributions being characterized.

7 comments

Thank you for the careful reading and for identifying the issue in Lemma B.5. Your counterexample is correct. The original proof used an observed invariance to infer a graphical separation, which Proposition 4.4 does not justify. We have revised the formulation and replaced the affected arguments.

On graphical stability and Lemma B.5

We adopt graphical stability witnesses, as you suggested. In Section 3, we first identify the genuinely changed mechanisms, including possible changes at latent variables. We then construct an observed mechanism-shift set I(i,j)I^{(i,j)} by graphical closure: an observed variable is added whenever no common observed conditioning set separates it from the auxiliary source F(i,j)F^{(i,j)} in both domain augmentations. The process continues until no further variable is added.

Consequently, an observed variable outside I(i,j)I^{(i,j)} has an unchanged mechanism and admits a common observable separating set:

\quad\text{and}\quad (F^{(i,j)}\perp X^{(j)}\mid W^{(j)}).$$ An unchanged mechanism can still belong to $I^{(i,j)}$ when such a common observed witness is unavailable. This distinction is necessary under latent confounding. Lemma B.5 now follows graphically. Variables in $I^{(i,j)}$ have direct $F$-arrows in both copies; variables outside it have the common separator supplied by the closure. These properties survive latent projection and passage to the twin augmented MAG. Thus, $$F^{(i,j)}\to X^{(i)} \iff F^{(i,j)}\to X^{(j)} \iff X\in I^{(i,j)}.$$ The proof no longer infers separation from a distributional equality. **Your counterexample under the revised definition** In your example, the genuine mechanism change is at $X$. With only $F\to X$, domain 1 has no observed separator between $F$ and $Y$: leaving $X$ unconditioned keeps $F\to X\to Y$ active, while conditioning on $X$ opens $F\to X\leftarrow U\to Y$. The graphical closure therefore gives $I^{(0,1)}=\{X,Y\}$, and the final augmented graphs have symmetric $F$-adjacencies. The equality $P^{(0)}(Y)=P^{(1)}(Y)=\mathcal N(0,3)$ still holds, but is an accidental invariance not implied by the graphical constraints. The distribution tuple satisfies the revised inv-Markov property and is inv-unfaithful. We have included this example in Appendix B to make the distinction explicit. **Scope of the characterization** We agree that defining stability graphically and restricting the characterized distributions to faithful ones are different remedies. We use the graphical formulation. For a fixed graph-shift tuple $(\mathbf G,\mathcal I)$, Definition 4.1 specifies the family of all distribution tuples satisfying its within-domain CI and cross-domain invariance constraints, including unfaithful tuples. Theorem 4.6 does not impose inv-faithfulness on membership in this family. We choose not to introduce inv-faithfulness at the direction because it is only required for learning. Sections 3–4 and Appendix B now state this formulation explicitly. We have also revised the bridge argument to handle changed latent mechanisms through a common observed post-intervention distribution.

On global graphical compatibility and the necessity direction of Theorem 4.6

For K≥3K\geq3, a witness construction must respect mechanism sharing across all domains simultaneously. If a mechanism is shared along a chain of domains, it must also be shared between the endpoints. These ties can imply invariances beyond those recorded by an individual pair's augmentation. Constructing witnesses independently for each pair would therefore be insufficient.

We make this issue explicit through global graphical compatibility (GC). For each observed variable XX, group domains connected by chains of pairs in which XX is stable, and let CX(i)C_X(i) denote the group containing domain ii. Define

A(i,j)={X:CX(i)≠CX(j)}⊆I(i,j).A^{(i,j)}=\{X:C_X(i)\neq C_X(j)\}\subseteq I^{(i,j)}.

GC requires that replacing the FF-targets I(i,j)I^{(i,j)} by A(i,j)A^{(i,j)} leaves all paired FF-separations with a common observed conditioning set unchanged. Thus, pairwise graphical constraints must already reflect the invariances implied by global mechanism sharing. For K=2K=2, A(1,2)=I(1,2)A^{(1,2)}=I^{(1,2)}, so GC is automatic. Its formal definition is Equation (8) in Appendix B.3. GC does not impose a common causal ordering or a reference domain.

Theorem 4.6 now explicitly assumes GC. We acknowledge that this restricts the graph-shift tuples covered by the characterization.

Replacement of Lemmas B.6–B.7

We agree that dependence between adjacent variables along an active path does not establish conditional dependence between its endpoints. We have replaced that argument with a global Gaussian witness construction:

  1. Lemma B.4, using the corrected Lemma B.5, establishes that non-equivalent twin MAGs differ on a testable within-domain CI or paired invariance query.

  2. Lemma B.6 constructs one Gaussian parameter family across all KK domains, tying local mechanisms wherever sharing is forced, including the ties required by latent mechanisms. For every query absent from the graphical signature, the proof constructs a violating parameter point within this same constrained family. For a missing invariance, an auxiliary Gaussian source supplies an intercept perturbation that respects all sharing ties and changes one domain's conditional law while leaving the comparison domain unchanged.

  3. Conditional Gaussian parameters are rational functions of the free coordinates. The explicit witnesses show that the relevant equality polynomials are not identically zero after imposing the sharing constraints. Each accidental equality therefore occurs on a null set; there are finitely many queries, so a jointly inv-faithful realization exists. The construction also realizes the prescribed graphical mechanism-shift collection.

Lemma B.7 then separates the two inv-Markov families: a faithful realization for one candidate violates the distinguishing constraint imposed by the other. This proves the necessity direction for general KK. Faithful distributions serve as witnesses in the proof; the characterized families themselves still include unfaithful distributions.

Effect on learning

Diverse-FCI is unchanged. Its soundness continues to rely on inv-faithfulness, now stated explicitly for all target variables, including those in I(i,j)I^{(i,j)}. This excludes cancellation invariances such as the equality in your example and ensures that invariance tests recover the graphically defined FF-adjacencies.

Under the causal-model setup of Section 3, inv-faithfulness also implies GC. The forced mechanism sharing yields a common observed bridge after intervention on A(i,j)A^{(i,j)}. If GC failed, that bridge would imply an observed invariance absent from the augmentation using I(i,j)I^{(i,j)}, contradicting inv-faithfulness. Remark B.9 gives this argument. Thus, GC adds no independent assumption to the learning guarantee, although it is an explicit condition of the characterization theorem.

We have updated the statements and proofs of Theorem 4.6 and Proposition 5.2, replaced Lemmas B.6–B.7, and revised the soundness proof of Theorem 6.3 accordingly. We thank the reviewers for identifying these issues. These comments helped us make the formulation and proof obligations precise. We are happy to take any further questions.

Overall, the authors have addressed my main theoretical concerns. Now, I have two more questions:

  1. For the synthetic experiments with three or more domains, can the authors confirm that the generated graph-shift tuples satisfy GC, and explain whether this condition was explicitly verified?
  2. Since the mechanism-shift set has been redefined graphically, can the authors provide precision and recall for the recovered (F)-adjacencies against the corresponding theoretical shift sets on synthetic data? This would help establish how reliably the algorithm recovers the distinctive information targeted by the revised framework.
Reviewer_HpwCOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

The paper studies causal discovery under heterogeneous environments where data comes from several domains that may have different causal mechanisms, without assuming the differences come from interventions. The authors identify a set of shifted variables per pair of domains and encode this through an extra “F-node” in the graph, and derive graphical sufficient conditions, called inv-constraints, for when a relationship stays the same across domains. Under an inv-faithfulness assumption, these inv-constraints are used to define an inv-Markov equivalence class over graph tuples, represented compactly per domain using twin augmented MAGs. Building on this, the authors proposed Diverse-FCI, an algorithm that is sound but not complete in the presence of latent confounding. Tested on synthetic data and two real datasets, the experiments show that accounting for domain invariances adds beneficial orientation information that grows with structural heterogeneity.

Critical strengths

1- The paper is well-motivated; it targets cases where datasets are collected from multiple sources, such as different hospitals or cohorts, creating different domain shifts in the datasets which may affect the overall causal mechanisms. 2- The authors designed their algorithm (Diverse-FCI ) to have a general setting, where domains are not required to come from interventions on one shared baseline graph, as each domain can have its own causal structure. 3- Diverse-FCI algorithm is supported with solid, extensive theory in both the main paper body and the appendix, and its performance is tested on synthetic data and two real-world datasets (SEER and MIMIC-IV). 4- The authors show that Diverse-FCI clearly beats pooled FCI (which ignores domain shifts), as indicated in Table 2 of the experiments, demonstrating that accounting for domain invariances adds beneficial orientation information. Table 3 further shows that this benefit becomes even more evident with increasing structural heterogeneity. 5- The authors are transparent with their algorithm limitations and list them in Appendix F.

Critical weaknesses

1- Diverse-FCI algorithm runtime increases a lot with the number of domains (K), refering to Table 2, where the number increases from 20.00s (K=2) to 147.35s (K=4). This highlights the algorithm's limited scalability, especially when applied to real-world datasets spanning multiple domains and larger sizes. It's also worth highlighting that the runtime for the real-world datasets (SEER and MIMIC-IV) is not reported directly in the paper; this is important because it shows how the algorithm performs in practice. 2- As per my understanding, the algorithm represents domain invariances and the mechanism shifts in the F-nodes, under the assumption that domain membership has no selection bias and that F node is exogenous. However, in many real cases, this may not be the case, where domains like insurance type, or ethnicity group could have other hidden factors. 3- All experiments are on a small number of variables and domains (at most 30 variables and 4 domains for synthetic data, Tables 1 and 2; and for real data, at most 19 variables with SEER using 2 domains (Appendix D.4) and 12 variables for MIMIX-IV using 3 domains (Appendix D.5). Even at this small scale, a substantial fraction of variables (5.6 out of 12 for MIMIC-IV and 7 out of 19 for SEER) were flagged as domain-sensitive, so the runtime and complexity problems above could even get more problematic for realistic cases. 4- The F-node specified per domain is generic and could include many different real causes, for example, different hospital scanning machines, human errors, different disease prevalence, etc. Hence, as per my understanding, the algorithm can indicate which variables are affected by the domain shift, but not what underlying real-world causes produced this shift. 5- The authors mentioned that the algorithm is sound but not complete when there are latent confounders; it outputs a set of separate graphs rather than one graph, and it heavily relies on cross-domain invariances, which could negatively affect the performance when the dataset tested is small and the domain shifts are weak.

Questions for authors

1- Is there a way to control the runtime growth for a higher number of domains, instead of always comparing every pair? Have the authors tested more than K=4 domains? 2- As per my understanding from the framewrok setup, the algorithm assumes domain membership (F-nodes) to be exogenous, with no selection bias. Still, real domains like insurance type or ethnicity group may be caused by latent or unmeasured patient factors. If this assumption does not hold, how will the F-nodes' adjacencies be affected, and is there a way to test it from the data? 3- Is it possible to decompose what is really causing an F-node's connection to a given variable, rather than referring to it all as a generic "domain-shift", to actually uncover the underlying causal structure?

5 comments

Thank you for the comprehensive and thoughtful comments. We appreciate your positive assessment of the motivation, theoretical results, and empirical benefits of exploiting domain invariances. We address the concerns in three parts.

On runtime growth and more domains (Weaknesses 1 and 3, Question 1)

We have extended the runtime evaluation beyond four domains (Section 7 and Appendix D.1, Table 4). For random binary models with latent confounding, 10 observed variables, and 5,000 samples per domain, mean sequential runtimes are 4.17, 9.06, 18.62, 36.57, and 72.58 seconds for K=2,3,4,6,8K=2,3,4,6,8, respectively.

We also evaluate parallel execution across domain anchors, where each task learns one domain's augmented graph by comparing it with the other domains. At n=10,K=8n=10,K=8, mean anchor-parallel time is 39.07 seconds, compared with 72.58 seconds sequentially. At n=20,K=8n=20,K=8, the corresponding times are 223.69 and 434.91 seconds. These measurements include all pairwise comparisons. The parallel timer includes process-pool startup but excludes shared data preparation. The sequential timer covers the full discovery call.

The implementation considers K(K−1)/2K(K-1)/2 unordered domain pairs. Its cost also depends on graph size and conditioning-set searches. A retained F-edge can require exhausting the candidate conditioning sets. Parallelization reduces elapsed time, while the total statistical search remains substantial.

To reduce the comparison count itself, one computational option is to select a reference domain rr and analyze only pairs (r,i)(r,i) if one is only interested in one specific domain, giving K−1K-1 two-domain analyses. Under the corresponding oracle assumptions, each selected pairwise fit retains the soundness guarantee of Theorem 6.3 for its own pair. Omitted comparisons may contain additional invariance and orientation information, so equivalence with the full all-pairs analysis is not guaranteed. The reference is a computational choice and requires no intervention interpretation. This option has not been benchmarked in the revision.

On the real-data runtimes

The previously omitted runtimes are now reported in Section 7 and Appendices D.4–D.5. On one CPU core, the selected 19-variable SEER analysis with K=2K=2 and 15,000 cases per domain takes approximately 391 seconds. The selected 12-variable MIMIC-IV insurance analysis with K=3K=3 and 12,000 stays per domain takes approximately 32 seconds. Both timings exclude preprocessing.

On larger graphs

We have added experiments with 50 and 100 observed variables, latent confounding, and 20,000 samples per domain (Table 3). Here, mm counts structural modifications, and each non-reference domain also receives an additional soft mechanism shift. Domain-averaged partial-graph SHDs, reported as mean ±\pm standard error over ten shared seeds, are:

(n,K,m)=(50,4,20)(n,K,m)=(50,4,20): Diverse-FCI 97.62±1.8997.62\pm1.89, per-domain FCI 105.42±2.30105.42\pm2.30, and CD-NOD 133.80±2.18133.80\pm2.18.

(100,2,40)(100,2,40): Diverse-FCI 199.25±4.33199.25\pm4.33, per-domain FCI 206.35±4.37206.35\pm4.37, and CD-NOD 213.20±6.19213.20\pm6.19.

(100,2,20)(100,2,20): Diverse-FCI 205.10±2.40205.10\pm2.40, per-domain FCI 212.25±2.75212.25\pm2.75, and CD-NOD 204.65±4.32204.65\pm4.32.

These results show additional recovery gains over per-domain FCI in the listed settings, while comparison with CD-NOD depends on the setting. The listed Diverse-FCI runs use a conditioning-set cap of 2 and should be interpreted as bounded-search experiments.

The computational tradeoff remains substantial. Diverse-FCI takes 263.9 seconds in the 50-variable setting and 320.1 seconds in the 100-variable, m=40m=40 setting, versus 6.6 and 15.2 seconds for per-domain FCI. Thus, the revision broadens the empirical evaluation while documenting the remaining cost of exploiting cross-domain information. Efficient large-scale discovery remains an important direction for future work.

On F-node exogeneity and selection bias (Weakness 2, Question 2)

The distinction between the formal F-node and the real-world domain-assignment process is important. F(i,j)F^{(i,j)} is an auxiliary graphical encoding of invariance and shift constraints between domains ii and jj. Its outgoing edges are part of this representation. They do not assert that insurance type, ethnicity, or hospital membership was randomized or generated by an external intervention.

Nevertheless, this distinction does not remove the modeling assumptions. The learning guarantee assumes the causal-model setup and inv-faithfulness, and the introduction explicitly assumes no selection bias. Allowing latent confounding within domains does not automatically handle selection induced by conditioning on cohort membership.

Differences in latent patient factors can alter observable invariances and hence the inferred F-adjacencies. Conditioning on a domain label influenced by patient variables or their latent causes can also introduce selection effects, changing within-domain CI relations and cross-domain invariances. Such effects are not automatically handled by adding more F-edges. Soundness for the underlying population graph does not follow simply by treating selected cohorts as different domains.

In general, observed invariance patterns alone cannot certify the absence of selection bias or uniquely determine whether an F-adjacency originates from selection, latent population differences, measurement changes, or another source. A large F-adjacent set is not a test for selection bias. Information about cohort construction, sampling, and the domain-assignment process is needed to assess these assumptions.

We therefore interpret the clinical analyses as exploratory findings under the stated modeling assumptions. The ethnicity and insurance labels define the domain comparisons. The inferred graphs do not establish causal effects of those labels or identify the physical process responsible for each discrepancy.

On decomposing the causes of shifts (Weakness 4, Question 3)

We agree that identifying the real-world causes of a shift is valuable. Our present target characterizes graph tuples distinguishable from within-domain CIs and pairwise invariances. These constraints do not, in general, uniquely identify the physical causes producing the differences.

An F-edge is also not a direct measurement of a changed biological mechanism. The mechanism-shift set may include a variable whose underlying mechanism is unchanged but whose stability has no common observed graphical witness. Conversely, a marginal distribution can change while an appropriate conditional remains invariant.

The 7 of 19 F-adjacent variables in SEER and the average of 5.6 of 12 in MIMIC-IV indicate that many variables lack a detected common invariance conditoining set in those comparisons. Their interpretation depends on the model assumptions and statistical tests. They do not establish which underlying biological functions changed.

If scanner type, measurement protocol, or other candidate causes are measured, including them as observed variables can support a more detailed analysis of which invariances remain after accounting for them. Known intervention information can provide additional constraints. Across multiple domains, pairwise F-adjacency patterns can indicate which domain comparisons exhibit discrepancies, although this alone does not identify their physical causes.

The new Perturb-seq illustration in Appendix D.6 provides a concrete example. The F-node is adjacent to the ER-stress expression module and HSPA9 expression, but not to the balanced guide indicator. This localizes domain-sensitive quantities relative to a shared invariance anchor. It does not by itself explain which biological changes between the cell types produced those differences.

On soundness and incompleteness (Weakness 5)

We agree that soundness and completeness are different guarantees. Theorem 6.3 establishes soundness under inv-faithfulness and access to the population distributions, with the prescribed searches. It does not establish that Diverse-FCI recovers every invariant orientation in the characterized equivalence class.

Appendix E.4 gives an explicit example of this gap. The algorithm can leave an endpoint unresolved even when joint compatibility across domains forces its orientation. Deriving additional cross-domain orientation rules and establishing completeness remain important theoretical directions.

On the graph-tuple representation

The representation reflects the target of the problem: when domains have different causal structures, the output needs to retain those differences. Proposition 5.2 packages the pairwise information into one augmented graph per domain, with F-adjacencies encoding the corresponding mechanism-shift information.

A single common graph over the system variables would generally discard domain-specific differences. A unified representation could retain them through domain indexing or additional annotations. Our domain-centric tuple provides one way to organize that information. It also makes clear which within-domain structure and pairwise invariance information support each result.

On weak shifts and small samples

We agree that weak shifts and small samples can make invariance testing unreliable. The soundness theorem is a population-level result, rather than a finite-sample guarantee against statistical testing errors.

A false invariance decision can incorrectly remove an F-edge and affect subsequent orientations. Consequently, there is no general guarantee that estimation errors merely reduce the output to per-domain FCI or only leave additional endpoints unresolved.

The low-heterogeneity results in Appendix D.1 show mixed performance:

At n=50,K=2,m=2n=50,K=2,m=2, Diverse-FCI and per-domain FCI have mean partial-graph SHDs of 103.42±3.13103.42\pm3.13 and 104.05±2.87104.05\pm2.87, respectively.

At n=100,K=2,m=3n=100,K=2,m=3, the corresponding values are 214.55±3.55214.55\pm3.55 and 210.25±3.55210.25\pm3.55, so Diverse-FCI performs worse on average in this setting.

These results show that gains are not uniform when the domains are nearly shared. They also need to be distinguished from experiments on weak shift magnitude: mm counts structural modifications, rather than the magnitude of each change, and these experiments do not directly test small-sample robustness.

The power of invariance tests depends on conditioning sets, distributional properties, and sample sizes. It need not equal the power of within-domain CI tests. A dedicated evaluation varying shift magnitude and sample size remains a direction for further study. Appendix F already acknowledges the limitations associated with incompleteness, weak shifts, and finite samples.

We thank the reviewer again for the constructive comments. The added results in Section 7 and Appendix D better document the empirical benefits and computational costs, while the clarifications above explain the scope and interpretation of the guarantees. We are happy to take any further questions.

Reviewer_VwUTOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies simultaneous causal discovery over multiple environments, allowing for latent confounders and for different topological order between environments (unlike when each environment corresponds to an intervention on a baseline causal model). Specifically, all environments have the same set of observed variables V\mathbf{V}, and the paper considers invariant causal mechanisms: for two environments ii and jj, X∈VX \in \mathbf{V} is considered "stable" if its mechanism is invariant and if there is some witness W⊆V∖XW \subseteq \mathbf{V} \setminus \\{ X \\} such that P(i)(x∣w)=P(j)(x∣x)P^{(i)}(x \mid w) = P^{(j)}(x \mid x), otherwise XX belongs the mechanism-shift set I(i,j)I^{(i,j)}.

The paper goes through the usual characterizations for causal graphical models: they define the inv-Markov property, inv-Markov equivalence, provide a graphical characterization of inv-Markov equivalence (Theorem 4.6), and provide a concise representation of the inv-Markov equivalence class. Then, under an appropriate faithfulness condition (inv-faithfulness), they provide a constraint-based algorithm (extending the usual PC/FCI algorithm), show that the algorithm is sound (i.e., that the inferred skeleton is correct, and that all inferred orientations are correct), but give a counterexample to show that it need not be complete under latent confounding).

Synthetic experiments show that the method works as expected: in particular, it outperforms methods that were designed under different assumptions (e.g. FCI-JCI and ψ\psi-FCI).

Critical strengths

From the theoretical perspective, the paper is fairly complete, and the terminology/notation/proofs are on solid ground, building upon well-established causal structure learning literature, while providing new results. The main distinctions from existing work are clear, e.g., that the causal graphs for different environments do not need to share a common topological order / arise from interventions on a designated "observational" environment. The paper also provides clear examples to show these differences, to show their graphical construction, and to show incompleteness.

Critical weaknesses

The two main weaknesses are the incremental nature of the paper, and incomplete coverage of related work.

Incrementality: By now, the causal structure learning literature is very rich, especially in terms of the kinds of theoretical characterizations given in this paper. I acknowledge that each new characterization requires non-trivial new work, but (in my opinion) this paper does not provide the kind of deep/exciting/novel results that really push the field forward. Hence, the paper may be better-suited for a dedicated conference like CLeaR, or as a journal paper where the method is applied to a real dataset that could not be properly analyzed using previous algorithms. For ICLR/ICML/NeurIPS, the paper would at least need to provide a much more scalable approach with experiments in higher-dimensional settings.

Related work: The main paper does not reference Appendix A, which contains additional related work. However, even taking this section into account, much relevant discussion is missing. For example (though this is not comprehensive): the discussion of IGSP doesn't discuss the unknown-target version (Squires et al., 2020), many works are missing on joint/multi-domain estimation under a shared topological order (e.g., Wang et al., 2020; Chen et al. 2021; Lee and Cao 2022; Chang and Taskin, 2026), and some work is missing for joint/multi-domain estimation without a shared topological order (e.g., Oates et al., 2014). Given the setting being studied, I'd expect more targeted discussion on the joint/multi-domain setting, and less on the simpler single-domain setting, which can be outsourced to review articles.

References

Squires et al. (2020), Permutation-based causal structure learning with unknown intervention targets
Wang et al. (2020), High-dimensional joint estimation of multiple directed Gaussian graphical models
Chen et al. (2021), Multi-task learning of order-consistent causal graphs
Lee and Cao (2022), Bayesian joint inference for multiple directed acyclic graphs
Chang and Taskin (2026), Leveraging heterogeneity for identifiability: Bayesian order-based learning of multiple DAGs
Oates et al. (2014), Exact estimation of multiple directed acyclic graphs

Questions for authors
  1. Please discuss how you may address the weaknesses mentioned above.

  2. Can you give convincing examples (with appropriate references) of applications where causal directions may actually reverse, but where the other modeling assumptions (e.g., acyclity) are expected to hold, at least up to a good approximation?

6 comments

Thank you for the comprehensive and thoughtful comments. We appreciate your recognition of the theoretical results and the distinctions from existing formulations. We address the concerns in three parts.

On novelty

Our central contribution is to establish foundations for causal discovery from diverse data: what causal information is available from within-domain conditional independences and cross-domain invariances when domains have different causal structures? Each domain has its own acyclic causal model, potentially with latent confounding. Differences between domains need not have intervention semantics relative to a designated baseline, and the graphs need not share a common causal ordering.

We address this identification question through three contributions:

We derive graphical conditions connecting cross-domain mechanism stability to observable conditional invariances.

We characterize inv-Markov equivalence for graph-shift tuples under global graphical compatibility, using augmented graphical representations.

We develop Diverse-FCI, a sound learning algorithm under inv-faithfulness that exploits both within-domain CIs and pairwise invariances.

The characterization specifies which graph tuples are distinguishable using the stated CI and distributional invariances within our model class. It establishes the graphical target for what is learnable from these constraints. The algorithm provides a constructive way to exploit that information and recover additional valid orientations, although it need not recover every invariant orientation in the target.

We believe this identification question is fundamental because allowing different graphs does not by itself explain when information can be transferred between them. With latent confounding and changing structures, an unchanged local mechanism does not justify copying an observed edge or arbitrary observed conditional across domains. Establishing the connection between mechanism stability, observable invariances, and graphical equivalence is therefore necessary for understanding the possibilities and limits of joint discovery.

On scalability and higher-dimensional evaluation

We agree that computational scalability and higher-dimensional evaluation are important. We have expanded Section 7 and Appendix D.1 with accuracy comparisons reaching 100 observed variables and a separate runtime benchmark reaching eight domains.

At (n,K,m)=(50,4,20)(n,K,m)=(50,4,20), Diverse-FCI reduces mean partial-graph SHD from 105.42±2.30105.42\pm2.30 for per-domain FCI to 97.62±1.8997.62\pm1.89. At (100,2,20)(100,2,20), the corresponding values are 212.25±2.75212.25\pm2.75 and 205.10±2.40205.10\pm2.40. Here, mm counts structural modifications, with an additional soft mechanism shift in each non-reference domain. These are bounded-search comparisons, and the gains are not uniform under low structural heterogeneity.

The separate domain-count benchmark evaluates K=2,3,4,6,8K=2,3,4,6,8. At n=20,K=8n=20,K=8, mean sequential and anchor-parallel times are 434.91 and 223.69 seconds, respectively (Table 4). Parallel execution retains all pairwise comparisons. Computational cost nevertheless remains substantially greater than independent per-domain discovery.

Our present theoretical contribution concerns identification and sound discovery. The additional experiments broaden its empirical evaluation, while faster search and more scalable implementations remain important directions for further work.

We agree that the related-work discussion should focus more directly on joint and multi-domain discovery. We have expanded Section 2 and Appendix A, and the main text now explicitly refers to the appendix. The discussion compares the information linking domains, ordering assumptions, treatment of latent variables, and theoretical targets.

Unknown intervention targets

Squires et al. (2020) is an important invariance-based predecessor. UT-IGSP combines observational CI information and conditional-invariance tests to recover unknown intervention targets and the interventional equivalence class, with consistency guarantees under its assumptions. Its setting is causally sufficient and relates observational and interventional distributions to a common DAG.

Our distinction is the equivalence characterization for structurally heterogeneous domain-specific graph tuples with latent confounding. The domains need not be interventions on a common underlying causal structure or share a common ordering.

Joint estimation under a shared ordering

Wang et al. (2020), Chen et al. (2021), and Lee and Cao (2022) establish important results for joint estimation of multiple Gaussian DAGs. Wang et al. and Chen et al. assume a shared unknown ordering, whereas Lee and Cao assume a known parent ordering.

These assumptions allow domain-specific edges and coefficients but exclude opposite directed edges across domains. Our framework instead links domains through pairwise mechanism stability and its observable invariances, while accommodating latent confounding and incompatible causal orders.

Identification gains from heterogeneity

Chang and Taskin (2026) is particularly relevant to our identification objective. Under a common-order assumption, their framework combines ordering constraints supplied by different domains, establishing conditions under which heterogeneity reduces ordering ambiguity. We explicitly acknowledge this conceptual connection.

Identification gains from jointly analyzing heterogeneous graphs are therefore already established. Our characterization concerns the constraints supplied by cross-domain conditional invariances in causal models with latent variables whose domain graphs need not admit a common ordering.

Joint estimation without a shared ordering

Oates et al. (2016; the 2014 preprint cited in the review) already permit multiple DAGs without a common ordering. Their joint estimation approach couples DAGs through structural-similarity priors. Accordingly, we do not claim that different orderings or edge reversals alone are novel.

Our contribution is the graphical characterization of equivalence induced by specified within-domain CI and cross-domain invariance constraints, together with a sound discovery procedure for compatible latent-variable graph tuples.

We have also retained detailed comparisons with CD-NOD and unknown general interventions in Appendices E.2–E.3. These comparisons acknowledge that previous formulations can allow structural changes. The distinctions concern how domains are related, which invariances are available, the handling of latent confounding, and the resulting identification target.

Perturb-seq across cellular contexts

We have added an analysis of real Perturb-seq data from K562 and RPE1 cells (Replogle et al., 2022), summarized in Section 7 and detailed in Appendix D.6. The domains are cellular contexts. One cell line is not assumed to be an intervention on the other.

Within each domain, we use a BiP-knockdown indicator AA, an ER-stress expression module XX, and HSPA9 expression YY. Guide proportions are balanced across domains, providing a marginal invariance anchor. Neither Diverse-FCI nor per-domain FCI receives background knowledge orienting edges out of AA.

BiP knockdown activates the stress module in both cell types, while HSPA9 increases substantially in RPE1 and changes little in K562. Diverse-FCI learns:

RPE1:A∘→X→Y,\text{RPE1:}\quad A\mathrel{\circ\rightarrow}X\rightarrow Y,

K562:A∘→X←∘Y.\text{K562:}\quad A\mathrel{\circ\rightarrow}X\mathrel{\leftarrow\circ}Y.

The estimated direction at the stress-module endpoint therefore changes across contexts. In RPE1, the X→YX\rightarrow Y orientation is obtained through cross-domain F-node information, whereas per-domain FCI leaves the chain unoriented. Across 500 design-preserving bootstrap resamples, the RPE1 orientation appears in 98.0% and the K562 arrowhead into XX appears in 86.8%. The full-sample patterns persist across the tested significance thresholds from 10−410^{-4} to 0.10.1 and conditioning-set sizes 1 and 2.

The K562 circle at YY remains unresolved. We therefore interpret this as a change in identified directional structure, retaining the PAG uncertainty. The appendix also discusses possible biological feedback and the limits of the selected acyclic approximation.

An exploratory screen identifies three further cases: EIF3E knockdown with RACK1, EIF3M knockdown with RPL14, and SART1 knockdown with EIF3F as the readouts. Each gives a collider-like configuration in K562 and a directed chain in RPE1, while per-domain FCI leaves the RPE1 chain unoriented. The RPE1 orientations appear in 99.0%, 89.7%, and 97.0% of 300 bootstrap resamples, respectively. These screened cases are exploratory follow-ups, rather than independent validation experiments (Table 7).

Poker as a supplementary positive control

We additionally analyze 45,029 hands from DeepStack's games against human players (Moravčík et al., 2017). The domains differ in who acts first before the flop. With the decision variables indexed by player, the protocol exchanges their temporal order across domains while preserving an acyclic ordering within each domain.

In the four-variable analysis, the observed variables comprise each player's private-card strength and first decision. Let XX denote the human's first decision and YY DeepStack's. Diverse-FCI learns:

Human acts first:X→Y,\text{Human acts first:}\quad X\rightarrow Y,

DeepStack acts first:Y→X.\text{DeepStack acts first:}\quad Y\rightarrow X.

Neither algorithm receives temporal orientation priors. Per-domain FCI already identifies opposite arrowheads, and Diverse-FCI additionally resolves the tails. The full-sample patterns hold in all four tested settings combining significance thresholds 0.01 and 0.05 with conditioning-set sizes 1 and 2.

This example directly illustrates how a protocol-defined change in who acts first can change the causal ordering of the selected variables without introducing a temporal cycle. It is a supplementary positive control. The statistical analysis remains exploratory and its approximate CI modeling is discussed in the appendix.

Figures 5–7 and Table 7 present the graphs, descriptive evidence, and bootstrap summaries. Together, these examples illustrate context-dependent directional structure, with the biological endpoint uncertainty and the protocol-defined nature of the poker example stated explicitly. Future work could explore whether similar changes in causal direction arise in climate and geological systems.

We thank the reviewer again for helping us improve the positioning and empirical evaluation of the work. We are happy to take any further questions.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Updated manuscript

We have uploaded an updated revision incorporating the changes discussed in our responses. We thank the reviewers for their constructive feedback, which helped improve the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.