acceptodds
Under review as a conference paper at ICLR 2027

HiBCD: Scalable Bayesian Causal Discovery for High-Dimensional Data

Abstract

Bayesian causal structure learning aims to infer causal graphs from observational data based on Bayesian posterior inference. However, the combinatorial nature of the graph space makes these methods computationally intractable for high-dimensional settings where the number of variables is large. To address this challenge, we propose HiBCD, a divide-and-conquer Bayesian causal discovery method for high-dimensional variables. The method first employs a hierarchical weighted sampling strategy to partition the high-dimensional variable set into several smaller, manageable subsets. Subsequently, for each subset, we infer a posterior over local causal orderings using a pre-trained meta estimator. The meta estimator features a symmetry-aware encoder and a Bayesian scoring head and is trained via a variational objective to ensure robustness to latent confounders. Finally, these local orderings are iteratively aggregated into a global causal topology, from which the full causal graph can be efficiently recovered. Experiments demonstrate that HiBCD significantly outperforms previous Bayesian causal discovery methods on large-scale data in terms of structural accuracy, while maintaining highly scalable computational efficiency.

Primary area:
causal reasoning
Keywords:
Bayesian Causal Discovery, High-dimensional Causal Structure Learning, Divide-and-conquer Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_VkfWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes HiBCD, a Bayesian causal discovery method for high-dimensional data. The method follows a divide-and-conquer strategy that repeatedly samples small subsets of variables, predicts local ancestral relations using a pre-trained amortized estimator, and aggregates the resulting local ordering information into a global causal order. Given the inferred global orders, the method further performs Bayesian edge scoring and model averaging to recover a global DAG. The method is evaluated on synthetic linear and nonlinear datasets as well as several benchmark Bayesian networks.

Critical strengths

The paper presents an interesting divide-and-conquer approach to scaling Bayesian causal discovery. In particular, predicting local ancestral relations rather than direct edges is a reasonable design choice when variables outside a sampled subset are effectively unobserved in the local problem. The empirical results also show promising structural recovery performance on graphs with hundreds of variables.

Critical weaknesses
  • The key local ancestral inference step lacks theoretical justification. The paper does not establish when ancestral relations in the full DAG are identifiable from the marginal distribution over a sampled subset, particularly when variables outside the subset induce latent confounding.

  • The local-to-global reconstruction lacks sufficient theoretical guarantees. The sampler guarantees node coverage, but not sufficient pairwise coverage, and no consistency guarantees are provided for recovering the global causal order or graph from local ancestral predictions.

  • The Bayesian interpretation of the framework is not fully justified. The local objective is a variational surrogate rather than an ELBO, while the final graph is obtained through order aggregation, consensus projection, and thresholding. It is therefore unclear whether the resulting probabilities can be interpreted as a well-defined posterior over causal graphs.

  • The scalability evaluation is incomplete. Although scalability is a central motivation of the paper, no quantitative runtime comparison with the competing methods is provided. Figure 2 only reports the runtime of HiBCD, and the efficiency evaluation further uses R=1R=1, whereas the main nonlinear experiments use R=128R=128. More importantly, several relevant baselines are missing, including Dong et al. (2025), Gu & Zhou (2020), Fast Scalable and Accurate Discovery of DAGs Using the Best Order Score Search and Grow-Shrink Trees (NeurIPS 2023), Large-Scale Differentiable Causal Discovery of Factor Graphs (NeurIPS 2022), and Effective Bayesian Causal Inference via Structural Marginalisation and Autoregressive Orders (AISTATS 2025). The first two are already discussed in the paper as motivating large-scale causal discovery methods.

  • The empirical evaluation does not sufficiently validate the claimed robustness and generality. The synthetic experiments are mainly limited to sparse ER DAGs with additive Gaussian noise. In addition, different baselines use different sample sizes and evaluation protocols, while the real-world benchmarks employ substantially different dataset-specific hyperparameters, making the empirical comparisons less controlled.

Questions for authors

Please see the weaknesses above for my main concerns.

Could the authors further validate the method on genuinely real high-dimensional biological data? For example, the Perturb-CITE-seq dataset from Multimodal pooled Perturb-CITE-seq screens in patient models define mechanisms of cancer immune evasion (Nature Genetics, 2021).

Reviewer_LBAWOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes HiBCD, a divide-and-conquer approach to bayesian causal discovery for high-dimensional data. The method repeatedly selects small subsets of variables and uses a pretrained estimator to predict ancestral relations and local causal orderings. These predictions are combined to infer causal orders over all variables. The method then uses bayesian conditional models to estimate edge probabilities under the inferred orders and combines the results to recover a DAG.

The main contribution is the combination of amortized local inference and order-based graph recovery to improve scalability. The authors evaluate HiBCD on linear and nonlinear synthetic datasets and three benchmark bayesian networks, comparing it with BCNP, AVICI, and DrBO. They also examine the effects of the encoder, sampling strategy, and subset size, and report inference time. HiBCD achieves lower SHD and higher F1 than the baselines in most evaluated settings.

Critical strengths
  1. Combining local ancestral inference with graph recovery HiBCD estimates ancestral relations on small subsets rather than inferring direct edges from each subset. This is important because causal paths may involve variables outside the selected subset. The method combines these local predictions to infer causal orderings and then uses bayesian conditional models to recover direct edges. Since the estimator operates on small subsets, the same pretrained model can be used for graphs with different numbers of variables.

  2. Graph recovery accuracy on larger graphs HiBCD achieves lower SHD and higher F1 than the baselines in most evaluated settings, with larger improvements in nonlinear settings. For example, with 200 variables, HiBCD achieves an F1 of 0.634, compared with 0.172 for AVICI. These results provide evidence that the proposed approach can improve graph recovery accuracy in high-dimensional settings.

Critical weaknesses
  1. Loss of information about dependence between variables in the estimator In Section 3.3, the estimator applies attention pooling to each variable separately before combining their representations. As a result, it summarizes each variable without keeping information about which values of different variables were observed together. For example, consider two datasets: • Dataset 1: A = [0, 0, 1, 1], B = [0, 0, 1, 1] • Dataset 2: A = [0, 0, 1, 1], B = [0, 1, 0, 1]

In Dataset 1, A and B are perfectly correlated, while in Dataset 2, they are uncorrelated. However, each variable contains the same values in both datasets. Therefore, the pooling operations produce identical representations, and the estimator predicts the same ancestral probabilities for both datasets.

This limitation may matter when different causal structures have the same marginal distributions. Consider two linear SEMs with independent Gaussian noise of variance 1: • SEM 1: C -> A -> B, with edge coefficients 0.75 and 0.6. • SEM 2: C -> B -> A, with the same edge coefficients. In both SEMs, C has variance 1, while A and B each have variance 1.5625. Since all variables are Gaussian, their marginal distributions are identical. However, A is an ancestor of B in SEM 1, while B is an ancestor of A in SEM 2.

The estimator may use differences in marginal distributions, such as variance, to infer causal orderings. However, these differences cannot distinguish the two structures above. The information needed to distinguish them lies in how the variables vary together, which is not preserved by the separate pooling operations.

This limitation may also affect graph recovery. In Section 3.4, the predicted ancestral probabilities are used to construct the order distribution in Eq. (13). The final graph then retains only edges consistent with the consensus order, as defined in Eq. (16). If the consensus order places B before A when the true graph contains A -> B, that edge cannot be recovered.

Although Figure 3 compares different observation encoders, the experiments do not establish how this limitation affects causal ordering and final graph recovery. In particular, it remains unclear how well the proposed method performs when marginal distributions provide insufficient information about ancestral relations. This is important because errors in the estimated orders can prevent true edges from being recovered.

  1. Limited evaluation of computational efficiency Scalability is one of the main contributions of the paper. However, the accuracy and runtime experiments use different inference settings. According to Table 3, the main experiments use R = 4 or 8 retained orders for linear SEMs and R = 128 for nonlinear SEMs. In contrast, Appendix A.3 states that the runtime evaluation uses R = 1 and does not use the nonlinear joint-chain inference budget from the main experiments.

Therefore, the runtime in Figure 2 does not represent the computational cost of achieving the SHD and F1 results in Tables 1–2. Figure 2 also reports only the runtime of HiBCD, without comparisons with BCNP, AVICI, or DrBO. The reported results show improvements in graph recovery accuracy, but it remains unclear how much computation is required to achieve this accuracy and how the runtime compares with existing methods. This makes it difficult to assess the claimed scalability under the main experimental settings.

Questions for authors
  1. How does the estimator perform when marginal distributions provide insufficient information about causal order? It would be helpful to examine settings where different causal structures have similar or identical marginal distributions. How does the estimator perform in these cases, and how do errors in ancestral predictions affect final graph recovery? A comparison with an encoder that preserves information about how variables vary together, while keeping the other experimental settings fixed, would help assess this limitation.

  2. What is the computational cost of achieving the reported graph recovery accuracy? What is the total inference time under the configurations used in Tables 1–2, including the nonlinear joint-chain inference procedure? It would also be useful to know how SHD and F1 change when using the reduced inference budget from Figure 2. In addition, a runtime comparison with BCNP, AVICI, and DrBO on the same datasets would help assess the computational efficiency of HiBCD relative to existing methods.

  3. How does the Aggregator compute edge probabilities and assign probabilities to causal orders? In Section 3.4, Eq. (14) defines a posterior distribution over regression coefficients and noise variance. The paper then introduces the posterior probability of the edge event E_ij, but it is unclear how this probability is obtained from the regression posterior. Does E_ij represent the inclusion of a variable in the parent set, or is it defined using the magnitude of a regression coefficient? A precise definition of this event and its probability calculation would help clarify what the estimated edge probabilities represent.

I would also appreciate clarification of the relationship between the order distributions in Eqs. (13) and (15). Does Eq. (15) use the order distribution from Eq. (13) directly, or are the order probabilities updated using the marginal likelihood of the conditional models? Please clarify how the order probabilities used for bayesian model averaging are obtained.

Reviewer_5suBOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

HiBCD samples small variable subsets, applies an amortized estimator of ancestral relations, and combines local evidence into global orderings. Conditional models then estimate edges, with averaging over retained orders. Adaptive sampling emphasizes variables inferred to occur early in causal orderings.

Critical strengths
  1. Combining local ancestral inference with global reconstruction is a plausible way to reduce the computational difficulty of high-dimensional structure learning. Modeling ancestry also acknowledges that omitted variables can mediate paths within a sampled subset.
  2. The paper reports linear, nonlinear, and benchmark-network experiments, and the appendix supplies useful architecture and inference-budget details.
Critical weaknesses
  1. The local estimator discards observation pairing before modeling relationships between variables.
  • Architecture. Section 3.3, Appendix A.2, and Figure 1 describe the same pipeline: embed each scalar observation, pool each variable's observations independently into a CLS representation, and only then apply attention across variables.
  • Consequence. The paper states invariance to a shared reordering of observations. Because the shared embedding and CLS pooling act on each column separately, the estimator is also unchanged when observations are permuted independently within each column. That operation can destroy the joint dependence relevant to ancestry.
  • Scope. Marginal distributions can contain ordering information under particular simulation assumptions, but the paper does not establish sufficient robustness beyond those assumptions. This objection concerns the local ancestral estimator; the later conditional regressions do use joint observations.
  1. The reported accuracy-efficiency advantage is not established under sufficiently comparable configurations.
  • Observation counts. Main HiBCD experiments use 400 observations, while appendix baseline configurations specify 500 or 1,000 observations and differing repetition counts. The larger observation counts for baselines do not by themselves establish a bias favoring HiBCD, and they do not explain its reported advantage. They do prevent a clean matched-dataset comparison.
  • Retained orders. The efficiency evaluation uses one retained order, whereas the main nonlinear accuracy experiments retain 128. The paper therefore does not identify one operating configuration that simultaneously achieves the headline accuracy and runtime advantages.
  1. There is also a concrete specification issue: Equations 5-6 update sampling weights toward scores in [−0.5,0.5][-0.5,0.5]. Repeated low-ranked updates can therefore produce negative weights, but no positivity transformation is specified for the weighted reservoir sampling, which requires positive weights. This establishes an omission in the written algorithm, not an observed implementation failure.
Questions for authors
  1. Is there any operation before per-variable pooling that preserves aligned observations across variables? If the description is accurate, how does local ancestry estimation perform after standardization, variable rescaling, and independent column permutations?
  2. Can all methods be compared on the same generated datasets, observation counts, and repetition protocol, with runtime measured for the exact configurations producing the accuracy results?
  3. How are sampling weights kept valid? Figure 4 suggests that removing variance-based initialization worsens reverse-pair convergence; how does it affect final SHD and F1?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.