acceptodds
Under review as a conference paper at ICLR 2027

Causal Neural Probabilistic Circuits

Abstract

Concept Bottleneck Models (CBMs) enhance the interpretability of end-to-end neural networks by introducing a layer of concepts and predicting the class label from the concept predictions. A key property of CBMs is that they support interventions, i.e., domain experts can correct mispredicted concept values at test time to improve the final accuracy. However, typical CBMs apply interventions by overwriting only the corrected concept while leaving other concept predictions unchanged, which ignores causal dependencies among concepts. To address this, we propose the Causal Neural Probabilistic Circuit (CNPC), which combines a neural attribute predictor with a causal probabilistic circuit compiled from a causal graph. This circuit supports exact, tractable causal inference that inherently respects causal dependencies. Under interventions, CNPC models the class distribution based on a Product of Experts (PoE) that fuses the attribute predictor's predictive distribution with the interventional marginals computed by the circuit. We theoretically characterize the compositional interventional error of CNPC w.r.t. its modules and identify conditions under which its upper bound is no larger than that of its NPC counterpart. Experiments on four datasets provide evidence that CNPC propagates interventions consistently with the encoded causal structure and achieves competitive intervention efficiency against five baselines.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Concept Bottleneck Models, Neuro-Symbolic AI, Neuro-Symbolic Predictors, Causal Inference, Causal Reasoning, Probabilistic Circuits, AI Interpretability

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_kKe7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Causal Neural Probabilistic Circuits (CNPCs) combine a neural attribute predictor with a causal probabilistic circuit. A product of experts (PoE), derived as a reverse KL-barycenter, fuses the clamped predictive distribution with the interventional marginal to approximate the input-conditioned interventional class distribution. The paper provides compositional error bounds and experiments on four datasets.

Critical strengths

1.The separation between input-dependent evidence and the causal PC is clear. Section 4.1 explicitly acknowledges the difficulty of obtaining the exact interventional conditional.

2.The oracle comparison, fixed-PoE-weight evaluation, learned-graph experiments, and domain-shift results are useful. The theoretical analysis states conditions for a tighter error bound rather than a universal accuracy guarantee.

Critical weaknesses

1.The main evaluation does not directly validate the interventional target. Appendix D.1.3 corrects attributes to their factual labels and evaluates against unchanged test labels. Figure 3 provides an oracle comparison, but the learned model's correction accuracy does not establish its accuracy as an input-conditioned interventional distribution.

2.Causal consistency after PoE fusion remains unclear. Even an independent, non-intervened attribute can have its predictive distribution changed by the fusion. Non-descendant accuracy can miss such probability changes, so it is insufficient to establish the intended intervention-propagation behavior.

3.The evidence for a general benefit from propagation is modest. Table 2 shows small task-accuracy gains over NPC, while Figure 10 favors zero PoE weight for several Sachs intervention budgets. Since CNPC also replaces the original PC, the contribution of propagation needs clearer separation from the circuit change.

Questions for authors

1.How is the exact input-conditioned target in Figure 3 computed, including the mechanism for X? Can the learned CNPC be evaluated against that same target?

2.Can the authors summarize the existing zero-weight and positive-weight results with the causal PC held fixed, and check probability changes on an independent non-descendant attribute?

3 comments

S1: The separation between input-dependent evidence and the causal PC is clear. Section 4.1 explicitly acknowledges the difficulty of obtaining the exact interventional conditional.

S2: The oracle comparison, fixed-PoE-weight evaluation, learned-graph experiments, and domain-shift results are useful.

We sincerely thank the reviewer for recognizing the clarity of our formulation and our explicit discussion of the difficulty of obtaining the exact interventional conditional. We also appreciate the positive assessment of our complementary experiments.

Q1+W1: How is the exact input-conditioned target in Figure 3 computed, including the mechanism for X? The main evaluation does not directly validate the interventional target. Can the learned CNPC be evaluated against that same target?

Thank you for the questions! We will answer them separately.

How is the exact interventional conditional computed?

In Figure 3, the exact interventional conditional is computed from the known synthetic data-generating process. We first sample attributes from the source Bayesian network and generate the input through

X=(1−ρ)g(A)+ρϵ−ms,ϵ∼N(0,I),ρ=0.5,X=\frac{(1-\rho)g(A)+\rho\epsilon-m}{s}, \qquad \epsilon\sim\mathcal N(0,I),\quad \rho=0.5,

where g(A)g(A) is the saved 32-dimensional autoencoder embedding, and mm and ss are the feature-standardization statistics fitted on the training split; division is elementwise. Thus, p(X∣A=a)p(X\mid A=a) is a known diagonal Gaussian with mean ((1−ρ)g(a)−m)/s((1-\rho)g(a)-m)/s and coordinate-wise standard deviation ρ/s\rho/s. In this synthetic generator, by invariance of causal mechanisms, the observation mechanism remains unchanged under the intervention, i.e., p(x∣a)=p(x∣a,do(Aj=v))p(x\mid a) = p(x\mid a,\mathrm{do}(A_j=v)).

Following the definition of an interventional distribution, we obtain p(a∣do(Aj=v))p(a\mid\mathrm{do}(A_j=v)) through truncated factorization by replacing the conditional probability table of AjA_j with a point mass at vv. We then compute

p(a∣x,do(Aj=v))=p(a∣do(Aj=v)) p(x∣a)∑a′p(a′∣do(Aj=v)) p(x∣a′).p(a\mid x,\mathrm{do}(A_j=v)) = \frac{ p(a\mid\mathrm{do}(A_j=v))\,p(x\mid a) }{ \sum_{a'} p(a'\mid\mathrm{do}(A_j=v))\,p(x\mid a') }.

Why is exact-target KL not used in the main evaluation?

We do not use exact-target KL as a general evaluation metric in the main experiments because the required ground-truth generative model is unavailable for real-image datasets such as CelebA.

How does learned CNPC compare with the exact target on synthetic datasets?

However, it is possible to measure the exact-target KL on the synthetic Asia and Sachs datasets. We therefore additionally evaluate the KL divergence from the exact interventional conditional to the learned CNPC on these datasets. Please refer to our response to Q3 for Reviewer BM38. The main takeaway is that the KL curves still exhibit a U-shaped trend, supporting our motivation for using PoE to address NPC’s limitation in propagating causal interventions.

Q2+W3: Since CNPC also replaces the original PC, the contribution of propagation needs clearer separation from the circuit change. Can the authors summarize the existing zero-weight and positive-weight results with the causal PC held fixed, and check probability changes on an independent non-descendant attribute?

Thanks for suggesting this controlled comparison. We will (1) clarify the differences between the model variants, and (2) isolate PoE’s contribution by comparing CNPC with α=0\alpha=0 and validation-selected α\alpha, while holding the causal PC fixed.

Differences between Model Variants:

All three model variants share the same frozen neural attribute predictor:

  • NPC uses the original PC to model P(Y∣A1:K)P(Y\mid A_{1:K}), together with the clamped neural attribute distribution.
  • CNPC with α=0\alpha=0 replaces the original PC with the CPC for P(Y∣A1:K)P(Y\mid A_{1:K}), while retaining the same clamped neural attribute distribution.
  • CNPC with validation-selected α\alpha holds this CPC fixed and replaces the clamped attribute distribution with the PoE formulation.

Thus, comparing the last two variants can isolate the effect of PoE. We retain the paper’s validation-selection protocol, which maximizes overall attribute-and-class accuracy separately for each dataset, seed, and intervention count. The selected α\alpha value can be zero.

Zoom-in on PoE’s Contribution:

Table 11. Validation-selected CNPC minus CNPC(α=0\alpha=0), averaged equally over partial intervention counts 1≤k<K1\leq k<K and three seeds.

DatasetΔ Non-desc.Δ DownstreamΔ Attr.Δ Task
Asia+0.16+0.54+0.13+0.23
Sachs+0.00+1.58+0.63−0.24
MNISTAdd—+0.09+0.04+0.09
CelebA+0.00+0.48+0.19+0.79

As requested, we also disaggregated the non-descendant accuracy changes. Under the paper’s cumulative intervention orders, non-descendant attributes remain only at k=1k=1:

DatasetCorrected attributeNon-descendant attributeAccuracy change (pp)
AsiaAsiaSmoke+0.10
AsiaLung+0.07
AsiaBronc+0.30
SachsPKCPlcg+0.00
PKCPIP3+0.00
PKCPIP2+0.00
CelebASmilingWearing_Lipstick+0.00

(1) Contribution to attribute prediction. The gains in attribute prediction equal those in the original NPC comparison (see Table 1) because NPC and CNPC(α=0\alpha=0) use the same clamped neural attribute distribution. The downstream improvements therefore arise from PoE in this controlled comparison, supporting its benefit for predicting downstream attributes after intervention.

(2) Contribution to class prediction. PoE improves average partial-intervention task accuracy on Asia, MNISTAdd, and CelebA, but decreases it on Sachs. On Sachs, the simultaneous attribute improvement and task decrease are consistent with a tradeoff under our validation objective: overall accuracy weights each of the ten attributes and the class equally, so improved attribute prediction need not coincide with improved task prediction.

Overall, this controlled experiment shows that the proposed PoE formulation improves average downstream attribute accuracy across all four datasets and average class prediction accuracy on three of them.

W2: Causal consistency after PoE fusion remains unclear. Even an independent, non-intervened attribute can have its predictive distribution changed by the fusion. Non-descendant accuracy can miss such probability changes, so it is insufficient to establish the intended intervention-propagation behavior.

We thank the reviewer for sharing this concern. We clarify below (1) the scope of our causal guarantee and (2) what the empirical results establish.

The causal guarantee applies to the interventional marginal computed by the causal PC. Within the encoded causal model, an intervention leaves the joint marginal of non-intervened non-descendants unchanged; its effects propagate only to descendants. However, the final predictive distribution combines this interventional marginal with the clamped neural predictor through PoE fusion (see L320). This reweighting does not generally preserve non-descendant predictive distributions and can change their probabilities. Thus, the circuit’s guarantee does not automatically extend to the fused predictions.

Table 1 nevertheless provides evidence that the predictive benefits are concentrated downstream, supporting the intended propagation behavior at the level of accuracy. We agree, however, that accuracy alone cannot establish invariance of the underlying predictive distributions. We thus characterize the PoE prediction rule as providing soft causal guidance, without claiming strict non-descendant invariance after fusion. We will incorporate the clarification into Section 4.

Thank you again for your constructive feedback, which has certainly helped us improve the clarity and positioning of our paper and provided us with an opportunity to further evaluate and demonstrate the effectiveness of the learned CNPC and the proposed PoE fusion. We sincerely appreciate your input. Please feel free to let us know if you have any further questions; we are keen on engaging.

Reviewer_PFXWOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors focus on making interventions causally relevant in the space of interpretable models, such as Concept Bottleneck Models and Neural Probabilistic Circuits. Specifically, they show how to make Neural Probabilistic Circuits handle concept interventions in a causally consistent way, deriving this property directly from first principles. They compare this methodology with non-causal Neural Probabilistic Circuits, as well as with different variations of Concept Bottleneck Models.

Critical strengths
  • The problem of ensuring that interventions in such networks do influence other causally related factors is crucial. This matters both from a simulation perspective (you are not changing factors that should change) and from a practical one (you want to perform as few corrections as possible while ending up in meaningful states), and the authors motivate this substantially.
  • The authors tackle the problem from first principles, showing how this architecture follows directly from deriving the desired property for Neural Probabilistic Circuits. They also show that Neural Probabilistic Circuits are a special case of the proposed Causal Neural Probabilistic Circuits, which makes perfect sense.
  • It is generally well written and easy to follow (even though some parts regarding the mathematical derivations and supporting material might be moved to the appendix or simplified to avoid breaking the reader's flow, as they require the reader to stop and reason more carefully about them, even when they are not crucial. e.g., Section 5). It is also well structured in terms of experimental setup and results. The authors use standard experiments and metrics, together with good baselines.
Critical weaknesses
  • My main concern is about novelty and relevance to the community. On the one hand, I agree that it is a natural advancement over NPCs, as Causal CGM was for CBMs. On the other hand, this problem has already been tackled by many other papers in the realm of CBMs. You mention that C2BM and Causal CGM only model linear relationships. Causal CGM allows for MLPs instead of linear heads, while C2BM is parameterized by linear layers that are instantiated on the fly with respect to the input, allowing them to approximate nonlinear functions. On top of that, CMR and HCMR (the causal version of CMR) approximate functions through logic rules and are universal approximators. What advantages do CNPCs offer over such methods? Why wouldn't other trivial solutions work (e.g., given a causal graph, directly creating multiple NPCs/PCs according to the graph, each approximating a child node given its parent nodes)?
  • The solution is sound, as highlighted above, but does it really move the needle? Even in terms of results, all the models seem pretty similar. Are there any specific experiments that show scenarios or practical advantages of your method over causal CBM-like baselines?
Questions for authors

In addition to my questions given in "Weaknesses", I would add:

  • How do you select the order of interventions in CBMs, since they are all at the same depth? Do you determine the order based on your causal graph and apply it to all models?
  • Does C2BM use the same graph as your model?
  • How is it possible that non-descendant nodes are affected by the intervention in your solution?
  • How did you choose K? Is K the total number of concepts?

UPDATE

The authors' clarifications and additional experiments address my main concerns, I have raised my score from 2 to 3.

6 comments

We sincerely thank the reviewer for recognizing the motivation behind our work and the principled derivation of our method. We also appreciate the positive assessment of the paper’s clarity and organization, as well as the experimental setup and choice of baselines.

W1: What advantages do CNPCs offer over such methods? Why wouldn't other trivial solutions work (e.g., given a causal graph, directly creating multiple NPCs/PCs according to the graph, each approximating a child node given its parent nodes)?

We thank the reviewer for this question. We respond by (1) clarifying our objective, (2) explaining CNPC’s practical and theoretical benefits, and (3) discussing the proposed alternative construction.

Clarification of our objective:

Our starting point is NPC’s inability to propagate concept interventions through causal dependencies. As the reviewer observes, CNPC is a natural extension of NPC in this direction. More broadly, we aim to offer the community a complementary approach to incorporating causal knowledge into concept-based prediction, without claiming to resolve the limitations of existing causal CBMs or to consistently outperform them.

CNPC's practical and theoretical benefits:

We agree that expressiveness alone does not distinguish CNPC from existing causal CBMs. When compared with them, our contribution concerns modular architectural design and compositional error analysis:

(1) Updating causal knowledge without retraining the neural attribute predictor. Thanks to its modular design, for a fixed set of attributes, CNPC can incorporate a revised causal structure by recompiling the causal PC and estimating its parameters while retaining the neural predictor. Appendix D.2.4 (L1121–1126) has demonstrated this with a learned graph; we will further emphasize this capability in the main paper. This separation is useful since causal knowledge may evolve in practice.

(2) An explicit compositional bound on interventional prediction error. CNPC extends NPC’s compositional error analysis to the PoE-based interventional predictor. Under the stated assumptions, its expected KL error is bounded by the error from the neural predictor and the error from the causal PC. This relates component quality to the overall prediction error—a guarantee that is not demonstrated for existing approaches.

Overall, CNPC allows the causal module to be updated independently of the visual predictor and provides a compositional bound on interventional prediction error. We present these properties as complementary contributions alongside existing approaches.

Discussion on the proposed alternative construction:

Regarding the suggested construction, learning a conditional PC for each node given its parents is a valid alternative starting point. Properly normalized local conditionals arranged according to a DAG define a Bayesian network. The issue is whether the resulting model supports tractable exact inference: tractability of each local component does not automatically imply tractability of marginal inference in the full model. In particular, directly multiplying these components need not produce a decomposable circuit because their scopes overlap. For example, the factors p(A)p(A), p(B∣A)p(B\mid A), and p(C∣A)p(C\mid A) all involve AA. Without appropriate structural guarantees, such as smoothness and decomposability, exact marginal and conditional inference can remain intractable even when queries to individual components are tractable.

In contrast, the causal PC used in CNPC is constructed through a principled compilation procedure that supports exact queries under the encoded causal model, with inference time linear in the compiled circuit’s size.

W2: Are there any specific experiments that show scenarios or practical advantages of your method over causal CBM-like baselines?

We thank the reviewer for acknowledging the soundness of our method. As discussed in our response to W1, a key practical benefit of CNPC is its modular design, which allows revised causal knowledge to be incorporated without retraining the neural attribute predictor.

Appendix D.2.4 provides a direct demonstration. We replace the original causal graph with one learned from data. For CNPC, we compile the revised graph into a causal PC, estimate its parameters, and reuse the existing neural predictor. In contrast, C²BM is retrained with the revised graph because its architecture depends on that structure.

This experiment illustrates a practical benefit beyond differences in predictive accuracy: when causal knowledge over the same attributes evolves, CNPC can update its causal module while avoiding neural retraining. Such a scenario arises when new data or domain expertise revises the relationships among existing concepts. We will highlight this experiment and its practical implications more clearly in the main paper.

Q1: How do you select the order of interventions in CBMs, since they are all at the same depth? Do you determine the order based on your causal graph and apply it to all models?

Thank you for these questions. Yes, we determine the intervention order from the causal graph underlying each dataset and apply the same order to all models. Following C²BM, we order attributes by their depth in this graph. Here, depth refers to the causal graph, rather than the model architecture, so the same ordering applies to standard CBMs whose concepts occupy a single layer. This protocol aims to ensure a fair comparison and is stated in Section 6.1 (L406–407), with the specific sequence for each dataset provided in Appendix D.1.1. We will highlight this distinction more clearly in the experimental setup.

Q2: Does C²BM use the same graph as your model?

Yes. To ensure a fair comparison, C²BM and CNPC use the same ground-truth causal graph for each dataset.

Q3: How is it possible that non-descendant nodes are affected by the intervention in your solution?

Thank you for raising this question. This behavior arises from the PoE-based attribute prediction rule. As stated in L320, we predict attribute labels from the fused joint distribution:

a^1:K=arg⁡max⁡a1:Kp~θ,w,αdoj(a1:K∣X),\hat a_{1:K} = \arg\max_{a_{1:K}} \tilde p^{\mathrm{do}_j}_{\theta,w,\alpha}(a_{1:K}\mid X),

where

p~θ,w,αdoj(a1:K∣X)=[pθclj(a1:K∣X)]1−α[pwdoj(a1:K)]αZα(X).\tilde p^{\mathrm{do}_j}_{\theta,w,\alpha}(a_{1:K}\mid X) = \frac{ \left[p_\theta^{\mathrm{cl}_j}(a_{1:K}\mid X)\right]^{1-\alpha} \left[p_w^{\mathrm{do}_j}(a_{1:K})\right]^\alpha }{ Z_\alpha(X) }.

Thus, attribute predictions depend on both the clamped neural outputs and the causal PC’s interventional marginal.

Within the encoded causal model, the interventional marginal preserves the distribution of non-intervened non-descendants. However, this invariance does not automatically carry over to the fused predictive distribution: PoE reweights and normalizes the joint attribute distribution, which can change non-descendant predictive probabilities and, consequently, their predicted labels.

Nevertheless, where both groups can be evaluated, Table 1 shows small or zero non-descendant accuracy differences alongside larger downstream accuracy gains. This supports the interpretation that the predictive benefits are concentrated in descendants, consistent with the intended causal hierarchy.

Q4: How did you choose K? Is K the total number of concepts?

KK denotes the total number of the (categorical) attributes used in each experiment, as defined in L175. It is determined by the dataset’s chosen attribute set, rather than tuned as a hyperparameter.

Specifically, our experiments on Asia, Sachs, MNISTAdd, and CelebA use 7, 10, 2, and 5 attributes, respectively, giving K=7,10,2,5K=7,10,2,5. The dataset-specific configurations are detailed in Appendix D.1.1.

Thank you for the clarifications and detailed responses.

Regarding W1 and W2: The objective was clear. However, updating knowledge without retraining the neural attribute predictor is also possible in C2BM or CausalCGM. In these approaches, one could similarly retrain only the affected nodes/edges while leaving the root predictors unchanged, as in CNPC. Could you clarify what specifically distinguishes CNPC in this regard? I also appreciate the discussion of simpler alternatives and their limitations.

Regarding Q3: Does the proposed approach bring CNPC closer to a Stochastic CBM rather than C2BM? In particular, would interventions then potentially affect all concepts, even if only minimally, rather than being constrained (by design) to causal descendants as expected from the underlying causal graph? If so, how is causal consistency preserved?

Reviewer_BM38OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper proposes Causal Neural Probabilistic Circuits (CNPC), combining a neural attribute predictor with a probabilistic circuit compiled from a causal graph. Under attribute interventions, CNPC uses a weighted product of experts (PoE) to combine clamped neural predictions with the circuit’s interventional attribute marginal. The resulting distribution approximates the input-conditioned interventional attribute distribution and is subsequently used for class prediction. The paper derives compositional KL error bounds and compares CNPC with five baselines on four datasets, with additional experiments involving learned graphs, fixed fusion weights, and distribution shift.

Critical strengths
  1. The modular architecture has practical value. CNPC separates attribute recognition from structured probabilistic inference, allowing the structural prior to be replaced while retaining the neural predictor. The learned-graph experiment illustrates this benefit by recompiling the causal module without retraining the neural attribute predictor. Interventional queries within the fitted causal model also have a clear computational implementation.
  2. The evaluation includes several useful complementary analyses. The paper compares against five relevant baselines on four datasets and reports results across three random seeds. The appendix includes experiments with fixed fusion weights, learned graphs, and distribution shift. In particular, the fixed-weight results suggest that some predictive benefits do not depend entirely on selecting a separate weight for each dataset and intervention budget. These findings support the framework’s potential as a practical prediction method.
Critical weaknesses
  1. The theoretical analysis provides general conditional error bounds but does not sufficiently substantiate the proposed interventional approximation. Section 5 establishes compositional KL bounds through general properties of marginalization, KL decomposition, and geometric pooling. These bounds are meaningful conditional on the component errors, but the inequalities themselves do not exploit the compiled circuit structure or establish when the proposed PoE accurately approximates the input-conditioned interventional distribution. In particular, the upper-bound comparison with NPC assumes that the causal expert is at least as close to the true interventional posterior as the clamped neural predictor. The analysis does not derive this advantage from the causal structure, the accuracy of the estimated mechanisms, or the stated training objectives. Similarly, the reverse-KL barycenter formulation justifies PoE as a compromise between its two sources, but does not establish its accuracy relative to the desired causal target. Consequently, the theoretical results primarily describe how assumed component errors propagate through the predictor. They provide limited explanation of why combining an input-dependent attribute prediction with an unconditional interventional marginal should yield an accurate interventional conditional, or when this design should improve upon clamping. This limits their support for the paper’s central methodological contribution.

  2. The experiments do not sufficiently validate the learned PoE approximation or isolate its contribution. Figure 3 directly compares PoE with the true interventional conditional, but does so using exact distributions in place of the learned components and sweeping the fusion weight. This demonstrates that PoE can closely approximate the target on the studied datasets under ideal component specifications and suitable weighting. In practice, CNPC uses imperfect learned components and selects its weight using a validation criterion averaging attribute and class accuracies. The paper lacks experiments connecting these two settings and does not sufficiently assess the discrepancy between the complete learned predictor and the true interventional posterior. At least on Asia and Sachs, where exact reference distributions are already available for Figure 3, the learned model should be evaluated directly, with the intervention settings and averaging procedure specified. Improvements in attribute and class accuracy support predictive utility, but do not establish the accuracy of the underlying probability approximation. Similarly, small changes in non-descendant label accuracy do not establish that the corresponding predictive distributions remain appropriately unchanged. Furthermore, the main NPC–CNPC comparison changes both the circuit structure and fitting procedure and the intervention prediction formula. Consequently, class prediction gains cannot be attributed entirely to PoE. Although the appendix includes weight sweeps, the highest mean task accuracy on Sachs occurs at α=0\alpha=0 for several partial intervention budgets, further motivating a separation of the effects of the probabilistic model and the fusion mechanism. Matched comparisons that fix both experts and compare PoE with other simple fusion rules would help establish whether the proposed fusion form provides an independent advantage. The current experiments therefore do not fully establish the connection between an accurate interventional approximation and the observed predictive improvements.

Questions for authors
  1. Does Eq. (3) additionally assume that the attributes remain sufficient in every relevant interventional environment? How does the observational conditional independence used in Appendix C imply the corresponding independence after intervention?
  2. How exactly is the oracle in Figure 3 constructed? Does the attribute prediction component use the true joint posterior or a product of true marginal posteriors? Which interventions are evaluated, and how are results averaged over inputs?
  3. In synthetic settings with exact reference distributions, what is the KL divergence or another distributional discrepancy between the complete learned CNPC and the true interventional posterior? Does selecting the fusion weight by validation accuracy also produce an accurate distributional approximation?
  4. With the neural predictor and probabilistic circuit fixed, what advantage does PoE provide over simple weighted mixtures or other fusion rules? How should the Sachs results—where several intervention budgets achieve their highest task accuracy at α=0\alpha=0 be interpreted?
6 comments

S1: The modular architecture has practical value. The learned-graph experiment illustrates this benefit by recompiling the causal module without retraining the neural attribute predictor. S2: The evaluation includes several useful complementary analyses. The appendix includes experiments with fixed fusion weights, learned graphs, and distribution shift. These findings support the framework’s potential as a practical prediction method.

We sincerely thank the reviewer for recognizing the practical value of CNPC’s modular architecture. We also appreciate the reviewer’s positive assessment of our complementary analyses.

Q1: Does Eq. (3) additionally assume that the attributes remain sufficient in every relevant interventional environment? How does the observational conditional independence used in Appendix C imply the corresponding independence after intervention?

Thank you for highlighting the distinction between observational and interventional sufficiency. Eq. (3) requires attribute sufficiency in the relevant interventional environments. In Assumption 1, we intended to use the notation ⊥\perp to represent the d-separation ⊥d\perp_d. To make this explicit, we will reformulate Assumption 1 as follows:

Assumption 1 (Structural sufficiency of attributes). The attributes A1:KA_{1:K} d-separate the input XX and class label YY in the underlying causal DAG containing these variables: Y⊥dX∣A1:KY\perp_d X \mid A_{1:K}.

Under the causal Markov property, this condition implies the observational independence Y⊥X∣A1:KY\perp X\mid A_{1:K}. It also justifies the corresponding interventional independence. Specifically, a fixed-value intervention do(Aj=aj)\mathrm{do}(A_j=a_j) removes the incoming edges to AjA_j. Because deleting edges preserves existing d-separations, A1:KA_{1:K} continues to d-separate XX and YY in the intervened graph. Applying the causal Markov property to that graph gives

Y⊥X∣A1:Kunder Pdoj.Y\perp X\mid A_{1:K} \quad\text{under }P^{\mathrm{do}_j}.

The same reasoning applies to interventions on multiple attributes. This justifies removing XX from the class conditional in the second equality of Appendix C.

We will update Assumption 1 in Section 3.1 and add this graphical argument to Appendix C, making explicit how the stated assumption supports both the observational and interventional derivations. Thank you for your comment again!

Q2: How exactly is the oracle in Figure 3 constructed? Does the attribute prediction component use the true joint posterior or a product of true marginal posteriors? Which interventions are evaluated, and how are results averaged over inputs?

We will clarify the oracle construction and experimental settings for Figure 3 below.

Construction of the oracle PoE:

The oracle PoE is constructed by replacing the two learned distributions in Equation (4) with the exact observational attribute distribution and the exact interventional marginal, respectively:

qoracle,α(a∣x,do(Aj=v))=pobs(a∣x)1−αpsource(a∣do(Aj=v))αZα(x,v).q_{\mathrm{oracle},\alpha}(a\mid x,\mathrm{do}(A_j=v)) = \frac{ p_{\mathrm{obs}}(a\mid x)^{1-\alpha} p_{\mathrm{source}}(a\mid \mathrm{do}(A_j=v))^\alpha }{ Z_\alpha(x,v) }.

The exact observational attribute posterior is

pobs(a∣x)=psource(a) p(x∣a)∑a′psource(a′) p(x∣a′),p_{\mathrm{obs}}(a\mid x) = \frac{p_{\mathrm{source}}(a)\,p(x\mid a)} {\sum_{a'}p_{\mathrm{source}}(a')\,p(x\mid a')},

where psource(a)p_{\mathrm{source}}(a) is obtained from the source Bayesian network and p(x∣a)p(x\mid a) is the known Gaussian observation density used to generate the synthetic inputs. The exact interventional marginal, psource(a∣do(Aj=v))p_{\mathrm{source}}(a\mid\mathrm{do}(A_j=v)), is obtained from the same source network by replacing the intervened variable’s conditional probability table with a point mass at vv and applying truncated factorization. It is a joint distribution over the attributes, marginalized over the class label and not conditioned on xx.

This construction allows us to examine the PoE approximation with exactly specified experts, removing parameter-estimation error and the neural predictor’s conditional-independence restriction. Thus, to answer your second question explicitly, the attribute prediction component uses the true joint posterior pobs(A∣x)p_{\mathrm{obs}}(A\mid x), rather than a product of true marginal posteriors.

Experimental Settings:

We intervene on a single attribute in each dataset: asia for Asia and PKC for Sachs, following the first attribute in the intervention order described in L406, by setting this attribute of each test example nn to its ground-truth value, i.e., dn=do(Aj=aj(n))d_n=\mathrm{do}(A_j=a_j^{(n)}).

For each α∈{0,0.1,…,1}\alpha\in\{0,0.1,\ldots,1\}, we compute the forward KL divergence over the complete joint attribute distribution for each example, then report its arithmetic mean over the 1,000 test examples, separately for each dataset:

11000∑n=11000DKL(pexact(A∣xn,dn)∥∥qoracle,α(A∣xn,dn)).\frac{1}{1000}\sum_{n=1}^{1000} D_{\mathrm{KL}}\left( p_{\mathrm{exact}}(A\mid x_n,d_n) \|\| q_{\mathrm{oracle},\alpha}(A\mid x_n,d_n) \right).

Thus, each plotted point averages per-input KL divergences.

Q3: In synthetic settings with exact reference distributions, what is the KL divergence or another distributional discrepancy between the complete learned CNPC and the true interventional posterior? Does selecting the fusion weight by validation accuracy also produce an accurate distributional approximation?

Thank you for the question! We will answer by recalling what Figure 3 describes, describing the setting for the required experiments, and presenting the results and analysis.

Description of Figure 3:

Figure 3 examines PoE fusion using oracle experts: the exact joint observational posterior over attributes and the exact interventional marginal. This removes learned-parameter error and the factorization restriction of the neural attribute predictor, allowing us to study the approximation introduced by combining these distributions.

Experimental Setting:

Here we present the KL divergence between the exact interventional conditional and the learned CNPC.

Computing the exact reference distribution on Asia and Sachs requires the original data-generating model, including the conditional means, noise variances, and feature-standardization parameters defining the Gaussian observation density p(X∣A)p(X\mid A). For Figure 3, we therefore regenerate both datasets that preserve these generation artifacts and retrain CNPC on them.

For Sachs, float32 underflow assigns zero probability to some states with positive exact probability, making forward KL infinite for every tested α>0\alpha>0. We therefore report a numerically stabilized float64 log-space diagnostic using the same learned experts and existing probability floor.

Results and Analysis:

Table 10. KL divergence from the exact interventional conditional to the learned CNPC on Asia and Sachs. The intervened attributes are asia and PKC, respectively. Bold entries indicate validation-selected α\alpha: 0.3 for Asia and 0.2 for Sachs.

αAsiaSachs
0.00.2379001.255944
0.10.1486900.997339
0.20.1346360.906659
0.30.1300200.863157
0.40.1358680.870680
0.50.1536910.935730
0.60.1856261.068688
0.70.2346361.285704
0.80.3048831.611995
0.90.4037492.088018
1.00.5551472.776674

Table 10 shows that:

(1) The minimum KL increases relative to the oracle experiments. This is consistent with the additional estimation error in the learned components and the factorized neural predictor’s inability to represent all conditional dependencies among attributes.

(2) The KL curves still exhibit a U-shaped trend. With an appropriate α\alpha, the PoE approximates the exact interventional conditional more closely than either NPC's clamped neural attribute approximation at α=0\alpha=0 or the causal PC’s interventional marginal at α=1\alpha=1. This supports our motivation for using PoE to address NPC’s limitation in propagating causal interventions.

(3) Validation performance provides a practical mechanism for selecting α\alpha. On Asia, the validation-selected α=0.3\alpha=0.3 is exactly the value that minimizes test KL over the evaluated grid. On Sachs, validation selects α=0.2\alpha=0.2, whose test KL is 0.906659, close to the minimum of 0.863157 at α=0.3\alpha=0.3. Thus, validation selection can select a weight that substantially improves the approximation over both endpoints.

Overall, these experiments support PoE fusion and validation-based weight selection as practical ways to improve the interventional conditional approximation.

Q4: With the neural predictor and probabilistic circuit fixed, what advantage does PoE provide over simple weighted mixtures or other fusion rules? How should the Sachs results—where several intervention budgets achieve their highest task accuracy at α=0\alpha=0 be interpreted?

Thank you for these questions! We will answer them by explaining (1) why we use PoE, (2) why we do not use the alternative fusion rule—mixture of experts (MoE), and (3) the Sachs results.

Why PoE?

Our choice of PoE is motivated by its variational formulation and empirical evidence. As described in L297, PoE is the normalized geometric mean that minimizes the weighted sum of reverse KL divergences to the two experts. This provides a principled formulation for balancing the two distributions. Empirically, Figure 3 shows that, with an appropriate α∈(0,1)\alpha\in(0,1), the oracle PoE closely approximates the exact interventional conditional. At α=0.1\alpha=0.1, its mean KL is 0.003375 on Asia and 0.010407 on Sachs, improving over both endpoints.

Why not MoE?

To evaluate simple weighted mixtures, we keep the same two exact expert distributions fixed and replace geometric fusion with an arithmetic mixture:

qMoE,α(A∣x,d)=(1−α)pobs(A∣x)+αpsource(A∣d).q_{\mathrm{MoE},\alpha}(A\mid x,d) =(1-\alpha)p_{\mathrm{obs}}(A\mid x) +\alpha p_{\mathrm{source}}(A\mid d).

We then compute the forward KL divergence from the same exact interventional conditional to this oracle MoE.

Table 12. KL divergence from the exact interventional conditional to the oracle MoE on Asia and Sachs.

αAsiaSachs
0.00.0214820.379773
0.10.0308890.430756
0.20.0526000.504147
0.30.0813000.591819
0.40.1161840.694903
0.50.1573850.816931
0.60.2056690.964300
0.70.2625621.148651
0.80.3309841.393953
0.90.4175881.764637
1.00.5547532.756055

The table shows that MoE’s KL increases across the tested α values on both datasets, attaining its minimum at α=0\alpha=0. Thus, in this experiment, arithmetic mixing does not improve the distributional approximation. We view this as evidence motivating our choice of PoE, rather than a guarantee that it will outperform MoE on downstream tasks in practice.

Sachs Performance:

We would like to provide both an empirical interpretation and a theoretical perspective on why some intervention budgets on Sachs achieve their highest task accuracy at α=0\alpha=0. Empirically, this suggests that, at these intervention budgets, increasing the weight on the input-independent causal marginal can reduce useful input evidence without a compensating improvement in task accuracy. Theoretically, L363–369 identifies a sufficient condition for CNPC’s interventional error bound to be no larger than NPC’s: the causal PC’s interventional marginal approximates the true interventional posterior at least as accurately as the clamped attribute predictor. When this condition is not satisfied, the bound can be larger. Although comparing upper bounds does not determine the ordering of actual task accuracies, this analysis is consistent with the observation that positive fusion weights do not always improve performance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.