- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Causal Neural Probabilistic Circuits (CNPCs) combine a neural attribute predictor with a causal probabilistic circuit. A product of experts (PoE), derived as a reverse KL-barycenter, fuses the clamped predictive distribution with the interventional marginal to approximate the input-conditioned interventional class distribution. The paper provides compositional error bounds and experiments on four datasets.
Critical strengths
1.The separation between input-dependent evidence and the causal PC is clear. Section 4.1 explicitly acknowledges the difficulty of obtaining the exact interventional conditional.
2.The oracle comparison, fixed-PoE-weight evaluation, learned-graph experiments, and domain-shift results are useful. The theoretical analysis states conditions for a tighter error bound rather than a universal accuracy guarantee.
Critical weaknesses
1.The main evaluation does not directly validate the interventional target. Appendix D.1.3 corrects attributes to their factual labels and evaluates against unchanged test labels. Figure 3 provides an oracle comparison, but the learned model's correction accuracy does not establish its accuracy as an input-conditioned interventional distribution.
2.Causal consistency after PoE fusion remains unclear. Even an independent, non-intervened attribute can have its predictive distribution changed by the fusion. Non-descendant accuracy can miss such probability changes, so it is insufficient to establish the intended intervention-propagation behavior.
3.The evidence for a general benefit from propagation is modest. Table 2 shows small task-accuracy gains over NPC, while Figure 10 favors zero PoE weight for several Sachs intervention budgets. Since CNPC also replaces the original PC, the contribution of propagation needs clearer separation from the circuit change.
Questions for authors
1.How is the exact input-conditioned target in Figure 3 computed, including the mechanism for X? Can the learned CNPC be evaluated against that same target?
2.Can the authors summarize the existing zero-weight and positive-weight results with the causal PC held fixed, and check probability changes on an independent non-descendant attribute?
3 comments
Q2+W3: Since CNPC also replaces the original PC, the contribution of propagation needs clearer separation from the circuit change. Can the authors summarize the existing zero-weight and positive-weight results with the causal PC held fixed, and check probability changes on an independent non-descendant attribute?
Thanks for suggesting this controlled comparison. We will (1) clarify the differences between the model variants, and (2) isolate PoE’s contribution by comparing CNPC with and validation-selected , while holding the causal PC fixed.
Differences between Model Variants:
All three model variants share the same frozen neural attribute predictor:
- NPC uses the original PC to model , together with the clamped neural attribute distribution.
- CNPC with replaces the original PC with the CPC for , while retaining the same clamped neural attribute distribution.
- CNPC with validation-selected holds this CPC fixed and replaces the clamped attribute distribution with the PoE formulation.
Thus, comparing the last two variants can isolate the effect of PoE. We retain the paper’s validation-selection protocol, which maximizes overall attribute-and-class accuracy separately for each dataset, seed, and intervention count. The selected value can be zero.
Zoom-in on PoE’s Contribution:
Table 11. Validation-selected CNPC minus CNPC(), averaged equally over partial intervention counts and three seeds.
| Dataset | Δ Non-desc. | Δ Downstream | Δ Attr. | Δ Task |
|---|---|---|---|---|
| Asia | +0.16 | +0.54 | +0.13 | +0.23 |
| Sachs | +0.00 | +1.58 | +0.63 | −0.24 |
| MNISTAdd | — | +0.09 | +0.04 | +0.09 |
| CelebA | +0.00 | +0.48 | +0.19 | +0.79 |
As requested, we also disaggregated the non-descendant accuracy changes. Under the paper’s cumulative intervention orders, non-descendant attributes remain only at :
| Dataset | Corrected attribute | Non-descendant attribute | Accuracy change (pp) |
|---|---|---|---|
| Asia | Asia | Smoke | +0.10 |
| Asia | Lung | +0.07 | |
| Asia | Bronc | +0.30 | |
| Sachs | PKC | Plcg | +0.00 |
| PKC | PIP3 | +0.00 | |
| PKC | PIP2 | +0.00 | |
| CelebA | Smiling | Wearing_Lipstick | +0.00 |
(1) Contribution to attribute prediction. The gains in attribute prediction equal those in the original NPC comparison (see Table 1) because NPC and CNPC() use the same clamped neural attribute distribution. The downstream improvements therefore arise from PoE in this controlled comparison, supporting its benefit for predicting downstream attributes after intervention.
(2) Contribution to class prediction. PoE improves average partial-intervention task accuracy on Asia, MNISTAdd, and CelebA, but decreases it on Sachs. On Sachs, the simultaneous attribute improvement and task decrease are consistent with a tradeoff under our validation objective: overall accuracy weights each of the ten attributes and the class equally, so improved attribute prediction need not coincide with improved task prediction.
Overall, this controlled experiment shows that the proposed PoE formulation improves average downstream attribute accuracy across all four datasets and average class prediction accuracy on three of them.
W2: Causal consistency after PoE fusion remains unclear. Even an independent, non-intervened attribute can have its predictive distribution changed by the fusion. Non-descendant accuracy can miss such probability changes, so it is insufficient to establish the intended intervention-propagation behavior.
We thank the reviewer for sharing this concern. We clarify below (1) the scope of our causal guarantee and (2) what the empirical results establish.
The causal guarantee applies to the interventional marginal computed by the causal PC. Within the encoded causal model, an intervention leaves the joint marginal of non-intervened non-descendants unchanged; its effects propagate only to descendants. However, the final predictive distribution combines this interventional marginal with the clamped neural predictor through PoE fusion (see L320). This reweighting does not generally preserve non-descendant predictive distributions and can change their probabilities. Thus, the circuit’s guarantee does not automatically extend to the fused predictions.
Table 1 nevertheless provides evidence that the predictive benefits are concentrated downstream, supporting the intended propagation behavior at the level of accuracy. We agree, however, that accuracy alone cannot establish invariance of the underlying predictive distributions. We thus characterize the PoE prediction rule as providing soft causal guidance, without claiming strict non-descendant invariance after fusion. We will incorporate the clarification into Section 4.
Thank you again for your constructive feedback, which has certainly helped us improve the clarity and positioning of our paper and provided us with an opportunity to further evaluate and demonstrate the effectiveness of the learned CNPC and the proposed PoE fusion. We sincerely appreciate your input. Please feel free to let us know if you have any further questions; we are keen on engaging.
We sincerely thank the reviewer for recognizing the clarity of our formulation and our explicit discussion of the difficulty of obtaining the exact interventional conditional. We also appreciate the positive assessment of our complementary experiments.
Thank you for the questions! We will answer them separately.
How is the exact interventional conditional computed?
In Figure 3, the exact interventional conditional is computed from the known synthetic data-generating process. We first sample attributes from the source Bayesian network and generate the input through
X=s(1−ρ)g(A)+ρϵ−m,ϵ∼N(0,I),ρ=0.5,
where g(A) is the saved 32-dimensional autoencoder embedding, and m and s are the feature-standardization statistics fitted on the training split; division is elementwise. Thus, p(X∣A=a) is a known diagonal Gaussian with mean ((1−ρ)g(a)−m)/s and coordinate-wise standard deviation ρ/s. In this synthetic generator, by invariance of causal mechanisms, the observation mechanism remains unchanged under the intervention, i.e., p(x∣a)=p(x∣a,do(Aj=v)).
Following the definition of an interventional distribution, we obtain p(a∣do(Aj=v)) through truncated factorization by replacing the conditional probability table of Aj with a point mass at v. We then compute
p(a∣x,do(Aj=v))=∑a′p(a′∣do(Aj=v))p(x∣a′)p(a∣do(Aj=v))p(x∣a).
Why is exact-target KL not used in the main evaluation?
We do not use exact-target KL as a general evaluation metric in the main experiments because the required ground-truth generative model is unavailable for real-image datasets such as CelebA.
How does learned CNPC compare with the exact target on synthetic datasets?
However, it is possible to measure the exact-target KL on the synthetic Asia and Sachs datasets. We therefore additionally evaluate the KL divergence from the exact interventional conditional to the learned CNPC on these datasets. Please refer to our response to Q3 for Reviewer BM38. The main takeaway is that the KL curves still exhibit a U-shaped trend, supporting our motivation for using PoE to address NPC’s limitation in propagating causal interventions.