- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies image generation under hidden confounding, where observational data may admit different intervention predictions. It introduces CauVaDE, a latent causal model with a discrete confounder, and proves an approximation result under regularity assumptions. It then varies a cluster-posterior entropy weight to generate alternative fitted models and intervention distributions while checking observational fit. Experiments on Color-MNIST, CelebA, and MIMIC-CXR compare predictions against causal bounds or unconfounded references. The method produces an exploratory collection of candidate causal explanations, without characterizing the full remaining uncertainty.
Critical strengths
- The paper provides a causally structured representation. It explicitly distinguishes observational conditioning from intervention and accommodates hidden confounding. Theorem 2.2 supplies an approximation guarantee for finite-confounder models under regularity assumptions.
- The method offers a practical procedure for exploring alternative candidate explanations. The entropy sweep generates different intervention predictions while monitoring observational fit. An incomplete candidate family could be valuable for finding counterexamples, guiding experimental design, or stress-testing decisions, provided its admissibility is established and its exploratory scope is explicit (see Weaknesses).
- The paper is mostly well-written, and the theoretical results appear sound.
Critical weaknesses
-
The paper does not establish that generated diversity reflects remaining causal uncertainty. For a fixed intervention, its candidate spread has neither demonstrated coverage of the identified set nor justified probabilistic weighting over mechanisms. Different interventions already produce different images, even deterministically. Exploratory candidate generation can be useful, but greater spread alone does not establish more informative causal inference; its relationship to the uncertainty permitted by the evidence needs assessment.
-
The connection between Eqs. (5)–(6), identification, and causal inference remains insufficiently established. The assumed structural model defines how to compute intervention predictions, but intervention semantics alone do not establish that fitted mechanisms satisfy the evidence. The paper should explain how its objective constrains learned mechanisms using the available information, and which aspects of their intervention predictions are thereby determined or remain unresolved. The gap concerns the causal justification of the fitted models, beyond whether they generate plausible images or respond to entropy regularization.
-
The connection between the local theory and implemented training remains unestablished. Theorem 3.2 assumes a positive-definite baseline Hessian and a nonzero interventional response, without establishing their applicability to the fitted networks. Any dedicated parameters omitted from the loss would also introduce flat directions. Moreover, the theorem follows a single smooth branch near zero entropy weight, whereas separate training runs at finite weights need not follow that branch or inherit its conclusions.
-
The evaluation does not isolate target nonidentification from estimation, regularization, or optimization error. Failure to recover latent confounders does not establish nonidentification of intervention predictions. The paper should distinguish persistent ambiguity allowed by the maintained evidence from disagreement caused by imperfect fitting, and test whether additional identifying information contracts the candidate predictions appropriately.
Writing issues. Some explanations needed to follow the argument are deferred to the appendix. Example 1 should state its outcome mechanism and treatment-assignment probabilities. Between Eqs. (3) and (4), the main text should explain that Y is unobserved and integrated out, defining the resulting image likelihood before using it in the ELBO.
Questions for authors
I think the authors should address two of 1 ~ 3; addressing all three will make a strong paper.
-
Please specify the information set defining admissible mechanisms and explain what uncertainty statements the candidate family supports. Are candidates assigned plausibility weights, or are they illustrative scenarios? Please also define Fig. 2’s error bars and distinguish their meaning from variation across entropy weights.
-
On tractable benchmarks with known identified sets, can you separately assess model admissibility, target-set membership, and coverage, including missed regions? Include an identified target and test whether disagreement for a fixed intervention disappears (in the correctly specified limit) as observational fitting errors vanish.
-
How does Theorem 3.2 apply to the implemented training procedure? Please justify assumptions (A4)–(A5) for the fitted networks, clarify how possible Hessian flat directions are handled, and assess whether separate training runs at finite entropy weights follow the local branch analyzed by the theorem.
-
Within the same model class, how does the candidate family respond to additional valid identifying information? Please add labels, graph restrictions, or intervention data and assess whether the generated spread tracks the resulting contraction of admissible predictions.
4 comments
Reply [2/3]
W3 / Q3: Thm. 3.2 assumes a positive-definite baseline Hessian and a nonzero interventional response without establishing them for the fitted networks; parameters omitted from the loss would add flat directions, and separate runs at finite entropy weights need not follow the analyzed branch.
We agree, and we give the full answer in our response to Reviewer iz6K, W2 (a). In brief, Thm. 3.2 is a local motivation for the regularizer, not a guarantee for the trained networks, and we do not verify (A4)–(A5) for them. (A4) as stated fails because of reparameterization symmetries and empty components; a version on the quotient by these symmetries is plausible but not proved. On parameters omitted from the loss, the treatment head is the generative treatment mechanism and is trained by (iz6K, W1 (a)), so the treatment mechanism has no parameters outside the loss. Candidates at finite are trained independently and need not follow the analyzed branch. We have not obtained a reliable small- continuation for this rebuttal. Admissibility of the reported candidates therefore rests only on the observational check, not on Thm. 3.2.
W4 (a) / Q2: The evaluation does not separate nonidentification from estimation, regularization, or optimization error, and failure to recover latent confounders does not establish nonidentification of intervention predictions. On benchmarks with known identified sets, can admissibility, target-set membership, and coverage (including missed regions) be assessed separately? Include an identified target and test empirically whether disagreement vanishes as observational fitting error goes to zero.
We agree that failing to recover (Table 11) does not establish nonidentification, and we will remove that argument from App. E.5.3 and FAQ 4. On Confounded Color-MNIST, nonidentification follows from the construction itself. The images are generated from alone; carries no further information about . Every value inside the Manski bound is therefore attained by some SCM compatible with the evidence.
At , restarts with the same configuration differ by 0.030 and 0.067 (10 seeds). This bounds the disagreement that optimization alone produces at a fixed setting. The accepted candidates across differ 4–8 times more (iz6K, Q3).
Admissibility, membership, coverage (Color-MNIST, Table 1):
- Admissibility: every in Table 1 passes the check in iz6K, W1 (b).
- Membership: all candidate predictions lie inside the Manski region.
- Coverage: the candidates span 0.49–0.72 for , 43% of the Manski width, and 0.54–0.83 for , 63%. The missed regions are and for , and and for . The true value for , 0.40, lies in a missed region. The family therefore does not cover the identified set, and we will report these coverage figures in Sec. 4.
Reply [3/3]
Identified target. If a target is a functional of , two models that reproduce exactly assign it the same value, so disagreement must vanish in the correctly specified limit. We evaluated 59 candidates (five configuration/seed groups, each over the grid of Table 4) on the identified target and on (Table R2). On the training set, the identified target differs across all candidates by at most 0.013 and 0.007, and by at most 0.004 once the image-level MMD is within the data-to-data level. Within a configuration, varying changes it by at most 0.013 (Table R3). Over the same candidates, the interventional prediction differs by 0.21–0.45. Disagreement therefore vanishes on the identified target at matched fit, but not on the interventional prediction.
Table R2. Identified vs. non-identified targets as the image-level fit threshold tightens.
| Threshold on | Candidates retained | Spread of , training set | Spread of , held-out set | Spread of |
|---|---|---|---|---|
| none | 59 | 0.013 / 0.007 | 0.099 / 0.007 | 0.408 / 0.450 |
| 30 | 0.013 / 0.004 | 0.099 / 0.006 | 0.377 / 0.336 | |
| 15 | 0.004 / 0.000 | 0.001 / 0.000 | 0.326 / 0.335 | |
| 8 | 0.004 / 0.000 | 0.001 / 0.000 | 0.319 / 0.309 | |
| 4 | 0.002 / 0.000 | 0.000 / 0.000 | 0.214 / 0.217 |
Table R3. Spread across within each configuration
| Configuration | values | Spread of | Spread of |
|---|---|---|---|
| , seed 3337 | 7 | 0.009 / 0.004 | 0.379 / 0.402 |
| , seed 3337 | 13 | 0.013 / 0.006 | 0.393 / 0.414 |
| , seed 8888 | 13 | 0.007 / 0.004 | 0.397 / 0.450 |
| , seed 1111 | 13 | 0.002 / 0.000 | 0.333 / 0.353 |
| , seed 3337 | 13 | 0.004 / 0.002 | 0.353 / 0.351 |
W4 (b) / Q4: Within the same model class, how does the candidate family respond to additional valid identifying information (labels, graph restrictions, or intervention data), and does the spread contract accordingly?
Thank you. A systematic study of how the candidate family responds to additional identifying information is beyond the scope of this paper, which works with observational data under the canonical diagram (Fig. 3a). In principle, such information shrinks the set of compatible models; when it identifies (e.g., a restriction making independent of ), all compatible fits share the same , so the spread across should contract to seed variation. We will explore additional graph restrictions and interventional data in future work. Labels of add no information on Color-MNIST, since is a deterministic function of the image.
Writing: Some explanations are deferred to the appendix; Example 1 should state its outcome mechanism and treatment-assignment probabilities, and the text between Eqs. (3) and (4) should explain that is unobserved and integrated out, defining the resulting image likelihood before the ELBO.
Thank you for pointing this out; we will make both changes. In the main text, Example 1 will state that , , and . These give and , against the interventional values and .
Between Eqs. (3) and (4), we will state that is not observed and is produced inside the model by the outcome head. We will then define the image likelihood , with integrated out (currently Eq. (10) in App. B), before using it in the ELBO. We will also move the definition of admissibility and the acceptance procedure from the appendix to Sec. 3.
Thank you again for your review. We wanted to check whether our response has addressed your concerns, and we would be happy to clarify any remaining questions before the discussion period closes.
Reply [1/3]
We thank the reviewer for the constructive review and precise questions, and for recognizing that an incomplete candidate family can be useful provided its admissibility and exploratory scope are made explicit. We address each point below.
The evidence is the observational distribution P(X,I) under the canonical diagram (Fig. 3a). A fitted model is admitted when it passes the observational check in our response to Reviewer iz6K, W1 (b): reconstruction MSE and NLL within the range of γ=0 seeds, and label RMSE of P(X,Y) within twice its sampling standard error of the baseline. This is a finite, relative check, so the admitted set approximates the models compatible with P(X,I) only up to these tolerances.
The candidates are illustrative scenarios, not a posterior. We assign no plausibility weights, since observational data cannot rank models that fit them equally well. The spread is therefore not a credible interval, and it does not claim coverage of the identified set (see W4 (a)). What the family supports is existence statements: each admitted candidate is a model consistent with the evidence, up to the tolerances, that implies the reported Px.
The whiskers appear only on the baselines (VAE, GAN, DDPM, ANCM) and show the mean ± one standard deviation over five seeds at a fixed configuration, as in Table 1. The CAUVADE bars are single models at γ∈{0,1,10,100} and carry no error bars. For CAUVADE, seed variation at γ=0 is 0.030 for do(X=0) and 0.067 for do(X=1) (10 seeds), while the spread across γ is 4–8 times larger (iz6K, Q3). We will state this in the caption.
The evidence enters only through the observational fit. Pxθ in Eq. (3) is computed from the fitted mechanisms after training; it does not constrain them. D in Eq. (5) is a weighted version of the observational joint ELBO for P(X,I), whose treatment term is the cross-entropy LX of the treatment head pθ(X∣Z,C) (our response to Reviewer iz6K, W1 (a)). Eq. (6) adds −γH, which brings no information beyond P(X,I). Training therefore constrains the learned mechanisms only through the model-implied Pθ(X,I); since D is a weighted surrogate, we check this fit separately (iz6K, W1 (b)). Each fitted model is thus a causal model compatible with the data, and Pxθ is one possible prediction.
This fit determines every functional of P(X,I), including P(X,Y) and P(Y∣X) with Y read from I. For discrete (X,Y), it restricts P(Y∣do(x)) to the Manski region.
It does not identify where Px lies among the interventional distributions compatible with P(X,I). For the scalar query, this is the location inside the Manski region; for the image distribution, it is the corresponding unidentified part of Px(I). The entropy sweep is designed to move this unresolved part, while the observational checks keep the determined part within their tolerances.
We will add this paragraph after Eq. (6).