acceptodds
Under review as a conference paper at ICLR 2027

A Causal Bootstrap for Diverse Image Generation under Unobserved Confounding

Abstract

Deep generative models reproduce the observational distribution underlying the training data, which, in turn, inherit any spurious associations it contains. A common source is an unobserved confounder that simultaneously affects both a treatment attribute the user wants to control/edit (e.g., age, sex) and an observed outcome (e.g., an image) expected to vary in response. Existing causal inference methods resolve the resulting ambiguity by imposing structural assumptions strong enough to single out one interventional distribution. However, in practical domains like image generation, such assumptions are rarely warranted, and the interventional distribution is generally non-identifiable: distinct causal models can agree on the observed data yet imply different causal predictions. This paper proposes CauVaDE (Causal Variational Deep Embedding), a novel family of causal generative models in which the unobserved confounder collapses into a discrete latent cluster with bounded support without loss of generality. We prove that this family can approximate both observational and interventional image distributions with arbitrary accuracy in a canonical latent causal diagram. Building on this model representation, we introduce a novel causal bootstrap that varies the strength of confounder entropy to generate a diverse family of causal explanations, each defining a distinct interventional image distribution. Such a procedure produces distinct interventional images while maintaining a good fit to the observed data.

Primary area:
causal reasoning
Keywords:
Causal Inference, Graphical Model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_vPDyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies image generation under hidden confounding, where observational data may admit different intervention predictions. It introduces CauVaDE, a latent causal model with a discrete confounder, and proves an approximation result under regularity assumptions. It then varies a cluster-posterior entropy weight to generate alternative fitted models and intervention distributions while checking observational fit. Experiments on Color-MNIST, CelebA, and MIMIC-CXR compare predictions against causal bounds or unconfounded references. The method produces an exploratory collection of candidate causal explanations, without characterizing the full remaining uncertainty.

Critical strengths
  • The paper provides a causally structured representation. It explicitly distinguishes observational conditioning from intervention and accommodates hidden confounding. Theorem 2.2 supplies an approximation guarantee for finite-confounder models under regularity assumptions.
  • The method offers a practical procedure for exploring alternative candidate explanations. The entropy sweep generates different intervention predictions while monitoring observational fit. An incomplete candidate family could be valuable for finding counterexamples, guiding experimental design, or stress-testing decisions, provided its admissibility is established and its exploratory scope is explicit (see Weaknesses).
  • The paper is mostly well-written, and the theoretical results appear sound.
Critical weaknesses
  1. The paper does not establish that generated diversity reflects remaining causal uncertainty. For a fixed intervention, its candidate spread has neither demonstrated coverage of the identified set nor justified probabilistic weighting over mechanisms. Different interventions already produce different images, even deterministically. Exploratory candidate generation can be useful, but greater spread alone does not establish more informative causal inference; its relationship to the uncertainty permitted by the evidence needs assessment.

  2. The connection between Eqs. (5)–(6), identification, and causal inference remains insufficiently established. The assumed structural model defines how to compute intervention predictions, but intervention semantics alone do not establish that fitted mechanisms satisfy the evidence. The paper should explain how its objective constrains learned mechanisms using the available information, and which aspects of their intervention predictions are thereby determined or remain unresolved. The gap concerns the causal justification of the fitted models, beyond whether they generate plausible images or respond to entropy regularization.

  3. The connection between the local theory and implemented training remains unestablished. Theorem 3.2 assumes a positive-definite baseline Hessian and a nonzero interventional response, without establishing their applicability to the fitted networks. Any dedicated parameters omitted from the loss would also introduce flat directions. Moreover, the theorem follows a single smooth branch near zero entropy weight, whereas separate training runs at finite weights need not follow that branch or inherit its conclusions.

  4. The evaluation does not isolate target nonidentification from estimation, regularization, or optimization error. Failure to recover latent confounders does not establish nonidentification of intervention predictions. The paper should distinguish persistent ambiguity allowed by the maintained evidence from disagreement caused by imperfect fitting, and test whether additional identifying information contracts the candidate predictions appropriately.

Writing issues. Some explanations needed to follow the argument are deferred to the appendix. Example 1 should state its outcome mechanism and treatment-assignment probabilities. Between Eqs. (3) and (4), the main text should explain that Y is unobserved and integrated out, defining the resulting image likelihood before using it in the ELBO.

Questions for authors

I think the authors should address two of 1 ~ 3; addressing all three will make a strong paper.

  1. Please specify the information set defining admissible mechanisms and explain what uncertainty statements the candidate family supports. Are candidates assigned plausibility weights, or are they illustrative scenarios? Please also define Fig. 2’s error bars and distinguish their meaning from variation across entropy weights.

  2. On tractable benchmarks with known identified sets, can you separately assess model admissibility, target-set membership, and coverage, including missed regions? Include an identified target and test whether disagreement for a fixed intervention disappears (in the correctly specified limit) as observational fitting errors vanish.

  3. How does Theorem 3.2 apply to the implemented training procedure? Please justify assumptions (A4)–(A5) for the fitted networks, clarify how possible Hessian flat directions are handled, and assess whether separate training runs at finite entropy weights follow the local branch analyzed by the theorem.

  4. Within the same model class, how does the candidate family respond to additional valid identifying information? Please add labels, graph restrictions, or intervention data and assess whether the generated spread tracks the resulting contraction of admissible predictions.

4 comments
AuthorsOpenReview ↗

Reply [1/3]

We thank the reviewer for the constructive review and precise questions, and for recognizing that an incomplete candidate family can be useful provided its admissibility and exploratory scope are made explicit. We address each point below.

W1 / Q1: The paper does not establish that generated diversity reflects remaining causal uncertainty: the candidate spread has neither demonstrated coverage of the identified set nor a justified weighting over mechanisms. What information set defines admissible mechanisms, are candidates plausibility-weighted or illustrative scenarios, and what do Fig. 2's error bars mean relative to variation across entropy weights?

The evidence is the observational distribution P(X,I)P(X, I) under the canonical diagram (Fig. 3a). A fitted model is admitted when it passes the observational check in our response to Reviewer iz6K, W1 (b): reconstruction MSE and NLL within the range of γ=0\gamma=0 seeds, and label RMSE of P(X,Y)P(X,Y) within twice its sampling standard error of the baseline. This is a finite, relative check, so the admitted set approximates the models compatible with P(X,I)P(X,I) only up to these tolerances.

The candidates are illustrative scenarios, not a posterior. We assign no plausibility weights, since observational data cannot rank models that fit them equally well. The spread is therefore not a credible interval, and it does not claim coverage of the identified set (see W4 (a)). What the family supports is existence statements: each admitted candidate is a model consistent with the evidence, up to the tolerances, that implies the reported PxP_x.

The whiskers appear only on the baselines (VAE, GAN, DDPM, ANCM) and show the mean ± one standard deviation over five seeds at a fixed configuration, as in Table 1. The CAUVADE bars are single models at γ∈{0,1,10,100}\gamma\in\{0,1,10,100\} and carry no error bars. For CAUVADE, seed variation at γ=0\gamma=0 is 0.030 for do(X=0)do(X{=}0) and 0.067 for do(X=1)do(X{=}1) (10 seeds), while the spread across γ\gamma is 4–8 times larger (iz6K, Q3). We will state this in the caption.

W2: The connection between Eqs. (5)–(6), identification, and causal inference is insufficiently established: intervention semantics alone do not show that fitted mechanisms satisfy the evidence. How does the objective constrain the learned mechanisms, and which aspects of their intervention predictions are thereby determined or left unresolved?

The evidence enters only through the observational fit. PxθP^\theta_x in Eq. (3) is computed from the fitted mechanisms after training; it does not constrain them. DD in Eq. (5) is a weighted version of the observational joint ELBO for P(X,I)P(X, I), whose treatment term is the cross-entropy LX\mathcal L_X of the treatment head pθ(X∣Z,C)p_\theta(X \mid \mathbf Z, C) (our response to Reviewer iz6K, W1 (a)). Eq. (6) adds −γH-\gamma H, which brings no information beyond P(X,I)P(X, I). Training therefore constrains the learned mechanisms only through the model-implied Pθ(X,I)P^\theta(X, I); since DD is a weighted surrogate, we check this fit separately (iz6K, W1 (b)). Each fitted model is thus a causal model compatible with the data, and PxθP^\theta_x is one possible prediction.

This fit determines every functional of P(X,I)P(X, I), including P(X,Y)P(X, Y) and P(Y∣X)P(Y \mid X) with YY read from II. For discrete (X,Y)(X, Y), it restricts P(Y∣do(x))P(Y \mid \mathrm{do}(x)) to the Manski region.

It does not identify where PxP_x lies among the interventional distributions compatible with P(X,I)P(X, I). For the scalar query, this is the location inside the Manski region; for the image distribution, it is the corresponding unidentified part of Px(I)P_x(I). The entropy sweep is designed to move this unresolved part, while the observational checks keep the determined part within their tolerances.

We will add this paragraph after Eq. (6).

AuthorsOpenReview ↗

Reply [2/3]

W3 / Q3: Thm. 3.2 assumes a positive-definite baseline Hessian and a nonzero interventional response without establishing them for the fitted networks; parameters omitted from the loss would add flat directions, and separate runs at finite entropy weights need not follow the analyzed branch.

We agree, and we give the full answer in our response to Reviewer iz6K, W2 (a). In brief, Thm. 3.2 is a local motivation for the regularizer, not a guarantee for the trained networks, and we do not verify (A4)–(A5) for them. (A4) as stated fails because of reparameterization symmetries and empty components; a version on the quotient by these symmetries is plausible but not proved. On parameters omitted from the loss, the treatment head fXf_X is the generative treatment mechanism and is trained by LX\mathcal L_X (iz6K, W1 (a)), so the treatment mechanism has no parameters outside the loss. Candidates at finite γ\gamma are trained independently and need not follow the analyzed branch. We have not obtained a reliable small-γ\gamma continuation for this rebuttal. Admissibility of the reported candidates therefore rests only on the observational check, not on Thm. 3.2.

W4 (a) / Q2: The evaluation does not separate nonidentification from estimation, regularization, or optimization error, and failure to recover latent confounders does not establish nonidentification of intervention predictions. On benchmarks with known identified sets, can admissibility, target-set membership, and coverage (including missed regions) be assessed separately? Include an identified target and test empirically whether disagreement vanishes as observational fitting error goes to zero.

We agree that failing to recover UU (Table 11) does not establish nonidentification, and we will remove that argument from App. E.5.3 and FAQ 4. On Confounded Color-MNIST, nonidentification follows from the construction itself. The images are generated from (X,Y)(X, Y) alone; P(X,I)P(X, I) carries no further information about UU. Every value inside the Manski bound is therefore attained by some SCM compatible with the evidence.

At γ=0\gamma=0, restarts with the same configuration differ by 0.030 and 0.067 (10 seeds). This bounds the disagreement that optimization alone produces at a fixed setting. The accepted candidates across γ\gamma differ 4–8 times more (iz6K, Q3).

Admissibility, membership, coverage (Color-MNIST, Table 1):

  • Admissibility: every γ\gamma in Table 1 passes the check in iz6K, W1 (b).
  • Membership: all candidate predictions lie inside the Manski region.
  • Coverage: the candidates span 0.49–0.72 for do(X=0)do(X{=}0), 43% of the Manski width, and 0.54–0.83 for do(X=1)do(X{=}1), 63%. The missed regions are [0.28,0.49)[0.28, 0.49) and (0.72,0.82](0.72, 0.82] for do(X=0)do(X{=}0), and [0.42,0.54)[0.42, 0.54) and (0.83,0.88](0.83, 0.88] for do(X=1)do(X{=}1). The true value for do(X=0)do(X{=}0), 0.40, lies in a missed region. The family therefore does not cover the identified set, and we will report these coverage figures in Sec. 4.
AuthorsOpenReview ↗

Reply [3/3]

Identified target. If a target is a functional of P(X,I)P(X,I), two models that reproduce P(X,I)P(X,I) exactly assign it the same value, so disagreement must vanish in the correctly specified limit. We evaluated 59 candidates (five configuration/seed groups, each over the γ\gamma grid of Table 4) on the identified target P(Y=1∣X=x)P(Y{=}1 \mid X{=}x) and on P(Y=1∣do(X=x))P(Y{=}1 \mid do(X{=}x)) (Table R2). On the training set, the identified target differs across all candidates by at most 0.013 and 0.007, and by at most 0.004 once the image-level MMD is within the data-to-data level. Within a configuration, varying γ\gamma changes it by at most 0.013 (Table R3). Over the same candidates, the interventional prediction differs by 0.21–0.45. Disagreement therefore vanishes on the identified target at matched fit, but not on the interventional prediction.

Table R2. Identified vs. non-identified targets as the image-level fit threshold tightens.

Threshold on MMD2(X,I)\mathrm{MMD}^2(X, I)Candidates retainedSpread of P(Y=1∣X=x)P(Y{=}1 \mid X{=}x), training setSpread of P(Y=1∣X=x)P(Y{=}1 \mid X{=}x), held-out setSpread of P(Y=1∣do(X=x))P(Y{=}1 \mid do(X{=}x))
none590.013 / 0.0070.099 / 0.0070.408 / 0.450
≤0.048\le 0.048300.013 / 0.0040.099 / 0.0060.377 / 0.336
≤0.038\le 0.038150.004 / 0.0000.001 / 0.0000.326 / 0.335
≤0.030\le 0.03080.004 / 0.0000.001 / 0.0000.319 / 0.309
≤0.027\le 0.02740.002 / 0.0000.000 / 0.0000.214 / 0.217

Table R3. Spread across γ\gamma within each configuration

Configurationγ\gamma valuesSpread of P(Y=1∣X=x)P(Y{=}1 \mid X{=}x)Spread of P(Y=1∣do(X=x))P(Y{=}1 \mid do(X{=}x))
α=10\alpha=10, seed 333770.009 / 0.0040.379 / 0.402
α=15\alpha=15, seed 3337130.013 / 0.0060.393 / 0.414
α=15\alpha=15, seed 8888130.007 / 0.0040.397 / 0.450
α=50\alpha=50, seed 1111130.002 / 0.0000.333 / 0.353
α=50\alpha=50, seed 3337130.004 / 0.0020.353 / 0.351

W4 (b) / Q4: Within the same model class, how does the candidate family respond to additional valid identifying information (labels, graph restrictions, or intervention data), and does the spread contract accordingly?

Thank you. A systematic study of how the candidate family responds to additional identifying information is beyond the scope of this paper, which works with observational data under the canonical diagram (Fig. 3a). In principle, such information shrinks the set of compatible models; when it identifies PxP_x (e.g., a restriction making XX independent of (C,Z)(C, \mathbf Z)), all compatible fits share the same PxP_x, so the spread across γ\gamma should contract to seed variation. We will explore additional graph restrictions and interventional data in future work. Labels of YY add no information on Color-MNIST, since YY is a deterministic function of the image.

Writing: Some explanations are deferred to the appendix; Example 1 should state its outcome mechanism and treatment-assignment probabilities, and the text between Eqs. (3) and (4) should explain that YY is unobserved and integrated out, defining the resulting image likelihood before the ELBO.

Thank you for pointing this out; we will make both changes. In the main text, Example 1 will state that P(U=1)=0.4P(U{=}1)=0.4, Y=U⊕XY = U \oplus X, P(X=1∣U=0)=0.7P(X{=}1\mid U{=}0)=0.7 and P(X=1∣U=1)=0.3P(X{=}1\mid U{=}1)=0.3. These give P(Y=1∣X=0)=0.609P(Y{=}1\mid X{=}0)=0.609 and P(Y=1∣X=1)=0.778P(Y{=}1\mid X{=}1)=0.778, against the interventional values P(Y=1∣do(X=0))=0.40P(Y{=}1\mid do(X{=}0))=0.40 and P(Y=1∣do(X=1))=0.60P(Y{=}1\mid do(X{=}1))=0.60.

Between Eqs. (3) and (4), we will state that YY is not observed and is produced inside the model by the outcome head. We will then define the image likelihood pθ(I∣x,z,c)p_\theta(I\mid x, \mathbf z, c), with YY integrated out (currently Eq. (10) in App. B), before using it in the ELBO. We will also move the definition of admissibility and the acceptance procedure from the appendix to Sec. 3.

Thank you again for your review. We wanted to check whether our response has addressed your concerns, and we would be happy to clarify any remaining questions before the discussion period closes.

Reviewer_KCCZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes CAUVADE, a causal generative model for exploring multiple interventional image distributions under unobserved confounding. It introduces a canonical representation with a discrete confounder, instantiates this representation through a Gaussian-mixture VAE, and varies a cluster-posterior entropy regulariser to generate alternative causal explanations. Experiments use confounded versions of the Color-MNIST, CelebA, and MIMIC-CXR-JPG datasets.

Critical strengths
  • The paper addresses an important limitation of causal image generation, that observational data may support several incompatible intervention predictions. By explicitly exploring this ambiguity the authors open up a useful research direction and this is supported by the theoretical motivation provided by Theorem 2.2.

  • An empiricall strength is that the experiments demonstrate that varying the entropy weight changes interventional predictions beyond the variation observed across baseline random initialisations. On MIMIC-CXR-JPG, Table 2 also reports improved average FID against the reference distribution.

Critical weaknesses
  • One key weakness is that the training objective does not clearly establish a fitted observational causal model. It is not clearly explained how p_\theta(X | Z,C) (from Eq 2) is learned or how the resulting observational joint distribution is computed and fitted. An observational joint-likelihood objective would normally include a contribution from the generative treatment mechanism and the lack of this matters because the proposed method aims to produce different causal models compatible with the same confounded observations.

  • Another weakness is that the confounder entropy depends on the latent representation. Splitting a cluster into identical subclusters can increase posterior entropy without changing observational or interventional predictions so entropy variation need not represent additional causal ambiguity. Adding experiments to investigate sensitivity to cluster count and redundant mixture components would help establish the regulariser’s causal interpretation.

Questions for authors
  • How is the generative treatment mechanism estimated?
  • How sensitive are the resulting intervention ranges to the number of latent clusters and the reconstruction, KL?
  • Do models with comparable observational fit produce substantially different ranges under these choices?
3 comments
AuthorsOpenReview ↗

Reply [1/2]

We thank the reviewer for the thoughtful review and for recognizing the value of exploring several intervention predictions compatible with the same observations, and that varying the entropy weight moves the predictions beyond random-restart variation. We address each point below.

W1 / Q1: The objective does not clearly fit an observational causal model: how is pθ(X∣Z,C)p_\theta(X \mid \mathbf Z, C) (Eq. 2) learned, and how is the observational joint distribution computed? An observational joint likelihood would normally include the treatment-mechanism term.

We completely agree with the reviewer that a valid observational causal model must incorporate the generative treatment mechanism into the joint observational likelihood. We realize that the disconnected description in Sec. 3 caused this misunderstanding, and we apologize for the confusion.

The training objective does include this treatment term. In our implementation, pψp_\psi is not a separate network: the decoder's treatment head fXf_X computes the treatment loss LX\mathcal L_X during training and serves as the generative treatment mechanism pθ(X∣Z,C)p_\theta(X \mid \mathbf Z, C) during sampling. Since fXf_X is part of the generative model θ\theta and is trained through LX\mathcal L_X, the observational joint bound decomposes as

log⁡pθ(X,I)≥Eqϕ[log⁡pθ(I∣X,Z,C)]−DKL(qϕ(Z,C∣I)∥pθ(Z∣C)P(C))⏟Eq. (4): bound on log⁡Pxθ(I) at x=X+Eqϕ[log⁡pθ(X∣Z,C)]⏟−LX,\log p_\theta(X, I) \ge \underbrace{\mathbb E_{q_\phi}\bigl[\log p_\theta(I \mid X, \mathbf Z, C)\bigr] - D_{\mathrm{KL}}\bigl(q_\phi(\mathbf Z, C \mid I) \Vert p_\theta(\mathbf Z \mid C)P(C)\bigr)}_{\text{Eq. (4): bound on } \log P^\theta_x(I) \text{ at } x = X} + \underbrace{\mathbb E_{q_\phi}\bigl[\log p_\theta(X \mid \mathbf Z, C)\bigr]}_{-\mathcal L_X},

Because α\alpha and λ\lambda weight the terms differently, DD in Eq. (5) is a weighted version of this bound rather than the likelihood itself.

The model-implied observational probabilities used in our checks are computed from posterior reconstructions; the procedure and the acceptance rule are given in our response to Reviewer iz6K, W1 (a)–(b). We will write pθp_\theta and LX(θ,ϕ)\mathcal L_X(\theta,\phi), drop ψ\psi from DD, η\eta and Algorithm 1, and remove the clause in Sec. 3 stating that the two are not identified.

W2: Cluster-posterior entropy depends on the latent representation: splitting a cluster into identical subclusters raises entropy without changing observational or interventional predictions, so entropy variation need not reflect causal ambiguity.

We agree with the example, and we do not use HH as a measure of causal ambiguity. HH is a perturbation that moves the fitted model; App. A already notes that γ\gamma has no calibrated interpretation. Diversity is measured in output space, by differences between the candidates' PxθP^\theta_x, which a relabeling or splitting of CC leaves unchanged.

Thm. 3.2 accounts for this case. It is conditional on two local assumptions, and the reviewer's split violates both:

  • (A5). A pure split direction has zero interventional derivative, dPx(η0)[v]=0\mathrm d\mathsf P_x(\eta_0)[v]=0, so ρx=0\rho_x=0. Along such a direction, Thm. 3.2 does not predict a change. Sec. 3 states that parameters can move without changing the generated distribution, and (A5) is the condition that excludes this.
  • (A4). If η0\eta_0 contained two identical clusters, the proportion of prior and posterior mass between them would be a flat direction of DD, contradicting ∇2D(η0)≻0\nabla^2 D(\eta_0)\succ0. If η0\eta_0 has no duplicate clusters, creating one requires moving a whole component, which is not a local move. So under (A4), the local branch cannot increase HH by splitting alone.

The theorem therefore claims only that, when the entropy direction has a nonzero interventional component, PxP_x moves at first order while DD moves at second order.

We also checked whether the entropy increases in our trained models come from such splits. On Confounded Color-MNIST we took 120 trained models: K∈{8,16}K\in\{8,16\}, β∈{0.05,0.1,0.5}\beta\in\{0.05,0.1,0.5\}, reconstruction weight ∈{10,50}\in\{10,50\}, and γ\gamma from 0 to 10. For each model, we compared the interventional image distributions pθ(I∣do(X=x),c)p_\theta(I\mid do(X{=}x), c) of every pair of occupied components (cc with aggregated posterior mass >0.01>0.01). We used an RBF-kernel MMD and declared two components duplicates when their MMD was within three times the within-component resampling floor.

No model has a duplicate pair, under either do(X=0)do(X{=}0) or do(X=1)do(X{=}1). The smallest MMD between two occupied components is 0.063, more than 20 times the largest duplicate threshold (0.003). Therefore, merging duplicates leaves HH unchanged in every model, including those where γ\gamma raises HH from about 0.01 at γ=0\gamma=0 to as much as 0.88.

AuthorsOpenReview ↗

Reply [2/2]

So the entropy increases we observe come from posterior overlap between components that generate different interventional images, not from splitting a component into identical copies.

We will add a remark after (A5) that cites the reviewer's example, explains why (A4)–(A5) exclude it, and notes that we assess diversity in the output space, not by HH, along with this check.

Q2: How sensitive are the intervention ranges to the number of clusters KK and to the reconstruction and KL weights?

We report the interventional ranges on Confounded Color-MNIST under three kinds of choices: the number of confounder states KK (App. E, Table 9), the KL weight β\beta, and the reconstruction weight α\alpha. The β\beta and α\alpha rows are new experiments. Each varies one weight around the configuration used for Fig. 7 (K=8K=8, α=10\alpha=10, β=0.1\beta=0.1, λ=5\lambda=5). For each setting, we trained models independently at γ∈{0,1,2,3}\gamma\in\{0,1,2,3\} with 4 seeds, giving 16 models per setting at the same training budget.

All new interventional predictions are computed from samples of the joint generative model: c∼P(C)c\sim P(C), then z∣c\mathbf z\mid c, then XX, then the image. (X,Y)(X, Y) are read off the generated images with classifiers that are independent of the model. The ranges for the new rows include all 16 models of each setting.

ConfigurationP(Y=1∣do(X=0))P(Y{=}1\mid do(X{=}0))P(Y=1∣do(X=1))P(Y{=}1\mid do(X{=}1))
Paper default (Table 1, K=8K=8, γ∈{0,1,10,100}\gamma\in\{0,1,10,100\})0.49–0.720.54–0.83
K=2K=2 (Table 9)0.50–0.67 (14/28 in bounds)0.49–0.67
K=4K=4 (Table 9)0.39–0.74 (26/28 in bounds)0.46–0.75
K=6K=6 (Table 9)0.59–0.81 (28/28 in bounds)0.50–0.68
β=0.05\beta=0.050.67–0.750.53–0.66
β=0.5\beta=0.50.68–0.690.65–0.69
α=1\alpha=10.69–0.750.64–0.73

The Manski region is [0.28,0.82][0.28, 0.82] for do(X=0)do(X{=}0) and [0.42,0.88][0.42, 0.88] for do(X=1)do(X{=}1). The KK rows aggregate the γ\gamma-sweep of App. E; their ranges span the in-bounds estimates. In the β\beta and α\alpha rows, every one of the 48 models lies inside the Manski region.

Sensitivity to KK. The number of confounder states matters most at small KK. At K=2K=2, only 14 of the 28 swept estimates fall inside the Manski region, compared with 26 of 28 at K=4K=4 and all 28 at K=6K=6. This matches the scope of Thm. 2.2, which approximates every compatible ASCM only when KK is large enough and does not cover a fixed small KK. Beyond that point, KK changes where the range lies rather than whether it is admissible. K=4K=4 gives the widest range for do(X=0)do(X{=}0) (width 0.35), while K=6K=6 shifts it upward to 0.59–0.81. At K=100K=100 (App. E, Table 10), only 12–21 of the 100 components are occupied and the holdout BCE is flat across γ\gamma, so a larger KK does not by itself create additional distinct components.

Sensitivity to the reconstruction and KL weights. The KL weight controls how far γ\gamma can move the predictions. At β=0.5\beta=0.5 they concentrate near the observational marginal P(Y=1)≈0.70P(Y{=}1)\approx 0.70, and do(X=1)do(X{=}1) spans only 0.65–0.69, 9% of the Manski width. At β=0.05\beta=0.05, do(X=1)do(X{=}1) spans 0.53–0.66, 28% of the width. With the reconstruction weight lowered to α=1\alpha=1, do(X=1)do(X{=}1) concentrates at 0.64–0.73. None of these choices pushes a prediction outside the Manski region.

Q3: Do models with comparable observational fit produce substantially different ranges under these choices?

Yes. The models fit the observational data comparably. For all 48 new models, the RMSE of the model-implied P(X,Y)P(X, Y), computed with posterior latents, is at most 0.021; for the paper's models, all settings except Color-MNIST γ=30\gamma=30 pass the check in our response to Reviewer iz6K, W1 (b).

Across settings, the ranges still differ in both width and location. For example, do(X=1)do(X{=}1) spans 0.53–0.66 at β=0.05\beta=0.05 but 0.64–0.73 at α=1\alpha=1, and do(X=0)do(X{=}0) spans 0.39–0.74 at K=4K=4 but 0.59–0.81 at K=6K=6.

So KK, α\alpha and β\beta, like γ\gamma, select among observationally comparable models. We read this as further evidence that the confounded data do not pin down PxP_x. The candidate family should be reported as the union over these choices, not over γ\gamma alone, and the resulting range is not an estimate of the identified set. We will state this in Sec. 4.

We understand this is a busy period, and we would appreciate any feedback on whether our reply has addressed your concerns.

We would also like to add further evidence on your first concern: whether our objective fits the observational distribution, including the treatment mechanism pθ(X∣Z,C)p_\theta(X \mid \mathbf Z, C). We compute the model-implied P(X,Y)P(X, Y) from posterior reconstructions and compare it with the empirical P(X,Y)P(X, Y) (when both XX and YY are labeled). Since XX in the model-implied P(X,Y)P(X, Y) is drawn from the treatment head pθ(X∣z,c)p_\theta(X \mid \mathbf z, c), this check also tests the fitted treatment mechanism. On Color-MNIST, a model passes when its label RMSE is at most 0.0780.078, the γ=0\gamma = 0 value plus twice the sampling standard error.

All 48 new models varying the KL and reconstruction weights have label RMSE at most 0.0210.021. Among the models in the paper, every setting passes except Color-MNIST γ=30\gamma = 30, which passes the label check but fails the image-level check; we exclude it from the reported ranges. The intervention ranges in our reply therefore come from models that fit the observational distribution comparably.

We are happy to answer any further questions.

Reviewer_iz6KOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a novel Gaussian-mixture variational autoencoder for generating multiple candidate interventional image distributions in the presence of hidden confounding. It includes theoretical contributions, such as a finite-confounder approximation theorem. The empirical results are promising and provide evidence for observational compatibility, but valid intervention behavior remains incomplete.

Critical strengths

Well-motivated problem, capable of distinguishing between generating plausible causal alternatives and the true mechanism

Theorem 2.2 provides a useful approximation result.

The empirical results provide a good demonstration of the effectiveness.

Critical weaknesses
  1. It lacks clarity on how the generative treatment mechanism and the model's observational probabilities are learned. In addition to the experimental results, the theoretical claims need empirical validation, and key quantitative comparisons are missing.

  2. The assumptions of some theoretical results need verification for CAUVADE. Moreover, one theorem applies only to an abstract model rather than the implemented architecture. It is unclear whether the proposed regularizer empirically produces meaningful causal alternatives.

  3. In the experiments, it is unclear how outcomes, images, and confounders are generated or preserved. The paper would benefit if the authors demonstrated that only the treatment mechanism changes, while all other mechanisms remain fixed.

  4. The change in predicted colors across different treatment values in Table 7 does not look obvious. Lower FID scores may not indicate the correctness of the intervention.

Questions for authors
  1. Explain how the generative treatment mechanism is fitted and how observational joint probabilities are generated. It lacks some key hyperparameters, such as held-out joint-distribution discrepancies, fixed acceptance thresholds, and acceptance rates.

  2. Are assumptions A4–A5 supported empirically? Did you observe the predicted cost and distributional changes from a small-γ continuation experiment? How can entropy changes be prevented from affecting only the encoder?

  3. If we consider the setting with matched observational fit and training budget, how does the accepted interventional diversity compare with other components, such as varying another existing loss weight or additional restarts? Show stability across seeds.

My main concerns from my weaknesses and questions have been well addressed by the author's rebuttal. Good job!

6 comments
AuthorsOpenReview ↗

Reply [1/3]

We thank the reviewer for the careful review and for recognizing the motivation of the problem, the distinction between generating causal alternatives and identifying the true mechanism, and the approximation argument of Theorem 2.2.

W1 (a) / Q1 (a): The auxiliary treatment classifier is separate from the generative treatment mechanism; how is that mechanism learned, and how are model-implied observational probabilities computed?

There may be some confusion due to our notation. The auxiliary classifier and the generative treatment mechanism are the same module: pψp_\psi is not a separate network. The decoder’s treatment head fXf_X computes LX\mathcal L_X and also serves as the generative treatment mechanism used for sampling. fXf_X is trained only through LX\mathcal L_X. With the same variational family, the observational joint bound decomposes as

log⁡pθ(X,I)≥Eqϕ[log⁡pθ(I∣X,Z,C)]−DKL(qϕ(Z,C∣I)∥pθ(Z∣C)P(C))⏟Eq. (4): bound on log⁡Pxθ(I) at x=X+Eqϕ[log⁡pθ(X∣Z,C)]⏟−LX,\log p_\theta(X, I) \ge \underbrace{\mathbb E_{q_\phi}\bigl[\log p_\theta(I \mid X, \mathbf Z, C)\bigr] - D_{\mathrm{KL}}\bigl(q_\phi(\mathbf Z, C \mid I) \Vert p_\theta(\mathbf Z \mid C)P(C)\bigr)}_{\text{Eq. (4): bound on } \log P^\theta_x(I) \text{ at } x = X} + \underbrace{\mathbb E_{q_\phi}\bigl[\log p_\theta(X \mid \mathbf Z, C)\bigr]}_{-\mathcal L_X},

For the observational checks in W1 (b), model-implied probabilities are computed from posterior reconstructions. For each image II in the evaluation set, we draw z∼qϕ(z∣I)\mathbf z \sim q_\phi(\mathbf z \mid I) and c∼qϕ(c∣z)c \sim q_\phi(c \mid \mathbf z), take XX from fX(z,c)f_X(\mathbf z, c), and decode I′∼pθ(I∣X,z,c)I' \sim p_\theta(I \mid X, \mathbf z, c). We then read YY off I′I' with a classifier trained independently of the model. Averaging over the evaluation set gives the model-implied P(X,Y)P(X,Y) in Tables 4–5, which the acceptance rule below uses.

We will write pθp_\theta and LX(θ,ϕ)\mathcal L_X(\theta,\phi), drop ψ\psi from DD, η\eta and Algorithm 1, and remove the clause in Sec. 3 stating that the two are not identified.

W1 (b) / Q1(b): Tables 4–5 diagnose encoder reconstructions, not joint samples; no acceptance procedure for Cor. 3.3 is reported; MIMIC lacks quantitative results. Report fixed thresholds and acceptance rates for every γ\gamma.

We state the rule applied to every reported candidate. It is a non-inferiority check against γ=0\gamma=0, with thresholds fixed from quantities that do not depend on the γ>0\gamma>0 candidates. Two caveats apply. First, because the rule is relative to γ=0\gamma=0, the baseline-slack condition of Cor. 3.3 holds by construction, and the rule does not certify fit to the data in absolute terms. On Color-MNIST, the baseline's own label RMSE (0.052) is about four sampling standard errors. Second, the diagnostics in Tables 4–5 are computed from posterior reconstructions, not from samples of the joint generative model; the rule inherits this limitation. A candidate is accepted if both of the following hold.

  1. Its reconstruction errors satisfy MSE≤εMSE\mathrm{MSE} \le \varepsilon_{\mathrm{MSE}} and NLL≤εNLL\mathrm{NLL} \le \varepsilon_{\mathrm{NLL}}, where each tolerance is the worst value over ten γ=0\gamma = 0 seeds: the Table 4 run plus nine reruns under the Table 3 configuration. On Color-MNIST, these span MSE∈[0.0307,0.0341]\mathrm{MSE} \in [0.0307, 0.0341] and NLL∈[0.1161,0.1297]\mathrm{NLL} \in [0.1161, 0.1297], and the Table 4 run is the best of the ten. Hence εMSE=0.0341\varepsilon_{\mathrm{MSE}} = 0.0341 and εNLL=0.1297\varepsilon_{\mathrm{NLL}} = 0.1297, i.e., 11.1%11.1\% and 11.7%11.7\% above that run. On CelebA and MIMIC-CXR-JPG, where we did not rerun seeds, we apply the same relative widths; on CelebA this gives εMSE=0.0240\varepsilon_{\mathrm{MSE}} = 0.0240 and εNLL=0.600\varepsilon_{\mathrm{NLL}} = 0.600.

  2. Its label RMSE of P(X,Y)P(X, Y) satisfies RMSE≤εRMSE=RMSE0+2SE\mathrm{RMSE} \le \varepsilon_{\mathrm{RMSE}} = \mathrm{RMSE_0} + 2\mathrm{SE}. Here RMSE0\mathrm{RMSE_0} is the γ=0\gamma = 0 value, and SE=14∑x,ypxy(1−pxy)/n\mathrm{SE} = \sqrt{\tfrac14 \sum_{x,y} p_{xy}(1 - p_{xy})/n} is the sampling standard error of the empirical joint. Color-MNIST has no held-out split, so we use all n=998n = 998 images: SE=0.013\mathrm{SE} = 0.013 and εRMSE=0.052+0.026=0.078\varepsilon_{\mathrm{RMSE}} = 0.052 + 0.026 = 0.078. On CelebA, we use n=289n = 289 held-out images and the worst case pxy=1/4p_{xy} = 1/4: SE=0.025\mathrm{SE} = 0.025 and εRMSE=0.010+0.051=0.061\varepsilon_{\mathrm{RMSE}} = 0.010 + 0.051 = 0.061.

Table R1. Outcome of the rule for every evaluated γ\gamma.

Datasetγ\gammaMSENLLΔ\DeltaLabel RMSE vs. γ=0\gamma=0Outcome
Color-MNISTall except 300.0303–0.03360.1153–0.1260−0.045 to +0.0002accepted
Color-MNIST300.03660.1362−0.021rejected
CelebA1, 10–100+0.0% to +9.7%+0.1% to +1.0%−0.005 to +0.029accepted

The rule accepts 11 of 12 settings on Color-MNIST and all 11 on CelebA. The only rejection, Color-MNIST γ=30\gamma=30, fails both image-level thresholds. No setting fails the semantic criterion. Every γ\gamma reported in Table 1 is accepted. In the revision we will mark γ=30\gamma=30 as rejected wherever it appears.

AuthorsOpenReview ↗

Reply [2/3]

MIMIC. We agree that App. E.3 described this comparison without reporting numbers. Reconstruction NLL on the 1,000 held-out images:

γ\gammaReconstruction NLL (per pixel)
00.5025
10.5015
100.5026
1000.5037

NLL stays within 0.3% of the γ=0\gamma=0 run, so all three settings (γ=1,10,100\gamma = 1,10,100) pass the image-level NLL threshold (11.7%). We have not computed MSE or a semantic check on MIMIC for this rebuttal; we will add both, together with this table, to App. E.3.

W2 (a) / Q2: Thm. 3.2 relies on (A4)–(A5), which are not established for CAUVADE, and its local branch need not describe independent training at large γ\gamma (App. C). What evidence supports (A4)–(A5), and what prevents entropy changes from affecting only the encoder?

Thank you. (A4)–(A5) are assumptions of a local analysis that motivates the entropy regularizer. We do not claim that they hold for the trained networks, and candidates trained independently at larger γ\gamma need not lie on the analyzed branch (App. C). We will state this explicitly after (A4)–(A5).

Our support is empirical. The candidates in Table 1 keep a good observational fit, passing the acceptance rule in W1 (b), while their interventional distributions differ: their estimates of Px(Y=1)P_x(Y{=}1) move well beyond the γ=0\gamma = 0 seed range in Table 8. Interventional sampling uses only the generative model and never the encoder (Eq. (3), Alg. 2), so these differences cannot come from changes to the encoder alone.

W2 (b): Thm. 2.2 concerns the abstract CASCM class rather than the implemented architecture.

Thm. 2.2 is a representation result; it does not concern training or the finite network. It shows that restricting the exogenous confounder to a single discrete variable loses no generality up to arbitrary W1W_1 approximation. Every compatible ASCM in the regular class (A1)–(A3) is approximated, jointly over its observational and interventional image distributions, by a CASCM with finitely many confounder states.

CAUVADE's family relates to the CASCMs as follows. The mixture component CC plays the role of UU, with KK the number of confounder states. The Gaussian bundle Z\mathbf Z carries ZZ and the exogenous noises, and the decoders realize the structural functions. Eq. 2 differs from Def. 2.1 in two ways. First, the image decoder also receives CC and the full bundle, whereas Def. 2.1 renders II from (X,Y,Z,EI)(X, Y, Z, E_I) only. Second, the bundle's moments may vary with CC, whereas Def. 2.1 requires the noises to be independent of UU. A CASCM is recovered when these extra dependencies are switched off. With sufficiently expressive decoders, the family therefore contains a CASCM approximant of every compatible ASCM, provided KK is large enough. Thm. 2.2 covers neither a fixed finite KK nor finite-width decoders with a fixed likelihood family (App. D does not claim universal approximation for the architecture). Thm. 2.2 therefore motivates the model class. It says nothing about the particular fit that training reaches, or about whether that fit's interventional distributions are meaningful; those are empirical questions. We will state this scope after Thm. 2.2 and describe the two differences between Eq. 2 and Def. 2.1.

W3: App. E.1.2 resamples on (Z, X, W) while evaluation concerns Y, so the outcome mechanism is unspecified; (ii) App. E.1.3 assigns treatment from age and view but samples images from (X, Y) pools, so it is unclear how the confounder and outcome mechanisms are preserved. A valid reference requires that only the treatment mechanism changes.

(i) CelebA. App. E.1.2 describes the construction inaccurately, and we will rewrite it; the construction itself changes only the treatment mechanism. Every record keeps its own CelebA attributes: UU = Attractive, XX = Young, YY = Heavy_Makeup, ZZ = Male, WW = Wearing_Earrings. The images therefore carry the confounder. The confounded set and the reference are resampled from the corpus, with records drawn uniformly within each (Z,U,X,W,Y)(Z, U, X, W, Y) cell, so P(I∣Z,U,X,W,Y)P(I \mid Z, U, X, W, Y) is the corpus conditional in both sets. Measured on the two sets:

  • P(U)P(U) and P(U∣Z)P(U \mid Z) are the same in both: P(U=1)=0.52P(U{=}1)=0.52; P(U=1∣Z=0)=0.60P(U{=}1\mid Z{=}0)=0.60, P(U=1∣Z=1)=0.40P(U{=}1\mid Z{=}1)=0.40.
  • XX depends on UU in the confounded set, with P(X=1∣U=0)=0.90P(X{=}1\mid U{=}0)=0.90 and P(X=1∣U=1)=0.10P(X{=}1\mid U{=}1)=0.10. In the reference, XX is independent of (U,Z)(U, Z): P(X=1∣U,Z)=0.50P(X{=}1\mid U, Z)=0.50 in every stratum.
  • The remaining mechanisms agree: P(W∣X,U)P(W\mid X,U) within 0.005 and P(Y∣X,U,Z)P(Y\mid X,U,Z) within 0.02 in every cell.

As a cross-check, back-door adjustment for UU on the confounded set gives P(Y=1∣do(X=0))=0.32P(Y{=}1\mid do(X{=}0)) = 0.32 and P(Y=1∣do(X=1))=0.45P(Y{=}1\mid do(X{=}1)) = 0.45, against 0.33 and 0.46 in the reference (Table 1). The CelebA reference is therefore an interventional reference for the confounded set, relative to the corpus conditionals.

AuthorsOpenReview ↗

Reply [3/3]

(ii) MIMIC. The phrase "four atomic pools by (X,Y)(X, Y)" in App. E.1.3 was inaccurate and does not reflect our implementation, which we now describe precisely. Using frontal radiographs (one per study), we take XX = Pneumonia, U=(A,V)U = (A, V) with age AA linked from MIMIC-IV and V=1{AP}V = \mathbb{1}\lbrace\text{AP}\rbrace, and YY = Lung Opacity, which is used only for evaluation and never for training or resampling (this is what "unlabeled" meant). Patients are first split into two disjoint pools, one for the confounded training set and one for the unconfounded reference, so no patient appears in both. Within each pool, we estimate the natural propensity P^0(X∣A,V)\hat P_0(X \mid A, V) by logistic regression and resample N=20,000N = 20{,}000 images with weights wi=P∗(xi∣ai,vi)/P^0(xi∣ai,vi)w_i = P^{\ast}(x_i \mid a_i, v_i) / \hat P_0(x_i \mid a_i, v_i), where P∗P^{\ast} is Eq. (14) for the confounded set and P∗(X=1)=0.5P^{\ast}(X=1) = 0.5 for the reference. Because the weights depend only on (A,V,X)(A, V, X), both sets share P0(A,V)P_0(A, V) and P0(Y,I∣X,A,V)P_0(Y, I \mid X, A, V), which are inherited from the source data, and differ only in the treatment mechanism. The reference is therefore exactly the interventional distribution Px(I)P_x(I) of the stipulated SCM, as the reviewer requires.

W4 (a): The treatment loss does not require generated images to express the intervened value. Table 7’s predictions are nearly unchanged across four treatments; this may be compatible with non-identifiability but gives little evidence of intervention sensitivity. Independent semantic evaluation of generated treatments and outcomes is needed.

The generated images do express the intervened value, as the samples under fixed interventions in Figs. 6, 8 and 12 show. To quantify this, we sampled from Pxθ(I)P^\theta_x(I) as in Alg. 2 and read the digit with classifiers trained independently on real images. On binary Color-MNIST, the fraction of generated images classified as the intervened digit is 1.001.00 under both do(X=0)\mathrm{do}(X{=}0) and do(X=1)\mathrm{do}(X{=}1), for all 13 Color-MNIST models behind Table 1 (10 seeds at γ=0\gamma = 0, plus γ∈{1,10,100}\gamma \in \{1, 10, 100\}). On the 3-value task, a 4-class classifier with validation accuracy 0.990.99 gives 0.98,1.00,0.63,1.000.98, 1.00, 0.63, 1.00 for x=0,1,2,3x = 0, 1, 2, 3 at γ=1\gamma = 1, and 0.96,1.00,0.65,1.000.96, 1.00, 0.65, 1.00 at γ=10\gamma = 10. The treatment is thus expressed reliably for all values except x=2x = 2.

On the 3-value task, the fitted candidates instead select a near-null effect: in Table 7, each channel varies by at most 0.0170.017 across the four treatments. This is one explanation compatible with the data, but we agree the table cannot show intervention sensitivity, which our binary experiments demonstrate (Table 1, Fig. 7). We will state this scope in App. E.4.2, report classifier-based treatment accuracy and outcome distributions for all settings, and drop the in-bounds column from Table 7.

W4 (b): Lower MIMIC FID measures aggregate image agreement and cannot establish treatment correctness.

We agree. The FID in Table 2 does not establish treatment correctness. What the lower FID does indicate is that, at every evaluated γ\gamma, CAUVADE's generated radiographs are closer in distribution to the MIMIC reference set than those of the VAE and ANCM baselines (276.64–289.51 vs. 354.97 and 407.73). We will present this as an image-quality comparison only, and revise the corresponding claims in the abstract, Sec. 4 and the conclusion.

Q3: At matched observational fit and training budget, how does accepted interventional diversity compare with varying another loss weight or with additional restarts, and is it stable across seeds?

At γ=0\gamma=0, restarts with the same budget concentrate P(Y=1∣do(x))P(Y{=}1\mid do(x)) (Table 8, Fig. 13). Across all 10 seeds trained for γ=0\gamma=0, do(X=0)do(X{=}0) stays within 0.570–0.600 and do(X=1)do(X{=}1) within 0.576–0.643. The models in Table 1 span 0.49–0.72 and 0.54–0.83, 4–8 times the restart range.

We ran new experiments with K=8K=8, α=10\alpha=10, β=0.1\beta=0.1, λ=5\lambda=5 and the same budget for every run, varying one weight at a time and training each model independently with 4 seeds. Varying the KL weight β∈{0.05,0.1,0.5}\beta\in\{0.05, 0.1, 0.5\} at γ=0\gamma=0, the models span 2% of the Manski width for do(X=0)do(X{=}0) and 17% for do(X=1)do(X{=}1). Varying γ∈{0,1,2,3}\gamma\in\{0,1,2,3\} at β=0.1\beta=0.1, they span 10% and 31%. Changing the number of components also moves the predictions within the Manski region (Table 9).

Across seeds, the direction of the γ\gamma-induced change is stable but its magnitude is not. In all 4 seeds, P(Y=1∣do(X=1))P(Y{=}1\mid do(X{=}1)) at γ=2\gamma=2 and γ=3\gamma=3 is lower than at γ=0\gamma=0 with the same seed, but at γ=2\gamma=2 it ranges from 0.54 to 0.65 across seeds.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.