acceptodds
Under review as a conference paper at ICLR 2027

Doubly Robust Proxy Causal Learning with Neural Mean Embeddings

Abstract

We propose a novel doubly robust neural mean-embedding framework for proxy causal learning (PCL) for continuous and structured treatments. PCL uses proxy variables and bridge functions to identify causal effects under unobserved confounding: either an outcome bridge or a treatment bridge can identify the target, while doubly robust methods combine both. Our framework learns both bridges with data-adaptive neural features and combines them into doubly robust estimators of population, treatment-conditional, and heterogeneous dose-response functions. Two features of our result are particularly notable: the treatment bridge can be learned without density ratios, and the same treatment bridge yields both population and heterogeneous responses, eliminating the need for a separate bridge fit. Our analysis shows that the doubly robust error is controlled by final-stage error and the smaller of the two projected bridge errors, with no separate density-ratio estimation error. Across challenging continuous- and structured-treatment benchmarks, our method outperforms neural single-bridge estimators and fixed-feature doubly robust estimators.

Primary area:
causal reasoning
Keywords:
proxy causal learning, doubly robust estimation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_fBujOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies causal-effect estimation under unobserved confounding. The authors adopt the proxy causal learning framework in which two observed proxies carry complementary information about a latent confounder. They propose DR-PCL-NET, a doubly robust neural estimator that learns two bridges, one from the treatment side and one from the outcome side, and combines them so that error in either bridge is mitigated by the other. Their treatment bridge requires no density-ratio estimation, a step that burdens earlier methods, and a single bridge fit serves all subgroup and heterogeneous queries. The paper further establishes nonasymptotic projected error bounds showing that the doubly robust error is governed by the smaller of the two bridge errors rather than by a separate density-ratio term. The authors stress-test each bridge route in isolation and then evaluate on benchmarks that include continuous treatments and image-valued structured treatments. Across these settings DR-PCL-NET attains the best or near-best causal mean squared error relative to single-bridge and fixed-kernel baselines.

Critical strengths

1、It removes the density-ratio estimation bottleneck and extends proxy causal learning to continuous and image-valued treatments.

2、Its doubly robust estimator is provably governed by the smaller bridge error and yields all three response types from a single fit.

Critical weaknesses
  1. The presentation is difficult to follow, with an overloaded formula-heavy main text, an identification argument that is hard to trace, and figures too small to read, which makes the contributions hard to assess.

  2. The nonasymptotic bounds are stated only for approximate empirical minimizers, whereas the estimator is trained by a nonconvex stochastic procedure whose convergence the authors explicitly leave open, so the theory does not certify the algorithm behind the reported results.

Questions for authors

1、Please remake the figures with legible text, as the current text is difficult to read. The writing and figure-drawing skills require improvement.

2、The guarantees hold for approximate empirical minimizers, while the method is trained by a nonconvex stochastic procedure without a convergence result. Can you report the optimization error actually attained on the reported benchmarks and compare it with the MSE gaps between your method and the baselines, so the empirical support for the theory becomes visible?

3、Since the submission cites but never compares Bingxi Zhang, Tao Shen, and Yifan Cui. Neural estimation of treatment bridge functions for proximal causal inference. Statistical Analysis and Data Mining: An ASA Data Science Journal, 18(5): e70045, 2025. doi: https://doi.org/10.1002/sam.70045.the closest neural treatment-bridge estimator.

3 comments

Thank you for your review. We aim to address each of your concerns and questions below as well as in the updated manuscript. We are happy to provide further clarifications if needed. If our revisions and responses resolve your concerns, we would appreciate it if you would increase your score.

Presentation and figures (Weakness 1 / Question 1).

We enlarged the fonts and graphical elements in Figures 1–2 and the fonts in Tables 1–2. To make identification easier to follow, Section 3 now explains the roles of target-specific reweighting, doubly robust residual correction, and conditional normalization, through which one population bridge serves heterogeneous response targets. We welcome specific pointers to any remaining steps that would benefit from clarification.

We hope these revisions make the connection between our identification results, estimators, and statistical guarantees clearer. Together, these contributions provide a framework for population, heterogeneous, and treatment-conditional response estimation under continuous or structured treatments. Under the stated assumptions, our projected-error analysis establishes causal-response consistency without requiring strong L2L^2 convergence of either bridge.

Optimization guarantees and attained error (Weakness 2 / Question 2).

Our bounds quantify how statistical, approximation, and optimization errors affect causal estimation. They are conditional on approximate empirical minimization; we do not claim a global convergence guarantee for the implemented alternating neural optimizer. Section 5 and Theorem 5.1 retain the optimization terms, and the limitations identify optimizer convergence as an open problem.

For example, Chen et al. (2025) establish optimization guarantees for two-layer non-parametric instrumental variable regression (NPIV) networks using a dedicated bilevel mean-field Langevin algorithm with entropy and quadratic regularization. Their analysis suggests a direction for developing optimization guarantees for our method, which is why we cite it when discussing future work. Extending these ideas to our deeper architectures and alternating updates would constitute a much more general and open research problem than the reduced scope of our present statistical analysis for our proxy causal learning problem. Our bounds explicitly quantify how the achieved optimization accuracy affects the statistical error.

Regarding the requested empirical gap, Assumption H.1 requires approximate empirical minimization at each stage:

R^j(ϑ^j)≤inf⁡ϑ∈VjR^j(ϑ)+ξj.\widehat{\mathcal{R}}_{j}(\widehat{\vartheta}_{j}) \le \inf_{\vartheta \in \mathcal{V}_{j}} \widehat{\mathcal{R}}_{j}(\vartheta) + \xi_{j}.

The global infimum is unknown, so training or validation loss cannot be reported as the attained global optimization gap. Moreover, bridge objectives are not causal MSE: their optimization errors must be propagated through the statistical bound, rather than compared directly with MSE differences between methods. Remark H.44 specifies the optimization accuracy sufficient to preserve the statistical rates.

Comparison with Zhang et al. (Question 3).

Zhang et al. study neural treatment-bridge estimation for binary treatments, whereas our main contribution targets continuous and structured treatment spaces. Since binary treatment is a special case of our framework, we added a direct comparison of TreatmentNet with DeepMMR-U and DeepMMR-V on their S1 and S4 designs at n∈{2,000,4,000}n\in\{2{,}000,4{,}000\}. We use the authors' released DGPs, DeepMMR training code, and per-arm configurations.

All methods use a common 75% fit/25% validation split and an independent test sample of 1,000. We compute ATE MSE as (ATE^−ATE)2(\widehat{\mathrm{ATE}}-\mathrm{ATE})^2, average three repetitions within each of ten data-generating seeds, and report the mean ± standard error across seed means.

DesignnnDeepMMR-UDeepMMR-VTreatmentNet
S12,0000.1935 ± 0.03950.1487 ± 0.05210.1860 ± 0.1321
S14,0000.1334 ± 0.06280.0942 ± 0.03190.0267 ± 0.0075
S42,0000.1277 ± 0.01900.1249 ± 0.04330.1232 ± 0.0725
S44,0000.1316 ± 0.08840.0960 ± 0.05260.0233 ± 0.0067

TreatmentNet achieves the lowest mean ATE MSE in three of four settings. At n=4000, it outperforms both DeepMMR variants in mean ATE MSE on both designs. The comparison, DGPs, and experimental protocol are included in Appendix I.8, Table 8, of the revised manuscript.

Thank you for the reply. The DeepMMR comparison answers my question, and I am glad to see TreatmentNet wins in three of the four settings. The Natality study and the ablations are also real work, I appreciate them. I also read your explanation about the optimization question, I will not push it further.

My main complaint is still the presentation. Enlarging the fonts helps, but the real problem is not the font size. The main text is still one formula after another, and there is still no figure that gives an overview of the framework, how the pieces are connected and where the final estimators come from. To be honest, the figure problem in the submitted version is something one careful read by the authors themselves would find. As I wrote in the review, the writing and the figures need more care. So my suggestion is simple: continue to polish the paper, the writing, the figures, and add one roadmap figure of the whole framework. The method may be good, but the current form makes the reader work too hard to see it.

With the new evidence from the rebuttal, I raise my score from 1 to 2. I hope the authors continue improving the manuscript.

Thank you for reconsidering your assessment. We are glad that the new DeepMMR comparison answered your question, and we appreciate your positive feedback on the Natality study and the ablations. During the rebuttal, we initially focused on the additional experiments you and the other reviewers requested.

To address your concern about presentation, we have now added Figures 2 and 3 and Algorithm 1 to Section 4. Figure 2 shows the treatment and outcome bridges side by side and points to the losses used to learn each one. Figure 3 shows how we combine the two bridges to obtain a doubly robust estimate of the population dose-response. We weight the OutcomeNet residuals with the treatment bridge, regress them on dose, and add the fitted correction to the outcome-bridge average. Algorithm 1 brings these steps together in pseudocode so readers can see how to obtain the final estimate at a given dose. In the final version, we will further refine these figures and use them to guide readers through the overall algorithm and the derivation of the estimators.

In light of these revisions, we would be grateful if you would consider raising your recommendation to accept.

Reviewer_3BXSOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DR-PCL-Net, a neural doubly robust method for proxy causal learning with continuous and structured treatments. It introduces TreatmentNet, which learns the treatment bridge with neural features and a density-ratio-free loss, and combines it with OutcomeNet and a final residual regression. The method covers ATE, CATE, and ATT: a single ATE bridge is reused for CATE, and one joint bridge yields the full ATT surface. The authors derive projected-norm error bounds and evaluate the method on synthetic, semi-synthetic, and image-treatment benchmarks.

Critical strengths
  • Useful extension: doubly robust PCL for continuous and image-valued treatments, covering ATE, CATE, and ATT in a single framework.
  • Thoughtful design: the CATE estimator reuses the ATE bridge, and the ATT bridge does not need to be retrained for each reference treatment.
  • Practical engineering: the einsum computation and the KKT normalization make training faster and more stable.
Critical weaknesses
  • The doubly robust guarantee is weak. The error bound only controls the remainder by the smaller of two first-order projected bridge errors, not by their product; the authors state this explicitly after Corollary H.32. The bound also requires the unprojected (strong-norm) bridge error on the complementary route to remain stochastically bounded (Assumption H.31). For bridges that solve ill-posed inverse problems, such boundedness is not automatic, and the paper gives no sufficient condition under which it holds.

  • The theory and the algorithm do not match. The analysis assumes that Stage 1 and Stage 2 are fitted on disjoint folds, with the Stage-1 estimate substituted into Stage 2. The implemented algorithm, however, recomputes an auxiliary first-stage operator on the fly on the current Stage-2 mini-batch and uses it inside the Stage-2 loss on that same batch (Appendix E). This breaks the sample-splitting structure that the proofs rely on, and the limitations paragraph only mentions the lack of optimizer guarantees, not this discrepancy.

  • TreatmentNet alone performs poorly. In Table 2 its error on Demand is far above every other method, and in Figure 3 its error on the original dSprite benchmark does not decrease with sample size. In several settings this also leaves DR-PCL-Net behind single-bridge baselines.

  • Missing experiments. The only CATE experiment uses a binary treatment (Appendix I.4.7), even though continuous-treatment CATE is a stated contribution. There is no sensitivity study for the clipping constant in the self-normalized CATE estimator (Algorithm 5), no ablation isolating the proximal closed-form update, and no runtime report for the pairwise ATT stage. All benchmarks are synthetic or semi-synthetic; there is no evaluation on fully real data.

Questions for authors
  1. Why is the auxiliary first-stage operator recomputed on the Stage-2 batch rather than on the Stage-1 fold, which would preserve sample splitting? Does the theory still apply to the implemented procedure?
  2. Under what conditions does Assumption H.31 hold for bridges that solve ill-posed inverse problems? Can the bound be strengthened to a product rate?
  3. Why does TreatmentNet perform so poorly on Demand and on the original dSprite benchmark, and why does its error not decrease with sample size there?
  4. Could the authors add a continuous-treatment CATE experiment and a sensitivity study for the clipping constant?
5 comments
AuthorsOpenReview ↗

Official Comment by Authors - Part 1/3

We thank you for your careful review. We have updated our manuscript and respond to each of your points below and are happy to clarify any remaining issues. If our responses and revisions resolve your concerns, we would appreciate it if you would consider increasing your score.

Weakness 1 / Question 2: projected double robustness and the bounded strong-error assumption H.31.

Assumption H.31 requires bounded complementary bridge errors, not their convergence to zero. The sufficient condition added after H.31 in Appendix H.6 combines enforceable fitted-bridge bounds with conditional second-moment bounds for the true bridges, without requiring strong recovery of the ill-posed bridges.

A sufficient condition for H.31. Assumptions H.6 and H.15 imply the following almost sure upper bound on the L∞L^{\infty}-norm of the estimators.

sup⁡a,x,w∣h^(a,x,w)∣2≤Mh,sup⁡a,x,z∣φ^(a,x,z)∣≤Mφ,\sup_{a,x,w}\\|\widehat h(a,x,w)\\|_2\le M_h, \qquad \sup_{a,x,z}|\widehat\varphi(a,x,z)|\le M_\varphi,

with constants independent of sample size. For neural bridges, bounded feature norms and coefficient norms enforce these envelopes.

We have shown in Appendix H.3 and Assumption H.20 that

E[∣φ0ATE(a,X,Z)∣2∣A=a]1/2≤Cφ,ATE<∞,E[∣h0(a,X,W)∣22∣A=a]1/2≤Ch<∞,\mathbb E[|\varphi_0^{\mathrm{ATE}}(a,X,Z)|^2\mid A=a]^{1/2} \le C_{\varphi,\mathrm{ATE}} < \infty, \qquad \mathbb E[\\|h_0(a,X,W)\\|_2^2\mid A=a]^{1/2} \le C_h < \infty,

for PAP_A-almost every aa. A straightforward triangular inequality argument yields

Be,φ≤Mφ+Cφ,ATE,Be,h≤Mh+Ch.B_{e,\varphi}\le M_\varphi+C_{\varphi,\mathrm{ATE}}, \qquad B_{e,h}\le M_h+C_h.

Thus Assumption H.31 holds uniformly in sample size.

Projected double robustness and ill-posedness. We emphasize that the key challenge and distinction of our work from prior efforts in the literature on semiparametric estimation in proximal causal learning is the fact that we target an infinite dimensional non-pathwise differentiable target, namely the proximal dose response curve. Optimal estimation and inference theory for the harder problem that we tackle remains in its infancy in the broader statistical literature, and does not address the challenging machine learning setting considered in our work. We emphasize that the familiar phenomenon of bias bounds in terms of a product of estimation rates is inherently attached to the pathwise differentiable setting, which is generally only the case when the treatment is categorical valued and the estimation target is finite dimensional.

The idea in our guarantees can be conveyed by the following identity. Let ff denote the dose response curve and fˉ\bar f the plug-in doubly robust functional (see Lemma H.26). We omit covariates X for simplicity.

f(a)−fˉ(a)=E[φ0(a,Z)−φ^(a,Z)h0(a,W)−h^(a,W)∣A=a].f(a)-\bar{f}(a) = \mathbb{E}[\\{\varphi_0(a,Z) - \widehat{\varphi}(a,Z)\\}\\{h_0(a,W) - \widehat{h}(a,W)\\} \mid A=a].

This is the familiar product error structure, pointwise for a fixed aa. We can project ZZ on (W,A)(W,A) or WW on (Z,A)(Z,A). Indeed,

f(a)−fˉ(a)=E[E[φ0(a,Z)−φ^(a,Z)∣W,A=a]h0(a,W)−h^(a,W)∣A=a]=E[φ0(a,Z)−φ^(a,Z)E[h0(a,W)−h^(a,W)∣Z,A=a]∣A=a].\begin{aligned} f(a)-\bar{f}(a) &= \mathbb{E}[\mathbb{E}[\varphi_0(a,Z) - \widehat{\varphi}(a,Z)\mid W, A=a]\\{h_0(a,W) - \widehat{h}(a,W)\\} \mid A=a]\\\\ &= \mathbb{E}[\\{\varphi_0(a,Z) - \widehat{\varphi}(a,Z)\\}\mathbb{E}[\\{h_0(a,W) - \widehat{h}(a,W)\\}\mid Z, A=a] \mid A=a]. \end{aligned}

Notice that error E[φ0(a,Z)−φ^(a,Z)∣W,A=a]\mathbb{E}[\varphi_0(a,Z) - \widehat{\varphi}(a,Z)\mid W, A=a] is particularly nice, because φ0\varphi_0 solves

E[φ0(A,Z)∣A,W]=PAPWPAW.\mathbb{E}[\varphi_0(A,Z)\mid A,W] = \frac{P_AP_W}{P_{AW}}.

The error with conditional expectation can be interpreted as the training loss for φ^\widehat{\varphi}. Our argument in the paper is nothing else than saying controlling the training loss is much easier than controlling a different loss with an implicit distribution shift. Having controlled the "projected error" we bound the other term by its supremum. This bound has the important advantage of being completely agnostic to how ill-posed the inverse problem is, as it only depends on the estimator function class and our a priori assumption on the true bridge function. We emphasize again that ill-posedness only affects convergence rates, and not L∞L^{\infty} bounds on the estimators. The same observation has also been made in Deaner 2022, Chernozhukov, Newey, and Singh (2023, Theorem 1) and Bennett et al. (2026, Eq. (20)). Chernozhukov et al. call this “double robustness to ill-posedness.” In our setting, conditional Cauchy–Schwarz and integration over treatment values give two projected–strong product bounds (Lemma H.27). Taking their minimum yields

∣fˉATEDR−fATE∣L2(PA;RdY)2≤min⁡Be,φ2Rhproj,Be,h2Rφproj.\\|\bar f_{\mathrm{ATE}}^{\mathrm{DR}}-f_{\mathrm{ATE}}\\|_{L^2(P_A;\mathbb R^{d_Y})}^2\le\min\\{B_{e,\varphi}^2\mathcal R_h^{\mathrm{proj}},B_{e,h}^2\mathcal R_\varphi^{\mathrm{proj}}\\}.

References Victor Chernozhukov, Whitney K. Newey, and Rahul Singh (2023). A simple and general debiased machine learning theorem with finite-sample guarantees. Biometrika. (See Theorem 1 and its discussion, p. 262.)

AuthorsOpenReview ↗

Official Comment by Authors - Part 2/3

Weakness 2 / Question 1: auxiliary Stage-1 refitting and applicability of the theory.

TreatmentNet (and similarly OutcomeNet) distinguishes the persistent Stage-1 operator from an auxiliary profile used to learn the Stage-2 representation.

  • Persistent operator V^t\widehat V_t. This conditional-feature operator is updated on B1,t⊂I1B_{1,t}\subset I_1, with the current proxy featurizer held fixed (Eq. (47)).

  • Auxiliary profile Vˇt\check V_t. This is recomputed on B2,t⊂I2B_{2,t}\subset I_2 using the current proxy features. Differentiating the Stage-2 loss through this closed-form solve trains the second stage proxy featurizer; the (A,X)(A,X) featurizer is updated directly through the same loss. The resulting projected features also enter the update of the final treatment-bridge weights (Eq. (52)). The auxiliary profile does not replace the persistent Stage-1 operator.

Using B2,tB_{2,t} keeps the auxiliary refit, representation update, and final-weight solve within the current Stage-2 minibatch, using the current proxy representation.

This auxiliary computation does not by itself invalidate the conditional statistical guarantee. The uniform bounds allow the Stage-2 featurizers and final treatment-bridge weights to be learned on I2I_2. The guarantee requires the returned estimator to satisfy the approximate-minimization conditions in Assumption H.1 for the criteria using I1I_1-fitted profiles and I2I_2-evaluated bridge losses. The bounds retain these optimization gaps explicitly; establishing sufficiently small gaps for the implemented stochastic optimizer remains open, as stated in the limitations.

Weakness 3 / Question 3: TreatmentNet's standalone performance on Demand and original dSprite.

The relative performance of the two bridge routes depends on the data-generating process. Our controlled experiments illustrate this dependence, while the Demand and dSprite results show that the treatment bridge can improve the DR estimator even when TreatmentNet is less accurate on its own.

Complementary routes. Table 1 demonstrates both directions. In Scenario B, weakening the outcome-proxy measurement increases the variability of the outcome bridge while leaving the exact treatment bridge and its nuisance-learning problem unchanged. TreatmentNet has lower mean MSE than OutcomeNet at four of the five proxy-strength levels. Scenario A produces the opposite ordering. These results motivate learning both routes.

Demand. This benchmark was introduced for a neural outcome bridge, and OutcomeNet is particularly strong. Nevertheless, DR-PCL-Net improves on OutcomeNet at all five reported sample sizes and has the lowest mean MSE among all methods at three sizes. At n=20,000n=20{,}000, it reduces MSE from 20.1820.18 to 15.7715.77. TreatmentNet's own MSE does not determine the value of its bridge within DR-PCL-Net, where it weights outcome-bridge residuals to form a correction.

Original dSprite (Figure 3). In this DGP, ZZ directly copies the scale, rotation, and horizontal-position coordinates of UU, whereas the outcome's confounding term depends on vertical position. The latter is shared by the treatment image AA and proxy image WW before independent pixel noise is added. Thus, WW directly encodes the outcome-relevant latent coordinate, while ZZ copies the other independently sampled coordinates. This asymmetric construction can make treatment-bridge learning difficult and motivated our adapted benchmark, where the treatment and both proxies depend on separate noisy measurements of a common latent confounder.

We retain the original design and results in Appendix I.4.5 and Figure 3 for comparison with prior work. Moreover, DR-PCL-Net achieves lower mean MSE than OutcomeNet at every tested sample size on both the original and modified dSprite benchmarks (Figures 3 and 2), supporting the value of the treatment-bridge correction even when TreatmentNet is less accurate on its own.

To isolate the value of avoiding density-ratio estimation, we also added a matched ablation on modified dSprite. Ratio-free TreatmentNet has lower mean MSE than all evaluated plug-in variants at every tested sample size. The protocols and results are included in Appendix I.7, Table 7, and our response to Reviewer XQK7.

AuthorsOpenReview ↗

Official Comment by Authors - Part 3/3

Weakness 4 / Question 4: real-data evaluation, nonbinary CATE, clipping sensitivity, update ablation, and ATT runtime.

Real-data application and nonbinary CATE. The revised manuscript includes an observational study of 410,597 eligible Pennsylvania births from the 1989–1991 NBER/NCHS Natality Detail Files. Treatment is reported cigarettes smoked per day during pregnancy; the outcome is infant birth weight in grams. We follow Singh's (2023) proxy allocation. Cigarette counts are integer-valued, nonbinary doses, which our models represent on a continuous scale.

Figure 6 reports heterogeneous response curves for White non-Hispanic, Black non-Hispanic, and Hispanic mothers. One population treatment bridge is reused across groups through conditional self-normalization. Figures 4 and 5 additionally report population and treatment-conditional curves; all three figures compare OutcomeNet, TreatmentNet, and DR-PCL-Net.

Without causal ground truth, we assess curve plausibility, split/initialization stability, and treatment support rather than causal accuracy. Section 6 summarizes the application; Appendix I.5 provides the proxy rationale, assumptions, fitting protocol, and detailed results. The study extends evaluation beyond binary CATE.

Clipping sensitivity. Denominator clipping safeguards self-normalization against near-zero estimates. Across all displayed Natality groups, doses through 40, seeds, and rotations, the minimum raw CATE denominators are 0.4520.452 for TreatmentNet and 0.4320.432 for DR-PCL-Net. Thus, any ϵD∈[0,0.4]\epsilon_D\in[0,0.4] leaves the curves unchanged: clipping never activates on the reported grid.

In our synthetic CATE experiment, the estimated denominators were also sufficiently far from zero (approximately 0.680.68) that clipping never activated for any tested ϵD∈0.01,0.05,0.1,0.25,0.5\epsilon_D\in\\{0.01,0.05,0.1,0.25,0.5\\}.

History-centered update ablation. We adopt the approach of Galashov et al. (2025), combining gradient-based feature learning with history-centered closed-form updates of the linear heads. Their experiments report improvements over standard SGD under squared loss. History centering carries information from previous minibatches into each new head solve as the neural features change. Under their approximate Kalman-filter interpretation, the previous head serves as the mean of a Gaussian tracking prior: centering the penalty at that head discourages abrupt changes while retaining a closed-form update.

To assess the effect of history centering, we have added an ablation that changes penalty centering jointly for both final layers: the Stage-1 conditional-feature head and the Stage-2 bridge head. For both OutcomeNet and TreatmentNet, we compare λ∣B−Bt∣F2\lambda\\|B-B_t\\|_F^2 with zero-centered ridge λ∣B∣F2\lambda\\|B\\|_F^2, where BB denotes the coefficient matrix or vector being updated. Both variants retain the same closed-form head solves.

On the synthetic ATT benchmark at n=2,000n=2{,}000, we use ten data-generating seeds, three repetitions per seed, three-way cross-fitting, and four ATT anchors. All other experimental settings are matched. Training times sum all three cross-fitting folds.

EstimatorHistory-centered MSEZero-centered MSEHistory-centered time (s)Zero-centered time (s)
OutcomeNet0.0700 ±\pm 0.00493.1155 ±\pm 0.0120125.7 ±\pm 1.1125.9 ±\pm 1.5
TreatmentNet0.1805 ±\pm 0.00700.3134 ±\pm 0.0122198.1 ±\pm 1.9197.9 ±\pm 2.3
DR-PCL-Net0.0581 ±\pm 0.00500.3030 ±\pm 0.0122307.3 ±\pm 2.5308.1 ±\pm 4.2

Entries are means ±\pm standard errors across the ten seed mean. At matched regularization settings, history centering lowers mean causal MSE for all three estimators at essentially unchanged measured training time.

Pairwise ATT runtime. The treatment bridge's pairwise second stage takes 48.67±0.3948.67\pm0.39 seconds with history-centered updates and 48.73±0.4148.73\pm0.41 seconds with zero-centered ridge, summed across all three cross-fitting folds and already included in the estimator times above. Each fold uses one joint ATT bridge fit for all four anchor curves. Appendix I.6, Table 6, reports the ablation and timing protocol.

Reviewer_XQK7OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies proxy causal learning (PCL) when unobserved confounders are present. It develops a neural doubly robust PCL framework for continuous and structured treatments, covering population, heterogeneous, and treatment-conditional response functions. It introduces neural mean embedding to learn both the treatment and outcome bridge, with a treatment-side objective that avoids explicit density-ratio estimation. It also establishes projected-error bounds and demonstrates competitive or superior performance across synthetic and semi-synthetic benchmarks.

Critical strengths
  1. Solid methodological and theoretical development: The usage of neural mean embedding in the treatment bridge route is sound. The result that a single population bridge can serve all supported subgroups is useful. The projected error bound analysis also provides a consistency result without the strong L2 convergence requirement.

  2. Strong Performance: The experiments include several recent baselines, and demonstrate competitive or superior performance on a diverse set of benchmarks.

Critical weaknesses
  1. Moderate theoretical novelty: This paper combines several existing components, including extensions to continuous treatments and treatment-conditional dose responses, doubly robust PCL, closed form updates of both bridges, and density-ratio-free formulations for treatment bridge identification. The major contributions, adaptive density-free treatment bridge learning, single population bridge used for all subgruops, and the projected error bound, provide moderate novelty. Although these contributions are clearly articulated and useful, I view the theoretical advances as moderate extensions of the existing framework.

  2. Limited evidence of applicability to real observational data. The evaluation relies on synthetic and semi-synthetic benchmarks. These are valuable for measuring causal estimation error against known ground truth, but they provide limited evidence about practical challenges such as selecting suitable proxies and assessing the plausibility of the identifying assumptions in an actual application.

Questions for authors
  1. Table 2 showed that TreatmentNet demonstrated a substantially higher MSE than Outcomenet, DR-PCL-Net, and most baselines. Can you comment on the gap? Does the density-ratio-free formulation provide better empirical performance than the regular density-ratio plug-in approach or simply provide a more convenient interface to estimate continuous treatment response curve? If possible, could you compare Treatmentnet with the conventional plug-in approach as an additional ablation?

  2. Could the authors include a real-world observational case study, or discuss a concrete application in greater detail? Such a study need not establish superior counterfactual accuracy, since ground truth is generally unavailable. Instead, it could illustrate proxy selection, justify the identifying assumptions, and assess the plausibility and stability of the estimated response functions.

4 comments
AuthorsOpenReview ↗

Official Comment by Authors - Part 1/3

Thank you for your detailed feedback. We aim to address your questions and concerns below, as well as in the updated manuscript and would welcome any further questions. If our clarifications and revisions resolve your concerns, we would appreciate it if you would consider increasing your score.

Weakness 1: novelty of the learning procedure, target-specific constructions, and theory.

Below we recall the main novel findings. Our contribution is a representation-learning procedure for doubly robust estimation of population, heterogeneous, and treatment-conditional response functions with continuous or structured treatments. Its novelty lies in the learning and identification constructions that make this procedure possible, together with their statistical guarantees.

Trainable ratio-free TreatmentNet. The computational obstacle is that the ratio-free risk depends on the unobserved conditional projection TφφT_\varphi\varphi, rather than an observed pointwise label. TreatmentNet makes this risk trainable through a linear-in-feature bridge, a learned conditional mean embedding, and differentiable profiling with history-centered head updates. This learns the representations needed for the treatment bridge and its conditional projection, without separately estimating a density ratio or the full conditional proxy law. The construction accommodates high-dimensional and structured treatments and proxies.

New heterogeneous (CATE) and treatment-conditional (ATT) constructions. Building on this learning procedure, we introduce two target-specific constructions. Estimating these response surfaces requires more than adding conditioning variables:

  • Heterogeneous responses: The native target-specific ratio-free risk involves conditional-product expectations indexed by VV (see Appendix F.2). We instead prove a conditional normalization identity: a valid population treatment bridge, divided by its conditional normalizer, induces a valid CATE bridge over supported (a,v)(a,v) (Proposition 3.4). This avoids the additional conditional-product estimation problem. A single population bridge and joint numerator–denominator regression support estimation of the heterogeneous surface through a doubly robust representation (Appendix F.2–F.3). The contribution is the population-bridge reduction and the estimator it enables.

  • Treatment-conditional responses: Starting from established treatment-bridge identification, we construct a joint-anchor ratio-free U-objective that learns the bridge jointly over the intervention aa and factual anchor a′a'. An ordered-pair final regression combines the anchor-dependent residual correction with the transported outcome-bridge prediction. These constructions estimate the full treatment-bridge and doubly robust response surfaces without refitting at each anchor a′a' (Appendix G.2–G.3).

Projected-error analysis. To establish when these fitted response functions are consistent, our analysis propagates errors from the learned embeddings, U-statistic bridge objectives, and final regressions to the fitted causal responses. Under bounded complementary errors, it separates final-stage terms from a doubly robust remainder controlled by the smaller projected bridge error, without requiring strong recovery of the ill-posed bridges or introducing a separate density-ratio estimation error. The guarantees also cover the conditional normalization in CATE and the joint-anchor treatment-conditional construction. For approximate empirical minimizers, they retain approximation, sampling, and optimization errors explicitly (Section 5 and Appendix H).

Question 1: standalone TreatmentNet performance and comparison with density-ratio plug-ins.

Standalone TreatmentNet performance. TreatmentNet can be less accurate in some data-generating processes, which motivated the controlled experiments in Table 1 to examine settings favoring either bridge route. In Scenario B, we weaken the outcome-proxy measurement while keeping the treatment-bridge learning problem fixed. TreatmentNet achieves lower mean MSE than OutcomeNet at four of the five stress levels. These results demonstrate that there are also regimes in which the treatment-bridge route is preferable. (Continued in Part 2/3: density-ratio plug-in comparison..)

AuthorsOpenReview ↗

Official Comment by Authors - Part 2/3

Question 1 (continued): density-ratio ablation.

Density-ratio plug-in ablation in a high-dimensional setting. To directly assess the empirical benefit of avoiding separate density-ratio estimation, we have added a plug-in ablation on our modified dSprite benchmark, where both the treatment and outcome proxy are 64×6464\times64 images. We compare ratio-free TreatmentNet with classifier-based and latent-space-based density-ratio plug-ins. Each plug-in is evaluated under independent and overlapping ratio-fitting protocols. All methods use the same total sample budget and a common 10% validation holdout:

  1. Ratio-free, three folds: circular roles I1I_1 for conditional-feature estimation, I2I_2 for bridge learning, and I3I_3 for the final response. No density ratio is estimated.
  2. Plug-in, four folds: circular roles (R,I1,I2,I3)(R,I_1,I_2,I_3), with the density ratio fitted on the separate fold RR and evaluated on I2I_2.
  3. Plug-in, three folds: the original three-fold construction, with the density ratio fitted on I1∪I2I_1\cup I_2. This overlaps the bridge-fitting observations while excluding I3I_3 and the external validation set.

The bridge architectures, epochs, ensemble sizes, regularization coefficients, optimizers, learning rate schedules are held fixed. The plug-in replaces the ratio-free bridge criterion with squared-error regression on the estimated density ratio:

R^PI(φ)=1∣I2∣∑i∈I2[g^φ(Ai,Wi)−r^(Ai,Wi)]2.\widehat{\mathcal R}_{\mathrm{PI}}(\varphi) = \frac{1}{|I_2|} \sum_{i\in I_2} \left[ \hat g_\varphi(A_i,W_i)-\hat r(A_i,W_i) \right]^2.

Here g^φ\hat g_\varphi is the estimated conditional projection of the bridge, and r^\hat r is fixed during bridge learning.

The four-fold protocol allocates a separate fold to ratio estimation, leaving fewer observations for each remaining role; its results therefore reflect this sample-allocation choice as well as ratio estimation. In the three-fold plug-in, ratio predictions on I2I_2 are in-sample. Ratio fitting and calibration exclude I3I_3 and the external validation set in both protocols.

Density-ratio estimators. The classifier-based estimator uses joint-versus-permuted pair classification and converts the fitted class probabilities to product-to-joint density-ratio odds. The latent-space-based estimator applies KLIEP (Sugiyama et al., 2007) to learned image representations. Appendix I.7 of the revised manuscript describes both density-ratio estimation procedures in detail.

Results. The table reports causal MSE for ratio-free TreatmentNet and its plug-in variants as mean ±\pm standard error across the ten seeds.

nnRatio-free (3 folds)Plug-in: classifier-based (4 folds, independent)Plug-in: classifier-based (3 folds, overlap)Plug-in: latent-space-based (4 folds, independent)Plug-in: latent-space-based (3 folds, overlap)
2,00017.597 ± 0.41922.052 ± 0.25318.568 ± 0.27021.979 ± 0.20818.821 ± 0.189
5,00015.030 ± 0.21918.057 ± 0.17116.435 ± 0.25918.107 ± 0.14316.665 ± 0.164
10,00015.264 ± 0.21516.709 ± 0.17215.957 ± 0.21816.930 ± 0.08116.617 ± 0.093
15,00014.830 ± 0.31516.253 ± 0.19515.976 ± 0.13516.799 ± 0.07616.684 ± 0.131
20,00014.162 ± 0.25816.002 ± 0.13915.770 ± 0.18316.615 ± 0.05516.700 ± 0.132

Ratio-free TreatmentNet has lower mean causal MSE than all four evaluated plug-in variants at every tested sample size. These results support avoiding separate density-ratio estimation in this structured benchmark under the evaluated protocols and training settings. Such estimation introduces an additional nuisance error alongside the embedding and bridge-learning errors, whereas our criterion learns the bridge without fitting this nuisance. We have included the comparison and experimental protocols in Appendix I.7, Table 7, of the revised manuscript.

Reference

Sugiyama, M., Nakajima, S., Kashima, H., von Bünau, P., and Kawanabe, M. (2007). Direct Importance Estimation with Model Selection and Its Application to Covariate Shift Adaptation. Advances in Neural Information Processing Systems 20.

AuthorsOpenReview ↗

Official Comment by Authors - Part 3/3

Weakness 2 / Question 2: observational evidence, proxy selection, and response plausibility.

We have added an observational case study using the 1989–1991 NBER/NCHS Natality Detail Files. After restricting to eligible Pennsylvania-resident singleton births with complete records, the cohort contains 410,597 births. The non-binary treatment is reported cigarettes smoked per day during pregnancy, and the outcome is infant birth weight in grams. This application demonstrates how our method can be applied to observational data with recorded treatments and outcomes.

Proxy selection and assumptions. Selecting suitable proxies is a difficult substantive problem requiring application-specific causal knowledge and input from domain experts. We therefore adopt the published allocation of Singh (2023, Section 6.2 and Appendix J), which was developed in consultation with physicians, and fix it before fitting our models.

Maternal and paternal education form the treatment-side proxies ZZ; live-birth order, infant sex, and Rh sensitization form the outcome-side proxies WW. The adjustment variables XX include parental demographics, prenatal care, alcohol use, medical risks, county, and year. Following Singh, the intended latent confounder is household income, with the partition positing Z⊥Y∣A,U,XZ\perp Y\mid A,U,X and W⊥(A,Z)∣U,XW\perp(A,Z)\mid U,X. These restrictions and completeness remain substantive assumptions. Assessing them, and whether adjusters such as prenatal care and pregnancy weight gain may be downstream of smoking, requires domain judgment.

Targets and fitting protocol. We estimate the population response, the treatment-conditional surface at observed-dose anchors a′∈0,5,10,20a'\in\\{0,5,10,20\\}, and heterogeneous responses for White non Hispanic, Black non-Hispanic, and Hispanic mothers. This illustrates our population, treatment-conditional, and CATE constructions in one application.

Treatment-bridge regularization and initialization were selected on a 20,000-record development subset using outcome-blind held-out bridge risk, checked at 100,000 records, and fixed for the full cohort. We report three random seeds, each with three circular cross-fitting rotations. Their dispersion measures split/initialization stability on this cohort, not sampling uncertainty.

Findings. Over the range of 0–20 cigarettes/day, the fitted population responses are:

Methodf(0)f(0)f(20)f(20)Estimated change
OutcomeNet3,439 g3,198 g−241-241 g
TreatmentNet3,449 g3,184 g−265-265 g
DR-PCL-Net3,437 g3,200 g−237-237 g

All three methods yield negative 20-versus-zero contrasts and similar response scales. The treatment conditional and heterogeneous curves also show negative dose contrasts, while allowing different baseline levels across anchors and groups.

Our high-dose heterogeneous responses remain approximately 2.9–3.2 kg at 40 cigarettes/day. By comparison, Singh's published curves approach a few hundred grams or less for Black and Hispanic smokers at high doses (see Figure 5 and 6 in Singh (2023, Section 6.2)). Our fitted curves therefore avoid these extreme mean birth-weight predictions and are more compatible with the recorded outcome scale. Notice that the eligible cohorts and target populations differ, since Singh analyzes smokers separately by race, whereas our primary cohort also includes nonsmokers.

The revised numerical section summarizes the application and fitted contrasts. Appendix I.5 provides the proxy rationale, cohort construction, support counts, and stability diagnostics. Figures 4–6 show the population (ATE), treatment-conditional (ATT), and heterogeneous (CATE) response curves, respectively, for OutcomeNet, TreatmentNet, and DR-PCL-Net.

Thank you for the detailed response and substantial revision. I appreciate the authors' clarification of the methodological contributions, as well as the additional ablation and real-world observational case study, which substantially address my concerns. I acknowledge that the revision has strengthened the paper and will maintain my positive score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Response to All Reviewers: Additional Experiments and Manuscript Updates

Dear reviewers,

Thank you for your careful reviews. We aim to address your comments in our individual responses and have expanded the empirical evaluation in the revised manuscript, with additions marked in blue. The new experiments include:

  • Natality observational study and nonbinary CATE: Birth-weight responses to smoking dose across population, treatment-conditional, and race-specific targets, using 410,597 births (Section 6; Appendix I.5; Figures 4–6).

  • Density-ratio plug-in ablation: On dSprite benchmark, ratio-free TreatmentNet achieves lower mean MSE than all four evaluated plug-in variants at every tested sample size (Appendix I.7, Table 7).

  • DeepMMR comparison: Using the released code and binary-treatment designs of Zhang et al. (2025), TreatmentNet achieves lower mean ATE MSE than both DeepMMR variants in both designs at 4,000 samples (Appendix I.8, Table 8).

  • History-centered final-layer update ablation: Comparing history-centered and zero-centered penalties jointly for both stage heads shows lower mean ATT MSE with history centering at matched regularization and similar training time (Appendix I.6, Table 6).

We welcome further questions on these experiments or our individual responses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.