- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This work uses a plug-in estimate of the EIF residual variance as a differentiable gating objective to learn a variance-efficient (but not necessarily unbiased) covariate subset in an outer optimization loop around an arbitrary differentiable causal estimator, substantially reducing the base learner’s ATE/CATE estimation errors in high-dimensional, small-sample settings where is small.
Critical strengths
(1) The paper introduces the concept of a variance-efficient adjustment set with an overlap-based worst-case OVB bound.
(2) This work establishes the consistency of the gating objective as a plug-in estimator of under a fixed mask and provides a finite-sample decomposition.
Critical weaknesses
(1) Corollary 3.2 fails to justify VERA's central bias–variance claim. Its bound on the squared omitted-variable bias includes irreducible outcome noise unrelated to confounding and becomes vacuous as , particularly under poor overlap or in high-dimensional settings. Consequently, it cannot establish that VERA's deliberate bias–variance trade-off reduces finite-sample MSE.
(2) The paper does not discuss or compare against the two most relevant lines of work: representation learning methods based on variable disentanglement (DR-CFR and TEDVAE), and TMLE-based methods (Dragonnet and C-TMLE).
Questions for authors
(1) Similar to Weakness (1) (2)
(2) The paper includes the R-learner, which is based on Neyman-orthogonalization as a baseline. However, I would also like to see comparisons with DR-learners (such as DML) to provide a more comprehensive evaluation against doubly robust approaches.
3 comments
Baselines and base learners
(2) The paper does not discuss or compare against the two most relevant lines of work: representation learning methods based on variable disentanglement (DR-CFR and TEDVAE), and TMLE-based methods (Dragonnet and C-TMLE).
Thank you for pointing us towards these lines of work. We consider representation learning methods as complementary to variable selection, since VERA can also select among learned features. We have added DR-CFR, TEDVAE and the TMLE-based methods to the revised related-work section.
We also add Dragonnet and C-TMLE as baselines (see App. G.6), including a version (Dragonnet-TP) that uses the same initialization and training protocol as the other neural learners. VERA + TARNet has the lowest ATE RMSE among these methods at and full-sample IHDP, while C-TMLE performs best in the synthetic setting with . The table reports ATE RMSE:
| Setting | Repetitions | Dragonnet | Dragonnet-TP | C-TMLE | VERA TARNet |
|---|---|---|---|---|---|
| Independent, | 1000 | 0.863 | 1.138 | 0.677 | 0.859 |
| Correlated, | 1000 | 0.760 | 1.115 | 0.635 | 0.764 |
| Independent, | 1000 | 1.434 | 1.326 | 1.159 | 0.738 |
| Correlated, | 1000 | 1.476 | 1.713 | 1.111 | 0.754 |
| IHDP, | 1000 | 3.754 | 2.331 | 2.726 | 2.948 |
| IHDP, | 1000 | 0.342 | 0.583 | 0.748 | 0.289 |
The synthetic settings use , and IHDP uses out-of-sample evaluation.
We also apply VERA to Dragonnet-TP. VERA lowers Dragonnet-TP's CATE error at both sample sizes and it lowers ATE RMSE at but raises it at . The table below reports out-of-sample ATE RMSE and CATE error (mean root-PEHE) over 100 repetitions.
| Metric | Dragonnet | Dragonnet-TP | C-TMLE | VERA TARNet | VERA Dragonnet-TP | |
|---|---|---|---|---|---|---|
| 50 | ATE RMSE | 3.833 | 2.496 | 3.268 | 2.581 | 3.088 |
| 50 | Root-PEHE | 6.222 | 4.820 | 5.312 | 4.762 | 4.575 |
| 672 | ATE RMSE | 0.262 | 0.538 | 0.685 | 0.240 | 0.424 |
| 672 | Root-PEHE | 1.279 | 1.418 | 2.152 | 0.747 | 1.197 |
Dragonnet uses the initialization and optimizer schedule in Shi et al. (2019)'s reference implementation. C-TMLE uses a flexible Super Learner outcome model with regression models, random forests and gradient boosting.
(2) The paper includes the R-learner, which is based on Neyman-orthogonalization as a baseline. However, I would also like to see comparisons with DR-learners (such as DML) to provide a more comprehensive evaluation against doubly robust approaches.
Please note that DML (with XGBoost) is already included as a baseline in the submitted paper. We additionally evaluate AIPW with VERA using T-learner and TARNet outcome models and five-fold cross-fitting of the neural nuisance models in App. G.6. VERA improves the base AIPW learner in all four configurations below, but AIPW has higher RMSE than the outcome-model estimates reported in the submitted paper at this sample size.
The following table reports ATE RMSE over 200 repetitions. At , we compare original VERA with the ungated AIPW models with T-Learner and TARNet as outcome models respectively.
| AIPW outcome model | No gating | VERA | |
|---|---|---|---|
| 100 | T-learner | 2.187 | 1.927 |
| 100 | TARNet | 2.066 | 1.665 |
| 10 | T-learner | 1.542 | 1.479 |
| 10 | TARNet | 1.595 | 1.473 |
Thank you for the detailed rebuttal and additional experiments. The derivation of the new OVB bound, together with the supporting experimental results, has adequately addressed my previous concerns. I will therefore raise my score to Weak Accept. Best of luck!
Thank you for your careful review. We address your concerns one by one in the following.
New OVB bound
We agree that the bound in Corollary 3.2 is loose because it bounds omitted outcome information by the full residual variance and uses a worst-case overlap constant. We replace it with a bound that uses Vres(m∗)≤Vres(mℓ) for a valid adjustment mask mℓ⊇m∗. The new bound is informative for weak confounders, but conservative when an omitted variable strongly predicts both treatment and outcome.
We clarify the connection to MSE in our response to Reviewer w4XD based on the example of the AIPW estimator, where we give a condition for MSE improvement of oracle AIPW (assuming true nuisance functions) based on the increase in outcome-prediction loss.
For the tighter bound, let em=P(A=1∣Xm), αm=A/em−(1−A)/(1−em) and σℓ2=Var(Y∣A,Xmℓ). Under positivity and finite expectations,
\mathbb{E}\left[\sigma_{\ell}^{2}(\alpha_{\ell}^{2}-\alpha_{*}^{2})\right] \mathbb{E}\left[\frac{(e_{\ell}-e_{*})^{2}}{e_{\ell}(1-e_{\ell})}\right].$$ The proof is in Appendix A.2 of the revised manuscript. The variance comparison limits the outcome information that can be lost when covariates are dropped. The first factor bounds this loss after propensity weighting, and the second measures the change in propensities. If the residual variance of the full model is constant with $\sigma^2$, the first factor is $\sigma^2\mathbb E[(\alpha_\ell-\alpha_*)^2]$. **Numerical example.** Consider equally likely treatment groups, Gaussian covariates within each group with common covariance, and a linear outcome with noise SD 1. The kept covariates have modest treatment-group imbalance (a combined standardized mean difference of $\sqrt{1/2}\approx0.71$). We vary the standardized mean difference (SMD) between the treatment groups of one additional confounder which is independent of the other covariates within each treatment group. Dropping it reduces $V_{\mathrm{res}}$ only when the share of residual outcome variance is below the shown threshold, i.e. when it does explain less than a certain amount of outcome variance. The bias is measured in outcome-noise SDs. | Confounder SMD | Variance threshold | Max absolute bias | New absolute-bias bound | |---|---|---|---| | 0.10 | 0.4% | 0.006 | 0.006 | | 0.25 | 2.2% | 0.038 | 0.039 | | 0.50 | 9.2% | 0.16 | 0.17 | | 1.00 | 38% | 0.78 | 1.04 | | 1.50 | 75% | 2.6 | 5.1 | | 2.00 | 95% | 8.7 | 32 | This bound replaces the overlap-based bound in Corollary 3.2 of the revised paper.