acceptodds
Under review as a conference paper at ICLR 2027

CUBO: Uncertainty-Aware Causal Policy Learning via Bayesian Bootstrap

Abstract

Classical causal policy learning methods seek treatment rules via empirical welfare maximization. However, this can select a rule with high estimated welfare but substantial downside uncertainty, resulting in poor generalizability of the policy. We propose CUBO, an uncertainty-aware framework for causal policy learning by controlling policy-targeted uncertainty measured by welfare posterior distribution. CUBO combines common Bayesian-bootstrap weights with a nuisance posterior, inducing a joint law of estimated welfare across candidate policies, and the policy is learned by maximizing a reward that combines posterior mean welfare with lower-tail conditional value-at-risk (CVaR). We establish theoretical guarantee on welfare regret bound, and the performance is comprehensively evaluated on IHDP and ACIC datasets. This work provides a novel policy uncertainty quantification from Bayesian perspective, filling the gap of existing methods to learn trustworthy causal policy in real-world applications.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Causal Policy Learning, Causal Effect, Bayesian Bootstrap, Uncertainty

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_EyS8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CUBO combines Bayesian-bootstrap population weights with a fitted nuisance posterior and learns a treatment policy using posterior mean welfare plus lower-tail CVaR. The paper derives conditional simulation and welfare-regret bounds and evaluates several Bayesian neural backbones on IHDP and ACIC policy-learning tasks.

Critical strengths
  1. A coherent decision-level uncertainty objective. Common posterior scenarios and whole-function nuisance draws preserve relevant dependence across policies; posterior CVaR targets aggregate-welfare uncertainty rather than individual outcome risk.

  2. Useful theoretical care. The analysis covers the actual mean–CVaR decision and finite draw bank, retains optimization error explicitly, and distinguishes cross-fitted from full-sample nuisance estimation.

Critical weaknesses
  1. The central objective is not isolated by the ablations. Table 1 changes nuisance fitting and policy optimization together, without a posterior-mean plug-in or a λ=0\lambda=0 learner using the same Bayesian nuisance model. Figure 4 starts at λ=0.05\lambda=0.05, while bootstrap-only and nuisance-only variants are also absent; the gains therefore do not establish whether the proposed tail objective and combination of uncertainty sources add value beyond the changed estimator.

  2. The evaluation does not directly test the advertised downside protection. Section 5 reports average normalized regret and policy accuracy, which are useful outcome measures but do not reveal whether posterior tail scenarios correspond to genuinely low-welfare decisions. The limited outcome-generating settings leave this question unresolved; a focused repeated-dataset check within an existing benchmark would be more informative about the central claim than further optimization runs.

  3. The combined welfare posterior rests on an unvalidated modeling approximation. Equation (7) assumes conditional independence between population and nuisance uncertainty, although both are learned from the same observations. The paper explicitly acknowledges this approximation, so it is not a hidden error, but does not establish its effect on adverse welfare scenarios; this matters because CVaR makes those scenarios directly determine the learned policy.

  4. The experiments do not directly instantiate the main theoretical construction. Sections 3.3–4 formulate cross-fitted nuisance scores, whereas the reported experiments use full-sample fitted effects. A separate full-sample extension is provided, but its additional complexity and nuisance-bias conditions are not checked for these learners; I therefore cannot treat the theoretical guarantees as established explanations for the observed experimental gains.

  5. The comparison with confidence-based policy learning mixes objectives and policy classes. Section 3.1 contrasts posterior CVaR with a simultaneous lower-confidence-bound objective, but Table 1 implements PoLeCe over a finite grouped-assignment class while CUBO learns a neural policy. This is a legitimate comparison of complete procedures, yet cannot determine whether the Bayesian tail criterion itself provides the claimed advantage over confidence-based decision making.

Questions for authors
  1. Using the existing fitted nuisance draws and the same policy class and budget, how do CUBO, λ=0\lambda=0, the posterior-mean plug-in, and the two single-source uncertainty variants compare?

  2. Within an existing benchmark's outcome-generating setting, does a focused repeated-dataset check show that optimizing the posterior lower tail reduces actual low-welfare outcomes? What evidence supports the combined posterior's representation of those scenarios?

  3. Which full-sample theoretical conditions are intended to cover the reported learners, and how should Table 1's PoLeCe comparison be interpreted given the different policy classes?

2 comments

Thank you for recognizing the value of our decision-level uncertainty objective! We also greatly appreciate your careful reading and constructive feedback, and we are more than glad to provide our responses below.

R1. Can the objective and uncertainty sources be isolated using the existing nuisance draws?

Yes. The added controls reuse the saved nuisance draws and keep the candidate policies fixed. With τˉ(Xi)=S−1∑sτθs(Xi)\bar\tau(X_i)=S^{-1}\sum_s\tau_{\theta_s}(X_i), the four scenario contributions are:

ControlContribution to scenario welfare
Both sourcesBisτθs(Xi)B_{is}\tau_{\theta_s}(X_i)
Nuisance onlyτθs(Xi)/n\tau_{\theta_s}(X_i)/n
Bootstrap onlyBisτˉ(Xi)B_{is}\bar\tau(X_i)
Neither sourceτˉ(Xi)/n\bar\tau(X_i)/n

Removing CVaR sets λ=0\lambda=0 while retaining the mean component. The following comparisons use the same fitted model, eight candidate rules, 100-draw bank, and fixed loss settings, with β=.5\beta=.5, α=.10\alpha=.10, and λ=.75\lambda=.75. Entries are test means over 200 realizations in outcome units. Bold full-method values are lower than every displayed ablation in the same row.

Dataset / backboneMetricFull CUBOWithout CVaRWithout bootstrap uncertaintyWithout nuisance uncertaintyWithout either uncertainty
IHDP / BTARNetUpper 5% regret2.8963.0132.9103.0003.000
IHDP / BTARNetUpper 20% regret1.0341.0971.0401.0941.094
IHDP / BDragonnetUpper 5% regret2.8062.9872.8482.9872.987
IHDP / BDragonnetUpper 20% regret1.0011.1191.0221.1241.124
ACIC 2017 / BT-LearnerUpper 5% regret0.5100.7390.5100.7390.739
ACIC 2017 / BT-LearnerUpper 20% regret0.1960.3120.1960.3120.312

Full CUBO achieves the lowest upper-5% and upper-20% regret in every reported matched case. Thus the saved-draw controls demonstrate gains from CVaR and nuisance variation, with smaller gains from bootstrap reweighting. For the positive mean–CVaR reward alone, fixed effects and unrestricted treatment make the sign plug-in optimal in every bootstrap-only scenario. Appendix D.6, Table 4 reports these matched comparisons.

R2. Do repeated datasets show actual downside protection?

Thanks for the concern. Following your suggestion, we have added results evaluating individual-regret tails across repeated realizations in datasets. For a test split of size mm, the paper's notation gives τi=μ1,0(Xi)−μ0,0(Xi)\tau_i=\mu_{1,0}(X_i)-\mu_{0,0}(X_i), πi=π(Xi)\pi_i=\pi(X_i), and ri=max⁡(τi,0)−πiτir_i=\max(\tau_i,0)-\pi_i\tau_i. We evaluate $ \rm{Reg}(\pi)=\frac1m\sum_i r_i,$ and

\quad \rho\in\\{0.05,0.20\\}.$$ We average these metrics equally over 200 distinct realizations per dataset. Each Bayesian plug-in is followed by CUBO using the same fitted model and candidate rules. Bold CUBO entries improve over the plug-in method. **Noisy IHDP and ACIC 2017 high-noise over 200 realizations.** | Method | IHDP upper 5% regret | IHDP upper 20% regret | ACIC 2017 upper 5% regret | ACIC 2017 upper 20% regret | |---|---|---|---|---| | T-Learner | 4.384 | 1.953 | 0.923 | 0.422 | | BT-Learner plug-in | 3.207 | 1.343 | 0.739 | 0.312 | | BT-Learner + CUBO | **2.765** | **1.051** | **0.510** | **0.196** | | CFRNet | 4.614 | 2.150 | 1.085 | 0.731 | | BCFRNet plug-in | 1.382 | 0.421 | 1.064 | 0.734 | | BCFRNet + CUBO | 1.382 | 0.421 | **0.956** | **0.395** | | TARNet | 4.896 | 2.117 | 1.133 | 0.535 | | BTARNet plug-in | 3.000 | 1.094 | 0.765 | 0.328 | | BTARNet + CUBO | **2.896** | **1.034** | **0.590** | **0.218** | | Dragonnet | 4.895 | 2.124 | 1.092 | 0.473 | | BDragonnet plug-in | 2.987 | 1.124 | 0.826 | 0.337 | | BDragonnet + CUBO | **2.806** | **1.001** | **0.689** | **0.247** | | DeRCFR | 2.029 | 0.709 | 0.525 | 0.249 | | BDeRCFR plug-in | 1.620 | 0.507 | 0.435 | 0.224 | | BDeRCFR + CUBO | **1.588** | **0.480** | **0.403** | **0.182** | | ESCFR | 1.685 | 0.563 | 0.995 | 0.682 | | Bayesian ESCFR plug-in | 1.389 | 0.429 | 0.970 | 0.671 | | Bayesian ESCFR + CUBO | **1.361** | **0.415** | **0.963** | **0.600** | | CEVAE | 2.870 | 0.952 | 1.463 | 0.923 | | PT-CATE | 5.828 | 3.333 | 1.496 | 0.805 | | DR-EWM | 6.752 | 4.168 | 1.116 | 0.441 | | PoLeCe | 6.732 | 4.046 | 0.882 | 0.420 | CUBO reduces both regret tails for five of six Bayesian backbones on noisy IHDP and all six on ACIC 2017. For example, BT-Learner upper-5% regret on IHDP decreases from 3.207 to 2.765. These reductions indicate lower severity of costly treatment errors on both datasets. Across the two noisy benchmarks, adding CUBO lowers both regret tails in almost all Bayesian comparisons. These results support its effectiveness in reducing policy regret across repeated datasets. The revised paper reports the noisy regret tails in Appendix D.5, Tables 2–3, and distinguishes them from posterior welfare CVaR.

R3. Is conditional independence between population and nuisance uncertainty validated?

Thanks for your good question. Conditional independence defines the scenario law in Equation (7):

L(B,θ∣Dn)=L(B∣Dn)qn(θ).\mathcal L(B,\theta\mid\mathcal D_n)=\mathcal L(B\mid\mathcal D_n)q_n(\theta).

We added concordant and discordant whole-draw pairings that preserve the saved population-weight and nuisance marginals while changing their dependence. For noisy IHDP with Bayesian ESCFR, mean upper-5% regret is 1.797 under concordant pairing and 1.461 under discordant pairing, compared with 1.666 under independent pairing. Their purpose is to quantify how coupling can change the decision. For welfare laws PVπP_V^\pi and P~Vπ\widetilde P_V^\pi, the hinge representation gives

∣Rα,λ,nP(π)−Rα,λ,nP~(π)∣≤(1−λ+λα)W1(PWπ,P~Wπ).|R^P_{\alpha,\lambda,n}(\pi)-R^{\widetilde P}_{\alpha,\lambda,n}(\pi)| \le\left(1-\lambda+\frac{\lambda}{\alpha}\right) W_1(P_W^\pi,\widetilde P_W^\pi).

The factor makes dependence sensitivity particularly relevant at small α\alpha. We have stated the conditional-independence approximation in Section 2.3.

R4. Do the experiments instantiate the cross-fitted theoretical construction?

The experiments use full-sample fitted effects. The cross-fitted orthogonal result and Proposition 2 cover different constructions. Cross-fitting permits conditional empirical-process control for held-out scores. For the experimental score ϕθ(O;π)=π(X)τθ(X)\phi_\theta(O;\pi)=\pi(X)\tau_\theta(X), Proposition 2 instead includes joint policy–nuisance complexity:

\qquad \mathcal F=\{\phi_\theta(\cdot;\pi):\pi\in\Pi,\theta\in\Theta\}.$$ Its nuisance bias is first order, i.e., $b_n\le\int\|\tau_\theta-\tau_0\|_{L_2(\mathcal{P}_0)}q_n(d\theta).$ Freezing a model before policy optimization fixes the conditional simulation problem, while its statistical dependence on the fitting sample remains part of the full-sample analysis. We have made the score distinction explicit in Section 3.3 and Appendix D.3 and clarified the full-sample guarantee's scope in Section 4 and Appendix C.4. ### R5. How should PoLeCe be interpreted when its policy class differs? **The main-paper comparison evaluates complete procedures. We added a separate common-class experiment to isolate the selection criterion.** All four criteria receive the same eight candidate policies, observed-validation posterior DR scores, nuisance information, and observations. CUBO uses $\beta=.5$, $\alpha=.10$, and $\lambda=.75$. The simultaneous LCB uses its internal 95% confidence calibration and 2,000 multiplier draws. The following BDeRCFR results are means over 200 realizations: | Criterion | IHDP upper 5% regret | IHDP upper 20% regret | ACIC upper 5% regret | ACIC upper 20% regret | |---|---|---|---|---| | Posterior mean | 1.934 | 0.657 | 0.513 | 0.224 | | Mean–CVaR | 1.953 | 0.670 | 0.620 | 0.294 | | CUBO | **1.823** | **0.595** | **0.423** | **0.173** | | Simultaneous LCB | 1.977 | 0.682 | 0.661 | 0.309 | The table shows that CUBO achieves the lowest upper-5% and upper-20% regret on both datasets within the same candidate-policy class. Bold values are best within each metric column. Thus these results further support CUBO's contribution to reducing upside regret under shared nuisance information. Because validation also selected the nuisance checkpoint, the comparison evaluates criterion behavior under shared information. Appendix D.2 describes the PoLeCe baseline, and Appendix D.6, Table 5 reports this comparison.
Reviewer_bedSOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how to learn treatment policies when estimates of their expected outcomes are uncertain. The proposed CUBO method combines Bayesian bootstrap weights over the observations with draws from fitted nuisance models to obtain a distribution of welfare for each candidate policy. It then selects a policy by balancing average posterior welfare against performance in the lower tail of the welfare distribution, thereby accounting for both expected outcomes and downside risk.

Critical strengths

The method accounts for uncertainty in the welfare of an entire treatment policy. Drawing whole nuisance functions preserves dependence among predicted treatment effects, while the bootstrap captures uncertainty in population-level averages. Using the same posterior scenarios to evaluate different policies also makes their comparison more consistent.

The theoretical analysis provides a solid foundation for the proposed decision rule. The proofs address several important technical challenges, including bootstrap normalization, dependence introduced by cross-fitting, and uniform approximation over the policy class.

Critical weaknesses

The paper needs to clarify more precisely how its methodological contribution differs from existing work. BROIL (Brown et al., NeurIPS 2020) and PG-BROIL (Javed et al., ICML 2021) already combine posterior expected policy performance with lower-tail CVaR under reward uncertainty. PG-BROIL also develops policy updates that focus on adverse posterior scenarios. Ye's Bayesian policy-learning framework (2026), which the paper cites, uses bootstrap welfare distributions and establishes posterior regret bounds under a known propensity score. CUBO extends this line of work by incorporating uncertainty from fitted nuisance models and deriving corresponding error bounds. The paper should more clearly explain what is gained by combining these elements in a causal policy learning framework and be more precise in its claims about introducing a new CVaR-based measure of uncertainty. The regret guarantees establish theoretical properties of the proposed rule under the stated assumptions, but they do not, by themselves, demonstrate the practical benefit of prioritizing lower-tail welfare.

The experiments do not establish how much the uncertainty treatment contributes or how effectively it protects against poor welfare outcomes. The main comparisons with plug-in methods vary several factors at once. As a result, it is difficult to determine whether the reported improvements come from the uncertainty treatment or from other differences in the methods. The sensitivity analysis would be more informative if it included posterior-mean welfare optimization using the same fitted model and policy class, along with ablations that separately remove bootstrap uncertainty and nuisance-model uncertainty.

Appendix D.1 also relies on a single selected IHDP realization and one ACIC condition and outcome file. Results on these examples provide limited evidence about how consistently the method reduces adverse welfare outcomes across datasets or how well the estimated lower tails reflect actual policy performance. This is particularly important because protecting against poor welfare outcomes is central to the paper's motivation. The quality of the uncertainty approximation also needs closer examination. In Appendix D.3, the authors adjust BDragonnet's dropout rates based on validation results for ACIC because the original posterior was too diffuse for policy optimization. This raises questions about how sensitive CUBO’s performance is to the calibration of the underlying posterior and whether its benefits persist when uncertainty is estimated without such adjustments.

Questions for authors

What is the main contribution beyond existing methods? Which aspect of CUBO's causal welfare construction goes beyond Bayesian-bootstrap policy learning and the posterior mean-CVaR objectives used in BROIL and PG-BROIL? A more direct comparison of the statistical objectives and theoretical guarantees would help clarify the paper's contribution.

How much does the proposed criterion contribute to performance? Using the same fitted Bayesian model, policy class, and optimization budget, how does CUBO compare with optimizing posterior-mean welfare alone? What happens when population weights are fixed or nuisance parameters are held fixed?

Does CUBO reliably reduce poor welfare outcomes, and how well does it estimate downside risk? Across repeated benchmark runs, does the method reduce the frequency or severity of poor realized welfare outcomes? How well do the estimated lower tails predict these outcomes, and how sensitive are the results to the dropout approximation and its settings?

Are the assumptions behind the theoretical guarantees satisfied by the implemented method? Appendix D.3 uses effect-based scores estimated from the same observations used for policy training and notes the additional conditions required by Proposition 2. Which of these conditions can be verified for the models used in the experiments? In particular, how are score boundedness, nuisance-estimation error, and the complexity of the joint class involving the fitted treatment-effect input addressed?

2 comments

Thank you for recognizing the strengths of our uncertainty formulation! We appreciate your helpful suggestions and address your concerns below. The added tables use 200 distinct realizations of noisy IHDP and ACIC 2017 high-noise data.

R1. What is the contribution beyond BROIL, PG-BROIL, and Bayesian-bootstrap policy learning?

CUBO's contribution is the construction and analysis of learned causal-welfare scenarios and the policy selected from them. Posterior mean–CVaR optimization is established in BROIL and PG-BROIL. The five distinctions are:

  • Problem. BROIL and PG-BROIL optimize returns under uncertain task rewards in sequential decision problems. CUBO targets one-stage causal treatment welfare W(π;P0)=E0[π(X){Y(1)−Y(0)}]W(\pi;\mathcal P_0)=E_0[\pi(X)\{Y(1)-Y(0)\}] from observational data, where one potential outcome is observed and assignment may depend on covariates. Ye's Bayesian-bootstrap policy learning shares this causal welfare target.

  • Formulation. Ye's main construction reweights reduced-form contributions using known propensities. CUBO forms Ws(π)=∑iBisϕθs(Oi;π)W_s(\pi)=\sum_iB_{is}\phi_{\theta_s}(O_i;\pi), allowing both population masses and learned causal-score contributions to vary. Whole nuisance-function draws preserve dependence across individuals. The mean–CVaR component follows BROIL, applied here to a learned causal-welfare distribution.

  • Technical challenge. Learned scores introduce nuisance bias and fitting dependence. CUBO analyzes orthogonal-score bias through the outcome–propensity error product, effect-score bias through effect error, and simulation error uniformly over policies and CVaR thresholds. The returned-policy bound also retains the optimizer's actual gap. These terms connect causal estimation, a shared finite scenario bank, and policy selection.

  • Insights. A policy's welfare uncertainty depends on whom it treats and on covariance across fitted effects. Pointwise CATE intervals alone do not determine this distribution. The factor 1−λ+λ/α1-\lambda+\lambda/\alpha quantifies how greater tail sensitivity increases nuisance and simulation accuracy requirements. Ye analyzes posterior optimal welfare, optimal rules, and posterior expected-welfare choice. CUBO analyzes the mean–CVaR reward and the returned policy's optimization gap.

  • Applicable settings. CUBO addresses offline observational treatment choice with flexible nuisance models, under consistency, unconfoundedness, overlap, and the stated score conditions. Ye's analyzed formulation uses known assignment probabilities. BROIL and PG-BROIL address sequential reward uncertainty, with PG-BROIL using policy rollouts. These settings motivate different score constructions and data requirements. For example, the orthogonal causal score is

Γθ(O)=μ1,θ(X)−μ0,θ(X)+A{Y−μ1,θ(X)}eθ(X)−(1−A){Y−μ0,θ(X)}1−eθ(X).\Gamma_\theta(O)=\mu_{1,\theta}(X)-\mu_{0,\theta}(X) +\frac{A\{Y-\mu_{1,\theta}(X)\}}{e_\theta(X)} -\frac{(1-A)\{Y-\mu_{0,\theta}(X)\}}{1-e_\theta(X)}.

For the mean–CVaR reward, Corollary 2 bounds the returned policy's regret by 2Δn,S+ξn,S2\Delta_{n,S}+\xi_{n,S}, where Δn,S\Delta_{n,S} includes statistical, nuisance, and finite-bank approximation errors, and ξn,S\xi_{n,S} is the optimization gap. Appendix A introduces Ye's Bayesian policy-learning framework and explains the distinctions in welfare construction and decision analysis. We have revised the contribution statement in Section 1 around this learned-score and decision analysis and highlighted the guarantee's scope in Section 4, with details in Appendix C.2.

R2. How much does the criterion contribute under matched models, classes, and budgets?

Thanks for your concern. We add an ablation comparison isolating CVaR and the uncertainty sources under shared nuisance models, candidate rules, and optimization settings. All columns use the same eight candidate rules and 100-draw bank, with β=.5\beta=.5, α=.10\alpha=.10, and λ=.75\lambda=.75. Removing CVaR sets λ=0\lambda=0, removing bootstrap uncertainty fixes Bis=1/nB_{is}=1/n, and removing nuisance uncertainty fixes τθs(Xi)=τˉ(Xi)\tau_{\theta_s}(X_i)=\bar\tau(X_i).

Each entry is a mean over 200 realizations in original outcome units. Bold full-method values are lower than every displayed ablation in the same row.

Dataset / backboneMetricFull CUBOWithout CVaRWithout bootstrap uncertaintyWithout nuisance uncertaintyWithout either uncertainty
IHDP / BTARNetUpper 5% regret2.8963.0132.9103.0003.000
IHDP / BTARNetUpper 20% regret1.0341.0971.0401.0941.094
IHDP / BDragonnetUpper 5% regret2.8062.9872.8482.9872.987
IHDP / BDragonnetUpper 20% regret1.0011.1191.0221.1241.124
ACIC 2017 / BT-LearnerUpper 5% regret0.5100.7390.5100.7390.739
ACIC 2017 / BT-LearnerUpper 20% regret0.1960.3120.1960.3120.312

Full CUBO achieves the lowest value in all six reported regret comparisons. Removing CVaR or nuisance variation increases both regret tails, while population reweighting contributes smaller gains. Because nuisance fits and candidate policies are unchanged, these differences demonstrate the contribution of CUBO's criterion and scenario construction in the reported cases. Appendix D.6, Table 4 reports the same matched ablations.

R3. Does the lower tail predict actual poor welfare, and how sensitive is it to dropout?

Thank you for raising this point. We now evaluate upper-5% and upper-20% regret over 200 distinct realizations per dataset. These metrics quantify the severity of the most costly individual treatment errors. On noisy IHDP and ACIC 2017, adding CUBO reduces both regret tails in almost all Bayesian comparisons. Tables 2–3 in Appendix D.5 report these results, providing evidence of reduced individual decision risk. Posterior CVaR, in turn, targets uncertainty in aggregate welfare.

Regarding dropout, its settings affect the nuisance posterior approximation and hence the welfare-tail distribution. Appendix B.2 explains dropout training and weight regularization, while Appendix D.3 specifies the settings and validation-based calibration. Within each matched comparison, dropout settings and the fitted posterior are held fixed, so the regret differences reflect the policy criterion under a shared uncertainty law. Welfare-tail fidelity therefore conditions on the fitted approximation.

R4. Which theoretical assumptions are verified for the full-sample learners?

Proposition 2 is the relevant result for the experimental full-sample effect scores ϕθ(O,π)=π(X)τθ(X)\phi_\theta(O,\pi)=\pi(X)\tau_\theta(X). Its sampling term includes policy and nuisance complexity CMv+dF+log⁡(1/δ)n,CM\sqrt{\frac{v+d_{\mathcal F}+\log(1/\delta)}{n}}, where F={ϕθ(⋅,π):π∈Π,θ∈Θ}.\mathcal F=\{\phi_\theta(\cdot,\pi):\pi\in\Pi,\theta\in\Theta\}.

For these scores, nuisance bias is first order, bn≤∫∥τθ−τ0∥L2(P0)qn(dθ).b_n\le\int\|\tau_\theta-\tau_0\|_{L_2(P_0)}q_n(d\theta). The implementation verifies bounded treatment probabilities and numerical optimization within the fixed rule bank. The deterministic population score envelope, posterior-integrated nuisance accuracy, and joint-class entropy rate remain theoretical assumptions, rather than experimentally verified rates. The neural optimization gap remains explicit.

We have clarified the experimental full-sample score in Section 3.3 and Appendix D.3 and its theoretical applicability limits in Section 4. You can check the full-sample conditions in Appendix C.4 and the finite-bank decision guarantee in Appendix C.2.

Reviewer_PmNJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes CUBO, a framework for learning treatment policies that accounts for uncertainty in estimated welfare. It combines common Bayesian-bootstrap weights, representing finite-sample uncertainty, together with a fitted posterior over nuisance components, representing estimation uncertainty. These draws produce a joint posterior distribution of estimated welfare across candidate policies. Policy learning then maximizes a weighted combination of posterior mean welfare and lower-tail conditional value-at-risk (CVaR), the average welfare in the least favorable posterior fraction.

The main contributions of this paper are (1) the explicit treatment of policy-targeted uncertainty through the posterior distribution of welfare from a Bayesian perspective, (2) its incorporation into a posterior mean-CVaR objective, and (3) an alternating optimization algorithm for the policy and CVaR threshold. The theoretical analysis provides posterior welfare-regret and uniform Monte Carlo approximation bounds under bounded-score, policy-class regularity, and nuisance-accuracy assumptions. Experimental validation makes use of two well-known real-world benchmark datasets: IHDP and ACIC 2016, where treatment effects are known. CUBO is evaluated with five Bayesian CATE neural backbones against treatment-effect plug-ins and policy learners, including DR-EWM and PoLeCe. The reported empirical results show lower test regret than each matched non-Bayesian plug-in and the best overall test regret on both benchmarks. IHDP ablations examine the CVaR weight, tail probability, and policy-network depth.

Critical strengths

(S1) The focus on uncertainty at the policy-welfare level seems to be well-motivated. A treatment rule can look attractive on average while its estimated benefit remains highly uncertain. CUBO brings this concern into policy learning procedure by combining posterior mean welfare with a lower-tail criterion. Its treatment of both sampling and nuisance-estimation uncertainty makes the formulation relevant to the decision being made.

(S2) The concrete regret analysis is another substantive strength. Appendix C.2 covers the policy selected by optimizing the mean–CVaR objective, including its finite-simulation implementation, and retains an explicit optimization error. These connections to the implemented alternating procedure for policy learning via CUBO seems to be useful. Under the stated assumptions, the results clarify how statistical estimation, posterior simulation, and optimization each affect the learned policy’s performance.

Critical weaknesses

(W1) It seems that the paper does not yet show that CUBO reliably avoids poor policy choices. The evaluation reports regret and policy accuracy on one selected instance of each benchmark, without variability estimates in the main comparisons. However, these results do not tell us how often this happens across different training datasets, or how severe those failures are. The theoretical results also do not establish that CUBO avoids these failures more effectively than posterior-mean learning methods. Since avoiding unfavorable outcomes is the main motivation for introducing CVaR, the lack of direct evidence for this benefit is a significant concern.

(W2) The experiments do not clearly show whether the uncertainty objective of CUBO is responsible for the reported gains. Compared with the corresponding plug-in methods, CUBO also changes how the treatment-effect models are fitted and adds a separate policy optimization step. Either change could contribute to the improvement. The key missing comparison is with a version using the same Bayesian models and policy optimization procedure, but without CVaR (λ=0\lambda = 0). Similarly, comparing bootstrap uncertainty alone, nuisance-model uncertainty alone, and both together would show whether combining the two sources helps. These comparisons matter because they would show whether the main methodological developments of the paper provide a meaningful benefit.

Questions for authors

Most of my questions are included in the "Critical Weaknesses" section. The following questions are related to the notations and minor technical details of the paper:

(Q1) The notion of weights ωμ\omega_{\mu}, ωe\omega_{e}, and ωτ\omega_{\tau} was a bit confusing. What is the key difference against the nuisance components μ\mu, ee, and τ\tau?

2 comments

Thank you for recognizing the motivation and regret analysis of our work! We appreciate your constructive feedback and address your concerns below.

R1. Does CUBO reliably avoid poor policy choices across training datasets?

We now evaluate costly treatment errors and PA over 200 distinct realizations per dataset. The added benchmarks are IHDP with residual SD multiplied by five and official ACIC 2017 high-noise data with η=1.25\eta=1.25. Following the paper, for a test split of size mm, write τi=μ1,0(Xi)−μ0,0(Xi)\tau_i=\mu_{1,0}(X_i)-\mu_{0,0}(X_i) and πi=π(Xi)\pi_i=\pi(X_i). Individual and mean regret are

{\rm Reg}(\pi)=\frac1m\sum_{i=1}^m r_i.$$ For $\rho\in\\{0.05,0.20\\}$, upper-tail regret is \$ \rm{UReg}\_{\rho}(\pi)=\inf_{t\in\mathbb{R}}\\{ t+\frac{1}{\rho m}\sum_{i=1}^m(r_i-t)_+\\}.\$ This averages the largest $\rho$ fraction of individual regrets, with fractional mass at the cutoff. PA measures expected agreement with the oracle rule: $$\rm{PA}(\pi)=\frac{100}{m}\sum_{i=1}^m \left[\pi_i\mathbf1\\{\tau_i>0\\}+(1-\pi_i)\mathbf{1}\\{\tau_i\le0\\}\right].$$ We compute these metrics within each realization and average them equally over 200 realizations. Lower regret is better. The following table includes all ten main-paper baselines. Each Bayesian plug-in is followed by CUBO using the same fitted model, posterior bank, and candidate rules. **Bold CUBO entries improve over the immediately preceding Bayesian plug-in.** **Noisy IHDP and ACIC 2017 high-noise: test means over 200 distinct realizations per dataset.** | Method | IHDP upper 5% regret | IHDP upper 20% regret | ACIC 2017 upper 5% regret | ACIC 2017 upper 20% regret | |---|---|---|---|---| | T-Learner | 4.384 | 1.953 | 0.923 | 0.422 | | BT-Learner plug-in | 3.207 | 1.343 | 0.739 | 0.312 | | BT-Learner + CUBO | **2.765** | **1.051** | **0.510** | **0.196** | | CFRNet | 4.614 | 2.150 | 1.085 | 0.731 | | BCFRNet plug-in | 1.382 | 0.421 | 1.064 | 0.734 | | BCFRNet + CUBO | 1.382 | 0.421 | **0.956** | **0.395** | | TARNet | 4.896 | 2.117 | 1.133 | 0.535 | | BTARNet plug-in | 3.000 | 1.094 | 0.765 | 0.328 | | BTARNet + CUBO | **2.896** | **1.034** | **0.590** | **0.218** | | Dragonnet | 4.895 | 2.124 | 1.092 | 0.473 | | BDragonnet plug-in | 2.987 | 1.124 | 0.826 | 0.337 | | BDragonnet + CUBO | **2.806** | **1.001** | **0.689** | **0.247** | | DeRCFR | 2.029 | 0.709 | 0.525 | 0.249 | | BDeRCFR plug-in | 1.620 | 0.507 | 0.435 | 0.224 | | BDeRCFR + CUBO | **1.588** | **0.480** | **0.403** | **0.182** | | ESCFR | 1.685 | 0.563 | 0.995 | 0.682 | | Bayesian ESCFR plug-in | 1.389 | 0.429 | 0.970 | 0.671 | | Bayesian ESCFR + CUBO | **1.361** | **0.415** | **0.963** | **0.600** | | CEVAE | 2.870 | 0.952 | 1.463 | 0.923 | | PT-CATE | 5.828 | 3.333 | 1.496 | 0.805 | | DR-EWM | 6.752 | 4.168 | 1.116 | 0.441 | | PoLeCe | 6.732 | 4.046 | 0.882 | 0.420 | The table shows lower regret tails for CUBO in five of six Bayesian comparisons on noisy IHDP and all six on ACIC 2017. For example, BT-Learner upper-5% regret on IHDP decreases from 3.207 to 2.765. On ACIC 2017, BDeRCFR + CUBO achieves the lowest upper-5% regret (0.403) and upper-20% regret (0.182). Appendix D.5 reports the evaluation protocol and these results in Tables 2–3.

R2. Are the gains due to CVaR, nuisance fitting, or policy optimization?

Thanks for your concern. We add ablation comparisons isolating CVaR and each uncertainty source within the same policy class. Each column uses the same fitted Bayesian model, frozen posterior bank, eight candidate rules, and fixed loss settings, with β=.5\beta=.5, α=.10\alpha=.10, λ=.75\lambda=.75, and S=100S=100. Removing CVaR sets λ=0\lambda=0 while retaining the posterior mean component. Removing bootstrap uncertainty fixes Bis=1/nB_{is}=1/n. Removing nuisance uncertainty fixes τθs(Xi)=τˉ(Xi)\tau_{\theta_s}(X_i)=\bar\tau(X_i). Removing both fixes both quantities.

Matched ablations: test means over 200 realizations. Bold values are lower than every displayed ablation in the same row.

Dataset / backboneMetricFull CUBOWithout CVaRWithout bootstrap uncertaintyWithout nuisance uncertaintyWithout either uncertainty
IHDP / BTARNetUpper 5% regret2.8963.0132.9103.0003.000
IHDP / BTARNetUpper 20% regret1.0341.0971.0401.0941.094
IHDP / BDragonnetUpper 5% regret2.8062.9872.8482.9872.987
IHDP / BDragonnetUpper 20% regret1.0011.1191.0221.1241.124
ACIC 2017 / BT-LearnerUpper 5% regret0.5100.7390.5100.7390.739
ACIC 2017 / BT-LearnerUpper 20% regret0.1960.3120.1960.3120.312

Full CUBO achieves the lowest upper-5% and upper-20% regret in all three reported matched comparisons. Removing CVaR or nuisance variation increases both regret tails, while removing bootstrap reweighting also gives smaller increases. Because the fits and candidate policies are shared, these results support the contributions of the risk criterion and combined uncertainty construction to CUBO's downside protection. Appendix D.6, Table 4 reports these ablations in the manuscript's result-table format.

R3. What is the difference between the model weights and nuisance functions?

Thanks for the notice. We have clarified this distinction in Section 2.1, with nuisance-function draws and bootstrap population weights described in Appendix B.2–B.3:

NotationMeaning
wμ,we,wτw_\mu,w_e,w_\tauResponse, propensity, or effect-model parameters
μa,θ(x)\mu_{a,\theta}(x)Fitted conditional outcome mean
eθ(x)e_\theta(x)Fitted treatment propensity
τθ(x)\tau_\theta(x)Fitted effect, directly modeled or obtained from outcome means
BisB_{is}Bootstrap population mass for observation ii in scenario ss

A parameter draw θs\theta_s induces one whole nuisance function τθs(⋅)\tau_{\theta_s}(\cdot). Together with a bootstrap draw, it gives Ws(π)=∑iBisπ(Xi)τθs(Xi)W_s(\pi)=\sum_iB_{is}\pi(X_i)\tau_{\theta_s}(X_i). Every candidate policy is evaluated with the same scenario pair (Bs,θs)(B_s,\theta_s).

Comments on the paper

6 comments

Thank you for the detailed response. The clarified contribution, matched uncertainty ablations, and additional results address important parts of my concerns. The common-policy-class comparison and dependence-sensitivity experiment are also helpful. My main remaining concern is the relationship between the evaluated risk metric and the method’s objective. The upper-5% and upper-20% regret summaries measure individual treatment errors, but posterior CVaR targets uncertainty in aggregate policy welfare. Could you report the aggregate welfare or aggregate regret of the selected policy in each realization, including its distribution across repeated training datasets, and compare it with matched posterior-mean optimization? Please also clarify how the 200 realizations and eight candidate rules are constructed, and explain the exact distinction between the "Mean-CVaR" and "CUBO" criteria in the common-class table. Evidence on posterior lower-welfare-quantile calibration and sensitivity to dropout settings would address the remaining uncertainty concern. I appreciate the clarification of the theoretical assumptions, but currently retain my rating because the aggregate-welfare downside-protection claim remains insufficiently evaluated.

AuthorsOpenReview ↗

Thanks for your reply!

Thank you for the follow-up and for recognizing the clarified contribution of this work. We are on the same page that individual-regret tails do not directly evaluate downside protection in aggregate welfare, and we really appreciate this opportunity to address your concerns below.

R1. Aggregate welfare across repeated training datasets.

For the policy selected in realization rr, we evaluate held-out welfare relative to never treating and aggregate regret:

R_r=\frac{1}{m_r}\sum_i\max(\tau_{ri},0)-W_r.$$ BDeRCFR results summarize **200 per-realization aggregate values**, using the same candidate rules and validation DR scores. Lower-5% welfare averages the ten smallest $W_r$, and upper-5% aggregate regret averages the ten largest $R_r$. Validation also selected nuisance checkpoints, so this is a matched diagnostic. |Dataset|Criterion|Mean welfare ↑|Lower-5% welfare ↑|Upper-5% aggregate regret ↓| |---|---|---:|---:|---:| |Noisy IHDP|Mean-only|5.411|3.484|0.543| |Noisy IHDP|Mean–CVaR|5.424|3.486|0.476| |ACIC 2017 high-noise|Mean-only|0.526|0.020|0.161| |ACIC 2017 high-noise|Mean–CVaR|0.541|0.061|0.119| Mean–CVaR improves all three aggregate summaries. We also report upper quantiles of **aggregate regret across realizations**, using linear interpolation of $\\{R_r\\}_{r=1}^{200}$. Each $R_r$ averages test-population regret within a realization; these are not individual-regret quantiles. |Dataset|Criterion|$Q_{.80}(R)$ ↓|$Q_{.90}(R)$ ↓|$Q_{.95}(R)$ ↓| |---|---|---:|---:|---:| |Noisy IHDP|Mean-only|0.226|0.343|0.437| |Noisy IHDP|Mean–CVaR|0.216|0.279|0.363| |ACIC 2017 high-noise|Mean-only|0.087|0.120|0.142| |ACIC 2017 high-noise|Mean–CVaR|0.053|0.075|0.089| Mean–CVaR lowers every displayed upper-regret quantile on both datasets. The 90th/95th percentiles decrease by **18.6% / 16.8% on IHDP** and **37.1% / 37.5% on ACIC**. Aggregate regret measures the shortfall from each realization's oracle welfare, accounting for differences in attainable welfare across realizations. These matched results show less severe aggregate policy errors in the upper tail across training datasets. **R2. Construction of the realizations and candidate rules.** Noisy IHDP uses source realizations 1–200 with $Y^{\mathrm{new}}(a)=\mu_a+5\{Y^{\mathrm{original}}(a)-\mu_a\}$, preserving true treatment effects. ACIC uses 200 distinct official high-noise realizations with $\eta=1.25$, 12–13 from each of 16 conditions, with 50 per error type. Each method uses one training seed per realization, while the 100 posterior draws within a fit are uncertainty samples. Writing $\bar\tau_r(x)=S^{-1}\sum_s\tau_{\theta_{rs}}(x)$, the candidate class is $$h_{r0}(x)=\mathbf{1}\\{\bar\tau_r(x)>0\\},\qquad h_{rq}(x)=\mathbf{1}\\{Q_q(\\{\tau_{\theta_{rs}}(x)\\}_{s=1}^{S})>0\\},$$ with $\mathcal Q=\\{.05,.10,.25,.50,.75,.90,.95\\}$ and $\Pi_r=\\{h_{r0},h_{rq}:q\in\mathcal Q\\}.$ Rules use the fitted posterior without oracle effects. The common-class comparison selects one of these eight rules. Because benchmark realizations vary in their data-generating conditions, their welfare distribution describes benchmark-wide performance, and paired within-realization comparisons isolate criterion differences. **R3. Mean-only and the updated mean–CVaR objective.** Mean–CVaR denotes the updated CUBO objective. With $M_{\mathrm{post}}(\pi)$ denoting posterior-mean causal-score welfare, the two targets are $$F_{\mathrm{mean}}(\pi)=M_{\mathrm{post}}(\pi),\quad J_{\alpha,\lambda}(\pi)=(1-\lambda)\widehat E_s[W_s(\pi)] +\lambda\,\widehat{\mathrm{LCVaR}}_{\alpha}\{W_s(\pi)\},$$ $$F_{\mathrm{mean\text{-}CVaR}}(\pi)=M_{\mathrm{post}}(\pi)-\beta J_{\alpha,\lambda}(\pi).$$ Both criteria use the same eight rules, nuisance information and draws. The hyperparameters are chosen from validation set. **R4. Dropout sensitivity.** R1 reports the empirical assessment of welfare tails and aggregate regret. We evaluate ACIC BDragonnet at four dropout settings over the same 200 realizations. Within each setting, mean-only and mean–CVaR share the fitted model, posterior draws, candidate rules, and fixed objective parameters. |Dropout rates|Mean-only aggregate regret ↓|Mean–CVaR aggregate regret ↓|Reduction| |---|---:|---:|---:| |$(.10,.50)$|0.057|0.041|28.3%| |$(.10,.10)$|0.072|0.050|30.5%| |$(.05,.05)$|0.085|0.060|29.3%| |$(.02,.02)$|0.094|0.075|20.6%| Entries are mean aggregate test regret; reductions use unrounded means. Mean–CVaR reduces regret by **20.6–30.5%** at every tested setting. Mean welfare also increases by **0.016–0.025**, and lower-5% welfare increases by **0.012–0.054**. CUBO's advantage persists across tested dropout settings. Thank you again for your continued engagement and thoughtful follow-up! We hope these additional results and clarifications fully address your concerns. If any questions or concerns remain, we would be happy to discuss them further.

Thank you for the additional results. These address important parts of my previous concerns and make the empirical evidence stronger. My main remaining question concerns the objective used to select policies. Your response describes subtracting a positive combination of average and lower-tail welfare from posterior-mean welfare. If this objective is maximized with positive weights, it appears to favor poorer lower-tail welfare when the mean-welfare terms are held fixed. Please clarify whether this subtraction is intended, whether the implementation maximizes or minimizes the objective, and whether any of these quantities represents a loss. Please also explain how the implemented objective corresponds to the theoretical guarantee and why the earlier table presented "Mean-CVaR" and "CUBO" as separate methods. This may be a notation issue, but it matters for interpreting the results. I retain my current rating pending this clarification and would reconsider it once the objective is clear. Please also distinguish the observed improvements in poor-outcome cases from evidence that the posterior accurately estimates the likelihood of those outcomes.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.