acceptodds
Under review as a conference paper at ICLR 2027

Proximal Causal Learning of Optimal Individualized Dose Rule

Abstract

Learning optimal individualized treatment regimes from observational data is of great importance in precision medicine. However, when treatment is continuous and unmeasured confounding is present, identifying such regimes remains a significant challenge. In this paper, we leverage the proximal learning framework to nonparametrically identify and estimate the optimal individualized dose rule in the presence of unmeasured confounders. Corresponding theoretical guarantees for the proposed estimators are also established, including the excess risk bound and optimal convergence rate. Simulation studies and a synthetic experiment confirm the practical utility of our framework.

Primary area:
causal reasoning
Keywords:
proximal causal inference, policy learning, individualized dose rules

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_1M3COpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposed a way of leveraging the proximal causal learning to solve the problem of unobserved confounders. Providing two identification results with two different proxies.

Critical strengths
  1. The paper is well-written and there is paras before different section serves as the connections which help the readers understand.
  2. The experiments compared with a lot of the baselines.
Critical weaknesses
  1. In the experiment section, there is only simulation studies and synthetic experiments. Which is not enough, should at least has semi-synthetic experiments or real-world experiments to show that the results of the methods.
  2. Missing citations about the decision making for example: for the potential outcome framework, did you follow the Rubin 1974's framework or any other streams. Also for example, in the assumption could cite some well-known papers instead just saying that "general assumptions in the causal inference literature".
  3. Before identifying your target estimand, the population where the data is sampled from is missing.
  4. Minor: There is a space missing in Line 322, "our proposed estimators.We".
Questions for authors
  1. To leverage the proxy causal framewok, one of the preliminary is that you could dividing L into three parts, how is step is conducted? Does the author pre-decide the three subsets X, W, Z, or the algorithm learn it by it self. And how different division will affect the estimation results.
  2. Could the author shows the confidence intervals of the proposed methods, do the CIs could cover the oracles with valid coverage percent?
  3. I would like to ask about sensitivity analysis of the bandwidth of the kernel estimation. Since if the bandwidth is too small, there will be hard to get enough sample to estimate with large variance, and if bandwidth is too large then the bias will be a problem.
3 comments

Thank you for the encouraging feedback on our writing and experimental comparisons, as well as the helpful suggestions. We address each point below.

W1: Only simulation studies and synthetic experiments; should at least have semi-synthetic or real-world experiments.

We thank the reviewer for this valuable suggestion. In the revised manuscript, we have added a semi-synthetic experiment based on real-world ICU records from MIMIC-IV v3.1 in Section 7 (with full DGP details in Appendix G and results in Figure 3), evaluating initial norepinephrine dosing for critically ill adults. All six proposed proximal rules outperform all competing baselines, achieving higher policy values while avoiding the upward confounding bias of Naive methods and the extreme boundary dosing of PMMR, VCNet-TR, CPO-DR, and CPO-IPW.

W2: Missing citations about decision making (e.g., Rubin 1974 for potential outcomes; well-known papers for general assumptions).

We thank the reviewer for catching these missing references. We have added the missing citations in Section 2 and Section 3.1, explicitly citing Rubin (1974) for the potential-outcomes framework and Hernán & Robins (2006); Kennedy et al. (2017) for consistency, positivity, and unconfoundedness.

W3: Before identifying your target estimand, the population where the data is sampled from is missing.

We thank the reviewer for pointing out this omission. In Section 2, we now explicitly state: "The observed data {(Li,Ai,Yi)}i=1n\{(L_i, A_i, Y_i)\}_{i=1}^n are independent and identically distributed draws from the observed-data distribution of this population. Our objective is to learn a dose rule for new units drawn from the same population." (See also Section 3.)

W4: Minor: There is a space missing in Line 322, "our proposed estimators.We".

We thank the reviewer for spotting this typo. We have fixed this spacing typo in Section 5.

Q1: How is dividing LL into (X,W,Z)(X, W, Z) conducted? Pre-decided or learned, and how does different division affect estimation?

We thank the reviewer for this important question. Following the standard proximal causal inference literature (Miao et al., 2018; Tchetgen Tchetgen et al., 2024; Cui et al., 2024), the partitioning of LL into (X,W,Z)(X, W, Z) is pre-specified using domain knowledge and causal/temporal ordering rather than learned purely data-driven, because the conditional independence conditions in Assumption 1 W⊥!!!⊥(A,Z)∣U,XW \perp\\!\\!\\!\perp (A, Z) \mid U, X and Y⊥!!!⊥Z∣A,U,XY \perp\\!\\!\\!\perp Z \mid A, U, X) are generally untestable from observational data alone. XX contains measured adjustment variables. For WW, we need a pre-treatment measurement informative about the latent confounder whose association with dose is accounted for by (U,X)(U,X). For ZZ, we need a measurement informative about treatment assignment with no direct effect on the outcome after accounting for (A,U,X)(A,U,X).

To illustrate how this is done in practice, consider our MIMIC-IV semi-synthetic study (Section 7) on initial norepinephrine dosing in ICU patients:

  • Observed confounders XX: Lactate, blood urea nitrogen, bicarbonate, creatinine, potassium, and the missing-immature-granulocyte indicator capture acute metabolic acidosis and renal/perfusion status, which directly guide vasopressor dosing (AA) and affect short-term blood pressure recovery (YY).
  • Outcome-inducing proxy WW (pre-infusion platelet count): Platelet count reflects underlying sepsis severity and coagulopathy (UU) and correlates with hemodynamic outcomes (YY). Because it is measured prior to infusion initiation, it cannot be affected by AA, and conditional on (U,X)(U, X) it does not directly drive the initial norepinephrine infusion rate—making W⊥!!!⊥(A,Z)∣(U,X)W \perp\\!\\!\\!\perp (A, Z) \mid (U, X) clinically plausible.
  • Treatment-inducing proxies Z1,Z2Z_1, Z_2 (white blood cell count and neutrophil percentage): Measured at blood-culture sampling before norepinephrine initiation, these inflammatory markers reflect acute infection severity (UU) and influence clinicians' assessment of septic shock and initial dosing (AA), while having no direct physiological effect on 1-hour mean arterial pressure (YY) once (U,X,A)(U, X, A) are accounted for—making Y⊥!!!⊥Z∣(A,U,X)Y \perp\\!\\!\\!\perp Z \mid (A, U, X) clinically plausible.

Regarding how different divisions affect estimation:

  1. Misclassifying variables such that Assumption 1 is violated can introduce bias (as discussed in Section 8).
  2. Within valid treatment-inducing proxies (Z1,Z2)(Z_1, Z_2), swapping which variable enters d2d_2 versus serving as the auxiliary proxy Z∗Z^* affects finite-sample performance: as shown in Section 6 (Figure 2), assigning the more informative proxy (Z2Z_2) to Z∗Z^* yields better performance than the reverse.

Q2: Could the author show the confidence intervals of the proposed methods, and do the CIs cover the oracles with valid coverage percent?

We thank the reviewer for raising this question on statistical inference. Our theoretical analysis in Section 5 establishes a finite-sample high-probability excess-risk (regret) bound for V(d2∗)−V(d^2)V(d_{2}^{\ast}) - V(\hat{d}_{2})(Theorem 4) rather than an asymptotic normality theorem for V(d^)V(\hat{d}) or V(d∗)−V(d^)V(d^*) - V(\hat{d}). Deriving asymptotic normality and valid confidence intervals for non-regular, nonsmooth continuous-dose O-learning estimators—which involve a vanishing kernel bandwidth ϕn→0\phi_n \to 0 and RKHS regularization—is a challenging theoretical problem beyond the scope of this work. Empirically, the boxplots over 100 replications (Figures 2, 3, 5, 7, 8) and the means ±\pm standard deviations in Table 5 reflect the finite-sample dispersion of our estimators relative to doracled_{\text{oracle}}.

Q3: Sensitivity analysis of the bandwidth ϕn\phi_n of the kernel estimation (bias-variance trade-off).

We thank the reviewer for this insightful suggestion. We have added a dedicated sensitivity analysis in Appendix E.4 (Figure 6, pp. 29–30), evaluating ϕn=cσ^An−1/6\phi_n = c\hat{\sigma}_A n^{-1/6} across c∈{0.5,1,1.5,2,2.5,3,3.5}c \in \{0.5, 1, 1.5, 2, 2.5, 3, 3.5\} for d1(X,W)d_1(X, W) under both Scenario 1 and Scenario 2 at n=2,000n = 2{,}000. Consistent with the reviewer's intuition, Figure 6 exhibits a clear bias-variance trade-off: small cc (e.g., c=0.5c = 0.5) yields high variance due to sparse local samples in the kernel window, large cc (e.g., c≥3.0c \ge 3.0) increases smoothing bias, and the policy value peaks at c=2c = 2.

References:

  • Cui, Y., Pu, H., Shi, X., Miao, W., & Tchetgen Tchetgen, E. (2024). Semiparametric proximal causal inference. Journal of the American Statistical Association, 119(546), 1348–1359.
  • Hernán, M. A., & Robins, J. M. (2006). Estimating causal effects from epidemiological data. Journal of Epidemiology and Community Health, 60(7), 578–586.
  • Kennedy, E. H., Ma, Z., McHugh, M. D., & Small, D. S. (2017). Non-parametric methods for doubly robust estimation of continuous treatment effects. Journal of the Royal Statistical Society Series B: Statistical Methodology, 79(4), 1229–1245.
  • Miao, W., Geng, Z., & Tchetgen Tchetgen, E. J. (2018). Identifying causal effects with proxy variables of an unmeasured confounder. Biometrika, 105(4), 987–993.
  • Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66(5), 688.
  • Tchetgen Tchetgen, E. J., Ying, A., Cui, Y., Shi, X., & Miao, W. (2024). An introduction to proximal causal inference. Statistical Science, 39(3), 375–390.
Reviewer_kZYjOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies learning optimal individualized dose rules for continuous treatments in the presence of unmeasured confounding. The authors combine proximal causal learning (Wu et al., 2023) with outcome-weighted learning (Chen et al., 2016) for continuous treatments. They first identify the value of policies of the form d1(X,W),d_1(X,W),, where WW is an outcome-inducing proxy, using a treatment confounding bridge. They then consider another policy class d2(X,W,Z),d_2(X,W,Z), which additionally incorporates a treatment-inducing proxy ZZ, while requiring the availability of an auxiliary proxy Z∗Z^* for identification following multiple proxies assumptions/scenarios by Cui etal.,2024;Yuetal.,2025.

For estimation, the paper uses kernel smoothing to handle continuous treatments, estimates the treatment bridge through a minimax procedure, and formulates direct policy learning via outcome-weighted learning. A DC algorithm (similar to Chen et al., 2016) is developed for linear and RKHS dose rules. The paper also derives approximation and excess-value/risk bounds for the learned IDR. Empirical validation consists of simulation studies under linear and nonlinear optimal rules and a synthetic airline-demand experiment. The proposed proximal policies generally outperform competing methods in these experiments.

Critical strengths

1. Important and relatively underexplored problem. The paper addresses individualized continuous-dose decision making under unmeasured confounding, an important problem in precision medicine. Existing proximal continuous-treatment methods primarily focus on estimating causal dose-response functions like ADRF, whereas this paper directly targets optimal individualized dose rules.

2. Interesting extension to policies incorporating treatment-inducing proxies. The formulation of d2(X,W,Z),d_2(X,W,Z), which allows part of the treatment-inducing proxy to enter the decision rule while retaining an auxiliary proxy Z∗Z^* for identification, is conceptually interesting. It provides a principled way to use proxy information about latent confounders for treatment personalization. The proposed direct optimization algorithm and accompanying theoretical analysis further contribute to the practical development of this framework.

Critical weaknesses

1. A potential gap in the main theoretical result.

Theorem 4 assumes that the optimal rule satisfies d2∗∈B1,∞α(Rd).d_2^* \in B^\alpha_{1,\infty}(\mathbb R^d). However, in Appendix B.3, the proof additionally assumes dϕn∗∈B1,∞α(Rd)d_{\phi_n}^* \in B^\alpha_{1,\infty}(\mathbb R^d) to establish the RKHS approximation error bound. It is unclear whether this additional smoothness condition follows from the assumptions stated in Theorem 4. Moreover, the Besov-space assumption appears incompatible with the unbounded linear oracle rule considered in Simulation Scenario 1.

2. The methodological novelty and numerical study could be strengthened.

Several key methodological ingredients, including kernel smoothing for continuous treatments and minimax estimation of treatment confounding bridges, have been developed in the existing proximal causal learning literature, particularly Wu et al. (2023) and the algorithm mimic Chen et. al ,2016. The main novelty of the current paper lies in individualized policy identification and direct policy learning, but this distinction could be more clearly presented. Moreover, the empirical evaluation is limited to simulations and a synthetic demand experiment. Stronger comparisons or better examples to compare with existing proximal continuous-treatment approaches would make the argument that there are practical advantages of the proposed method more convincing. Current synthetic analysis adds little value to the paper, as it is not more complex/realistic compared to simulation analysis.

Questions for authors

1. Theoretical assumptions in Theorem 4 and their applicability to linear decision rules.

Theorem 4 assumes that the optimal decision rule satisfies d2∗∈B1,∞α(Rd).d_2^* \in B^\alpha_{1,\infty}(\mathbb R^d).

However, under the definition provided in the paper, this Besov space requires the function to be essentially bounded. A nonconstant linear rule d(x)=β⊤x+bd(x)=\beta^\top x+b on Rd\mathbb R^d is unbounded and therefore does not belong to this space. In particular, the linear oracle rule in Simulation Scenario 1 depends on covariates with unbounded support. Does this mean that the theoretical guarantees in Theorem 4 do not apply to the linear rules considered in the simulations? If the intended assumption concerns a bounded covariate domain, should the theorem and proof be reformulated accordingly?

Furthermore, the proof in Appendix B.3 additionally assumes that the surrogate-risk minimizer satisfies dϕn∗∈B1,∞α(Rd),d_{\phi_n}^* \in B^\alpha_{1,\infty}(\mathbb R^d), although this condition is not stated in Theorem 4. Does this additional assumption follow from the smoothness of d2∗d_2^*, or must it be imposed separately? If it is a separate assumption, could the authors explain its justification and implications for the stated convergence rate?

2. Relationship to existing continuous-treatment proximal learning.

Wu et al. (2023) have developed proximal causal learning methods for continuous treatments, including kernel smoothing and minimax estimation of treatment confounding bridges. Could the authors clarify the precise methodological and theoretical advances of the present work beyond combining these techniques with the continuous-dose outcome-weighted learning framework of Chen et al. (2016)? In particular, it would be helpful to distinguish the new policy-identification results from existing proximal estimation procedure.

3. Sample splitting and theoretical guarantees.

The proof of Theorem 4 assumes that the treatment bridge estimator is obtained using sample splitting independently of policy learning. Is this sample-splitting procedure implemented in the numerical experiments? If not, how do the theoretical guarantees apply to the estimator used in practice?

4. Some presentation issues Most of the paper is well organized, and the connection between proximal identification, continuous-dose outcome-weighted learning, and the proposed optimization algorithm is generally well explained. However, the synthetic analysis is somewhat confusing. For example, in the main text, the authors describe Z1Z_1 and Z2Z_2 as fuel-cost variables without explaining why fuel cost needs to be represented by two variables. After reading Appendix E, I found that these variables are generated by sine and cosine transformations of the latent demand variable UU, without explaining their economic interpretation. What do these two fuel-cost variables represent in practice? Since the proposed d2d_2 approach requires an additional treatment-inducing proxy, it would also be helpful to discuss whether this experiment is representative of practical settings where only one fuel-cost variable is available.

3 comments

Thank you for the thoughtful and detailed review, and especially for identifying the technical gap in Theorem 4 and the sample-splitting alignment. We have addressed the concerns in the revised manuscript as detailed below.

W1 & Q1: Potential gap in Theorem 4: the proof in Appendix B.3 additionally assumes dϕn∗∈B1,∞α(Rd)d_{\phi_n}^* \in B_{1,\infty}^\alpha(\mathbb{R}^d), and the Besov-space assumption appears incompatible with the unbounded linear oracle rule in Simulation Scenario 1.

We thank the reviewer for catching both points, which we have resolved in the revision:

  1. Eliminating the extra assumption on dϕn∗d_{\phi_n}^*: In the revised proof of Theorem 4 (Appendix C.5, pp. 23–25), we no longer introduce the surrogate-risk minimizer dϕn∗d_{\phi_n}^* or assume dϕn∗∈B1,∞α(Rd)d_{\phi_n}^* \in B_{1,\infty}^\alpha(\mathbb{R}^d). Instead, we bound Vϕn(d2∗,qˉ1)−Vϕn(d^2,qˉ1)=Rϕn(d^2,qˉ1)−Rϕn(d2∗,qˉ1)V_{\phi_{n}}(d_{2}^{\ast}, \bar{q}_{1}) - V_{\phi_{n}}(\hat{d}_{2}, \bar{q}_{1}) = \mathscr{R}_{\phi_{n}}(\hat{d}_{2}, \bar{q}_{1}) - \mathscr{R}_{\phi_{n}}(d_{2}^{\ast}, \bar{q}_{1}) directly with qˉ1=1K∑k=1Kq^1(−k)\bar{q}_{1} = \frac{1}{K}\sum_{k=1}^{K} \hat{q}_{1}^{(-k)} and construct the Gaussian-RKHS approximation d2,γ=Kγ(r)∗d2∗d_{2,\gamma} = K_\gamma^{(r)} * d_2^* directly from d2∗d_2^* (Step I, p. 23). Thus, no separate smoothness assumption on dϕn∗d_{\phi_n}^* is needed.
  2. Compatibility of B1,∞α(Rd)∩L∞(Rd)B_{1,\infty}^\alpha(\mathbb{R}^d) \cap L_\infty(\mathbb{R}^d) with linear rules and bounded covariates: We revised Assumption 8 (p. 7) to explicitly state d2∗∈B1,∞α(Rd)∩L∞(Rd)d_2^* \in B_{1,\infty}^\alpha(\mathbb{R}^d) \cap L_\infty(\mathbb{R}^d). In Scenario 1 (Appendix E.1, p. 28), we updated the data-generating mechanism so that all covariates and proxies (X,W,Z)(X, W, Z) are drawn from uniform distributions and hence have bounded support (replacing the previously unbounded normal noise in Z1Z_1 with Uniform[−3,3]\mathrm{Uniform}[-\sqrt{3}, \sqrt{3}]). On a bounded covariate domain, the linear oracle rule d∗(X,W)d^*(X, W) is bounded and admits a smooth, compactly supported extension to Rd\mathbb{R}^d belonging to B1,∞α(Rd)∩L∞(Rd)B_{1,\infty}^\alpha(\mathbb{R}^d) \cap L_\infty(\mathbb{R}^d). Therefore, Theorem 4 applies directly to Scenario 1.

W2 & Q2: Methodological/theoretical advances beyond Wu et al. (2023) and Chen et al. (2016), and strengthening the empirical evaluation.

We thank the reviewer for raising this important question, which gives us the opportunity to clarify why our work is far from a simple combination of Wu et al. (2023) and Chen et al. (2016), as well as how we have strengthened our empirical evaluation.

  1. Advances beyond Wu et al. (2023) (ADRF Estimation vs. Policy Identification): While we adopt CNFs and minimax bridge estimation following Wu et al. (2023) for nuisance estimation, Wu et al. (2023) estimate the average dose-response function (ADRF) E[Y(a)]\mathbb{E}[Y(a)] at a fixed constant dose aa, whereas we identify and learn optimal individualized dose rules (IDRs) d(⋅)d(\cdot) that map random proxies to doses. This is not a trivial substitution because the proxy inputs of d(⋅)d(\cdot) enter the conditioning set of the kernel indicator: Theorem 1 leverages joint latent unconfoundedness (Assumption 1(3)) so that q0(Z,A,X)q_0(Z, A, X) can identify d1(X,W)d_1(X, W) without conditioning on WW, while Theorem 2 introduces a new proxy-splitting scheme (Z,Z∗)(Z, Z^*) and auxiliary bridge q1(Z∗,A,X)q_1(Z^*, A, X) (Assumptions 5–6) to identify the expanded class d2(X,W,Z)d_2(X, W, Z) where q0q_0 would otherwise fail.

  2. Algorithmic and theoretical advances beyond Chen et al. (2016): Extending Chen et al. (2016) from randomized trials—where there is no unmeasured confounding and p(A∣X)p(A \mid X) is known by design—to observational studies with unknown treatment assignment and unmeasured confounders (via proximal causal learning with cross-fitted bridge estimators q^(−k)\hat{q}^{(-k)}) introduces three fundamental challenges that Chen et al.'s framework cannot handle:

    • Theoretical guarantees under unmeasured confounding with bandwidth-coupled nuisance error (Section 5 & Appendix C.5): Chen et al. (2016, Theorem 2) do not account for unmeasured confounding and assume the propensity density p(A∣X)p(A \mid X) is known by design. In contrast, our Theorem 4 addresses observational settings with unmeasured confounders and accounts for the cross-fitted bridge estimation error Op(n−φ)O_p(n^{-\varphi}), shifting the optimal bandwidth trade-off and convergence rate from n−1/4n^{-1/4} to n−1/6n^{-1/6}.
    • Optimization challenges induced by proximal causal learning: signed weights and a redesigned DC / dual QP algorithm (Section 4.2 & Appendix D): Chen et al. (2016) assume randomized trials with strictly positive weights, under which samples outside the ϕn\phi_n-tube cancel out in the DC subproblem. In proximal causal learning, however, the estimated bridge q^(−k)\hat{q}^{(-k)}—and thus the sample weight wi=Yiq^(−k)w_i = Y_i\hat{q}^{(-k)}—can be negative, which flips the convexity/concavity of the surrogate loss and invalidates their DC decomposition. We partition samples into positive- and negative-weight sets (P,N\mathcal{P}, \mathcal{N}) to derive a new convex majorization and unified dual QP (Equations 6–7).
    • Refined proof without extra assumptions on dϕn∗d_{\phi_n}^*: Unlike Chen et al. (2016), who invoke Theorem 7.23 of Steinwart & Christmann (2008) under positive weights and additionally assume fϕ∗∈B1,∞α(Rd)f_\phi^* \in B_{1,\infty}^\alpha(\mathbb{R}^d), we handle signed fold weights via a shifted nonnegative loss (Proposition 2) and approximate d2∗d_2^* directly, eliminating the extra smoothness assumption on dϕn∗d_{\phi_n}^*.
  3. Strengthened empirical evaluation: To further demonstrate practical utility beyond synthetic simulations, we added a MIMIC-IV v3.1 semi-synthetic experiment (Section 7, Figure 3, Appendix G) modeling realistic clinical confounding across 30 ICU variables, as well as Scenarios 3 and 4 (Appendix E.5, Figures 7–8) where d∗d^* depends on UU, and moved the Demand study to Appendix F.

Q3: The proof of Theorem 4 assumes that the treatment bridge estimator is obtained using sample splitting independently of policy learning. Is this implemented in the numerical experiments?

We thank the reviewer for pointing out this discrepancy. In the revised manuscript, we now implement KK-fold cross-fitting (K=5K = 5) in all numerical experiments to estimate q^0(−k)\hat{q}_0^{(-k)} and q^1(−k)\hat{q}_1^{(-k)} out-of-fold (Sections 3.2 and 3.3, pp. 4–5). Correspondingly, we updated Theorem 4 and its proof (Appendix C.5, Proposition 2 and Proof of Theorem 4, pp. 21–25) to explicitly analyze the KK-fold cross-fitted bridge estimator qˉ1=K−1∑k=1Kq^1(−k)\bar{q}_{1} = K^{-1}\sum_{k=1}^{K} \hat{q}_{1}^{(-k)} (conditioning on F−k\mathcal{F}_{-k} per fold and applying a union bound over the KK folds). The theoretical guarantees and practical implementation are now fully consistent.


Q4: Presentation issues in the Demand synthetic study (economic interpretation of Z1,Z2Z_1, Z_2 and single-proxy settings).

We thank the reviewer for pointing out these presentation issues. The Demand synthetic data-generating process is adopted directly from Hartford et al. (2017) and Xu et al. (2021). In practice, airline fuel costs are driven by seasonal travel demand UU through multiple cost components that are not strictly synchronized over the demand cycle. Specifically, Z1Z_1 and Z2Z_2 can be interpreted as two complementary, phase-shifted fuel-cost indicators. For example, contemporaneous spot jet fuel prices (Z1Z_1) and lagged hedged fuel contract costs or refining spreads (Z2Z_2). Because a single cyclical cost indicator can take the same value during both the rising and falling phases of a demand cycle, pairing two out-of-phase cost proxies (modeled mathematically via sine and cosine transformations of UU) captures both the level and phase of latent demand UU while affecting ticket pricing AA without directly influencing passenger sales YY conditional on (A,U)(A, U).

We moved this experiment to Appendix F so the main text centers on the MIMIC-IV clinical semi-synthetic study (Section 7). Regarding settings where only a single treatment-inducing proxy is available (so no auxiliary Z∗Z^* exists), our first framework d1(X,W)d_1(X, W) (Section 3.2, Theorem 1) remains directly applicable without requiring Z∗Z^*, whereas d2(X,W,Z)d_2(X, W, Z) is designed for settings with multiple treatment-inducing proxies (such as WBC count Z1Z_1 and neutrophil percentage Z2Z_2 in MIMIC-IV).

References:

  • Chen, G., Zeng, D., & Kosorok, M. R. (2016). Personalized dose finding using outcome weighted learning. Journal of the American Statistical Association, 111(516), 1509–1521.
  • Hartford, J., Lewis, G., Leyton-Brown, K., & Taddy, M. (2017). Deep IV: A flexible approach for counterfactual prediction. ICML, 1414–1423.
  • Steinwart, I., & Christmann, A. (2008). Support Vector Machines. Springer Science & Business Media.
  • Wu, Y., Fu, Y., Wang, S., & Sun, X. (2023). Doubly robust proximal causal learning for continuous treatments. arXiv preprint arXiv:2309.12819.
  • Xu, L., Kanagawa, H., & Gretton, A. (2021). Deep proxy causal learning and its application to confounded bandit policy evaluation. NeurIPS, 34, 26264–26275.
Reviewer_wmPQOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies individualized treatment regime (ITR) learning where the treatment is continuous and unmeasured confounding is present. Integrating the proximal causal inference framework and kernel-smoothed value estimator by Chen et al (2016), the paper establishes two value identification results under various sets of conditions, develops their corresponding restricted in-class optimal ITRs, and proves their theoretical properties. The practical performance of the proposed optimal ITRs is compared with existing methods by a simulation study and a synthetic study.

Critical strengths

ITR learning for continuous treatments is not well studied, especially when unmeasured confounding is present. The paper potentially enriches the toolbox of ITR learning.

Critical weaknesses
  1. The novelty and practical benefit of the proposed methods beyond existing works is unclear. First, the ITR problem considered by this paper seems a special case of Li et al. (2026), which is not cited or compared. It is unclear why the method by Li et al. (2026) cannot be used for ITR learning or worse than the proposed methods. Second, the paper does not include a real data application, so the practical utility of the proposed methods cannot be assessed.

  2. The proposed methods are not clearly justified. First, some existing works have developed ITR learning methods under the proximal causal inference framework based on the outcome confounding bridge h(A,W,X)h(A, W, X) or a doubly robust ITR learner based on both h(A,WX)h(A, W X) and q(Z,A,X)q(Z, A, X). It is not explained in the paper why these approaches are not viable or why/how much they are worse than the proposed methods. Second, the necessity of developing d^2\hat{d}_2 in the presence of d^1\hat{d}_1 seems questionable. The DAG in Figure 3 becomes the DAG in Figure 1 if Z∗Z^{\ast} is absorbed into XX. The numerical advantage of d^2\hat{d}_2 over d^2\hat{d}_2 is not demonstrated. Even if d^2\hat{d}_2 is useful, the paper does not provide any numerical evidence or practical guidance about how to choose Z∗Z^{\ast} properly in practice.

Reference:

Li, Y., Han, E., Hu, Y., Zhou, W., Qi, Z., Cui, Y., & Zhu, R. (2026). Reinforcement learning with continuous actions under unmeasured confounding. Journal of the American Statistical Association, 121(553), 209-222.

Questions for authors

Q1. The meanings of some notations are not given, such as p(A∣L)p(A | L), p(A∣U,X)p(A | U, X), ∂A\partial \mathcal{A}, choice of distdist, eie_i in Lemmas 1 and 2.

Q2. Comments on the assumptions are missing, such as sufficient conditions on the existence of q0q_0 in Assumption 3 and of q1q_1 in Assumption 5.

Q3. Colors of different methods in Figure 2 are too similar. The legends in Figure 2 are inconsistent with the descriptions in Section 6.

Q4. In Section 7, how well do PMMR and VCNet-TR perform if allowing nonlinear policies?

3 comments

We sincerely thank you for the careful reading and constructive feedback. We have revised the manuscript accordingly and address all weaknesses and questions point-by-point below.

W1: The novelty and practical benefit of the proposed methods beyond existing works is unclear. First, the ITR problem considered by this paper seems a special case of Li et al. (2026), which is not cited or compared. Second, the paper does not include a real data application.

We sincerely thank the reviewer for bringing Li et al. (2026) to our attention. While single-stage decision-making can broadly be viewed as a special case of reinforcement learning, we respectfully disagree that our problem is a special case of Li et al. (2026) for the following three reasons:

  1. Different policy classes and underlying assumptions: Li et al. (2026) consider stochastic policies characterized by conditional action densities π(a∣o)\pi(a \mid o), subject to regularity and coverage conditions (Assumptions 5 and 10). In contrast, our work focuses on deterministic individualized dose rules d(H)d(H), which correspond to Dirac measures. Such policies do not admit Lebesgue densities and generally violate the absolute continuity condition in Assumption 10 of Li et al. (2026). Consequently, our policy class falls outside the scope of their framework, and their learning method and theoretical guarantees are not directly applicable.
  2. Different policy learning approaches: Li et al. (2026) optimize parameterized stochastic policies using policy gradient methods (with Gaussian and Beta policy distributions). In contrast, our approach combines a kernel-smoothed surrogate loss with difference-of-convex (DC) optimization, allowing us to directly learn flexible deterministic decision functions—including nonlinear functions in reproducing kernel Hilbert spaces (RKHSs)—without specifying a parametric family of stochastic policy distributions.
  3. Different theoretical contributions: The regret analysis in Li et al. (2026) relies on uniform control of policy value estimation errors over their stochastic policy class, accounting for policy optimization and bridge estimation errors, but not the kernel-smoothing bias arising in our deterministic setting. In contrast, we establish finite-sample regret bounds for deterministic dose rules through kernel smoothing (Theorems 3 and 4), explicitly balancing smoothing bias, confounding bridge estimation error, and statistical learning error via bandwidth selection.

We have explicitly cited and discussed these distinctions from Li et al. (2026) in the revised manuscript.

On real-data evaluation: In the revised manuscript, we have added a semi-synthetic experiment based on real-world ICU records from MIMIC-IV v3.1 (Section 7, Figure 3, and Appendix G). In observational real-world datasets with unmeasured confounding, counterfactual outcomes under alternative continuous doses are inherently unobserved, making objective policy comparison impossible without a structural benchmark. Our MIMIC-IV semi-synthetic study preserves realistic clinical confounding and covariate structures across 30 baseline ICU variables while providing a known counterfactual benchmark for initial norepinephrine dosing. As shown in Figure 3(b), all six proposed proximal rules achieve higher mean policy values and substantially smaller variability than all competing baselines, while avoiding the upward confounding bias of Naive O-learning and the extreme boundary-collapsing behavior of PMMR, VCNet-TR, CPO-DR, and CPO-IPW (Figure 3(a)).

W2: The proposed methods are not clearly justified (comparison with h(A,W,X)h(A,W,X)-based or doubly robust proximal ITR learners; necessity of d^2\hat{d}_2 over d^1\hat{d}_1; guidance on choosing Z∗Z^{*}).

We thank the reviewer for these thoughtful comments, which allow us to better clarify the positioning and motivation of our methods.

  1. Comparison with existing proximal ITR learners: Existing proximal ITR methods such as Qi et al. (2024) and Shen & Cui (2023) primarily focus on binary treatments and formulate policy learning as a weighted classification problem, which cannot be directly applied to continuous treatments. To provide a direct comparison, we include a regression-based proximal baseline (PMMR; Mastouri et al., 2021) in Section 6 and Section 7, which estimates the outcome confounding bridge h(A,W,X)h(A, W, X) via conditional moment restrictions given (A,Z,X)(A, Z, X) and selects the dose maximizing h^(a,W,X)\hat{h}(a, W, X). This serves as a continuous-treatment Q-learning counterpart to the outcome-bridge approach in Qi et al. (2024). Across our simulations (Figures 2, 5, 7, 8) and the MIMIC-IV semi-synthetic study (Figure 3), our direct outcome-weighted DC learning methods consistently outperform this regression-based baseline.

  2. Necessity of d^2\hat{d}_2 over d^1\hat{d}_1 and structural distinction: A fundamental property of policy optimization is that expanding the admissible policy class cannot decrease the optimal value: if D1⊆D2\mathcal{D}_{1} \subseteq \mathcal{D}_{2}, then sup⁡d∈D2V(d)≥sup⁡d∈D1V(d).\sup_{d \in \mathcal{D}_{2}} V(d) \ge \sup_{d \in \mathcal{D}_{1}} V(d). However, in proximal ITR learning, the decision space is constrained by the domains of confounding bridge functions and structural assumptions such as latent ignorability. Consequently, existing approaches, such as Qi et al. (2024), generally consider policies based on either (X,W)(X,W) or (X,Z)(X,Z), rather than directly optimizing over a general class of policies defined on (X,W,Z)(X,W,Z). Shen and Cui (2023) partially address this limitation through a two-stage structure: they use XX to determine whether d(X,W)d(X,W) or d(X,Z)d(X,Z) is preferable and then implement the selected policy. Consequently, their approach optimizes over a restricted class of combined policies, rather than directly optimizing over a general class of decision rules on (X,W,Z)(X,W,Z). Moreover, the additional selection step may introduce prediction error and instability. The key distinction is therefore not merely that our policy class is larger, but that our framework allows a richer decision space while retaining the proxy information required for proximal identification. Specifically, by leveraging multiple treatment-inducing proxies and incorporating a subset of them into the decision space, our method provides a trade-off between identification stability and the richness of the policy class. This flexibility allows researchers to select the decision space according to the available proxy variables and their identification requirements.

  3. Numerical advantage of d^2\hat{d}_2 over d^1\hat{d}_1 and guidance on choosing Z∗Z^*: In the revised manuscript, we added Scenarios 3 and 4 (Appendix E.5, Figures 7 and 8) where the oracle optimal rule d∗d^* directly depends on the latent confounder UU. Because d2d_2 leverages information about UU carried by ZZ, d2d_2 clearly outperforms d1d_1 in both scenarios (e.g., d2L(X,W,Z1)d_2^{L}(X, W, Z_1) and d2L(X,W,Z2)d_2^{L}(X, W, Z_2) in Figure 7, and d2G(X,W,Z1)d_2^G(X, W, Z_1) and d2G(X,W,Z2)d_2^G(X, W, Z_2) in Figure 8). Similarly, in the Demand synthetic study (Appendix F, Table 5), d2L(W,Z1)d_2^L(W, Z_1) and d2L(W,Z2)d_2^L(W, Z_2) outperform d1L(W)d_1^L(W). Moreover, as shown in Section 6 (Figure 2), d2(X,W,Z1)d_2(X, W, Z_1) (using Z2Z_2 as Z∗Z^*) outperforms d2(X,W,Z2)d_2(X, W, Z_2) (using Z1Z_1 as Z∗Z^*) because Z2Z_2 has lower noise variance and is a more informative proxy for UU. Thus, in practice, when multiple treatment-inducing proxies are available, one should allocate the stronger/more informative proxy to Z∗Z^* for stable bridge identification, while including the remaining proxy in the decision rule.

Q1: The meanings of some notations are not given, such as p(A∣L)p(A \mid L), p(A∣U,X)p(A \mid U, X), ∂A\partial \mathcal{A}, choice of dist\mathrm{dist}, eie_i in Lemmas 1 and 2.

We thank the reviewer for pointing out these missing notation definitions. We have added a comprehensive notation summary in Appendix A.1 (Table 1, pp. 15–16) defining all notations used in the main text and proofs, including conditional probability densities p(⋅∣⋅)p(\cdot \mid \cdot), the treatment space boundary ∂A\partial\mathcal{A}, the Euclidean set distance dist(a,S)=inf⁡s∈S∣a−s∣\mathrm{dist}(a, S) = \inf_{s \in S}|a - s|, and the jj-th dyadic entropy number ej(A,E)e_j(\mathcal{A}, E).

Q2: Comments on the assumptions are missing, such as sufficient conditions on the existence of q0q_0 in Assumption 3 and of q1q_1 in Assumption 5.

We thank the reviewer for this helpful suggestion. We have added detailed discussions following each assumption in Sections 3.1 and 3.3:

  • Assumption 2 (Section 3.1, p. 3): Completeness requires WW to be sufficiently informative about UU conditional on (A,X)(A, X), which holds for suitable exponential-family models (Newey & Powell, 2003); we also cite D'Haultfoeuille (2011) for further sufficient conditions.
  • Assumption 3 (Section 3.1, p. 3): Assumption 3 concerns the solvability of a linear integral equation of the first kind (Kress et al., 1989), and we refer readers to Appendix B of Cui et al. (2024) for sufficient conditions ensuring the existence of q0q_0 (including completeness between WW and ZZ and regularity conditions).
  • Assumptions 4–6 (Section 3.3, p. 5): Assumption 5 has an analogous integral-equation structure to Equation (1) with Z∗Z^* as the bridge argument and (W,Z)(W, Z) as the conditioning variable, so q1q_1 exists under conditions analogous to q0q_0 when an additional valid treatment-inducing proxy Z∗Z^* is available (Cui et al., 2024; Yu et al., 2025). We also discuss the trade-off introduced by Assumption 6 between expanding the decision-making variables and the required strength of the completeness condition.

Q3: Colors of different methods in Figure 2 are too similar. The legends in Figure 2 are inconsistent with the descriptions in Section 6.

We thank the reviewer for catching this presentation issue. We have updated the color palette and added distinct hatching patterns in Figure 2 (as well as Figures 3, 5, 7, and 8) to clearly separate linear rules, Gaussian-kernel rules, and baselines. We also updated the legend labels to doracled_{\text{oracle}}, d1L(X,W)d_1^L(X, W), d1G(X,W)d_1^G(X, W), d2L(X,W,Z1)d_2^L(X, W, Z_1), d2G(X,W,Z1)d_2^G(X, W, Z_1), d2L(X,W,Z2)d_2^L(X, W, Z_2), d2G(X,W,Z2)d_2^G(X, W, Z_2), Naive dL(X,W,Z1,Z2)\text{Naive } d^L(X, W, Z_1, Z_2), Naive dG(X,W,Z1,Z2)\text{Naive } d^G(X, W, Z_1, Z_2), PMMR, VCNet-TR, CPO-DR, and CPO-IPW, which are now fully consistent with Section 6.

Q4: In Section 7, how well do PMMR and VCNet-TR perform if allowing nonlinear policies?

We thank the reviewer for this insightful question. In the revised manuscript, the Demand synthetic study has been moved to Appendix F (while Section 7 now presents the MIMIC-IV semi-synthetic study, which already includes PMMR and VCNet-TR). To directly answer the reviewer's question on the Demand study, we evaluated our Gaussian-kernel proximal rules (d1G(W)d_1^G(W), d2G(W,Z1)d_2^G(W, Z_1), d2G(W,Z2)d_2^G(W, Z_2)) against PMMR and VCNet-TR across sample sizes n∈{1000,2000,4000}n \in \{1000, 2000, 4000\} over 100 replications:

Methodn=1000n = 1000n=2000n = 2000n=4000n = 4000
PMMR56.7834±1.753856.7834 \pm 1.753858.1241±1.477158.1241 \pm 1.477159.2093±1.553459.2093 \pm 1.5534
VCNet-TR42.3686±8.149642.3686 \pm 8.149640.7194±6.063040.7194 \pm 6.063040.3960±5.538640.3960 \pm 5.5386
d1G(W)d_1^G(W)63.9819±2.511763.9819 \pm 2.511763.9341±2.465663.9341 \pm 2.465664.7247±0.3730\mathbf{64.7247 \pm 0.3730}
d2G(W,Z1)d_2^G(W, Z_1)63.2503±3.162663.2503 \pm 3.162662.8245±3.849262.8245 \pm 3.849262.9588±2.911262.9588 \pm 2.9112
d2G(W,Z2)d_2^G(W, Z_2)64.1737±0.3550\mathbf{64.1737 \pm 0.3550}64.1417±0.2951\mathbf{64.1417 \pm 0.2951}64.2051±0.251664.2051 \pm 0.2516

The Gaussian-kernel proximal methods consistently outperform both PMMR and VCNet-TR by a large margin across all sample sizes.

Reference:

Li, Y., Han, E., Hu, Y., Zhou, W., Qi, Z., Cui, Y., & Zhu, R. (2026). Reinforcement learning with continuous actions under unmeasured confounding. Journal of the American Statistical Association, 121(553), 209–222.

Qi, Z., Miao, R., & Zhang, X. (2024). Proximal learning for individualized treatment regimes under unmeasured confounding. Journal of the American Statistical Association, 119(546), 915–928.

Shen, T., & Cui, Y. (2023). Optimal treatment regimes for proximal causal learning. Advances in Neural Information Processing Systems, 36, 47735–47748.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.