acceptodds
Under review as a conference paper at ICLR 2027

Orthogonal Learner for Estimating Heterogeneous Long-Term Treatment Effects

Abstract

Estimation of heterogeneous long-term treatment effects (HLTEs) is relevant for personalized decision-making in marketing, economics, and medicine, where short-term observational datasets are often combined with long-term observational datasets. However, HLTE estimation is challenging due to limited overlap in treatment assignments or in long-term outcomes for certain subpopulations, which can lead to unstable HLTE estimates with large finite-sample variance. To address this challenge, we introduce the LT-O-learners (Long-Term Orthogonal Learners), a set of novel orthogonal learners for HLTE estimation in the canonical HLTE setting with surrogacy. The key idea of our LT-O-learners is to retarget the loss via custom overlap weights that downweight low-overlap samples. We show that the retargeted loss recovers the true HLTE pointwise and satisfies Neyman-orthogonality. We further prove two key theoretical results: (i) The nuisance error enters the error bound only through higher-order terms, which means our learners are robust to nuisance misspecification. (ii) Under a linear function class, the retargeting effectively controls the asymptotic variance of the HLTE estimator via the overlap weights in low-overlap regimes. We conduct experiments on synthetic and real-world datasets to confirm the theoretical properties of our LT-O-learners, particularly robustness in low-overlap regimes. To our knowledge, ours are the first orthogonal learners for HLTE estimation robust to low overlap in long-term settings.

Primary area:
causal reasoning
Keywords:
Causal inference, causal machine learning, long-term treatment effect analysis, orthogonal statistical learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_ZTQwOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies heterogeneous long-term treatment effects estimation, with a particular focus on low treatment overlap and low long-term outcome overlap. It adapts existing meta-learners to the HLTE setting and proposes a class of orthogonal learners that improve estimations by downweighting samples with poor overlap.

Critical strengths

The paper addresses an important and practical problem of estimating HLTE when short-term and long-term outcomes are observed in separate datasets, and under low-overlap settings. The proposed method is intuitive and theoretically justified, and the empirical results show its effectiveness under low-overlap setups. The paper is also clearly written and easy to follow.

Critical weaknesses
  1. The meta-learners used for benchmarking and discussed in the related work are mostly methods proposed before 2023, while several improved variants have been published in the past two years. E.g:

[1] Liang, Zhongyuan, Lars van der Laan, and Ahmed Alaa. "Hybrid Meta-learners for Estimating Heterogeneous Treatment Effects."

[2] Lan, Hui, et al. "A meta-learner for heterogeneous effects in difference-in-differences."

[3] Frauen, Dennis, Konstantin Hess, and Stefan Feuerriegel. "Model-agnostic meta-learners for estimating heterogeneous treatment effects over time."

The related work section would benefit from discussing how these more recent variants could be adapted to the HLTE settings and possibly include them in the empirical performance comparisons.

  1. The main synthetic evaluation uses one generic data-generating process and focuses particularly on low-overlap settings. Prior literature has shown that different meta-learners can perform differently across different DGPs. It is unclear how the proposed method would perform in more general and more diverse DGPs.
Questions for authors

Please see my weakness above. It would be helpful if the authors could revise the paper to discuss how more recent meta-learners relate to the described HLTE setting and the proposed method.

4 comments

We thank the reviewer for the careful reading and the constructive comments. We address each concern below. We further updated the submission PDF in response to the comments (labeled as Action) and highlighted the changes in blue color.

W1 & Q: Discussion with recent meta-learners and the comparison

We thank the reviewer for bringing up these references. We discuss them one by one below:

  • Meta-learner for DiD [2]: The learner of [2] targets the CATT under conditional parallel trends, which requires pre- and post-treatment outcomes of the same units. So it does not apply to our setting, where we target the long-term CATE.

  • Meta-learners for CATE over time [3]: [3] studies the CATE under a sequence of treatments over time. Essentially, the learners that [3] proposed are the longitudinal extension (using the g-computation principle) of the T-, IPW-, and DR-learners with a single treatment. Its IVW-DR-learner applies inverse-variance weighting, but the learner is not orthogonal, unlike our proposed LT-O-learners. Importantly, we proposed learners that both apply overlap weighting and retain orthogonality.

  • H-learner [1]: The H-learner regularizes the target CAPO functions with a loss that is a weighted combination of the plug-in loss and the RA loss. Hence, it does not provide any new confounding adjustment strategy. However, its regularization technique can indeed be extended to our setting, and it would be interesting to compare the performance as an auxiliary baseline.

We implemented the long-term analog of the H-learner (LT-H-learner) and tested its performance under the setting of the synthetic experiment. Below, we report its PEHE (mean ± std) across 5 runs, together with the scores of the LT-T- and LT-RA-learners.

LearnerHealthyLow treatment overlapLow outcome overlapDual overlap
LT-T-learner0.21 ± 0.020.31 ± 0.020.19 ± 0.040.26 ± 0.04
LT-RA-learner0.19 ± 0.020.61 ± 0.030.18 ± 0.020.66 ± 0.07
LT-H-learner (λ = 0.25)0.20 ± 0.020.52 ± 0.030.19 ± 0.040.56 ± 0.07
LT-H-learner (λ = 0.5)0.21 ± 0.020.58 ± 0.050.20 ± 0.050.63 ± 0.04
LT-H-learner (λ = 0.75)0.21 ± 0.030.59 ± 0.050.18 ± 0.040.66 ± 0.07
LT-H-learner (best λ in hindsight)0.20 ± 0.02 (λ = 0.25)0.52 ± 0.03 (λ = 0.25)0.18 ± 0.04 (λ = 0.75)0.56 ± 0.07 (λ = 0.25)

We find that, across all regimes, the LT-H-learner does not beat the better of the LT-T- and LT-RA-learners. This is consistent with the construction of the LT-H-learner’s loss function, which is a λ\lambda-weighted combination of the plug-in loss (highly correlated with the T-learner) and the RA-learner’s loss. Therefore, the newly implemented LT-H-learner does not beat our LT-O-DO-learner in any regime (Table 2).

Action: We added a paragraph on [1–3] to the extended related work (Appendix A.1). We added the H-learner, adapted to our setting, as a baseline to the synthetic experiments (Appendix B.7, Table 9).

[1] Liang, Zhongyuan, Lars van der Laan, and Ahmed Alaa. "Hybrid Meta-learners for Estimating Heterogeneous Treatment Effects." AISTATS, 2026.

[2] Lan, Hui, et al. "A meta-learner for heterogeneous effects in difference-in-differences." ICML, 2025.

[3] Frauen, Dennis, Konstantin Hess, and Stefan Feuerriegel. "Model-agnostic meta-learners for estimating heterogeneous treatment effects over time." ICLR, 2025.

W2: Performance in more general and more diverse DGPs

We thank the reviewer for the suggestion to report the performance on more diverse DGPs. This helps to demonstrate the general capability of our learners across DGPs. We constructed two DGPs in addition to the one of the main experiment (DGP A):

  • DGP B (non-smooth baseline, complex effect). We replace the short-term outcome model of DGP A by the baseline and effect functions of Setup B of [4], extended by interaction and quadratic terms (Appendix G.1).

  • DGP C (complex baseline, constant short-term effect). We replace the short-term outcome model of DGP A by the baseline and effect functions of Setup C of [4], so that the HLTE is heterogeneous only through the nonlinear long-term outcome model (Appendix G.1).

We evaluate both DGPs under the same four overlap regimes as in Table 2. The tables report the PEHE (mean ±\pm std. dev. over 5 runs; best per regime in bold).

DGP BD∗\mathcal D_*Dt\mathcal D_tDo\mathcal D_oDt+o\mathcal D_{t+o}
LT-T0.132±0.0190.132\pm0.0190.164±0.0230.164\pm0.0230.126±0.0070.126\pm0.0070.127±0.033\mathbf{0.127\pm0.033}
LT-RA0.129±0.0140.129\pm0.0140.251±0.0270.251\pm0.0270.139±0.0190.139\pm0.0190.251±0.0270.251\pm0.027
LT-IPW0.134±0.0140.134\pm0.0140.891±0.6690.891\pm0.6690.128±0.0240.128\pm0.0240.783±0.1620.783\pm0.162
LT-O-DR0.033±0.0060.033\pm0.0060.629±0.5300.629\pm0.5300.201±0.0610.201\pm0.0614.676±4.6624.676\pm4.662
LT-O-TO0.032±0.005\mathbf{0.032\pm0.005}0.100±0.024\mathbf{0.100\pm0.024}0.162±0.0770.162\pm0.0770.223±0.0340.223\pm0.034
LT-O-LO0.037±0.0070.037\pm0.0070.752±0.4680.752\pm0.4680.101±0.029\mathbf{0.101\pm0.029}6.220±7.8366.220\pm7.836
LT-O-DO0.032±0.007\mathbf{0.032\pm0.007}0.105±0.0360.105\pm0.0360.114±0.0200.114\pm0.0200.246±0.0500.246\pm0.050
DGP CD∗\mathcal D_*Dt\mathcal D_tDo\mathcal D_oDt+o\mathcal D_{t+o}
LT-T0.197±0.0470.197\pm0.0470.208±0.0160.208\pm0.0160.187±0.0430.187\pm0.0430.361±0.3260.361\pm0.326
LT-RA0.202±0.0150.202\pm0.0150.244±0.0210.244\pm0.0210.183±0.0330.183\pm0.0330.228±0.0570.228\pm0.057
LT-IPW0.227±0.0280.227\pm0.0280.707±0.2520.707\pm0.2520.218±0.0600.218\pm0.0600.885±0.1510.885\pm0.151
LT-O-DR0.011±0.0030.011\pm0.0030.271±0.0880.271\pm0.0880.142±0.0870.142\pm0.0870.620±0.3730.620\pm0.373
LT-O-TO0.013±0.0050.013\pm0.0050.038±0.017\mathbf{0.038\pm0.017}0.220±0.0460.220\pm0.0460.120±0.0610.120\pm0.061
LT-O-LO0.009±0.002\mathbf{0.009\pm0.002}0.240±0.1330.240\pm0.1330.069±0.0200.069\pm0.0200.469±0.2020.469\pm0.202
LT-O-DO0.009±0.003\mathbf{0.009\pm0.003}0.044±0.0070.044\pm0.0070.058±0.019\mathbf{0.058\pm0.019}0.106±0.020\mathbf{0.106\pm0.020}

In 7 of the 8 settings, one of our LT-O-learners achieves the lowest PEHE, and the LT-O-DO-learner improves on the unweighted LT-O-DR-learner in all six low-overlap settings. The exception is Dt+o\mathcal D_{t+o} of DGP B, where the LT-T-learner is best. This suggests that the performance of our learners is not restricted to a specific DGP.

Action: We added DGP B and DGP C to the description of the synthetic data (Appendix G.1) and their results under all four overlap regimes to Appendix B.6 (Table 8).

We hope this clarifies the relation to recent meta-learners and the scope of our experiments, and we are happy to answer further questions.

[4] Xinkun Nie and Stefan Wager. “Quasi-oracle estimation of heterogeneous treatment effects.” Biometrika, 108(2):299–319, 2021.

I thank the authors for their rebuttal, which I believe strengthens the paper. I have therefore increased my score.

Thank you for reading our response and for raising your score. Your constructive comments helped us improve the paper, and we are happy to clarify any further questions.

Reviewer_ovxKOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies the estimation of heterogeneous long-term treatment effects (HLTEs) under limited overlap in treatment assignment and long-term outcome observation. The authors propose a class of long-term orthogonal learners (LT-O-learners) that retarget the loss function using overlap weights to downweight observations with limited overlap. The proposed loss preserves Neyman orthogonality and recovers the true HLTE under appropriate assumptions. The authors establish theoretical results on robustness to nuisance estimation errors and asymptotic variance stabilization under a linear HLTE model. Numerical experiments demonstrate improved estimation performance under various limited-overlap scenarios.

Critical strengths
  1. The paper provides systematic investigation of estimating HLTE using retargetted loss.
  2. The theory is clear and solid.
Critical weaknesses
  1. The connection between optimal weighting and efficient HLTE estimation is unclear. The proposed overlap weights are motivated by the efficiency bound for the weighted average long-term treatment effect (WALTE). However, minimizing this efficiency bound does not necessarily lead to efficient estimation of HLTEs. Although the authors establish variance stabilization under a linear model, it remains unclear whether the proposed weighting schemes are optimal or generally preferable for HLTE estimation.

  2. The optimal weighting scheme may depend on the structural assumptions imposed on HLTEs. For example, the optimal weights for estimating the coefficients of a linear HLTE model may differ from those for other parametric or nonparametric models. Moreover, under model misspecification, different weighting schemes generally correspond to different weighted projections of the true HLTE function. Thus, it is unclear how useful the proposed weights are for the ultimate goal of estimating HLTEs, rather than estimating a retargeted loss.

  3. Potential optimization difficulties due to negative weights. The orthogonalized weights can be negative, resulting in a potentially nonconvex empirical objective. Although the authors discuss this issue and establish theoretical well-posedness, these results do not necessarily guarantee numerical stability or reliable optimization in finite samples, particularly with flexible machine learning models.

Questions for authors

Q1. Although the authors establish variance stabilization, this does not necessarily imply that the weighted estimator is more efficient than the unweighted estimator in general. Under what conditions can the proposed weighting schemes guarantee improved estimation efficiency relative to the unweighted learner?

Q2. How does the choice of overlap weights affect the estimation target and prediction accuracy under model misspecification? Since different weights can lead to different weighted projections of the true HLTE, could the authors investigate the implications for HLTE estimation under misspecified structural models?

Q3. How do the authors handle negative orthogonalized weights in practical optimization? In particular, how frequently do they cause nonconvexity or optimization instability?

4 comments

We thank the reviewer for the careful reading and the constructive comments. We address each concern below. We further updated the submission PDF in response to the comments (labeled as Action) and highlighted the changes in blue color.

W1 & Q1: Connection between the optimal weight and efficient HLTE estimation

Thank you for raising this important question. In our derivations of the LT-O-learners, we followed the line of reasoning below (similar to how overlap-weighting is motivated for CATE [1], policy values [2], or time-varying CAPOs [3]):

  1. First, we derive (a) the efficient estimator of the averaged long-term effect.

  2. We find that it contains inverse overlap weights (which, under (i) the assumption of homoscedasticity, are the main drivers of large asymptotic variance).

  3. We use those overlap weights to redefine the causal estimand [4]: averaged overlap-weighted LT effect.

  4. We derive (b) the efficient estimator for the averaged overlap-weighted LT effect (and it does not have the inverse overlap weights by design).

  5. Under (i) the homoscedasticity and (ii) the assumption that the averaged overlap-weighted LT effect equals the original LT effect, it is easy to see that (b) is strictly more efficient than (a) (see [4]).

A similar line of reasoning can be repeated for meta-learners (namely, what we do in our work), yet with one important caveat: Now there are two targets of inference, namely the target risk (= intermediate target) and the target model (= final target that minimizes the target risk). This is similar to the distinction between the policy value and the learned optimal policy [2]. Here, the analogous semi-parametric efficiency arguments (1)-(5) can be applied w.r.t. the target risk but not the target model itself (unless the target model is parametric, e.g., a linear model). That is, for the target model, the semi-parametric efficiency is not defined and is replaced with quasi-oracle efficiency.

Most importantly, the quasi-oracle efficiency by itself cannot answer which Neyman-orthogonal learner yields a generally “more efficient” target model (the unweighted or overlap-weighted learner); it only describes at what rate different nuisance errors contribute to the final target model errors. Then, to answer this question, we would need to know an exact formula of how the optimized target model depends on the observational data and nuisance functions (this is known, for example, for a linear model; see Theorem 5.3). Therefore, our LT-O-learners are both general and model-agnostic, and optimal in some special cases (in the same way that analogous overlap-weighted learners in other settings are).

With these arguments in mind (and given similar literature on CATE/CAPOs/policy learning), we still think that our paper provides a valuable contribution to causal ML: We introduced an entire family of overlap-weighted learners that (under additional assumptions and in special cases) are optimal.

W2 & Q2: Dependence of the weights on the model class

We thank the reviewer for pointing out that the estimator is subject to the model class G\mathcal{G} when the true target is not contained in it (τ0∉G\tau^0 \notin \mathcal{G}). Intuitively, the overlap-adaptive re-targeting downweights the regions that are intrinsically difficult (and thus disruptive for the target loss objective). These weights serve to stabilize the loss, which eventually leads to overall improved quality of the HLTE target. The effectiveness has been extensively validated in the overlap weighting literature [1, 3, 5]. A strict guarantee for the final target, however, would rely on the actual instantiated model family (see our derivation for the linear model as an example). In our paper, we focus on developing the general, model-agnostic theory, and hence do not provide such guarantees, which vary case by case.

We also provide empirical evidence under misspecification in our paper. In Appendix B.5, we report the performance of a linear model class fitted on a dataset with an actual nonlinear HLTE. => Our results show that our LT-O-DO-learner outperforms the non-weighting baselines (see Table 6).

Action: We added the performance under misspecification to the experiment section (Section 6), with a pointer to Appendix B.5.

W3 & Q3: Negative orthogonalized weights in practical optimization

We thank the reviewer for raising this point. The empirical loss is well-posed in that it converges to the strongly convex asymptotic population loss, and is bounded below regardless of the signs of ω^i∗\hat\omega^*_i (see Theorem 5.1). Hence, the minimizer exists for bounded outputs (Appendix F.6). As a result, negativity of ω^i∗\hat\omega^*_i does not invalidate the empirical loss as a whole. Indeed, in all of our experiments, the LT-O-learners optimize smoothly and effectively.

Why do negative weights emerge? Proposition C.3 shows that the emergence of negative weights is actually inevitable. Finite-sample convexity and an orthogonalized variance-stabilizing weight are thus incompatible. This is a limitation of the setting, not of our method. In fact, signed weights arise in other orthogonal overlap-weighted learners as well [1, 3, 6]. We choose to preserve orthogonality and the overlap weight, and ensure well-posedness through the conditions above.

How we handle them in practice (Q3). We keep the sign of the weights and trim their absolute values away from 0 at 10−610^{-6}, because negative weights are essential for ensuring the key orthogonality. With negative weights, the LT-O-DO-learner has the lowest PEHE in the Dt\mathcal{D}_t and Dt+o\mathcal{D}_{t+o} regimes (see Table 2) on average over 5 runs, demonstrating the robustness of the optimization. Therefore, the empirical study also suggests that the signed weights do not affect the well-posedness of the optimization.

Action: We added the handling of signed weights to Appendix I.

We hope this clarifies the role of the weights and the handling of negative weights, and we are happy to answer further questions.

[1] Paweł Morzywołek, Johan Decruyenaere, and Stijn Vansteelandt. “On a general class of orthogonal learners for the estimation of heterogeneous treatment effects.” arXiv:2303.12687, 2023.

[2] Nathan Kallus. “More efficient policy learning via optimal retargeting.” arXiv:1906.08611, 2019.

[3] Konstantin Hess, Dennis Frauen, Mihaela van der Schaar, and Stefan Feuerriegel. “Overlap-weighted orthogonal meta-learner for treatment effect estimation over time.” ICLR, 2026.

[4] Richard K. Crump, V. Joseph Hotz, Guido W. Imbens, and Oscar A. Mitnik. “Moving the goalposts: Addressing limited overlap in the estimation of average treatment effects by changing the estimand.” NBER Technical Working Paper 330, 2006.

[5] Xinkun Nie and Stefan Wager. “Quasi-oracle estimation of heterogeneous treatment effects.” Biometrika, 108(2):299–319, 2021.

[6] Dennis Frauen, Maresa Schröder, Konstantin Hess, and Stefan Feuerriegel. “Orthogonal survival learners for estimating heterogeneous treatment effects from time-to-event data.” NeurIPS, 2025.

I would like to thank the authors' efforts to address my concerns. Although the response cannot address all my concerns since some comments are fundamental, I am happy to move to a score of 3.

Thank you for reading our response and for raising your score. Your constructive comments helped us improve the paper, and we are happy to clarify any further questions.

Reviewer_3EwyOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LT-O-learners for estimating heterogeneous long-term treatment effects by combining short-term treatment data with historical long-term outcomes. The method uses overlap-weighted orthogonal losses to address limited treatment and outcome overlap. The authors establish theoretical properties and evaluate the learners through simulations and a GAIN–PSID application.

Critical strengths
  1. The paper addresses a meaningful problem and clearly distinguishes treatment overlap from long-term outcome overlap. Extending orthogonal learning to account for both is a useful methodological contribution.

  2. The theoretical development is substantial, covering oracle equivalence, nuisance-estimation error, and variance stabilization. The simulations complement the theory with overlap-stratified results, bias–variance analysis, and misspecification experiments.

Critical weaknesses
  1. The scope of the robustness claims should be clearer. The variance-stabilization result assumes a correctly specified linear effect model and a sufficiently supported region that identifies its parameters. These are reasonable conditions, but the broader statements about robustness in low-overlap populations need this qualification.

  2. The real-data application transfers a comparability correction from year 9 to years 11–12. This requires the cross-dataset discrepancy to remain sufficiently stable over time, which is consequential for the estimated effects but not directly validated.

Questions for authors
  1. Could you clarify which robustness guarantees extend beyond the correctly specified linear setting, particularly for prediction in low-overlap subgroups?

  2. How sensitive are the GAIN–PSID estimates to the comparability correction and its transfer across outcome horizons?

2 comments

We thank the reviewer for the careful reading and the constructive comments. We address each concern below. We further updated the submission PDF in response to the comments (labeled as Action) and highlighted the changes in blue color.

W1 & Q1: Scope of the robustness guarantees beyond the correctly specified linear setting

We thank the reviewer for the request to clarify the scope of provable robustness. To the best of our knowledge, it is still an open question how to strictly prove, in the non-parametric setting, that the mean squared error of the predicted conditional treatment effects can be reduced in low-overlap regions [1].

Nevertheless, our method is still built on concrete theoretical robustness guarantees that apply to the general setting:

  1. Robustness to nuisance errors. Neyman orthogonality (Theorem 5.1) and the error bound (Theorem 5.2) hold for any closed convex class G\mathcal{G} (Assumption F.4).

  2. The target is preserved. By oracle equivalence (Theorem 5.1), if τ0∈G\tau^0\in\mathcal{G}, the weighted loss recovers τ0\tau^0 for every xx, including low-overlap subgroups. If τ0∉G\tau^0\notin\mathcal{G}, then bias is inevitable. Our method achieves lower MSE by reducing the high variance in the low-overlap region.

  3. The source of the instability is removed. Lemma K.6 shows that overlap weighting keeps the conditional variance of the oracle residual in the low-overlap regions bounded under Assumption K.8(d)–(e). This establishes stability of the learning signal in low-overlap regions.

From both practical and theoretical perspectives, the low overlap of treatment / observation causes high variance in the low-overlap region. Our idea is to directly control the variance of the loss via overlap-adaptive weighting, while preserving the target under oracle nuisances. While the variance is reduced, this might introduce bias into the HLTE estimates. Empirical evidence shows that it is often the variance that dominates, not the bias, in the case of low overlap.

Indeed, empirical evidence indicates that variance is more important to address than the bias under this bias–variance trade-off. In Figure 2, the variance of the LT-O-DO-learner stays low as overlap decreases, while that of the LT-O-DR-learner increases. In Appendix B.2, we show that, stratified by overlap, the LT-O-DO-learner has a similar error in the lowest- and highest-overlap strata, whereas the LT-O-DR-learner has its largest mean error in the lowest-overlap stratum (see, e.g., Figure 6). In Appendix B.5, we show that under misspecification, the LT-O-DO-learner still attains the lowest PEHE in the low-overlap stratum (Table 6). These results together demonstrate the effectiveness of our method’s variance reduction for low-overlap regions.

Action: We added an explicit discussion of the implications of this bias–variance trade-off and the underlying reasoning on why we can stabilize the estimation in low-overlap regions to the end of Section 5.3.

[1] Paweł Morzywołek, Johan Decruyenaere, and Stijn Vansteelandt. “On a general class of orthogonal learners for the estimation of heterogeneous treatment effects.” arXiv:2303.12687, 2023.

W2 & Q2: Transfer of the comparability correction from year 9 to years 11–12

We thank the reviewer for the careful examination of our experiments. First of all, comparability across two datasets is, in principle, hard to verify, and GAIN has no earnings after year 9 to check the transfer. This is, by the way, exactly the reason why researchers need to combine the two datasets together to compute long-term estimates. To make the problem tractable, we therefore assume that the same type of outcome (here: earnings) changes smoothly over years, and so does the gap between the two datasets. This lets us extrapolate the correction based on earlier years and give a range for it.

We re-estimated the correction with earnings in years 7 and 8, which both datasets observe, and extrapolated linearly to years 11–12. The table shows the correction (1000 USD) with its bootstrap standard error.

SubgroupYear 7Year 8Year 9Years 11–12 (extrapolated)
All0.068 ± 0.0300.117 ± 0.0300.066 ± 0.0260.063 ± 0.046
Age <300.007 ± 0.0340.080 ± 0.0340.010 ± 0.0370.013 ± 0.076
Age 30–390.112 ± 0.0450.165 ± 0.0440.102 ± 0.0390.089 ± 0.063
Age 40–490.038 ± 0.0400.064 ± 0.0440.067 ± 0.0380.102 ± 0.067
Age ≥500.086 ± 0.0440.074 ± 0.0510.030 ± 0.044−0.040 ± 0.074
HS diploma0.125 ± 0.0500.199 ± 0.0490.118 ± 0.0430.109 ± 0.074
No HS diploma0.010 ± 0.0200.033 ± 0.0220.013 ± 0.0230.016 ± 0.044
Prior earnings0.059 ± 0.0380.118 ± 0.0420.052 ± 0.0390.045 ± 0.065
No prior earnings0.077 ± 0.0340.116 ± 0.0350.078 ± 0.0290.080 ± 0.052
White0.129 ± 0.0540.211 ± 0.0510.131 ± 0.0430.135 ± 0.076
Black0.046 ± 0.0380.085 ± 0.0380.048 ± 0.0370.051 ± 0.066
Other ethnicity0.022 ± 0.0230.043 ± 0.0270.011 ± 0.028−0.002 ± 0.058

Then, we apply the shift to every learner equally. We find that, under the extrapolated shift (with the plausible range), the LT-O-DO- and LT-O-TO-learners continue to give more stable subgroup estimates than the LT-O-DR-learner (Figure 10; cf. Figure 3, Table 7). => Therefore, the result is not sensitive to the correction itself.

Action: We added the smoothness assumption, the new results (new Table 11), and the subgroup estimates with the uncertainty of the correction term (new Figure 10). See also our updated Appendix G.2.3, which we cross-reference in Section 6.

We hope this clarifies the scope of our guarantees and of the real-world application, and we are happy to answer further questions.

Comments on the paper

1 comment

Thank you for handling our submission and for the constructive reviews.

We respond to each reviewer’s comment in detail, and we revised the paper according to the reviewers’ suggestions. All changes are highlighted in blue in the updated PDF. Below we summarize the main improvements:

  • We added the H-learner as a new baseline, adapted to our long-term setting (LT-H-learner). In all four overlap regimes, it does not improve on the better of the LT-T- and LT-RA-learners, and its PEHE stays above that of our LT-O-DO-learner (new Appendix B.7; see our answer to Reviewer ZTQw; W1).
  • We added experiments on two additional data-generating processes, built on Setups B and C of Nie & Wager (2021). One of our LT-O-learners achieves the lowest PEHE in 7 of the 8 settings, and the LT-O-DO-learner improves on the unweighted LT-O-DR-learner in all low-overlap settings (new Appendix B.6; see our answer to Reviewer ZTQw; W2).
  • We added a sensitivity analysis of the comparability correction on GAIN–PSID. We re-estimated the correction at years 7, 8, and 9 and extrapolated it to years 11–12 with its uncertainty. Our LT-O-DO- and LT-O-TO-learners continue to give more stable subgroup estimates than the LT-O-DR-learner (Appendix G.2.3; see our answer to Reviewer 3Ewy; W2).
  • We clarified the scope of our robustness guarantees beyond the linear class, with a discussion of the bias–variance trade-off of overlap weighting (Section 5.3; see our answer to Reviewer 3Ewy; W1).
  • We now report the performance under a misspecified model class in the main paper. Our LT-O-DO-learner still attains the lowest PEHE in both the low- and high-overlap strata (Section 6 and Appendix B.5; see our answer to Reviewer ovxK; W2).
  • We added the practical handling of signed weights: we keep the sign of the weights, which is required for orthogonality, and trim their absolute values away from zero (Appendix I; see our answer to Reviewer ovxK; W3).
  • We added a discussion of recent meta-learners for difference-in-differences, treatments over time, and regularization (Appendix A.1; see our answer to Reviewer ZTQw; W1).

We further explain the connection between the optimal weights and efficient HLTE estimation in our answer to Reviewer ovxK (W1).

Given these improvements, we are confident that we have addressed all the concerns. We are happy to engage in any further discussion. We believe that our novel meta-learners for heterogeneous long-term treatment effects will make a nice contribution to ICLR.

Thank you!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.