- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies heterogeneous long-term treatment effects estimation, with a particular focus on low treatment overlap and low long-term outcome overlap. It adapts existing meta-learners to the HLTE setting and proposes a class of orthogonal learners that improve estimations by downweighting samples with poor overlap.
Critical strengths
The paper addresses an important and practical problem of estimating HLTE when short-term and long-term outcomes are observed in separate datasets, and under low-overlap settings. The proposed method is intuitive and theoretically justified, and the empirical results show its effectiveness under low-overlap setups. The paper is also clearly written and easy to follow.
Critical weaknesses
- The meta-learners used for benchmarking and discussed in the related work are mostly methods proposed before 2023, while several improved variants have been published in the past two years. E.g:
[1] Liang, Zhongyuan, Lars van der Laan, and Ahmed Alaa. "Hybrid Meta-learners for Estimating Heterogeneous Treatment Effects."
[2] Lan, Hui, et al. "A meta-learner for heterogeneous effects in difference-in-differences."
[3] Frauen, Dennis, Konstantin Hess, and Stefan Feuerriegel. "Model-agnostic meta-learners for estimating heterogeneous treatment effects over time."
The related work section would benefit from discussing how these more recent variants could be adapted to the HLTE settings and possibly include them in the empirical performance comparisons.
- The main synthetic evaluation uses one generic data-generating process and focuses particularly on low-overlap settings. Prior literature has shown that different meta-learners can perform differently across different DGPs. It is unclear how the proposed method would perform in more general and more diverse DGPs.
Questions for authors
Please see my weakness above. It would be helpful if the authors could revise the paper to discuss how more recent meta-learners relate to the described HLTE setting and the proposed method.
4 comments
W2: Performance in more general and more diverse DGPs
We thank the reviewer for the suggestion to report the performance on more diverse DGPs. This helps to demonstrate the general capability of our learners across DGPs. We constructed two DGPs in addition to the one of the main experiment (DGP A):
-
DGP B (non-smooth baseline, complex effect). We replace the short-term outcome model of DGP A by the baseline and effect functions of Setup B of [4], extended by interaction and quadratic terms (Appendix G.1).
-
DGP C (complex baseline, constant short-term effect). We replace the short-term outcome model of DGP A by the baseline and effect functions of Setup C of [4], so that the HLTE is heterogeneous only through the nonlinear long-term outcome model (Appendix G.1).
We evaluate both DGPs under the same four overlap regimes as in Table 2. The tables report the PEHE (mean std. dev. over 5 runs; best per regime in bold).
| DGP B | ||||
|---|---|---|---|---|
| LT-T | ||||
| LT-RA | ||||
| LT-IPW | ||||
| LT-O-DR | ||||
| LT-O-TO | ||||
| LT-O-LO | ||||
| LT-O-DO |
| DGP C | ||||
|---|---|---|---|---|
| LT-T | ||||
| LT-RA | ||||
| LT-IPW | ||||
| LT-O-DR | ||||
| LT-O-TO | ||||
| LT-O-LO | ||||
| LT-O-DO |
In 7 of the 8 settings, one of our LT-O-learners achieves the lowest PEHE, and the LT-O-DO-learner improves on the unweighted LT-O-DR-learner in all six low-overlap settings. The exception is of DGP B, where the LT-T-learner is best. This suggests that the performance of our learners is not restricted to a specific DGP.
Action: We added DGP B and DGP C to the description of the synthetic data (Appendix G.1) and their results under all four overlap regimes to Appendix B.6 (Table 8).
We hope this clarifies the relation to recent meta-learners and the scope of our experiments, and we are happy to answer further questions.
[4] Xinkun Nie and Stefan Wager. “Quasi-oracle estimation of heterogeneous treatment effects.” Biometrika, 108(2):299–319, 2021.
I thank the authors for their rebuttal, which I believe strengthens the paper. I have therefore increased my score.
Thank you for reading our response and for raising your score. Your constructive comments helped us improve the paper, and we are happy to clarify any further questions.
We thank the reviewer for the careful reading and the constructive comments. We address each concern below. We further updated the submission PDF in response to the comments (labeled as Action) and highlighted the changes in blue color.
W1 & Q: Discussion with recent meta-learners and the comparison
We thank the reviewer for bringing up these references. We discuss them one by one below:
Meta-learner for DiD [2]: The learner of [2] targets the CATT under conditional parallel trends, which requires pre- and post-treatment outcomes of the same units. So it does not apply to our setting, where we target the long-term CATE.
Meta-learners for CATE over time [3]: [3] studies the CATE under a sequence of treatments over time. Essentially, the learners that [3] proposed are the longitudinal extension (using the g-computation principle) of the T-, IPW-, and DR-learners with a single treatment. Its IVW-DR-learner applies inverse-variance weighting, but the learner is not orthogonal, unlike our proposed LT-O-learners. Importantly, we proposed learners that both apply overlap weighting and retain orthogonality.
H-learner [1]: The H-learner regularizes the target CAPO functions with a loss that is a weighted combination of the plug-in loss and the RA loss. Hence, it does not provide any new confounding adjustment strategy. However, its regularization technique can indeed be extended to our setting, and it would be interesting to compare the performance as an auxiliary baseline.
We implemented the long-term analog of the H-learner (LT-H-learner) and tested its performance under the setting of the synthetic experiment. Below, we report its PEHE (mean ± std) across 5 runs, together with the scores of the LT-T- and LT-RA-learners.
We find that, across all regimes, the LT-H-learner does not beat the better of the LT-T- and LT-RA-learners. This is consistent with the construction of the LT-H-learner’s loss function, which is a λ-weighted combination of the plug-in loss (highly correlated with the T-learner) and the RA-learner’s loss. Therefore, the newly implemented LT-H-learner does not beat our LT-O-DO-learner in any regime (Table 2).
Action: We added a paragraph on [1–3] to the extended related work (Appendix A.1). We added the H-learner, adapted to our setting, as a baseline to the synthetic experiments (Appendix B.7, Table 9).
[1] Liang, Zhongyuan, Lars van der Laan, and Ahmed Alaa. "Hybrid Meta-learners for Estimating Heterogeneous Treatment Effects." AISTATS, 2026.
[2] Lan, Hui, et al. "A meta-learner for heterogeneous effects in difference-in-differences." ICML, 2025.
[3] Frauen, Dennis, Konstantin Hess, and Stefan Feuerriegel. "Model-agnostic meta-learners for estimating heterogeneous treatment effects over time." ICLR, 2025.