- Rating
- 4(+3)
- Confidence
- 4
- Informativeness
- 2(+1)
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
This paper proposes an estimator for the distributional treatment effect (DTE) in settings where a surrogate outcome is fully observed but the primary outcome is only partially observed under a missing-at-random (MAR) labeling mechanism. The estimator is built on doubly robust and Neyman-orthogonal (DML) principles: it combines two conditional CDFs (one using only covariates and one augmented with the surrogate) into an AIPW score, estimated via cross-fitting. Experiments on synthetic data and real-world applications demonstrate the method's effectiveness and improved precision over the chosen baselines.
Critical strengths
-
The empirical evaluation covers four real-world applications spanning LLM evaluation, natural language, and clinical trials, which strengthens the practical relevance of the method.
-
The theoretical results are detailed.
Critical weaknesses
-
The role and assumptions of the surrogate outcome are not formally defined. The paper introduces the surrogate only descriptively ("a surrogate available for everyone") without specifying its structural relationship to the primary outcome or the conditions it must satisfy. It is also unclear how to assess whether the surrogate carries useful information beyond covariates. Moreover, the real-world scenarios under which both full surrogate observability and MAR are plausible are not discussed.
-
The discussion of and experimental comparison with the treatment effect estimators designed for partially observed outcomes are absent. While the paper builds on Kallus & Mao (2025), it does not deeply discuss the broader literature on treatment effect estimation with partially observed outcomes [1-11], nor does it include experimental comparisons against relevant methods from that line of work. This gap makes it difficult to assess the relative novelty and empirical soundness of the proposed approach.
-
The methodological novelty needs clearer positioning. The estimator combines surrogate-assisted doubly robust estimation with DML-style orthogonalization and distributional inference, which is a combination of well-established ideas. The paper should more explicitly clarify its specific contributions relative to existing doubly robust methods for partially observed outcomes (some of which also handle distributional estimands [10, 11] or even MNAR settings [6, 7, 9, 11]).
Reference
[1] Heckman, J.J.: Sample selection bias as a specification error. Econometrica: Journal of the econometric society, 153–161 (1979).
[2] Robins, J.M., Rotnitzky, A., Zhao, L.P.: Analysis of semiparametric regression models for repeated outcomes in the presence of missing data. Journal of the american statistical association 90(429), 106–121 (1995).
[3] Rotnitzky, A., Robins, J.M., Scharfstein, D.O.: Semiparametric regression for repeated outcomes with nonignorable nonresponse. Journal of the american statistical association 93(444), 1321–1339 (1998).
[4] Wooldridge, J.M.: Econometric Analysis of Cross Section and Panel Data. MIT press, Cambridge (2010).
[5] Ding, P.: Bayesian robust inference of sample selection using selection-t models. Journal of Multivariate Analysis 124, 451–464 (2014).
[6] Tchetgen Tchetgen, E.J., Wirth, K.E.: A general instrumental variable framework for regression analysis with outcome missing not at random. Biometrics 73(4),1123–1131 (2017).
[7] Sun, B., Liu, L., Miao, W., Wirth, K., Robins, J., Tchetgen, E.J.T.: Semiparametric estimation with data missing not at random using an instrumental variable. Statistica Sinica 28(4), 1965 (2018).
[8] Little, R.J., Rubin, D.B.: Statistical Analysis with Missing Data. John Wiley & Sons, Hoboken (2020).
[9] Li, W., Miao, W., Tchetgen Tchetgen, E.: Non-parametric inference about mean functionals of non-ignorable non-response data without identifying the joint distribution. Journal of the Royal Statistical Society Series B: Statistical Methodology 85(3), 913–935 (2023).
[10] Yifan Cui, Michael R Kosorok, Erik Sverdrup, Stefan Wager, and Ruoqing Zhu. 2023. Estimating heterogeneous treatment effects with right-censored data via causal survival forests. Journal of the Royal Statistical Society Series B: Statistical Methodology 85, 2 (2023), 179–211.
[11] Miao, W., Liu, L., Li, Y., Tchetgen Tchetgen, E.J., Geng, Z.: Identification and semiparametric efficiency theory of nonignorable missing data with a shadow variable. ACM/JMS Journal of Data Science 1(2), 1–23 (2024).
Questions for authors
Please see Weaknesses.