- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes Q-loss, a learning objective that can be used to directly train meta learners with the Q-statistic without access to counterfactuals. Motivated by a study on real world data in which no heterogeneity was found, the authors assert that standard meta learners are insufficient for CATE estimation as they rely on surrogates, and instead propose an end-to-end learning framework. The proposed learner is validated on real and synthetic data.
Critical strengths
- The paper is mostly self-contained.
- The idea of directyl learning CATE estimators without second-stage regression sounds interesting.
- Empirical results look somewhat promising.
Critical weaknesses
-
The authors propose Q-loss. However, completign the squares in their loss exactly yields standard pseudo outcome regression. There is nothing novel in their estimand (see for example Curth et al., 2021 and Kennedy, 2023). To be fair with the authors, they acknowledge this equivalence, but as a result, they strongly overclaim their contribution.
-
Proposition 1, as stated, is incorrect. The minimizer is only CATE on population level, not for an empirical finite sample estimator.
-
Proposition 2 is also, as stated, incorrect. Consistency requires assumptions on the neural network class / complexity class of the estimator.
-
Proposition 3 is overclaiming what the feature gates are doing. If a feature is irrelevant for CATE estimation, this only means that optimal prediction does not use this feature. Without regularization, there is no reason to assume that the feature gate will go to 0.
-
Experiments are insufficient: The most important comparison would be against neural DR learners, which are missing in the experiments.
-
Lastly, but this should not be the main reason for rejection: The motivation is questionable. I checked the referenced paper by Yu & Sun (2025), which serves as the main motivation. Here, a number of CATE learners was evaluated on RCTs, and a large number of them revealed no HTE. The authors call this degeneracy of the learners. I have severe doubts that this can be asserted so easily. Instead, what is more likely is either: i) there was no treatment effect (hence, is correct since ), or ii) there was a treatment effect, but no heterogeneity (and hence is correct as well.
The only novelty I see in this work is the integration of a Gumbel softmax function, but this is not enough for an ICLR contribution.
Alicia Curth & Mihaela van der Schaar. Nonparametric Estimation of Heterogeneous Treatment Effects: From Theory to Learning Algorithms. In AISTATS, 2021.
Edward H. Kennedy. Towards Optimal Doubly Robust Estimation of Heterogeneous Causal Effects. arXiv preprint, arXiv:2004.14497.
Questions for authors
-
Could the authors explain where exactly the novelty in their estimator lies, especially compared to standard learners?
-
Could the authors correct Propositions 1-3?
-
Could the authors explain why neural DR estimators are missing as baselines in their experiments?