- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces computational identifiability, a practical alternative to traditional theoretical identifiability in causal inference. While theoretical identifiability asks whether a causal query can be uniquely determined under idealized assumptions, computational identifiability asks whether an estimator can be found that achieves a specified error tolerance and confidence level with a finite sample and computational search procedure. The paper formalizes this framework by connecting causal effect estimation with meta-learning. The paper empirically demonstrates the framework in several settings.
Critical strengths
The paper introduces a new perspective on causal identifiability through the notion of computational identifiability. The central idea is novel and promising, and I believe it is worth pursuing further.
Critical weaknesses
My main concern is that the introduced notion of computational identifiability depends fundamentally on the chosen hypothesis space of candidate estimators and the computational search procedure. Consequently, the conclusions may be highly sensitive to these choices, which are not related to the problem itself. It is therefore unclear how robust the results are or how useful they are in guiding practical decisions as the conclusions depend strongly on some arbitrary model choices. Additionally, the proposed approach relies on the strong assumption that a prior distribution over SCMs is known.
I found it difficult to assess what substantive insights the experiments provide beyond the behavior of the particular estimator classes and settings considered. For example, in Section 4.1, it is unclear whether the results actually offer general guidance on which adjustment set is preferable, since the conclusions depend on the hypothesis space of estimators considered (in addition, the confidence intervals seem to be too wide to draw definite conclusions). Similarly, in Section 4.2, I am not sure what general insight about identifiability with mixed observational and interventional data can be drawn from the reported results.
Overall, although the proposed perspective is interesting, further development and stronger empirical validation are needed to demonstrate the robustness and practical usefulness of the framework.
Questions for authors
-Could the authors provide an intuitive explanation of what the causal mixture distribution represents in Definition 2?
-In Definition 4, what does it mean for a causal mixture distribution to be given? Specifically, what information is assumed to be available?
-Line 412: What does the of the meta-model represent?
-Line 447: The paper states that “computational identification of the CATE works in all settings.” What criteria are used to determine that identification is successful?
-Line 463: The paper states that “ITE estimation can be substantially worse.” I understand some data may not be helpful, but do authors have an explanation when or why having more data can make the estimation worse?
-Is it possible to design experiments to compare the predicted probability of identifiability against the ground truth? For example, in settings where a causal query is known to be identifiable, and where sharp analytic bounds for the causal query are known.