acceptodds
Withdrawn from ICLR 2027

Computational Identifiability

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Identification conditions describe the computability of a target query or parameter of interest as a function of the type and amount of information available. In causal identification, this information is often expressed in the form of a causal graph, and data are observed or collected for some subset of variables in the graph. Target queries may be for a single effect alone or for a class of effects in a given model. The derivation of an identification algorithm then defines mathematically the process by which the desired causal effect(s) can be uniquely determined, theoretically, in expectation. Identifiability in expectation, or 'theoretical identifiability,' generally assumes asymptotic properties, infinite data, or other mathematically idealized conditions. In this paper, we explore a fundamental distinction between this theoretical, idealized notion of identifiability and a proposed alternative that is computation-bound. The framework we propose — 'computational identifiability' — is to instead define a finite computational search procedure for an empirical estimator. If this process finds an estimator empirically, within a desired error tolerance, then identifiability is satisfied, conditional on the specified assumptions of the search (i.e., a prior distribution over the parameters) and conditional on the search procedure itself. Through several experiments, we demonstrate how this framework allows us to answer fine-grained, practical identification questions, such as identification with small finite samples, with ambiguous graphical criteria, with mixed observational-interventional data, and across counterfactual data and estimands.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
meta-inference, causal inference, identification

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Q8AyOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces computational identifiability, a practical alternative to traditional theoretical identifiability in causal inference. While theoretical identifiability asks whether a causal query can be uniquely determined under idealized assumptions, computational identifiability asks whether an estimator can be found that achieves a specified error tolerance and confidence level with a finite sample and computational search procedure. The paper formalizes this framework by connecting causal effect estimation with meta-learning. The paper empirically demonstrates the framework in several settings.

Critical strengths

The paper introduces a new perspective on causal identifiability through the notion of computational identifiability. The central idea is novel and promising, and I believe it is worth pursuing further.

Critical weaknesses

My main concern is that the introduced notion of computational identifiability depends fundamentally on the chosen hypothesis space of candidate estimators and the computational search procedure. Consequently, the conclusions may be highly sensitive to these choices, which are not related to the problem itself. It is therefore unclear how robust the results are or how useful they are in guiding practical decisions as the conclusions depend strongly on some arbitrary model choices. Additionally, the proposed approach relies on the strong assumption that a prior distribution over SCMs is known.

I found it difficult to assess what substantive insights the experiments provide beyond the behavior of the particular estimator classes and settings considered. For example, in Section 4.1, it is unclear whether the results actually offer general guidance on which adjustment set is preferable, since the conclusions depend on the hypothesis space of estimators considered (in addition, the confidence intervals seem to be too wide to draw definite conclusions). Similarly, in Section 4.2, I am not sure what general insight about identifiability with mixed observational and interventional data can be drawn from the reported results.

Overall, although the proposed perspective is interesting, further development and stronger empirical validation are needed to demonstrate the robustness and practical usefulness of the framework.

Questions for authors

-Could the authors provide an intuitive explanation of what the causal mixture distribution represents in Definition 2?

-In Definition 4, what does it mean for a causal mixture distribution to be given? Specifically, what information is assumed to be available?

-Line 412: What does the R2R^2 of the meta-model represent?

-Line 447: The paper states that “computational identification of the CATE works in all settings.” What criteria are used to determine that identification is successful?

-Line 463: The paper states that “ITE estimation can be substantially worse.” I understand some data may not be helpful, but do authors have an explanation when or why having more data can make the estimation worse?

-Is it possible to design experiments to compare the predicted probability of identifiability against the ground truth? For example, in settings where a causal query is known to be identifiable, and where sharp analytic bounds for the causal query are known.

Reviewer_z2JPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces computational identifiability, a framework that characterizes causal identifiability through finite-sample estimation performance under a specified prior and hypothesis space. The authors use meta-learning to empirically evaluate this notion across several causal inference settings.

Critical strengths
  1. The paper addresses an interesting and important problem by connecting causal identifiability with practical estimation under finite samples.

  2. The framework is general and considers several settings, including hidden confounding, observational-interventional mixtures, and counterfactual estimation.

Critical weaknesses
  1. The proposed notion of computational identifiability appears closely related to existing concepts in statistical learnability and finite-sample estimation. The paper does not sufficiently clarify what fundamentally new insights this definition provides beyond evaluating estimator performance under a specified prior.

  2. The proposed framework makes identifiability dependent on the hypothesis space, architecture, and optimization procedure. Consequently, poor estimation performance may reflect model limitations rather than intrinsic non-identifiability. This distinction is not adequately addressed.

  3. Despite introducing a new notion of identifiability, the paper provides limited theoretical characterization of its properties or its relationship to classical identifiability. Stronger formal results would substantially improve the contribution.

  4. The experiments primarily demonstrate the behavior of meta-learned estimators on synthetic causal models. It remains unclear whether the proposed framework provides actionable insights beyond standard estimation-error analysis or generalizes to more realistic settings.

Questions for authors
  1. How does the proposed notion of computational identifiability fundamentally differ from existing concepts of statistical learnability and finite-sample estimation?

  2. How can the authors distinguish intrinsic non-identifiability from poor estimation performance caused by limited model capacity or optimization failure?

  3. Can the authors provide stronger theoretical justification for the relationship between computational identifiability and classical causal identifiability?

  4. What additional insights does the proposed framework offer beyond evaluating the estimation accuracy of meta-learned models under different data distributions?

Reviewer_Qe9BOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes a notion of "computational" identifiability wherein the usual infinite sample/population statistic level identifiability/recoverability guarantees are replaced with a with what probability can you achieve this task formalism.

Critical strengths

Computation bound processing is a very critical topic for bridging the gap between the ideal mathematical constructions that are either too idealized (e.g. brittle algorithms) or too harsh (e.g., data processing inequality vs. self-feedback LLM progress) compared to their finite-sample finite-compute counterparts in practice. Formalizing such a concept in the causality context too is going to be a big step forward, even if analysis of the object itself may not be theoretically amenable at first.

Critical weaknesses

This being more of a position paper, I would like to criticise the definition of computational identifiability chosen. The main issues relating to computational bottlenecks are generally in the estimation and optimization steps of the problem. In the current formalism, the candidate set H\mathcal{H} is assumed given/fixed, and there is no formalization on how an h∈Hh \in \mathcal{H} is chosen, which goes exactly opposite to this intuitive -- and as far as I know more universal -- motivation. Formalizing the randomness in the "world as is" is more natural from a modelling point of view, and is valuable by itself if it can explain observed phenomena, but this should NOT be called computational identifiability per se.

Questions for authors

I am simply uncomfortable with the notion of "computation" ending up with a limited scope object in this field. If the authors can expand upon the reasons behind their specific choices of name and formalism or even just change the name, I'd love to increase my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.