EICR: Executable Identifiability Counterfactual Reasoning for Large Language Models

Counterfactual reasoning requires more than executing an alternative action. A model must first infer which hidden explanations remain compatible with the factual observation, and then determine whether those explanations agree after intervention. We identify a failure mode in which a model selects one factually consistent explanation, obtains a locally valid counterfactual value, and reports it as a uniquely supported answer even though another equally valid explanation gives a different result. We call this phenomenon counterfactual identifiability collapse. We introduce EICR, an executable completion-set reasoning framework that represents a counterfactual answer by all observation-consistent structural completions, their branch-wise rollouts, the resulting answer set, and an instance-level identification label. We construct EIBench as a unified benchmark with an oracle-driven construction protocol over a common symbolic task space rendered as executable code, structural causal models (SCMs/DAGs), and natural language. It distinguishes point-identified, set-identified, interval-partially-identified, and unresolved cases. Under an explicit and complete candidate mechanism family, we establish soundness and completeness properties for the oracle certificate, define a training objective for learning task-dependent certificates, specify representation-aware dataset splits, and introduce metrics for false certainty, false uncertainty, set coverage, interval coverage, bound error, and certificate validity. Experiments across five model backbones and three representations show that completion-set supervision reduces false certainty on executable tasks and improves recovery of the supported answer set, while natural-language parsing remains more challenging than structured representations. These results support preserving the evidence-compatible completion set rather than committing prematurely to a single abductive explanation.