acceptodds
Under review as a conference paper at ICLR 2027

Mitigating Hidden Confounding by Progressive Confounder Imputation via Large Language Models

Abstract

Hidden confounding remains a central challenge in estimating treatment effects from observational data, as unobserved confounders can lead to biased causal estimates. Although recent work has explored the use of large language models (LLMs) for causal inference, existing approaches still rely on the unconfoundedness assumption and therefore cannot address hidden confounding. In this paper, we make the first attempt to mitigate hidden confounding using LLMs. We propose ProCI (Progressive Confounder Imputation), a framework that leverages two key strengths of LLMs: their ability to interpret rich textual information in causal benchmarks, which is often ignored by prior methods, and their internalized world knowledge, which enables the generation of plausible latent confounders and supports counterfactual reasoning. ProCI alternates between imputing a plausible hidden confounder based on the semantics of observed variables, and validating the imputation by empirically assessing whether the new confounder can help restore conditional independence between treatment and outcome. To enhance robustness, ProCI adopts a distributional reasoning strategy rather than direct value imputation, preventing collapsed outputs. Extensive experiments demonstrate that ProCI uncovers meaningful hidden confounders and substantially improves treatment effect estimation across diverse datasets, base models, and LLM backbones.

Primary area:
causal reasoning
Keywords:
Causal Inference, Hidden Confounder, Large Language Model

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_P5QCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

In causal inference, hidden confounding could bias the treatment effect estimation and is very common in real-world practice. This paper propose ProCI (Progressive Confounder Imputation) framework, which use large langue models (LLMs) to address this challenge by providing additional informaton. This framework is built on two assumptions (line 19-22) i) LLM can interpret rich text information in causal benchmarks; ii) LLM has world knowledge that allows generating latent confounders and support causal reasoning.

ProCI estimates treatment effect in two steps, confounder imputation and unconfoundness validation (Section 3). In confounder imputation phase, ProCI let LLM generate hidden confounder U based on the semantic relationship among observed covariate X, outcome Y, and treatment T, along with a text description. LLM is then use to impute the value of U by inferring its distribution type and parameters. After imputation, the observed confounder X is then augmented by incorporating U. After phase 1, ProCI performs an conditional independence test to assess whether the generated confounder U captures sufficient hidden bias. If the test fails, it returns to Phase 1 to generate an additional confounder; if it passes, it proceeds to CATE estimation.

The authors then conduct experiments to answer research questions (RQs) 1 to 5 (line 305-311). In RQ1, they show ProCI can improve the performance of existing CATE estimatiors on the evaluated dataset. In RQ2, ProCI demonstrated better improvement with domain-adaptive LLM routing. In RQ3, the authors show ProCI gives higher conditional mutual information (CMI) than random. In RQ4, the authors testes the sensitivity of ProCI to the number of hidden confounders. In RQ5, the author did an ablation study to evaluate the contribution of components of ProCI.

In conclusion, this paper propose ProCI, a framework to mitigate hidden confounding in treatment effect estimation.

Critical strengths
  1. Important research question. This paper studies how to address hidden confounding in treatment effect estimation. It is a fundamental problem in causal inference and addressing that can reduce bias in estimation and better inform the final decision making. Exploring whether LLMs encode hidden confounding information and how to use that information is an interesting research direction.

  2. Clear writing. The paper is well structured. I can easily follow the reasoning flow and understand the proposed framework. The progressive imputation procedure is easy to understand and well explained. The paper also provide many experiment to support the argument, including many CATE baselines, different LLM models, and ablation studies on the level of hidden confounding and components of the proposed method.

Critical weaknesses
  1. The statistical validity of LLM-generated confounder in phase 1 is not sufficiently generated. This paper propose using LLM to generate hidden confounding. Specifically, in equation (2), U^,U^exp=LLM(Pvar(X,T,Y))\hat{U}, \hat{U}_{\mathrm{exp}} = \mathrm{LLM}(\mathcal{P} {\mathrm{var} (X, T, Y))}. In addition, ProCI also let LLM infer the distribution and its parameter, in Equation (3) and (4), respectively. Therefore, the proposed hidden confounder UU not only depend on observed covariate XX, but also on treatment TT and outcome YY. This causes severe identification issue, since a valid adjustment variable needs to precede treatment and outcome. The paper lacks sufficient justification that the imputed variables belong to valid adjustment set. As a result, this procedure may lead to strong bias. Moreover, the paper assumes LLM can provide correct information for hidden confounding. However, LLM may hallucianate and the paper does not give sufficient discussion on the consequence of that.

  2. The unconfoundedness validation in phase 2 is not neccessarily correct. To test whether LLM generated confounders are valid, the paper proposes using conditional independence test (Y(0),Y(1))⊥T∣X,U^(Y(0), Y(1)) \perp T | X, \hat{U}. To address the problem that the counterfactual outcomes, Y(1)Y(1) or Y(0)Y(0), are never observed, the authors use LLM to impute the missing outcomes, in Equation (5). Assuming LLM can correctly impute unobserved outcomes is a very strong assumption, which is unlikely to hold in practice.

Even though we assume that LLMs may correctly impute the the unobserved outcomes, the conditional independence test cannot justify that generated confounders are valid adjustment set in general. For example, let U^=g(T)\hat{U} = g(T), where gg is some measurable function. Then (Y(0),Y(1))⊥T∣X,U^(Y(0), Y(1)) \perp T | X, \hat{U} even U^=g(T)\hat{U} = g(T) is not a valid adjustment variable. Conditioning on X,U^X, \hat{U} will break the positivity assumption.

Questions for authors
  1. Under what conditions can LLM provide valid adjustment set rather than variables inferred post-hoc from TT and XX? Furthermore, can we verify these conditions in real-world application? If so, how?

  2. Under what conditions can the proposed unconfoundedness validation correctly verify that current covariates are sufficient for identification? If the conditional independence test in Equation (6) is needed for that, please justify why LLM give correct imputation for the unobserved outcomes Y(1)Y(1) or Y(0)Y(0).

Reviewer_PZJ1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

Existing LLM-based treatment effect estimation method still mostly relies on the unconfoundedness assumption, which assumes that no unobserved variables can cause confounding bias. To this end, this paper proposes ProCI (Progressive Confounder Imputation), an LLM-based framework for mitigating hidden confounding from observational data. Specifically, they employ LLMs for iteratively proposing hidden variables and value imputation. To assess whether the imputed confounders are sufficient, they employ a KCIT. If the test fails, the ProCI continues to propose a new hidden variable until the test is passed.

Critical strengths
  • The paper is generally clear and easy to follow.
  • The presentation is good.
Critical weaknesses
  • How do you ensure variable imputation is correct? Passing the KCIT does not guarantee that the generated variable and values are correct. Although in Appendix E.3, you have used a control experiment to let the LLMs recover the missing variable, it remains unknown whether the values are correct. Additional metrics, such as MAE of the generations and original values need also to be reported.
  • Missing necessary baselines. The authors only try LLMs for both variable generation and value imputation. What about traditional hidden variable discovery method?
  • The used LLMs are a little bit outdated. Including more recent LLMs would help to evaluate whether the proposed method achieves better results with stronger models.
Questions for authors

See Critical Weakness

Reviewer_iAsbOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This study proposes a novel framework called ProCI (Progressive Confounder Imputation) for estimating causal effects under the possible influence of latent confounders by using the intrinsic knowledge learned by an LLM. ProCI mainly alternates among four steps: 1) guessing a plausible missing confounder using an LLM based on the semantics of the observed variables, 2) imputing its values for each sample by using an LLM to estimate the distributional type and individual-specific distributional parameters of the candidate confounder, 3) generating counterfactual outcomes using an LLM based on the observables and the generated latent confounders, and then 4) evaluating the validity of the estimated latent confounder using a kernel-based conditional independence test (KCIT). The numerical experiments showed improvements in causal inference performance across multiple datasets, LLM models, and causal inference methods.

Critical strengths
  1. The problem setting seems to be novel and would contribute to addressing the difficult problem of accounting for latent confounders in the field of causal inference.

  2. The study presents extensive numerical experiments and shows improvements in causal effect estimation when combined with several existing causal inference methods, robustness to the choice of LLM model, and the importance of each proposed component through an ablation study.

Critical weaknesses
  1. Untestable assumptions: This study relies on an LLM to infer a latent confounder variable, its distributional type, its individual-specific distributional parameters, and counterfactual outcomes. However, it is unclear to what extent the LLM can reliably perform these steps. In particular, the study does not sufficiently justify why the LLM can generate counterfactual data that are accurate enough for causal effect estimation.

  2. Related to the above: Although Theorem 1 seems to be introduced to justify the proposed framework, it is based on the assumption that Yhat^0 and Yhat^1 are sampled from the true distributions belonging to a specific class. Although this is simply described as “regularity condition and the class of imputation distributions” and first presented in the Appendix, this reviewer thinks this is a central assumption that should be explicitly stated in the main text, with sufficient discussion of its strength and the conditions under which it may hold.

  3. The framework would depend on the quality and detail of the variable names and descriptions provided by the human user. However, it is unclear to what extent missing information or differences in wording affect its performance.

  4. Regarding RQ3, measuring the conditional mutual information I(U, T|X) and I(U, Y|X) does not necessarily justify interpreting the additional information captured by the imputed confounders U beyond the observed covariates as “confounding information”; U could simply be correlated with T and Y, or could actually be a collider instead of a confounder.

  5. The numerical experiments are limited to the usual Twins and Jobs datasets. This raises the concern that the reported performance may partly reflect the LLM's prior knowledge of these benchmarks, leaving the framework's generalizability to less familiar settings unclear.

Questions for authors

See Weaknesses

Reviewer_C8zxOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

ProCI uses an LLM to propose semantically plausible unobserved confounders and impute individual values through predicted distribution families and parameters. It alternates this augmentation with counterfactual-outcome imputation and a kernel conditional independence test, stopping at the first non-rejection or an iteration limit. The augmented data are then supplied to standard treatment-effect estimators. Experiments on Twins and Jobs cover several estimators and LLMs, domain routing, component ablations, variable removal, and an alternative outcome-imputation stage. The paper claims that this procedure mitigates hidden confounding and that its imputation-based test asymptotically agrees with testing the true latent variables.

Critical strengths
  1. The paper addresses an important problem and proposes a concrete, interpretable workflow for using semantic prior knowledge to suggest candidate missing variables. Separating variable naming from distributional parameter inference is a useful design choice for avoiding identical-value imputation and makes the method straightforward to combine with several downstream estimators.

  2. The experimental coverage includes multiple estimators and LLM backbones, component ablations, and an explicit attempt to investigate circularity by changing the outcome-imputation model. These are useful exploratory checks, and some estimator–dataset combinations show appreciable improvements, even though they do not establish the claimed causal validity.

Critical weaknesses
  1. The identification and validation argument is not established. Eq. (4) constructs U_hat from the factual X,T,Y, so it is not automatically a pre-treatment adjustment variable or a valid proxy. Assumption 2 posits away the key problem rather than showing when semantic knowledge supplies the missing identifying information. In Appendix C, Lemma 2 assumes the imputed and true centered kernel matrices are already close, which is not derived from the LLM procedure. Moreover, Eq. (6) calls KCIT a p-value, whereas Appendix Eq. (11) and the proof concern a statistic; closeness of an unscaled statistic alone does not establish calibrated, equally powerful tests. Stopping at the first non-rejection cannot certify unconfoundedness. For example, conditioning on a generated variable that encodes T can make independence vacuous while violating positivity. These gaps undermine the central claimed guarantee.

  2. The experiments do not distinguish causal recovery from outcome-informed feature construction. Since T and Y are inputs to U_hat, higher I(U_hat;T|X) and I(U_hat;Y|X) are not evidence that a genuine pre-treatment confounder has been recovered. Appendix E.3 checks semantic labels rather than recovery of individual hidden values; E.5 compares downstream errors, not the false-pass rate or power of the stopping test. The protocol also does not clearly state how out-of-sample U_hat is produced without test outcomes. Finally, the claim of consistent improvement is contradicted by Table 1: on Jobs, TARNet's out-of-sample ATT error rises from 0.1466 to 0.3223 with ProCI-4o; on Twins, S-Learner's out-of-sample ATE error rises from 0.0037 to 0.0077 and PEHE from 0.2819 to 0.2867. Selective gains do not resolve the identification problem.

Questions for authors
  1. What explicit structural or external-information assumptions make the generated U_hat a valid adjustment variable despite its dependence on factual T,Y? Please establish positivity and clarify the estimand after augmentation. Can the method distinguish observationally equivalent causal models with different treatment effects, and if so, what external information supplies that distinction?

  2. Can Theorem 1 be stated and proved for the actual p-value/stopping procedure, with explicit imputation convergence rates, kernel bandwidth and regularization conditions, and test power? Please measure false acceptance of insufficient adjustment on known SCMs, including deliberately wrong imputations, rather than using non-rejection itself as evidence of success.

  3. Please specify the complete train/validation/test imputation protocol, particularly whether factual test outcomes enter U_hat. Can you report X-only test-time imputation, quantitative recovery of held-out confounder values, and matched non-LLM feature-construction baselines, while accounting for the degradations in Table 1? These results would determine whether the empirical gains support a narrower, valid contribution.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.