acceptodds
Under review as a conference paper at ICLR 2027

Hallucinated Heterogeneity: LLM Social Simulations Exaggerate Who Responds to Interventions

20 views

Abstract

Large language model (LLM) social simulations are increasingly used as a fast, cheap substitute for human experiments, and are typically validated on whether they recover average treatment effects. But targeting and personalization depend on a different question: who responds? We introduce H-Bench (Heterogeneity Bench), a benchmark of 138 randomized experiments spanning economics, politics, health, and other domains, which uses causal inference to test whether simulations recover treatment effect heterogeneity, from subgroup differences to the value of targeting decisions, against randomized human outcomes. Across nine LLMs, we find Hallucinated Heterogeneity: simulations track average effects far better than who responds, and they exaggerate how much people differ. Claude Sonnet 5's average effects correlate with human estimates at r=0.74, but its subgroup differences in treatment effects correlate at only r=0.14 (p<0.001 for the difference), a gap that persists after correcting for human sampling noise (0.80 vs. 0.29, p=0.04). Eight of nine models overstate heterogeneity by 1.4–2.7×, and LLM-based targeting barely beats random assignment, even after calibration on human data. We explain why this failure arises and goes undetected. Simulated responses are often far from human ones, but the errors are similar with and without treatment and cancel in the average effect, so standard validation passes. And the heterogeneity models produce follows familiar divides such as partisanship, where models exaggerate it without identifying which groups respond more. Together, these results show that LLM social simulations must be validated at the level at which they will be used before they can support targeting.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Large Language Models, Synthetic Respondents, Social Simulation, Heterogeneous Treatment Effects

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_A17mOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether llms can accurately simulate heterogeneous treatment effects in social science experiments. The authors construct H-Bench by aggregating 138 existing randomized experiments from five publicly available data sources, resulting in 530 treatment–outcome contrasts and 2,906 subgroup comparisons.

The authors find that models predict ATEs more accurately than subgroup-level treatment interactions. They further report that most evaluated models overestimate treatment-effect heterogeneity. LLM-based targeting policies also offer limited improvement over random treatment assignment.

The paper also investigates potential explanations for these observations, including the cancellation of prediction errors across treatment conditions and the exaggeration of familiar political and demographic differences.

Critical strengths
  • The paper attempts to address an important problem in social simulation using llms.
  • The dataset is comprehensive and built from real-world experiments, which is needed in this field.
Critical weaknesses
  • Several related observations have already been documented in previous literature. For example, some previous work [1] has investigated the discrepancy between LLMs' ability to predict aggregate experimental effects and their substantially weaker performance on demographic interactions. They showed that strong correlations with average treatment effects do not necessarily extend to demographic interaction effects. Other works [2,3,4] have already investigated related failure modes in persona-based simulations and proposed some solutions. Also, Section 5.1 highlights that prediction errors in the treatment and control groups can cancel out, leading to seemingly accurate ATE estimates. This is closely related to existing work on prediction-powered inference (PPI), which addresses biased model predictions by leveraging real observations for statistical correction.
  • The paper introduces H-Bench as a new benchmark, but the underlying human experimental data are drawn from existing sources.

[1] Anthis, Jacy Reese, et al. “Position: LLM Social Simulations Are a Promising Research Method.” 2025

[2] Wang, Angelina, et al. “Large Language Models That Replace Human Participants Can Harmfully Misportray and Flatten Identity Groups.” 2025

[3] Cheng, Myra, et al. “CoMPosT: Characterizing and Evaluating Caricature in LLM Simulations.” 2023

[4] Zhang, B., Li, J., Hortaçsu, A., Ye, X., Chernozhukov, V., Ni, A., & Huang, E. W. Agentic economic modeling. 2025

Questions for authors
  • Do you think there are temporal effects on the dataset? For example, some datasets were used to train the models you used, but some were not. How could that influence the outcome?
2 comments

We thank the Reviewer for the thoughtful comments. We have addressed each point below and marked the corresponding revisions in the main text and appendix in orange.

W1: Novelty relative to prior work

We respectfully disagree that our main findings have been documented before. To our knowledge, no prior work systematically tests, across models and against randomized human outcomes, whether LLM simulations recover who responds to an intervention. Our paper establishes four results that the cited works do not:

  1. A causal benchmark for heterogeneity. H-Bench evaluates subgroup interactions, model-ranked groups, and targeting decisions, all identified by randomization (Table 1).
  2. Hallucinated heterogeneity. Across eleven models and 2,906 subgroup comparisons, simulations recover average effects far better than interactions, even after correcting for sampling noise in the human data, and ten of eleven models overstate heterogeneity by 1.4–2.7×.
  3. The cost for decisions. Model rankings used for targeting perform near random assignment, and calibration with human data does not close the gap (Section 4; Appendix C.4).
  4. Why validation misses it. Arm-level errors cancel in the average effect, masking wrong heterogeneity.

Anthis et al. [1] is a position paper reviewing LLM social simulations; we now cite it in Related Work. Its remark on demographic interaction effects is a one-sentence summary of earlier work in its appendix, concerning a single model. It reports no analysis of the size, direction, cause, or decision cost of the gap, which are results 2–4 above.

Wang et al. [2] and Cheng et al. [3] study how groups are portrayed without an intervention. We study whether differences in response to an intervention are preserved, a causal question. The distinction matters empirically: eight of eleven models recover baseline group differences reasonably well (r = 0.64–0.76), but interaction correlations range only from −0.01 to 0.23. Evaluations of portrayal therefore cannot detect the failure we document.

PPI and Zhang et al. [4] use real observations to correct model predictions and obtain valid inference for a chosen estimand. Section 5 addresses a different question: it is a diagnosis of why validating simulations on average effects can pass while their heterogeneity is wrong, not a correction method. The two are complementary; we now cite both in Related Work.

W2: H-Bench builds on existing human data

Using existing randomized experiments is deliberate: a heterogeneity benchmark needs randomized outcomes with respondent-level characteristics, and peer-reviewed, publicly archived experiments are the most credible and reproducible source. H-Bench contributes what the sources do not provide: (i) a harmonized benchmark of 530 contrasts and 2,906 subgroup comparisons; (ii) matched simulations of every observed profile under every arm for eleven models, including frontier proprietary models (GPT-5.6 Luna, Claude Sonnet 5); and (iii) an evaluation framework with noise correction, cross-fitting, and a held-out targeting design. We will release all data, outputs, and code.

Q1: Temporal effects and training-data exposure

If models had memorized published results, exposure could explain why they recover average effects but not heterogeneity. We dated every experiment by the earliest article, working paper, or dissertation reporting its results (searching the literature as of November 2026) and recomputed the correlations for all eleven models within three groups (new Appendix E and Table 9).

ATE r / interaction rPublished by 2021Published 2022 or laterNo publication found
Contrasts / subgroup comparisons241 / 1,313231 / 1,25958 / 334
Claude Sonnet 50.77 / 0.280.67 / −0.010.82 / 0.27
GPT-5.6 Luna0.54 / −0.010.54 / −0.020.83 / −0.06
Qwen3.5 397B-A17B0.80 / 0.370.61 / −0.020.87 / 0.29
Median of eleven models0.63 / 0.280.54 / 0.040.77 / 0.08

The results do not support exposure as an explanation. In all three groups, every model except the smallest (Gemma 3 4B) recovers average effects better than interactions. Models recover average effects on unpublished experiments at least as well as on older ones (median r = 0.77 vs. 0.63), so this accuracy does not depend on having seen published results. Median interaction correlations are higher for experiments published by 2021, which could reflect exposure or study composition, but this would only flatter models on published data.

Thanks the author for the review. I think the clarification on the related work and the new exposure experiment addressed some of my concerns. I would like to raise my score to weak acceptance.

Reviewer_yUxJOpenReview ↗
Rating
3
Confidence
2(+1)
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper shows that LLM social simulations show stronger agreement with human estimates of average treatment effects (ATEs) and baseline gaps than with differences in subgroup treatment effects. Their rankings also provide limited targeting value. Therefore, validating LLM simulations directly for their intended use, including subgroup analysis and targeting decisions, is important. The paper presents a benchmark called H-Bench that combines 138 randomized experiments from five public sources to test LLM simulations. The authors also demonstrate that LLM simulations inflate the estimated heterogeneity, which they call hallucinated heterogeneity. They empirically illustrate how errors in predicted treatment and control means can cancel, allowing agreement on ATEs despite inaccurate outcome-level predictions.

Critical strengths

Overall, I find the paper strong and clearly written. The paper provides a substantial benchmark for evaluating an important limitation of LLM social simulations. H-Bench combines 138 randomized experiments and evaluates nine LLMs, extending validation beyond average effects to subgroup treatment effects and targeting decisions. The results show that agreement on average effects can coexist with inaccurate or exaggerated heterogeneity. I think this is very intriguing finding. The targeting experiment in Section 4 is particularly interesting for practical decisions. At a target treatment budget, the tested LLM policies generally provide limited improvement over random allocation across the ten selected experiments, while the human-data policy achieves a positive estimated gain. This shows why it is very important to validate LLM simulations for their intended use, because the results may otherwise be misleading in practice. The analysis also address human sampling noise. This addresses an alternative explanation for the weaker agreement on subgroup effects.

Critical weaknesses

It is not clear to me how the targeting value in Eq. (3) is estimated from observed data. The main text and Appendix D.2 would benefit from further details, particularly the explicit estimator of V(q)V(q) and the computation of the error bars in Figure 2. I also find Appendix D.1 quite compact; a clearer explanation about the ExtraTrees scores SikS_{ik}, the variable Γik\Gamma_{ik}, and the BLP regression would be helpful.

Small points:

Although Section 3.1 reports noise-corrected correlations and Appendix C.3 describes the procedure for addressing the human sampling noise, the discussion can be expanded in the main text as it is outlining an important concept in the paper. I think this is crucial to interpret the reported results in the main text. Lie 937 and 981: please clarify what Nature experiments refer to. Are these the experiments from Ashokkumar et al., 2026? Line 1086: Specify what BLP is in the text.

Questions for authors

Regarding the points about V(q)V(q) above, could the authors provide the explicit formula for Γik\Gamma_{ik} and the observed-data estimator used to calculate the targeting values V(q)V(q) in Figure 2? How are the error bars computed? Section 5 provides useful evidence that correlated arm-level errors can cancel in ATE predictions. However, the underlying mechanism remains unclear in my opinion. I recognize that identifying this mechanism is difficult and may be beyond the paper’s scope. Could the authors clarify what the analysis establishes and what remains open? Also regarding Section 5, I am wondering about the reliability of ATE predictions for new interventions when human validation is unavailable. How stable is the cancellation of arm-level errors across interventions, and under what conditions does it break down?

2 comments

We thank the Reviewer for the detailed feedback. We have addressed each point below and marked the corresponding revisions in the main text and appendix in pink.

Targeting-value estimator and error bars

Appendix D.2 now gives the observed-data estimator. Since no respondent is observed in both arms, we replace the individual effect in Eq. (3) with a transformed outcome whose expectation is the human CATE, which yields a Horvitz–Thompson version of the PAPE estimator with the ten experiments weighted equally. Error bars are 95% percentile intervals from 5,000 bootstrap resamples of the experiment-level estimates, conditional on the discovery-half selection and fitted policies; Section 4 and the Figure 2 caption now state this.

ExtraTrees scores and BLP

Appendix D.1 now defines the transformed outcome, explains how ExtraTrees produces cross-fitted CATE scores from it, and describes the best linear predictor (BLP) regression, whose slope tests whether higher scores predict larger human effects.

Human sampling noise

Section 3.1 now explains that sampling noise attenuates correlations, far more for interactions (76% of variance) than for ATEs (14%), and how Appendix C.3 corrects for it. After correction, all eleven models retain a positive ATE–interaction gap, and corrected interaction correlations remain well below corrected ATE correlations.

What the error-cancellation analysis establishes

Section 5 establishes an empirical error structure, not its cause. Simulated outcome levels are substantially wrong (median arm-level RMSE about 0.6 human SD), but the errors in the treatment and control arms of a contrast are strongly correlated (r = 0.45–0.94 across eleven models; Figure 3c). Because ATE error equals e1-e0, correlated arm errors cancel, so for ten of eleven models the ATE error is smaller than either arm error (Figure 3d). This explains how ATE validation can pass while simulated responses are wrong. What remains open is why arm errors are correlated (for example, how models use response scales, persona conditioning, or prior beliefs about a topic). Answering this question requires controlled interventions on prompts or model representations.

Reliability for new interventions

Cancellation is common but not guaranteed. Arm errors are positively correlated for every model, but the correlation varies across models (0.45–0.94) and across sources: it is weaker in the persuasive-video experiments than in those of Ashokkumar et al. (median 0.51 vs. 0.71). Cancellation breaks down when the intervention changes how the model answers the question, so that the two arms carry different errors. Cancellation explains why ATE errors are smaller than arm errors, but it does not guarantee an accurate ATE: the remaining error reflects whether the model predicts how the intervention shifts responses. Whether cancellation holds, or whether an ATE is accurate, for a new intervention cannot be determined without human validation.

Nature experiments and BLP

Yes, these are the experiments of Ashokkumar et al. (2026). We now cite them explicitly instead of using the shorthand, and BLP is spelled out at its first occurrence.

Reviewer_yUxJOpenReview ↗

update following author response

Dear authors, thank you for the responses. Your answers and the revisions to the manuscript have addressed my comments. I will keep my positive rating and increase my confidence.

Reviewer_1xG6OpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

We introduce H-BenchBench, a benchmark to evaluate whether LLMs will misjudge who responds to interventions in LLM-based simulation. The proposed benchmark consists of 138 randomized human experiments and spans various domains. Evaluations are conducted over nine LLMs in four families and reveal that the simulations track average effects far better than they track who responds, and they exaggerate how much people differ.

The paper is well-motivated with an example. The benchmark is useful and the limitation of state-of-the-art LLMs in social simulations is revealed. In addition, the paper also discusses the cause for this gap and attributes it to simulation errors cancelled out in the average effect.

Critical strengths

S1. Rather than average effects, this paper investigate whether LLM-based simulation can accurately simulate who will respond to interventions. This is a question ignored by existing studies.

S2. The proposed benchmark covers various domains.

S3. The paper is well-motivated with a running example in the beginning of the paper.

S4. In addition to the discovery of the limitation of existing LLMs in judging who responds, the cause of the limitation is also investigated.

Critical weaknesses

W1. The paper is mostly diagnostic but the discussion on the remedy is limited. For example, a simple calibration is tested but it fails to close the gap.

W2. The paper gives little positive guidance on when LLM-based simulation remains usable (see Q1 below).

W3. While the evaluation covers nine LLMs, most open models evaluated are rather small (see Q2 below).

Questions for authors

Q1. A short section on trustworthy use cases, ideally with the evidence from the analysis already in this paper, would substantially strengthen the paper's impact.

Q2. It is suggested that more advanced open models, as well as more model families, are considered for evaluation, e.g., most recent Qwen, DeepSeek, Mistral, or GLM models. The model sizes are larger, up to several hundred billion parameters. It is interesting to see whether the observed results also apply to these models, and if so, whether the same cause of the gap can be attributed.

2 comments

We thank the Reviewer for the constructive suggestions. We have addressed each point below, with all revisions to the manuscript marked in blue.

W1: Discussion on the remedy

Section 4 already tests calibration against randomized human data: affine, isotonic, and flexible calibrators fitted on the discovery half do not close the targeting gap (Figure 2d). The new Appendix C.4 (Table 6) adds a cross-experiment calibration: for each model, we fit a linear map from simulated to human interactions and apply it to held-out experiments. This significantly lowers average-effect error for all eleven models but does not fix heterogeneity. It shrinks simulated interactions toward zero, turning overstatement (median inflation ratio 1.64) into significant understatement (0.35), and no model becomes significantly more accurate than predicting no difference between groups. We also added a paragraph to the Discussion on directions that could change the simulated heterogeneity signal itself, such as fine-tuning on randomized experiments with subgroup outcomes and richer respondent representations.

W2 / Q1: Practical guidance

We have added a paragraph to the Discussion explaining where simulations remain useful and where human validation is essential.

What simulations can be used for.

  • Average effects. Simulated ATEs are positively correlated with human ATEs across 530 contrasts (Section 3.1). Simulations may therefore offer a rough, preliminary signal about the direction and relative size of average effects.
  • Baseline subgroup differences. Models recover baseline subgroup differences reasonably well (Appendix Table 2), and both models and humans show heterogeneity concentrated in political rather than demographic characteristics (Section 5.2). Simulations thus capture which groups differ, a pattern that does not extend to how groups respond to treatment (see below).

What simulations cannot yet be relied on for. Our results do not support using simulations to:

  • Determine which groups benefit more, or decide whom to treat. Interaction recovery is weak even when baseline differences are recovered (r = −0.01–0.23), and ten of eleven models exaggerate heterogeneity (Sections 3.1–3.2), including along the political divides where it concentrates (Section 5.2). Consequently, targeting based on model rankings performs close to random assignment (Section 4).
  • Predict absolute outcome levels or assume precise effect sizes from ATE correlations. Substantial treatment- and control-arm errors can cancel in average effects; correlation alone does not establish numerical accuracy (Section 5.1).

W3 / Q2: Evaluation of larger open models from additional families

We have added full evaluations of Qwen3.5 397B-A17B and Mistral Small 4 119B, spanning two recent open-model families.

ModelATE rBaseline rInteraction rLevel RMSE
Qwen3.5 397B-A17B0.7230.7570.1380.508
Mistral Small 4 119B0.4790.5900.1071.074
GPT-5.6 Luna0.5820.663-0.0130.801
Claude Sonnet 50.7390.7200.1430.468

The new models do not remove the gap. Their raw ATE–interaction correlation gaps are 0.585 for Qwen3.5 and 0.371 for Mistral. After correcting for sampling noise in the human data, with intervals from the same 5,000 bootstrap resamples of experiments, the results are:

ModelCorrected ATE rCorrected interaction rCorrected gap95% gap intervalp
Qwen3.5 397B-A17B0.7780.2840.494[0.077, 0.859]0.026
Mistral Small 4 119B0.5150.2210.294[-0.057, 0.495]0.077

Both models retain a positive corrected gap, significant for Qwen3.5, so the finding is not specific to small open models or a single family. They also show the same signatures that Section 5 identifies: baseline group differences transfer much better than treatment interactions, and for Mistral the large arm-level errors (about 1.1 human SD) largely cancel in the average effect (0.35). Across eleven models, the median RMSE is 0.59 and 0.63 human SD for control and treatment means, versus 0.41 for average effects (updated Figure 3d). Larger and newer models can improve average-effect recovery, but they do not by themselves make simulated heterogeneity reliable.

Thanks for the response. My rating maintains.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.