acceptodds
Under review as a conference paper at ICLR 2027

Economic-Informed Persona Generation for Behaviorally Grounded LLM Populations

Abstract

Large language models are increasingly used as persona-conditioned user simulators, yet a plausible persona prompt is not a behavioral specification: the same text can induce different choice patterns across simulators, and matching observed behavior may not identify responses to interventions. We introduce Economically-Informed Persona Generators (EIPG), a framework that calibrates persona generators from discrepancies between simulated and observed behavior rather than from persona labels. In a controlled benchmark, weak intervention coverage can improve calibration while worsening unseen counterfactuals; richer interventions make the calibration signal more identifying and improve frozen test performance. For natural-language personas, an anchored parameterization preserves authored prompts while exposing sparse, interpretable preference adjustments. Across four local LLMs from three model families, useful corrections are model-dependent, while structured economic search transfers more reliably than more flexible LLM- or text-evolution search in our finite-budget experiments. On public discrete-choice data, calibration improves an initially misspecified synthetic population on held-out human choices, although direct human estimation remains substantially stronger on the primary held-out choice metrics. EIPG therefore reframes persona grounding as an identification problem: the goal is not merely to write plausible personas, but to construct synthetic populations whose behavior remains credible under interventions.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
LLM personas, synthetic populations, behavioral calibration, economic feedback, counterfactual evaluation, identification, intervention design, discrete choice, agent simulation, prompt refinement

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_wA8iOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposed EIPG to calibrate persona generations using the difference between the simulated behavior and the reference. The structured search summarizes the behavior of synthetic populations and adjusts the persona distribution through an optimization loop. The natural-language method modifies prompts through constrained preference adjustments. The framework was evaluated on synthetic environments and real world datasets across several different LLMs. The results show that more intervention coverage improves counterfactual prediction. However, calibration can also lead to worse performance on unseen interventions.

Critical strengths
  • The paper attempts to address an important question in the domain of simulating economic behaviors using LLMs. The paper provides a meaningful failure mode where calibration can lead to worse counterfactual predictions.
  • The paper provides insightful reasons and supporting experiments to study why calibration can make counterfactual predictions worse.
Critical weaknesses
  • My primary concern is that it is unclear from the paper how the M target is constructed. Including more details of this part can make the paper easier to read.
  • The experiments need more ablation and comparative studies to show the effectiveness of the framework. For example, why is the MNL needed in the middle of structured method? And some previous work [1, 2] also attempts to construct personas or use customer preference data for outcome calibration.

[1] Bohan Zhang, Jiaxuan Li, Ali Hortacsu, Xiaoyang Ye, Victor Chernozhukov, Anqi Ni, and Edward W Huang. 2026. Agentic Economic Modeling. [2] Mengxin Wang, DennisJZhang, andHengZhang.2026. Large language models for market research: A dataaugmentation approach. Marketing Science

Questions for authors
  • Regarding the first weakness, how is the target matrix constructed?
  • The MNL model fitted directly to human data substantially outperforms EIPG in one of the experiments. While I understand the proposed framework aims to construct reusable synthetic populations, what additional practical benefit does the calibrated population provide over the directly fitted MNL?
2 comments

We thank the reviewer for the constructive feedback. We agree that the construction of MtarM_{\mathrm{tar}} needs to be explained more clearly, as its source differs across experiments. In the controlled benchmark, reference moments are computed from choices generated by a predefined synthetic population, including choice shares, chosen-attribute means, and responses to interventions. In the natural-language experiment, we instead calculate the targets from a fixed synthetic reference model. We specify MNL preference coefficients for three segments (budget-sensitive, quality-oriented, and sustainability-oriented) and use them to calculate choice probabilities over predefined product contexts.

The resulting target vector contains 19 moments. Fifteen are segment-specific chosen-attribute means: for each segment and attribute, we multiply each product's attribute value by its reference choice probability and average across contexts. The remaining four capture intervention responses. For each intervention, we modify one attribute of a product while holding the others fixed, calculate the change in its choice probability, and average that change across reference segments and paired contexts.

For example, the budget-sensitive segment has a target mean chosen price of 1.4960. When we increase the intervened product's price by 25%, the aggregate target change in its choice probability is −0.0988. These values are fixed before calibration. The persona optimizer receives the discrepancies between its simulated moments and these targets, but not the underlying reference coefficients or individual reference probabilities.

We will add the full target vector, reference coefficients, and intervention specifications to the appendix. These experiments test calibration against known synthetic behavior, not human behavioral fidelity. In Nutri2Cycle, by contrast, the reference statistics are calculated from observed human choices.

Regarding the rol of MNL, we consider MNL also plays a specific, rather than essential, role in the framework. In the structured implementation, we fit MNL to choices generated by each candidate population and compare its implied moments with the reference. It provides a common, interpretable summary of candidate behavior,, but also introduces a modeling approximation. The natural-language implementation computes moments directly from LLM choices, without an intermediate MNL. We will clarify this distinction in the method section.

the comparison with direct-human MNL is particularly important. In the submitted Nutri2Cycle experiment, direct MNL predicts held-out choices substantially better than structured EIPG (share L2 of 0.0758 versus 0.4491). we do not claim that producing synthetic personas is inherently preferable to estimating a choice model directly. The potential advantage we want to investigate is whether pretrained behavioral knowledge can reduce the human data needed to calibrate a useful population. That motivated two new experiments:

We therefore run a new experiment and evaluated both approaches across human-label budgets. At 48 respondents (144 labeled choices), the calibrated LLM-persona mixture achieved held-out L2 of 0.12, compared with 0.18 for regularized direct MNL. The advantage was concentrated at smaller budgets; direct MNL became more accurate as additional observations were available. This would identify a specific data-scarcity use case for pretrained personas, rather than general predictive superiority. A convincing comparison also requires selecting MNL regularization using training data alone, which we would document. The result would not establish lower total computational cost or prove that pretraining itself causes the advantage.

We also calibrated a persona population on Nutri2Cycle Tasks 1–2, froze its descriptions and mixture weights, and evaluated it on Tasks 3–4 without further calibration or human labels. Its mean choice-share L2 was 0.08, compared with 0.11 for MNL trained on the same source observations, and it better reproduced the change in choices between the two price scenarios. MNL can also transfer without refitting, so the claim is limited to better predictive transfer in these particular within-domain scenarios. Task 4 had previously been examined in our submitted evaluation, making this an exploratory result rather than an independent confirmation.

Finally, regarding related work, we agree that using personas and human evidence to model economic behavior is not new. Zhang et al. also investigate synthetic agents for economic modeling, while Wang et al. develop a data-augmentation approach for statistical inference. Our focus is on calibrating the persona-generating mechanism against explicit behavioral discrepancies, including intervention responses. this produces an explicit population of agents rather than only corrected predictions or parameter estimates. We will clarify both the overlap and this methodological distinction.

I thank the authors for clarifying the construction of M and look forward to seeing these additions in the revised paper. I will maintain my weak accept recommendation.

Reviewer_ujHVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes EIPG, a framework for calibrating persona generators against behavioral evidence. It combines economic moment matching with either structured preference distributions or small, interpretable edits to persona prompts. Experiments cover a controlled synthetic benchmark, four local LLMs, and a public human-choice dataset. The main finding is that better calibration fit does not necessarily imply better counterfactual predictions; intervention coverage and the choice of prompt search method matter.

Critical strengths

The distinction between matching observed choices and predicting responses to interventions is useful. The controlled experiments give a concrete example where calibration improves while counterfactual performance gets worse, and show that increasing the inner model’s capacity does not automatically fix this.

Critical weaknesses
  • The connection to human-grounded LLM populations is still incomplete. The LLM experiments use three authored personas, while the human-data experiment evaluates a structured generator. The source and construction of the LLM study’s target moments, particularly the segment-specific targets, are not clearly explained. The authors acknowledge this, but the experiments do not yet show what useful capability the calibrated persona population adds over a directly fitted choice model.

  • The benefit of broader intervention coverage is unclear. Richer calibration beats price-only in only 2/5 final benchmarks (Table 15). On human data, gains over anchor-only are small, with no advantage on the respondent holdout, while response error remains slightly worse than the initial generator’s. Paired uncertainty estimates and matched observation budgets would make this claim more convincing.

Questions for authors
  • Where do the target moments in the LLM experiments come from? How are segment-specific targets obtained, and what information about the target population is assumed available?

  • Do price-only and richer calibration use the same total number of reference choice observations? Can you provide paired uncertainty estimates for their final-test difference across independent benchmark realizations?

  • Have you evaluated the anchored LLM method directly on held-out human choices, with a choice model trained on the same calibration evidence as a baseline? This would help connect the paper’s two main empirical components.

  • How does EIPG compare with MoP [1], which also learns a persona-based population simulator from observed responses without persona labels?

[1] Bui, Ngoc, et al. "Mixture-of-personas language models for population simulation." Findings of the Association for Computational Linguistics: ACL 2025. 2025.

1 comment

Thank you for your feedback! The review raises two important questions: what the natural-language experiments establish about human behavior, and whether the apparent benefit of multiple interventions is reliable.

The synthetic reference in the language-based experiment is deliberate. It lets us test whether economic feedback can correct the behavior induced by persona prompts under known preferences and controlled changes in product attributes. We specify three reference segments with fixed MNL preferences and calculate 19 behavioral targets: 15 segment-specific chosen-attribute means and four intervention responses. The optimizer does not receive the reference coefficients or individual choice probabilities; it receives discrepancies between the target moments and those generated by the personas.

This setting allows us to test whether calibration recovers specified behavioral relationships, rather than simply producing more plausible descriptions. We then freeze the personas and evaluate them on separately generated contexts. In the submitted experiments, held-out weighted moment error decreases across all four calibration/test seed pairs, with improvements ranging from 3.2% to 42.2%. This supports calibration against a controlled behavioral reference, not human behavioral fidelity. We will make that limitation explicit and add the full target vector, reference coefficients, and construction equations to the appendix.

The original intervention-coverage comparison also needs correction. In Table 15, four-intervention calibration beats price-only in only two of five independent benchmark realizations. Moreover, the four-intervention condition uses 2,400 reference observations, compared with 600 for price-only. The observed difference therefore cannot be attributed specifically to intervention diversity.

To alleviate that, we repeated the comparison at equal total observation budgets of 600 and 2,400, holding the anchor data, search budget, and evaluation protocol fixed. At 600 observations, four-intervention calibration reduced final-test L2 by 0.012 on average (95% paired CI [−0.021, −0.003]); at 2,400 observations, the reduction was 0.010 (CI [−0.018, −0.002]). It performed better in four of five independent benchmark realizations at each budget. These results would support an intervention-diversity effect beyond the original difference in reference-data volume.

The submitted Nutri2Cycle results do not demonstrate the same benefit. Its interventions vary prices rather than multiple product attributes, and the improvement in price-response prediction over anchor-only calibration is small. Furthermore, the human-grounded experiment uses a structured generator, whereas the natural-language experiments use synthetic targets. Evidence from one setting cannot establish the human validity of the other.

To examine this more directly, we now calibrated three authored bread-shopping personas against Nutri2Cycle choices from 618 training respondents (Tasks 1–3). We then froze the calibrated population and evaluated it on Task 4 choices from 154 separate respondents. Direct MNL was trained on the same human calibration observations. The LLM population achieved held-out choice-share L2 of 0.081, compared with 0.101 for direct MNL, and better reproduced the observed price response. The uncalibrated personas performed substantially worse. This would provide evidence that calibrating actual language-based personas can improve predictions against human observations, rather than only synthetic targets.

We would nevertheless retain the submitted structured-generator result, in which direct-human MNL is considerably better (L2 of 0.0758 versus 0.4491). These experiments differ in representation and respondent split; the new result would identify a potentially useful implementation, not overturn the earlier comparison. The Task-4 design was also previously examined, so the new evidence remains exploratory.

finally, Mixture-of-Personas (Bui et al., 2025) is closely related and should have been discussed. MoP also learns mixtures of persona-conditioned language-model agents to approximate population behavior. We therefore do not claim novelty in learned persona mixtures themselves. Our emphasis is on calibrating persona-induced behavior against explicit economic moments, particularly responses to controlled interventions, while permitting interpretable edits to preference descriptions. Matching observed choice distributions does not necessarily imply matching responses to interventions. We will clarify this distinction in the related-work section

Reviewer_9r2QOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes an algorithm called EIPG that calibrates populations of simulated shoppers against reference choices, adjusting numerical preferences or making small edits to written personas. The paper conducts experiments and shows that the method generally improves predictions, but gains vary across models and settings. A standard choice model fitted directly to human data remains substantially more accurate.

Critical strengths
  1. The paper demonstrates an important failure: fitting observed choices better can make predictions in new situations worse. In the controlled experiments, testing responses to several product changes helps, while using a more complex choice model does not. This gives a concrete lesson about what evidence is needed to validate simulated users.

  2. The authors evaluate selected personas on fresh situations after freezing their edits. Across four language models, they find useful but different corrections—including opposite brand-preference adjustments for two Gemma models. This supports the practical finding that a persona needs behavioral validation whenever the underlying model changes.

Critical weaknesses
  1. The target behavior for the LLM experiments needs more explanation. Section 4.3 and Appendix D give many details about the search, but I am still unclear how the target choice statistics are constructed and how they relate to real people. The human-data experiment uses numerical preference profiles, so it does not directly validate the written LLM personas.

  2. The practical benefit is still not very clear. On the bread-choice test, EIPG’s choice-share error is 0.4491, while a standard model fitted directly to human data achieves 0.0758. The authors argue that a reusable simulated population has other uses, but there is no downstream experiment showing that advantage. The extra complexity needs a stronger justification.

  3. The richer-intervention result is less consistent than the average suggests. In Table 15, four-attribute calibration beats price-only calibration in only two of five independent benchmarks after averaging search seeds. The reported average improvements of 6.6% versus 1.1% are encouraging, but there is only five independent benchmarks and no uncertainty estimate for this comparison. It is hard to judge how stable the advantage is.

  4. The human results are weak on predicting behavioral changes. Table 26 shows that richer calibration improves response error over anchor-only calibration, from 0.0348 to 0.0181. However, the original population already achieves 0.0174. On unseen respondents, the calibrated methods tie on choice-share error and neither improves response error. Since these tasks only vary price, they also cannot validate the broader claim about testing several preference dimensions.

  5. The GEPA comparison mixes two different changes. Coordinate search explores 30 fixed preference adjustments, while GEPA writes free-text suffixes. The reported gains of 16.4% versus 5.7% therefore do not tell us whether the search method is better or the restricted editing space simply works better here. The simulator-call budget also does not capture reflection costs or actual runtime.

Questions for authors
  1. How are the target choice statistics in the LLM experiments generated? Are any of these targets estimated from actual human responses?

  2. In Table 15, four-attribute calibration beats price-only calibration in only two of five independent benchmarks after averaging search seeds. Can you report the paired differences and their uncertainty, using independent benchmarks as the units? How reliable is the advantage beyond the aggregate average?

  3. Does the direct human choice model use exactly the same respondents and task outcomes available to EIPG?

1 comment

Thank you for you feedback ! The reviewer identifies an important distinction: improving agreement with a synthetic reference is not the same as demonstrating practical value for modeling human behavior. We will separate these claims more clearly.

In the natural-language experiments, three predefined MNL preference vectors generate synthetic reference behavior. We summarize it in 19 moments (15 chosen-attribute means and four intervention responses). The optimizer sees moment discrepancies, not the reference coefficients or choice probabilities. This isolates whether persona edits recover known behavioral patterns. Held-out error falls by 3.2%–42.2% across four seed pairs, but this is evidence of synthetic calibration, not human validity.

Human grounding remains a limitation of the submitted results. On Nutri2Cycle Task 4, direct-human MNL outperforms structured EIPG (share L2: 0.0758 vs. 0.4491). Although both use the same 772 respondents, MNL fits individual choices while EIPG calibrates aggregate moments; the information available to each model differs..

we therefore calibrated three language-based personas on Tasks 1–3 from 618 respondents and evaluated the frozen population on Task 4 from 154 separate respondents. Its share L2 was 0.081 versus 0.101 for direct MNL trained on the same choices, with lower price-response error. This would support the language-based approach against observed behavior, without overturning the weaker structured-generator result. Because Task 4 had already been examined in the submitted work, the analysis remains exploratory.

in addition, calibrating on Tasks 1–2 and transferring unchanged to Tasks 3–4 gave mean share L2 of 0.08 versus 0.11 for MNL fitted on the same source tasks, without new human labels or refitting. MNL is also reusable; this would show a benefit only for the tested bread-price scenarios, not broader agent-level transfer.

We agree that the intervention comparison in Table 15 is confounded: four-intervention calibration beats price-only in only 2/5 independent benchmarks and uses 2,400 rather than 600 reference observations. We re-run the experiment , and at matched totals of 600 and 2,400 observations, keeping anchor and search budgets fixed, four-intervention minus price-only final-test L2 was −0.012 (95% paired CI [−0.021, −0.003]) and −0.010 ([−0.018, −0.002]); it won in 4/5 independent benchmarks at each budget. This would support an intervention-diversity effect beyond observation count, though five benchmarks limit certainty

Similarly, the GEPA comparison also changes two factors at once. We thank the reviewer for raising this issue. Coordinate enumeration searches 30 predefined preference edits; GEPA produces free-text suffixes. Thus the submitted 16.4% versus 5.7% held-out gains do not establish optimizer superiority.

In a new experiment, we constrained GEPA to the same edit space, with identical calibration contexts, simulator, scoring and 3,720 behavioral-choice calls. Structured GEPA achieved a 15.1% mean held-out gain, close to coordinate enumeration's 16.4%, versus 5.7% for free-text GEPA. This suggests the structured representation explains much of the gap, not that enumeration is a generally better optimizer; the 30-move space is particularly amenable to exhaustive search.

We also counted reflection/proposal calls, tokens and end-to-end runtime. At 3,720 choice calls, enumeration took 3.6 hours; structured GEPA took 5.1 hours, including 42 reflection calls and about 420,000 additional tokens. At a matched 3.6-hour runtime budget, their held-out gains were 16.4% and 12.9%, respectively. Simulator tokens and cache hits were also recorded; budget checkpoints used calibration-only selection on identical hardware. We will report the full accounting and performance-versus-budget curves, rather than treating equal choice calls as equal total cost.

Together, these findings would support a narrower claim: economically structured edits enable effective calibration in this low-dimensional setting, and enumeration offers a favorable performance–cost trade-off here. They would not establish general optimizer superiority or human realism across tasks. We will revise the claims accordingly.

Comments on the paper

1 comment

To the Area Chair and reviewers,

As we are almost finishing the discussion period, we would like to clarify what we see as the "central" contribution of EIPG, particularly in light of the concerns raised in the reviews.

our starting point is that a plausible persona description is not a behavioral specification. The same persona prompt can produce different preference patterns across language models, and a synthetic population can fit observed choices while predicting incorrect responses to interventions. This raises a question that we believe is important for (any) LLM-based user simulation: how can we determine whether a population has learned the behavioral relationships needed for credible counterfactual analysis?

EIPG approaches this as a problem of behavioral calibration and identification. Rather than attempting to recover a correct persona description, we calibrate the persona-generating mechanism against interpretable behavioral evidence, including choice patterns and intervention responses.

We believe the experiments contribute two findings in particular. in the first place , better calibration fit does not necessarily imply better counterfactual behavior. Our controlled experiments demonstrate this failure mode and investigate why the information supplied through interventions matters, while increasing the complexity of the intermediate choice model alone does not resolve it. Second, the natural-language experiments show that useful preference corrections depend on the underlying LLM. Even identical starting personas can require different, sometimes opposite, adjustments. This motivates treating persona grounding as specific to the deployed simulator rather than assuming that authored descriptions transfer reliably between models.

We also recognize the limitations emphasized by the reviewers. The natural-language reference behavior is synthetic, the human-data experiment favors direct MNL on predictive accuracy, and some comparative findings require more careful interpretation. We do not claim that EIPG replaces direct estimation from human choices or that we have demonstrated general human behavioral fidelity.

Our intended contribution is therefore not a universally superior predictor, but a framework for constructing and evaluating synthetic populations in terms of the behavioral constraints they satisfy, together with concrete evidence of why baseline fit and plausible persona descriptions can be insufficient.

We appreciate the reviewers' careful examination of these issues :). We hope this clarification helps the Area Chair assess the contribution and its limitations together when forming the final recommendation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.