acceptodds
Under review as a conference paper at ICLR 2027

Who Is at Risk? Reading LLM Decision Biases into Evacuation Simulations

Abstract

In a tsunami, who survives depends on when and how each resident decides to leave. Language models now offer a way to supply these decision traits, such as how strongly a resident relies on an official warning, to every resident of an evacuation simulation, and such simulations are usually judged by whether the death toll looks plausible. This test cannot tell whether each resident received the right traits: when residents are placed at random, shuffling their traits leaves the expected death toll unchanged but changes who dies. We therefore propose checks on who holds which trait: whether shuffling changes who dies, whether the groups that fare worst match evidence from real evacuations, whether a direction read from the model does better than random directions, and whether different models agree. The checks need no data on individual behavior, and they catch errors we inject on purpose. We read six decision biases from the hidden states of four open models into two established tsunami evacuation simulators. Shuffling the scores barely changes the death toll but changes about half of the victims, and sources with similar death tolls differ on the checks. Warning reliance is consistent across models while herd influence is not, and all four models rate older residents as the most warning-reliant, the opposite of 2011 mortality. A plausible death toll, and even agreement between models, can hide errors in who is put at risk.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
agent-based modeling (ABM), disaster evacuation, LLM hidden states

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_TxGtOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
2(+1)
Soundness
1
Read review
Paper summary

The paper proposes a resident-level evaluation protocol for evaluating LLM-derived behavioral trait assignments in agent-based evacuation simulations. It shows that simulations can predict similar aggregate outcomes (total death count) even when assigning risk to different residents, and that the proposed checks can detect author-injected score corruptions.

Critical strengths
  1. The paper provides good observations that a simulation may look promising when looking at the aggregate outcomes alone (e.g., total death count), but the simulation may not be accurate at the individual resident level. The motivation is intuitive: inaccurate or biased risk assignments could lead to suboptimal/bias/unfair planning decisions, although the manuscript does not clearly establish the connection to real-world consequences.

  2. The paper reiterates some insightful findings in LLM research and demonstrates their relevance to Evacuation agent-based models. For example, the paper suggests that cross-model agreement does not necessarily mean validity, and direct prompting can be highly sensitive to wording.

Critical weaknesses
  1. The experiments evaluate one injected failure error at a time, with each error closely aligned with one or more proposed checks. While this is useful for unit testing, it may not be robust to real-world scenarios where multiple errors can coexist. Evaluating on combined errors or independently designed held-out errors would help demonstrate the robustness of the proposed protocol.

  2. Although the exact metrics and decision rules appear to be introduced in this paper, the individual checks mostly build upon ideas from areas such as ABM validation and simulation sensitivity analysis. The contribution seems to be primarily on combining these checks, rather than introducing new validation principles.

Questions for authors
  1. Would the same protocol apply if the resident scores are produced by other methods, e.g., a regression model, survey imputation, or handwritten rules? If so, what aspects of the protocol (e.g., C3) are specifically designed to address potential failures introduced by LLM-derived scores?

  2. The manuscript states that the proposed checks "establish consistency with evidence rather than individual accuracy," but also claims that they identify traits that can be “trusted”. Can the authors elaborate more on the level of validity supported by the checks, and what “trusted” means? Does passing all checks only indicate that the selected evidence and diagnostics found no inconsistency, or is it used to justify using the resulting resident-level assignments?

  3. When one or more checks fail, what are the implications, and how should practitioners determine what component to revise, e.g., the LLM-derived scores, the profile-to-score mapping, the score-to-behavior channel, or the underlying simulator? Can failure patterns across multiple checks reliably localize the problem, or do they only indicate that the end-to-end pipeline is inconsistent with the available evidence?

  4. The manuscript describes warning reliance as a trait that can be “trusted,” yet no score source passes all four C2 patterns under the stated rule. Can the authors clarify the basis for this conclusion? Are some checks considered more informative/important than others? How should one interpret potentially conflicting results across C1–C5 when determining whether a trait is trustworthy?

Reviewer_dewTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper aims to study whether the LLM-based simulation can assign risk to a certain resident group. authors extract the six behavioral scores from the LLM's hidden states and discover that shuffling these scores can lead to distinct individual outcomes. This paper also leverages controlled score corruptions to further verify the method.

Critical strengths
  1. This paper shows that evacuation planning depends on identifying the people and the locations that need help. Further experiments also demonstrate that considering only total fatalities can ignore differences in risk assignment.

  2. The authors conduct thorough experiments across different models, simulators, people, etc. The authors not only demonstrate the differences between the reported models but also verify that the method can detect certain errors.

Critical weaknesses
  1. This paper does not position its contribution clearly. This paper seems to use existing persona simulation and validation methods directly in tsunami models, without clearly explaining what new methodological contribution this provides. Besides, this paper also needs to clarify its unique contribution beyond applying existing methods to a certain domain.

  2. The usage and benefit of the LLM are not well justified. This paper needs to compare the LLM-based method with traditional methods to establish what benefit the LLM provides here. Similarly, the errors from the LLM are not well located. For example, the age mismatch may come from missing mobility constraints instead of the LLM's judgment. The authors need to clearly illustrate what benefit the LLM provides and where the LLM fails.

  3. The model selection may limit the conclusions. The experiments are mostly conducted with small LLMs in the 3B–7B range. It is unclear whether the observed errors would persist with larger or more capable models (at least 30 B), which limits the generalizability of the findings.

Questions for authors
  1. What can the proposed protocol diagnose that existing simulation-validation methods cannot?

  2. How can a failed check distinguish an inaccurate LLM score from a missing or incorrect simulator mechanism?

3 comments

Thanks for the substantial revision. I think the new mobility and robustness analyses largely address my concern about whether the age-related failure could instead come from a missing simulator mechanism. My main concern, however, is still the role and necessity of the LLM component. The revision makes the positioning clearer, but I still see the protocol mainly as a combination of existing validation tools applied to a new evaluation target. Similarly, saying that LLMs provide broader coverage does not fully answer why the LLM component is needed or preferable to simpler alternatives.

Model scale is not my main concern, but I still think a larger-model comparison would be useful. The cited earlier work does not show that the same failures would necessarily persist for current larger models, especially since recent results suggest that scaling effects in social/persona simulation are mixed and task-dependent. Ziems et al. (2026) find substantial gains from scaling on many social-simulation tasks, while Lutz et al. (2025) show that larger models are not consistently better for persona simulation. Overall, the revision strengthens the paper, but it does not change my main concerns about the necessity of the LLM component and the generalizability of the findings across model scales, so I plan to keep my score unchanged.

References

Ziems, Caleb, et al. “Will Scaling Improve Social Simulation with LLMs?” arXiv preprint arXiv:2607.02464, 2026.

Lutz, Marlene, et al. “The Prompt Makes the Person(a): A Systematic Evaluation of Sociodemographic Persona Prompting for Large Language Models.” Findings of EMNLP, 2025.

Thank you for engaging with the revision and for confirming that the mobility and robustness analyses address the attribution concern.

On the necessity of the LLM component. The paper does not claim that LLMs are necessary or better than other sources; §1 states that “we do not claim that LLM scores improve the simulation.” LLMs are already being used to supply behavior in evacuation simulations (Yang et al., 2026; Li et al., 2025c). The coverage they offer (§7) is the reason to use them, and it is exactly what needs checking: coverage helps only if the right traits reach the right residents. Our main finding is partly negative for LLMs. All four models rate residents over 60 as the most warning-reliant, against 2011 mortality and survey data, and the scores’ contribution to this age gap stays negative after a mobility mechanism is added (read − shuffled: −4.6 pp [−6.2, −3.0]). The protocol’s value does not depend on the LLM either. App. L runs the same checks on a warning-reliance rule built only from published effect directions, with no LLM input. It gives a similar death toll (14.0% vs. 15.5%), yet 62.6% of its victims are different residents, so the assignment problem is not specific to LLMs.

On combining existing tools. We agree that the individual ingredients have precedents (§2). What we add is (i) Eq. 1, which shows that aggregate fit cannot identify the assignment when residents are placed at random, and therefore why standard ABM validation does not catch these errors, and (ii) checks targeted at that assignment, including C3 for hidden-state sources, with measured detection and false-alarm rates (App. J).

On model scale. All claims are scoped to the 3–7B models tested; since the protocol treats each model as one more source, a larger model runs through it unchanged.

[1] When agents learn to think: Large language model-enhanced agent-based modeling for crowd evacuation in disaster scenarios. Reliability Engineering & System Safety, 269:112056, 2026. [2] What makes LLM agent simulations useful for policy practice? An iterative design study in emergency preparedness.

Thank you for the response. While I appreciate the additional analyses, they do not change my overall assessment. I will therefore maintain my current rating.

Reviewer_PH4hOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper focuses on how LLM decision-making can be biased in evacuation simulation using the case of a tsunami. The use of contrastive differences in LLMs' hidden state direction for each resident factor. They apply various existing hypotheses that are widely recognized as true to test whether, when LLMs are simulating, the results of different runs align with the hypotheses. The experiments show that LLM scoring has positive contribution on the simulation but still have room to improve. For example, reshuffling scores doesn't impact total deaths but changes the victims, and the simulated impact of some axes doesn't align with existing hypotheses.

Critical strengths
  • The simulation design is novel and solid.
  • The presentation is clear.
  • It is novel to go deeper than population-level statistics to look at the individual. The conclusion is important and may have real-world impact.
  • The evaluation protocol is reasonable.
Critical weaknesses
  • The evaluation protocol covers some aspects but doesn't ensure it is comprehensive enough to discover all weaknesses or possible errors in the system.
  • The evaluation protocol is more focused on diagnosis but has a gap in validating predictions.
  • The transfer from standardized scores to behavior channels (Table 1) is hand-crafted. Lack of reference for why design in such way and where the parameters are from.
  • Some assumptions in the method design can be too strong. For example, equation 1 assumes profiles and locations are independent, but this may not be the real-world case.
Questions for authors
  • For each feature, we use PC1 as the direction. But actually, the hidden assumption here is that the features should be represented equivalently by 1 direction. What if some features are more complex to model and cannot be represented by a single direction?
  • This is an open question: for those cases where a feature from the LLM is not aligned with existing hypotheses in simulation, can we know if the LLM is having different but consistent reasoning on the opposite side, or if it is simply not aware of it? Some mechanistic interpretability methods may be useful.
  • This is an open question: for the channels 1 design, can we also use a learned method like regression for a function and parameter?
2 comments
AuthorsOpenReview ↗

Kind reminder

Dear Reviewer PH4h,

Thank you again for your positive and constructive review. Your suggestions on the channel constants and on involving evacuation-policy experts have both made their way into the revised paper. As the discussion period nears its end, we would be grateful if you could let us know whether our responses address your concerns. We appreciate your time.

The authors

Reviewer_PH4hOpenReview ↗

Thank you

Thank you for the response! I appreciate the additional information and maintain my positive assessment (3). I encourage the AC/SAC to consider accepting this paper.

Comments on the paper

5 comments

Thank you for the detailed response and robustness experiments. The new experiments demonstrate that C3 is particularly effective for hidden-state-derived scores, which is the paper’s primary setting. Could the authors clarify whether the intended contribution is specifically a protocol for hidden-state-parameterized simulations, or a broader framework covering prompted and non-LLM sources? More importantly, is there any relevant work suggesting that hidden-state-derived resident traits are an important or emerging input to practical evacuation simulations? These clarifications would help establish whether the protocol addresses a likely practical need or primarily evaluates a technically plausible but currently hypothetical pipeline.

Thank you for clarifying that this is an emerging, currently unpublished application. I encourage the authors to make this scope explicit in the manuscript. Putting that aside, my remaining concerns are:

  • C3 could help screen hidden-state directions before using them as traits, but it is specific to this narrow interface. The other checks are more general but largely adapt existing validation ideas.
  • Although evacuation provides a life-critical motivation, passing these checks indicates only that no tested inconsistency was found, not that the assignments are accurate or safe for planning.

Overall, I view this work as a potentially useful starting point and screening procedure for an emerging interface. However, given the limited scope and unclear significance of the current contribution, I will maintain my score for now. That said, I am less familiar with how methodological contributions are typically assessed in the ABM community, and I remain open to revising my score if the discussion or other reviewers’ expertise indicates that this constitutes a substantial contribution in that area.

Thank you for your openness to the ABM perspective. We will make the scope explicit in the paper. On the remaining concerns:

Building from existing ideas is how ABM methods usually advance. Two of the most widely adopted contributions in the field, the ODD protocol for describing models (Grimm et al., 2006) and the TRACE framework for documenting model testing (Grimm et al., 2014), introduced no new principles. Their contribution was to turn known ideas into a standard that modelers can follow, and they are now standard practice. We do this for a problem that had not been addressed: whether each agent gets the right traits.

Passing is not accuracy, we agree. No check can prove a simulation right, only show where it is wrong (Oreskes et al., 1994). What matters is what the checks catch: here, all four models treat people over 60 as among the safest, when in 2011 they died at the highest rates, and the death toll hid it. A screen that catches this before a plan is made is what the field lacks.

References

Grimm, V., et al. A standard protocol for describing individual-based and agent-based models. Ecological Modelling, 198(1–2), 115–126, 2006. doi:10.1016/j.ecolmodel.2006.04.023

Grimm, V., et al. Towards better modelling and decision support: Documenting model development, testing, and analysis using TRACE. Ecological Modelling, 280, 129–139, 2014. doi:10.1016/j.ecolmodel.2014.01.018

Oreskes, N., Shrader-Frechette, K., Belitz, K. Verification, validation, and confirmation of numerical models in the earth sciences. Science, 263(5147), 641–646, 1994. doi:10.1126/science.263.5147.641

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.