- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper evaluates whether standard behavioral instruments is predictive of realistic LLM behaviors in terms of risk preferences. The authors measure risk preferences of 54 LLMs using surveys, lottery choices questions, and human-written risky dilemmas. They conduct a online human experiment to collect real-life risk dilemmas from human subjects and evaluate risk preferences. They found that LLMs' risk preferences estimated from survey questions are predictive to their risk preferences in real risky dilemmas, while lottery questions do not.
Critical strengths
- The paper's most significant strength is the real human experiment, where subjects write real-world risky dilemmas, respond to others' dilemmas, and rate LLM responses with different risk-preference levels.
- The paper has a clear finding that LLMs' risk preference in surveys are predictive to it in risky dilemmas, while lottery questions are not.
Critical weaknesses
My main concern is that the generalizability and practical implication of the results remain unclear:
- The finding seems to be the averaged performance across 54 LLMs, it does not imply any conclusion for individual LLMs. However, in experiments we typically use a single or a few LLMs.
- It is also not applicable to LLMs outside the tested ones. The results may also be sensitive to model configurations, prompt design, harness, etc.
- The finding only applies to the off-the-shelf LLMs. However, a growing trend is to use specifically fine-tuned foundations models to serve as human digital twins for social science experiments. It is unclear whether the results would apply to those models.
- Lastly the practical implications remain unclear. Lottery choice questions are designed to test specific economics choice axioms, which aims more at understanding and modeling the mechanism behind human behaviors rather than being predictive. It is well know that human's actual behavior deviates from these axioms. Meanwhile, the survey questions shares more similar contexts as the risky dilemmas. Thus, it is not surprising that the lottery questions are not predictive for the real-world dilemmas while the survey questions are.
Questions for authors
- What is the individual result for the 54 LLMs? Does any universal pattern arises when scaling up the models, etc.?
- What is the sensitivity result given changes in temperature, reasoning levels, etc?
- What is the practical implications of the result?
6 comments
Response to Q6nv part 2 (high-level motivation behind the work)
Lottery choice questions are designed to test specific economics choice axioms, which aims more at understanding and modeling the mechanism behind human behaviors rather than being predictive. It is well know that human's actual behavior deviates from these axioms. Meanwhile, the survey questions shares more similar contexts as the risky dilemmas. Thus, it is not surprising that the lottery questions are not predictive for the real-world dilemmas while the survey questions are.
First: One way economists use lottery evaluations is for the obvious application: quantifying how people make decisions about risky versus safe payments. To correct a misconception in your review, lottery evaluations are not limited to measuring whether human behavior conforms to economic choice axioms. Instead, these kinds of evaluations can be, and have been, used to detect all sorts of risk-related behavioral (i.e., not fully “rational”) effects. Two examples include myopic loss aversion (Gneezy and Potters ‘97) and cognitive uncertainty (Enke and Graeber ‘23). Economists have studied lottery evaluations and their variants for decades, because they have been found to be useful for understanding many different subtle aspects of how people make decisions under (financial) risk.
Second: Lab-field generalizability studies in economics have shown that lottery evaluations are predictive of plenty of risk-related behavior outside financial domains. For example, Barsky, Juster, Kimball, and Shapiro ‘97 show that lottery evaluations predict many non-financial risk-related behaviors, such as smoking and drinking. In light of results like these, our main finding -- that lottery evaluations are not predictive of LLMs’ dilemma evaluation scores -- is not a priori obvious.
In summary: our interest in testing lottery evaluations is motivated by them being grounded in decades of research in economics. Their use in economics is far broader than merely testing whether behavior conforms to economic choice axioms such as expected utility. And in fact, lottery evaluations have been shown to be useful in predicting non-financial risk-related behavior in humans. For these reasons, we think they are important to consider in the context of LLM evaluations, and we think our result -- that lottery evaluations are not predictive of dilemma evaluations -- is important to communicate.
What is the practical implications of the result?
We improved the writing of the paper to make the answer to this question clearer. First, the new analysis on held-out LLMs (Section 5.5, also mentioned above) illustrates how one can apply the findings from our work. Second, we fleshed out the actual discussion section (Section 8, see also Appendix B) a bit more. It now both points to novel challenges in LLM behavioral evaluation design (and how they relate to / differ from challenges we face with humans), and, in the last paragraph, gives a more high-level take on the broader implications of our work.
Hi, since the rebuttal phase is nearly over, we wanted to follow up. Does our response address all your concerns? If it does, we’d really appreciate your increasing your score to reflect this.
Response to Q6nv part 1 (incl. prompt and reasoning level robustness)
Thanks for your thoughtful review. Your comments on sensitivity analysis were particularly helpful, and helped us uncover an additional interesting finding. We substantially revised the paper to address your and the other reviewers’ comments. See our top-level OpenReview comment for a complete summary; below we respond directly to the points you raised. Hopefully our response addresses your concerns. If it does, we’d really appreciate your increasing your score to reflect this, considering how you overall seemed to like the paper.
Based on this feedback, we added two new sections. First, Section 5.2, which analyzes the risk preference evaluation scores of individual LLMs (also fleshed out further in Appendix E). Second, Section 5.5, which demonstrates concretely how our results on the full set of LLMs can be applied to individual new, held-out LLMs.
In Section 5.1, we added an analysis looking at how LLM reliability scores (i.e., risk preference coherence scores) relate to overall LLM capabilities. We find, across all eight of our risk preference evaluations, that more capable LLMs (as measured by the Epoch Capabilities Index, an aggregate capability benchmark) exhibit more coherent risk preferences. This is in line with Mazeika et al. (2025); Fish et al. (2026); Slama et al. (2026), who find similar results in different domains. This suggests that the kinds of measurements we make on risk preferences of LLMs will become more informative as LLM capabilities progress. We also discuss this point more in Section 8.
This comment was particularly helpful in helping us improve the paper. Based on it we ran some additional experiments with interesting results. For all 29 LLMs, and each of the four risk preference evaluation types, we re-ran the main experiment on 3 prompt variants and (when available) 3 reasoning levels. Overall, we find that the paper’s main generalizability result (Section 5.3) is robust to changes in the prompt, and for the two higher reasoning levels. (For the lowest reasoning level, the result persists for RatingResponses, but for WritingResponses, the slope is not statistically distinguishable from 0.) We also noticed an interesting pattern in which the strength of the relationship between each dilemma evaluation and DospertSurvey increases with inference-time compute. This suggests that LLMs that employ more reasoning might exhibit more coherent risk preferences. The full details of this experiment are in Section 6.2. (This of course also is closely connected to the analysis we added in Section 5.1, which shows that more capable LLMs also exhibit more coherent risk preferences.)
Building LLMs that act as human digital twins / “in silico” simulations is also an interesting way that techniques from the social sciences can be used in machine learning applications, but it is not the focus of our paper. Rather, the focus of our paper is to understand the character, preferences, and values of off-the-shelf LLMs’ default “Assistant” personas. We are interested in understanding differences in risk preferences in, say, ChatGPT versus Grok, because these differences impact user experiences with these two consumer chatbots. And indeed, right now, the typical user is more likely to use a standard consumer chatbot than they are to have a customized fine-tuned setup. So in terms of broader impacts of our work, this also motivates our choice to focus on LLMs’ off-the-shelf behavior.
Of course, there is plenty of interesting overlap with the human digital twin / “in silico” simulation literature. Indeed, the work we do in our paper has the potential to be very useful in those settings. For example, we collect a large-scale dataset of human risk preferences that includes risk preferences in human-AI interaction tasks, which has the potential to be very useful for accurate “in silico” simulations of humans interacting with LLM chatbots. We think our dataset (which we will release upon acceptance) has the potential to be very fruitful for follow-up work in this area.