acceptodds
Under review as a conference paper at ICLR 2027

Human-Like Generalization of Risk Preferences in LLMs

Abstract

The success of real-world interactions between humans and LLM-based chatbots depends on the character, preferences, and values of the LLM's "Assistant'" persona. However, the extent to which isolated measures of LLM behavior—e.g., evaluations using standard instruments from economics, cognitive science, or psychology—are predictive of realistic LLM chatbot behavior is unclear. In this work, we investigate this question of generalizability in the specific domain of *risk preferences*. We measure the risk preferences of 54 LLMs using three kinds of evaluations: surveys, lottery choice questions, and responses to human-written risky dilemmas. The first two evaluation categories are standard in psychology and experimental economics, and the last evaluation category reflects how LLM chatbots are used in practice. We find that survey evaluations predict the degree of riskiness of an LLM's responses to human-written risky dilemmas, while lottery evaluations do not, matching human risk preference generalization behavior. Our risky dilemma evaluation is conducted via a large-scale human experiment, in which participants write risk-related dilemmas from their own life and rate which LLM responses they prefer.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI behavioral science, economic preferences, generalizability, risk aversion, human-AI interaction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Q6nvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper evaluates whether standard behavioral instruments is predictive of realistic LLM behaviors in terms of risk preferences. The authors measure risk preferences of 54 LLMs using surveys, lottery choices questions, and human-written risky dilemmas. They conduct a online human experiment to collect real-life risk dilemmas from human subjects and evaluate risk preferences. They found that LLMs' risk preferences estimated from survey questions are predictive to their risk preferences in real risky dilemmas, while lottery questions do not.

Critical strengths
  1. The paper's most significant strength is the real human experiment, where subjects write real-world risky dilemmas, respond to others' dilemmas, and rate LLM responses with different risk-preference levels.
  2. The paper has a clear finding that LLMs' risk preference in surveys are predictive to it in risky dilemmas, while lottery questions are not.
Critical weaknesses

My main concern is that the generalizability and practical implication of the results remain unclear:

  1. The finding seems to be the averaged performance across 54 LLMs, it does not imply any conclusion for individual LLMs. However, in experiments we typically use a single or a few LLMs.
  2. It is also not applicable to LLMs outside the tested ones. The results may also be sensitive to model configurations, prompt design, harness, etc.
  3. The finding only applies to the off-the-shelf LLMs. However, a growing trend is to use specifically fine-tuned foundations models to serve as human digital twins for social science experiments. It is unclear whether the results would apply to those models.
  4. Lastly the practical implications remain unclear. Lottery choice questions are designed to test specific economics choice axioms, which aims more at understanding and modeling the mechanism behind human behaviors rather than being predictive. It is well know that human's actual behavior deviates from these axioms. Meanwhile, the survey questions shares more similar contexts as the risky dilemmas. Thus, it is not surprising that the lottery questions are not predictive for the real-world dilemmas while the survey questions are.
Questions for authors
  1. What is the individual result for the 54 LLMs? Does any universal pattern arises when scaling up the models, etc.?
  2. What is the sensitivity result given changes in temperature, reasoning levels, etc?
  3. What is the practical implications of the result?
6 comments
AuthorsOpenReview ↗

Response to Q6nv part 1 (incl. prompt and reasoning level robustness)

Thanks for your thoughtful review. Your comments on sensitivity analysis were particularly helpful, and helped us uncover an additional interesting finding. We substantially revised the paper to address your and the other reviewers’ comments. See our top-level OpenReview comment for a complete summary; below we respond directly to the points you raised. Hopefully our response addresses your concerns. If it does, we’d really appreciate your increasing your score to reflect this, considering how you overall seemed to like the paper.

The finding seems to be the averaged performance across 54 LLMs, it does not imply any conclusion for individual LLMs. However, in experiments we typically use a single or a few LLMs. [...] What is the individual result for the 54 LLMs?

Based on this feedback, we added two new sections. First, Section 5.2, which analyzes the risk preference evaluation scores of individual LLMs (also fleshed out further in Appendix E). Second, Section 5.5, which demonstrates concretely how our results on the full set of LLMs can be applied to individual new, held-out LLMs.

Does any universal pattern arises when scaling up the models, etc.?

In Section 5.1, we added an analysis looking at how LLM reliability scores (i.e., risk preference coherence scores) relate to overall LLM capabilities. We find, across all eight of our risk preference evaluations, that more capable LLMs (as measured by the Epoch Capabilities Index, an aggregate capability benchmark) exhibit more coherent risk preferences. This is in line with Mazeika et al. (2025); Fish et al. (2026); Slama et al. (2026), who find similar results in different domains. This suggests that the kinds of measurements we make on risk preferences of LLMs will become more informative as LLM capabilities progress. We also discuss this point more in Section 8.

It is also not applicable to LLMs outside the tested ones. The results may also be sensitive to model configurations, prompt design, harness, etc. [...] What is the sensitivity result given changes in temperature, reasoning levels, etc?

This comment was particularly helpful in helping us improve the paper. Based on it we ran some additional experiments with interesting results. For all 29 LLMs, and each of the four risk preference evaluation types, we re-ran the main experiment on 3 prompt variants and (when available) 3 reasoning levels. Overall, we find that the paper’s main generalizability result (Section 5.3) is robust to changes in the prompt, and for the two higher reasoning levels. (For the lowest reasoning level, the result persists for RatingResponses, but for WritingResponses, the slope is not statistically distinguishable from 0.) We also noticed an interesting pattern in which the strength of the relationship between each dilemma evaluation and DospertSurvey increases with inference-time compute. This suggests that LLMs that employ more reasoning might exhibit more coherent risk preferences. The full details of this experiment are in Section 6.2. (This of course also is closely connected to the analysis we added in Section 5.1, which shows that more capable LLMs also exhibit more coherent risk preferences.)

The finding only applies to the off-the-shelf LLMs. However, a growing trend is to use specifically fine-tuned foundations models to serve as human digital twins for social science experiments. It is unclear whether the results would apply to those models.

Building LLMs that act as human digital twins / “in silico” simulations is also an interesting way that techniques from the social sciences can be used in machine learning applications, but it is not the focus of our paper. Rather, the focus of our paper is to understand the character, preferences, and values of off-the-shelf LLMs’ default “Assistant” personas. We are interested in understanding differences in risk preferences in, say, ChatGPT versus Grok, because these differences impact user experiences with these two consumer chatbots. And indeed, right now, the typical user is more likely to use a standard consumer chatbot than they are to have a customized fine-tuned setup. So in terms of broader impacts of our work, this also motivates our choice to focus on LLMs’ off-the-shelf behavior.

Of course, there is plenty of interesting overlap with the human digital twin / “in silico” simulation literature. Indeed, the work we do in our paper has the potential to be very useful in those settings. For example, we collect a large-scale dataset of human risk preferences that includes risk preferences in human-AI interaction tasks, which has the potential to be very useful for accurate “in silico” simulations of humans interacting with LLM chatbots. We think our dataset (which we will release upon acceptance) has the potential to be very fruitful for follow-up work in this area.

AuthorsOpenReview ↗

Response to Q6nv part 2 (high-level motivation behind the work)

Lottery choice questions are designed to test specific economics choice axioms, which aims more at understanding and modeling the mechanism behind human behaviors rather than being predictive. It is well know that human's actual behavior deviates from these axioms. Meanwhile, the survey questions shares more similar contexts as the risky dilemmas. Thus, it is not surprising that the lottery questions are not predictive for the real-world dilemmas while the survey questions are.

First: One way economists use lottery evaluations is for the obvious application: quantifying how people make decisions about risky versus safe payments. To correct a misconception in your review, lottery evaluations are not limited to measuring whether human behavior conforms to economic choice axioms. Instead, these kinds of evaluations can be, and have been, used to detect all sorts of risk-related behavioral (i.e., not fully “rational”) effects. Two examples include myopic loss aversion (Gneezy and Potters ‘97) and cognitive uncertainty (Enke and Graeber ‘23). Economists have studied lottery evaluations and their variants for decades, because they have been found to be useful for understanding many different subtle aspects of how people make decisions under (financial) risk.

Second: Lab-field generalizability studies in economics have shown that lottery evaluations are predictive of plenty of risk-related behavior outside financial domains. For example, Barsky, Juster, Kimball, and Shapiro ‘97 show that lottery evaluations predict many non-financial risk-related behaviors, such as smoking and drinking. In light of results like these, our main finding -- that lottery evaluations are not predictive of LLMs’ dilemma evaluation scores -- is not a priori obvious.

In summary: our interest in testing lottery evaluations is motivated by them being grounded in decades of research in economics. Their use in economics is far broader than merely testing whether behavior conforms to economic choice axioms such as expected utility. And in fact, lottery evaluations have been shown to be useful in predicting non-financial risk-related behavior in humans. For these reasons, we think they are important to consider in the context of LLM evaluations, and we think our result -- that lottery evaluations are not predictive of dilemma evaluations -- is important to communicate.

What is the practical implications of the result?

We improved the writing of the paper to make the answer to this question clearer. First, the new analysis on held-out LLMs (Section 5.5, also mentioned above) illustrates how one can apply the findings from our work. Second, we fleshed out the actual discussion section (Section 8, see also Appendix B) a bit more. It now both points to novel challenges in LLM behavioral evaluation design (and how they relate to / differ from challenges we face with humans), and, in the last paragraph, gives a more high-level take on the broader implications of our work.

Hi, since the rebuttal phase is nearly over, we wanted to follow up. Does our response address all your concerns? If it does, we’d really appreciate your increasing your score to reflect this.

Reviewer_DsJqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The main theme of the paper is to understand how theories about humans in economics can be applied to LLMs and if LLMs have the behavioral capability operationalized as performance in economic tasks and reliability. The paper claims to have shown the generalization of risk behavior survey results provided by the LLM to the degree of risk present in the LLM response to participant dilemmas. The paper then further claims to have developed the novel risk preference evaluations grounded in human-ai interactions: AI’s preference to give risky advise and participants preference of accepting risky advise from LLMs.

Critical strengths

I think the paper does well in motivating the premise and grounding their methods and evals. Creating theory on LLMs behavior when it’s such a moving target is hard and I like that the authors remain explicit in the limitations and they have checks on if the llms would even qualify to have a theory build. Furthermore, using a new model which has just been released outside of the models tested was a good idea. The pre-registration is also a nice touch and very encouraging to see. The authors also did a case study on human evaluation design vs applying it to llm evaluation which is also a nice addition

Critical weaknesses

I think the paper’s writing does not do a good job in providing details about the two novel risk evaluations developed. When I hear evaluation, I want to see a proper dataset report, metrics and model results; none of these were provided and the only main highlight from the results section seems to be correlation between measures but not the individual reporting of the results on the measures itself.

As mentioned in my strengths, this paper does a lot but that is also it’s weakness. I feel a lot of the evaluation, analysis and even the qualitative analysis was not deep enough. Furthermore, the authors spent a lot of words explaining the exclusion criteria which I felt was unnecessary. To highlight the problem, the main results are smaller than the exclusion critera section. I also felt that the discussion was a bit lacking and did not really try to explain the results or the broader implications.

I think the paper although distinguishes itself from earlier risk-preference studies, but leaves its relationship to prior work on evaluation validity largely unstated[1,2]. There are also works on user simulations which talk about using human theories to ground llm behavior which the paper does not address.

There is also a lack of explanation provided in the lack of significance of the lottery evaluations.

[1] https://aclanthology.org/2024.acl-long.816/ [2] https://arxiv.org/pdf/2412.00207

Questions for authors

(1) the riskiness of the advice the subject gives, and (2) the riskiness of advice the subject prefers. The amount of risk given by the model will depend on the amount of risk preferred by the participant. There are works on confirmation bias in llms that show this phenomenon. So the subject in 1 and 2 should be different. Can the authors defend their choice in choosing both subjects as the LLM and what does it tell us?

5 comments
AuthorsOpenReview ↗

Response to DsJq (part 1: writing fixes)

Thanks for your thoughtful review. Your comments on what parts of the paper were more/less important to you were really helpful when editing the paper to improve presentation. We substantially revised the paper to address your and the other reviewers’ comments. See our top-level OpenReview comment for a complete summary; below we respond directly to the points you raised. Hopefully our response addresses your concerns. If it does, we’d really appreciate your increasing your score to reflect this, considering how you overall seemed to like the paper.

I think the paper’s writing does not do a good job in providing details about the two novel risk evaluations developed. When I hear evaluation, I want to see a proper dataset report, metrics and model results; none of these were provided and the only main highlight from the results section seems to be correlation between measures but not the individual reporting of the results on the measures itself.

This is good feedback, thanks. We improved this aspect of the writing in the rebuttal draft. We added Section 5.2, which describes the individual risk preference score reports of both humans and LLMs, as well as Appendices E and N, which provide a conventional dataset report and some additional analysis.

As mentioned in my strengths, this paper does a lot but that is also it’s weakness. I feel a lot of the evaluation, analysis and even the qualitative analysis was not deep enough. Furthermore, the authors spent a lot of words explaining the exclusion criteria which I felt was unnecessary. To highlight the problem, the main results are smaller than the exclusion criteria section.

This is also helpful writing feedback. In the rebuttal draft, we improved this aspect in a couple of ways:

  • We shortened the discussion of the exclusion criteria to a single paragraph, so that the main body only describes the essential aspects. See the first paragraph of Section 5.1.
  • We used that gained space to report on the results in greater depth. In particular, we added several new sections. The main results section (Section 5) has been expanded: we now report on the individual risk preference scores of humans and LLMs (Section 5.2), the relationship between LLM capabilities and coherence (Section 5.1, second paragraph), and an analysis of the risk preferences of newer held-out LLMs (Section 5.5). And, we added an entire separate robustness section (Section 6), which studies robustness to the choice of LLM judge (Section 6.1), robustness to the choice of reasoning level and prompt (Section 6.2), robustness of the statistical analysis (Section 6.3), robustness to differences in LLMs’ writing styles (Section 6.4), and robustness to differences in risk preferences of the dilemmas’ authors (Section 6.5).

We also made some other minor edits throughout the paper to condense aspects that are not central to the main focus of the paper, and expand aspects that are important.

I also felt that the discussion was a bit lacking and did not really try to explain the results or the broader implications.

We also improved this aspect of the writing. First, the new analysis on held-out LLMs (Section 5.5, also mentioned above) illustrates how one can apply the findings from our work. Second, we fleshed out the actual discussion section (Section 8) a bit more. It now both points to novel challenges in LLM behavioral evaluation design (and how they relate to / differ from challenges we face with humans), and, in the last two paragraphs, gives a more high-level take on the broader implications of our work.

I think the paper although distinguishes itself from earlier risk-preference studies, but leaves its relationship to prior work on evaluation validity largely unstated[1,2]. There are also works on user simulations which talk about using human theories to ground llm behavior which the paper does not address.

Thanks for this feedback. Our original submission had already discussed prior work on evaluation validity (Lines 39-45) and user simulations (Lines 82-85), but in a limited capacity due to space constraints. We’ve expanded this discussion in the rebuttal version (Appendix A), and cited [1] and [2] in the process. We also added a section on how our study relates to the HCI/CASA literature, which may be of interest.

AuthorsOpenReview ↗

Response to DsJq (part 2: incl. robustness to potential confirmation bias confound)

There is also a lack of explanation provided in the lack of significance of the lottery evaluations.

To be honest, from a metascientific perspective, it’s difficult to provide a comprehensive explanation for why a given effect is not present. Still, we added some analysis and discussion in the paper (Section 7, which is expanded out in Appendix C) to give some intuition for this finding. Here’s a high-level summary:

The explanation as to why SmallLotteries is not predictive is simple (Appendix C.2) -- many LLMs interpret the problem as a utility maximization problem, so then the behavior is degenerate. (This is in spite of the prompt being very clear that the question is a matter of personal preference, and has no correct answer.) We are able to eliminate this effect with BigLotteries, which uses the exact same prompt, but has larger stakes for the lotteries.

However, of course, still, BigLotteries does not correlate with the other risk preference evaluation scores. Appendix C.3, and also Figure 10, give a bit of a sense why. Some LLMs’ preferences are stable between BigLotteries and DospertSurvey -- for example Grok models and pre-Gemini-3 Google models. But then there is this marked pattern where the Claude models consistently have high DospertSurvey scores, but low BigLotteries scores. When we look at the LLMs’ stated explanations, this seems to be due to Claude models having high risk tolerance in the “openness to experience” sense, but low risk tolerance in the sense of risky gambles.

This suggests a lot of interesting follow-up analyses one could consider -- for example, decomposing the general domain of risk preferences into many specific sub-components, and measuring their correlation structure. For example: the original authors of DOSPERT consider six subcategories of risk preferences. We had the thought to break our analysis down by these categories, but were unable to do so because our human-written dilemma dataset is unbalanced -- the vast majority of the dilemmas are social. So, we think there is a lot of interesting follow-up work one could conduct in this space, but it would require collecting an entirely separate paper’s worth of additional human data.

(1) the riskiness of the advice the subject gives, and (2) the riskiness of advice the subject prefers. The amount of risk given by the model will depend on the amount of risk preferred by the participant. There are works on confirmation bias in llms that show this phenomenon. So the subject in 1 and 2 should be different. Can the authors defend their choice in choosing both subjects as the LLM and what does it tell us?

This is a great point, thanks for raising it. We added a new regression analysis (Section 6.5) that checks for this. Specifically, we run a regression that predicts whether the LLM gives overall risky (1) or safe (0) advice, from two quantities: first, the LLM’s DospertSurvey score, and second, the dilemma author’s DospertSurvey score. So we are estimating two coefficients, which we call $\beta_{LLM}$ and $\beta_{user}$. If $\beta_{LLM}$ is positive, it means that the LLM’s risk preferences (as measured by DospertSurvey) predict the riskiness of the advice it gives. If $\beta_{user}$ is positive, it means that the user’s risk preferences (as measured by DospertSurvey) predict the riskiness of the advice the LLM gives. That is, $\beta_{LLM}$ is our main quantity of interest, and (closely related to) the thing we measure in the main analysis in Section 5.3, and $\beta_{user}$ detects for whether confirmation bias influences the results. When we run the regression, we find that $\beta_{LLM}$ is positive (p < 0.05), while $\beta_{user}$ is not statistically distinguishable from 0. That is, we do not detect any influence of confirmation bias on the riskiness of the LLMs’ dilemma responses.

Hi, since the rebuttal phase is nearly over, we wanted to follow up. Does our response address all your concerns? If it does, we’d really appreciate your increasing your score to reflect this.

Reviewer_dLyeOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether standard risk-preference evaluations predict LLM behavior on realistic advice-seeking dilemmas. It evaluated 54 LLMs (29 retained after filtering), and conducted two pre-registered human experiments. The authors conclude that survey scores predict LLM advise riskiness and response preferences, while lottery scores do not. The study contributes a nice dataset with 500+ human-written dilemmas and useful evidence about cross-instrument validity.

Critical strengths
  • The pre-registered human experiments provide a valuable comparison and dataset for future use.
  • The real dilemmas provide more nuanced evaluation for AI behavior than survey and game tasks.
Critical weaknesses
  • While the study provides useful results and advance understanding of LLM's behavior, it is less clear what's the takeaway for the machine learning community. It may be a better fit for a behavioral science journal or a more applied AI conference.
  • Preference between two models' responses may confound risk tolerance with the language style and quality, which needs to be properly controlled.
Questions for authors

Reference measurements for Grok vs. GPT responses differ along dimensions beyond risk. Can you isolate risk preference from the style & language quality of the responses? You may control these factors by matching responses for style, helpfulness, and language quality, or replicate this result with multiple model pairs that reflects diverse response styles and quality.

5 comments
AuthorsOpenReview ↗

Response to dLye (incl. robustness to style/quality experiments)

Thanks for your thoughtful review. We substantially revised the paper to address your and the other reviewers’ comments. See our top-level OpenReview comment for a complete summary; below we respond directly to the points you raised. Hopefully our response addresses your concerns. If it does, we’d really appreciate your increasing your score to reflect this, considering how you overall seemed to like the paper.

While the study provides useful results and advance understanding of LLM's behavior, it is less clear what's the takeaway for the machine learning community. It may be a better fit for a behavioral science journal or a more applied AI conference.

We agree the work reads as social-science-y, because of its interdisciplinary nature. We revised the technical sections of the paper, and added new content (Sections 5.5 and 6), to make its machine learning contribution clearer and stronger. Briefly: the purpose of this paper is to provide a blueprint for using and adapting instruments from the social sciences to develop practical risk preference evaluations for LLMs. That is, we are applying tools from the social sciences to problems that are of interest to the machine learning community (and specifically, the community conducting behavioral evaluations of LLMs).

To illustrate how our work can be of interest to the machine learning community, some closely related ICLR/ICML/NeurIPS papers are listed below. [1-4] use techniques from economics, psychology, and cognitive science to evaluate LLMs (and [5] is a position paper arguing for this approach). [6-7] study coherence of LLM preferences. Our paper contributes to both areas. (Of course, it goes without saying that there are many more applicable papers, both from ICLR/ICML/NeurIPS and outside these conferences -- this of course is just a subset for demonstration.)

Preference between two models' responses may confound risk tolerance with the language style and quality, which needs to be properly controlled. [...] Reference measurements for Grok vs. GPT responses differ along dimensions beyond risk. Can you isolate risk preference from the style & language quality of the responses? You may control these factors by matching responses for style, helpfulness, and language quality, or replicate this result with multiple model pairs that reflects diverse response styles and quality.

This is a good point, thanks for raising it. We added a comprehensive analysis in Section 6.4 that should address this concern, both for the WritingResponses and RatingResponses cases.

For WritingResponses, we consider an alternate scoring method “modulo style”. For each dilemma, we use an LLM judge to write two contrasting advice labels, e.g. “quit your job” and “keep your job'”, and label which is riskier vs. safer. Then, we use an LLM judge to assign each response to one of these labels, based on which label the advice most closely falls under. The WritingResponses scores modulo style are then computed using a Bradley-Terry fit using the same process as the standard WritingResponses scores, except using these binary response risk labels instead of LLM judge labels. This yields a scoring process that controls for style, at the cost of being coarser. What we find is that these WritingResponses scores modulo style are very similar to the original WritingResponses scores (Spearman $\rho$ is 0.81 on human data, $\ge 0.89$ on LLM data). That is, the scores are robust when controlling for writing style.

For RatingResponses, we use 5 LLMs from different providers—Grok 4.20, GPT 5.6 Terra, Claude Opus 5, Gemini 3.6 Flash and Kimi K2.7-Code—to rewrite dilemma responses in their own voice, preserving the content. We then recompute separate RatingResponses scores for each (ordered) pair of rewriter LLMs. What we find is that these RatingResponses scores using rewritten responses are very similar to the original RatingResponses scores (Spearman $\rho$ is at least $0.92$ across all 25 pairs of rewriter LLMs). That is, again, the scores are robust when controlling for writing style.

Here are the papers referenced in our above response:

[1] Erik Jones and Jacob Steinhardt. Capturing Failures of Large Language Models via Human Cognitive Biases. NeurIPS 2022.
[2] Nino Scherrer, Claudia Shi, Amir Feder, and David M. Blei. Evaluating the Moral Beliefs Encoded in LLMs. NeurIPS 2023.
[3] Ryan Liu, Theodore R. Sumers, Ishita Dasgupta, and Thomas L. Griffiths. How Do Large Language Models Navigate Conflicts between Honesty and Helpfulness? ICML 2024.
[4] Hadi Hosseini and Samarth Khanna. Distributive Fairness in Large Language Models: Evaluating Alignment with Human Values. NeurIPS 2025.
[5] Hanna Wallach, Meera Desai, A. Feder Cooper, et al. Position: Evaluating Generative AI Systems Is a Social Science Measurement Challenge. ICML 2025.
[6] Mantas Mazeika, Xuwang Yin, Rishub Tamirisa, Jaehyuk Lim, Bruce W. Lee, Richard Ren, Long Phan, Norman Mu, Oliver Zhang, and Dan Hendrycks. Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs. NeurIPS 2025.
[7] Naama Rozen, Liat Bezalel, Gal Elidan, Amir Globerson, and Ella Daniel. Do LLMs Have Consistent Values? ICLR 2025.

Hi, since the rebuttal phase is nearly over, we wanted to follow up. Does our response address all your concerns? If it does, we’d really appreciate your increasing your score to reflect this.

Comments on the paper

1 comment
AuthorsOpenReview ↗

High-level Rebuttal Summary

Thanks to the reviewers for their thoughtful reviews. Considering how coarse the overall ICLR scoring system is this year, we were encouraged to see consistently high subscores in presentation (332), soundness (322), and informativeness (222). We were also happy to see that reviewers saw the value in our approach (e.g., dLye/DsJq praise our evaluation design) and execution (e.g., all three reviewers praise our preregistered human experiments).

Based on the reviewers’ comments, we substantially improved the paper in a few ways:

Robustness section. We added an entirely new section (Section 6) dedicated to robustness checks suggested by the reviewers. Section 6.1 checks robustness to the choice of LLM judge, Section 6.2 checks robustness to the choice of reasoning level and prompt, Section 6.3 checks robustness of the statistical analysis, Section 6.4 checks robustness to differences in LLMs’ writing styles, and Section 6.5 checks whether confirmation bias is a confound. These sections (which involved extensive additional experiments) help flesh out the main results.

Held-out LLM analysis. We added a new subsection (Section 5.5) evaluating the risk preferences of 7 LLMs released after the ICLR 2027 submission deadline. The idea of this analysis is to demonstrate the practical implications of our results. Because we show that survey evaluations are predictive of dilemma evaluations on our original set of 29 LLMs, when we encounter a new LLM, we can get some signal on its performance on the dilemma evaluations (which, because they reflect realistic LLM-user interactions, are comparatively expensive) by first looking at its scores on a survey evaluation (comparatively cheap). Thinking at scale, one could imagine auditing a new LLM using hundreds of (cheap) surveys, and then based on these initial results, targeting more effectively which LLM-user interaction evaluations to develop and run.

Fleshed-out analysis of individual LLMs’ risk preferences and coherence. We added new subsections (Sections 5.1 and 5.2) to report humans’ and LLMs’ individual risk preference and coherence scores in a more fleshed-out way. Section 5.2 analyzes how the risk preferences of the LLM vs. human populations compare. Section 5.1 looks at the coherence of LLMs’ risk preferences, and finds that more capable LLMs exhibit more coherent risk preferences, in line with the findings of Mazeika et al. (2025), Fish et al. (2026), and Slama et al. (2026).

Writing and presentation improvements. Our scores for presentation (332) were already quite high. Still, based on helpful reviewer feedback, we made some minor fixes to the prose to improve clarity and structure. (In particular, condensing less interesting parts such as the exclusion criteria, and expanding more interesting parts such as the results.)

This was a substantial undertaking, but we believe with these fixes the paper is much stronger. We’ve left detailed responses for the individual reviewers explaining the specific changes we made to address their concerns. Hopefully, with these changes, the paper meets the bar for ICLR.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.