- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies the consistency of preference probability distributions generated by LLMs. The authors use nine open-weight models from three families (Llama, Qwen, and Mistral) and examine choices related to air travel, restaurants, and consumer products. Instead of asking LLMs to make one choice, they ask the models to generate explicit probability distributions over a list of alternatives. The paper then studies how these distributions change across repeated calls, different models, and prompt variations.
The main finding is that the same model tends to produce relatively consistent distributions across repeated calls, especially for its top choices. However, different models produce quite different distributions, even when belonging to the same model family. For example, the top-five choices of Llama 70B and 405B have little overlap, and transferring a top-five shortlist between these two models reduces its probability coverage by about 17.7 percentage points, evaluated against the target model's own outputs. The paper also shows that prompt wording and the processing of generated responses can substantially affect the resulting distributions.
Critical strengths
I like that the paper goes beyond simply showing LLM outputs are sensitive to prompts. It asks whether changing models changes an entire probability distribution, and how much this matters for reusing model-generated choices. The 17.7 percentage point difference in shortlist coverage is a useful way to quantify model disagreement, although the coverage is evaluated against the LLM's own distribution rather than actual human choices. This is relevant for applications where simulated distributions are used repeatedly.
In addition, I appreciate the effort in auditing the outputs and distinguishing between matched and unmatched probabilities. For example, the paper shows that only around 69.8% of Yelp and 65.1% of Amazon probability mass is retained under the respective matching rules. The authors also recognize that some stabilization patterns arise mechanically from averaging repeated calls, and they use a stationary random baseline to check this. These are useful methodological details that are often overlooked in LLM simulation studies.
Critical weaknesses
My biggest concerns is that I am not convinced that the paper is measuring preference distributions in an economically meaningful sense.
The paper asks LLMs to provide probability distributions over 300 alternatives, but what exactly is this probability supposed to represent? For example, the travel prompt asks about travel plans over the next 12 months, and the restaurant and product prompts ask about visiting or purchasing items in the next month. These are not necessarily mutually exclusive events. A consumer may visit multiple restaurants, buy multiple products, or travel to multiple destinations. However, the prompts force the probabilities across all alternatives to sum to one. This is not the same as either marginal purchase probabilities or a standard discrete-choice probability distribution.
There is also very little information about the actual choice environment. For travel, there are no ticket prices, budgets, trip purposes, or other constraints. For restaurants, distance, price and quality information are largely absent from the prompt. The generated distributions may therefore reflect what destinations or product names are more salient in the model's training data, rather than actual consumer preferences.
More importantly, repeated calls with the same demographic persona do not generate a sample of different consumers. They are repeated numerical responses from the same model under the same instruction. Averaging these responses may reduce generation noise, but it does not necessarily recover preference heterogeneity in a population.
I understand the authors explicitly state that they do not measure human-choice accuracy. But I think the issue here is more fundamental. Even before comparing with real human data, it is unclear what behavioral object the generated probability vector represents.
Without a clearer definition or validation, I find it difficult to interpret the results as evidence about the reliability of LLM-generated preferences, rather than the reproducibility of numbers written by LLMs.
Questions for authors
- Clarification regarding the meaning of preference probability.
What is the precise interpretation of the probability assigned to each alternative? Is it the probability of selecting one alternative from the entire choice set, or the probability of visiting/purchasing that alternative within a period? If it is the latter, why should probabilities sum to one when multiple events can occur? This clarification is important for interpreting the main results.
- Evidence based on a more meaningful economic choice task.
Would the major cross-model findings remain if the prompt defined one mutually exclusive decision, such as the destination of the next single trip, or the restaurant for one particular dining occasion, with an outside option? I think this would help establish whether the results reflect differences in preference predictions or simply differences in how models generate numerical lists. Even a relatively small controlled experiment would be informative.
2 comments
Response to Reviewer MHyE (2/2)
Weaknesses, paragraph 6: Without a clearer definition or validation, I find it difficult to interpret the results as evidence about the reliability of LLM-generated preferences, rather than the reproducibility of numbers written by LLMs.
In the simulation uses we target, the numbers a model writes are what the simulator consumes, so their reproducibility within a model and their agreement across models are the reliability properties at stake. They are also necessary conditions for any behavioral reading: two models that disagree on their most probable outcomes under an identical prompt cannot both match the same behavioral target. For the same reason, shortlist coverage is scored on the target model's own held-out calls: it measures what is lost when one model's shortlist stands in for another's, not human-choice accuracy (Section 5.3). Validation against human choices is the next step (Section 7).
Question 2: Would the major cross-model findings remain if the prompt defined one mutually exclusive decision, such as the destination of the next single trip, or the restaurant for one particular dining occasion, with an outside option?
Question 2: I think this would help establish whether the results reflect differences in preference predictions or simply differences in how models generate numerical lists.
Thank you for this concrete suggestion. We will run this controlled experiment for the Llama 70B/405B and Qwen 72B/235B pairs (20 calls per model): the prompt asks for the destination of the traveler's next single trip, with "no trip" as an outside option, keeping the persona, choice list, and decoding settings of the matched pairs, and the analysis follows Tables 15 and 16. The Llama-3.1-405B endpoint used in the paper has since been retired, and no serverless inference provider we checked offers this checkpoint, so we will deploy it on our own GPU server, which takes longer than the discussion period; we will include those experiment results in the camera-ready version. For Llama-3.3-70B, the most probable destination in the 20 pooled verbalized IND calls, ORD (Figure 9), is the airport returned in all 300 single-choice draws at temperature 0.2 (Appendix A). We will also compare models on token-level single-choice probabilities, which involve no numerical list, and we will include those experimental results in the camera-ready version.
Response to Reviewer MHyE (1/2)
Thank you for the thoughtful review and for valuing the shortlist-coverage measure and the audit of matched versus unmatched probability. This part answers Question 1 and the concerns about the choice environment and repeated calls; Part 2 answers the validation concern and Question 2.
It is the former, as the sum-to-one instruction implies: the probability of selecting each alternative on one choice occasion (a trip, a dining visit, a purchase) within the stated window; over several occasions, it is the expected share of occasions. This is the standard discrete-choice probability conditional on a choice occasion [1, 2], and it mirrors travel demand modeling, where trip frequency (trip generation) is modeled separately from where each trip goes (destination choice) [3]. The window sets the context without assuming exclusivity, and for the restaurant and product prompts the sum-to-one instruction resolves the wording toward choice shares. Section 3.2 will state this definition.
The context is minimal and, within each compared pair, identical (the same persona text, origin, and choice set; the adapted Mistral-24B prompt adds one sentence, Appendix I.5), so missing attributes cannot by themselves explain why the two models of a pair disagree. If outputs partly reflect salience from training data, the finding that differently sized models of one family emphasize different alternatives is what a practitioner needs to know before reusing such a distribution. Section 6 will state that the elicited distributions are conditional on this minimal context, which includes no prices, budgets, or trip purposes.
Repeated calls are not used to represent different consumers. They measure how stable one model's stated distribution is for one fixed profile (Section 3.3), a precondition for using that distribution in a simulator. Heterogeneity across profiles is examined separately with a persona grid of five age groups by four education groups (Appendix C.1). In the specified contrasts (Appendix K), changing the age within an education group, or the education within an age group, moves the 70B distribution by mean JS 0.330 and 0.258 (Table 18), and no pair of Llama models gives an identical top five for any of these 20 personas (Table 3).
References
[1] McFadden, D. (1974). Conditional logit analysis of qualitative choice behavior. In P. Zarembka (Ed.), Frontiers in econometrics (pp. 105–142). Academic Press.
[2] Train, K. E. (2009). Discrete choice methods with simulation (2nd ed.). Cambridge University Press. https://doi.org/10.1017/CBO9780511805271
[3] Ortúzar, J. de D., & Willumsen, L. G. (2011). Modelling transport (4th ed.). Wiley. https://doi.org/10.1002/9781119993308