acceptodds
Under review as a conference paper at ICLR 2027

Democratic Value Trade-offs in Large Language Models: A Conjoint Audit Against Human Choices

Abstract

Large language models (LLMs) summarize public consultations, mediate deliberation, and simulate citizens, so how they weigh democratic institutions against other outcomes depends on the political judgments they inform. People's stated support for democracy is close to universal, so political science measures their commitment by what they would give up democratic institutions for. Conjoint experiments do this by setting institutions such as free elections against outcomes such as a strong economy. Studies of LLMs values elicit stated positions or weigh one value against another, and none compares how LLMs make this trade-off with how people make it. We present the tasks of such an experiment, answered by 4,130 respondents in four countries, to 23 LLMs, each prompted with the background of the respondent who faced the task, and compare the trade-offs implied by the models' choices with the respondents'. Most models penalize unfair elections less than the respondents do and a low-income economy and placement in the smallest minority group more than a logistic regression fitted to the respondents' choices. In 19 of 23 models, a high-income economy outranks low corruption as compensation for a democratic loss, reversing the respondents' order, and all 16 models that predict individual choices best show the reversal. The reversal recurs across eight model families and in 17 of 23 models under a richer persona, free-text answers keep the smaller penalty on unfair elections, and fine-tuning three open models on the respondents' choices restores the corruption–economy order and reduces the error in their choice probabilities. Overall, most models, including the most accurate, accept unfair elections in return for a high-income economy more often than the respondents, so a model can agree with people's individual choices without sharing their democratic priorities.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
large language models, synthetic respondents, democratic trade-offs, conjoint experiments, survey simulation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_h9fzOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper measures how LLMs trade democratic institutions against material and social outcomes. It replays the conjoint tasks of Chu et al. (2025), answered by 4,130 respondents in the US, India, Italy and Thailand, through 23 LLMs, each prompted with the background of the respondent who faced the task, and uses a cross-fitted logistic model that answers deterministically as a baseline separating the answering rule from the model's weights. Models match individual choices above chance, but most under-penalize unfair elections, and in 19 of 23 models a high-income economy offsets a democratic loss more than low corruption does, reversing the respondents' order; LoRA fine-tuning on respondents' choices restores that order in three open models.

Critical strengths

This is the right measurement and it is new. Stated support for democracy is near universal, which is why political science measures commitment by what people are willing to trade for it, and porting that logic to LLMs on the respondents' own randomized tasks is a genuine contribution. The fitted logistic baseline is the best methodological decision in the paper and should be imitated: a temperature-0 model answers deterministically, which by itself inflates the seven penalties 1.60-1.85x, and most LLM-versus-survey work never separates that from the preference. Robustness is broad (persona variants, prompt formats, survey languages, 576 free-text narratives) and null results are reported as such.

Critical weaknesses

Mostly framing, nothing fatal. The abstract is more negative than the results: Section 4.3 shows that of the nine human orderings with intervals excluding zero, most models reverse only two, so the story is one consistent exchange-rate error (corruption vs economy) rather than broad divergence, which is a sharper and better supported claim. Relatedly, the paper itself notes the reversal is the economy weight "expressed as a ranking," so 4.3 largely re-expresses 4.2. The human corruption-economy gap is not established in every country (Thailand's interval touches zero), yet reversals are counted there as well. Finally, the main replay stored only the parsed answer, so the main analysis sits on the decision scale the paper itself shows is contaminated by the answering rule; storing next-token probabilities for the open-weight models would make 4.2-4.3 much harder to argue with. Three GPT-5 models also ran at provider defaults rather than temperature 0. A careful paper with an important result.

Questions for authors
  1. Do the election penalty and the corruption-economy reversal survive on the probability scale for the open-weight models?
  2. How should the reversal counts be read in countries where the respondents' own corruption-economy gap does not exclude zero?
  3. Section 4.4 finds minority heterogeneity amplified and education heterogeneity compressed in every model. Does that suggest the persona elicits a stereotype of the described person rather than the respondent's trade-off?
2 comments

Thank you for your thoughtful and encouraging review, and for recognizing the motivation behind our measurement, the fitted logistic baseline and the range of robustness checks. We conducted new analyses to address your questions. In summary:

  • Probability scale (Q1): in three open models, the corruption–economy gap and the penalty ratio differ from the respondents' on the probability scale (all p = .001), and the election penalty is smaller in all three (significant in one).
  • Countries (Q2): where the respondents' ordering is established (US and Italy), 21 and 20 of 23 models reverse it (point estimates).
  • Persona and stereotypes (Q3): a plausible interpretation that our design cannot confirm; we will note it in §4.4.
  • GPT-5 defaults: excluding the three GPT-5 models, 17 of 20 still have a negative corruption–economy gap.

Line numbers refer to the submitted version.

Q1. Probability scale. As the paper notes (L251–253), the main replay stored only the final answers. The three results below show, in the four open models whose token probabilities we could score, that the differences behind §4.2–§4.3, the smaller election penalty relative to the economy and the corruption–economy gap, are not produced by deterministic answering: they appear on the probability scale, and the two scales agree.

1. New runs that record answer-token probabilities. If deterministic answering created the differences, the probability scale would not show them. We ran three open models without a persona on 1,033 respondents (design in our response to Reviewer A2jd, 3/3). We take the log-probabilities of the token carrying the answer letter (top 20 stored), merge variants of the same letter, and normalize over A and B. When one letter falls outside the stored top 20 tokens (0, 19 and 1,330 of 3,099 tasks), we set it to zero. Its probability is then at most one minus that of the chosen letter: at most 2.4×10⁻⁴ for Gemma-4-26B and 3.2×10⁻⁶ for Gemma-4-31B, too small to change the sign of any reported gap or whether it falls outside the ±0.02 tie band. As in the main replay, the decision scale is the generated answer, which matches the most probable letter in 98.8%, 99.7% and 100% of these tasks. Qwen3.5-27B's probabilities are less concentrated than the Gemma models' (median 0.93 for the chosen letter; below 0.9 in 42.8% of tasks), so for this model the agreement of the two scales is not automatic. On this sample, the respondents' gap is +0.054 and their ratio 0.38.

QuantityQwen3.5-27BGemma-4-26B-A4BGemma-4-31B
Corruption–economy gap, probability / decision scale−0.033 / −0.045−0.056 / −0.055−0.031 / −0.031
Gap minus respondents', probability scale−0.088 (p = .001)−0.110 (p = .001)−0.085 (p = .001)
Penalty ratio, probability / decision scale1.50 / 1.481.50 / 1.491.00 / 1.01
Log ratio minus respondents', probability scale+1.36 (p = .001)+1.36 (p = .001)+0.96 (p = .001)
Unfair-election penalty below respondents', probability scalep = .001p = .057p = .36

The two scales agree closely (gaps within 0.012, ratios within 0.02). For the two Gemma models, the election-penalty difference has the same direction but is not significant.

2. The §4.6 base models, with personas, on a separate sample. This extends item 1 to persona-conditioned prompts, the paper's main condition. Table G.6 reports only point estimates for these models. On the probability scale, the gaps of Qwen3.5-9B, Qwen3.5-27B and Gemma-4-26B-A4B fall below the respondents' by 0.10, 0.18 and 0.13 (p = .043, .001 and .011; respondents +0.098 on that sample). For Qwen3.5-9B the model's own gap is a tie (−0.002), so the difference is a shift toward the economy.

3. The penalty ratio. The §4.2 ratio changes little with the answering rule. Answering with its probabilities, the fitted logistic model reproduces the respondents' unfair-election penalty (−0.189 vs −0.190). Answering deterministically, it inflates the seven penalties by similar factors (1.60–1.85), so its ratio stays near the respondents' (0.48 vs 0.46), and its corruption–economy gap stays positive (0.066 vs 0.046; L370–371). In the new runs, the ratio is nearly identical on the two scales (table above). In the main replay, the ratio exceeds the respondents' in 20 of 23 models (19 with q < .05 in a new paired-bootstrap test; GPT-5.4-Nano's ratio cannot be tested because its unfair-election penalty is near zero), and the unfair-election penalty, as a multiple of the respondents', falls below the model's average multiple in 22 of 23 (Table D.3). Both comparisons are robust to the near-uniform inflation the answering rule produces.

Framing. Thank you for this suggestion. We will make the specificity of the finding explicit in the abstract and introduction: the median model weighs a low-income economy 1.38 times as heavily as unfair elections, against 0.46 for the respondents (20 of 23 models), and of the nine orderings the respondents establish, a majority of models reverse only two, corruption against the economy and corruption against minority treatment (Table E.2). On §4.3: as the paper notes (L373–375), the reversal is the §4.2 economy weight expressed as a ranking. §4.3 reports the ranking because it is the form in which a study using the models would draw conclusions, and it shows that the economy outranks low corruption in 16 to 21 of 23 models within each country and contrast, and in all 16 most accurate models (L378–384). §4.3 will open by stating this relation.

GPT-5 models at provider defaults. The paper's counts do not depend on these three models. When the three GPT-5 models are excluded and the BH correction is recomputed over the remaining 20 models, 17 have a negative corruption–economy gap (15 beyond the tie band with q < .05), and 16 penalize unfair elections less than the respondents (15 with q < .05). In the new no-persona runs, the smaller election penalty also appears in the three GPT-5 models at provider defaults as well as the three GPT-5.4 models at temperature 0 (all six with p < .05).

Q2. Countries where the respondents' gap does not exclude zero. We agree that "reversal" should be used only where the respondents' own ordering is established, and we will revise the text accordingly. In the US and Italy, where the respondents' interval excludes zero (Table B.1(b)), 21 and 20 of 23 models have a negative gap (point estimates). For all four countries, we also report the difference between each model and the respondents, which does not depend on the sign of the respondents' gap. It is shifted toward the economy in:

CountryModels below respondentsSignificant (q < .05)
US2221
Italy2120
Thailand2016
India1812

L378–381 will say "reversal" for the US and Italy and "shifted toward the economy relative to respondents" for India and Thailand.

Q3. Stereotypes of the described person. These differences may reflect a stereotype the models hold of the described group. Our design cannot establish whether this is the cause, but we consider it a plausible interpretation, since each prompt gives only the respondent's background (L139–143) and the respondents' own choices barely differ by minority status (L398–400). We will note this interpretation in §4.4.

We will upload the revised PDF with these changes during the discussion period.

Thank you again for engaging so closely with the paper. Your suggestions make the main claim sharper, and the new evidence gives §4.2–§4.3 a firmer basis.

Reviewer_A2jdOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The authors compare LLMs with 4,130 human respondents on a conjoint experiment. Models receive each respondent’s background and choose between the same hypothetical countries, each describe with a list of attributes. Experiment results show some patterns: for example, most models give fair elections less weight relative to economic prosperity than actual human respondents do. In 19 of 23 models, a strong economy outweighs low corruption as compensation for weakened democratic institutions, reversing the human ordering. Additional experiments examine richer personas, response formats, and fine-tuning.

Critical strengths

I think the authors' experiment breadth and task-level pairing make is a careful methodological design. The experiment covers 23 models from 8 families in 4 survey languages, and every model is compared with humans on the same respondents' tasks. The task itself, using the conjoint analysis, is interesting, too.

Critical weaknesses
  1. The paper is hard to read by humans. For example, this is the one sentence:

(Line 56) Studies of LLM values elicit stated positions, whose ordering changes with prompt and response format (Moore et al., 2024; Tang et al., 2026; Röttger et al., 2024), or set one value against another rather than against a material good (Chiu et al., 2026; Liu et al., 2026; Shen et al., 2025), and studies of LLMs and democracy score stated support for democratic institutions (Piedrahita et al., 2026) or model assessments of them (Loaiza et al., 2025), which, as among people, does not settle how a model treats them: models that generally favor democratic leaders favor authoritarian figures more when prompted in Mandarin (Piedrahita et al., 2026).

  1. The paper's contribution is narrower than the framing suggests.

Prior work already shows that LLMs can resemble human survey responses while failing to reproduce underlying relationships (Bisbee et al., 2024 (also cited in the paper)), and recent work evaluates synthetic respondents in conjoint experiments ("Multi-dimensional Bias in Modeling Multi-dimensional Preferences: Evaluating the Ability of Synthetic Agents to Replace Human Participants in Conjoint Experiments", Hung et al., 2026), to name a few among papers in the field claiming LLMs behave differently compared to human respondents. Measuring model priorities through trade-offs also has existing works, such as, Mazeika et al. ("Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs", 2025). The remaining contribution in this paper is specific election-economy-corruption pattern in an existing human study. The paper should distinguish and discuss this more clearly from the established cautions previous works have already mentioned.

  1. Authors' interpretation of the experimental results exceeds what the persona-conditioned LLM shows.

Every model is asked to simulate a particular respondent, so the results is on the fidelity of that respondent simulation. They do not directly establish the model’s own value priorities / trade-offs as the title suggests. An experiment condition without a persona prompting would help distinguish these explanations.

Questions for authors

No additional questions -- please refer to the critical weakness above.

5 comments

Thank you for the careful review and for recognizing the breadth of the experiments and the task-level pairing with the respondents. Line numbers refer to the submitted version.

1. Readability. We will revise overly long sentences and awkward expressions, including the sentence at L55–61.

2. Contribution. Thank you for these references. We would like to clarify how the paper's contribution differs from them. The paper is not a study of conjoint methodology, nor of whether synthetic respondents can replace survey participants in general. Its question is how LLMs trade democratic institutions against material and social outcomes, compared with people (L54, L63–64). The conjoint data of Chu et al. is the instrument that allows this comparison on the same respondents' tasks. The human data come from Chu et al.; the model side, the task-by-task comparison and the answering-rule baseline are new, and the paper's framing and contributions are stated for exactly this comparison (L63–64, L78–85).

The cited studies answer different questions:

  • Bisbee et al. (2024), "Synthetic Replacements for Human Survey Data? The Perils of Large Language Models" (Political Analysis), show that LLM survey answers can resemble human ones while missing underlying relationships. The paper cites this as a general caution (L97–98); it does not concern democratic trade-offs.
  • Hung et al. (2026), "Multi-dimensional Bias in Modeling Multi-dimensional Preferences: Evaluating the Ability of Synthetic Agents to Replace Human Participants in Conjoint Experiments", ask whether synthetic agents can replace participants in conjoint experiments, using policy and candidate conjoints. Their results do not show how models weigh democratic institutions against prosperity. We will add it to §2.
  • Mazeika et al. (2025), "Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs", infer exchange rates between values from a model's preferences, without human choices on the same tasks (cited at L93–94).

These cautions establish that models can diverge from people. They do not show whether models give democratic institutions more or less weight than people do when set against material outcomes. That direction is what matters, because political science measures democratic commitment by what people give up institutions for (L46–53), and §2 (L109–118) explains why a model's departures in this direction matter for the judgments it informs. The paper's results on this are:

  • 20 of 23 models, from eight families, weigh a low-income economy more heavily relative to unfair elections than the respondents do (§4.2; 19 with q < .05 in a new paired-bootstrap test; GPT-5.4-Nano gives unfair elections almost no penalty, so its ratio cannot be tested).
  • 19 of 23 rank a high-income economy above low corruption as compensation for a democratic loss, reversing the respondents' order (§4.3), in 8 of 10 closed-weight and 11 of 13 open-weight models (Appendix D).
  • Of the nine orderings the respondents establish, a majority of models reverse only two, corruption against the economy and corruption against minority treatment (Table E.2).
  • Used as synthetic respondents, these models would overstate the choice rate of an unfair-election, high-income country by about 10 percentage points at the median (0.596 against 0.497; Appendix H).

The Discussion states these conclusions and their uses for researchers who simulate citizens, for value evaluations and for alignment (L453–466), and Appendix H develops them.

The paper also shows what validating synthetic respondents requires beyond accuracy:

  • Accuracy does not reveal it (§4.3, L383–384). All 16 models with the highest task match rate have a negative corruption–economy gap (15 beyond the tie band). Within a family, the higher-capacity model has the higher match rate in 12 of 14 pairs but the larger recovery-rate difference in only 7 of 14 (Table D.7).
  • The answering rule does not explain it (§3.4). A logistic model fitted to the respondents and answering deterministically, as a temperature-0 model does, enlarges the penalties 1.60–1.85 times, yet its penalty ratio (0.48) and its choice rate for unfair-election, high-income countries (0.501) stay close to the respondents' (0.46, 0.497), far from the model medians (1.38, 0.596).
  • Probabilities and answers must be evaluated separately (§4.6). Fine-tuning on the respondents' choices turns the corruption–economy gap positive and lowers the probability-scale CFE, while the decision-scale CFE rises.

The paper draws this distinction in §1–§2 (L54–63, L109–120); we will make it more explicit there.

3. Persona conditioning. Thank you for this point, which helps us clarify what the paper measures. We first explain why the paper conditions on the respondent's persona, then report the no-persona condition you suggested.

Why the persona. The survey question asks in which country the respondent would rather be born and grow up, and two of its ten attributes are the wealth and the group position the country assigns to the respondent (L66–69, L128–129). The persona tells the model whose life these attributes describe, and makes each answer comparable with that of the person who faced the same task (L73–75). It also follows the paper's motivation: LLMs are used to simulate citizens and survey respondents and to inform political judgments (L13–15, L39–45), and a model that gives up democratic institutions for material gains more readily than people "would carry that weaker commitment into the judgments it informs" (L116–118). The persona-conditioned trade-off is the quantity the simulation uses rely on, and the first use the Discussion names is for researchers who simulate citizens (L461–463).

Relation to prior studies. Several studies condition a model on a person or group and draw conclusions about the model itself:

  • Gupta et al., "Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs" (ICLR 2024): assigned personas reveal the models' own reasoning biases.
  • Rozen et al., "Do LLMs have Consistent Values?" (ICLR 2025): demographic and persona prompts are used to study the models' value structure.
  • R. Liu et al., "Large Language Models Assume People are More Rational than We Really are" (ICLR 2025): models simulate and predict people's choices, and the departure from actual choices is read as "the implicit decision-making models of LLMs".
  • Jahanparast et al., "What Do Large Language Models Know About Opinions?" (ICLR 2026): group-conditioned prompts are used to probe the models' internal knowledge of the opinions of 22 demographic groups.

Our design follows the same logic: the persona fixes whose task is answered, and a departure that holds in aggregate over 4,130 respondents' personas and within each of the four countries is a property of how the model answers for people, not of any one persona.

A prompt without a persona is also a condition: the model then answers in its default assistant role, which Tan & Lee, "Unmasking Implicit Bias: Evaluating Persona-Prompted LLM Responses in Power-Disparate Social Scenarios" (NAACL 2025), describe as "the default persona that LLMs tend to adopt in the absence of explicit demographic prompts", and whose answers Durmus et al., "Towards Measuring the Representation of Subjective Global Opinions in Language Models" (COLM 2024), find closer to the opinions of certain populations. We therefore read both conditions as the model's trade-offs under a stated condition. The persona condition has the advantage that it evaluates how the model answers when given the observed characteristics of the respondent who completed each task.

The persona design also gives the comparison its breadth. Each model answers as each of 4,130 respondents in four countries on three continents (the US, India, Italy and Thailand), in the survey language, with that respondent's own demographic fields, on the same 12,390 tasks the respondent faced; 23 models from eight families answer each task in eight prompt versions, 2.28 million queries in all (Appendix A). This allows testing whether the difference holds within each country (Table E.4) and within each demographic subgroup and level of stated democratic commitment.

Following your suggestion, we ran nine of the paper's models without a persona.

(a) Runs without a persona. What this tests: the paper's central claim is that models give fair elections less weight relative to the economy than people do (§4.2; L453–455). If this difference required a persona, it would disappear without a persona; it does not. The nine models (Qwen3.5-27B, Gemma-4-26B-A4B, Gemma-4-31B and the six GPT models) were re-run with the respondent description removed from every prompt. The rest of each prompt is unchanged, and decoding stays at temperature 0 (provider defaults for the three GPT-5 models). The sample is the 1,033 respondents of §4.5 with their original profile order (3,099 tasks), and comparisons with the respondents use a paired bootstrap over respondents (B = 1,000).

Without a persona, compared with the respondentsModels in this directionp < .05
Unfair-election, high-income country chosen more often9 / 99
(free-election, low-income) − (unfair-election, high-income) smaller9 / 98
Unfair-election penalty smaller9 / 97
Low-income / unfair-election penalty ratio larger8 / 87
Corruption–economy gap lower6 / 95

Counts are unchanged after Benjamini–Hochberg adjustment over the nine models. GPT-5.4-Nano is left out of the ratio row (its unfair-election AMCE is +0.006). For the three open models, answer-token probabilities give the same result for the gap and the ratio (all p = .001; see our response to Reviewer h9fz, 1/2).

The smaller weight on electoral fairness relative to the economy therefore does not require a persona. The corruption–economy reversal is less frequent without one: the gap is negative in 5 of 9 models (3 beyond the ±0.02 tie band), against 7 of 9 (all beyond the band) in the paper's persona answers on the same tasks. The no-persona gap is higher than the persona gap in 7 of 9 models (p < .05), and still lower than the respondents' in 5 of 9 (p < .05).

(b) Respondent subgroups. What this tests: if the models were imitating a type of respondent, some group of respondents would make the trade-off as the models do. Using the paper's persona answers, we split the respondents into 19 subgroups by country, importance of democracy, ideology, SES, education, age, gender and minority status. In every subgroup, the median model's penalty ratio (1.05–1.61) lies above the respondents' 95% interval, and no subgroup of respondents has a ratio above 0.62. The respondents' corruption–economy gap is positive in every subgroup (+0.002 to +0.082), whereas 16–21 models have a negative gap on each subgroup's tasks. No single-characteristic subgroup of respondents makes the trade-off the way the models do.

(c) Personas that describe the respondent's politics. What this tests: if the economy-over-corruption ordering (§4.3) came from a persona that omits the respondent's politics, stating them should remove it. It does not. The paper's richer persona (P2) adds eleven fields, including political orientation, party and rated importance of democracy. With its four prompt variants pooled, the reversal holds in 17 of 23 models under P2, against 18 under the basic persona (Table E.5). In a new analysis, among respondents who rate democracy 10 out of 10, whose own gap is +0.065, 18 of 23 models still rank the economy above low corruption under P2. The paper also shows that removing democracy-related fields changes the sign of fewer than 5% of 92 comparisons involving match rates, attribute effects and CFE (Table C.1), and that free-text answers keep a smaller election penalty than the respondents' in all six models tested (Table F.2).

On the title. The paper studies the democratic trade-offs models express when simulating citizens or survey respondents. We assess these trade-offs by comparing attribute effects, conditional choice rates and compensation orderings estimated from model and human choices on the same tasks. The title refers to these trade-offs, and their central difference, the smaller weight on fair elections relative to the economy, also appears without a persona (table above). We will state this aim explicitly in §1 and §3.3 and add the no-persona results as an appendix.

We will upload the revised PDF with these changes during the discussion period. We are grateful for this suggestion, which makes the paper's result clearer. If the clarification and the new experiment address your concerns, we would be grateful if you would take them into account in your assessment.

Reviewer_xbYBOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The authors raise the question on how LLMs trade-off democratic values compared to how humans do. They build upon an existing dataset that asks users about their preference on two hypothetical states. Both states are described by attributes such as their leadership selection process, level of corruption and so on. The LLMs are also provided information on the individual answering the preference questions. The authors then analysis to which extend different LLMs agree with the human trade-offs.

Critical strengths

The fundamental idea of comparing how similar LLMs preference is to those of humans is worth research. Given how often LLMs are used as judges and the broader field about the alignment problem getting a better understanding of their internal values and trade offs is interesting.

Critical weaknesses

I find the paper hard to follow. The method as well as the results section tries to discuss various thoughts and ablation studies along the way without a clear line of thought. Taking a look in the appendix or the original paper by Chu et al. the very core idea is not that complicated after all, but the paper does not manage to convey the main message. This critic materialises at various points along the paper. While the original paper consistently reports the effect on the probability of preferring one country description over the other, this paper fails to make such a pick and instead provides an extensive list of metrics and descriptives one might want to look at. This shortcoming then also affects the interpretation of the presented results. Instead of highlighting and explaining interesting specific findings the paper tends to summarize aggregated findings without clear takeaways. Second major weakness is on the identified gap. The authors state that other surveys have “set one value against another rather than against a material good (line 056 f.) and state this to be one of their contribution (line 063). I am willing to accept that this gap does existing. But then I would expect the paper to also discuss this exact gap in detail. Nowhere in the results or discussion section is this potentially new angle discussed in detail.

Questions for authors

No questions that would materially affect my assessment.

4 comments

Thank you for your review and for noting that comparing how LLMs and people make value trade-offs is worth studying. Your review raises three points: (1) the choice of metrics and the line of thought, (2) the takeaways, and (3) the discussion of democratic institutions against material outcomes. Each is addressed in the submitted version, and below we point to the passages (line numbers refer to that version).

1. Metrics and line of thought. The paper asks whether models answering the respondents' tasks make the same trade-offs as the respondents (L63–64, L73–75).

  • Building on Chu et al.'s conjoint analysis. The paper starts from the quantity Chu et al. report, the AMCE, "the average change in the probability that a profile is chosen" (L198–200), and §4.2–§4.3 report differences in this probability. Chu et al. analyze trade-offs by crossing a democratic attribute with an outcome attribute; the conditional choice rates and the compensation c(i | d) (Eq. 1) make this comparison on choice rates, showing "how often a degraded profile is still chosen" (L204–205) and "what respondents accept in return for the degradation" (L216–217). Chu et al. also compare respondent subgroups, including by minority status; ΔΔ and HDR apply this to the models (L209–213).
  • Quantities needed because the comparison is with models. Task match rate tests whether agreement on individual answers carries over to the trade-offs (L269); it does not, since all 16 models with the highest match rate rank a high-income economy above low corruption (15 beyond the tie band), the opposite of the respondents (L383–384). CFE and COD measure a model's distance from the respondents on these quantities, against a split-half noise scale (L235–238). The logistic model fitted to the respondents shows "how far the answering rule alone moves the estimates" (L240–249).

2. Takeaways. The overall takeaway is the first sentence of the Discussion (L453–455): "most of the 23 models weigh democratic institutions against other outcomes differently from the respondents and in the same direction: they give free and fair elections less weight and a strong economy and group position more."

The findings form one line of argument in §4 (L261–265): accuracy does not reveal the difference (§4.1), which lies in the exchange between elections and the economy (§4.2), reorders the outcomes that offset a democratic loss (§4.3) and extends to the respondents' own characteristics (§4.4); §4.5–§4.6 test whether a fuller prompt or fine-tuning removes it.

  • Elections are exchanged for the economy at a different rate. For the respondents, a low-income economy costs 0.46 times as much as unfair elections; for the median model, 1.38 times, and 20 of 23 models exceed the respondents' ratio (§4.2, L309–311).
    • Why it matters: because the models penalize some democratic losses more than people and others less, the difference cannot be read as weaker or stronger support for democracy, which studies of LLMs and democracy measure. It is a different rate of exchange between elections and the economy, the kind of exchange through which backsliding proceeds (L116–118, L317–321).
  • The models reverse what offsets a democratic loss. For the respondents, low corruption offsets a degraded institution more than a high-income economy; 19 of 23 models reverse this, including all 16 with the highest match rate (§4.3, L369–384).
    • Why it matters: a model can match people's individual choices and still make different trade-offs, so accuracy alone cannot validate it (§5, L459–460).
  • A model panel would mislead a study that uses it. A country with unfair elections and a high-income economy is chosen at 0.497 by the respondents and more often by 21 of 23 models (Table 1(b)).
    • Why it matters: the median model's rate is about 10 percentage points higher (0.596 against 0.497), so a study using such models as synthetic respondents would overstate how far a strong economy offsets an unfair election (Appendix H, L2111–2119). This can be checked from answers already collected (L461–463).
  • The models attribute to minority respondents a difference these respondents do not show. The respondents' choices barely differ by minority status, whereas this difference is larger in all 23 models (19 with q < .05; §4.4, L398–400).
    • Why it matters: a model simulating minority respondents attributes to them trade-offs their own answers do not show, which the paper identifies as a fairness concern (L402–403).
  • Fine-tuning lowers the probability-scale CFE but raises the decision-scale CFE. Training on the respondents' choices also restores the corruption–economy order (§4.6).
    • Why it matters: a model's probabilities can move closer to people's choices while the choice rates from its answers move further away, so alignment must evaluate the two separately (L427–429, L464–466).

3. Institutions versus material outcomes. This comparison is the main result of the paper, and the submitted version develops it in both the results and the discussion:

  • §4.2 (L309–321) quantifies the election–economy exchange (0.46 for the respondents against 1.38 for the median model) and explains why the difference is not a matter of more or less support for democracy, but of how elections are weighed against the economy.
  • §4.3 (L367–391) shows how this changes the ranking of outcomes that offset a democratic loss: a high-income economy outranks low corruption in 19 of 23 models, the most accurate ones included.
  • §5 (L457–459): the first conclusion is that "the models differ from the respondents on the trade-off between institutions and outcomes, which stated support, near-universal among people, cannot show".
  • Appendix H (L2111–2119) works out its consequence for studies that use models as synthetic respondents. In a new tabulation, respondents choose a free-election, low-income country more often (0.597) than an unfair-election, high-income one (0.497), whereas 15 of 23 models reverse this order (medians 0.545 and 0.596).
  • §6 (L484–485) closes the paper with it: "Measuring how LLMs treat democratic institutions requires setting them against other outcomes and comparing the trade-offs with people's."

Why this angle is new. Stated support for democracy is near-universal among people, so political science measures commitment by what people give up an institution for (L46–53). The paper already cites the prior studies below and states how it differs from them, in §1 (L54–63) and §2 (L90–103, L108–120):

  • Stability of value positions and rankings (L55–56, L90–92).
    • Moore et al., "Are Large Language Models Consistent over Value-laden Questions?" (EMNLP Findings 2024), compare a model's answers across paraphrases, formats and translations of 8,000 value-laden questions, and find them less consistent on controversial topics such as euthanasia.
    • Röttger et al., "Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models" (ACL 2024), show on the Political Compass Test that models answer differently when not forced into a multiple-choice format, and differently depending on how they are forced.
    • Tang et al., "Probing the Plasticity and Correlation of LLM Value Systems: LLM Value Rankings Are Not Stable" (ACL Findings 2026), show that models' value rankings shift under different prompting strategies.
  • Value priorities inferred from choices (L56–57, L92–93).
    • Chiu et al., "LitmusValues: Will AI Tell Lies to Save Sick Children? Litmus-Testing AI Values Prioritization with AIRiskDilemmas" (ICLR 2026), pit AI-relevant values against each other in dilemmas and show that the resulting priorities predict risky behaviors.
    • Liu et al., "Generative Value Conflicts Reveal LLM Priorities" (ICLR 2026), generate scenarios in which two values conflict and find that the priorities models express shift between multiple-choice and open-ended settings.
  • Value–action consistency (L57, L94–95). Shen et al., "Mind the Value-Action Gap: Do LLMs Act in Alignment with Their Values?" (EMNLP 2025), compare the values a model states with the actions it chooses and find that they often diverge.
  • LLMs and democracy (L58–61, L109–114). Piedrahita et al., "Democratic or Authoritarian? Probing a New Dimension of Political Biases in Large Language Models" (EACL 2026), find that models generally favor democratic values and leaders, yet become more favorable to authoritarian figures when prompted in Mandarin; as the paper notes, such stated support does not settle how a model treats democratic institutions (L113–114).
  • Simulated respondents (L61–63, L95–97). Argyle et al., "Out of One, Many: Using Language Models to Simulate Human Samples" (Political Analysis 2023), condition a model on the backgrounds of real survey respondents and find that its answers reproduce the distributions of human answers across subgroups; our §4.1 shows that a model can agree with most individual answers while its trade-offs differ.

These studies do not estimate how LLMs trade democratic institutions against material outcomes relative to human choices on the same tasks, and so cannot show in which direction a model departs from people. How the paper differs from the synthetic-respondent, conjoint and value trade-off studies raised by Reviewer A2jd (Bisbee et al., 2024; Hung et al., 2026; Mazeika et al., 2025) is explained in our response to Reviewer A2jd.

We will revise the paper so that its line of thought is easier to follow, and upload the revised PDF during the discussion period.

We hope these clarifications address your concerns, and we would be grateful if you would take them into account in your assessment. Thank you again for your time and comments.

AuthorsOpenReview ↗

Reminder: Request for Feedback on Our Rebuttal

Thank you again for your review. As the discussion period is coming to a close, we wanted to briefly follow up on our rebuttal.

If you have a chance to review our response, we would appreciate any further feedback, particularly if you have any remaining concerns. We would also appreciate your consideration of our response in your final assessment.

Thank you for your time and consideration.

Thank you again for the extensive response. In our review, we stressed that "Instead of highlighting and explaining interesting specific findings the paper tends to summarize aggregated findings without clear takeaways". Unfortunately, the made adjustments hardly follow this feedback. Similar to A2jd, the current form of the paper (and rebuttal) does not allow for sufficient human readability. Still, we would encourage you to rework the paper and resubmit.

Comments on the paper

1 comment

We thank the Reviewers xbYB, A2jd and h9fz; the reviews led to several improvements in the revised manuscript (key additions in blue). Page numbers below refer to the revised PDF.

We appreciate that the reviewers found it worth studying how LLMs weigh democratic institutions against other outcomes, and recognized the main elements of the design: measuring these trade-offs as political science measures democratic commitment, by what is accepted in exchange for democratic institutions, with the choices of 4,130 respondents in four countries as the reference; comparing 23 LLMs from eight families with these respondents task by task; a logistic baseline fitted to the respondents' choices that answers as the models do, separating a model's preferences from its answering rule; and checks across persona, prompt format and response format.

The major revisions are as follows:

Line of thought and metrics (Reviewer xbYB): The reviewer asked for a clearer line of thought and choice of metrics. We: 1) added a roadmap at the start of §3 (p. 3); 2) state at the start of §3.3 (p. 4) which quantities follow Chu et al. (2025) and which are added because the comparison involves models.

Takeaways and framing (Reviewers xbYB, h9fz): Reviewer xbYB asked for clear takeaways, and Reviewer h9fz found the abstract more negative than the results. The abstract (p. 1) and contributions (p. 2) now state: the median model weighs a low-income economy 1.38 times as heavily as unfair elections, against 0.46 for the respondents (20 of 23 models higher; §4.2, p. 7), and of the nine orderings the respondents establish, only two are reversed by most models (§4.3, p. 8).

Institutions against material outcomes and related studies (Reviewers xbYB, A2jd): Reviewer xbYB asked for more discussion of this comparison, and Reviewer A2jd for the difference from Bisbee et al., Hung et al. and Mazeika et al. We: 1) added Hung et al. (2026) to §2 (p. 2) and state the distinctions there, with further discussion in §5 (p. 10); 2) report (§4.3, p. 8; Table E.9, p. 41) that the respondents' choice rate for free-election, low-income profiles (0.597) exceeds that for unfair-election, high-income profiles, whereas 15 of 23 models reverse this order.

Persona conditioning (Reviewer A2jd): The reviewer asked whether the results reflect the persona and suggested a run without one. We: 1) state our aim in §1 (p. 2) and §3.3 (p. 4); 2) report runs of nine models without a persona (§4.5, p. 9; Table E.6, p. 38): all nine choose the unfair-election, high-income profile more often than the respondents (all p < .05) and penalize unfair elections less (seven with p < .05), while the corruption–economy gap is negative in five of the nine, against seven with the persona.

Readability (Reviewer A2jd): We divided the long sentence the reviewer quoted (submitted L55–61; §1, p. 2) and others.

Probability scale (Reviewer h9fz, Q1): §4.5 (p. 9) and Table E.7 (p. 39) report that in three open models run without a persona the gap and penalty ratio differ from the respondents' (all p = .001), and the unfair-election penalty is smaller in all three (p < .05 in one).

§4.3, countries and stereotypes (Reviewer h9fz, Q2–Q3): The reviewer noted that §4.3 largely re-expresses §4.2 and asked how to read counts where the respondents' interval includes zero, and whether the persona elicits a stereotype. We: 1) state at the start of §4.3 (p. 7) that the ranking is the §4.2 economy weight expressed as an ordering; 2) by country, report a reversal only for the United States and Italy (21 and 20 models) and a shift toward the economy for India and Thailand (18 and 20 models; 12 and 16 with q < .05; §4.3, p. 8; Table E.4, p. 37); 3) note the stereotype reading as a possible interpretation in §4.4 (p. 9).

GPT-5 decoding (Reviewer h9fz): Three GPT-5 models were not decoded at temperature 0; §4.3 (p. 8) and Table E.9 (p. 41) report that, of the other 20 models, 17 have a negative gap and 16 penalize unfair elections less than the respondents.

Locating the rebuttal's references: The rebuttal cites the submitted PDF's line numbers; existing table, figure and section numbers are unchanged. Four tables are new: E.6 (p. 38, no persona), E.7 (p. 39, probability scale), E.8 (p. 40, 19 subgroups, richer persona) and E.9 (p. 41, ratio test, without GPT-5 models, choice rates). Line and page numbers shifted. Submitted lines map to: L13–15 Abstract; L39–85 §1; L90–120 §2; L128–143 §3.1–3.2; L198–217 §3.3; L235–253 §3.4; L261–269 §4–4.1; L309–321 §4.2; L367–391 §4.3; L398–403 §4.4; L427–429 §4.6; L453–466 §5; L484–485 §6; L2111–2119 Appendix H.

We thank the reviewers again for their careful reading and questions, which have made the findings of the paper more specific. We would be glad to answer any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.