acceptodds
Withdrawn from ICLR 2027

Auditing the Economic Alignment of Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Language models are increasingly used to make purchases and to negotiate on people's behalf. An agent trusted with such decisions should at least hold a coherent preference: whether it accepts a routine trade at a given price should be well defined, should not change when the same offer is reworded, and should not depend on details that are irrelevant to the trade, such as the names of the two parties. This paper tests that requirement directly. Eight open-weight instruction-tuned models were each asked, one price at a time, whether to buy a fixed quantity of water or of wheat, over dense price grids and under controlled variations of the prompt, yielding about 1.1 billion single-token yes/no decisions. Each price sweep is first tested for a clear switch from buying to declining inside the tested prices; where one exists, the switching price is located with a logistic fit. The requirement is met only partially. Where a switching price exists, rewording the same offer moves it by more than a factor of five, and sampling the output at temperature one instead of decoding greedily makes six of the eight models flip between buying and declining at 20-50% of adjacent prices. Replacing the generic labels “Buyer” and “Seller” with names changes the share of prices at which a model buys by up to 95 percentage points for the typical name pair, and which names are used moves the switching price across a 2.5-fold range in the median model-commodity panel; most of this variation is attributable to individual names rather than to the region or gender labels the names carry. A coherence leaderboard summarizes the eight models on responsiveness to price and invariance to irrelevant cues; on it, an agent that never changes its decision outscores half the models. An economic delegate must therefore be evaluated on its whole decision curve and its invariances, not on a single reservation price. The 1.1 billion model responses and the analysis code will be released after the review period.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
economic alignment, delegated purchasing agents, reservation price, price sweeps, decision boundary, name-based invariance, buyer–seller pairing effects, open-weight instruction-tuned models, revealed preference, machine psychology, price tokenization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_FcqPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper audits whether language models acting as purchasing agents hold coherent preferences, using the weakest requirement of economic rationality: a monotone decision boundary in price. The method sweeps dense price grids while holding the transaction fixed (about 137M prompts, 1.1B single-token greedy yes/no decisions), classifies each decision curve (constant, near-constant, or containing a step), fits a logistic boundary with instability metrics, and tests invariance under paraphrase, quantity changes, and economically irrelevant buyer/seller names and occupations. Eight open-weight models (0.5B to 32B) are scored on a coherence leaderboard. Findings: coherent step curves are the exception (30 of 64 baseline curves); equivalent paraphrases move boundaries by factors above 5; replacing generic roles with names shifts demand by up to 95 percentage points (Llama-3.1-8B on water: 24 percent generic versus 94 percent for the median named pair) with boundary ratios up to 16.5x; quantity direction is always correct; half the models score worse than an agent that never changes its decision.

Critical strengths

The empirical execution is thorough in ways that matter: name-level permutation tests, a replication with a second validated name set, threshold sensitivity analyses, and a temperature ablation are all present, and the headline name effect is large enough (24 to 94 percent) to be robust to reasonable analytic choices. The documentation of identity-sensitive purchasing decisions is a finding of real concern and, to my knowledge, new at this granularity.

Critical weaknesses
  1. The gap between what is measured and what is claimed is large, and it is consequential because the paper's framing ("economic alignment", a coherence leaderboard) invites conclusions the data cannot support. The task is a single-shot yes/no purchase of water or wheat with no budget, no alternatives, and no interaction, and the model pool contains only open-weight models up to 32B. Deployed purchasing agents are built on frontier closed models, none of which are tested, although the method needs only a yes/no API call. As a result the leaderboard, the most visible artifact of the paper, ranks models nobody deploys for this purpose on a task far below deployment complexity.

  2. The main results use greedy decoding, and the paper's own Appendix I shows this choice is load-bearing: at temperature 1, boundaries disappear for five of the seven panels that had one, and turbulence rises to 0.2 to 0.5 for six of eight models. Since deployment uses sampling, the main tables likely overstate coherence, and the ablation that shows this should be in the main text rather than the appendix. The authors are candid that the leaderboard is "a summary, not a test" with no uncertainty estimate, which further limits what the ranking can support. I also note Figure 1 is not legible at print size (16 compressed panels whose axis labels and annotations cannot be read), which matters because it is the main qualitative evidence for the curve taxonomy.

Questions for authors
  1. Can you run the price sweep on at least one frontier closed model? The per-decision cost is a single output token, and this would determine whether the incoherence findings describe deployed systems or only small open-weight models.

  2. Would the leaderboard ranking survive at the sampling temperatures used in deployment (e.g. T = 0.7)? If not, which results should readers treat as the operative ones?

  3. For the panels that are saturated (always buy or never buy), how much of the name-effect analysis do they carry?

1 comment

Thank you for your time and efforts for a detailed review. Your recognition of the thoroughness of the empirical work, including the permutation tests, replication and ablations, and of the identity-sensitive purchasing findings, which you described as new at this granularity, is greatly appreciated. Also, your points on the gap between the task and deployed agents, the role of decoding temperature, and the clarity of the presentation are valuable. This submission will be withdrawn so that we can address your points properly in our next revision. Thank you again for your thoughtful contribution.

Reviewer_wDZ8OpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper audits eight open-weight instruction-tuned language models on single-token purchase decisions for water and wheat. The main corpus contains roughly 1.10 billion decisions; the second-name-set replication and temperature ablation bring the total to approximately 2.28 billion. Dense price sweeps vary paraphrases, quantities, buyer/seller professions, and name pairs. The analysis first tests for an interior decision boundary, then reports logistic thresholds, widths, demand, and adjacent-price reversals. Only 30 of 64 baseline curves have a boundary; some thresholds vary more than fivefold across wording. Name-pair and permutation analyses find substantial individual-name sensitivity, with replication on a partially overlapping second name set. A three-seed temperature ablation and a descriptive coherence leaderboard summarize additional instability and invariance results.

Critical strengths
  1. The curve-level audit makes a useful methodological distinction between absent boundaries, unstable thresholds, and genuine price reversals. Explicit range limits, quantity positive controls, refusal handling, alternative curve summaries, and temperature tests make the findings more interpretable than a single elicited reservation price. This provides a reusable evaluation framework for delegated decisions.
  2. The identity analysis moves beyond coarse group heatmaps. Full buyer-seller crosses, name-level permutation units, self-pair removal, additive versus pairing effects, and a second name set show that much of the response variation is individual-name-specific. The paper appropriately limits demographic interpretations and reports incomplete overlap and saturated panels, supporting the credibility of this central empirical observation.
Critical weaknesses
  1. Adjacent-price flips under independent stochastic decoding do not alone demonstrate incoherent latent preferences. Even a perfectly monotone purchase-probability curve produces flips when a fresh answer is drawn at each price, especially near probability one half. Three complete sweeps assess seed stability but do not estimate per-price probabilities well enough to separate ordinary sampling variability from non-monotone decision probabilities. This distinction materially affects the claim that temperature destroys economic coherence, although it does not remove the observed deployment variability or greedy reversals.
  2. The largest naming contrast is causally confounded: replacing Buyer/Seller with names simultaneously removes explicit role cues. Section 5.4 acknowledges this, but a role-preserving named condition or neutral-label replacement is absent. In addition, the leaderboard's name component aggregates region-cell demand ranges even though the main analysis shows most variation occurs within regions. The causal headline and descriptive ranking therefore need controls and sensitivity checks before being treated as measures of identity invariance.
Questions for authors
  1. On representative panels, are estimated purchase probabilities monotone with price under repeated sampling at each price? How much observed turbulence exceeds a monotone-probability sampling baseline? This would distinguish stochastic execution noise from inconsistent preference probabilities.
  2. Does the large generic-to-named effect persist when prompts retain explicit Buyer/Seller roles, and how does it compare with replacing those role nouns by neutral identifiers?
  3. How stable is the coherence ordering when identity invariance uses individual-name or name-pair variation instead of region-cell ranges, and under alternative metric weights? Flag For Ethics Review
1 comment

Thank you for your time and for such a clear, helpful review. We appreciate your recognition of the curve-level distinction between absent boundaries, unstable thresholds and genuine price reversals, and of the identity analysis that goes beyond group-level heatmaps to name-level effects. We have decided to withdraw the submission so that we can take the time to address your reservations carefully in our next revision. Thank you again for your thoughtful contribution.

Reviewer_dYSpOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies whether LLMs can make economically coherent purchasing decisions. The authors propose that an economically aligned AI agent should exhibit a stable reservation price, purchase a product at lower prices rather than higher prices, and remain invariant to irrelevant information such as prompt wording and buyer/seller identities.

The paper evaluates eight open-weight LLMs using around 1.1 billion deterministic yes/no purchasing decisions across two commodities, water and wheat. The authors construct demand curves by changing the offered prices, and measure decision boundaries, non-monotonicity, and sensitivity to prompts, quantity, names, professions, and sampling temperature. The paper also proposes a coherence leaderboard to compare the performance of different models.

The main findings are that many models do not exhibit a clear switching price within the tested range, and even when they do, the boundary can vary significantly with prompt wording. The authors also report large changes in purchasing decisions when buyer and seller identities are changed. These variations appear to be driven more by individual names than by broad demographic categories. The paper concludes that the economic alignment of LLMs should be evaluated using entire decision curves rather than only a single willingness-to-pay estimate.

Critical strengths

I appreciate the motivation to evaluate whether AI agents can make consistent economic decisions, particularly when they are used for purchasing and negotiation on behalf of consumers. The paper also goes beyond measuring one willingness-to-pay estimate and attempts to understand the entire decision curve. This is potentially useful for evaluating AI agents.

The authors collect a large number of model responses and conduct experiments involving prices, quantities, prompt wording, names, and professions. The individual-name effects are interesting, especially the finding that a substantial amount of variation cannot be explained by the demographic categories assigned to the names. The additional name-set replication is also useful.

Critical weaknesses
  1. The proposed economic coherence framework is conceptually problematic.

My main concern is whether the paper actually measures economic rationality or alignment. First, the absence of a reservation-price boundary within the tested range does not necessarily imply incoherent behavior. A rational consumer may consistently accept or reject all offers if their valuation falls outside the price range. But the paper penalizes such cases in its leaderboard, affecting 34 of the 64 baseline curves.

Second, the paper confuses stochastic choice with preference inconsistency. Even when the underlying purchase probability decreases monotonically with price, independently sampled binary decisions can frequently reverse. Thus, the increased turbulence under higher temperatures does not necessarily indicate economic irrationality.

More fundamentally, the purchasing tasks do not specify consumer budgets, utility, outside options, or actual preferences. Without these, it is unclear what the model should be optimizing or whose preferences it should align with. I think the paper does not sufficiently distinguish output consistency, economic rationality, and alignment, which raises fundamental concerns about its main contribution.

  1. Important identification problems and limited theoretical and empirical contributions.

I have several concerns about the experimental design and interpretation.

First, the assumption that buyer and seller identities are economically irrelevant is questionable. Names and professions may signal income, preferences, or transaction reliability, which can influence willingness to pay. Moreover, the comparison in Section 5.4 simultaneously introduces names and removes the generic Buyer/Seller labels, making it difficult to isolate the effect of identity.

Second, despite the impressive number of model responses, the experiments involve only two commodities and highly repetitive purchasing scenarios. Billions of similar binary decisions do not necessarily provide stronger evidence about economic behavior. A smaller set of more carefully designed economic environments may be more informative.

Third, the average-demand metric depends on the arbitrarily chosen price grid, while the leaderboard combines heterogeneous measures with equal weights without economic justification. An agent that never changes its decision even outperforms half the models, which raises questions about the validity of the ranking. There is also no evidence that higher-ranked models achieve better economic outcomes.

Overall, the paper largely documents LLM sensitivity to prompts and identities, which is already well established. I do not see sufficient new theoretical insight or empirical evidence connecting these variations to actual economic misalignment. These concerns are fundamental.

Questions for authors
  1. Economic coherence versus stochastic choice. How would the main results change if the authors measured purchase probabilities using repeated samples at each price, and tested the monotonicity of those probabilities rather than the monotonicity of realized binary responses? I think this is necessary to establish whether the observed temperature effects represent actual incoherence or expected stochastic variation.

  2. Interpretation of constant decision curves. Why should a model that always declines or always purchases within the tested price range be considered economically incoherent? Can the authors provide an economic justification for the no-boundary penalty in the leaderboard, and show how the results change if these cases are separated from genuine non-monotonicity?

  3. Identification of identity effects. Can the authors conduct a factorial experiment that separately changes explicit buyer/seller role labels and personal names, while fixing consumer budgets, preferences, seller reliability, and other relevant information? Without this separation, I do not think the paper can establish that the headline name effects represent violations of economically irrelevant-cue invariance.

  4. Economic validity of the leaderboard. Is there any evidence that models ranked higher on the proposed coherence score make better decisions in terms of actual economic outcomes? For example, do they better maximize consumer surplus, respect user budgets, or follow known consumer preferences? Without such validation, why should the proposed ranking be interpreted as economic alignment rather than a descriptive measure of output stability?

  5. Contribution beyond prompt sensitivity. What new economic insight does the paper provide beyond showing that LLM outputs are sensitive to prompt wording and identity cues? Is there a specific economic mechanism explaining the observed differences, or a prediction from the proposed framework that can be independently tested? I think a clearer answer is important to establish the contribution relative to existing work.

1 comment

Thank you for your time and for such a thorough review. Your recognition of the motivation for evaluating purchasing agents on the whole decision curve rather than a single willingness-to-pay estimate is greatly appreciated. Additionally, your points on separating output consistency, economic rationality and alignment, on stochastic choice, and on the economic relevance of identity are valuable to us.

The submission will be withdrawn so that we can take the time we need to address these points carefully in our next revision. The review process is intensive and we appreciate the great deal of effort you made to give us this feedback which is extremely valuable.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.