acceptodds
Under review as a conference paper at ICLR 2027

The Origins of Price Sensitivity in Large Language Models

Abstract

Price sensitivity is where an LLM's numeracy becomes an economic preference: acting on a price means representing it, comparing it, and deciding how much it counts against everything else. Models differ in their price sensitivity, which shapes the purchase advice they give; little research has investigated how training produces those preferences. OLMo 2 7B's public checkpoints and training data let us follow representation, comparison and weighting through pretraining, supervised fine-tuning, and direct preference optimization (DPO). Our central finding: accuracy in comparing prices does not reveal how strongly price influences a choice. Internally, magnitude sits closer to a logarithmic scale than a linear one, the same shape the price response takes. By the earliest checkpoint probed, the model already distinguishes numbers of equal length, which digit count alone cannot. A lower-price preference develops during pretraining, alongside improving comparison accuracy and before preference optimization. The training text offers a plausible source: an estimated third of genuine detected prices in sampled web text occur in choice-resolving comparisons. In the plain prompt, annealing and supervised fine-tuning reduce price weight relative to decisiveness about non-price attributes. DPO partly reverses this decline; its effect is smaller in the chat template and in four other models we checked. Separating numerical knowledge from its influence on decisions gives a framework for auditing purchasing agents' economic preferences.

Primary area:
interpretability and explainable AI
Keywords:
large language models, interpretability, training dynamics, probing, numerical representations, direct preference optimization, price sensitivity, economic preferences

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_FMLKOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper traces how price sensitivity forms during training, using the public checkpoints and data of OLMo 2 7B. It separately measures 1. numerical representation (i.e. linear probes on hidden states), 2. price comparison accuracy, and 3. price weighting (i.e. a conjoint-style hotel-choice task with shuffled prices). The authors report several findings. The paper reports that the comparison accuracy does not determine how strongly price influences choices. Also, a cheaper-option preference appears during pretraining, even before preference labels. Annealing and SFT lower the relative price weight, mainly by sharpening non-price choices, and DPO partly reverses this. Both the price response and the internal number scale are closer to logarithmic than linear. The weighting study is rerun on five other models and four DPO retraining runs, and the authors frame the full pipeline as a reusable audit template.

Critical strengths
  • The motivation is timely as purchasing agents are being deployed and tracing agents' economic preference is a reasonable question.
  • The author provide careful and transparent measurement design. Several design choices, such as separating price weight from overall decisiveness, can go beyond typical LLM preference studies
  • The code release give the work some practical values. The authors also report limitations candidly, including invalid stage-1 measurements, checkpoint-to-checkpoint variance beyond bootstrap intervals, run-to-run variation, and dependence on normalization.
Critical weaknesses
  • The evidence does not yet support the claims about the “origins” of the preference. The trajectory analysis covers one model lineage, one training run per phase, and one task domain (binary hotel choices in New York City). “Training stage” is also not a standardized notion across model families; stages such as mid-training, annealing and cool-down differ in content and schedule. The stage-wise conclusions may therefore not transfer beyond the OLMo 2 recipe. OLMo 2 is widely used, but a detailed analysis of a single model has limited implications for how price sensitivity forms in general.
  • Early checkpoints have little question-answering ability. The early findings such as comparison preference of the early checkpoints, are therefore measured on log-odds between letters the model rarely produces. In my reading, they reflect a mix of emerging task-following ability and preference, which the design cannot separate.
  • Neighboring checkpoints differ far more than task-bootstrap intervals predict. For example, two DPO retraining seeds on the same recipe give a plain-prompt increase of 1.45× and 2.84×, with non-overlapping intervals.
  • Also, elicitation through first-token A/B log-odds mixes preference with the model’s general capability, such as instruction following, sensitivity to context. The strong dependence of the results on prompt format supports this concern. For example, the SFT→DPO step is 2.21× in the plain prompt, 1.07× in the chat template, 1.55× with a system message, and not detectable in the cue format.
  • The findings rely on strong modeling assumption. The price weight relies on a specific discrete-choice specification which might not naturally satisfy, such as a log-price term, additive effects and no interactions with other attributes. Also, key conclusions also depend on the normalization chosen.
  • Crucially, if the framework is meant to be general, it is not clear why it focuses on price rather than LLM choice criteria broadly. If price is the special focus, the paper should show what is specific to price.
  • As I understand it, the comparison task is basically the accuracy when asked to pick the lower of two prices, while the weight of price in a multi-attribute choice is a trade-off. It is not surprising that the first does not determine the second, as first task is closer to math comparison. Therefore the claim that “a test of a capability does not certify a preference” seems close to expected and its novelty is limited.

[minor]

  • The introduction section is very dense and previews nearly every result and number from the later sections. A shorter introduction focused on the question, the approach and two or three key findings would be easier to read.
  • The paper’s framing is primarily economic and regulatory. And I have a small concern whether there would be a better fit for behavior analysis on economic. Emphasizing what it contributes to ICLR-relevant questions, such as training dynamics, interpretability of numerical representations, or how post-training changes model decisions, would strengthen its fit for this audience.
Questions for authors
  • By the paper’s own validity rule (Appendix A.3), 11 of 15 stage-1 checkpoints would be excluded. How should the stage-1 results be interpreted, given that most stage-1 checkpoints fail your own validity rule?
  • To what extent are the findings specific to price? If the analysis were rerun with, for example, distance as the focal attribute, or even some known bias such as gender bias, would the same stage-wise story hold?
  • Given that the log-vs-linear gain in held-out MSE is modest (3.6–10.3%) and the five-bin model fits better after post-training, how strongly do you interpret the logarithmic shape?
  • Regarding the untrained-network results, what does it mean concretely for a random-weight network to exhibit “full-range logarithmic spacing”? Different random initializations could behave differently, and with only two initializations it is unclear how stable this reference is. Because untrained networks put almost no probability on A/B, how meaningful are the behavioral metrics for them?
  • Extending the weighting study to other domains (e.g., e-commerce products) and to more than two options would help establish generality.
4 comments

(1/3)

Thank you for the thoughtful review. Your comments on early-checkpoint validity, checkpoint noise and generality led to three changes. The validity rule is now applied to the pretraining checkpoints. The annealing decline is tested across groups of checkpoints. The cross-model results move into the main text as Table 3.

W1. One lineage, one run per phase, one domain. We agree that a single trajectory cannot show how price sensitivity forms in general. The paper no longer claims that. "Origin" now means where in training the preference becomes measurable and which later stages change it. The paper suggests sources but does not isolate causes.

The cross-model results, previously an appendix table, are now Table 3 in Section 5. Because training stages are not standardized across families, Table 3 compares each model's own released SFT and DPO checkpoints. Three patterns recur across five models and four retraining runs. A lower-price preference is present before post-training in all three base models that pass the validity rule. SFT and DPO raise decisiveness in all 24 steps. DPO raises the raw price weight in 19 of 20. What varies is how much the relative price weight changes at DPO, because decisiveness rises alongside the price weight. The step is 2.21× in OLMo 2 7B's plain prompt and 0.73× to 1.10× in the released chat-template pairs (W4). The trajectory remains a single lineage. We therefore claim its timing for OLMo 2 7B only and rest the cross-model claims on the three patterns above. On other domains, see Q5.

W2 and Q1. Early checkpoints and measurement validity. We agree that the submission placed too much weight on checkpoints that fail our own validity rule: 11 of the 15 stage-1 checkpoints fail at least one criterion. The rule is now applied throughout the trajectory (App. A.3). The failing checkpoints are shaded in Figure 3 and their scores are reported as descriptive.

The central finding survives, because it needs only that a lower-price preference is measurable before any preference labels. Its timing becomes less precise. On the four valid checkpoints, all between 3.880T and 3.896T, the price weight is positive with its interval above zero. Relative price weight there is 0.60 to 0.92, well above the untrained band (upper edge 0.13). We keep that conclusion and move the onset from 3.146T to "by 3.880T". The checkpoints that dated it earlier compare prices accurately but put too little probability on the answer letters.

Two further checks show that the stage-1 scores respond to the displayed price rather than to answer position or to the regression specification. First, refitting on the half of tasks with less position bias changes the estimates at the nine checkpoints from 3.146T onward by at most 11% (Table A7). Second, a measure that uses no regression agrees. It counts how often the model's choice reverses when the shuffle reverses which option is cheaper, net of how often it reverses when the cheaper option stays the same. It tracks relative price weight across stage 1 (Spearman 0.95; App. C.1). Neither check separates preference from emerging task-following. Your reading that early scores mix the two is now stated in Section 4.

W3. Checkpoint noise. You are right. Our intervals capture one source of uncertainty, the sample of 600 tasks. Appendix A.4 now quantifies what they miss. Within a training phase, relative price weight varies across checkpoints 2.7 to 3.4 times as much as task sampling predicts.

The annealing result in the submission rested on a single step between neighboring checkpoints, so we retested it as a comparison between groups of checkpoints (Table A9). Against the average of the last five stage-1 checkpoints, all 11 annealing states have a lower relative price weight in every bootstrap draw. The highest annealing value (0.40) is below the lowest late-stage-1 value (0.60), a gap of three to five between-checkpoint standard deviations. Averaged over the tracked annealing run, relative price weight is about a third of its late-stage-1 level (0.32 [0.25, 0.40]), because decisiveness rises 1.77-fold while the price weight itself falls to 0.59 times its level. The result is the same against the four late checkpoints that pass the validity rule (0.34 [0.27, 0.42]; App. C.5) and under the price-window and trimmed fits (0.40 and 0.35; App. B.3). A gap this large and this consistent is well beyond the checkpoint noise we measured.

On the two DPO seeds (1.45× and 2.84×), we agree that one run cannot fix the size of the DPO effect. Section 5 now reports both. The direction recurs. All four retrained runs raise the relative price weight in the plain prompt. DPO raises the price weight in 19 of the 20 steps we measured.

(2/3)

W4. Elicitation through letter log-odds. We agree that first-token log-odds mix preference with format compliance. Table A14 therefore scores the SFT→DPO step in four prompt formats. DPO raises the price weight in all four: 3.54-fold in the plain prompt, 1.91-fold in the chat template, 1.68-fold with the answer cue inside the chat template and 2.96-fold with a system message. It raises decisiveness in all four as well, by 1.60- to 1.91-fold. The price preference therefore strengthens at DPO in every format we scored. So do the preferences over the other attributes.

Relative price weight is the ratio of the two. It doubles in the plain prompt, where the price weight rises about twice as much as decisiveness. It barely moves in the chat template, where the two rise by about the same factor. The format changes how far the price weight outpaces the other attributes. The two formats also agree on where DPO ends: relative price weight is 0.258 in the plain prompt and 0.289 in the chat template (Table 2). The same holds across models. DPO raises the price weight in 19 of 20 steps and SFT and DPO raise decisiveness in all 24, while the change in the ratio ranges from 0.73× to 2.21×. This is why the paper reports the price weight, decisiveness and their ratio separately for each format, with slot-lock beside every number. It also asks audits to name the format and any system message (Section 7).

The scores are also read from tokens the model would produce. At every released stage the higher-scored letter holds more probability than all non-letter tokens combined on 98.6% to 100% of prompts, so greedy decoding would begin with that letter (App. D.2). Scoring generated recommendations on the same tasks is the natural next step and is named in the Limitations.

W5 and Q3. Modeling assumptions and the log shape. The concern is that the price weight depends on one regression form. Table A4 refits it under six specifications: the six-control fit, price-by-attribute interactions, hotel fixed effects, a price window of 100 to 600 dollars, a 5% trim and 71 controls. All six keep log price. The post-training contrasts are stable across them:

Contrast (ratio of r)Range across the six specifications
Base→SFT, plain0.29 to 0.38
SFT→DPO, plain2.19 to 2.29
SFT→DPO, chat1.06 to 1.09

The interactions do show that higher-starred hotels are less price-sensitive after post-training, which the additive fit averages over. One contrast is sensitive to the specification: the single step from 3.896T to +5B, within the price window. Stated as the comparison between phases in W3, the annealing result is robust. On normalization, the two post-training steps keep their direction under all four denominators we tried. Whether DPO ends below the pretraining levels does depend on the denominator (App. D.1).

On the log shape, we read the response as concave and far from linear, with the logarithm as an approximation. In behavior the gain over a linear fit is modest, as you note. A power-transformation fit, in which exponent 0 is logarithmic and 1 is linear, puts the exponent at −0.10 [−0.30, 0.05] at the end of stage 1 and between −0.40 and −0.80 after annealing and post-training (Table A5). The response is logarithmic at first and more concave than logarithmic later. The five-bin curve picks up that extra concavity. In the representation the evidence is stronger. The logarithmic relation beats the linear one in all 176 readout cells (median held-out R² 0.84 against 0.39). It does so again within equal digit counts (0.71 against 0.19; App. B.2). Section 3 now calls the logarithmic form an approximation.

Q4. Untrained networks. Random weights can preserve how a number is written, including its length. Length is a coarse logarithmic cue, because each extra digit means a tenfold larger value. A readout fitted on a random-weight network recovers that cue. This is all that "full-range logarithmic spacing" means there.

We therefore compare only numbers with the same digit count, such as 200 and 800, which length cannot tell apart. On those pairs the untrained readouts do worse than predicting the mean (R² −0.07 and −0.12). The trained model reaches 0.39 at 399B, the first checkpoint probed. It stays above the untrained values at every later checkpoint (0.65 to 0.74 across annealing and Base). Training therefore adds magnitude information that digit count cannot supply.

On stability, the two initializations differ by at most 0.04 in full-range R² and 0.08 in same-digit R². The paper now describes the reference as the two tested initializations (App. A.4).

You are right that the behavioral metrics of untrained networks do not measure behavior. These networks put less than 0.0001 of their probability on A or B. Their accuracy, w and r are therefore estimator floors. Section 3 now labels them as such.

(3/3)

W6 and Q2. What is specific to price. Your question is whether the stage-wise story is about price or about how any attribute comes to be weighted. We tested this by rerunning the trajectory with star rating as the focal attribute (App. C.6, Table A10). If the story were generic, star rating would follow the same path as price. It does not:

Price weight wStar-rating coefficient
Detectable at 399B, the first checkpointNoYes
Trend over stage 1 (Kendall τ)Rises (+0.73)Falls (−0.51)
AnnealingTracked-run mean falls to 0.59× the late-stage-1 mean (Table A9)Rises 4.2× by +5B and keeps rising
Base→SFT, plain1.26×2.5×
SFT→DPO, plain3.5×1.3×

The price preference is absent at the first checkpoint, builds over stage 1 and falls across annealing. The star-rating preference is present from the first checkpoint, weakens over stage 1 and strengthens across annealing. The two pretraining findings, a preference that builds over stage 1 and declines across annealing, are therefore specific to price among the attributes we can test. Post-training sharpening is shared. SFT and DPO raise every attribute coefficient, SFT the quality attributes most (2.5× to 8.2×) and DPO in the plain prompt price most. The comparison is descriptive, because star rating is listed rather than shuffled and the early checkpoints fail the validity rule.

On distance, your example, we have a smaller test. A separate instrument varies seven quantities one at a time at six checkpoints (App. D.4). Price has the largest normalized coefficient at every released stage. Distance is the only other quantity with a consistent sign, favoring the closer hotel (−0.65 at 3.775T, −0.27 at Base, −0.12 at SFT, −0.16 at DPO; App. C.6). Social biases such as gender are outside this instrument.

Why price? It is the attribute a purchasing agent trades against everything else. It is also the one where the capability (comparing numbers) and the preference (favoring the cheaper option) can be measured separately on the same checkpoints. The method itself, a randomized attribute with a decisiveness denominator, applies to any attribute.

W7. Novelty. We agree that comparing two prices and trading price against quality are different constructs and that saying so is not a finding. The finding is empirical. It runs against the simple expectation that a model which compares prices better also weighs price more.

We tested that expectation in two steps. First, 18 pretraining checkpoints all pick the lower price at least 97% of the time. Their relative price weight still ranges from 0.09 to 0.87. Equal accuracy therefore comes with very different weights on price. Second, accuracy at this level cannot rank the checkpoints, so we use a finer measure of the same skill. The comparison slope records how strongly the answer responds to the size of the price gap. Unlike accuracy, it does not saturate. If better comparison meant more weight on price, the checkpoints with a steeper slope would have a higher relative price weight. Across the 18 checkpoints they tend to have a lower one (Spearman −0.64). The slope instead follows decisiveness, the strength of the model's choices on the other attributes (0.69; Section 4). Checkpoint noise of the size we measured does not produce this reversal. If one shared factor drove both measures, the correlation would still be positive, about 0.57 (App. C.4).

The practical point is that an evaluation can show that a released agent compares prices correctly and still say nothing about the weight it puts on price. The paper supplies the measurement that does. The dissociation stays as a contribution, now stated as the measured result above. The other contributions are measured results as well. A lower-price preference is measurable on valid pretraining checkpoints, before any preference labels. SFT lowers the relative price weight by raising decisiveness. DPO raises the price weight in every format. The size of the relative change depends on the format and the run (Sections 4 and 5).

Q5. Other domains, more than two options. Both remain limitations. The Limitations now name other products, wordings and larger choice sets as future work.

Minor. Presentation and fit. The introduction is rewritten around the training-dynamics question, with almost every number moved out. The market motivation is one paragraph. The regulatory quote moves to the Discussion.

We would welcome further questions.

I thank the authors for the detailed response and the revisions. I appreciate applying the validity rule throughout the trajectory, retesting against checkpoint noise, and moving the cross-model results into the main text, which I believe add more clarity and reliability to the contribution. These changes address my concerns about checkpoint noise and the robustness of the annealing result. However, the following concerns remain.

  1. The claims may still go beyond what the valid evidence supports. I appreciate that the revised paper now acknowledges this directly. However, the checkpoints that pass the validity rule all lie at the very end of stage 1. As I read it, they show that a preference exists by the end of pretraining, but not that it develops during pretraining, since the rising trend relies mostly on checkpoints that fail the rule. Separation from the untrained band also seems to hold only after normalization: on the raw price weight, only 3 of the 15 stage-1 checkpoints lie above the band (Table A8). Finally, cross-model evidence for a pre-post-training preference comes only from OLMo-family base models. I suggest aligning the title, abstract and Section 4 heading with this narrower evidence.
  2. The comparison–weighting dissociation may partly reflect a phase transition. I appreciate that the authors moved from a conceptual argument to an empirical test, however I am not yet fully convinced by the evidence. Within stage 1, the comparison slope and relative price weight move together. The overall negative correlation therefore appears to come mainly from the transition into annealing. The slope rise appears in only one of the two annealing runs, it coincides with a change in answer format (App. C.4), and some of the included checkpoints fail the validity rule. Could the authors report this relationship within phases or on valid checkpoints only, and clarify whether the one-factor simulation accounts for this phase structure?
  3. I thank the authors for clarifying that the logarithmic form is an approximation. The new power fits, however, seem to raise a related question. After annealing and post-training, the behavioral exponent is −0.40 to −0.80, while the internal readout exponent stays near 0.04. If I understand correctly, the abstract’s statement that the internal scale has “the same shape the price response takes” would then hold mainly at the end of stage 1. It may help to qualify this statement or explain its basis.
  4. The headline post-training results depend on format. The response shows that the raw price weight rises at DPO in all four formats. However, the paper’s main metric, relative price weight, changes for example by 1.07× in chat template, 1.55× in system message)and 2.21× in plain prompt. The abstract’s “partly reverses” appears to rest mainly on the plain prompt. That is not the format the model was trained on, and the post-trained checkpoints are slot-locked in it. With two retraining seeds per arm giving 1.45× and 2.84×, the size of the DPO step remains uncertain. I suggest presenting the DPO result in the abstract and conclusion as format- and run-dependent, rather than as a stage-level finding.
Reviewer_eu9nOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2(−1)
Read review
Paper summary

This paper explores where models’ price sensitivity emerges, probing pretraining checkpoints, annealing, SFT, and DPO of a single model. The authors find strong evidence for a logarithmic representation of price, and they show that a detectable preference for lower prices is present before any post-training begins. Interestingly, post-training seems to enhance models’ preferences for non-price attributes.

Critical strengths

The writing is clear and lucid, probably in the 90th+ percentile of clarity, making it easy to follow.

The methodological choices, such as using OLMo in order to access pretraining checkpoints, and situating the task in a travel-booking setting, are sensible and sound.

The evidence presented is convincing and tells a coherent, robust story.

Critical weaknesses

The scope of the investigation is relatively narrow. It’s not entirely clear to what extent insights about price sensitivity generalize to other models, other preferences, or other traits. I’m persuaded that understanding LLM price sensitivity is important given that LLMs are and will be making consumer choices on behalf of users, yet I’m not sure what the findings as presented do to inform that in an actionable way.

Questions for authors

This paper is about a single model. Do we have any evidence that these findings generalize to other models?

This paper is specifically about price sensitivity. Do we have any evidence that these findings generalize to other properties or other preferences?

The motivation for this paper is anchored in agents that are “beginning to book and pay for users.” For companies building such agents, or users employing them, what insights are revealed by the evidence you present here?

I’m particularly interested in the finding about slot-lock, which seems to be a contribution in its own right. Are there actionable implications there for makers or users of decision-tools?

1 comment

Thank you for the careful summary and for the questions about generality and use.

Q1. Other models. The submission did rerun the weighting study on five other models and four retraining runs, but that evidence sat in an appendix table. You are right that the trajectory itself is one lineage. The cross-model results now sit in Section 5 as Table 3. We state explicitly what recurs and what does not:

PatternRecurs across models?
A lower-price preference before post-trainingYes, in all three base models that pass the validity rule
SFT and DPO raise decisivenessYes, 24 of 24 steps
DPO raises the raw price weightYes, 19 of 20 steps
Dividing by decisiveness narrows the differences between training objectivesYes: seven objectives trained from one checkpoint differ 5.9× in price weight but only 1.5× in relative price weight
How much the relative price weight changes at DPONo. It ranges from 0.73× to 1.10× in the released chat-template pairs and reaches 2.21× in OLMo 2 7B's plain prompt

The trajectory is the worked example. The rows above list what recurs in the models and formats we measured (the step counts include each model in more than one format).

Q2. Other properties and preferences. Price behaves differently from the quality attributes during pretraining. Post-training sharpens both. We reran the stage-wise analysis with star rating as the focal attribute on the same checkpoints (App. C.6, Table A10). The star coefficient is present at the first checkpoint and falls over stage 1 while the price weight rises. Across annealing the star coefficient rises more than fourfold while the price weight falls below its late-stage-1 mean. SFT and DPO raise both, by different factors, with DPO in the plain prompt amplifying price most. The comparison is descriptive, because star rating is listed rather than shuffled.

On the seven-quantity instrument (App. D.4), price has the largest normalized coefficient at every released stage. Distance is the only other quantity with a consistent sign. We do not claim that the trajectory generalizes to non-numerical traits. The method does. It needs a randomized attribute, a decisiveness denominator without that attribute and a paired task bootstrap.

Q3. What builders and users learn. Section 7 now has a paragraph on this. Four points:

  1. To locate where a preference came from, audit the trajectory and the training data as well as the released model, because the preference predates the preference labels.
  2. Report the price weight together with decisiveness, because SFT and DPO raised decisiveness in every model and run we measured. Name the model, the format, the version and any system message, because the size of the DPO change depends on all of these (Table 3).
  3. Treat the answer-letter diagnostics, letter mass and comparison accuracy, as a validity gate for any evaluation scored from answer letters. Report slot-lock beside every number.
  4. Under a logarithmic response, equal percentage discounts move the agent equally whatever their dollar amounts.

In interpretable units (Table A5), doubling one option's price lowers its model-implied choice probability by 1.0 percentage point at Base and by 3.7 to 5.3 points after DPO, depending on the format. DPO and Instruct trade one additional star for a 2.9× to 4.8× price multiple, against a 1.77× gradient of listed prices per star in the hotel pool. What weight an agent should give price is its user's decision. The audit exists so that it can be checked.

Q4. Slot-lock. We agree that it is a contribution in its own right and now present it that way. Three implications:

  • Slot-lock separates score sensitivity from reliable option selection. A model can be slot-locked on 97% of tasks and still carry a measurable price preference in the margin between the letters, so the margin is worth reporting. For an application that needs dependable selection, the lock itself is the finding. The headline contrasts keep their direction on the half of tasks with less position bias, though the small chat-template DPO step (1.07×) is 1.05× [0.99, 1.15] there (Table A7).
  • For anyone evaluating models from letter probabilities, slot-lock and letter mass should accompany every number. Appendix A.3 offers our rule as a default. Section 4 now states its consequence for our own data: 11 of the 15 stage-1 checkpoints fail it.
  • Lower slot-lock did not come with higher price sensitivity. In the chat template, DPO cut slot-lock from 97% to 45% while relative price weight stayed between 1.0 and 1.1 times its SFT value. The two should be audited separately.

We would welcome further questions.

Reviewer_UkhzOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates the LLM behaviour of price sensitivity, which is an essential topic in economics. The author explores when this behaviour is trained into the model. This paper uses OLMo 2 7B, an open-source model for which training checkpoints are available. The author uses 600 NYC hotel choices to find the reaction of the LLM at each stage, including pretraining, annealing, SFT, DPO, and Instruct. The main claims are that price-comparison accuracy does not reveal how much price drives a choice, and that a lower-price preference already emerges during pretraining, before DPO. The paper also links this to a log-like internal number scale and to price comparisons in the pretraining data.

Critical strengths
  1. Price sensitivity is an important topic in economics. Exploring the LLM's behaviour is an interesting question. The author also explores which stage actually learns, starting from pretraining, annealing, SFT, DPO, and instruction tuning.
  2. Some of the experimental setup is good. For example, considering the selection bias of LLMs that the model have preference to option A over B. And the price shuffle is also a good design.
Critical weaknesses
  1. The intro is all about agents shopping and paying for users. The experiments are something else. There’s no user in the tasks. No budget, no preferences. So we can’t tell whether the agent serves anyone’s interest. The abstract says price sensitivity shapes purchase advice, but no advice is ever generated. Additionally, I'm still wondering about the purpose of this paper. Do we want to align the LLM with the human? Or investigate whether there is a difference between LLMs and humans? If so, conducting some analysis of it is necessary.
  2. The paper is hard to read, and the writing needs to be improved. Also, it may have some missing references. The intro (lines 81–105) throws a dozen-plus numbers before anything is defined. Use lots of colons and semicolons that make the sentences broken. For the missing reference:

a) In lines 146- 153, you mentioned selection bias and cite Zheng (2024). Have you ever considered mitigating the bias? Such research are talking about it.

[1] Unveiling Selection Biases: Exploring Order and Token Sensitivity in Large Language Models (ACL 2024)

[2] Mitigating Selection Bias with Node Pruning and Auxiliary Options (ACL 2025)

b) Some price / economic related LLM research can also be considered as reference

[3] How Does Cognitive Bias Affect Large Language Models? A Case Study on the Anchoring Effect in Price Negotiation Simulations (EMNLP 2025)

[4] Bias Beware: The Impact of Cognitive Biases on LLM-Driven Product Recommendations (EMNLP 2025)

[5] How price sensitive is AI? Benchmarking LLM-generated willingness to pay in pricing analytics

[6] Pay What LLM Wants: Can LLM Simulate Economics Experiment with 522 Real-human Persona?

Questions for authors
  1. Why is the validity rule skipped for the OLMo 2 7B pretraining checkpoints?
  2. With checkpoint noise this large, how much agreement between the comparison slope and price weight would you even expect if they shared one latent factor? Could the three annealing runs work as replicates here?
  3. Would a human conjoint study on the same 600 tasks give a reference point for r? Any human check on the corpus judge? And could it record which option each comparison favors?
  4. What price weight do you think an agent acting for someone should have? Is the log-shaped, human-like pattern actually desirable?
4 comments

(1/3)

Thank you for the careful reading. Two of your points changed the paper most: the gap between the introduction and the experiments (W1) and the readability of the text (W2). References below are to the revised PDF.

We use a few terms throughout. The price weight is how strongly a lower price pulls the model toward a hotel. The relative price weight (r) compares that pull with the pull of the other attributes, such as star rating and review score. Pretraining has two parts, stage 1 and annealing. The released checkpoints are Base (the end of pretraining), SFT, DPO and Instruct.

W1. Purpose and framing. You asked whether the goal is to align models with humans or to compare models with humans. Neither is the goal. The paper contains no human choices. It asks two questions about training. At which stage does a model begin to prefer cheaper options? Which later stages change that preference? The answers matter for anyone who wants to check a shopping agent. To check or change the weight an agent puts on price, one must measure that weight and know which training stage set it.

You noted that the introduction is about shopping agents while the experiments are something else. The revised introduction opens with what the experiments measure, a model's price preference traced through training. It keeps one paragraph on shopping agents as motivation.

You are right that the tasks name no user, no budget and no preferences, so they cannot show whether an agent serves a user's interest. They measure the model's default, that is, the hotel it favors when it is told nothing about the user. With that number, a user or a regulator can compare the default with what the user wants (see Q4).

The abstract no longer mentions purchase advice, because no advice is generated. We read the model's preference from the probabilities it gives to the two answer letters, A and B. A pretraining checkpoint cannot write a recommendation, so following the preference through training needs a measurement that does not depend on generated text. For the released checkpoints we checked how this measurement relates to the model's output. On 98.6% to 100% of prompts, the higher-scored letter is also the model's most probable first token, so a generated answer would begin with it (App. D.2). We did not generate and evaluate recommendation text. The Limitations list that evaluation as a next step.

W2. Readability. The introduction no longer leads with numbers. It states the findings in words and defines each measure before using it. The numbers appear in Sections 3 to 5. On sentence structure, the rewritten introduction has no sentences joined by semicolons and fewer colons.

W3. References. All six are added. You asked whether we considered mitigating selection bias. We did. We chose to cancel the bias in the measurement and leave the model unchanged. Every task is asked twice, with the two hotels swapped between positions A and B. A model that always leans toward the letter A adds the same push in both versions. The push cancels when the two versions are combined. A mitigation method would change the model's answer probabilities, which are what we measure. Mitigation is the right tool when the goal is a model that selects options reliably. Our goal is to measure the model as it is. Section 2 now explains this choice and cites both selection-bias papers. As a check, we repeated the analysis on the half of the tasks where the selection bias is weaker. The main results hold there (Table A7). The four papers on economic behavior are discussed in Related Work.

(2/3)

Q1. Why the validity rule was not applied to the pretraining checkpoints. The submission used the rule only to choose among released models and prompt formats. For the pretraining checkpoints it reported every score and stated which ones failed. We agree that the rule should apply there too. The revision applies it (App. A.3).

The rule has two conditions. First, the checkpoint must answer with a letter. At least half of its probability at the answer position must fall on A or B. Second, it must compare prices correctly. It must pick the cheaper price in at least 90% of the questions that ask which of two prices is lower. Of the 15 stage-1 checkpoints, 11 fail and four pass. The four are among the last five of stage 1 (3.880T to 3.896T training tokens, where T is trillion). The failing checkpoints still appear in Figure 3, shaded. The pretraining claims now rest on the four that pass. This changes three things.

  • The main finding holds. The model prefers the cheaper hotel during pretraining, before DPO trains it on preference labels. On the four passing checkpoints the relative price weight is 0.60 to 0.92. Untrained networks, which have the same architecture and random parameters, reach at most 0.13.
  • The first appearance of the preference is dated later. The submission dated it at 3.146T. The revision says "by 3.880T". The checkpoints behind the earlier date compare prices correctly but put less than half of their probability on the letters.
  • The fall during annealing holds. The paper reports that the relative price weight falls during annealing. Averaged over the annealing checkpoints we follow, it is about a third (0.34 times) of its level on the four passing checkpoints (App. C.5).

The post-training results are unchanged, because the four released checkpoints pass the rule.

Q2. Checkpoint noise, expected agreement, replicates. The noise is large, as you say. Within stage 1 and within annealing, the relative price weight varies across checkpoints 2.7 to 3.4 times as much as the sampling of the 600 tasks alone would explain (App. A.4).

(a) How much agreement would one latent factor produce? The question concerns two measures. One is the relative price weight. The other is the comparison slope, which is how strongly the model's answer responds to the size of a price gap in questions that ask only which of two prices is lower. If one latent factor drove both, their correlation across checkpoints would be about 0.57. The observed correlation is −0.64.

We obtained 0.57 by simulation. We assumed one latent factor behind both measures and added independent noise at each checkpoint. We estimated the size of the noise from neighboring checkpoints. We ran 10,000 simulations over the 18 pretraining checkpoints that answer at least 97% of the price comparisons correctly. The median simulated correlation is 0.57. Noise lowers the correlation. It did not produce a value as low as −0.64 in any of the 10,000 simulations. The two measures therefore do not move together as one latent factor would predict. This supports the paper's point that comparing prices well does not tell how much weight a model puts on price. The simulation assumes independent noise at each checkpoint. Other kinds of noise are not ruled out (App. C.4).

(b) Can the annealing runs serve as replicates? Partly. There are three annealing runs. One, which we call the tracked run, has nine scored checkpoints. A second has only its endpoint scored. The third is not scored. Base averages the parameters of all three runs, so it is not an independent replicate.

The fall in relative price weight appears at both scored endpoints and at Base. There it is 0.35 to 0.54 times its average over the last five stage-1 checkpoints (App. C.5). The direction replicates and the size differs. A second finding, the rise of the comparison slope during annealing, does not replicate. It appears in the tracked run only (Section 4, App. C.4).

(3/3)

Q3. Human reference points and the corpus judge.

(a) Would a human study on the same 600 tasks give a reference point for r? Yes. The paper has no such study. The Limitations name it as the next step.

In its place, Table A5 expresses the model's price sensitivity in two everyday units. The first is the drop in the probability of choosing a hotel when its price doubles. A larger drop means the model is more sensitive to price. The second is the price multiple the model accepts for one extra star. A value of 4.8× means that one extra star offsets a price 4.8 times higher. A larger multiple means the model gives up more money for quality.

CheckpointDrop in choice probability when the price doubles (percentage points)Price multiple accepted for one extra star
End of stage 10.61.5×
Base1.08.8×
DPO, plain prompt3.74.8×
DPO, chat template5.33.3×

DPO is scored in two prompt formats, the plain task text and the model's chat template. For comparison, in the hotel listings one extra star goes with a price 1.77 times higher. These units show the size of the effect. They do not replace a human benchmark.

(b) Is there a human check on the corpus judge? No. (c) Could it record which option each comparison favors? It could, but in this study it did not. The revision states both limits where the corpus analysis is reported (App. E.1). It presents the finding on price comparisons in the pretraining text as exploratory.

Q4. What weight should an agent have? Is a log-shaped response desirable? Our view is that the weight should follow the user's own preferences. No single value is right for every user, so the paper does not prescribe one. It supplies the measurement. A user, a business or a regulator can then compare an agent's weight with their own standard.

We take no position on whether the log shape is desirable. It is worth knowing because it has a practical consequence. The agent responds to percentage discounts. The same percentage discount moves it equally on a cheap room and on an expensive room. A seller of an expensive room therefore needs a larger dollar discount to move the agent by the same amount. The new paragraph "What this means for builders and users" in Section 7 makes these points.

We would welcome further questions.

Reviewer_UkhzOpenReview ↗

Reply to the author response

Thanks to the authors for the response and the revised version.

However, I still have two concerns. First, after applying the validity rule, only four checkpoints are left, and all of them sit at the very end of stage 1. To me, this shows that the model already prefers lower prices by the end of pretraining, but it doesn't really tell us how that preference forms during pretraining. So I think the "origins" framing still claims more than the evidence supports. Reframing the paper as a training-dynamics study is clearer, but it also makes the contribution narrower than before. Second, Q3 is still open.

For these reasons, I'll keep my original score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the revision for the AC and reviewers

Summary of the revision (revised PDF uploaded). We thank the three reviewers. The reviews identified three issues: (1) the reliability of the early-checkpoint measurements, (2) the distinction between training-stage changes and checkpoint noise, (3) the generality of the findings across models and prompt formats. We addressed each with additional analyses of the model outputs saved for the submission. One clause of the abstract is reworded to match the measured outcome.

The revised paper makes narrower but better-supported claims:

  • The lower-price preference is measurable during pretraining. Eleven of the 15 stage-1 checkpoints fail our own validity rule. The rule is now applied to the trajectory. The pretraining claims rest on the four checkpoints that pass (relative price weight 0.60 to 0.92). The earlier scores are reported as descriptive. The onset moves from 3.146T to "by 3.880T".
  • The annealing decline is now tested across groups of checkpoints. All 11 annealing states lie below the mean of the late stage-1 checkpoints (relative price weight 0.32 [0.25, 0.40] times the last-five mean, 0.34 [0.27, 0.42] times the valid-only mean). A shared-factor noise model shows that the disagreement between comparison slope and relative price weight is not what checkpoint noise alone would produce.
  • The direction of the post-training effects recurs across models. Their size does not. The cross-model results move into the main text (Table 3). Decisiveness rises in every SFT and DPO step we measured. DPO raises the price weight in 19 of 20. The size of the DPO change in relative price weight depends on the format and the run. The plain-prompt doubling in OLMo 2 7B is the largest among the released checkpoints, while the released chat-template steps lie between 0.73× and 1.10×. Two retraining seeds differ 1.96-fold.
  • The framing is now the training-dynamics question. The introduction is rewritten and "origin" is defined narrowly. Almost every number leaves the introduction. The Discussion says what builders and users should take from the results.

The table lists the changes by concern, with their locations.

Concern (reviewers)What changedWhere
Framing and purpose; dense introduction (Ukhz W1–W2, FMLK minor)Introduction rewritten; numbers moved to Sec. 3–5; "origin" defined narrowly; abstract clause rewordedSec. 1; abstract
Stage-1 checkpoints fail the validity rule (FMLK W2/Q1, Ukhz Q1)Rule applied; failing checkpoints shaded; claims restated on valid checkpoints; half-split and regression-free checksSec. 4; Fig. 3; App. A.3, C.1, Tables A6–A7
Checkpoint noise; single-run steps (FMLK W3, Ukhz Q2)Spread quantified; multi-checkpoint baseline with paired intervals; shared-factor noise model; seed spread statedSec. 4–5; App. A.4, C.4, C.5, Table A9
One lineage (FMLK W1, eu9n Q1)Cross-model steps promoted to the main text; portable claims stated as suchSec. 5, Table 3; App. D.3
Letter log-odds; format dependence (FMLK W4, Ukhz W1)Format dependence reported as a result; answer-letter diagnostics proposed as a validity gateSec. 5, Table 3; Sec. 7; App. D.2
Modeling assumptions; log shape (FMLK W5, Q3)Headline contrasts under six specifications; behavioral Box–Cox exponent; effect sizes in choice probabilities and willingness to paySec. 3; App. B.3, Tables A4–A5
What is specific to price (FMLK W6/Q2, eu9n Q2)Star rating as the focal attribute on the same checkpoints, reported as descriptiveSec. 4–5; App. C.6, Table A10
Novelty of "accuracy does not reveal weight" (FMLK W7)Contributions restated as measured results; early below-chance accuracy explainedSec. 1; App. C.2
Untrained networks (FMLK Q4)Term defined; two-seed agreement; behavioral values labeled as floorsSec. 3; App. B.2
Actionability; slot-lock (eu9n Q3–Q4, Ukhz Q4)"What this means for builders and users" paragraphSec. 7
Missing references (Ukhz W3)All six added and discussedSec. 2, Sec. 6
Human reference points; judge validation (Ukhz Q3)Model-implied effect sizes beside the listed-price gradient; judge limits stated; human choices named as future workApp. B.3, Table A5; App. E.1; Sec. 7

What we did not do. A human conjoint study and a human check of the corpus judge were not possible within the discussion window. The Limitations name human choices on the same tasks as the next step. Generated choices, a second product domain and a second trajectory at another model size each need new model evaluations. Isolating the source of the preference needs controlled retraining. All are stated as open questions in the Limitations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.