- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
The paper asks a genuinely interesting question, which is, given that LLMs appear to be significantly more likely to buy something at, e.g., $4.99 than $5.00, does this reflect a "perceptual" emphasis on the leftmost digit, or a "semantic" effect coming from the distribution of prices in the pretraining corpus?
To establish the premise, they use prompting to determine models' willingness to purchase products at various prices, looking at various one-cent increases compared to the x.99->(x+1).00 boundary, showing that there is indeed a significant "threshold"-like effect.
They use five different experiments to attempt to distinguish whether this effect is "perceptual" or "semantic". They use visual models to test the "perceptual" thesis in a visual setting. They use two attempts to disentangle a model's positive preference for round numbers with its negative aversion to whole-dollar threshold jumps. Finally they test with different prompts and different currencies. The authors argue broadly for the "semantic" explanation, i.e., that LLMs are influenced by the prevalence of prices in pretraining.
Critical strengths
The round-dollar threshold effect is very clearly and convincingly demonstrated, and is an interesting finding in its own right.
The question of whether this is "perceptual" or "semantic" is a fruitful distinction to make, and the comparisons/distinctions with human studies are fruitful.
Counting the prevalence of prices in pretraining corpora is a useful contribution.
The idea of physically varying the size of a price in a graphic is an interesting and creative approach.
Critical weaknesses
Overall, the paper is more difficult to read than it needs to be. The abstract uses language that doesn't make sense until one reads the paper; figures use language that doesn't make sense until one reads the appendix; etc.
Overall, the figures use confusing and dubious conventions. Figure 6 says "gpt-4o-mini shows a monotonic gradient", but the figure is not monotonic (it goes up from 100% to 70% then down again to 50%). Figure 7 is a linear regression using an arbitrary "roundness" score from 0-3 on the x-axis, throwing away the "0" scores (which are the vast majority of the data), and running a line from the mean "1" score through one single "2" datum to extrapolate the single "3". It's hard to take this seriously as having the level of rigor it seems to be purporting.
The prompting intervention does not seem convincing at all to me. Simply saying "retail prices in the country conventionally end in .77" breaks ecological validity, and I'm not sure that the resulting outputs are particularly meaningful. A much better approach IMO would be to use SFT to actually train the model on different price distributions.
The vision data are n=3, the three models disagree wildly, including in the direction of the effect, and so I'm not sure what story we are even meant to take away from them.
§4.4 claims, "Presenting the same prices in 5 formats (standard, spaced, digit-separated, ISO, spelled out in words) gives a pooled estimate of −0.26 [−3.42, 2.99]. The word rendering, which removes the digit structure entirely, gives 6.28. The profile therefore persists across representations". Consulting against Table 43, I note that +6.28 this points in the opposite direction of digitspace, plain, and spaced, which are all negative. The main text claims that a profile "persists across representations" but the data appear to say precisely the opposite.
Questions for authors
In F6, you claim that gpt-4o-mini shows a monotonic gradient, but the image appears to show the opposite. Am I misinterpreting the image?
If the word rendering in T43 is +6.28 and digitspace, plain, and spaced are all negative, then in what sense is it the case that a shared profile "persists across representations"?
2 comments
Reply to Reviewer tyQh [2/2]
W2: Figure 7 and the roundness score. Thank you for raising this concern. We clarify first that no observations with were dropped: every step ends at a multiple of ten, so these endings do not occur as destinations, but they enter the analysis as starting points and through within-block detrending. On the broader concern about the roundness score, we agree that the ordinal fit relies on limited support at higher values: eight placebo steps have , and only one has , making extrapolation to weakly determined. We originally used this simple specification to put the effect on a common scale across experiments, but it is not the only way we assess the effect. The earlier appendix reported a few alternative specifications. We have added another approach and made these complementary analyses more prominent in the revision.
Specifically, Section 6 now foregrounds two estimates that require no roundness score, with their raw data shown in the revised Figure 7:
- Matched design (E0): Two step-10 jumps both end at .00 and differ only in whether the leftmost digit changes. Their difference is −6.14 pp [−8.36, −3.86], negative in 7 of 9 models.
- Conservative bound (core sweep): The step-10 effect minus the mean of the nine placebo effects is −2.50 pp [−3.91, −1.28]. Under the assumption that a whole-dollar ending is at least as attractive a destination as any within-dollar ending, consistent with .00 having the largest effect in the ending profile, this contrast is an upper bound on the true effect and therefore understates its magnitude when negative.
The ordinal estimate (−3.25 pp [−5.64, −0.99]) remains in Table 38 as one of five specifications, serving as a common-scale summary rather than the sole basis for our conclusion. We also report an alternative specification using the observed corpus frequency of each ending instead of the ordinal score; this yields −3.52 pp [−5.44, −1.73]. The corresponding ordinal-fit figure has been moved to Appendix C (Figure 11), with its caption clarifying the limited support at higher roundness values.
W3: The stated-convention intervention. We thank the reviewer for raising this concern. The fictional convention was intentional: using an unfamiliar ending such as .33 or .77 allowed us to test whether models would adapt their preferences to an explicitly stated pricing convention, with less influence from familiar price-ending associations. The experiment therefore tests whether ending preferences are responsive to stated conventions, rather than whether the native preference for .99 originates in training data. We have clarified this distinction in the revised Section 4.1.
Two further results help contextualise the intervention. Naming .00, a real and common price ending, moves the dollar-step estimate from −0.87 [−7.16, 7.51] to +28.71 [21.73, 36.81] (Table 28). Moreover, the change in sign across non-price domains (Table 15) and the disappearance of the penalty when currency is written as a word (Table 19) occur without any convention-setting instruction. Finally, we agree that fine-tuning on controlled price distributions would provide a more direct test of the training-data explanation and thank the reviewer for the suggestion. The revised Discussion identifies this as an important direction for future work.
W4: vision with three models. We agree with your description. The three models disagree: the slope is significant only for gpt-4o-mini (25.53 [0.88, 50.45]), the other two give −0.21 [−2.93, 2.56] and −2.74 [−18.02, 11.90], and the pooled slope has an interval that includes zero (5.65 [−2.05, 14.95]). These results support a limited conclusion: a perceptual component is detectable in one model, but the evidence does not suggest that it is a general property of vision models. We have revised Contribution 3 and Section 5 to reflect this clearly.
We hope these revisions address your concerns, and we would be glad to clarify anything further.
Reply to Reviewer tyQh [1/2]
Thank you for the close reading and the specific questions, which led to clear improvements in the revision. We answer your two questions first. Section, table, and figure numbers refer to the revised PDF.
Q1 and W2 (Figure 6). Your reading is right, and "monotonic" was the wrong word. The
gpt-4o-miniestimates at 100%, 70%, 50%, and 35% cents height are −0.19, +3.31, −3.74, and −18.76. The effect concentrates at the smallest size, and the slope of a line fitted to the four estimates is 25.53 pp per unit of height [0.88, 50.45], driven by the 35% condition. The revised Section 5 reports the four values, and we also updated the Figure 6 caption.Q2 and W5 (word rendering). We thank the reviewer for catching this inconsistency. Our earlier statement that "the profile therefore persists across representations" was too broad: the positive estimate for the word-rendered condition does not support this conclusion. We have removed the statement and revised Section 4.4 to describe the results for each representation separately, without implying a common pattern across formats (Table 19; Appendix Table 44, previously Table 43).
W1: readability. Thanks, we have revised the paper throughout with this in mind. The abstract now describes each design in plain words, so it can be read without the paper. Every figure caption is self-contained, and figure and table labels that used internal shorthand ("installed", "arm", "naive") now use plain descriptions. We define terms such as block, cell, and the roundness score where they first appear, and the appendix now provides the exact prompts for the non-price quantities and the stated conventions.