- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper argues that when a model is trained using an AI judge, it can learn to talk down to certain users, because the judge rewards this and the users themselves never get a say. The authors test this using age. They take everyday requests where age doesn't change the right answer, and add either "I'm 45" or "I'm 79" to the request.
The study has four parts. First, six language models give older users roughly the same content, but more often mention their age, suggest asking family for help, or add reassurance. Second, AI judges comparing the same two answers are more likely to pick the one written for the older user when they are told the user is 79. Third, training a model (Qwen3-8B) for three rounds with such a judge raises the share of answers that mention a 79-year-old's age from 30% to 68.5%, while hiding the age from the judge prevents most of this increase. Finally, adults aged 65 and over, comparing the same pairs of answers without knowing which was which, show no preference for the answers written for the older user, and rate them as only slightly more likely to treat the reader as less capable.
Critical strengths
-
A clean test of the judge. Each pair of answers is judged twice, once with the judge told the user is 45 and once told 79, with the answers unchanged and in both orders. This shows clearly that the judge's choice depends on the stated age. It matters to my assessment because it is the part of the paper that holds up well, and it is a reusable way to check judges for other user attributes.
-
A control that identifies the cause of the amplification. Training with the same setup but with the age hidden from the judge barely changes how often the model mentions age, while training with the age visible more than doubles it. This shows that the increase comes from the judge seeing the age, and it gives a simple, practical fix. It is the most concrete result in the paper, though as I note below it is also expected.
Critical weaknesses
-
The findings are largely expected. Language models and AI judges learn the associations people hold about age from their training data, so it is not surprising that they respond to a stated age, and biases of this kind have been documented extensively. That training amplifies whatever an AI judge prefers is also a known pattern, and it would happen in the same way for a preference that is good for users. This matters most to my assessment because it leaves the paper's contribution resting on its claim of harm (point 2). The narrow scope adds to this: one attribute in one direction, markers taken from research on how people talk to older adults (so other biases, for example toward younger users, would not show up), one small trained model, and one judge chosen because it showed the shift.
-
The harm to older users is assumed rather than shown. The paper calls this a trap because the judge rewards something older users do not want, but its recipient study does not support this. Asked which answer they would rather receive, older raters chose the one written for the older user 49% of the time. The primary result (+4.8 pp on "treats the reader as less capable") is below the authors' own 10 pp threshold, and it and the secondary result (+4.4 pp) barely exclude zero. Under the prompt used for training, the judge's preference does not differ significantly from the raters'. Only older adults were recruited, and in most of the pairs they compared (106 of 144) neither answer mentions age. This is consequential because it is the one finding that would make the paper new, and the title and abstract present it more strongly than the results allow.
Questions for authors
- In most of the pairs shown to recipients, neither answer mentions age. Could you report the recipient results on the 30 pairs where only the older user's answer mentions age? With only about 50 ratings on these pairs this would be exploratory, but it would show whether the indifference holds where the behavior is actually present. A study designed around this behavior (question 4) would be needed to settle it.
- On pairs where neither answer mentions age, which differences between the answers explain the judge's shift (reassurance, family referrals, length, reading level, content)? This would show whether the judge rewards a condescending tone or something harmless.
- The reproducibility statement says everything is available at the linked repository, and the paper refers to the code release for the full register patterns. When I checked, the repository only said "Sorry, we are still preparing the code. Please check back soon!". Could you make the code, rubrics, register patterns and recipient data available during the review period?
- The paper does not report what recipient study participants were paid, or whether the study received ethics approval or an exemption. Could you state the compensation (and the equivalent hourly rate) and the approval status?
Response to Reviewer vbcJ
We thank Reviewer vbcJ for the comments and review, and for the questions that led to the correction in our general response #1.
W1 (Expected findings and narrow scope).
W2 ("The harm to older users is assumed rather than shown").
We agree with the facts and with the criticism of the wording. The primary estimate is below our threshold, older adults do not prefer the favored answer (49%), and under the training prompt the judge does not differ from them, so the study shows a level of preference rather than a harm. Please refer to our general responses #5 and #6. We defined the term by the loop of Section 3.1, stated there that perceived condescension is not established, and reworded the abstract and the Introduction to match.
Questions for authors.
Q1 (The pairs in which only the 79-answer mentions age). Please refer to our general response #5 and Table 11 of the revised manuscript. On the 39 such pairs (65 ratings), the estimate is +20.8 points, with no preference for receiving that answer. As the reviewer anticipated, this is exploratory.
Q2 (The judge shift where neither answer mentions age). Part of it came from 12 pairs that our rule misclassified (general response #1). On the remaining 94 pairs, the two open-weight judges are close to zero over five runs, and GPT-5.5 and Claude Sonnet 5 keep a small shift. In an exploratory regression, length, reading grade, list structure, reassurance words, referral and rubric coverage explain almost none of it.
Q3 (Repository). The anonymous repository now contains the analysis code with the full register patterns, the rubrics, the generations and the 144 natural pairs with all judge outputs, and its README links the pseudonymized recipient ratings and the trained adapters. We apologize for the late upload.
Q4 (Compensation and ethics review). Participants were paid £7.87 for a study estimated at 20 minutes (£24.85 per hour at the median completion time of 19 minutes), including the three participants excluded for failing the final attention check. The study was an online survey of adult volunteers on Prolific, with a consent page and no names or contact details collected. We added these points to a new Ethics Statement, with the details in Appendix G.
Please let us know if you have any other comments or questions!
Sincerely, Authors