- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper reports empathic accommodation: when told only the user's role, with no stated opinion, LLMs shift information in the user's favor. The evidence is five studies and the authors conclude that models track users' inferred emotional stakes.
Critical strengths
S1. The paper documents a consequential, hard-to-notice distortion across several domains and model families with large samples. The medical study is the most striking to me.
S2. The paper is transparent, which makes the claims auditable and the proposed mitigation easy to re-test.
Critical weaknesses
W1. The role cues are explicit and task-relevant, so the design cannot tell empathic accommodation apart from legitimate personalization to the user's needs. The SI prompts state the role directly:
Study 1: "Owns a Peloton bike" in a user-memory block. This is consumer ownership, not brand ownership as the main text says.
Study 2: "Identifies as a Democrat."
Study 4: "The patient label will be shared directly with the patient."
Study 5: "The user has already invested $10,000."
Under each cue, a different answer can be a reasonable response to what the user needs: patient-facing wording, a hold-vs-buy framing, or experience-focused summaries for existing customers. It need not reflect an inference about the user's feelings. Relatedly, several tasks are not clearly "objective": Study 2 asks whether a policy is "good or bad", so the premise that a faithful answer should not depend on who is asking is contestable. Without conditions that separate the user's information needs from their emotional stakes, the shift is shown but the mechanism in the title is not. Examples of such conditions: matched needs with a reversed stake (e.g., a user who is short the stock), a length-matched neutral cue, or contrasting "do not tailor to the user's needs" with "do not consider how the user will feel".
W2. The model selection is dated and limited. GPT-4o (2024), Gemini 2.5, and Llama 3.3 are used, mostly in mini/Flash/Lite/Haiku tiers, and thinking mode is disabled in Studies 2–5. Study 1 itself finds that thinking attenuates the effect. The motivating health and finance use cases are where users are most likely to consult current flagship reasoning models, so whether the effect persists in those models is central to the paper's practical claim.
W3. Most results are reported as long inline strings of statistics, and the main text has only two figures. A cross-study summary table and per-model effect-size plots would make the evidence much easier to evaluate.
Questions for authors
See weaknesses. I would be happy to improve my score if my questions could be reasonably resolved.