acceptodds
Withdrawn from ICLR 2027

Characterizing Age Conditioned Affective Reasoning in Vision Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision language models now support personalized and emotionally aware interactions, but demographic conditioning may reflect stereotypes rather than real human variation. We test this for age-conditioned affect using all 900 OASIS images and 17 proprietary and open-weight VLMs under no-age, young, middle, and older-adult conditions. We collect valence and arousal ratings, emotion labels, and open-ended descriptions, and compare them with participant-level judgments from 822 human raters. Models show high baseline competence (median valence ) and strong age responsiveness: 16 of 17 lower valence and all 17 lower arousal for older personas. Yet image-specific fidelity remains near zero ( to , ceiling ), even though model age effects agree with one another (mean pairwise ). The mismatch is content-dependent: on pleasant sensitive images, models lower older-persona valence by points while older human raters shift in the opposite direction (), and older-persona descriptions use more memory and vulnerability language. We also test six agent-based interventions on 300 images; reasoning alone does not improve alignment, while human grounding improves the broad direction of age effects but leaves substantial image-specific error, with the best agent reaching . These results show that matching average ratings or producing a plausible age persona is not enough: faithful personalization requires models to change on the same stimuli, and in the same direction, as the human group they aim to represent. Code and all model responses are available in the supplementary material.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Vision-Language Models, Affective Computing, Emotion Recognition, Demographic Bias, Age Conditioning, Multimodal Large Language Models, Human-AI Alignment, Computational Psychology, Personalization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_pZ9JOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper evaluates whether telling a VLM to answer as a young, middle-aged, or older adult changes its ratings of the 900 OASIS images the way human age groups actually differ. 17 models match pooled human valence and shift strongly under the older persona, but their per-image shifts do not correlate with human per-image age differences, while they correlate with each other. On pleasant sensitive images the models move opposite to older human raters.

Critical strengths
  1. The paper's framing asks the right question (same images, same direction as humans) and controls for the obvious confound (pleasantness explains most of the human age effect), with ceilings, FDR, and bootstraps.
  2. The negative result is clean and useful: a shared, reliable model “older persona” that is orthogonal to human data and reverses it on a specific content class.
Critical weaknesses
  1. The headline result (“no model tracks human age effects”) rests on a target that is not measurable and on the wrong comparison. The human reference is thin and inconsistently reported: 25 raters aged 60+ for valence (16 for arousal). The per-image human effect is a linear age slope fit mostly on under-60 raters, while the model effect is a step at 60, and the attainable ceiling is 0.86 in the abstract/§4.1 but 0.60 in Table 1/§4.4. Moreover, the human age effect is SD ≈ 0.1 points per SD of age on a 7-point scale, in contrast to model persona shifts which are 0.2-1.5 points. The clearest results are the magnitude mismatch (caricature) and the sensitive-content gate, not r_part against a near-noise target on which the framing rests. Finally, the wrong comparison is performed. OASIS older adults rated images as themselves, where the models were asked to imagine being 60+. The right control for a role-play prompt is humans asked to role-play an older rater (the classic stereotype paradigm), who would likely also produce frailty/nostalgia and miss item-level effects. Without that baseline, the paper cannot say whether VLMs are worse than people at perspective-taking or merely as bad, and the “stricter test for personalization” demands item-level fidelity no evidence shows humans can deliver.

  2. The stereotype and mitigation results are confounded or inconclusive. The prompt induces the “stereotype language.” §B.2.1 explicitly requests the description “from the perspective of the specified age group.” Asking for perspective-taking and then counting nostalgia and grandchildren as stereotype content is partly circular. Instead, the rating fields would be the cleaner evidence. Furthermore, the mitigation study gets headline status but is inconclusive: one model, 300 images, best arm chosen post hoc, gains not significant after correction, and the grounded agents’ direction agreement matches a pleasantness-only rule.

Questions for authors
  1. Which ceiling applies, and what are the corrected “% of attainable” numbers?
  2. Does a per-image 60+ vs. under-60 contrast (instead of a linear slope) change the fidelity result?
  3. Replicate the best mitigation arm on the held-out 600 images?
  4. What do young humans asked to rate “as a 60+ adult” produce on these images (even a small pilot)?
3 comments
AuthorsOpenReview ↗

Response to Reviewer pZ9J (Part 1/3)

We thank Reviewer pZ9J for a close technical reading, and for recognizing that the framing asks the right question (same images, same direction as humans), that the paper controls the main confound (pleasantness) with ceilings, FDR, and bootstraps, and that the negative result is clean: a shared, reliable older persona that is orthogonal to human data and reverses it on one content class. The review questions whether the human target is measurable, whether a step-shaped model response should be compared with a linear human slope, which ceiling applies, whether role-play humans are the correct comparison, and whether the stereotype and mitigation results are confounded. We ran new analyses for each; the central result holds, and we corrected the ceiling.


Weaknesses

  • W1a / Q2. Is the human target measurable, and does a 60+ contrast change the result?

    • Response: We agree these are the two conditions the headline depends on, so we tested both directly.

      Measurability. Splitting raters into two disjoint halves, estimating the pleasantness-adjusted age effect in each, and applying Spearman-Brown gives a split-half reliability of 0.39 [0.26, 0.50] and a ceiling of 0.62. This is an estimate independent of the model-based ceiling (0.595) and agrees with it. A power simulation planted model effects of known fidelity into the real OASIS design (same raters, ages, lists, noise levels, integer ratings) and ran our pipeline 300 times per setting: power is 41% at a true fidelity of 0.10, 92% at 0.20, and 100% at 0.30. A true fidelity of 0.30 would produce the best model's observed 0.093 in 1.7% of runs. The target is measurable, and the design detects a modest fidelity.

      Step vs slope (Q2). We re-estimated the human effect ten ways and recomputed r_part for all 17 models.

      Table R1. Fidelity under alternative estimators of the human age effect (valence).

      Human age effect estimated asOlder ratersMedian r_partModels CI > 0 (images)Split-half ceiling
      Linear slope, pooled (paper)all 4130.03720.62
      60+ vs under-60, pooled250.04360.09
      60+ vs under-60, raw means250.04770.00
      50+ vs under-50, pooled780.00300.52
      Hinge at 45 (nonlinear)all 4130.04640.47

      The 60+ contrast gives median r_part 0.047, so it does not change the result. It also cannot serve as a target: with 25 older raters, the 60+ vs under-60 contrast has reliability near zero (ceiling 0.00 to 0.09). Its weak positive values do not survive resampling raters and images (0 of 17 models above zero; consensus 0.126 [−0.034, 0.209]). The 50+ contrast, with 78 raters and a usable ceiling of 0.52, gives 0.003. Two further facts justify the slope. First, the human data are graded, not a step: a pure step at 60 would give the linear slope an SD of 0.016 across images, and the observed SD is 0.106. Second, the model side is already a contrast: Eq. 2 weights the three personas by (−43, −1, 44)/1893, so ΔM is a scaled older-minus-young difference, and correlation is invariant to that scale.

      The step-shaped model response is therefore compared with the most reliable human target available, and every alternative target, including the contrast the reviewer suggested, gives the same near-zero fidelity.

  • W1b / Q1. Which ceiling applies?

    • Response: Thank you for catching this inconsistency. The correct ceiling for r_part is 0.595 (pleasantness-adjusted effect); 0.86 applies to the raw age effect. The split-half estimate (0.62) confirms 0.595.

      Corrected values: best model (GLM-4.5V) 15.6% of the attainable value (previously reported as 11%); median model 6.3% (previously 4%); held-out S4 17.6%.

      Revised text (Abstract and Section 4.1):

      Abstract: "Yet image-specific fidelity remains near zero (r_part = −0.030 to 0.093, attainable ceiling 0.595; no interval excludes zero once raters are resampled)." Section 4.1: "[...] against an attainable ceiling of 0.595. The best model attains 15.6% of what is attainable and the median 6.3%."

      The correction makes the paper internally consistent and, if anything, gives the models more credit; fidelity remains a small fraction of what the data allow.


AuthorsOpenReview ↗

Response to Reviewer pZ9J (Part 2/3)

  • W1c. Magnitude: a 0.1-point human effect vs 0.2 to 1.5-point model shifts

    • Response: We agree the magnitude mismatch is a finding in its own right. The two numbers are in different units, though: the human value is points per SD of age, and the model values are older-minus-young points. We put both in the same units (rating points, older at 67 vs young at 23.5 years).

      Table R2. Human and model image-specific age effects in the same units (valence).

      SD across imagesRatio to human SD
      Human, image-specific effect0.4411
      Human, after removing pleasantness0.2811
      Models, median of 170.627 (range 0.327 to 0.852)1.42 (2.20 after removing pleasantness)

      In matched units, model shifts are a median 1.4 times the human ones (2.2 times after removing pleasantness). The caricature is real, and the revised paper reports it in these units; the human target is not near noise (0.441 points SD; 691 of 900 images carry a permutation-supported effect).

      Reported in matched units, the caricature becomes a quantified finding rather than an artifact of comparing slopes with contrasts.

  • W1d / Q4. Should the comparison be humans asked to role-play an older rater?

    • Response: We agree this is an informative baseline, and we did not run it. We address the reviewer's argument in three parts.

      (i) The target follows from the use case. Persona simulation is proposed as a stand-in for the group's own responses (Argyle et al., 2023; Aher et al., 2023; Santurkar et al., 2023 compare with real group members). The question our paper answers is whether the persona matches the group it claims to represent. Whether VLMs are worse than people at imagining older adults is a different question, and the paper makes no claim about it.

      (ii) We expect role-play humans to show the same pattern, and that supports our interpretation. Lacey et al. (2006) found that younger and older adults both believe happiness declines with age, while self-reported happiness rises with age. A role-play baseline would likely show the frailty and nostalgia pattern, which is what we mean by stereotype: the model persona matches a common belief about ageing rather than the measured responses of older raters.

      (iii) Item-level fidelity is attainable from human data. Plain young-rater means predict the ratings of real 60+ raters better than any model variant (median RMSE 0.583 vs 0.756 with no age and 0.789 with the older persona), and the S4 arm, given human evidence from other images, reaches r_part = 0.105 [0.014, 0.196] on 600 held-out images. The stricter test can be met with human data; prompting does not meet it.

      Revised text (Discussion):

      "Human role-players may share the models' pattern, as people of all ages expect affect to decline with age (Lacey et al., 2006); our target is the represented group's own ratings."

      The missing baseline would refine how the stereotype is interpreted, but it would not change the fidelity result, which is defined against the group the persona claims to represent.

  • W2a. The prompt requests a description from the persona's perspective, so the stereotype language is partly induced

    • Response: We agree the rating fields are the cleaner evidence. Ratings are generated before the description in the JSON field order (Appendix B.2.2), and we ran two prompts that request no description at all.

      Table R3. Valence with no description requested (six models, 300 images).

      PromptModels with negative older − youngMedian shift on pleasant flagged (17)r_part differs from submission
      Submission prompt6/6−1.09n/a
      Rating-only, third person6/6 (largest p = 0.001)−1.390 of 6
      Direct rating only, first person (5 models)4/5−0.6000 of 5

      Without a description, the downshift and the reversal on pleasant flagged images remain, and the reversal becomes larger under the third-person prompt. The perspective framing also does not produce the memory content by itself: in the same template with the age sentence replaced, a no-age "cautious person" uses memory words in 0 to 1% of descriptions, against 4 to 72% for the older persona and 7 to 78% for "retired." We agree that word counts alone are partly induced; the stereotype claim rests on the ratings and the role controls. One change: the arousal downshift holds in 2 of 6 models under the third-person prompt, so we scope the arousal claim to first-person prompts.

      The cleaner evidence the reviewer asked for, ratings with no description, therefore supports the valence stereotype claim more strongly than the original prompt did.


AuthorsOpenReview ↗

Response to Reviewer pZ9J (Part 3/3)

  • W2b / Q3. The mitigation study: one model, 300 images, best arm chosen post hoc, gains not significant, direction agreement matches a pleasantness-only rule
    • Response: We agree with each element and addressed them with a held-out replication. The fixed S4 arm (same prompts, model, and parameters) was run on the 600 OASIS images outside the mitigation subset (3,707 calls, 100% valid).

      Table R4. Fixed S4 arm on development and held-out images.

      ImagesSystemDirection agreementMean absolute errorr_part [95% CI]
      Held-out 600S0 direct prompt38.1%0.511−0.025 [−0.101, 0.053]
      Held-out 600S4 grounded panel85.3%0.2100.105 [0.014, 0.196]
      Held-out 600Pleasantness-only rule (no model)94.3%0.133n/a
      Development 300S4 grounded panel87.5%0.2050.268 [0.161, 0.373]

      On held-out images, S4 is above zero (p = 0.011) and above direct prompting (gain +0.130 [0.006, 0.254], p = 0.041; 400 directions fixed vs 35 broken). With one pre-specified arm, no correction applies. The held-out r_part (0.105) is lower than the development value (0.268), so the original figure was optimistic, and the revised abstract reports both. On direction and error, the pleasantness-only rule beats S4 on both splits, confirming the reviewer's point: those metrics measure what pleasantness predicts, which is why the paper's target is r_part. The revised Section 4.4 reports the rule next to S4 and presents the mitigation study as a diagnostic: reasoning alone lowers fidelity (S1 0.002, S2 −0.018 vs S0 0.081), and human evidence is what raises it.

      Following the reviewer's suggestion turned the mitigation result into a held-out replication of a fixed arm, evaluated on the pleasantness-controlled target beside a no-model baseline.


Questions

  • Q1. Which ceiling, and corrected percentages? 0.595; best model 15.6%, median 6.3% (W1b).
  • Q2. Does a 60+ vs under-60 contrast change fidelity? No: median r_part 0.047, reliability near zero, and 0 of 17 models above zero once raters and images are resampled (W1a, Table R1).
  • Q3. Replicate the best arm on the held-out 600? Done: r_part 0.105 [0.014, 0.196], p = 0.011 (W2b, Table R4).
  • Q4. Young humans rating "as a 60+ adult"? Not run. We explain why the represented group is the correct target, why a role-play baseline would likely share the stereotype (Lacey et al., 2006), and why item-level fidelity is attainable from human data (W1d).

In summary, the human target is measurable (split-half ceiling 0.62, 92% power at fidelity 0.20), every alternative age estimator gives near-zero fidelity, and model and human magnitudes differ by a factor of 1.4 in matched units. The ceiling is corrected to 0.595. The valence stereotype and the reversal persist when no description is requested, and a no-age "retired" persona reproduces them while a "cautious" persona does not. The best mitigation arm replicates on held-out images at r_part 0.105, which we now report alongside a pleasantness-only baseline.

We hope these clarifications and the additional analyses presented above address your concerns. We respectfully ask you to consider updating your score in light of this response.


References

  • Aher, Arriaga, and Kalai. 2023. Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies. ICML 2023.
  • Argyle, Busby, Fulda, Gubler, Rytting, and Wingate. 2023. Out of One, Many: Using Language Models to Simulate Human Samples. Political Analysis 31(3):337-351.
  • Lacey, Smith, and Ubel. 2006. Hope I die before I get old: Mispredicting happiness across the adult lifespan. Journal of Happiness Studies 7(2):167-182.
  • Santurkar, Durmus, Ladhak, Lee, Liang, and Hashimoto. 2023. Whose Opinions Do Language Models Reflect? ICML 2023.
Reviewer_KBNoOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies whether VLMs can reproduce age differences in how people rate images for pleasantness and arousal. It compares responses to age prompts with age patterns estimated from OASIS ratings and finds weak agreement. It also tests whether different prompts, additional reasoning, and human examples improve this agreement.

Critical strengths
  • The evaluation separates agreement with average human ratings from agreement with human age differences. This helps avoid treating good average accuracy or responses to age prompts as evidence of successful persona simulation.
Critical weaknesses
  1. The task itself has little value for the broader community. Reproducing age differences in the scores of one OASIS participant sample is a highly specific objective with little practical relevance. Findings from this task do little to advance practical personalization or the broader study of personas. The paper demonstrates no benefit for an actual application.

  2. The main insights are already known in persona research. Models can match average responses while failing to reproduce demographic differences [a]. Image emotion ratings and demographic prompting in VLMs have also been studied [b]. The main addition is a more detailed analysis of age effects within OASIS. This adds little insight beyond established failures of persona simulation.

  3. The experimental design is inadequate for the stated research question. The paper asks whether VLMs can predict how emotional responses to the same image differ with age. Its evaluation cannot support a general conclusion about this ability because the result depends on one task, one prompt template, and estimated age differences from a poorly balanced participant sample.

  • All main tests use OASIS and one English prompt template. Testing many models on the same task does not establish that the findings hold for other images, participant groups, or persona formats. Persona simulations are known to depend on how roles and demographic information are specified [c].

  • Only 25 participants aged 60 or older rated pleasantness, and 16 rated arousal. The central comparison relies on age effects estimated under a linear relationship between age and ratings. No reported sensitivity analysis establishes that the findings survive alternative ways of estimating these effects.

  • The confidence intervals for the main measure resample images. No corresponding analysis resamples participants and estimates the human age effects again. These intervals therefore do not establish whether the result survives a different sample of older participants.

  1. The psychological comparison does not test the relationship used to interpret the results. The cited psychological study measures the relationship between pleasantness and arousal within each person. OASIS assigns these ratings to separate groups, so that relationship cannot be measured from these data. Relationships across image averages do not validate the same psychological finding. The data also compare different people at different ages and cannot show how a person’s responses change as they age.

  2. The intervention experiments do not establish an effective solution. No intervention produces a statistically significant improvement over direct prompting on the main measure after correction for multiple comparisons. Improvements in the direction and absolute error of predicted age differences do not establish success on the central target of reproducing differences across images. Evaluation on one VLM and 300 OASIS images also provides no evidence of usefulness beyond this setting.

  3. In almost all figures and tables (Figures 1, 2, 3, 5, 6, 7, and 8, and Table 1), the spacing between the figures/tables and their captions, as well as between the captions and the main text, appears substantially smaller than in the official template. These formatting violations must be fixed.


References

[a] "Synthetic Replacements for Human Survey Data? The Perils of Large Language Models"

[b] "Visual Affect Analysis: Predicting Emotions of Image Viewers with Vision-Language Models"

[c] "The Prompt Makes the Person(a): A Systematic Evaluation of Sociodemographic Persona Prompting for Large Language Models"

Questions for authors

Please refer to Critical Weaknesses.

5 comments
AuthorsOpenReview ↗

Response to Reviewer KBNo (Part 1/5)

We thank Reviewer KBNo for a detailed, expert review, and for recognizing the paper's central design choice: separating agreement with average human ratings from agreement with human age differences, so that average accuracy or prompt responsiveness is not mistaken for successful persona simulation. We answer each of the nine concerns with new evidence: 13 analyses, including 5 new model runs (53,207 calls), covering prompt formats, a second dataset, rater resampling, alternative estimators, power, matched human geometry, downstream prediction, and a held-out mitigation test.


Weaknesses

  • W1. What practical decision does the benchmark inform?
    • Response: We agree that predicting one OASIS sample cannot establish a benefit for an individual user. We respectfully disagree that this leaves the task with little practical value. A developer considering age conditioning faces a concrete choice: given the same image and the same model, should the system keep its baseline estimate of viewer pleasantness, or revise it when told the audience is older? Both choices produce a prediction before any human rating is consulted, so their difference in error measures the value of using the age cue for that decision. We scored both against real OASIS raters.

      Table R1. Does supplying the age cue improve prediction of real older raters? (Median RMSE over 17 models; worse/better counts use 95% image-bootstrap intervals on the change in squared error.)

      Target, imagesModel, no ageModel, older personaWorse / better (of 17)Young humans (18-29)Young humans + model age adjustmentWorse / better (of 17)
      60+ raters, all 9000.7560.7897 / 00.5830.69617 / 0
      60+ raters, pleasant flagged (54)0.7541.21416 / 00.6221.31417 / 0
      50+ raters, all 9000.6830.73110 / 00.4950.63617 / 0

      Across 900 images, supplying the age cue raised median error against adults aged 60+ from 0.756 to 0.789; 7 of 17 models became worse and none better. The aggregate conceals the most consequential pattern: on the 54 pleasant images flagged as sensitive, median RMSE rose from 0.754 to 1.214 and 16 of 17 models became worse. The second test gives the model a more informative starting point, the observed mean of raters aged 18-29 (RMSE 0.583). Adding each model's predicted age adjustment raised median RMSE to 0.696 and worsened prediction for 17 of 17 models. For this task, even a developer who already has human norms loses accuracy by applying the model's demographic adjustment.

      The broader contribution is an audit design for personalized foundation models wherever the intended adaptation can be checked against an independent outcome. The Princeton CITP Algorithms + Elections field audit (2026) shows why answers should be examined across user contexts: it asked the same election questions through 160 users' own ChatGPT accounts with memory on and off, and found that personalized answers differed from non-personalized ones more than repeated answers in the same setting did. That audit measures how the answer changes; it does not score answers against an outcome. When the goal is prediction, one further comparison is required, which our benchmark supplies: does the context-conditioned answer outperform the same system with that context withheld, scored against independent outcomes? A practitioner evaluating another population or a memory-based assistant can reuse this comparison with outcomes suited to that setting. We claim direct evidence for age-conditioned visual affect on OASIS; a deployed system would also need evaluation with its own users.

      The benchmark therefore answers a question a practitioner actually faces, whether to use the age cue, and on that decision the answer is measurable and negative: the cue lowers accuracy, even for a developer who already has human norms.


AuthorsOpenReview ↗

Response to Reviewer KBNo (Part 2/5)

  • W2. What does this add to earlier persona research [a], [b], [c]?
    • Response: All three works are cited and discussed in the submission (Section 2; Appendix A.5). They establish that synthetic survey responses can misrepresent groups [a], that VLMs can predict average viewer affect [b], and that persona wording changes responses [c]. Those observations leave an empirical question open: can accurate average prediction, consistent age adjustments across models, and systematically misdirected adjustments coexist on the same stimuli? Here they do. Median pooled-valence correlation is 0.891. Models agree with one another about which images to adjust (mean pairwise r = 0.428, equal within and across developers), yet their median image-specific agreement with the human age effect is 0.037 after removing the part predicted by pooled pleasantness. On pleasant flagged images, models lower older-persona ratings by 0.845 points on average, while older human raters move upward by 0.443. A conventional pooled-accuracy evaluation would miss this reversal; the paired prediction test (W1) gives its cost in rating units, and the content analysis locates where the error concentrates.

      Table R2. What prior work established and what this paper adds.

      WorkWhat it establishedWhat remained untestedWhat this paper establishes
      [a] Bisbee et al. 2024Text-LLM survey responses can look plausible on average yet misrepresent relationships between respondent traits and outcomesImages; continuous affect; whether failure is noise or a shared structureThe failure is structured: per-image age shifts agree across 17 models from 7 developers, but not with humans
      [b] Nowicki et al. 20269 VLMs align with pooled ratings (r > 0.75); rater-conditioned prompting gives small, inconsistent changesWhether persona changes land on the images where groups differ16 of 17 models shift consistently, but on the wrong images; shift size and fidelity are uncorrelated (r = −0.038)
      [c] Lutz et al. 2025Prompt format changes text-LLM portrayals; interview-style reduces stereotypingWhether reduced stereotyping yields item-level fidelityInterview-style removes the persona response (pleasant flagged −0.065 vs −0.812) but fidelity stays near zero (r_part 0.015)

      The paper also contributes the evaluation protocol that exposes this: a stimulus-matched, list-centered estimand that removes pleasantness (which alone correlates with the human age effect at r = 0.897), with a reliability ceiling, rater resampling, and power. Without this control, a rule using no model reaches 95.1% direction agreement and beats every agent we built, so naive metrics would report success where none exists.

      In short, [a] to [c] show that personas can fail. Our paper shows that high competence, consistent cross-model adjustments, and misdirected adjustments coexist on the same images, and it measures where the error concentrates and what it costs. We believe this is a distinct and practically relevant contribution.


AuthorsOpenReview ↗

Response to Reviewer KBNo (Part 3/5)

  • W3 / W4. One task, one prompt template, one dataset

    • Response: We agree that testing many models on one template does not establish generality, and that Lutz et al. (2025) show persona results depend on format. We ran three tests.

      (a) Seven formats, five models from five developers, 300 images, 31,500 calls. Each format changes one element of the submission prompt.

      Table R3. Valence results per format (median over five models).

      FormatOlder − youngShift on pleasant flagged (17)r_partModels with r_part CI > 0
      Submission prompt−0.065−0.8120.0851/5
      Chain of thought−0.031−0.8650.0860/5
      Direct rating only−0.096−0.6000.0880/5
      Paraphrase A−0.099−0.7530.0930/5
      Paraphrase B−0.069−0.7590.1200/5
      Numeric age (24/45/67)−0.047−0.7120.0760/5
      System prompt−0.090−0.7650.0980/5
      Interview-style [c]−0.024−0.0650.0150/5

      Six formats reproduce all valence findings; 0 of 35 new r_part intervals lie above zero and 0 of 35 paired differences from the submission prompt exclude zero. With about ±0.13 interval width on 300 images, this rules out a format with clearly higher fidelity.

      (b) Rating-only, third-person prompt, six models: 6 of 6 lower valence for the older persona (largest p = 0.001); no r_part differs from its submission value.

      (c) NAPS, six models, 450 images, 5,400 calls: median baseline valence r 0.89 (OASIS 0.90); calibration slope above 1 for 6 of 6 models on both datasets; the young persona, matched to NAPS raters, improves 0 of 6. NAPS releases pooled norms without rater ages linked to ratings; its norming sample was 204 mostly young volunteers (Marchewka et al., 2014), so the young persona is the matched condition, and the age-difference target can be built only from OASIS participant-level data.

      The arousal downshift holds in all first-person formats but in 2 of 6 models under the third-person prompt, so the revised paper scopes the arousal claim to first-person prompts. Valence, the primary dimension, is unaffected.

      The paper's conclusions therefore rest on seven formats, two prompt perspectives, and two image sets rather than one template, which addresses the generality concern directly.

  • W5. 25 and 16 older raters; linear age assumption

    • Response: We agree a sensitivity analysis was missing. We re-estimated the human age effect ten ways and recomputed r_part for all 17 models.

      Table R4. Fidelity under alternative estimators of the human age effect (valence).

      Human age effect estimated asOlder ratersMedian r_partSplit-half ceiling
      Linear slope, pooled (paper)all 4130.0370.62
      60+ vs under-60, pooled250.0430.09
      50+ vs under-50, pooled780.0030.52
      Hinge at 45 (nonlinear)all 4130.0460.47
      Log age (nonlinear)all 4130.0250.62
      60+ vs under-60, raw means250.0470.00
      Linear slope, no pooling (OLS)all 4130.037n/a

      Median r_part is 0.05 or below under all ten estimators (three further rows in the revised appendix). The table also shows why the paper uses the linear slope: the 60+ vs under-60 contrasts, built on 25 raters, have reliability near zero (ceiling 0.00 to 0.09), so they cannot serve as a target. The 50+ contrast, with 78 raters and a usable ceiling of 0.52, gives median r_part 0.003. The weak positive values under 60+ contrasts do not survive resampling raters and images (0 of 17 models above zero).

      A power simulation on the real OASIS design (same raters, lists, assignments, noise levels, integer ratings) gives 92% power at a true fidelity of 0.20, and a true fidelity of 0.30 would produce the best model's observed 0.093 in only 1.7% of runs. If human age effects were a pure step at 60, the linear slope would have an SD of 0.016 across images; the observed SD is 0.106, so the data support graded effects.

      The fidelity null therefore does not depend on the linear form, the 60+ cutoff, or pooling, and the design had the power to detect a modest fidelity had one existed.


Reviewer_QKC8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper tests how modern VLMs rate images when asked to respond as people of different ages. Detailed experiments show that models generally agree with mean human ratings of how pleasant the images are. However, they poorly match their response to age differences as compared to human ratings. The paper also shows that giving one model some human evidence can improve its predictions, but the gains may not be statistically significant. Overall, the study is interesting, but remains illustrative due to limited datasets, an imbalanced human rater population, and just one prompt setting.

Critical strengths
  1. The core question the paper asks - whether age prompts help models predict real differences between younger and older people - is interesting.
  2. The experimental results are broad, covering 17 models on the OASIS dataset.
  3. The paper also tests the results on a wide range of human ratings across 822 raters. This makes the contributed corpus useful for future studies.
Critical weaknesses
  1. While the paper tests a variety of models, all results rest on just one dataset. I also don’t see any prompt sensitivity tests, even on a small sample, to understand how model behavior differs based on the supplied input. I agree that this doesn’t immediately invalidate most of the analysis, but for a paper that tests prompting effects, making sweeping generalizations from limited data on one prompting method should be avoided. If the authors could provide a generalization study to NAPS, much of this concern would be addressed.

  2. The human study used in the paper is broad (it covers 822 people), but imbalances in the participant population remain poorly addressed. Looking at the full specification the authors provide in App. B.1, only 41 participants were aged >60. Of these, 25 rated valence and 16 rated arousal. This is again split across four image limits, making per-sample participant count even smaller. The sample size is quite small for the general claim the paper makes from it that models “respond strongly to age prompts yet do not reliably identify the images where human age differences occur” (Lines 480-482), and the paper itself acknowledges it in Lines 366-367. I’m particularly worried as this claim is critical to the work (“VLMs reproduce a stereotype of emotional ageing, not the human pattern”). The participant imbalance’s effect on their claim must be addressed appropriately.

  3. Similarly, the paper’s claim that “VLMs reproduce a stereotype of emotional ageing” in the Figure 1 caption isn’t clearly isolated. While shared errors and nostalgia/vulnerability language can be plausible reasons for stereotyping, what makes the authors certain that they are different from conventions or safety behaviors common for the age or the task? I suggest the claim be toned down to what the experiments can reliably support.

  4. Notably, in writing the paper, the authors acknowledge several limitations with their results and experiments (e.g., Lines 461-463 note that the contrastive panel method, their “best” intervention, does not show a statistically significant improvement over direct prompting). Yet, the claims remain substantially strong.

  5. In many instances, the paper introduces variables and shorthand in the main text (such as S1-S4 in Figure 8 and Section 4.4) that are never defined. This makes the paper very difficult to read.

Questions for authors
  1. Can the authors justify the imbalance in the human rater age pool? Is it intentional to reflect any demographic?
  2. Will the paper’s claimed findings generalize across prompt formats? I recommend studying a few standard methods in the literature, including chain-of-thought, direct responses, etc.
  3. Can the authors show with some controlled experiments that the main claims generalize to the NAPS dataset?
4 comments
AuthorsOpenReview ↗

Response to Reviewer QKC8 (Part 1/3)

We thank Reviewer QKC8 for recognizing that the core question (whether age prompts track real differences between younger and older people) is interesting, that the evaluation covers 17 models on OASIS, and that the 822-rater reference makes the corpus useful for future work. The review raises five concerns: one dataset and one prompt, the small number of raters aged 60+, the basis of the stereotype claim, claim strength, and undefined notation. We addressed each with new evidence: 13 analyses, 5 of them new model runs (53,207 calls). The main result holds under every test, and we corrected three statements where the new evidence called for it.


Weaknesses

  • W1 / Q2 / Q3. One dataset and one prompt format
    • Response: We agree that a paper about prompting must show that its findings do not depend on one template. We ran three tests.

      (a) Seven new prompt formats. Five models from five developers (GPT-5.6-Luna-Pro, Claude-Sonnet-4.6, Gemini-3-Flash, Qwen3-VL-235B, Llama-4-Scout) rated the 300-image subset under young, middle, and older personas in seven formats, each changing one element of the submission prompt (31,500 calls, 99.99% valid).

      Table R1. Valence results over five models per format (median values). Pleasant flagged = 17 sensitive images humans rate 4 or higher; older OASIS raters rate these images higher than young raters.

      FormatOlder − youngModels with negative shiftShift on pleasant flaggedr_partModels with r_part CI > 0
      Submission prompt−0.0654/5−0.8120.0851/5
      Chain of thought−0.0314/5−0.8650.0860/5
      Direct rating only−0.0964/5−0.6000.0880/5
      Paraphrase A−0.0994/5−0.7530.0930/5
      Paraphrase B−0.0695/5−0.7590.1200/5
      Numeric age (24/45/67)−0.0475/5−0.7120.0760/5
      System prompt−0.0904/5−0.7650.0980/5
      Interview-style−0.0243/5−0.0650.0150/5

      Six of seven formats reproduce the downshift, the reversal on pleasant flagged images, and near-zero fidelity; the models move largely the same images (median per-image r with the submission prompt 0.67 to 0.77). Chain of thought, which the reviewer asked about, leaves r_part unchanged (0.086 vs 0.085). Across the 35 new model-format runs, 0 r_part intervals lie above zero and 0 paired differences from the submission prompt exclude zero. The interview-style format, which Lutz et al. (2025) found reduces stereotyping, removes most of the persona response (pleasant flagged −0.065) but does not create fidelity (r_part 0.015). Changing the format either keeps the stereotyped response or suppresses the persona; no format makes the shift track humans.

      (b) Rating-only, third-person prompt (no description). On six models, 6 of 6 lower valence for the older persona (largest p = 0.001), and no model's r_part differs from its submission-prompt value. Three Qwen3-VL models on all 900 images give the same valence result.

      (c) A second dataset, NAPS (Q3). The NAPS release provides pooled image norms without rater ages linked to individual ratings. We therefore used the original norming paper to choose the comparison: Marchewka et al. (2014) describe 204 volunteers, mainly college students and young employees, mean age 23.9 years (SD 3.4). A young-adult prompt approximates this rater profile, and the no-age prompt is the baseline on the same images. The young prompt could improve, leave unchanged, or worsen prediction of the norms. An older prompt would show how much a model changes, but NAPS has no age-linked reference to score that change against. We sampled 450 images across the five NAPS categories and the valence range, mapped the 1-9 norms linearly to 1-7, and ran six models with each prompt (5,400 calls).

      Table R2. NAPS vs OASIS, same six models.

      Median no-age valence rCalibration slope rangeYoung persona moves model closer to young norms
      OASIS0.901.16 to 1.351 of 6 (3 further)
      NAPS0.891.08 to 1.330 of 6 (1 further)

      Baseline competence and over-dispersion (every slope above 1) replicate, and the persona matched to the rater population improves 0 of 6 models and worsens 1. This extends the pooled-rating result, and the test of adding a plausible age cue, to a second image set; the older vs young fidelity analysis requires OASIS participant-level ages.

      Taken together, these runs show that the paper's findings do not rest on one template or one image set. The one claim we narrowed is arousal: the arousal downshift holds in all first-person formats but in only 2 of 6 models under the third-person prompt, so the revised paper scopes the arousal claim to first-person persona prompts. Valence, the primary dimension, is unaffected.


AuthorsOpenReview ↗

Response to Reviewer QKC8 (Part 2/3)

  • W2 / Q1. Few raters aged 60+

    • Response: We agree this is the most important threat to a null result. On Q1: the imbalance is not our design. OASIS (Kurdi et al., 2017) is a public archive of an online sample (ages 18 to 74, mean 37); we collected no data and the age mix was not chosen to reflect a demographic. We designed the analysis around it: the partially pooled model uses continuous age, so all 413 valence raters inform every image's age slope, not only the 25 aged 60+. We then tested whether the null survives the sample.

      Table R3. Does the fidelity null survive the older-rater sample?

      TestResult
      Raters and images resampled, hierarchical model refitted (1,000 reps)0 of 17 models exclude zero; largest upper bound 0.187
      Leave one 60+ rater out (best model, GLM-4.5V)r_part stays in [0.083, 0.102]
      Split-half reliability of the human target0.39 [0.26, 0.50]; ceiling 0.62
      50+ vs under-50 contrast (78 older raters)ceiling 0.52; median r_part 0.003; 0 of 17 above 0
      Power, simulated on the real OASIS design (300 reps)41% at true fidelity 0.10, 92% at 0.20, 100% at 0.30
      P(observed ≤ 0.093, the best model) if true fidelity = 0.300.017

      The target is measurable: its split-half ceiling (0.62) agrees with the model-based ceiling (0.595), and the design would detect a true fidelity of 0.20 in 92% of runs. A true fidelity of 0.30 or more is excluded for every model. The null is therefore a measured absence of fidelity, not noise from 25 raters.

      The reversal on pleasant images also does not rest on 25 raters. With 78 raters aged 50+ and size-matched young subsamples, older raters rate pleasant images 0.373 points higher [0.221, 0.525] (60+: 0.309 [0.029, 0.585]), while 16 of 17 models rate them lower under the older persona. Against real raters, the older persona moves 16 of 17 models further from both 60+ raters (RMSE 0.754 to 1.214) and 50+ raters (0.611 to 1.160) on pleasant flagged images.

      We therefore believe the claim at Lines 480-482 is supported on both halves. "Respond strongly to age prompts" does not depend on the human sample at all (16 of 17 models; every format in Table R1 except interview-style). "Do not reliably identify the images" holds when raters are resampled, when the older group is three times larger, and in a design with 92% power to detect a modest fidelity.

  • W3. Is the "stereotype" isolated from task conventions or safety behavior?

    • Response: This is the right alternative to test. We ran persona controls with no age (6 models, 300 images, 3,600 calls): "a retired person" (a later-life role) and "a cautious person" (a safety-like style).

      Table R4. Control personas vs the older persona (median over six models, valence).

      OlderRetiredCautiousYoung
      Shift vs no age: all images−0.03+0.04−0.52+0.07
      Shift vs no age: pleasant flagged−0.76−0.68−1.46+0.10
      Per-image r with older shift10.750.460.22
      Memory words, range over models4 to 72%7 to 78%0 to 1%1 to 6%

      "Retired" reproduces the older response: same images (r = 0.75), similar size, same rise in memory language. "Cautious" lowers all images uniformly, leaves 64% to 99% of the older shift unexplained, and produces vulnerability words instead of memory words. Three further facts separate the response from a safety behavior: the young persona does not trigger it on the same sensitive images (+0.10); a numeric age of 67 with no age band reproduces it (Table R1); and it contradicts measured older raters (Table R3). Humans hold the same belief: Lacey et al. (2006) found that younger and older adults both expect happiness to decline with age, while self-reports rise with age.

      Revised text (Section 4.3):

      "The valence findings persist when no description is requested (6 of 6 models), and a no-age 'retired' persona reproduces the older shift (median per-image r = 0.75) and its memory language, whereas a 'cautious' persona lowers all images (r = 0.46). We therefore call the response a stereotype: it is shared across developers, contradicts the group's measured ratings, and is cued by a later-life role rather than by caution."

      With these controls, the stereotype claim no longer rests on interpretation: it is an operational definition with three tests, and the data pass each one, while the safety-style alternative fails the role and lexicon tests.


AuthorsOpenReview ↗

Response to Reviewer QKC8 (Part 3/3)

  • W4. Limitations acknowledged, yet claims remain strong

    • Response: We agree that every claim should be calibrated to its evidence, and we separate two kinds of claims.

      Headline claims (competence, responsiveness, and near-zero fidelity) are now backed by more evidence, not less: rater resampling, ten estimators of the human effect (median r_part 0.05 or below under every estimator), power analysis, seven formats, and NAPS.

      The mitigation study is diagnostic. Its role is to show that reasoning alone does not help (S1 and S2 fall below direct prompting) and that human evidence is what is missing. On the reviewer's point about Lines 461-463, we re-ran the fixed S4 arm, unchanged, on the 600 OASIS images it was never tuned on (3,707 calls). With one pre-specified arm, no multiple-comparison correction applies: r_part = 0.105 [0.014, 0.196], p = 0.011, a gain over direct prompting of +0.130 [0.006, 0.254], p = 0.041, with 400 image directions fixed vs 35 broken. The held-out value is lower than the 0.268 on the development images, which we now report.

      Revised text (Abstract):

      "Yet image-specific fidelity remains near zero (r_part = −0.030 to 0.093, attainable ceiling 0.595; no interval excludes zero once raters are resampled) [...] human grounding improves the broad direction of age effects but leaves most image-specific variation unexplained: the best agent reaches r_part = 0.27 on 300 development images and 0.105 on 600 held-out images."

      We also corrected the ceiling to 0.595 for r_part (the 0.86 value applies to the raw age effect), so the best model reaches 15.6% of what is attainable, not 11%. The arousal claim is scoped as described in W1.

      We hope this shows that the revision matches each claim to its evidence: the headline claims gain support, and the mitigation result is now a replicated finding on held-out images rather than a post hoc observation.

  • W5. Undefined notation (S1 to S4, Figure 8, Section 4.4)

    • Response: We agree; the arm labels were defined only in the appendix. The revised Section 3.4 defines every arm where it is introduced.

      Revised text (Section 3.4):

      "Beyond the direct prompt (S0), S1 adds an instruction not to rely on age stereotypes; S2 chains a perceiver, a persona rater, a stereotype critic, and a reviser, with no human data; S3 adds retrieved human age differences for similar images; S4 combines that evidence with five simulated raters per age band, a calibrator anchored on the no-age rating, and an auditor; S5 (VERA) keeps an age-difference rule only if it passes a re-test on held-out human cases; and VERA-Cal passes that rule to the S4 panels."

      A notation table now opens Appendix B, defining r_part, the ceiling, "flagged," "pleasant flagged," and every arm. A reader can now follow Section 4.4 and Figure 8 without consulting the appendix.


Questions

  • Q1. Justification of the rater-age imbalance: see W2. The imbalance comes from the public OASIS archive; our design (continuous-age partial pooling) and the tests in Table R3 address it.
  • Q2. Generalization across prompt formats (chain of thought, direct responses): see W1(a) and (b). Seven formats plus a rating-only third-person prompt; 0 of 35 new r_part intervals lie above zero.
  • Q3. Generalization to NAPS: see W1(c), which explains what NAPS can test and why we used no-age and young prompts there.

In summary, the three concerns the reviewer weighted most now have direct evidence. The findings hold across seven prompt formats and a second dataset. The fidelity null survives rater resampling and a power analysis that excludes a true fidelity of 0.30. A no-age "retired" persona reproduces the older response while a "cautious" persona does not, which separates the stereotype from a safety style. We also corrected the ceiling, scoped the arousal claim, and replicated the best mitigation arm on held-out images.

We hope these clarifications and the additional analyses presented above address your concerns. We respectfully ask you to consider updating your score in light of this response.


References

  • Marchewka, Żurawski, Jednoróg, and Grabowska. 2014. The Nencki Affective Picture System (NAPS). Behavior Research Methods 46(2):596-610.
  • Kurdi, Lozano, and Banaji. 2017. Introducing the Open Affective Standardized Image Set (OASIS). Behavior Research Methods.
  • Lacey, Smith, and Ubel. 2006. Hope I die before I get old: Mispredicting happiness across the adult lifespan. Journal of Happiness Studies 7(2):167-182.
  • Lutz, Sen, Ahnert, Rogers, and Strohmaier. 2025. The Prompt Makes the Person(a): A Systematic Evaluation of Sociodemographic Persona Prompting for Large Language Models. Findings of EMNLP 2025.

Dear Reviewer QKC8

We sincerely appreciate your great efforts in reviewing this paper. Your constructive advice and insightful comments really help improve our paper. We have carefully tried to address your concerns with detailed explanations, essential experiments, and the revised version of our manuscripts.

Considering the approaching deadline, we would appreciate it if you could spare some time to go over our response, and we can further address unclear explanations and remaining concerns if any. We sincerely hope that you can consider reevaluating the initial assessment if we have successfully addressed your concerns.

Best Regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.