acceptodds
Under review as a conference paper at ICLR 2027

Cultural Counterfactuals: Evaluating Cultural Biases in Large Vision-Language Models with Counterfactual Examples

Abstract

Large Vision-Language Models (LVLMs) have grown increasingly powerful in recent years, but can also exhibit harmful biases. Prior studies investigating such biases have primarily focused on demographic traits related to the visual characteristics of a person depicted in an image, such as their race or gender. This has left biases related to cultural differences (e.g., religion, socioeconomic status), which cannot be readily discerned from an individual's appearance alone, relatively understudied. A key challenge in measuring cultural biases is that determining which group an individual belongs to often depends upon cultural context cues in images, and datasets annotated with cultural context cues are lacking. To address this gap, we introduce Cultural Counterfactuals: a high-quality synthetic dataset containing nearly 60k counterfactual images for measuring cultural biases related to religion, nationality, and socioeconomic status. To ensure that cultural contexts are accurately depicted, we generate our dataset using an image-editing model to place people of different demographics into real cultural context images. This enables the construction of counterfactual image sets which depict the same person in multiple different contexts, allowing for precise measurement of the impact that cultural context differences have on LVLM outputs. We demonstrate the utility of Cultural Counterfactuals for quantifying cultural biases in popular LVLMs.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Social bias, cultural bias, fairness, counterfactuals

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_HnfdOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper describes a dataset for diagnosing cultural biases related to religion, nationality, and socioeconomic status, called Cultural Counterfactuals. Moreover, a bias evaluation framework is presented to analyze cultural bias in over 9 million responses produced by LVLMs. The results highlight the usefulness of the Cultural Counterfactuals dataset for diagnosing context-dependent biases in multimodal models.

Critical strengths

None

Critical weaknesses

None

Questions for authors

I'm wondering if the authors can suggest some reliable methods for bias mitigation for LVLMs, based on the bias evaluation strategy from the authors.

1 comment

Thank you for your positive assessment of our work and for recognizing the dataset's usefulness for diagnosing context-dependent bias.

"Can you suggest some reliable methods for bias mitigation for LVLMs based on the bias evaluation strategy?"

To address your question, we tested three bias mitigation strategies: prompt instructions, representation steering, and counterfactual-consistency fine-tuning (LoRA). Steering and fine-tuning were fitted on training sets and evaluated on a held-out panel of 180 sets per dimension by whether they removed the context effect, preserved context recognition, and avoided collapsing person-to-person variation.

Prompt instructions. First, we tested the anti-stereotype instruction from our appendix pilot (telling the model not to use perceived nationality, religion, socioeconomic status, race, age or gender as evidence about a person, and to avoid stereotypes) against a length-matched placebo asking it to inspect the image carefully. On 180 sets per dimension for Qwen2.5-VL and Molmo-7B, it never significantly reduced salary disparity relative to the unmodified prompt; relative to the placebo, it helped only Qwen on nationality (15% lower disparity) and did worse for Molmo (23% and 46% higher disparity). Second, we appended a sentence restating that the job is in San Francisco (or the room in Seattle) and that where the photo was taken is irrelevant, tested across five models on 180 sets per dimension (Holm-corrected over 30 tests). Qwen kept 75% of its SES salary gap (person variation unchanged); InternVL3-8B (HF)'s SES gap fell by 56% only because all person-to-person variation shrank equally; Gemma-3's rent effects grew; and Molmo and LLaVA showed no measurable change.

Representation steering. On Qwen2.5-VL-7B, we recorded residual-stream activations at the last salary-prompt token for 1,700 training images outside the test panel and centred them within each counterfactual set to isolate context differences from the person. At inference, we projected out each dimension's per-context mean subspace (5, 7 and 2 directions for religion, nationality and SES) at every token position in layer 24 (selected over layers 8–24 on 20 validation sets per dimension), alongside a same-size random-subspace placebo. On the held-out panel, the SES gap falls from $14,972 to $713 and the nationality gap from $5,528 to $192 (classification within 1 point; placebo has no effect), but 96% of answers become $120k (vs. 69% before), between-person variation falls by ~75%, and the −$34k age gap disappears. Steering thus removes across-context disparities only by collapsing response diversity.

Counterfactual-consistency fine-tuning. To teach the model to give the same answer across contexts in a set while allowing different answers across people, we trained a rank-16 LoRA on Qwen2.5-VL-7B for 263 steps on the salary prompt using 600 religion and nationality sets whose persons and photos never appear in the test sets. Every image's target in a set was the (lower) median of the base model's own salary answers for that set across contexts and seeds, removing context variation while keeping each person's salary level.

DimensionSalary disparity (within-set SD, % of mean), base → LoRA
Religion7.1% → 2.9% (−60%)
Nationality11.6% → 4.6% (−61%)
SES (never trained on)8.8% → 2.5% (−71%)

On Qwen2.5-VL-7B, the held-out SES gap falls from $14,972 to $2,972, classification accuracy changes by at most 1 point, person-level variation is preserved (set-mean r = 0.91–0.92 with the base model), and MMBench accuracy is unchanged (0.870 → 0.870). Transfer to the untrained rent prompt is partial (−18% to −31%), which training on several judgment prompts may improve.

Counterfactual-consistency fine-tuning is therefore the most reliable method we tested. More broadly, our evaluation strategy maps different failure modes to different interventions: counterfactual-consistency fine-tuning when a model exhibits seed-stable context sensitivity (such as Qwen2.5-VL), output calibration when apparent sensitivity mostly reflects same-image seed noise (such as Molmo-7B or LLaVA-v1.6), and refusal-parity checks when safety-tuned models abstain at context-dependent rates (such as Gemma-4-31B). We also recommend evaluating any mitigation with person-level checks rather than a disparity score alone, because steering looks successful while collapsing the answers. We will add a short main-text "Implications for mitigation" paragraph and an appendix section with these new results.

Reviewer_QQJmOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces a dataset that places the same person in different backgrounds related to religion, country, and income. It measures cultural bias by comparing descriptions of people, salary and rent suggestions, and harmful language across these backgrounds. It evaluates 8 VLMs using nearly 60,000 images and over 9M generated responses.

Critical strengths

The paper addresses cultural bias in VLM judgments about people. This is an important problem because such bias could lead to unfair treatment in salary and housing decisions.

Critical weaknesses
  1. The core counterfactual approach has already been used in prior work.
  • CultureMix [a] already places the same target in different cultural backgrounds and measures changes in VLM predictions. Both studies use real background photographs, concatenate target and background images before editing, and check and regenerate failed images. Some background datasets also overlap. The main evaluation task differs, but the central idea and image construction method are already present.

  • MIXCUBE [b] changes people's apparent ethnicity in images and measures changes in cultural recognition. Beyond Words [c] examines how cultural images, including photographs of people from different income groups, affect responses to questions about values.

  • In summary, the claim that this is the first use of a counterfactual framework to study cultural bias is invalid. Although the tasks differ, prior work already covers counterfactual evaluation of cultural bias, changes to cultural backgrounds, and the effects of visual cultural cues on model responses. None of these studies is cited.

  1. There are problems with how the benchmark is built.
  • The filtering keeps only backgrounds for which a VLM correctly predicts the source cultural label. This may favor settings that fit existing model associations and exclude valid but less typical examples. Agreement across several VLMs does not rule out shared stereotypes. The effect of this selection on the reported bias remains unclear.

  • Changing the background also changes architecture, indoor or outdoor settings, lighting, and scene layout. Prior work [d] shows that even small background changes can greatly alter measured gender bias. This benchmark does not separate cultural cues from other background differences, making it difficult to attribute the reported effects specifically to cultural bias.

  • Question diversity is limited. The evaluation mainly uses fixed English prompts for classification, salary, rent, keywords, and sentence completion. Nearly 60,000 images and over nine million responses do not establish broad question coverage. The results provide limited support for conclusions beyond these specific prompts.


References

[a] "World in a Frame: Understanding Culture Mixing as a New Challenge for Vision-Language Models", CVPR 2026

[b] "When Tom Eats Kimchi: Evaluating Cultural Bias of Multimodal Large Language Models in Cultural Mixture Contexts", C3NLP 2025

[c] "Beyond Words: Exploring Cultural Value Sensitivity in Multimodal Models", Findings of NAACL 2025

[d] "Bias in Gender Bias Benchmarks: How Spurious Features Distort Evaluation", ICCV 2025

Questions for authors

Please refer to Critical Weaknesses.

6 comments

Thank you for recognizing the importance of the problem we study. We respond to your questions and concerns below.

1. Novelty

"W1: The core counterfactual approach has already been used in prior work. CultureMix [a]...MIXCUBE [b]...Beyond Words [c]...None of these studies is cited."

Thank you for these references; we will cite all four works in Section 2 of our updated manuscript and will revise our two 'first' claims (Section 2: 'the first to employ a similar counterfactual framework for studying cultural biases'; Conclusion: 'the first counterfactual image dataset'). However, we respectfully disagree that "the central idea…[is] already present" in prior work such as CultureMix. None of these papers provide what our dataset contains: counterfactual image sets which depict the same person in different religious, national and socioeconomic contexts. Furthermore, none of these prior studies investigate how model judgments about a depicted person vary due to cultural context differences. The table below summarizes the differences between our work and these previous papers.

WorkImagesHeld fixedVariedThe personThe model is askedNormative invariance / What variation measures
CultureMix [a]~23kFood item50 landmark / street photosNo person is a targetThe food's name and countryAnswer is the food item; variation = object-recognition error under background distraction
MixCuBe [b]2,475Food, festival or clothing sceneThe person's ethnicityEdited (the perturbation)The item's country and nameAnswer is the cultural item; variation = cultural-recognition error under ethnicity edit
Beyond Words [c]Unedited web and Dollar Street photosSurvey questionWhich photo is shownUnedited family photosWorld Values Survey questionsAnswer is WVS country distribution; variation = cultural value alignment (no counterfactual sets)
Ours59,827 in 10,331 setsThe person17 religion / nationality / SES contextsHeld fixed (the subject)Salary, rent, descriptions and completions about the personAnswer should be invariant to context; variation = disparate treatment / bias toward the same person

CultureMix composites 247 food items onto 50 landmark and street photos (5 per continent) and asks for the food's name and country. Therefore, the subject of generated images (food in CultureMix, people in Cultural Counterfactuals) as well as the intended use of the datasets differs substantially. Our construction shares standard compositing primitives with CultureMix (real background photos, concatenation followed by FLUX.1-Kontext editing, and regeneration of failed edits). Because the background is our treatment on a human subject rather than a distractor behind an object, we added design elements specific to person-fixed auditing: balanced intersectional person synthesis across 144 (age × race × gender) demographic profiles (4,261 synthetic individuals, enabling person-fixed and intersectional analysis), foreground person segmentation and removal (RMBG-2.0) before context-detectability verification so the VLM judge cannot infer culture from facial features or attire, and multi-round context resampling from large human-annotated pools (500–2,000+ scenes) rather than a fixed set of 50 photos.

MixCuBe is the inverse of our counterfactual design: it keeps the cultural scene fixed, edits the person's ethnicity, and then asks about the item (as opposed to the person). Beyond Words has no matched image sets; it scores LLaVA's answers to World Values Survey questions against a country's survey responses.

A key distinction between our work and other studies is the type of evaluation that our dataset supports. In CultureMix and MixCuBe the correct answer depends on culture, so relying on the background or on the person is a recognition error. In our tasks the correct answer (e.g., pay for a fixed San Francisco job) should not depend on culture at all. Any change across contexts is therefore disparate treatment of the same person, which none of these three prior works can measure because none asks about the person. We will update our manuscript to clarify this distinction between our paper and prior work. We will revise our main novelty claim as follows: "To the best of our knowledge, Cultural Counterfactuals is the first dataset that holds a depicted person fixed while varying their religious, national or socioeconomic context, enabling the measurement of context-conditioned disparities in the judgments that LVLMs make about people."

2. Filter selection

"W2: The filtering keeps only backgrounds for which a VLM correctly predicts the source cultural label...Agreement across several VLMs does not rule out shared stereotypes."

All labels for our source images come from human-annotated sources (Google Landmarks, VIPPGeo geotags and Dollar Street income). The filter is therefore just intended to remove visually ambiguous images from this pool of human-labeled images, and it preserves 96.7% of religion and 88.2% of nationality source images. To test directly whether VLM filtering or shared multi-VLM stereotypes create our findings, we conducted two new analyses—one removing VLM filtering altogether on SES, and one stratifying by background typicality across dimensions.

First, for SES (where the filter discards 69.4% of source images), we rendered the same 287 people with identical image generation settings into three Dollar Street source pools: the current filter, a pool accepted by ≥ 3 of 5 diverse judges, and all 991/991 human-labeled sources with no VLM filter at all. Evaluating five LVLMs with two-way (person × photo) clustered 95% CIs, the unfiltered high − low salary gap remains significant: +6.9% for Qwen2.5-VL vs. +10.7% filtered (+10.3% on ≥ 3-of-5); +7.7% for InternVL3-8B (HF) vs. +12.4% filtered (+12.5% on ≥ 3-of-5); and +6.4% for Molmo-7B vs. +8.1% filtered (+11.0% on ≥ 3-of-5). Because the unfiltered evaluation retains every human-labeled Dollar Street photo with zero VLM selection, shared stereotypes across VLMs cannot explain the baseline disparity. VLM filtering sharpens the contrast by ~1.6× (because many global middle- and high-income Dollar Street homes look modest and are read as low income), but does not create it.

Second, to test whether more "stereotypical" or easily recognized scenes drive effects in nationality as well, we stratified accepted sets into quintiles of scene typicality using 24 blind classification votes across the eight evaluated LVLMs (3 seeds each; leave-own-model-out 21-vote typicality correlates r=0.995r = 0.995), with pigeonhole bootstrap 95% CIs over persons and photos. For nationality, Qwen2.5-VL's salary gap is completely flat across typicality quintiles (5.0%, 5.5%, 5.5%, 5.2%, 5.3% from least- to most-typical quintile; Q5 − Q1 = +0.4 pp [−0.6, +1.3]), and is likewise flat for Gemma-3-12b, Molmo-7B, InternVL3-8B, and InternVL3-38B—confirming that scene typicality does not drive nationality disparities. For SES, while Qwen's salary gap rises with typicality (+11.0% → +18.1%, Q5 − Q1 = +7.1 pp [2.0, 12.1]), it is already +11.0% in the least-typical quintile (Q1). We will reframe our discussion of the SES setting as "visual-affluence context" in our updated manuscript and include both analyses.

3. Background confounds (Hirota et al.)

"Changing the background also changes architecture, indoor or outdoor settings, lighting, and scene layout"

We agree that a scene changes cultural cues and other visual attributes together, and that some attributes, such as a mosque's architecture, partly define the context itself. Following Hirota et al. (2025) [d], we conducted additional experiments with controlled scene changes to address this concern.

For our primary control, we used a second photo and photometric perturbations. On 180 sets per dimension, we rendered each person into a second source photo of the same category (photo B). Photo B was chosen by a fixed rule before any outcome was seen and rendered with identical settings. We also made copies of the original photo (A) with brightness × 0.85 and saturation × 0.75. Qwen2.5-VL and InternVL3-8B (HF) were evaluated on the resulting images with three random seeds.

Model, contrast (USD [95% CI])Photo APhoto BB − ABright − ASat − A
Qwen SES salary11,92613,370+1,444 [−2,378, 5,267]−778 [−2,037, 481]−74 [−1,287, 1,139]
Qwen nationality salary5,6326,000+368 [−1,489, 2,225]−660 [−1,322, 3]−375 [−983, 233]
InternVL SES salary11,73611,422−314 [−2,749, 2,121]−725 [−2,565, 1,115]+156 [−1,793, 2,106]
InternVL nationality salary3,4513,845+394 [−1,004, 1,791]+248 [−762, 1,258]+754 [−176, 1,684]

The table above shows that our main findings hold under these controlled scene changes. Every SES effect (salary and rent) and every nationality salary effect keeps its sign and significance on photo B and under both perturbations, and no change from photo A is significant. The scenes also remain recognizable, with classification accuracy changing by at most 3 points. Swapping to another photo of the same category also changes salary substantially, but switching category changes it significantly (by +$2.2k to +$3.4k for SES and +$0.5k to +$1.0k for nationality across the two models).

3. Background confounds (continued)

We also tagged measured scene attributes including brightness, contrast, colourfulness, saturation, sharpness, edge density, person size, and CLIP indoor/night. Some attributes differ by context; for example, Shinto-shrine scenes have a standardized difference of +0.73 in edge density. Adjusting for these attributes leaves the SES and nationality salary gaps unchanged:

Model, contrastUnadjustedMatchedRegression
Qwen, SES$14,216$14,876$15,020
InternVL3-8B, SES$8,696$7,315$9,190
Molmo, SES$8,668$7,821$9,256
InternVL3-14B, nationality$2,290$2,172$2,251
InternVL3-38B, nationality$1,222$1,179$1,221

We do not adjust for scene type because it nearly determines the category (only 4–14% overlap). Crossed random effects for person set and photo change the estimates by about 1%. A text cue such as "This photo was taken in Brazil" reproduces part of the pattern (see our response to ZGWj).

In summary, the SES and nationality salary disparities survive a second photo and photometric changes, exceed the variation between photos of the same category, and are unchanged by adjustment for measured scene attributes. These controls cannot rule out every unmeasured difference between categories, so in our updated manuscript we will describe these effects as context-conditioned disparities rather than as effects of cultural cues alone.

4. Question diversity

"Question diversity is limited...Nearly 60,000 images and over nine million responses do not establish broad question coverage."

We ran three new experiments with Qwen2.5-VL and InternVL3-8B (HF) to address this. First, we produced three paraphrases each for the salary and rent prompts, plus two paraphrases each for Arrest and Bad Influence. Each paraphrased prompt kept the job, city and output format fixed and was run on 180 counterfactual sets per context type. Results for the salary prompt are summarized in the table below.

Model, contrastOriginalParaphrases P1–P3Pooled
Qwen SES salary+14.3%+5.0 to +12.3%+9.2% [6.6, 11.7]
Qwen nationality salary+5.4%+2.2 to +5.6%+4.7% [3.5, 5.8]
InternVL SES salary+13.1%+4.0 to +17.5%+8.9% [7.3, 10.5]
InternVL nationality salary+3.8%+1.0 to +4.8%+2.6% [2.0, 3.2]

The direction of the effects is robust across paraphrases while the magnitude of effects varies. Every SES and nationality salary effect keeps its sign and remains significant across paraphrases. For the completion prompts, wording changes refusal rates more than context does, but Qwen's toxicity excess persists across all paraphrases.

Second, we added three decision tasks in which the model acts on the person at a fixed US location with a constrained output format: approving a maximum personal loan amount in USD as a San Francisco bank officer, accepting or rejecting a room-rental applicant in downtown Seattle, and rating likelihood to hire a San Francisco tech applicant. We ran each task on 300 sets per dimension with three seeds, using the same contrasts and two-way clustered CIs as for salary.

TaskQwen, SES low → high incomeInternVL, SES low → high incomeNationality gap, Qwen / InternVL
Maximum loan$33.3k → $46.6k (+32.6% [25.0, 40.2])$14.5k → $29.8k (+68.0% [54.5, 81.4])+7.5% / +8.4%
Tenant accepted83.1% → 98.0% (+14.9 pp [10.2, 19.6])99.8% accepted overall (ceiling)n.s. / n.s.
Hiring score (1–10)5.71 → 7.22 (+1.51 [1.25, 1.76])6.37 → 7.06 (+0.69 [0.51, 0.86])+4.0% / +1.1% (p = .048)

Every SES decision that is not at ceiling moves in the same direction as salary and rent and is significant. Loan amounts cluster at a few values (Qwen answers almost only $5k or $50k), so percentages cannot be compared across tasks. We therefore compare standardized effects (d_z, the mean within-set gap divided by its SD across sets). Qwen's SES effect is 0.44–0.76 on the decision tasks, against 0.66–0.68 for salary and rent on the same images. The nationality gap appears in loan and hiring decisions for both models but not in tenant decisions.

Third, we ran the prompts in Chinese, Portuguese, Arabic and French on the same images. The SES and nationality salary gaps keep their sign and significance in all four languages for both models. Pooled over languages, Qwen's SES gap is +20.5% against +13.3% in English, and InternVL's is +11.1% against +13.1%. Qwen's Arrest toxicity excess is present in all four languages.

These new results show that the direction of the SES and nationality effects is robust across wordings, tasks and languages. We will add these new results to the updated version of our manuscript. Finally, we note that our main contribution is the dataset itself and that others can of course use it to investigate alternative prompts beyond those which we evaluated.

Reviewer_ZGWjOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduce Cultural Counterfactuals, a dataset of around 60k images in 10.3k counterfactual sets for measuring cultural bias in LVLMs (religion, nationality, socioeconomic status). Each set show the same synthetic person (generated with FLUX.1-dev) placed into real cultural context photos using FLUX.1-Kontext, so only the background context change. Images are filtered with CLIP similarity and Qwen2.5-VL-32B context detection, with regeneration loop, and checked by 5-VLM panel and small human validation. Authors evaluate eight open LVLMs (around 9M generations) with context classification, salary/rent prompts, keyword sensitivity, MaxToxicity, and topic modeling, and show that outputs change with cultural context, e.g., lower salary for Brazil/India contexts and stereotype completions for Mosque context.

Critical strengths
  1. The counterfactual design is good idea for cultural bias. Since person is fixed within a set, difference can be attributed to context and not to the person appearance. Cultural attributes like religion cannot be seen from face, so this fill a real gap, and the dataset will be useful for community.

  2. Authors do many robustness checks: re-generating images with new FLUX seeds (Tab. 11-12), 5-VLM panel for filtering (Tab. 8-9), sensitivity on correctness-filtered subsets (Tab. 19), MaxToxicity without refusals (Tab. 23), and placebo prompt in mitigation (Tab. 5). This make the main salary finding for Qwen2.5-VL quite convincing.

Critical weaknesses
  1. The metrics do not have null baseline, so it is hard to know how much is bias and how much is just sampling noise. For sensitivity, keywords are union over 3 seeds, and Gemma-3-12b has seed stability J=0.01 but sensitivity 0.96, which is basically random. Sensitivity should be compared to within-image variation across seeds. MaxToxicity is max minus min over contexts at temperature 1, so it is non-zero even without any context effect, and range naturally grow with number of contexts (3 for SES vs 6 and 8), so the dimensions are not comparable. For salary, deviation is only relative to set mean and not to neutral background, even though authors already have people-only images. Effect is also not reported as % of mean salary, and with ~2.6k sets the t-test will be significant for very small difference.

  2. Some headline findings may be overstated. The stereotype examples ("terrorist", "weapons of mass destruction") are very rare: the WMD topic is 0.19% of Arrest responses and religious violence topic 0.24% of Bad Influence responses (App. I.1), and mostly come from Molmo-7B which has no safety tuning. Also the SES filter discard 69.4% of images (87% of middle/high income), so kept images are the ones that look most stereotypically rich or poor, which can inflate the SES gap. Since salary and rent are both lower for poor context, model may be reading the scene as location and not judging the person, which is different kind of harm. Last, the pipeline and metrics mostly follow SocialCounterfactuals (Howard et al., 2024) and Howard et al. (2025), so main contribution is the dataset, and the models are older and small (Qwen2.5-VL-7B, LLaVA-1.6, Molmo-7B), with InternVL3-14B/38B missing for most prompts (Tab. 21).

Questions for authors
  1. Can you report sensitivity and MaxToxicity relative to a seed-only baseline (same image, different seeds)? This will change how I read Gemma-3-12b and Molmo-7B results.
  2. Can you compare salary/rent to the people-only (blank background) images, and report deviation as % of mean salary?
  3. How do you make MaxToxicity comparable across dimensions with different number of contexts?
  4. Does the SES finding still hold if you use images that pass a weaker filter (e.g., 3 of 5 panel VLMs)?
  5. Filtering is done with Qwen2.5-VL-32B, and Qwen2.5-VL-7B get highest classification accuracy. Did you check if this favor the Qwen family?

Suggestion: evaluating at least one newer or larger model (e.g., the ones already used in your filtering panel) would make the benchmark more useful.

4 comments

Thank you for recognizing that the person-fixed counterfactual design is a good idea that fills a real gap for attributes such as religion that are not visible from a face. We also appreciate your recognition of the robustness checks that make the Qwen2.5-VL salary finding convincing. We address your questions and concerns in order below.

1. Keyword sensitivity needs a seed-only null

W1/Q1: "Can you report sensitivity and MaxToxicity relative to a seed-only baseline?"

Thank you for this suggestion. The table below reports a new metric called excess sensitivity which is the cross-context keyword dissimilarity minus the same-image dissimilarity across seeds, with set-bootstrap CIs.

ModelPrompt A excess (share of cross-context dissimilarity)Prompt B excess
Qwen2.5-VL+0.58 to +0.63 (75–77%)+0.56 to +0.59 (93%)
Gemma-3-12b+0.004 (0.4%)+0.21 to +0.24 (37–40%)
InternVL3-8B / 14B+0.03 to +0.05+0.07 to +0.09
Molmo-7B+0.01 (~1%)+0.03 to +0.04
LLaVA-v1.6+0.01 (~1%)+0.01

Because our existing sensitivity metric pools keywords over 3 seeds, we also generated 3 new seeds for all 8 models on 300 sets per dimension (244,800 generations), so that the same image has a second 3-seed union to compare against. The union-level excess gives the same model ranking (religion / SES: Qwen +0.47 / +0.51; LLaVA and Molmo ≤ +0.01).

These results quantify the paper's existing conclusions, which attribute Gemma-3's high sensitivity and near-zero seed stability to stochastic variation rather than bias (Section 5; App. "Recommendations", item 1). Only +0.004 (0.4%) of Gemma's Prompt A sensitivity exceeds seed noise, versus 75–77% of Qwen's. On Prompt B, which restricts the output to exactly five keywords (thereby avoiding explanations that Gemma sometimes provides for Prompt A), Gemma-3’s excess sensitivity rises to a modest context effect of +0.21 to +0.24. Molmo-7B, which we report as having a low seed stability (J ≈ 0.06–0.07), has an excess of only 1–3%, like LLaVA, so its keyword variation is also mostly noise. These results support our recommendation that sensitivity be read together with stability.

2. MaxToxicity has no null and depends on the number of contexts

W1/Q3: "How do you make MaxToxicity comparable across dimensions with different number of contexts?"

To address your question, we added a seed-only null which compares the toxicity range over three responses to one image with the range over three contexts and reports the excess at the 95th percentile. We matched religion and nationality to K = 3 contexts by averaging over all 3-context subsets across distinct seeds and report the share of within-set variance due to context, with a permutation test of which context holds the maximum.

Qwen2.5-VL exceeds seed noise in all 18 cells (+0.004 to +0.134; e.g., religion Arrest 0.135 vs. 0.018 null, +0.118 excess), including on non-refused Arrest and Bad Influence responses alone, with context explaining 67–95% of its toxicity variance. Other models have excess between −0.001 and +0.015 (context share ≤ 0.28). While Molmo produces the highest raw MaxToxicity among 7B–12B models, it also has the largest seed-only range (highest in 15/18 cells), so its high values mostly reflect same-image variability; we will keep this observation and revise the interpretation that Molmo shows the greatest sensitivity to context.

However, a permutation test on which context holds the maximum in the top 5% of sets still rejects uniformity in 12/18 cells for Molmo—including religion Arrest (p = 0.0005) and SES Bad Influence (p = 0.0005, where low income holds the maximum in 45% of sets vs. 33% expected)—as well as 13/18 for Qwen, 15/18 for InternVL3-8B, and 12/18 for LLaVA. Context therefore shifts which group receives the most toxic outputs even where overall tail size is near noise (consistent with Point 4).

Matching K = 3 removes most cross-dimension differences: Qwen's Should nationality value falls from 0.345 to 0.177 (vs. 0.156 for SES), and Molmo's Arrest value falls from 0.222 to 0.143 (vs. 0.144 for SES). Qwen's excess also persists across every paraphrase and four other languages (see QQJm Point 4).

3. Neutral reference, % of mean, and significance with many sets

W1/Q2: "Can you compare salary/rent to the people-only (blank background) images, and report deviation as % of mean salary?"

For the people-only reference, we ran the salary and rent prompts on the 4,261 people-only images for all eight models with three seeds. The table below provides the change relative to the same person on a white background.

ModelSES lowSES highNat. BR/IN/MA/ZANat. DE/FR/US/CNReligion
Qwen2.5-VL salary−2.7%+11.6%+4.2%+9.8%+8.8%
Qwen2.5-VL rent−10.2%+2.3%−3.0%+0.6% (n.s.)+0.1% (n.s.)
InternVL3-8B (HF) salary−6.7%+8.0%+2.0%+4.7%+5.5%
Molmo-7B salary−9.1%−1.3% (n.s.)−6.0%−3.6% (n.s.)−4.1% (n.s.)
Gemma-3 salary+6.2%+6.1%+3.9%+5.2%+6.3%

The Qwen SES salary gap is two-sided, with high-income scenes raising salary more than low-income scenes lowering it. For the rent prompt with InternVL3-8B/14B/38B, low-income scenes lowering the judgment account for 70–87% of the gap, while Molmo places every scene below the blank. For nationality, Qwen and Gemma pay more in every country than on the blank, and more still for DE/FR/US/CN.

We also recomputed our original salary & rent prompt contrasts as a % of the mean, with 95% CIs clustered by person and by source photo. For Qwen2.5-VL the nationality gap is +5.3% [4.9, 5.7] and the SES high − low salary gap is +13.8% [11.9, 15.6]. The other models' SES gaps range from +3.0% (InternVL3-38B) to +9.2% (Molmo, whose gap is less stable across seeds but reappears at +8.1% with independent seeds; see Point 5), with Gemma-3 at −0.1% [−1.3, 1.0] and LLaVA not significant. Equivalence tests (TOST, ±2% of the mean) show that Gemma-3's SES gap is equivalent to zero, while no Qwen gap is equivalent to zero.

4. Stereotype examples are rare and mostly from Molmo-7B

W2: "The stereotype examples are very rare and mostly come from Molmo-7B which has no safety tuning."

Thank you for raising this concern. We will rewrite the introduction and abstract to name the model behind each example and also quantify its rate of occurrence. To check whether these rare examples still reflect a systematic association, we computed risk ratios for Mosque against the other religious contexts, with set-bootstrap CIs. For Molmo, the WMD topic appears in 1.58% of Mosque responses against 0.61% elsewhere (RR 2.60 [2.25, 2.98]), and the religious-violence topic has RR 1.86 [1.59, 2.15].

At the same time, Qwen shows significant over-representation in its own two Bad Influence BERTopic clusters (RR 2.47 for Mosque and RR 1.82 for Synagogue), and mentions of "terrorism" in Arrest completions show the same Mosque skew beyond Molmo: RR 2.03 [1.86, 2.23] for Molmo, 2.68 [1.95, 3.76] for Qwen (404 responses) and 2.50 [1.21, 4.76] for InternVL3-8B (42 responses), while LLaVA is not significant. The association is therefore rare in the other models, but not unique to Molmo.

5. The SES filter may select stereotypical scenes

W2/Q4: "Does the SES finding still hold if you use images that pass a weaker filter (e.g., 3 of 5 panel VLMs)?"

We ran additional experiments using the filtering process you proposed and found that our SES findings still hold. Specifically, we ran the five-judge panel on all 991 Dollar Street sources and used the votes to build three source pools (low / middle / high income): the current filter (200 / 83 / 20 sources), photos that at least 3 of the 5 judges accept (200 / 94 / 25), and all sources with no filter (201 / 492 / 298). We rendered the same 287 people into each pool with identical settings and then evaluated five models with three seeds each, with CIs clustered by person and photo. The table below provides the high − low gap as % of the mean.

Model, endpointCurrent≥ 3 of 5No filter
Qwen2.5-VL salary10.7%10.3%6.9%
Qwen2.5-VL rent9.8%8.8%3.0%
InternVL3-8B (HF) salary12.4%12.5%7.7%
InternVL3-8B (HF) rent11.3%14.0%9.9%
Molmo-7B salary8.1%11.0%6.4%
Gemma-3 salary0.1% (n.s.)−1.0% (n.s.)−3.8%

The ≥ 3-of-5 pool produces the same gap as the current filter for every model, and the same classification accuracy. Without any filter, the gap falls to about 60% of its filtered value (about 30% for Qwen rent), but it stays significant for Qwen, InternVL3-8B and Molmo salary. The gap shrinks because Dollar Street measures income on a global scale. Many unfiltered middle- and high-income homes look modest, so the models read them as low income, and the gap grows with how recognizable the income level is. The SES context therefore captures visible affluence, which we will clarify.

6. The model may read the scene as a location

W2: "model may be reading the scene as location and not judging the person, which is different kind of harm."

Our prompts already fix the decision location (e.g., San Francisco for salary), so using the scene as the location departs from the prompt instructions. We ran three new experiments to investigate this mechanism.

First, we appended instructions such as "The job is located in San Francisco; where the photo was taken is irrelevant." on 180 sets per dimension across five models (Holm-corrected over 30 tests). Qwen2.5-VL keeps 75% of its SES salary gap (person variation unchanged) and 86% of its nationality gap; its SES rent gap drops 79% only because 95% of answers collapse to $1,200 (vs. 48% before). InternVL3-8B (HF) loses 56% of its SES gap, but its person variation and age gap shrink equally and its nationality gap is unchanged. Gemma-3's rent effects grow, while Molmo and LLaVA show no change.

Second, on 300 sets per dimension, we asked "Where does this person live?" and "What is this person's annual income?" Every model places the person in the scene's country (+13 to +46 pp) and maps high-income scenes to high-income countries (+51 to +77 pp); stated income accounts linearly for 48–67% of Qwen's SES gap.

Third, adding a text cue ("This photo was taken in Brazil.") to people-only images (177 persons, 17 cues, 3 seeds) reproduces the SES direction, InternVL3-8B's nationality gap (text/image ratio 1.25 [0.93, 1.78]), and Qwen's religion pattern (6/6 signs, r = 0.97), plus a third of Qwen's nationality gap.

Thus, models infer residence and affluence from the background and rely on that inference despite the fixed prompt location. Either way, the same person receives different pay or rent due to the background; we will discuss location-inference harm alongside stereotyping and clarify that our design does not separate them.

7. The main contribution is the dataset

"the pipeline and metrics mostly follow SocialCounterfactuals so main contribution is the dataset"

We agree that our primary contribution is the Cultural Counterfactuals dataset. However, its construction differs from SocialCounterfactuals (Howard et al., 2024) and Howard et al. (2025) in four ways: it uses real, human-labelled context photos (synthetically generated scenes may misrepresent cultural locations), it fuses the person into the scene, it applies a detectability filter after foreground removal, and it resamples contexts over multiple rounds. Components of our evaluation framework, such as how it measures awareness, sensitivity and seed stability jointly and tests refusal parity across contexts, are also new.

8. Older and smaller models; missing InternVL3-14B/38B results

Thank you for pointing this out. We have completed generation experiments for InternVL3-14B and -38B for every prompt. Both models refuse most Arrest and Bad Influence completions (14B 84–85% / 96%; 38B 95–97% / 95–96%), and their MaxToxicity excess is at most 0.009.

We also evaluated two newer models from our filtering panel. Gemma-4-31B (open weights) was run on 180 sets per dimension with 6 prompts and 3 responses each (55,080 generations). Its classification accuracy is 0.79 / 0.75 / 0.64. It refuses every Arrest and Bad Influence completion and 38% of salary requests, and its salary refusals depend on the scene: from 32.6% for Germany to 43.1% for India, and 9.4 pp lower for high-income than middle-income scenes. Where it answers, the SES gap is +2.4% [1.4, 3.4], the nationality gap is not significant, and rent is a constant $1,200. Its keyword excess is 0.15–0.18, and it calls people in low-income scenes "humble" in 19.4% of responses, against 9.4% for high-income scenes.

GPT-5.5 (API) answered salary, rent and Arrest once per image on 300 sets per dimension. Its SES salary gap is +4.5% [2.5, 6.5], against +14.2% for Qwen on the same images. Its nationality gap is +1.3% [0.3, 2.2] and its SES rent gap +1.9%, and it refuses 99.8% of Arrest completions.

9. Does the Qwen2.5-VL-32B filter favour the Qwen family?

To address this question, we used all eight models to classify source photos without a person (including photos which the Qwen2.5-VL-32B filter had rejected). We compared accuracy on a "Qwen-free" pool (photos accepted by a majority of the four non-Qwen judges). For religious contexts, Qwen leads Gemma-3 by +5.7 points [3.1, 8.4] on the Qwen-free pool. For nationality, the Qwen lead shrinks to +1.1 [−1.0, 3.1]. For SES, Qwen is never the top model (it ranks 4th–7th), and the ≥ 3-of-5 pool gives the same SES effects (see Point 5 above). The filter therefore does not explain Qwen's religion lead, may partially explain its nationality lead, and is irrelevant for SES (since Qwen doesn’t lead in context classification accuracy for SES).

We will incorporate all of these new results and clarifications into our updated manuscript.

Thank you for your response; I will take the other reviewer's comments into account before making my final decision.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revised manuscript uploaded

We thank all reviewers for their constructive feedback. We have uploaded a revised manuscript that incorporates every new result and clarification from our responses. Changes are shown in blue. The major changes are as follows:

  1. Positioning and novelty (QQJm, ZGWj). Section 2 now cites and discusses CultureMix, MixCuBe, Beyond Words, and Hirota et al. (2025). We replaced the two "first" claims with a precise claim: Cultural Counterfactuals is the first dataset that holds a depicted person fixed while varying their religious, national, or socioeconomic context. App. C.1 (Table 7) compares the designs and lists how our construction differs from SocialCounterfactuals.
  2. What we measure (QQJm, ZGWj). Section 1 now defines our measure of cultural bias as context-conditioned disparities: differences in how the same person is judged across depicted contexts. App. A adds two limitations paragraphs, one on scene attributes that are correlated with the context and one on location inference. Section 3.1 states that our socioeconomic contexts capture visible affluence.
  3. Seed-only nulls (ZGWj). We added excess keyword sensitivity (Sections 4–5; App. E.5) and a seed-only null for MaxToxicity, with K-matching and permutation tests (App. H.4). We revised our interpretation of Molmo-7B's MaxToxicity accordingly (Section 5; App. B).
  4. Effect sizes and a neutral reference (ZGWj). App. G.1 reports contrasts as % of the mean with person × photo clustered CIs, d_z, and equivalence tests. App. G.2 compares each context with the same person on a blank background.
  5. Robustness (QQJm, ZGWj). A new main-text paragraph, "Robustness of Context-Conditioned Disparities" (Section 5), summarizes several new analyses:
    • filter-free and relaxed-filter SES pools, scene typicality, and the Qwen-filter check (App. C.8);
    • a second photo of the same category, photometric perturbations, and adjustment for measured scene attributes (App. K.1–K.2);
    • location inference (App. K.3);
    • paraphrases, three decision tasks, and four other languages (App. K.4, with prompts in App. D.3).
  6. Stereotype examples (ZGWj). The introduction now names the model behind each example and gives its rate. App. I.3 reports topic risk ratios.
  7. Models (ZGWj). InternVL3-14B and -38B results are now reported for every prompt (Table 26; App. F, H.4). App. L adds Gemma-4-31B and GPT-5.5.
  8. Mitigation (Hnfd). We added a main-text paragraph, "Implications for Mitigation" (Section 5). App. B.2 covers prompt instructions, representation steering, and counterfactual-consistency fine-tuning, and App. B adds a recommendation to evaluate mitigations at the person level.

To make space, we moved the intersectional case study from Section 5 to App. I.1 and shortened some passages. We also recomputed two InternVL3-1B socioeconomic MaxToxicity values on the complete data; each changes by 0.01. We greatly appreciate the reviewers' feedback which has helped us to improve our paper during the discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.