acceptodds
Under review as a conference paper at ICLR 2027

Beyond Person-Cue Suppression: Three-View Matched Contrastive Correction for Visual Cultural Attribution

Abstract

Multimodal large language models (MLLMs) can assign different cultural origins to the same content when depicted with different people. However, suppressing person-related cues alone does not ensure correct attribution: it may obscure relevant cultural evidence while leaving attribution errors unresolved. To address this limitation, we propose Three-View Matched Contrastive Correction (TVMCC), which uses the visual intervention itself to construct a reference for correcting the suppressed prediction. Task-aware masking produces a person-suppressed view and a complementary person-focused view from the same image while seeking to preserve the queried content. By contrasting candidate preferences in the original and person-focused views, TVMCC revises the suppressed ranking using a reference matched to the current image and intervention, without additional training or test labels. Across six MLLMs on Cultural Counterfactuals and MixCuBe, TVMCC outperforms original-image predictions in all 18 model–task settings, achieving the highest reported accuracy in 17. Nationality attribution gains on Cultural Counterfactuals range from 10.4 to 16.9 percentage points, exceeding person suppression on all six models. Reference shuffling on Cultural Counterfactuals reduces accuracy even with matched correction magnitudes, supporting the contribution of same-image correspondence beyond correction strength.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Visual Cultural Attribution, Multimodal Large Language Models

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_YDfCOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents the Three-View Matched Contrastive Correction (TVMCC) method, an inference-time method for visual cultural attribution that uses complementary views from the same image to correct person-suppressed predictions. The authors prove that TVMCC improves attribution accuracy over original-image predictions, manifesting how a visual intervention can provide both a basis for prediction and a matched reference for correcting errors that suppression alone leaves unresolved.

Critical strengths

None

Critical weaknesses

None

Questions for authors

What could be the most relevant working principle for TVMCC to link visual content to culturally specific knowledge? What is the main difference between TVMCC and Cultural VQA approaches?

1 comment
AuthorsOpenReview ↗

Response to Reviewer YDfC (1/1)

Thank you for your positive feedback and your accurate summary of TVMCC. We answer your two questions below.

Q1: What could be the most relevant working principle for TVMCC to link visual content to culturally specific knowledge?

A1: The frozen MLLM already links visual content to cultural knowledge. The working principle of TVMCC is to keep this link on the task: the model should attribute the cultural content that the question asks about, and should not be misled by task-irrelevant cues, such as the appearance of the depicted person.

The principle. No single view is sufficient, so TVMCC combines three views of the same image, each covering the weakness of the others. Example from Figure 5 (Gemma4-31B): the same British clothing is worn by different people. Here the cultural content is on the person, so the person cannot simply be removed.
First, the original image holds the clothing, but also the misleading person cue: for one depicted person the model answers "America", although it answers "Britain" for the same clothing on others.
Second, the person-suppressed view removes this cue, but does not by itself restore the answer: with the head blurred, the model answers "Azerbaijan". Suppression alone is not enough.
Third, the person-focused view shows what the person region suggests, and this is what TVMCC cancels: starting from the suppressed answer, TVMCC adds the support of the original image and subtracts the support of the person-focused view. The evidence from the clothing is kept, and the preference caused by the person is cancelled. The answer becomes "Britain", which neither single view gives.

Q2: What is the main difference between TVMCC and Cultural VQA approaches?

A2: Cultural attribution is itself a cultural VQA task, and Cultural VQA approaches can also improve it. The main difference is the source of error addressed. Cultural VQA approaches reduce errors caused by insufficient cultural knowledge. TVMCC reduces errors that remain when the knowledge is sufficient: the model is misled by task-irrelevant factors, such as the appearance of the depicted person.

Three differences follow.
First, the source of error is different: an attribution error can arise because the model does not recognize the cultural content, or because it recognizes the content but follows the person. Cultural VQA approaches target the first case, by adding knowledge through fine-tuning or retrieval. TVMCC targets the second: it identifies the preference caused by the person from the image itself and cancels it at inference time, without training or extra data.
Second, the second kind of error is large and increasingly relevant: with globalization, people from one country often appear in the cultural context of another. In such images, the same scene is attributed differently depending on who is depicted (the 43.9-point gap above). Blurring the person alone raises accuracy from 63.2% to 75.9%, so these are not knowledge errors. Added knowledge does not remove the person from the image, so this interference can remain.
Third, the two are complementary: their gains come from different sources of error. Knowledge-based methods improve what the model knows. TVMCC works with any model that outputs candidate scores, so it can be applied on top of them. It also transfers without modification to the open-ended cultural value attribution task of Howard et al. [1] (Appendix J).

In summary, both aim at correct cultural attribution. Cultural VQA approaches improve what the model knows; TVMCC ensures that this knowledge is applied to the cultural content, not to the person who happens to be in the image.

Thank you again for your valuable time and constructive comments. We are happy to provide any further clarification.

[1] Howard et al. Cross-Cultural Value Attribution in Large Vision-Language Models. Findings of EMNLP 2026.

Reviewer_in7GOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper proposes a method called Three-View Matched Contrastive Correction (TVMCC) which aims to improve cultural attribution in multimodal LLMs. TVMCC applies task-aware masking to produce a person-focused and person-suppressed variant of each image, using contrasting candidate preferences over these views and the original image to produce a refined cultural attribution prediction. Experiments are conducted using six multimodal LLMs on the Cultural Counterfactuals and MixCuBe datasets, showing that TVMCC outperforms directly predicting cultural attribution from the original images in 17 out of 18 model-task settings.

Critical strengths
  1. This work proposes an interesting, effective, and (to the best of my knowledge) novel approach for improving visual cultural attribution in MLLMs. It addresses an important problem and the introduction does a good job of motivating the need for TVMCC.

  2. The experimental results convincingly show that TVMCC outperforms existing methods across a broad range of methods, with additional ablation studies demonstrating the importance of different elements of the method. The same-image reference matching analysis and case study provide further evidence of TVMCC's effectiveness.

Critical weaknesses
  1. The evaluation setting is limited to directly prompting the MLLMs to articulate the culture depicted in the image. This makes sense given that it is the most straightforward way to measure visual culture attribution, which is the focus of this work. However, it would have been helpful to see if TVMCC could be applied to other tasks which more indirectly require cultural attribution, such as cultural value attribution [2].

  2. The methodology relies on several automated components (e.g., masking, blurring) which do not seem to have been individually validated for their effect on image quality. A small-scale human evaluation and/or a larger MLLM-judged analysis of image quality after applying these operations would be helpful.

[1] Howard, Phillip, Xin Su, and Kathleen C. Fraser. "Cross-Cultural Value Attribution in Large Vision-Language Models." Findings of the Association for Computational Linguistics: EMNLP 2026.

Questions for authors
  1. How does TVMCC compare to the other methods you evaluated in terms of inference efficiency? One practical downside of the approach seems to be the additional image processing and multiple forwarded passes required for the different contrastive views.

  2. CoT reducing cultural attribution accuracy in most settings is surprising. Did you conduct any qualitative analysis of the CoT sequences to determine why this might be the case?

  3. How much does the cultural attribution accuracy vary by candidate label for different methods? e.g., does TVMCC work better for certain religious / national contexts than others?

6 comments
AuthorsOpenReview ↗

Response to Reviewer in7G (1/3)

Thank you for your valuable and positive feedback. We appreciate your recognition of the novelty of TVMCC, its motivation, and our experiments and analyses. We respond point by point below.

W1: Can TVMCC be applied to tasks that require cultural attribution more indirectly, such as cultural value attribution [1]?

A1: Thank you for this suggestion. We applied TVMCC, unchanged, to the value attribution benchmark of [1].

(1) Setup. We follow [1] and change only the decoding.
Task. "Based on the depicted {national / religious / socioeconomic} context, what {moral / ethical / political} values does this person hold?" [1]. The culture must be inferred from the scene and is never named.
Data and models. Due to time limits, we randomly sampled 15,879 CC images (100 sets per race for nationality and religion; full SES), on InternVL3 (IVL3)-1B/8B/14B/38B (checkpoints of [1]) and Qwen3.5-9B.
TVMCC. The correction of Eq. 6 is applied to each token's log-probabilities, restricted to plausible tokens by the VCD constraint. Masks, blur and coefficients are unchanged.
Protocol check. The prompts, sampling, refusal rules, judge instruction and surveys (MFQ-2, WVS) of [1] are reused. Original decoding reproduces its key findings (e.g., the SES-Authority inversion at every scale).

(2) Result 1: with TVMCC, the attributed values follow the depicted context rather than the depicted person. The context share is the fraction of variation in the attributed values explained by the context rather than the person's race (1 = only context matters).

ContextDecodingIVL3-1BIVL3-8BIVL3-14BIVL3-38BQwen3.5-9B
NationalityOriginal0.120.060.140.080.11
TVMCC0.980.970.980.940.99
ReligionOriginal0.630.740.750.720.75
TVMCC0.970.980.980.980.98
SESOriginal0.320.310.260.550.20
TVMCC0.980.990.990.990.99

Three observations follow.
First, person interference is strong at every scale: the Original context share for nationality is 0.06 to 0.14. This agrees with [1], where scale does not improve value attribution.
Second, TVMCC removes it consistently: the context share rises to 0.94 to 0.99 in all 15 cells (all paired 95% CIs exclude zero).
Third, the outputs do not become uniform: the variance explained by context rises by 1.8 to 12.3 times, while that explained by race falls by one to two orders of magnitude.

(3) Result 2: TVMCC removes the race-conditional failure reported in [1]. Bias 2 of [1] is the SES grounding gap between Middle-Eastern subjects and the other races, against the human survey (0 = no gap).

ModelOriginalTVMCC
IVL3-8B-0.21-0.01
IVL3-14B-0.26+0.02
Qwen3.5-9B-0.21-0.04

Three observations follow.
First, the gap closes in every model where it is clearly present; all three paired CIs exclude zero.
Second, the text prior is untouched: Bias 1 of [1] (the SES-Authority inversion) persists without an image, so a visual correction should not remove it. It stays negative in all models.
Third, agreement with the human surveys is preserved: the mean correlation change is within -0.04 to +0.03 for all 5 models.

(4) Result 3: no baseline achieves this. The baselines of Table 1 are transferred to generation with the same per-token rule. The text-only probe of [1] is told the context in words and sees no person; it is a reference, not a deployable method. Results on IVL3-8B:

MethodContext share (nationality)Context share (religion)Context share (SES)Bias 2 (0 = no gap)
Original0.060.740.31-0.21
Debias Prompt0.020.820.63-0.01
CF-VQA0.020.610.23-0.33
VCD0.070.680.32-0.27
AGLA0.050.690.36-0.18
OA-VCD0.110.860.50-0.18
TVMCC0.970.980.99-0.01
Text-only reference [1] (no image)1.001.001.000.01

TVMCC is the only method that both reaches the text-only reference and closes Bias 2 Three observations explain the ranking.
First, CF-VQA and VCD cannot separate person from context: they subtract a blank or noised image, which removes both.
Second, OA-VCD is the strongest baseline but incomplete: it subtracts the same person-focused view, but lacks a view of the context without the person.
Third, Debias Prompt closes Bias 2 but leaves nationality at 0.02: an instruction cannot remove a cue the model still sees.

The same holds on IVL3-14B: context share 0.98 / 0.98 / 0.99 against at most 0.23 / 0.82 / 0.70 for baselines; Bias 2 is closed (+0.02, against -0.05 to -0.25).

In summary, person interference also drives reasoning about values, and the same fixed correction removes it. Full results are in Appendix J (Tables 36 to 38).

[1] Howard et al. Cross-Cultural Value Attribution in Large Vision-Language Models. Findings of EMNLP 2026.

AuthorsOpenReview ↗

Response to Reviewer in7G (2/3)

W2: The methodology relies on automated components (e.g., masking, blurring) that have not been individually validated. A small-scale human evaluation and/or a larger MLLM-judged analysis would be helpful.

A2: Thank you for this comment. Following your advice, we manually evaluated the masked views on 300 images.

(1) Protocol. We randomly sampled 300 images with a fixed seed, 50 per category, stratified by the person's race and by the context. All views come from the pipeline in the paper, without manual correction. For each image, the annotator compared the original, the mask overlay, the suppressed view s and the focused view a, and rated the five criteria in the table. We chose human rating over an MLLM judge, because an MLLM judge is itself exposed to the person cue that the paper studies.

(2) Results. Images that fully meet each criterion (%):

CategoryC1 person cue masked (s)C2 task cues visible (s)C3 face clear (a)C4 no biasing context (a)C5 no added noise
CC nationality9870968698
CC religion98461009298
CC SES988410096100
MixCuBe Food1009610094100
MixCuBe Festival98849892100
MixCuBe Clothes1007810096100
All (300)9976999399

Four observations follow.
First, the masks follow the person accurately: the person cue is masked in s and the face stays clear in a for almost all images (C1, C3). This is what the TVMCC contrast requires.
Second, masking the person can also hide task cues, and TVMCC is designed for this case: C2 is the only criterion with a clear gap. It is not a segmentation error, since C1 and C3 are near perfect. It arises because people often stand in front of, or overlap, the cultural content. In such images s alone lacks the cue, which is why suppression alone is not sufficient (Figure 1b). TVMCC always keeps the original view o, so the hidden evidence still enters the score.
Third, the gap depends on image composition: C2 is lower on CC than on MixCuBe, and lowest for CC religion. In CC, the person stands in the centre of the frame, in front of the scene or the place of worship. In MixCuBe, the target stays visible by design: Food masks exclude the dish, and Festival and Clothes masks cover only the head.
Fourth, the masking introduces no external bias: each view only blurs part of the original image toward a grey, low-frequency version of itself. Nothing is generated, inpainted or pasted in, so all evidence used by TVMCC comes from the original image. Consistent with this, almost no image shows added noise, and none was rated severe.

The protocol, the table, and examples of valid and failed masks are in Appendix I (Table 35, Figure 6).

Q1: How does TVMCC compare to the other methods in inference efficiency?

A1: Thank you for this practical question. TVMCC needs three forward passes, each scoring a single answer token, and generates no text. Its measured cost is about 3.6 times that of original-image scoring.

End-to-end time on the same 200 CC nationality images on one A100 (ms per image; Appendix G, Table 33):

MethodForward passesGenerated tokensQwen3.5-9BIVL3-8B
Original10209 (1.0x)247 (1.0x)
Debias Prompt10212 (1.0x)250 (1.0x)
CF-VQA20422 (2.0x)511 (2.1x)
VCD20508 (2.4x)591 (2.4x)
CoT1 + reasoning traceup to 3203383 (16.2x)1055 (4.3x)
TVMCC30759 (3.6x)893 (3.6x)

Three observations follow.
First, the cost grows linearly with the number of views: each view adds one forward pass that scores a single token. TVMCC stays below CoT, which must decode a reasoning trace token by token.
Second, masks are a one-time cost per image: they depend only on the image, not on the attribution model or the question. Grounding (about 0.66 s per image at batch size 32, Appendix G) is done once and reused. The three views are also independent, so they can be scored in one batch.
Third, the cost buys accuracy: TVMCC gives the highest accuracy in 17 of 18 settings in Table 1, whereas CoT costs more and lowers accuracy in 16 of 18.

AuthorsOpenReview ↗

Response to Reviewer in7G (3/3)

Q2: CoT reduces accuracy in most settings. Is there a qualitative analysis of why this happens?

A2: Thank you for this insightful question. We analysed all CoT reasoning traces behind Table 1 on CC. CoT turns the person cue into stated evidence, and the final answer then follows it.

(1) Example. Gemma4-31B, a South Asian woman placed in France: "Step 1: The woman is wearing a traditional Indian garment ... Step 2: The clothing and jewelry are characteristic of Indian culture. Step 3: India." The French street scene is never mentioned.

(2) Statistics. We coded the CoT reasoning traces on CC nationality with a fixed lexicon. A trace contains a person-based attribution when its reasoning assigns a culture to the person's clothing or appearance, as in Step 2 of the example. Results (%):

ModelTraces with a person-based attributionCoT accuracy with / without itCorrect Original answers turned wrong, with / without it
Qwen3.5-9B4853.6 / 70.111.2 / 5.4
Qwen3.5-27B4349.1 / 65.615.4 / 9.5
IVL3-8B2443.2 / 61.418.2 / 11.6
IVL3-38B3848.0 / 68.715.9 / 9.3
Gemma4-12B4844.4 / 54.314.0 / 9.2
Gemma4-31B4459.3 / 77.99.3 / 3.5

Three observations follow.
First, CoT tends to start from the person: 70 to 87% of traces describe the person before the scene, and 24 to 48% then make a person-based attribution.
Second, person-based attribution costs accuracy: in all six models, such traces are less accurate and turn correct Original answers wrong more often.
Third, the final answer often follows the stated cue: when a trace names the culture stereotypically associated with the person's race, and it differs from the true context, CoT selects it in 47 to 74% of such cases.

In summary, explicit reasoning does not filter out the person cue and can promote it to stated evidence. This supports correcting the cue at the score level. Full analysis is in Appendix K (Table 39), and Section 4.2.

Q3: How much does accuracy vary by candidate label? Does TVMCC work better for certain religious or national contexts?

A3: Thank you for this question. Yes: a label gains more where the depicted person misleads the model more.

(1) Per-label accuracy. CC nationality (%, mean over six MLLMs; best baseline: OA-VCD, or Debias Prompt):

LabelOriginalBest baselineTVMCC
Brazil56.766.081.9
Germany71.479.988.5
China60.765.376.8
France49.858.865.2
United States79.787.990.5
India76.478.586.7
South Africa40.247.050.1
Morocco70.975.979.4

Three observations follow.
First, TVMCC improves every label, and leads the best baseline in 44 of 48 model-label cells.
Second, the gain is largely predictable: it correlates with the share of a label's images whose person-only answer differs from the true label (Spearman 0.55).
Third, the errors removed are mostly stereotype-consistent: on nationality, the share of predictions that wrongly name the culture stereotypically associated with the person's race falls from 19.4% (Original) to 2.1% (TVMCC) (Appendix M).

(2) Accuracy gap between depicted races within the same label. Max-minus-min accuracy across races, averaged over labels and models:

SettingOriginalDebiasCoTCF-VQAVCDAGLAOA-VCDTVMCC
CC nationality43.931.745.839.841.645.229.76.7
CC religion18.416.019.319.918.722.912.23.9
MixCuBe15.712.619.214.014.117.611.23.7

Two observations follow.
First, a label's mean can hide a large person effect: with Original decoding, the gap between races averages 43.9 points on nationality. TVMCC brings it down to a few points.
Second, the person cue can both penalize and reward: for Mosque, accuracy falls for Middle-Eastern subjects and rises for East and South Asian subjects. The mean stays flat, but the race gap shrinks from 22.1 to 4.3 points.

(3) What remains is mostly scene difficulty. With the person removed, about half of the synagogue images are still classified as a Christian church. The remaining differences between labels mainly reflect scene knowledge, which a person-cue correction cannot fix. Full tables are in Appendix L (Tables 40 to 46).

Thank you again for your valuable time and constructive comments. We are happy to provide any further clarification.

Reviewer_nfH3OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper looks at how the person in an image can push MLLMs toward the wrong cultural attribution. The authors propose TVMCC: they blur the person to get a suppressed view, blur everything except the person to get a focused view, and then score with s + o − a (suppressed + original − focused). No training is needed. They test six MLLMs on Cultural Counterfactuals and MixCuBe and report gains over the original-image prediction in all 18 settings. They also do a shuffling experiment to argue that using the focused view from the same image matters.

Critical strengths
  1. I really like how thorough and honest the evaluation is. There are paired bootstrap CIs, fix/break counts, the full grid of score combinations, and the authors also report results that don't favor their method. It made the paper easy to check.

  2. The method is simple and needs no training, and the norm-matched shuffling control is a nice touch.

Critical weaknesses
  1. My main concern is that most of the improvement seems to come from suppression and the mask, and the correction itself adds little. Compared to same-mask suppression, TVMCC only adds around 0.2–3 pp, and three of those CIs include zero. On religion and MixCuBe, just using o + s is about as good. On CC religion, DeepLab-only suppression is better than grounded TVMCC for all six models. Suppression is also not in Table 1, and the MixCuBe "suppression only" point in Fig. 3 uses different masks, so I think the headline overstates what the correction does.

  2. I'm also not convinced the correction works for the reason given. In CC, the label only depends on the background, so anything that pushes toward non-person evidence will help. On MixCuBe, where the person can carry real cultural cues, the gains are smaller, and clothing accuracy drops for three models. On religion, blank and mean references do as well as or better than the same-image one, and the shuffling test only covers CC and the three smaller models.

Questions for authors

See weakness

6 comments
AuthorsOpenReview ↗

Response to Reviewer nfH3 (1/3)

Thank you for your detailed and constructive comments, and for recognizing our evaluation. All revisions are included in the updated paper.

We divide W1 into five parts:

  • Q1.1: The improvement from the correction is limited.
  • Q1.2: Combining the original and suppressed views performs comparably on religion and MixCuBe.
  • Q1.3: DeepLab-only suppression performs better on CC religion.
  • Q1.4: Suppression is not included in Table 1.
  • Q1.5: The MixCuBe point in Fig. 3 uses different masks.

Q1.1: The improvement from the correction is limited.

A1.1: The increment over same-mask suppression is indeed 0.2 to 3 points. It is small because suppression and the correction fix overlapping errors, so whichever is applied second adds less. Applied on its own, the correction improves every setting, and TVMCC exceeds same-mask suppression in all 18 settings.

(1) The correction is effective without suppression. We apply the same person-focused contrast to the original image instead of the suppressed image (Section 4.4; Appendix D, Tables 18 and 19). "Suppression" is the task-aware suppressed view, the first of the three modules of TVMCC, used alone, not generic person suppression. The reference point is Original: standard scoring of the unmodified image. Mean accuracy over six models (%, higher is better), with the gain over Original in parentheses:

TaskOriginalVCD (noise reference)Suppression alone (module 1)Correction aloneBoth (TVMCC)
CC nationality63.2163.81 (+0.60)75.86 (+12.65)69.75 (+6.54)77.38 (+14.17)
CC religion77.2977.85 (+0.56)80.19 (+2.90)79.72 (+2.43)81.49 (+4.20)
MixCuBe87.7888.07 (+0.29)89.39 (+1.61)89.87 (+2.09)90.82 (+3.04)

Per model: accuracy of the correction alone minus accuracy of Original (pp; e.g., Qwen3.5-9B, nationality: 72.04 - 64.47 = +7.57):

ModelCC nationalityCC religionMixCuBe
Qwen3.5-9B+7.57+2.94+3.03
Qwen3.5-27B+8.01+3.66+2.77
InternVL3-8B+8.01+1.86+1.51
InternVL3-38B+8.78+2.68+1.86
Gemma4-12B+3.65+1.57+2.07
Gemma4-31B+3.21+1.91+1.32

Three observations follow.
First, the correction alone improves all 18 model-task settings: on MixCuBe, it gains more than suppression alone.
Second, the matched reference is what makes it work: VCD uses a noise-distorted reference and gains at most 0.60.
Third, the two components fix overlapping errors: applied first, the correction adds 2.09 to 6.54; applied after suppression, 1.30 to 1.52.

(2) Suppression alone is not a reliable basis. On MixCuBe, person-level suppression falls below Original for all six models (87.78 to 84.91). Task-aware suppression avoids this loss, but still falls below Original in 5 of the 18 model-category settings; TVMCC is at or above Original in all five. In the three settings whose accuracy intervals include zero, TVMCC is still more accurate than suppression and makes significantly fewer person-driven errors (Appendix M).

Q1.2: Combining the original and suppressed views performs comparably on religion and MixCuBe.

A1.2: Adding the original and suppressed scores (Original + Suppression) is indeed close to TVMCC on these two tasks. However, it does not generalize across tasks, and it keeps more person-driven errors.

(1) TVMCC generalizes across tasks; Original + Suppression does not. "Suppression" is the first module of TVMCC (task-aware suppression), not generic person suppression, which falls below Original on MixCuBe (A1.1). Mean accuracy over six models (%, higher is better). The number in parentheses is the rank among the nine strategies of Tables 13 to 15 (1 = most accurate; Appendix D):

StrategyCC nationalityCC religionMixCuBeSettings where it is the best (of 18)
Suppression (module 1 of TVMCC)75.86 (2)80.19 (3)89.39 (4)0
Original + Suppression72.17 (4)81.11 (2)90.67 (2)3
TVMCC77.38 (1)81.49 (1)90.82 (1)15

Two observations follow.
First, no alternative is second best on every task: Original + Suppression ranks second on religion and MixCuBe, but fourth on nationality, where it falls below plain suppression for all six models.
Second, TVMCC ranks first on all three tasks: it is the best strategy in 15 of 18 settings.

(2) Original + Suppression keeps more person-driven errors. We count stereotype-consistent (SC) errors: wrong answers that follow the depicted person instead of the scene. For example, for a South Asian person shown in a Brazilian street, the answer "India" instead of "Brazil" follows the person's ethnicity. The mapping is fixed in advance. SC error rate (% of images, lower is better; mean over six models):

TaskOriginal + SuppressionTVMCC
CC nationality10.052.09
CC religion8.235.63
MixCuBe4.592.79

TVMCC has fewer SC errors in 17 of 18 settings.

AuthorsOpenReview ↗

Response to Reviewer nfH3 (2/3)

Q1.3: DeepLab-only suppression performs better on CC religion.

A1.3: DeepLab-only suppression is not the suppression module of TVMCC. It is the alternative mask of Section 4.3: the person region comes from the DeepLabV3 segmenter alone, without grounding or main-subject selection (Appendix B). The comparison in the review therefore changes the mask and the scoring together. With the mask held fixed, the correction still helps. Mean over six models (accuracy %, higher is better; SC errors %, lower is better; Appendices B and M):

TaskModule 1 of TVMCC (grounded mask): SuppressionGrounded mask: TVMCCDeepLab mask: SuppressionDeepLab mask: TVMCCSC errors, DeepLab mask: Suppression / TVMCC
CC nationality75.8677.3876.8477.962.98 / 2.53
CC religion80.1981.4983.1583.565.77 / 5.02

Three observations follow.
First, the mask and the correction are separate components: DeepLab masks raise both suppression and TVMCC on CC, and the best average result on both tasks is TVMCC on DeepLab masks.
Second, the correction still removes person-driven errors on the better mask: SC errors fall in 11 of 12 settings. On religion the accuracy gain is small (+0.41), because a mask that keeps more of the scene leaves less to correct.
Third, a person segmenter does not transfer to MixCuBe: there the person carries cultural evidence, and person-level suppression falls below Original (A1.1). The grounded pipeline therefore stays the default, and DeepLab is reported as a stronger mask for CC.

Q1.4: Suppression is not included in Table 1.

A1.4: Same-mask suppression is the first module and the foundation of TVMCC, not an external baseline. We therefore reported it in the ablation study (Fig. 3) and in Tables 2, 11 to 15 and 29. The revised Table 1 now reports it as the "Supp." column under "Ours" Averages (Original / Suppression / TVMCC): nationality 63.21 / 75.86 / 77.38; religion 77.29 / 80.19 / 81.49; MixCuBe 87.78 / 89.39 / 90.82. TVMCC exceeds same-mask suppression in all 18 rows, and the best method does not change in any row.

Q1.5: The MixCuBe point in Fig. 3 uses different masks.

A1.5: In the MixCuBe panel, the "suppression only" point removed the correction and the task-aware mask together. The revised Fig. 3 uses same-mask suppression in all panels, and the mask is ablated separately (Appendix D, Table 17). MixCuBe accuracy (%, higher is better; mean over six models):

CategoryOriginalPerson-level mask: SuppressionPerson-level mask: TVMCCTask-aware mask (module 1 of TVMCC): SuppressionTask-aware mask: TVMCC
Food83.6683.8985.9683.8985.96
Festival89.5778.2179.5791.6493.28
Clothes90.4392.6393.2192.6393.21
All87.7884.9186.2589.3990.82

Two observations follow.
First, the correction helps under both masks and in every category.
Second, the task-aware mask matters when the person carries evidence: for festivals, suppressing the whole person also removes costumes and props, and accuracy falls below Original; suppressing only the head keeps them. Food and Clothes use the same mask in both settings.

We divide W2 into four parts:

  • Q2.1: In CC, anything that pushes toward non-person evidence will help.
  • Q2.2: On MixCuBe, the gains are smaller, and clothing accuracy drops for three models.
  • Q2.3: On religion, blank and mean references do as well as or better than the same-image reference.
  • Q2.4: The shuffling test only covers CC and the three smaller models.

Q2.1: In CC, anything that pushes toward non-person evidence will help.

A2.1: We tested this hypothesis directly. Pushes that are not tied to the depicted person fall below Original or below plain suppression. Only the reference built from the same person improves on suppression (new Section 4.6, Table 3). TVMCC uses the person-focused view of the same image as its reference. We replace it with a blank grey image or with the mean person-focused scores of other images (App. D), and we replace the person mask with a random region of the same area. CC nationality, mean over six models (accuracy %, higher is better; SC errors %, lower is better):

MethodAccuracySC errors
Original63.2119.39
Suppression of a random region55.2515.03
TVMCC built on a random region58.6113.14
TVMCC with a blank reference72.799.49
TVMCC with a mean reference74.019.72
Suppression (module 1 of TVMCC)75.862.26
TVMCC (same-image reference)77.382.09

Two observations follow.
First, the region must be the person: suppressing a random region falls below Original, and so does TVMCC built on it.
Second, person-agnostic references do not improve on suppression: blank and mean references fall below it for all six models, with about four times more SC errors. Only the same-image reference improves on it.

AuthorsOpenReview ↗

Response to Reviewer nfH3 (3/3)

Q2.2: On MixCuBe, the gains are smaller, and clothing accuracy drops for three models.

A2.2: MixCuBe is the setting where the person can carry real cues, and TVMCC handles it through the task-aware mask. The gains are smaller in points because Original accuracy is already 87.78%; in error terms they exceed those on religion.

Three observations follow.
First, the task-aware mask keeps the cues the person carries: for Festival and Clothes it covers only the head, so garments and props stay visible. The correction then subtracts the preference induced by the face, not the evidence in the clothing.
Second, the MixCuBe gain is not small in error terms: TVMCC removes 25% of the Original errors on MixCuBe, against 18% on CC religion.
Third, on Clothes, TVMCC never falls below Original: it gains up to 7.27 points, and SC errors fall for all six models (Appendix M, Table 55). The three drops are relative to suppression: 2 to 6 of 660 images, for models already at 93 to 96% accuracy.

Q2.3: On religion, blank and mean references do as well as or better than the same-image reference.

A2.3: On accuracy this is correct: blank and mean references reach 81.89% and 82.16% on average, against 81.49% for TVMCC (Appendix D). They gain by a different route: a fixed reference shifts the label prior, and it does not remove the pull of the depicted person.

We take all images where adding a reference changes the suppressed answer. "Same image" is the default reference of TVMCC; "Blank" and "Mean" are the blank and mean references of A2.1. We report how often the abandoned answer is the stereotype label of the depicted person, and how often the change fixes an error (%, Appendix D, Table 24):

TaskAbandoned answer is the stereotype label: Same image / Blank / MeanChange fixes an error: Same image / Blank / Mean
CC nationality6.8 / 1.3 / 1.143.9 / 23.5 / 26.3
CC religion23.7 / 16.1 / 17.353.1 / 50.4 / 52.0
MixCuBe24.2 / 14.4 / 17.657.6 / 52.6 / 57.1

Three observations follow.
First, a fixed reference cannot depend on who is in the image: it is the same for every image, so subtracting it shifts all scores by one constant vector. The same-image reference changes with the person.
Second, only the same-image reference targets the person: it abandons the stereotype label most often in 17 of 18 settings. On religion, blank and mean references make more SC errors than TVMCC for all six models (7.86% and 7.45% against 5.63%; lower is better; Appendix M, Table 54).
Third, the same shift hurts where the person cue is strongest: on nationality, blank and mean references break more answers than they fix and fall below plain suppression (A2.1).

A prior shift is a calibration effect and is complementary to the correction. We therefore do not claim that the same-image reference gives the highest accuracy in every setting. It is the reference that addresses the cause studied in the paper, and it improves on suppression in all 18 settings.

Q2.4: The shuffling test only covers CC and the three smaller models.

A2.4: We extended the test to all six models and both datasets, with the protocol of Appendix D (Table 26). Each value is the accuracy of TVMCC with its own (matched) reference minus the accuracy of TVMCC with a reference shuffled from another image and rescaled to the same correction strength (pp; positive means the matched reference is better):

ModelCC nationalityCC religionMixCuBe
Qwen3.5-9B+1.40+1.57+0.86
Qwen3.5-27B+1.74+1.34+0.82
InternVL3-8B+2.43+1.15+1.28
InternVL3-38B+1.81+1.45+1.03
Gemma4-12B+1.10+0.99+1.41
Gemma4-31B+1.36+0.73+1.05
Pooled over six models+1.64+1.21+1.08

Two observations follow.
First, the matched reference is better in all 18 settings: on CC, all 12 intervals exclude zero, and the larger models behave like the smaller ones.
Second, MixCuBe is positive for every model: with 495 items per model the per-model intervals are wide, and the advantage is significant when the six models are pooled.

In summary: 1) the correction contributes beyond suppression: it is effective on its own, and TVMCC exceeds its suppression module in all 18 settings; 2) TVMCC generalizes across tasks: it ranks first on all three, and each alternative falls behind on at least one.

Revisions. Table 1 ("Supp." column under "Ours"); Fig. 3 (same-mask suppression in all panels); Section 4.3 and Appendix B (DeepLab masks on CC); Section 4.4 (correction without suppression); new Section 4.6 with Table 3 (person-agnostic controls); extended Appendix D (Tables 17 to 20, 22 to 24 and 26); new Appendix M (SC errors, Tables 47 to 55).

Thank you again for your valuable time and constructive comments. We are happy to provide any further clarification.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.