acceptodds
Under review as a conference paper at ICLR 2027

Scalable Audit and Mitigation of Intersectional Fairness Bugs in VLMs with Combinatorial Interaction Testing

Abstract

Intersectionality analysis is critical for fairness, since individuals hold multiple and overlapping identities whose intersections shape how they are perceived and treated, bringing distinct disadvantages as well as privileges. This matters even more for algorithmic systems such as Vision-Language Models (VLMs), which increasingly serve as auditors of other models, or as backbones of downstream applications. However, VLM fairness is often evaluated one axis at a time, leaving failures that emerge only in higher-order intersectional subgroups undetected: we frame this problem as intersectional fairness bugs. Such bugs remain hard to find: the number of attribute combinations grows combinatorially, making it prohibitively expensive to collect and label data for every subgroup. We propose a scalable and tunable audit for VLMs built on Combinatorial Interaction Testing (CIT) from software testing. Given a set of parameters and their potential values, CIT aims to test all interactions between at most parameters, instead of testing all combinations of parameter values, thus controlling the combinatorial explosion. Using CIT in conjunction with diffusion-based synthesis, we show that it is possible to both audit VLMs for intersectional fairness bugs and mitigate them by focusing on specific hard subgroups or augmenting training data with synthetic images representative of subgroups. Our empirical evaluation of CIT-based audit and mitigation shows that 1) 60 subgroup accuracies on synthetic images correlate with those on real images for four VLMs (Pearson –; Spearman –), and 2) when fine-tuning CLIP, audit-guided data allocation improves the mean accuracy of the worst 10% of subgroups () by up to 4.0 pp under small budgets, and mixing CIT images into real images improves it by up to 2.2 pp under tail data scarcity.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Intersectional Fairness, Vision-Language Models, Combinatorial Interaction Testing, Synthetic Test Input Generation, Targeted Bias Mitigation, Diffusion Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_avGrOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

This paper tackles the problem of intersectional bias in Vision-Language Models (VLMs). Intersectionality refers to the fact that a person can be defined by many axes at the same time (e.g., gender, age, disability, etc…) but often fairness is assessed on a single axis. One complication of intersection audit is the fact that the number of possible combinations grows combinatorially so it becomes often an intractable problem (and it can leave very few real labeled samples for every subgroup). The authors propose instead to use a techniques borrowed from software testing known as Combinatorial Interaction Testing (CIT). Given a set of axes and their potential values (called here attributes), a t-way CIT test set covers every combination of attributes of any t axes at least once (this is useful when t << number of axes).

The authors start with a list of 8 axes and 2, 3, and 4-way sets (reducing the total number of possible combinations from 362K to 2847 for the 4-way). Each row of the set becomes a prompt they use to generate 50 images using SDXL. After manual inspection of 690 images from the 2-way set two axes (faith and socioeconomic status) are dropped from scoring, although they stay in the prompts and images in order to keep the coverage complete. Some attributes from the “disability” and “appearance” axes are also dropped (only "wheelchair / no disability" and "fat / thin" are kept) because the SDXL model does not generate those axes and attributes reliably. The prompt's attributes are then used as ground truth.

The audit involves 4 different VLMs. For each image and each scored axis, the authors check whether the model recognizes the attribute named in the prompt. Three metrics are extracted: Attribute Recognition Accuracy for every axis, Mean Pairwise Delta (MDP), which measures how much the model confidence in an attribute moves when one other axis changes while the rest remains fixed (e.g., ability to recognize gender when age changes), and MinMax Fairness, which measure the worst-case accuracy for a given target axis across the values of the varying axis. The result of this audit is that every model has weak points at every t, and they mostly involve age and gender. The per-subgroup accuracy on synthetic images correlates with accuracy on real photos (UTKFace + FairFace, "UTK-FF") for all four models.

Then the authors show how the per-subgroup accuracy from the audit can be used for mitigation. Specifically, they fine-tune CLIP ViT-B/32 with a limited budget of N = 60 × K training images (K per subgroup on average). Each synthetic subgroup (called cell in the paper) is given a weight that is 1 - its synthetic-image accuracy, so more images are given to more difficult subgroups (“difficulty-weighted” or DW). Training on pure synthetic images gives worse accuracy and fairness metrics compared to using real images. Using real images (from UTK-FF) with the authors' DW improves faster than Real-Balanced (the same number K of real images for every cell) when the budget is small (e.g., K=8), but the advantage disappears as the budget grows. As the budget increases, a mixed approach helps: it keeps N fixed but replaces 25% or 50% of the real images with synthetic ones. Synthetic images fill these cells with new samples instead. At K = 128, Mixed-Balanced 50% raises one of the fairness metrics ( CVaR 0.1: the average accuracy of the worst 10% of subgroups) by 2.2 points over Real-Balanced (71.7% vs 69.5%).

Critical strengths
  • The paper tackles an important and difficult problem.
  • The method used could be of practical use.
Critical weaknesses

Soundness

  • One of the main claim is the ability to find “fairness bugs” in VLMs but one of the main problem is that it is not possible to isolate if “fairness bugs” found are VLM bugs or the image generator bugs.
    • The manual check was only done for t=2 and no info about this check is provided to understand if this was a reliable audit. E.g., what does “they are not realized consistently enough “ mean?
  • Additionally it is hard to distinguish if what is found is a fairness bug or something objectively more difficult to judge (e.g., age of infants)
  • The CIT is only partially used for mitigation. The papers shows mitigation only on 3 axes, for a total of 60 cells, but if I understand correctly that amounts to the full cartesian product so one could have done this audit and mitigation without CIT.
  • The mitigation gains could be an artifact of under-training. For K=8 the model is trained only with six steps (for K<=4 only 3 steps). In this regime, any method that unweights hard cells may do well simply because training stops before anything else can catch up. It would be better to think of ways to separate the training length from the data size? Maybe at small K train with more epochs? Or train each run to converge on a validation split?
  • One of the real problem of intersectionality is the sacristy of data and this method does not seem to be able to tackle this, not even in the synthetic audit. The CIT method only guarantees at least one prompt per combination. This means its 50 images are probably near copies of each other so it is not surprising that the actual results show a marginal improvement.
  • The ACC metrics for the 3 VQA models ignores false positives. So always answering yes would score 100% accuracy. Consider using per-class precision and recall or something that accounts also for false positives.

Some missing experiments

  • Simple random sampling baselines for the audit (and for the mitigation) are missing. CIT guarantees coverage, but whether it finds more or different bugs than random designs of the same size is not tested.
  • The DW strategy is lacking a comparison: random weights, or weights based on real-data difficulty measured on a small labeled set.
  • How much the fine-tuning affects the models is only tested on the same task but not on generic capabilities.
  • Also worth re-running the complete audit after the mitigation fine-tuning. Has the fine-tuning reduced the weak points previously identified?
  • Results do not seem unforeseeable from the single axes test. Age was already the hardest axis for CLIP by itself and most of CLIP's weak points involve age, either as the target or as the axis that shifts recognition of another one. Perhaps a convincing argument could be to show that by simply adding randomly sampled images across different values for those axis identified individually is not as good as using the more complex mix strategy?
  • The authors say that the weak points replicate on held-out prompts, but those are only tested within the same image generator.

Results are weak

  • CLIP correlation with UTK-FF is somewhat low r = 0.59, \rho = 0.56.
  • The gains are small: the DW gain disappears for larger K, and the Mixed gain is quantified only at K = 128.

Novelty is limited

  • As explained by the authors, CIT with Text-to-Image was already introduce in 2023. Additionally CIT for choosing training data was also already introduced in Cody et al., 2022 (“Systematic Training and Testing for Machine Learning Using Combinatorial Interaction Testing”).
  • The novelty relies on these applications of CIT to a slightly different tasks. The combination of limited novelty, lack of some baseline to see if CIT actually brings value and the weak results makes this work difficult to accept as is.

Ethical use

  • The use of axis like “race” to be detected in images is a questionable choice. Race or ethnicity have complex definitions and related to attributes that are not always identifiable in images. I find this especially critical for a fairness paper. I would recommend to use more objective axes instead, for example the skin color using the Monk Skin Tone scale.
  • The authors use UTKFace dataset which is a web-scraped dataset of faces collected without consent. In general, but especially in a fairness paper the use of these kind of datasets is questionable. I know that in practice many work use this dataset and that it is hard to find valid alternatives so I would not consider this a reason for rejection but it is also worth thinking about this.

Minor but worth thinking about it.

  • I am not sure about the use of the term “bug” for fairness topics. Regardless of the fact that might or not have been used in other works using this terms seems to imply an unintended unaware error that can be quickly fix. As also this paper show this is not really the case: these issues are foreseeable, not due to a “mistake” and definitely not easy to fix.
  • One of the metrics used is confidence but I do not see any evidence or tests that these models are calibrated. If they are not calibrated is this metric reliable?
Questions for authors
  • Could you show the value of CIT for mitigation in a case where you are not using the full cartesian product?
  • Is there a way to disentangle bugs due to the generator from those due to the VLM?
  • Can you show that the gains at small K is not artifact of the training set up? See my suggestions above.
  • Can you report a better metric than ACC, one that would not be 100% for a model that always answers YES?
  • Can you add the baselines mentioned above?
  • Is the test split for the mitigation experiment completely separated from the training split? Have you checked that some near duplicate did not end up in both splits?
  • The authors measured correlation but maybe it’s more informative to measure whether the worst cells agree, for example how many of the 6 worst synthetic cells are among the 6 worst real ones.
33 comments
AuthorsOpenReview ↗

Thank you for the detailed review (1/31)

We thank Reviewer avGr for the detailed review. Several of the suggested baselines and controls were concrete and actionable, and we have now run them ([W1, W4 - W9, W12 - W15, M2]); the results and analysis are reported below. We have also adopted the reviewer's point on terminology and plan to replace "fairness bug" with "fairness failure" throughout the paper (M1). On one point, we respectfully hold a different view: the use of perceived race as an audit axis (E1). We state our position and explain below why we believe it is the appropriate choice for this audit. We welcome further discussion on any of these points, including the point for which we differ. We respond to every point below. Those that bear directly on our main claims are marked [Main claim].

AuthorsOpenReview ↗

[Main claim] Q1 / W4. Contribution of CIT (2/31)

Q1: Could you show the value of CIT for mitigation in a case where you are not using the full cartesian product?

W4: The CIT is only partially used for mitigation. The papers shows mitigation only on 3 axes, for a total of 60 cells, but if I understand correctly that amounts to the full cartesian product so one could have done this audit and mitigation without CIT.

You are right that the 60 cells are the full product of three axes, and this is by design (Sec. 5). Harm falls on people, not on axes: ETH×AGE→GEN tells us where a disparity concentrates, but not who bears it. As Sec. 5 states, mitigation "aims to close the gap among subgroups," so every subgroup must be an enumerable cell we can train and evaluate on; skipping cells would recreate the blind spots intersectional analysis aims to remove. At this level, CIT does not reduce the space, and we do not claim it does.

What CIT provides is upstream, and it is precisely what makes the pipeline usable beyond this one scenario.

(1) One generation run serves every target intersection. The t=4 covering array contains every 4-way, and therefore every 3-way, attribute combination over the eight axes. Whichever three axes a practitioner chooses to intervene on, all of that scenario's cells are already present in the pool. Covering all 56 three-axis scenarios with purpose-built sets would require 8,237 cell-specific prompts; the t=4 array covers all of them, as well as every 4-way cell, with 2,847 prompts. The gender×age×race&eth scenario was not built into the array; it is one of the 56 triples the array covers by construction.

(2) Reused audit images are as effective as purpose-built ones. The natural concern is that reuse costs quality: CIT images also vary the other five axes, which might dilute the target cells. To test this directly, we generated a purpose-built set for this scenario: one prompt per cell specifying only race, gender, and age, with 128 images per prompt (7,680 images). We swapped it in as the synthetic half of Mixed 50%, keeping everything else fixed (seeds 0–9).

AllocationKΔCVaR₀.₁ (pp)ΔMPD
Mixed-Balanced 50%8−0.76 [−1.72, 0.20]+0.0081 [0.0037, 0.0125] → audit
Mixed-Balanced 50%32+1.18 [−0.03, 2.39]−0.0010 [−0.0041, 0.0021]
Mixed-Balanced 50%128+0.12 [−0.53, 0.77]−0.0016 [−0.0040, 0.0008]
Mixed-DW 50%8−0.61 [−1.93, 0.71]+0.0029 [−0.0026, 0.0084]
Mixed-DW 50%32+0.85 [0.25, 1.45] → purpose-built−0.0027 [−0.0046, −0.0008] → purpose-built
Mixed-DW 50%128+0.56 [−0.11, 1.23]−0.0030 [−0.0053, −0.0007] → purpose-built

Δ = purpose-built 3-axis − reused t=4 audit images, paired by seed; brackets are 95% CIs over seeds 0–9. Higher CVaR₀.₁ and lower MPD are better, so a positive ΔCVaR₀.₁ or a negative ΔMPD favors the purpose-built set. Bold: the interval excludes zero, followed by the source it favors.

In 8 of 12 comparisons the interval includes zero. The four that exclude zero point in both directions (three favor the purpose-built set, one favors the audit images), and all point estimates are within 1.2 pp CVaR₀.₁ and 0.009 MPD. Those CVaR₀.₁ differences remain smaller than the effect of mixing itself (+1.5 pp over Real-Balanced at K=128; 71.6 vs. 70.1), but the largest point estimate, 1.18 pp, is now close to that gain. Purpose-built images therefore give no consistent advantage, even though they were tailored to this exact scenario and the audit images were not.

(3) The audit chooses the target. Among 362,880 combinations, ETH×AGE→GEN was CLIP's top t=3 weak point by MPD (Fig. 1, Tables 10 and 11), which is why we chose this scenario. The top t=4 weak point under young-adult MPD is GEN×LOC×DIS→AGE (Sec. 5.3, Table 11). It could not be used only because no photo dataset labels all four attributes at a usable scale. Under MinMax, GEN×APP×DIS→AGE is still first, and the same data constraint applies. Given such data, the same images apply without regeneration.

Together, a single CIT generation run both identifies where to intervene and supplies mitigation data for any intersection it covers, with no consistent loss relative to generating a dedicated set per scenario. One caveat we will add: the covering array guarantees every cell at least one prompt (50 images), so for high-cardinality triples (e.g., location × disability × faith, 336 cells) the per-cell pool is smaller than in the scenario studied here.

AuthorsOpenReview ↗

[Main claim] Q2 / W1, W2, W3, W13. Generator Failure vs. VLM Failure (1) (3/31)

Q2: Is there a way to disentangle bugs due to the generator from those due to the VLM?

W1: One of the main claim is the ability to find “fairness bugs” in VLMs but one of the main problem is that it is not possible to isolate if “fairness bugs” found are VLM bugs or the image generator bugs.

W2: The manual check was only done for t=2 and no info about this check is provided to understand if this was a reliable audit. E.g., what does “they are not realized consistently enough” mean? 

W3: Additionally it is hard to distinguish if what is found is a fairness bug or something objectively more difficult to judge (e.g., age of infants)

W13: The authors say that the weak points replicate on held-out prompts, but those are only tested within the same image generator.

Q2, W1, W13 — Generator vs. VLM

Q2 and W1 ask whether a weak point found on synthetic images can be attributed to the VLM rather than to the image generator, and W13 notes that the replication was tested only within the same generator. Our answer is yes for the weak points whose axes have real labels, and we ran a stricter version of the check W13 asks for: rather than re-rendering with a second generator, we tested these weak points with no generator at all. AGE→GEN and ETH×AGE→GEN involve only gender, age, and race, which UTK-FF labels. We computed the same metrics on all 97,455 UTK-FF photographs, using the zero-shot predictions behind Fig. 2.

ModelAGE→GEN MPD ↑worse (syn → real)AGE→GEN MinMax ↓worse (syn → real)ETH×AGE→GEN MPD ↑worse (syn → real)ETH×AGE→GEN MinMax ↓worse (syn → real)
CLIP0.092 → 0.1060.886 → 0.7040.054 → 0.0680.941 → 0.785
ViLT0.205 → 0.2120.278 → 0.3180.109 → 0.1190.625 → 0.514
Qwen2.5-VL0.104 → 0.1560.731 → 0.5900.054 → 0.0940.873 → 0.726
PaliGemma 20.024 → 0.0800.973 → 0.7970.014 → 0.0540.983 → 0.839

Synthetic values come from the array where each item was found (t=2, 69 prompts, for AGE→GEN; t=3, 469 prompts, for ETH×AGE→GEN). All values are raw. Chance correction in the paper ranks items with different cell sizes using a prompt-block permutation, which does not apply to photographs; here each row compares one item with itself. Raw values, if anything, favor the synthetic side: its groups hold about 12–16 prompts against at least 208 photographs on UTK-FF, so noise inflates synthetic MPD and lowers synthetic MinMax more.

On photographs, MPD is at least as large for every model on both items, and MinMax is lower in seven of the eight cases (the exception is ViLT on AGE→GEN). The per-cell pattern also agrees. Infants have the lowest gender accuracy for all four models on both sides, and Spearman ρ over the 30 race×age cells is 0.75–0.92. These disparities exist without any generator, so the generator is not needed to explain them. We will add this table to the revision.

For weak points involving appearance, disability, or location, no labeled photographs exist at a usable scale. For those, synthetic images alone cannot fully separate the two sources, as stated in Sec. 7, and synthesis remains the only practical route to an audit. The pipeline does not depend on SDXL: the generator is a replaceable component, so these weak points can be re-rendered with a different generator and checked in the same way.

Reviewer_K5FkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a framework for auditing and mitigating intersectional fairness across different demographic groups in VLMs. It combines Combinatorial Interaction Testing with synthetic image generation. The proposed approach efficiently explores different combinations of demographic attributes to identify fairness disparities. The authors also introduce two mitigation strategies based on the auditing results, using difficulty-weighted sampling and synthetic data augmentation. Experiments show that the proposed approach can identify subgroup recognition disparities and improve worst-subgroup accuracy.

Critical strengths
  • The paper addresses the combinatorial explosion of intersectional fairness evaluation, which is usually omitted by most current fairness studies and benchmarks. It is a good complement to conventional single-axis fairness assessments.

  • The idea from software testing makes sense in the intersectional fairness domain with balanced interaction coverage against computational cost. And the method itself can be applied to different architectures of VLMs

  • Extensive empirical experiments.

Critical weaknesses
  • The VQA probes only ever ask questions whose answer is "Yes.", making the problem over simplied. A model that leans toward saying yes looks good under this setup, so it's hard to tell whether the gaps come from what the model "sees".
  • The paper defines intersectional fairness bugs primarily through overall accuracy disparities. Why is this particular fairness notion appropriate for evaluating VLMs? In particular, recognition disparities may arise from inherent differences in visual ambiguity across different groups rather than the model learning discriminatory patterns. Would this lead to a biased result relying on a single metric?
  • I am a little concerned about the dataset construction. For example, some categories like "Adult" and "Middle-aged" and then the model was asked to answer if someone is adult / if someone is middle-aged, this type of question may be ambiguous. Also, the authors merge UTKFace and FairFace, but UTKFace's broad "Asian" category is directly mapped to "East Asian," and Latino samples come exclusively from FairFace. I wonder whether there might be some distribution shifts between these two datasets that could affect the fairness evaluation results.
Questions for authors

see above

7 comments
AuthorsOpenReview ↗

W1. Yes-only probes. (1) (1/N)

We thank the reviewer K5Fk for comments that directly strengthened the validity of our measurement. Especially, prompted by the concerns about the overlap between age label, we re-ran part of the generation and audit and all mitigation experiments; the updated results are reported at common response and paper.

W1. Yes-only probes.

The VQA probes only ever ask questions whose answer is "Yes.", making the problem oversimplified. A model that leans toward saying yes looks good under this setup, so it is hard to tell whether the gaps come from what the model "sees."

Thank you for raising an important design question. We first explain 1) why we made this choice. We then 2) examine the concern using the results already in the paper. Finally, we describe 3) additional experiments that address it more directly.

1) Design choice. We intended to frame the audit as verification rather than classification, because the harm we target is a person not being recognized as who they are. For example, when a VLM is used to label the outputs of another model (Sec. 3.1), failing to recognize an elderly woman as female removes her from the count and hides the bias of the audited model. This failure corresponds to per-group TPR, i.e., equal opportunity (Hardt et al., 2016). Asking only whether the true attribute applies also keeps the probe from acting as a demographic classifier that chooses among labels (Ethics Statement). We acknowledge that this choice was not stated clearly in the paper, and that TPR alone is trivially satisfied by a model that always answers yes.

2) Existing experiment analysis. However, the issue you raise can already be examined partially in the original experiments.

  • (i) A tendency to say yes that is the same for all groups raises every cell equally, and that shift is absorbed by our permutation null and can only compress gaps, which makes the audit conservative.
  • (ii) CLIP is probed as a closed-set classifier and has no yes/no channel, yet the age–gender interaction (AGE→GEN or GEN→AGE) leads its t=2 MPD weak points, as it does for all three VQA models. For PaliGemma 2, GEN→AGE leads the t=2 MPD list on both the submitted and the revised list (Tables 10 and 11).
  • (iii) All mitigation experiments (Sec. 6) use CLIP and are unaffected by this concern.
AuthorsOpenReview ↗

W1. Yes-only probes. (2) (2/N)

W1. Yes-only probes.

The VQA probes only ever ask questions whose answer is "Yes.", making the problem oversimplified. A model that leans toward saying yes looks good under this setup, so it is hard to tell whether the gaps come from what the model "sees."

3) New experiment: negative probes. To test this directly, we asked each VQA model one additional question per image whose answer is clearly No, on the t=3 synthetic images (23,450 images after young adult regeneration). For age, we ask "Is the person elderly?" for Infant and Child images, "Is the person an infant?" for Teenager images, and "Is the person a child?" for young adult, Middle-aged, and Elderly images. For the other axes, we ask about the opposite gender, Fat↔Thin, and wheelchair↔without a disability. For race, each ground-truth label is paired with one other label drawn at random. For location, each ground-truth label is paired with one visually distant label, avoiding near pairs such as city/urban and neighborhood/suburban. We chose unambiguous foils deliberately.
The negative probes are still verification questions, so the probe still never asks the model to choose among demographic labels. For each model and axis, we report TPR, FPR, and AUROC over the yes-probability p̃. We also compute 2AFC accuracy: the fraction of images on which the true attribute receives a higher yes-probability than the foil.

AxisViLT TPRViLT FPRViLT AUROCPaliGemma 2 TPRPaliGemma 2 FPRPaliGemma 2 AUROCQwen2.5-VL TPRQwen2.5-VL FPRQwen2.5-VL AUROC
Age7250.958910.996900.99
Gender8480.9510021.009601.00
Location88120.9589150.958310.97
Race78720.5680200.857280.89
Appearance23210.527460.9474130.91
Disability48410.6598240.988830.98

TPR / FPR (%): rate of answering yes to the true attribute / to the foil. AUROC over the yes-probability p̃. Bold: yes/no bias (AUROC < 0.70). All probes use the wording in Table 5. 2AFC tracks AUROC (within 0.08) except ViLT disability (AUROC 0.65, 2AFC 0.78).

PaliGemma 2 and Qwen2.5-VL separate true from foil on every axis (AUROC 0.85–1.00). Qwen2.5-VL leans toward "no" (FPR 0–3% on age, gender, location, and disability), so its yes-only accuracies understate recognition. ViLT does not reliably separate the true attribute from a foil on race, appearance, and disability (AUROC 0.56, 0.52, 0.65, all below 0.70), so we no longer treat its answers on those axes as recognition and mark weak points that target them (e.g., GEN→DIS, AGE×GEN→APP) for yes/no bias. Items in which those axes only vary, with age, gender, or location as the target, stay as they are: the varying axis is the prompted attribute. We thank the reviewer; this check exposed a real limitation of the ViLT probe, which we will report.

Although the results above already support our claim, we note that the CIT audit does not depend on a particular probe. The covering array, the image set, and the chance-corrected metrics stay the same, and a balanced probe such as the one above can replace the yes-only probe without changing the rest of the pipeline.

The negative probes also show that the tendency to answer yes or no is not a single property of a model. It differs by axis within the same model. ViLT answers yes to race questions regardless of the image, Qwen2.5-VL leans toward no on age, and PaliGemma 2 has a higher FPR on disability (24%) than on age (1%). Which attributes a VLM affirms or denies without visual evidence, and whether this differs across demographic axes, is itself a fairness question that our framework can be used to study. We will note it as future work.

AuthorsOpenReview ↗

W2. Recognition disparity vs. discriminatory bias (3/N)

W2. Recognition disparity vs. discriminatory bias

The paper defines intersectional fairness bugs primarily through overall accuracy disparities. Why is this particular fairness notion appropriate for evaluating VLMs? In particular, recognition disparities may arise from inherent differences in visual ambiguity across different groups rather than the model learning discriminatory patterns. Would this lead to a biased result relying on a single metric?

Thank you for this question. We read it as three related concerns: 1) why accuracy disparity is an appropriate fairness notion for VLMs, and whether a single metric suffices; 2) whether the disparity can be separated from intrinsic differences in task difficulty, which we also acknowledge as a limitation (Sec. 7); and 3) whether the disparity matters if it stems from ambiguity rather than learned bias.

1) Why recognition-based metrics. Our audit asks whether a person who is both A and B is still recognized as A, so recognition itself is the object of study. This follows the explicit-bias definition of Luo et al. (2024) and the intersectional recognition gaps reported in Gender Shades (Buolamwini & Gebru, 2018). We also do not rely on a single overall accuracy. The axis-level audit uses confidence-based MPD and worst-case MinMax, both permutation chance-corrected (App. F, Eq. 7), and mitigation uses subgroup MPD and CVaR₀.₁. These capture confidence shifts, the worst-case floor, and tail subgroups respectively (probe validity is addressed in W1).

2) Ambiguity vs. model behavior. These metrics measure recognition disparity, not its cause (Sec. 7, Ethics Statement). Chance correction removes only part of the target-axis difficulty. The four models see the same images, so if the gap were only image ambiguity, axis accuracy would be similar. It is not: gender, under the same yes/no probe and chance of 0.5, is 0.84 for ViLT, 0.96 for Qwen2.5-VL, and 0.99 for PaliGemma 2 (Table 7, young adult). The W1 negative probe agrees (ViLT vs. PaliGemma 2: TPR 84 vs. 100, FPR 8 vs. 2, AUROC 0.95 vs. 1.00). Weak points overlap only in part (Table 11). Shared items, such as AGE→GEN, may be intrinsic ambiguity, for example infant gender. We plan a multi-annotator agreement study on the synthetic images.

3) Why the gap should be reduced regardless of cause. Whatever its origin, the gap should be reduced. When VLMs serve as auditors or backbones, a systematic recognition gap propagates to every downstream task, whether it comes from visual ambiguity or learned bias. The gap is also reducible in practice: fine-tuning raises worst-cell accuracy (Fig. 3d), and difficulty-weighted allocation targets exactly this reducible part.

Reviewer_ypGjOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The manuscript focuses on audit and mitigation of intersectional fairness in Vision-Language Models (VLMs).

The main challenge with intersectional fairness is that the number of protected/contextual attribute combination to be checked grows combinatorically much like hyper parameter grid search with many parameters. E.g. with 8 attributes and 5-6 level per attribute gives rise to 362K possible combinations.

The proposed solution is to reduce the search space using combinatorial testing which allows a near-equable coverage of the operational design domain while keeping the number of explored subgroups relatively small.

The adopted combinatorial testing method is combinatorial t-wise testing, individual attribute (e.g. age) correspond to covering arrays with t=1. For intersectional fairness, we are interested in t > 1. The manuscript explored t in {2,3,4}.

The final step will be pairing combinatorial t-wise testing with diffusion-based synthesis for auditing (checking whether the intersectional axis can be predicted accurately, checking the average (parity fairness) or worst case (minimax fairness) performance for target axis and the other axes) and mitigating (focusing fine-tuning on hard subgroups and using synthetic data to add more signals).

Critical strengths
  • Bias auditing and mitigation of intersectional fairness is an important problem
  • Experiments are performed on four VLMs with differing architecture, training paradigm and scale
Critical weaknesses
  • Combinatorial t-wise testing with diffusion-based synthesis has been proposed before in ICCV 2023 as mentioned by the manuscript. I understand the differentiation in terms of experiments (ICCV 2023 was on ImageNet) and focuses only on auditing/discovery. But I humbly think the current manuscript offers only minor contributions in addition to what we have learnt from ICCV 2023.
  • The main motivation is about auditing and mitigating bias in VLMs models, but I think the focus is only on the V(ision) but not the L(anguage). Perhaps to have additional contribution w.r.t. ICCV 2023, we could do analysis on the L aspect?
  • Considering the analogy with hyper parameter grid search, random search seems to work well for hyper parameter search (e.g. Bergstra and Bengio, JMLR 2012), would it be possible to understand the advantage of t-wise testing to random search?
  • Not a weakness, but rather a suggestion, I suppose: The statement on “synthetic-only training is weak on all three metrics”, the authors might want to explore literature on iterative re training, e.g. Yunzhen Feng, Elvis Dohmatob, Pu Yang, Francois Charton, and Julia Kempe. Beyond model collapse: Scaling up with synthesized data requires verification, ICLR 2025, Bingji Yi, Qiyuan Liu, Yuwei Cheng, and Haifeng Xu. Escaping model collapse via synthetic data verification: Near-term improvements and long-term convergence, arXiv 2026.
Questions for authors

I suppose the weakness points 2 and 3 related to language and random search would be good to hear authors’ perspective.

13 comments
AuthorsOpenReview ↗

W1: Comparison with Metzen et al. (ICCV 2023) (1/N)

We thank the reviewer ypGj for the accurate summary and for pointing to two concrete directions, the language side of VLMs and the comparison with random designs, that we agree sharpen the contribution beyond Metzen et al. (2023). We also appreciate the pointers to the literature on synthetic-data verification, which we plan to discuss in Section 7.

W1: Comparison with Metzen et al. (ICCV 2023)

Combinatorial tt-wise testing with diffusion-based synthesis has been proposed before in ICCV 2023, as mentioned by the manuscript. I understand the differentiation in terms of experiments (ICCV 2023 was on ImageNet) and that it focuses only on auditing/discovery. But I think the current manuscript offers only minor contributions in addition to what we have learnt from ICCV 2023.

We thank the reviewer for the careful reading and agree that the CIT + T2I backbone was introduced by Metzen et al. (2023). We will lower our novelty claim accordingly and rewrite Sec. 3.2 to state this explicitly. We would, however, like to clarify that the difference goes beyond the domain change (ImageNet vs. VLMs) and the addition of mitigation.

1) What is measured. Metzen et al. measure whether a class label is misclassified. Subgroup attributes enter only as the conditions under which that error occurs. We take the recognition of the identity attributes themselves as the object of the audit, and we measure how one identity axis shifts the recognition of another (S→AtargetS \rightarrow A_{target}). A classifier cannot express this comparison: its outputs are class labels, not identity attributes. We even extend it to higher-order interactions in VLM recognition, up to t=4t=4. That extension is where CIT becomes necessary. The number of axis combinations to compare grows with tt, and a covering array is what keeps every tt-way combination observable within a fixed budget.

2) Why the combinatorial explosion differs. When the search is over scene factors, the explosion is mainly a search-cost problem. In intersectional fairness it is also a data problem: every added axis shrinks the subgroup samples, and collecting labeled data on people is costly and sometimes raises consent and ethical concerns. This makes synthesis not just convenient but necessary here, as reflected in our mitigation scenario, where the rarest real photograph cell has only 42 training photographs.

3) Reusable probes. Connecting from 1), because each probe is a natural-language question, the same CIT image set can audit any attribute the model can be asked about, without retraining or relabeling. A fixed-label classifier audit cannot do this. The axes themselves were chosen for their relevance to intersectionality (Appendix A.1).

4) New evidence. In response to W3, we will add a comparison of CIT against random designs that select the same number of subgroup prompts, which directly tests whether CIT finds weak points that random selection misses. Tables R1 and R2 below are that comparison. Metzen et al. (2023) use combinatorial testing at a single strength (nC = 3) and do not compare it against random subgroup selection.

AuthorsOpenReview ↗

W2 / Q: The focus is on V rather than L. (2/N)

W2 / Q: The focus is on V rather than L.

The main motivation is about auditing and mitigating bias in VLM models, but I think the focus is only on the V(ision) and not the L(anguage). Perhaps, to have an additional contribution with respect to ICCV 2023, we could do analysis on the L aspect?

I suppose the weakness points 2 and 3 related to language and random search would be good to hear authors’ perspective.

Thank you for this suggestion. The language side is already what makes our audit possible, and that is actually how it sharpens the contribution beyond Metzen et al. (ICCV 2023).

Metzen et al. audit image classifiers, whose fixed label space leaves a single target: whether the object class is predicted correctly as scene factors vary. A VLM is instead queried about each attribute in natural language (a yes/no question for the VQA models, caption matching for CLIP). This lets a single model be probed on every identity axis, so any axis can serve both as the target and as the context that varies, which is the S→AtargetS \rightarrow A_{target} structure of Sec. 5.2. Asking whether an elderly woman is recognized as female as reliably as an adult woman, and whether her age is recognized as reliably as an elderly man's, requires exactly this.

However, we agree that the scope should be stated explicitly, and we will add it to our contributions: the scope of this work currently is visual identity recognition by VLMs. This scope still matters because fairness research itself increasingly relies on VLMs to recognize identity: as labelers (Zhang & Chunara, 2024; Shukla et al., 2025), as verifiers that filter synthetic images for debiasing (Howard et al., 2024), and as guidance for fair generation (Zhang et al., 2023). Uneven recognition across intersections therefore propagates into the conclusions and interventions built on these models.

We also still agree that a complementary question on the language side deserves direct study, and we do not regard this response as our final word on it. Our audit measures explicit bias as defined in Sec. 5: the failure to recognize an attribute that is explicitly present in the image. The yes/no probe, however, only asks the model to verify a label we supply. A complementary question is whether the model produces the right concept in its own words, for example when asked to describe the person with respect to each given axis. The CIT-guided images cover every intersection with the same prompt structure, so they could support this analysis without new data. We are currently exploring experimental design.

AuthorsOpenReview ↗

W3 / Q: CIT vs. Random (1) (3/N)

W3 / Q: CIT vs. Random

Considering the analogy with hyperparameter grid search, random search seems to work well for hyperparameter search (e.g. Bergstra and Bengio, JMLR 2012). Would it be possible to understand the advantage of tt-wise testing relative to random search?

I suppose the weakness points 2 and 3 related to language and random search would be good to hear authors’ perspective.

We thank the reviewer for this question, and answer it in three steps: 1) why the two problems have different objectives, 2) how much a random design of the same size misses, and 3) whether coverage translates into better audit estimates.

1) Different objectives. Choosing prompts is structurally similar to choosing hyperparameter configurations, and random search and t-wise testing share the same spirit: both avoid the waste of a full grid, because every row assigns an attribute to every axis and so covers many lower-order combinations at once.
However, the two problems differ in their objectives. Hyperparameter search is an optimization problem: it needs one good configuration, and random search works because only a few hyperparameters matter, so random draws quickly land in a good region. An audit is not an optimization problem, including the mirror-image problem of finding the single worst subgroup. It must characterize every subgroup. Our metrics aggregate over all of them (MPD over all pairs of attributes, CVaR0.1\mathrm{CVaR}_{0.1} over the worst 10% of cells), and the mitigation in Sec. 6 needs a score for every cell. Subgroups are also categorical, so a result on one subgroup carries no guarantee about another. An audit is therefore a coverage problem. Here, tt-wise testing offers what random search cannot: a deterministic guarantee that every interaction of up to tt attributes, i.e., every tt-way intersectional subgroup, is examined at least once.

This guarantee matters especially for fairness. A random design leaves some subgroups untested, and repeating the draw changes both how many are left out and which subgroups they are. One draw can cover more of the space than another draw of the same size, and the two draws can skip different subgroups. The design itself does not name the subgroups it skipped. An audit that omits subgroups in this way treats those subgroups unequally, and that inequality changes from draw to draw. With a covering array, the auditor can state and guarantee exactly what was tested ("every 2-, 3-, and 4-way subgroup, at least once"), and anyone can verify that statement. This gives the audit an accountability that a random design can provide only in expectation.

Comments on the paper

6 comments
AuthorsOpenReview ↗

General response (1/3)

We thank all three reviewers for their careful reading. Questions from one of the reviewers led us to re-run part of the generation and audit and all mitigation experiments, so several numbers in the revised PDF differ from the submission. We summarize the changes here, before the individual responses, so that they do not come as a surprise. None of our original conclusions is reversed. Two mitigation gains become smaller, and CLIP's synthetic–real correlation decreases (r/ρ 0.59/0.56 → 0.49/0.43) but remains substantial; the revised paper reports both.

How to read the revised paper. Changed text is in blue. Numbers appear as submitted / revised, with the revised value in blue. The ViLT yes/no bias revisions are in red: the second paragraph of Appendix E, the Fig. 9 caption, and the captions and struck-through items of Tables 10 and 11. Figures 1–3 and 9–11 show the submitted version ("Adult prompt") next to the revised one ("young adult"), and Appendix F reports the validation for both (Tables 8–13). For PaliGemma 2, the revised panels also include the scoring correction in §3. Unless stated otherwise, every revised number in the responses includes the young-adult regeneration.

AuthorsOpenReview ↗

General response (2/3)

1. "Adult" → "Young Adult" (Reviewer K5Fk, W3)

We agree with the reviewer; thank you for pointing this out. The 20–39 age attribute was labeled "Adult" in our prompts and probes, but the word "Adult" also covers middle-aged and elderly people, so "Is the person an adult?" does not single out that bin. To respond to this, we have regenerated every synthetic image prompted with "Adult" using "Young Adult". We re-ran every probe on these images (the age question is now "Is the person a young adult?"), and, because CLIP's age probe is a closed set over all six age captions, re-ran CLIP's age axis on all images with the new caption. The same label is used in the CLIP fine-tuning captions ("a photo of a young adult..."). The UTK-FF bin itself (ages 20–39) is unchanged. Tables 1, 3, 5, and 6, the Fig. 3 caption, Appendix B.2, Appendix C.3.1, Appendix D, Sec. 7, and Appendix G now record this wording. Example images in Appendices B and C will be updated.

1) Axis accuracy (Table 7) barely moves. Five of the 24 entries move by more than 1 pp: CLIP Age 0.468 → 0.487, ViLT Race 0.785 → 0.769, ViLT Appearance 0.220 → 0.234, PaliGemma 2 Appearance 0.675 → 0.686, and Qwen2.5-VL Disability 0.848 → 0.859. For PaliGemma 2, both the young-adult regeneration and the scoring correction in §3 are applied.

2) Weak points (MPD). The revised weak points largely overlap the submitted ones (PaliGemma 2 is in §3). Under MinMax, CLIP's overlap at t=2 is 2/3 (Fig. 1):

t=2t=3t=4
CLIP3/36/65/6
ViLT2/36/65/6
Qwen2.5-VL2/36/65/6

CLIP's top item is still AGE→GEN at t=2 and ETH×AGE→GEN at t=3, the interaction we mitigate. At t=4 under MPD, the main text now names GEN×LOC×DIS→AGE, up from fourth in the submitted list (Table 10). Under MinMax, GEN×APP×DIS→AGE is still first.

3) Synthetic–real agreement (Fig. 2, Pearson r / Spearman ρ).

ModelPearson r (submitted → revised)Spearman ρ (submitted → revised)
ViLT0.84 → 0.850.77 → 0.79
Qwen2.5-VL0.73 → 0.700.71 → 0.68
PaliGemma 20.68 → 0.680.67 → 0.65
CLIP0.59 → 0.490.56 → 0.43

The three VQA models move by at most 0.032 (Qwen2.5-VL ρ, 0.713 → 0.681); CLIP drops by 0.10 (r) and 0.135 (ρ, 0.561 → 0.426).

We attribute this to the property of CLIP's probe that Appendix D already gave as the likely reason for CLIP's lowest correlation in the submission: its age answer is a softmax over six ordered captions, so "young adult" competes with "teenager" and "middle-aged person" and errors spill into neighboring bins, which reorders the 60 cells (each cell score averages gender, age, and race). The VQA probes ask only about the ground-truth attribute. Since the three VQA models, which see the same regenerated images, barely move, the drop is more associated with CLIP's probe rather than with the images themselves. CLIP's correlation nonetheless remains substantial, with a 95% CI that excludes zero (ρ = 0.426 [0.200, 0.615]; per-source results in our response to Reviewer K5Fk, W3), so Sec. 5.3.3 still describes all four correlations as substantial. The mitigation in (4) below (Sec. 6) checks this directly: allocation guided by CLIP's synthetic scores still improves CVaR₀.₁ over Balanced at small K (+1.6 pp at K=8, up to +2.9 pp at K=2).

4) Mitigation CLIP ViT-B/32, CVaR$_{0.1}$ on the UTK-FF evaluation split, 20 seeds:

SubmittedRevised
Real-DW over Real-Balanced, small Klargest gap +4.0 pp at K=8 (64.1 vs. 60.1)+1.6 pp at K=8 (67.4 vs. 65.8); largest gap +2.9 pp at K=2 (62.2 vs. 59.3)
Mixed-Balanced 50% over Real-Balanced, K=128+2.2 pp (71.7 vs. 69.5)+1.5 pp (71.6 vs. 70.1)

Synthetic-only training remains the weakest (at K=128, 73.9% overall and 55.5% CVaR$_{0.1}$; submitted 76.2% and 55.9%). At K=128, the overall and macro accuracies of Natural and Real-Balanced move by at most 0.4 pp and keep their order. The abstract and introduction report the revised gains.

AuthorsOpenReview ↗

General response (3/3)

2. Near-duplicates between training and evaluation (Reviewer avGr, Q6)

Thank you for raising this; there were some. We remove a training image if it has the same MD5 as an evaluation image, or if it is the nearest training neighbor of an evaluation image and the correlation of the two images at 64×6464\times64 is at least 0.95. This removes 298 of the 77,868 training images (0.38%) and leaves 77,570. The evaluation split stays at 19,587.

Leaving the overlap in would change overall accuracy by 0.09–0.17 pp and move the methods relative to each other by at most 0.08 pp. The Natural vs. Real-Balanced gap is 0.9 pp in the submission (82.7 vs. 81.8) and 1.0 pp in the revision (82.8 vs. 81.8). Since we were re-running the experiments anyway, all revised mitigation numbers use the deduplicated pool of 77,570 (Sec. 6.2, Fig. 3). Appendix C.2 will report the new counts.

3. Correction: PaliGemma 2 audit (found while preparing our response to Reviewer K5Fk)

Unlike the ViLT and Qwen2.5-VL result files, the PaliGemma 2 file also contained answers for the appearance and disability attributes we excluded from scoring (Sec. 4.3), and our filtering step did not remove them for this model. PaliGemma 2's appearance and disability targets were therefore averaged over unscored attributes. On the scored attributes only, appearance accuracy is 0.675 / 0.686 and disability accuracy is 0.981 / 0.982 (Adult prompt / young adult), against 0.441 / 0.445 and 0.378 / 0.378 with the unscored labels included. Both lie above chance, so Appendix E now says near or below chance only for ViLT.

We re-ran t = 2, 3, and 4 on the young-adult images with scored attributes only. The revised top 10% overlaps the submitted lists as follows:

t=2t=3t=4
MPD3/35/65/6
MinMax1/35/66/6

The unscored labels were in the PaliGemma 2 file only, so this does not affect CLIP, ViLT, Qwen2.5-VL, the 60-cell synthetic–real comparison (gender, age, and race only), or any mitigation experiment.

4. ViLT yes/no bias (Reviewer K5Fk, W1 / Reviewer avGr, W7)

Reviewer K5Fk (W1) and Reviewer avGr (W7) noted that our probes only ask questions whose answer is yes, so a model that always says yes looks accurate. We checked this with a negative probe, a second question per image whose answer is no. That check, not the young-adult regeneration, is why some ViLT items are marked red. The lists themselves are unchanged.

ViLT does not reliably separate the true attribute from a foil on race, appearance, or disability, so weak points that target those axes stay in the lists but are not counted as recognition. They are struck through in Tables 10 and 11 and drawn in gray in Fig. 9; Appendix E names the items. PaliGemma 2 and Qwen2.5-VL discriminate on every axis, and no CLIP item is affected. Weak points in which these axes only vary, with age, gender, or location as the target, are unchanged. The probe design, TPR, FPR, and AUROC are in that response.

Three of the changes above alter numbers in the revised PDF. The age-label question (Reviewer K5Fk, W3) led to the young-adult regeneration, and the duplicate question (Reviewer avGr, Q6) led to the removal of near-duplicate training images. The PaliGemma 2 scoring error and the ViLT yes/no bias turned up while we prepared the reply to Reviewer K5Fk (W1) and Reviewer avGr (W7). The remaining points are/will be answered in each reviewer's thread.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.