- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2(+1)
Read review
Paper summary
This paper tackles the problem of intersectional bias in Vision-Language Models (VLMs). Intersectionality refers to the fact that a person can be defined by many axes at the same time (e.g., gender, age, disability, etc…) but often fairness is assessed on a single axis. One complication of intersection audit is the fact that the number of possible combinations grows combinatorially so it becomes often an intractable problem (and it can leave very few real labeled samples for every subgroup). The authors propose instead to use a techniques borrowed from software testing known as Combinatorial Interaction Testing (CIT). Given a set of axes and their potential values (called here attributes), a t-way CIT test set covers every combination of attributes of any t axes at least once (this is useful when t << number of axes).
The authors start with a list of 8 axes and 2, 3, and 4-way sets (reducing the total number of possible combinations from 362K to 2847 for the 4-way). Each row of the set becomes a prompt they use to generate 50 images using SDXL. After manual inspection of 690 images from the 2-way set two axes (faith and socioeconomic status) are dropped from scoring, although they stay in the prompts and images in order to keep the coverage complete. Some attributes from the “disability” and “appearance” axes are also dropped (only "wheelchair / no disability" and "fat / thin" are kept) because the SDXL model does not generate those axes and attributes reliably. The prompt's attributes are then used as ground truth.
The audit involves 4 different VLMs. For each image and each scored axis, the authors check whether the model recognizes the attribute named in the prompt. Three metrics are extracted: Attribute Recognition Accuracy for every axis, Mean Pairwise Delta (MDP), which measures how much the model confidence in an attribute moves when one other axis changes while the rest remains fixed (e.g., ability to recognize gender when age changes), and MinMax Fairness, which measure the worst-case accuracy for a given target axis across the values of the varying axis. The result of this audit is that every model has weak points at every t, and they mostly involve age and gender. The per-subgroup accuracy on synthetic images correlates with accuracy on real photos (UTKFace + FairFace, "UTK-FF") for all four models.
Then the authors show how the per-subgroup accuracy from the audit can be used for mitigation. Specifically, they fine-tune CLIP ViT-B/32 with a limited budget of N = 60 × K training images (K per subgroup on average). Each synthetic subgroup (called cell in the paper) is given a weight that is 1 - its synthetic-image accuracy, so more images are given to more difficult subgroups (“difficulty-weighted” or DW). Training on pure synthetic images gives worse accuracy and fairness metrics compared to using real images. Using real images (from UTK-FF) with the authors' DW improves faster than Real-Balanced (the same number K of real images for every cell) when the budget is small (e.g., K=8), but the advantage disappears as the budget grows. As the budget increases, a mixed approach helps: it keeps N fixed but replaces 25% or 50% of the real images with synthetic ones. Synthetic images fill these cells with new samples instead. At K = 128, Mixed-Balanced 50% raises one of the fairness metrics ( CVaR 0.1: the average accuracy of the worst 10% of subgroups) by 2.2 points over Real-Balanced (71.7% vs 69.5%).
Critical strengths
- The paper tackles an important and difficult problem.
- The method used could be of practical use.
Critical weaknesses
Soundness
- One of the main claim is the ability to find “fairness bugs” in VLMs but one of the main problem is that it is not possible to isolate if “fairness bugs” found are VLM bugs or the image generator bugs.
- The manual check was only done for t=2 and no info about this check is provided to understand if this was a reliable audit. E.g., what does “they are not realized consistently enough “ mean?
- Additionally it is hard to distinguish if what is found is a fairness bug or something objectively more difficult to judge (e.g., age of infants)
- The CIT is only partially used for mitigation. The papers shows mitigation only on 3 axes, for a total of 60 cells, but if I understand correctly that amounts to the full cartesian product so one could have done this audit and mitigation without CIT.
- The mitigation gains could be an artifact of under-training. For K=8 the model is trained only with six steps (for K<=4 only 3 steps). In this regime, any method that unweights hard cells may do well simply because training stops before anything else can catch up. It would be better to think of ways to separate the training length from the data size? Maybe at small K train with more epochs? Or train each run to converge on a validation split?
- One of the real problem of intersectionality is the sacristy of data and this method does not seem to be able to tackle this, not even in the synthetic audit. The CIT method only guarantees at least one prompt per combination. This means its 50 images are probably near copies of each other so it is not surprising that the actual results show a marginal improvement.
- The ACC metrics for the 3 VQA models ignores false positives. So always answering yes would score 100% accuracy. Consider using per-class precision and recall or something that accounts also for false positives.
Some missing experiments
- Simple random sampling baselines for the audit (and for the mitigation) are missing. CIT guarantees coverage, but whether it finds more or different bugs than random designs of the same size is not tested.
- The DW strategy is lacking a comparison: random weights, or weights based on real-data difficulty measured on a small labeled set.
- How much the fine-tuning affects the models is only tested on the same task but not on generic capabilities.
- Also worth re-running the complete audit after the mitigation fine-tuning. Has the fine-tuning reduced the weak points previously identified?
- Results do not seem unforeseeable from the single axes test. Age was already the hardest axis for CLIP by itself and most of CLIP's weak points involve age, either as the target or as the axis that shifts recognition of another one. Perhaps a convincing argument could be to show that by simply adding randomly sampled images across different values for those axis identified individually is not as good as using the more complex mix strategy?
- The authors say that the weak points replicate on held-out prompts, but those are only tested within the same image generator.
Results are weak
- CLIP correlation with UTK-FF is somewhat low r = 0.59, \rho = 0.56.
- The gains are small: the DW gain disappears for larger K, and the Mixed gain is quantified only at K = 128.
Novelty is limited
- As explained by the authors, CIT with Text-to-Image was already introduce in 2023. Additionally CIT for choosing training data was also already introduced in Cody et al., 2022 (“Systematic Training and Testing for Machine Learning Using Combinatorial Interaction Testing”).
- The novelty relies on these applications of CIT to a slightly different tasks. The combination of limited novelty, lack of some baseline to see if CIT actually brings value and the weak results makes this work difficult to accept as is.
Ethical use
- The use of axis like “race” to be detected in images is a questionable choice. Race or ethnicity have complex definitions and related to attributes that are not always identifiable in images. I find this especially critical for a fairness paper. I would recommend to use more objective axes instead, for example the skin color using the Monk Skin Tone scale.
- The authors use UTKFace dataset which is a web-scraped dataset of faces collected without consent. In general, but especially in a fairness paper the use of these kind of datasets is questionable. I know that in practice many work use this dataset and that it is hard to find valid alternatives so I would not consider this a reason for rejection but it is also worth thinking about this.
Minor but worth thinking about it.
- I am not sure about the use of the term “bug” for fairness topics. Regardless of the fact that might or not have been used in other works using this terms seems to imply an unintended unaware error that can be quickly fix. As also this paper show this is not really the case: these issues are foreseeable, not due to a “mistake” and definitely not easy to fix.
- One of the metrics used is confidence but I do not see any evidence or tests that these models are calibrated. If they are not calibrated is this metric reliable?
Questions for authors
- Could you show the value of CIT for mitigation in a case where you are not using the full cartesian product?
- Is there a way to disentangle bugs due to the generator from those due to the VLM?
- Can you show that the gains at small K is not artifact of the training set up? See my suggestions above.
- Can you report a better metric than ACC, one that would not be 100% for a model that always answers YES?
- Can you add the baselines mentioned above?
- Is the test split for the mitigation experiment completely separated from the training split? Have you checked that some near duplicate did not end up in both splits?
- The authors measured correlation but maybe it’s more informative to measure whether the worst cells agree, for example how many of the 6 worst synthetic cells are among the 6 worst real ones.
33 comments
[Main claim] Q1 / W4. Contribution of CIT (2/31)
Q1: Could you show the value of CIT for mitigation in a case where you are not using the full cartesian product?
W4: The CIT is only partially used for mitigation. The papers shows mitigation only on 3 axes, for a total of 60 cells, but if I understand correctly that amounts to the full cartesian product so one could have done this audit and mitigation without CIT.
You are right that the 60 cells are the full product of three axes, and this is by design (Sec. 5). Harm falls on people, not on axes: ETH×AGE→GEN tells us where a disparity concentrates, but not who bears it. As Sec. 5 states, mitigation "aims to close the gap among subgroups," so every subgroup must be an enumerable cell we can train and evaluate on; skipping cells would recreate the blind spots intersectional analysis aims to remove. At this level, CIT does not reduce the space, and we do not claim it does.
What CIT provides is upstream, and it is precisely what makes the pipeline usable beyond this one scenario.
(1) One generation run serves every target intersection. The t=4 covering array contains every 4-way, and therefore every 3-way, attribute combination over the eight axes. Whichever three axes a practitioner chooses to intervene on, all of that scenario's cells are already present in the pool. Covering all 56 three-axis scenarios with purpose-built sets would require 8,237 cell-specific prompts; the t=4 array covers all of them, as well as every 4-way cell, with 2,847 prompts. The gender×age×raceð scenario was not built into the array; it is one of the 56 triples the array covers by construction.
(2) Reused audit images are as effective as purpose-built ones. The natural concern is that reuse costs quality: CIT images also vary the other five axes, which might dilute the target cells. To test this directly, we generated a purpose-built set for this scenario: one prompt per cell specifying only race, gender, and age, with 128 images per prompt (7,680 images). We swapped it in as the synthetic half of Mixed 50%, keeping everything else fixed (seeds 0–9).
| Allocation | K | ΔCVaR₀.₁ (pp) | ΔMPD |
|---|---|---|---|
| Mixed-Balanced 50% | 8 | −0.76 [−1.72, 0.20] | +0.0081 [0.0037, 0.0125] → audit |
| Mixed-Balanced 50% | 32 | +1.18 [−0.03, 2.39] | −0.0010 [−0.0041, 0.0021] |
| Mixed-Balanced 50% | 128 | +0.12 [−0.53, 0.77] | −0.0016 [−0.0040, 0.0008] |
| Mixed-DW 50% | 8 | −0.61 [−1.93, 0.71] | +0.0029 [−0.0026, 0.0084] |
| Mixed-DW 50% | 32 | +0.85 [0.25, 1.45] → purpose-built | −0.0027 [−0.0046, −0.0008] → purpose-built |
| Mixed-DW 50% | 128 | +0.56 [−0.11, 1.23] | −0.0030 [−0.0053, −0.0007] → purpose-built |
Δ = purpose-built 3-axis − reused t=4 audit images, paired by seed; brackets are 95% CIs over seeds 0–9. Higher CVaR₀.₁ and lower MPD are better, so a positive ΔCVaR₀.₁ or a negative ΔMPD favors the purpose-built set. Bold: the interval excludes zero, followed by the source it favors.
In 8 of 12 comparisons the interval includes zero. The four that exclude zero point in both directions (three favor the purpose-built set, one favors the audit images), and all point estimates are within 1.2 pp CVaR₀.₁ and 0.009 MPD. Those CVaR₀.₁ differences remain smaller than the effect of mixing itself (+1.5 pp over Real-Balanced at K=128; 71.6 vs. 70.1), but the largest point estimate, 1.18 pp, is now close to that gain. Purpose-built images therefore give no consistent advantage, even though they were tailored to this exact scenario and the audit images were not.
(3) The audit chooses the target. Among 362,880 combinations, ETH×AGE→GEN was CLIP's top t=3 weak point by MPD (Fig. 1, Tables 10 and 11), which is why we chose this scenario. The top t=4 weak point under young-adult MPD is GEN×LOC×DIS→AGE (Sec. 5.3, Table 11). It could not be used only because no photo dataset labels all four attributes at a usable scale. Under MinMax, GEN×APP×DIS→AGE is still first, and the same data constraint applies. Given such data, the same images apply without regeneration.
Together, a single CIT generation run both identifies where to intervene and supplies mitigation data for any intersection it covers, with no consistent loss relative to generating a dedicated set per scenario. One caveat we will add: the covering array guarantees every cell at least one prompt (50 images), so for high-cardinality triples (e.g., location × disability × faith, 336 cells) the per-cell pool is smaller than in the scenario studied here.
[Main claim] Q2 / W1, W2, W3, W13. Generator Failure vs. VLM Failure (1) (3/31)
Q2: Is there a way to disentangle bugs due to the generator from those due to the VLM?
W1: One of the main claim is the ability to find “fairness bugs” in VLMs but one of the main problem is that it is not possible to isolate if “fairness bugs” found are VLM bugs or the image generator bugs.
W2: The manual check was only done for t=2 and no info about this check is provided to understand if this was a reliable audit. E.g., what does “they are not realized consistently enough” mean?
W3: Additionally it is hard to distinguish if what is found is a fairness bug or something objectively more difficult to judge (e.g., age of infants)
W13: The authors say that the weak points replicate on held-out prompts, but those are only tested within the same image generator.
Q2, W1, W13 — Generator vs. VLM
Q2 and W1 ask whether a weak point found on synthetic images can be attributed to the VLM rather than to the image generator, and W13 notes that the replication was tested only within the same generator. Our answer is yes for the weak points whose axes have real labels, and we ran a stricter version of the check W13 asks for: rather than re-rendering with a second generator, we tested these weak points with no generator at all. AGE→GEN and ETH×AGE→GEN involve only gender, age, and race, which UTK-FF labels. We computed the same metrics on all 97,455 UTK-FF photographs, using the zero-shot predictions behind Fig. 2.
| Model | AGE→GEN MPD ↑worse (syn → real) | AGE→GEN MinMax ↓worse (syn → real) | ETH×AGE→GEN MPD ↑worse (syn → real) | ETH×AGE→GEN MinMax ↓worse (syn → real) |
|---|---|---|---|---|
| CLIP | 0.092 → 0.106 | 0.886 → 0.704 | 0.054 → 0.068 | 0.941 → 0.785 |
| ViLT | 0.205 → 0.212 | 0.278 → 0.318 | 0.109 → 0.119 | 0.625 → 0.514 |
| Qwen2.5-VL | 0.104 → 0.156 | 0.731 → 0.590 | 0.054 → 0.094 | 0.873 → 0.726 |
| PaliGemma 2 | 0.024 → 0.080 | 0.973 → 0.797 | 0.014 → 0.054 | 0.983 → 0.839 |
Synthetic values come from the array where each item was found (t=2, 69 prompts, for AGE→GEN; t=3, 469 prompts, for ETH×AGE→GEN). All values are raw. Chance correction in the paper ranks items with different cell sizes using a prompt-block permutation, which does not apply to photographs; here each row compares one item with itself. Raw values, if anything, favor the synthetic side: its groups hold about 12–16 prompts against at least 208 photographs on UTK-FF, so noise inflates synthetic MPD and lowers synthetic MinMax more.
On photographs, MPD is at least as large for every model on both items, and MinMax is lower in seven of the eight cases (the exception is ViLT on AGE→GEN). The per-cell pattern also agrees. Infants have the lowest gender accuracy for all four models on both sides, and Spearman ρ over the 30 race×age cells is 0.75–0.92. These disparities exist without any generator, so the generator is not needed to explain them. We will add this table to the revision.
For weak points involving appearance, disability, or location, no labeled photographs exist at a usable scale. For those, synthetic images alone cannot fully separate the two sources, as stated in Sec. 7, and synthesis remains the only practical route to an audit. The pipeline does not depend on SDXL: the generator is a replaceable component, so these weak points can be re-rendered with a different generator and checked in the same way.
Thank you for the detailed review (1/31)
We thank Reviewer avGr for the detailed review. Several of the suggested baselines and controls were concrete and actionable, and we have now run them ([W1, W4 - W9, W12 - W15, M2]); the results and analysis are reported below. We have also adopted the reviewer's point on terminology and plan to replace "fairness bug" with "fairness failure" throughout the paper (M1). On one point, we respectfully hold a different view: the use of perceived race as an audit axis (E1). We state our position and explain below why we believe it is the appropriate choice for this audit. We welcome further discussion on any of these points, including the point for which we differ. We respond to every point below. Those that bear directly on our main claims are marked [Main claim].