acceptodds
Under review as a conference paper at ICLR 2027

Bias Before Pixels: A Surrogate Framework for Auditing Demographic Bias in Text-to-Image Models

Abstract

Text-to-image (T2I) models reproduce demographic biases from their training data, but auditing this bias traditionally requires generating and classifying large image batches for every evaluated prompt. We train a surrogate that predicts the demographic distribution of a model's generated images directly from the prompt, so new prompts can be audited without generating any images. The surrogate can reuse the target model's own text encoder, or use an external encoder that requires no access to the model's weights, making it also suitable for API-only systems. We construct a dataset of long and short prompts paired with measured gender and race distributions for three T2I models (FLUX.2-klein, SDXL Turbo, Z-Image Turbo), on which our surrogate improves gender bias-detection accuracy by 11–25% over the strongest text-based or diffusion-internal baseline, with consistent gains on race. Since each audit takes a single forward pass, we can compare bias across models and find concepts and words whose demographic skew differs sharply between them. This efficiency also lets us test all 9,396 object–activity combinations, which shows that activities shape gender bias more than objects. Finally, the surrogate improves an existing debiasing pipeline by predicting failed interventions before image generation, raising gender fairness from 0.62 to 0.74 without reducing image quality. Surrogates thus make it practical to audit bias at scale, before generating a single image and even without access to model weights.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Bias Discovery, Fairness, Text-to-Image Generation, surrogate models, demographic bias

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_EYutOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an Attribute Distribution Surrogate (ADS) that predicts the demographic distribution of images generated by a text-to-image model directly from its input prompt. The method uses either the model's native text encoder or an external encoder and is evaluated on three T2I models. The authors further apply the surrogate to cross-model bias analysis and improve an existing debiasing pipeline.

Critical strengths
  1. A well-motivated and practical problem. Generating and labeling images for every audit prompt is costly. A prompt-only surrogate offers a promising way to scale bias auditing, potentially including API-only models.
  2. Promising results and practical utility. The method shows competitive results across three T2I models for both gender and race (Tables 1–2). It also improves an existing debiasing pipeline without noticeably compromising image quality (Table 3).
Critical weaknesses
  1. I'm not convinced the gains in Table 1 come from the proposed method itself. The Text Embedding Probe and Diffusion Activation Probe use pronoun-derived labels, while ADS is trained on distributions measured from generated images. A comparison using the same training data would help clarify how much of the improvement comes from the model rather than the supervision.
  2. The conclusions from the large-scale analysis seem stronger than the evidence. The 9,396 object–activity combinations are evaluated using surrogate predictions, with only 90 checked against actual generations. The TIFA analysis also lacks generation-based validation. This makes the broader findings difficult to fully trust.
  3. I'm also concerned about the quality of the training labels. The descriptive training set uses only one generated image per prompt, and uncertain demographic labels are discarded. How much does this affect the estimated distributions, especially for the six-category race task?
Questions for authors

1.How would the Diffusion Activation Probe perform if trained on the same prompt–distribution pairs as ADS? Would the performance gap remain? 2.Can the authors provide more generation-based validation for the object–activity and TIFA analyses, beyond the 90-prompt check already reported?

1 comment
AuthorsOpenReview ↗

Awaiting Reviewer Response

Dear Reviewer,

We would like to thank you for your efforts and feedback. We believe our revision and responses address all concerns mentioned in the review. We would greatly appreciate getting feedback on whether there are any remaining concerns from your end.

Best, the Authors.

Reviewer_b8LvOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary
  • The paper introduces Attribute Distribution Surrogate (ADS), a surrogate model that is trained to predict demographic distribution of a Text-to-Image (T2I) model's generated images from textual prompts.
  • The authors propose a new prompting dataset with mixed length and measured gender / race distributions of several state-of-the-art image generation models.
  • The paper established a cross-model comparison on demographic associations with concepts and words, as well as auditing a TIFA benchmark.
  • The authors show that the proposed ADS method can replace the costly image generation part in existing debiasing pipelines to predict generated demographic distribution directly from prompts.
Critical strengths
  • The proposed ADS approach is meaningful to improve efficiency debiasing in T2I generation, by avoiding costly image generation of multiple images for each of the evaluation prompts.
    • Experiment results also show that both ADS-Native (using model text encoder) and ADS-External outperforms previously proposed methods like diffusion activation prob, text embedding probe, etc.. on bias detection accuracy, distribution variance distance, and spearman / entropy correlation.
  • What's more, the proposed ADS method demonstrates downstream value to enhance debiasing pipelines beyond bias prediction accuracy.
    • For instance, incorporating it into the FairImagen framework improves the reported gender fairness score from 0.623 to 0.745 without hurting image quality and text-to-image similarity scores.
Critical weaknesses
  • Experiments on how ADS improves downstream debiasing pipelines is limited to FairImagen on FLUX.2-klein model. It therefore remains unclear whether the proposed ADS method can reliably improve other inference-time debiasing pipelines as well.
  • As the authors also discuss in Appendix A, the proposed surrogate’s reliability with active generator updates (e.g. during SFT or post-training) could also invalidate the original learned predictions. Therefore, I feel like ADS is less effective in tracking debiasing progress during training, which would have been a meaningful contribution.
Questions for authors
  • How well does ADS perform on long, descriptive, and complex prompts? (e.g. those specifying details like background, lighting, style, etc.. that aligns better with real world use cases). I read that you included long prompts in dataset construction, but does ADS fail more often with longer prompts with more constraints? How can we resolve this?
3 comments
AuthorsOpenReview ↗

Awaiting Reviewer Response

Dear Reviewer,

We would like to thank you for your efforts and feedback. We believe our revision and responses address all concerns mentioned in the review. We would greatly appreciate getting feedback on whether there are any remaining concerns from your end.

Best, the Authors.

Thank you for the clarification and further results. Since I already gave a positive rating, I will maintain my score. Good luck!

Thank you for the follow-up and for your continued support of our work. We appreciate your time and feedback throughout the review process.

Best, the Authors.

Reviewer_UxaEOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper trains a small model that guesses, from the prompt alone, how often a text to image model would draw men versus women or people of different races. This lets bias be measured without generating any images. The authors test it on image models, show it works better than existing probes, and use its speed to compare models.

Critical strengths
  • The idea is simple and useful. Auditing any new prompt takes a single forward pass instead of generating and classifying images, which makes large scale and cross model analyses practical.
  • The baselines make a clear point. Probes trained on explicit pronouns do not work on neutral prompts, and reading diffusion internals does not help either, so the curated dataset is what drives the gains.
Critical weaknesses
  • The Descriptive test set includes prompts that still say he or she. Those are easy to predict from text alone, so the reported accuracy may be inflated. Results on neutral prompts should be shown separately.
  • The ground truth itself is noisy, since training labels come from one image per prompt and labelers disagree on some images, but the paper never measures this noise. So it is unclear how much of the surrogate's error is actually the surrogate's fault.
Questions for authors
  • Some Descriptive test prompts still contain explicit pronouns or strongly gendered words such as dress. These are easy to predict from text alone. Can the authors report results on fully neutral prompts separately, so that the gain over the baselines reflects the surrogate's ability to predict bias rather than prompt adherence?
  • The external encoder variant is presented as black box, but it is only evaluated on open models. Have you tested it on a closed API model, and if not, what would you expect to change when the training labels must come from an API with its own sampling and safety filters?
1 comment
AuthorsOpenReview ↗

Author Response (1/2)

Pronoun Cues (W1, Q1): We explicitly re-evaluated different splits of our evaluation set for a more detailed analysis (Table 1). Explicit pronouns (e.g., "she") strongly constrain output distributions, aiding surrogate prediction; however, performance remains below 100%100\%, likely due to semantic ambiguity in prompts containing multiple contrasting pronouns. Without explicit pronouns, the surrogate infers gender from appearance cues or subtle activity/setting context containing generator bias. We show that ADS-External maintains top performance across all metrics on fully neutral prompts, proving gains are not merely driven by explicit pronouns (which comprise only 8.6%8.6\% of the test set) or gendered nouns and appearance cues such as 'dress' (excluded in the No cues split).

Table 1: FLUX.2-klein Descriptive test prompts split into those with a gendered pronoun (Pron.), other gender cues without a pronoun (Cues), e.g. ''woman'', ''beard'', ''dress'', and neither (Neutral)

MethodAcc. (Pron.)Acc. (Cues)Acc. (Neutral)Acc. (All Prompts)TV (Pron.)TV (Cues)TV (Neutral)TV (All Prompts)Spearman (Pron.)Spearman (Cues)Spearman (Neutral)Spearman (All Prompts)
Diff. Activation Probe0.87790.72920.63960.72210.10160.18020.22440.18320.86120.76470.64950.7597
RepE Reading Vec.0.43020.35480.37840.36650.33410.37450.34330.36410.53050.37500.27470.3608
InterpretFlux0.39530.34610.39410.36100.42110.45480.41750.44360.17280.0112-0.03250.0194
Text Embedding Probe0.79650.54530.36710.52730.19070.27150.30390.27180.83980.73330.54700.7191
ADS-Native0.88950.81610.68240.79270.07840.11860.19030.13100.92710.88490.73520.8659
ADS-External (T5-XXL)0.91280.85730.73420.83480.05730.08610.14960.09770.94160.92050.82630.9073

Image Budget/Prompt Diversity (W2): To evaluate whether single-image label noise limits surrogate performance, we conducted an ablation experiment using ADS-External (T5-Large) evaluated on the Descriptive test set (Table 2). We compared two regimes: (1) expanding the number of sampled images per prompt while keeping all prompts, and (2) sampling more images per prompt under a fixed total compute budget (which reduces unique prompt coverage by 5x and 10x). As seen in Table 2, accuracy remains flat (0.829 to 0.831) when adding more images per prompt. Because a single generated image is an unbiased sample from the T2I distribution, soft-label loss naturally averages out noise across 10k training prompts. Crucially, under a fixed compute budget, reducing prompt diversity to collect multi-image labels degrades accuracy from 0.829 to 0.768, proving that broad prompt coverage matters far more than per-prompt label precision. We have added both of the above experiments to section 5.1 of the main paper.

Table 2: ADS-External (T5-Large) on FLUX.2-klein trained on different versions of the ground truth, Descriptive set. Best per column in bold.

Training dataAcc.TVSpearman
All prompts, images per prompt
10.82920.10180.9021
50.81970.10310.9007
100.83120.10270.9058
Fixed image budget, fewer prompts
1 image (all prompts)0.82920.10180.9021
5 images (5x fewer prompts)0.80420.13030.8624
10 images (10x fewer prompts)0.76820.14740.8388

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.