acceptodds
Withdrawn from ICLR 2027

Rethinking Fairness in Multimodal Large Language Models for Medical Imaging

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multimodal large language models (MLLMs) now answer clinical questions and draft radiology reports at expert-competitive quality, yet early audits show that their outputs differ in quality across demographic groups. Fairness in medical imaging has been studied extensively, but almost entirely for models that predict a label from an image alone, and the few studies of medical MLLMs are purely empirical and narrow in scope. As a result, there is no theoretical account of how demographic bias arises in an MLLM, which is conditioned on both an image and a textual query and produces open-ended text, and no systematic benchmark to measure it. We address both gaps with MedFairVLM. We first define a fairness criterion for open-ended generation as the disparity in LLM-as-a-judge scores between demographic groups. Building on it, we derive an upper bound on the fairness gap of an MLLM that decomposes it into five sources of bias: disparities in the image distribution, in the query distribution, in the two conditional dependencies between image and query, and in text generation given both inputs. We extend the bound to distribution shift and provide an estimator that quantifies each source from data. We then benchmark ten open-source and commercial MLLMs on eight datasets spanning four imaging modalities, four organs, and three sensitive attributes, across closed-ended visual question answering (VQA), open-ended VQA, and report generation, under zero-shot, fine-tuning, and distribution-shift scenarios. Bias is prevalent across all models and settings; under fine-tuning, fairness behavior is highly correlated with the bias sources identified by our theory; existing fairness methods fail to mitigate bias consistently; and distribution shift makes fairness harder still to control.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
fairness, bias, multimodal, LLM, multimodal LLM, medical imaging, VQA

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_3WP1OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents MedFairVLM, a benchmark for evaluating fairness in medical multimodal LLMs. It evaluates 10 models on 8 datasets across different demographic groups, tasks, and settings. The paper also proposes a theoretical upper bound that decomposes fairness disparity into five sources of bias, and analyzes their correlations with model fairness.

Critical strengths

The experiments are relatively comprehensive, covering zero-shot, fine-tuning, and distribution shift.

Critical weaknesses

1: Limited novelty and scientific findings; Most conclusions are rather expected. Different demographic groups have different data distributions and model performance, and distribution shift may affect these groups differently. Similar issues have already been studied in medical AI fairness. The paper does not provide sufficiently new insights beyond these observations.

2: The proposed five bias sources do not really explain model unfairness; They are derived from data distributions and are model-independent. Therefore, the same five sources are shared by different MLLMs on the same dataset, even though their fairness behavior can be very different. More importantly, correlations between distribution differences and fairness gaps cannot establish which source actually causes unfairness. The paper does not provide controlled experiments to verify these causal relationships.

  1. The theoretical contribution is not well connected to the experiments: The upper bound is derived for FEMI, rather than the actual F1 or LLM-judge fairness gaps. In practice, the proposed estimators use MMD instead of the JS distances in the theorem, so the bound is not directly verified. Moreover, FEMI shows relatively weak correlations with the actual fairness gaps. It is unclear what additional understanding this theory provides.

4 The fairness evaluation may be confounded by clinical differences: Different demographic groups naturally have different disease prevalence and clinical characteristics. Even a model with identical sensitivity and false-positive rate across groups can have different F1 scores. The paper does not sufficiently distinguish such data heterogeneity from algorithmic unfairness. Therefore, some reported fairness gaps and bias sources may simply reflect differences in disease distributions.

Questions for authors

See the weakness.

Reviewer_3PkWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces MedFairVLM, a framework and benchmark for demographic fairness in medical multimodal LLMs. It measures group disparities in task utility, introduces an effective-mutual-information fairness proxy, and derives an upper bound involving five data-distribution discrepancies. Kernel-based estimators are used to examine these discrepancies empirically. The benchmark evaluates ten models on eight datasets across closed-ended QA, open-ended QA, report generation, fine-tuning, and distribution shift. Results document widespread performance disparities and inconsistent benefits from two fairness interventions, while identifying correlations between data-level discrepancies and observed utility gaps.

Critical strengths

Broad and practically relevant experimental coverage. The benchmark covers eight datasets, four imaging modalities, ten models, and multiple tasks and deployment settings. This provides substantially broader evidence than a single-dataset or zero-shot-only audit.

An explicit attempt to connect fairness measurement with distributional analysis. The framework distinguishes image, query, cross-modal, and reference-response discrepancies, providing a useful vocabulary for investigating group disparities. The paper supplies complete proofs and acknowledges that its practical estimators do not constitute certified finite-sample bounds.

Detailed and informative empirical reporting. The appendices include dataset construction, training settings, evaluation prompts, confidence intervals, and extensive subgroup results. The comparison of zero-shot, fine-tuned, and shifted settings highlights that better utility does not automatically imply smaller demographic disparities.

Critical weaknesses

The theoretical proxy is insufficiently connected to the fairness outcomes being evaluated. The theorem bounds FEMI rather than the actual F1 or judge-score disparity. Table 2 reports only modest correlations between FEMI and observed utility gaps: 0.29, 0.40, and 0.17 across the three tasks. Furthermore, the empirical discrepancies are kernel-based substitutes for the theorem’s JS distances. Although these limitations are acknowledged, they substantially weaken the claim that the theory explains the measured unfairness.

The claimed bias attribution is stronger than the evidence supports. An upper bound involving several dependent discrepancies does not identify their individual contributions or establish that intervening on one will reduce the observed gap. Source (v) concerns reference-response conditionals, rather than directly identifying decoder bias. The evidence is primarily correlational and lacks controlled source-specific interventions or recovery tests with known sources of disparity.

Unadjusted utility gaps can conflate demographic performance differences with case-mix differences. F1 depends on prevalence, while judge scores can depend on disease complexity and reference-report characteristics. The paper does not sufficiently establish which gaps persist after accounting for these factors. Such adjustments are important for distinguishing descriptive group inequality from model-specific, potentially remediable error disparities.

The automated evaluation requires stronger clinical and subgroup validation. GPT-5-mini is used for much of the QA construction and for open-ended evaluation, without quantified expert agreement across demographic groups and modalities. The adapted GREEN prompt also instructs the judge both to count false or missed findings and to evaluate only findings appearing in both reports. This ambiguity could affect the central report-generation results.

The statistical analysis does not adequately address dependence among observations. Appendix C.1 treats image–question rows as independent despite multiple questions or images potentially originating from the same image or patient. The pooled correlation analysis also repeats model-independent source estimates across models. Cluster-aware uncertainty estimates and hierarchical analyses are needed to establish the robustness of the reported significance and associations.

Questions for authors

How reliable is FEMI as a proxy for the fairness criterion used in the benchmark? Given the correlations of 0.29, 0.40, and 0.17 in Table 2, could you characterize when FEMI agrees or disagrees with actual utility gaps? Which assumptions connecting EMI to judge scores are empirically satisfied in these medical tasks?

Can the five-source framework recover known sources of disparity under controlled interventions? Could you construct settings that separately alter image distributions, query selection, cross-modal pairing, or reference responses, and test whether the estimators correctly identify the manipulated component? How do conditional estimates behave when groups have limited input-support overlap?

Which disparities remain after controlling for clinical case mix? Please report disease- and severity-stratified or matched analyses, together with sensitivity and specificity for closed-ended QA. Can report-generation gaps be separated from differences in report length, finding count, and clinical complexity?

How have the judges been validated across groups and modalities? Could you provide expert agreement and independent-judge sensitivity analyses? Please also clarify or correct the GREEN instruction restricting evaluation to findings present in both reports, since this appears incompatible with counting false and missed findings.

Do the statistical conclusions hold under cluster-aware analysis? Could you bootstrap at the patient or image level, account for multiple testing, and reassess correlations with clustering by dataset–attribute setting? Please also distinguish a significant mean signed gap across models from agreement among models on the favored group.

How sensitive are the conclusions about mitigation to hyperparameter and method selection? Could you evaluate multiple regularization strengths and reweighting settings, report fairness–utility frontiers, and include an additional applicable mitigation approach? This would clarify whether failures reflect the methods themselves or the selected operating points.

Reviewer_aGWTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents MedFairVLM, a framework for evaluating fairness in medical MLLMs. It introduces an EMI-based fairness bound decomposed into five distributional disparities and benchmarks 10 models across 8 datasets, 3 tasks, and multiple evaluation settings. The experiments reveal widespread demographic performance gaps and inconsistent effectiveness of existing mitigation methods.

Critical strengths
  1. Comprehensive evaluation. The benchmark covers diverse medical imaging modalities, models, tasks, and distribution-shift settings, providing a useful empirical resource for studying fairness in medical MLLMs.

  2. Structured bias analysis. The framework attempts to connect demographic performance disparities with image, query, cross-modal, and reference-response distributions, offering an interpretable perspective on fairness.

Critical weaknesses
  1. The core theoretical framework closely follows Oh et al. (ICML 2025), which already introduced EMI and analyzed the same five types of distributional discrepancies. The main extension is applying this analysis to demographic groups. Furthermore, FMBench, CARES, and FairMedFM have already investigated fairness in medical vision-language models. The incremental contribution is insufficiently distinguished from these works.

  2. The five proposed bias sources are largely model-independent data discrepancies, which do not necessarily explain model-specific unfairness. The experiments establish correlations rather than causal attribution. Moreover, FEMI correlates only weakly with actual utility gaps (Spearman ρ = 0.17–0.40), and the pooled correlation analysis repeatedly uses identical source values across different models. These limitations undermine the central claim that the framework can reliably identify the origins of unfairness.

Questions for authors
  1. What is the fundamental theoretical novelty beyond applying the EMI-based distribution-shift analysis of Oh et al. (2025) to demographic groups?

  2. How is FEMI justified as a proxy for the actual fairness metrics, given its relatively weak empirical correlations?

  3. Do the reported bias-source correlations remain significant under dataset-level analysis or leave-one-dataset-out evaluation, particularly after controlling for disease prevalence and task difficulty?

Reviewer_n4MDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces MedFairVLM, a theoretical framework and benchmark for evaluating fairness in medical MLLMs. The authors derive an upper bound decomposing unfairness into five distributional sources and extend the analysis to distribution shifts. Experiments involving ten MLLMs and eight datasets across four imaging modalities demonstrate widespread demographic disparities, inconsistent effectiveness of fairness interventions, and increased fairness challenges under distribution shifts.

Critical strengths
  1. The evaluation spans multiple models, imaging modalities, clinical tasks, and deployment scenarios, providing valuable insights into fairness in medical MLLMs.

  2. The five-source decomposition offers an interpretable framework for investigating potential origins of fairness disparities beyond conventional performance comparisons.

Critical weaknesses
  1. Limited causal validation of the proposed bias sources. The theoretical decomposition provides an upper bound, but the empirical analysis primarily relies on correlations. It remains unclear whether these sources directly contribute to unfairness or merely reflect underlying distributional differences. Controlled experiments would strengthen the claims.

  2. Potential bias from automatically generated questions and evaluation. Open-ended questions are generated using GPT-5-mini, which also serves as the evaluation judge. This introduces potential systematic biases that could affect subgroup comparisons. Independent evaluation would strengthen confidence in the findings.

  3. Oversimplification of demographic attributes. Binarizing race, age, and skin type may obscure important subgroup and intersectional disparities. This limits the clinical representativeness and generalizability of the fairness conclusions.

Questions for authors
  1. Can the authors provide controlled experiments demonstrating that reducing individual bias sources leads to measurable fairness improvements?

  2. How well do the proposed fairness measures correlate with clinically meaningful errors?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.