acceptodds
Under review as a conference paper at ICLR 2027

SAFER: When Text Beats Vision in Radiology VLM Hallucination Detection

Abstract

A text-only classifier with no image access reaches AUROC 0.918 on a benchmark designed to test visual hallucination detection in radiology vision-language models (VLMs) — far exceeding every evaluated VLM, including the best real model (MedGemma-27B, AUROC 0.700). This is the central finding of SAFER (Structured Assessment of Faithfulness and Error in Radiology), an 8,294-sample hallucination-detection benchmark for chest radiograph VLMs (OpenI Indiana Chest X-ray and MIMIC-CXR), evaluated across eight VLMs and five constructed baselines, with a nine-part failure analysis that interrogates the benchmark and its evaluation metric rather than presenting them as settled instruments. Through four rounds of controlled elimination, we trace this leakage: grammar and collocation artifacts explain negligible variance; the closed 50-word swap vocabulary explains just over half (≈55%, ∆AUROC 0.229 of a 0.418 above-chance gap, chance=0.500); report length, a 53-term vocabulary list, and cross-source formatting do not explain a material fraction of the remainder under the controls we tested. A residual AUROC of 0.65–0.70 is observed separately in both evaluated sources, as an open limitation of this specific lexical-substitution injection design and a signal that cross-model AUROC comparisons on benchmarks built this way cannot, without further analysis, be attributed to visual grounding. Beyond the leakage analysis, we show that a composite metric redesigned to punish degenerate classifiers still rewards a near-constant classifier as top-ranked (a candidate MCC-floor gate is demonstrated, though not independently validated); that four of eight evaluated VLMs collapse to an almost-exactly-constant output while one additional model (MedGemma) is the only non-degenerate VLM and shows the strongest evidence of image-dependent behaviour; and that calibration terms in composite metrics can promote a text-only baseline above all real VLMs under calibration-emphasised weighting. We release all raw metrics, per-sample outputs, and the nine-experiment analysis pipeline for independent verification, and recommend eliminative, control-paired validation as standard practice for benchmarks that use model self-report as a proxy for visual clinical reasoning.

Primary area:
datasets and benchmarks
Keywords:
hallucination detection, vision-language models, radiology, benchmark validity, dataset artifacts, shortcut learning, medical imaging, chest X-ray, model evaluation, faithfulness

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_4YdbOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper introduces SAFER, a benchmark of 8,294 chest radiograph and report pairs. The authors insert errors through single word substitutions. They evaluate eight vision and language models and several control systems. A classifier trained only on report text achieves an area under the receiver operating characteristic curve of 0.9183 on a held out split. The strongest evaluated vision and language model achieves 0.7001 on the full benchmark. The paper also examines lexical shortcuts, nearly constant model outputs, and weaknesses in its composite score.

Critical strengths
  • Its score of 0.9183 is striking. Removing the original and substituted word spans reduces the score to 0.654. These results give a concrete reason to question whether performance on this benchmark measures visual grounding.
  • CheXagent obtains the highest composite score among the evaluated models despite a specificity of only 0.032. The reported ranking score for MedGemma changes by just 0.006 when images are shuffled. Both checks prevent an overly favorable reading of the model results.
Critical weaknesses
  • The paper identifies Experiments 6 through 9 as its primary methodological contribution. Appendix A.9, however, ends after Experiment 4. Table 4 provides scores without the exact transformations, retraining procedures, or uncertainty estimates. This matters for the claim that the fixed substitution vocabulary explains approximately 55 percent of the text signal. It also matters because only reports containing eligible words can receive an injected error. A comparison using clean and altered versions of the same eligible reports would test that remaining source of signal. The related work also omits SHOVIR [1], which studies visual shortcuts through controlled radiology image perturbations, and ReXTrust [2], which studies finding level hallucination detection. Their tasks differ from SAFER, but both are relevant to the claimed contribution. arxiv.org

  • The text classifier is evaluated on 2,042 held out cases. MedGemma is evaluated on all 8,294 cases. Appendix A.7 treats a cross source difference of 0.0415 as a bound on the resulting discrepancy in area under the curve. Appendix A.9.3 identifies that 0.0415 value as a difference in balanced accuracy. It cannot bound a difference in another metric across different sample sets. Patient identifiers are also available for MIMIC-CXR [3]. At minimum, they permit a patient separated analysis on that source. The reported gap is large, but the current calculations do not establish its precise size under matched evaluation.

[1] Ruffini, F., Salmé, M., Sicilia, R., Guarrasi, V., and Soda, P. “SHOVIR: A Benchmark for Evaluating Vision Shortcut Learning in Radiology Report Generation.” arXiv:2606.30201, 2026.

[2] Hardy, R., Kim, S. E., Ro, D. H., and Rajpurkar, P. “ReXTrust: A Model for Fine-Grained Hallucination Detection in AI-Generated Radiology Reports.” Proceedings of Machine Learning Research, 281:173–182, 2025.

[3] Johnson, A., Pollard, T., Mark, R., Berkowitz, S., and Horng, S. MIMIC-CXR Database, version 1.0.0. PhysioNet, 2019.

Questions for authors

What exact transformations and retraining procedures produced Experiments 6 through 9? Does the residual text signal persist when clean and altered reports are paired from the same eligible originals?

What are the text classifier and MedGemma results on the identical 2,042 cases, with paired uncertainty estimates? How does the text classifier perform with patient separated training and testing on MIMIC-CXR?

2 comments

Thank you for the careful reading. We address each point.

Missing appendix. You are right. Appendices A.9.5–A.9.13 are added with the exact masking rules, both vocabulary lists, the 30-run protocol and the retraining procedures. They distinguish the two ablations (A.9.6):

  • 0.654 applies the unmodified Experiment 5 model to test reports with injected and original spans deleted, without retraining.
  • 0.689 was the single-split result of excluding the swap vocabulary and retraining, now replaced by the 30-run mean 0.674 ± 0.010.

Uncertainty (Table 4). Over 30 re-split, re-trained runs: swap vocabulary Δ 0.239 [0.223, 0.260] (57.9% ± 2.2% of the above-chance gap); grammar and length/broader-vocabulary intervals include zero.

A.7 and matched evaluation (Table 3; A.7). You are correct that 0.0415 is a balanced-accuracy gap and cannot bound an AUROC difference. It is replaced by the direct paired comparison on the identical 2,042 cases: TF-IDF 0.918 [0.906, 0.930], MedGemma 0.707 [0.687, 0.726], paired Δ 0.212 [0.190, 0.234].

Patient separation (A.6). Our earlier statement about MIMIC-CXR identifiers was wrong. In our subset every study comes from a distinct subject, so the source is patient-disjoint by construction; a MIMIC-CXR-only classifier gives 0.887 subject-grouped vs. 0.885 report-level (paired Δ 0.002 [−0.019, 0.036]).

Paired eligible originals (§6.3). Clean vs. injected versions of the same reports give 0.675 with the swap vocabulary excluded, matching the benchmark's 0.674. This exposed a substring-matching bug in our injector, which accounts for ≈33% of the above-chance gap; <2% remains unexplained. We corrected the injector and release SAFER v1.1, on which the vocabulary-excluded residual falls to 0.552.

Re-evaluation on v1.1 (Exp. 12; §6.5; Table 12). All seven VLMs, baselines and image ablations are re-evaluated. The 1,417 changed studies are run afresh; unchanged-study outputs are reused after 200 anchor studies were re-run (verdict agreement ≥0.995); image ablations are re-run on all studies. The leak survives the fix: on the identical 2,042 studies TF-IDF reaches 0.907 vs. MedGemma 0.673 (gap 0.234 [0.211, 0.256]); on all 8,166 studies MedGemma falls from 0.700 to 0.661. Elsewhere, results are on v1 unless marked v1.1.

Related work (§2). SHOVIR and ReXTrust are now discussed. We also use ReXErr, from the ReXTrust group, as an independent test of the protocol (Exp. 13, §6.6): a text-only probe reaches 0.876 on its same-report clean/edited pairs, and zero-shot MedGemma with no image 0.768.

Presentation. The paper follows one evaluation chain (probe → elimination with intervals → attribution → image ablation → v1.1 → external benchmark), with metric material moved to A.9.13. If any section remains hard to follow, we would be grateful to know which.

Reviewer_4YdbOpenReview ↗

Official Comment

I appreciate the authors’ thorough response. My concerns are mostly resolved within the limited time available, and I would like to increase my rating.

Reviewer_Y1XiOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents SAFER as a benchmark audit of radiology VLM hallucination detection. It builds 8,294 chest X-ray/report examples from OpenI and MIMIC-CXR, leaving some reports intact and introducing one of three types of single-word errors into others. Eight VLMs and several baselines are asked to judge whether each report contains an error and to give an error probability. The most striking result is that a supervised TF-IDF classifier using only the report reaches AUROC 0.918 on its held-out split, while the best tested VLM, MedGemma-27B, reaches 0.700 on the full benchmark. This is evidence that success on this benchmark cannot automatically be read as visual grounding.

The paper's main methodological contribution is a sequence of nine checks on the benchmark and its composite HalluScore. Eliminating the closed 50-word substitution vocabulary reduces text-only AUROC from 0.918 to 0.689, explaining about 55% of the above-chance gap. The remaining grammar, report-length, expanded-vocabulary and source-format controls leave substantial text-only signal, about 0.65–0.70 within the two sources. Separately, several VLMs produce nearly constant decisions; CheXagent, for example, has recall 0.979 but specificity 0.032 while receiving the highest HalluScore among real VLMs. The authors release per-sample outputs and the analysis pipeline and explicitly acknowledge that the proposed metric repair and the residual shortcut are unresolved.

Critical strengths

The text-only result is a strong validity test of the benchmark. A simple classifier with no image access reaches 0.918 AUROC and 0.683 MCC, compared with 0.700 AUROC and 0.395 MCC for the strongest tested VLM. The precise ranking is less important than the fact that an image-free model can recover so much of the label. For a benchmark advertised as detecting visual hallucinations, that observation changes what the reported VLM numbers mean.

The follow-up analysis adds substance. The drop after masking the swap vocabulary is large (0.229 AUROC), whereas grammar repair changes AUROC by only 0.002 and the length/broader-vocabulary control by 0.006 relative to the preceding test. The paper also shows why a high composite score or high model agreement can coexist with poor discrimination. I appreciate that it treats the MCC floor as a candidate fix and reports that its own redesigned score still rewards a nearly constant classifier. This self-auditing makes the contribution useful beyond the particular model list. The MedGemma image-shuffle control is especially sobering: its AUROC changes only from 0.7001 to 0.6941, so even the best VLM's score is weak evidence of image use on this benchmark.

Critical weaknesses

The 0.918-versus-0.700 headline uses different evaluation populations: the TF-IDF score comes from 2,042 held-out examples within an 8,166-example intersection, whereas MedGemma is scored on 8,294 examples. Appendix A.7 argues that a cross-source AUROC gap of 0.0415 is too small to explain the observed 0.218 difference, but this is an empirical analogy, not a bound on the actual subset-composition effect. A direct paired comparison on the same 2,042 examples is available in principle from the released VLM outputs and would make the main figure much cleaner. The text split is also not confirmed patient-disjoint. Appendix A.6 gives plausible evidence against a large diffuse patient-style shortcut, yet those controls were not designed to test patient identity directly.

  1. The construction itself creates another possible shortcut: only reports containing an eligible term can receive an injected error. If untouched reports include many ineligible examples, the two classes need not have comparable starting distributions even after the obvious swapped tokens are removed. The paper identifies this concern but does not directly match eligible clean reports to injected ones. Thus the residual 0.65–0.70 AUROC is evidence that the benchmark remains text-readable, while its mechanism remains open. The small control deltas are point estimates without bootstrap intervals; I would avoid interpreting a 0.002 or 0.006 change as an established null effect. There is also an asymmetry in comparing a supervised text classifier with prompted VLMs: it is a valid benchmark leakage probe, but the numerical gap should not be framed as a fair contest between detection systems. These issues narrow the causal explanation, though they do not erase the broader warning about lexical-substitution benchmarks.
Questions for authors

Could the authors recompute MedGemma and the other VLM AUROCs on exactly the same 2,042 examples used for the TF-IDF test score, with paired uncertainty intervals? This is the direct check on the headline comparison; the cross-source gap in Appendix A.7 is not a substitute for it.

Can the authors rerun the text-only control after restricting both classes to reports eligible for the same substitution, or otherwise match eligibility before assignment? If the residual AUROC falls substantially, the interpretation of Experiments 7–9 changes. Please also clarify whether exact or near-duplicate reports were removed across the train/test split and provide confidence intervals for the small elimination deltas.

1 comment

Thank you for a precise and constructive review. We ran each check you asked for.

Q1: Paired comparison (§6.1, Table 3; all systems in A.7, Table 7). On the identical 2,042 test studies: TF-IDF 0.918 [0.906, 0.930], MedGemma 0.707 [0.687, 0.726], paired Δ 0.212 [0.190, 0.234]. Every TF-IDF − system difference is at least 0.212. We agree the former A.7 was an analogy, not a bound, and removed it. On the supervised-vs-prompted asymmetry: MedGemma prompted with no image reaches 0.695 on the same studies.

Q2: Eligibility (§6.3, Table 4). We ran this as you suggested, and it changed our interpretation:

  • Clean vs. injected versions of the same eligible reports (split by report) give 0.924 with full text and 0.675 with the swap vocabulary excluded, matching the benchmark's 0.674. The residual is therefore not selection.
  • Its source is substring matching in our injector: 28.5% of injections fall inside longer words.
  • Restricting the clean class to eligible reports, after excluding perturbation-site tokens, gives 0.508.

This revises our reading of Experiments 7–9; the paper reports the full decomposition (≈58% vocabulary, ≈33% substring artifacts, ≈8% eligibility, <2% unexplained). We then corrected the injector and re-evaluated all seven VLMs on SAFER v1.1 (Exp. 12, §6.5): the substring residual disappears, but the vocabulary leak remains (TF-IDF 0.907 vs. MedGemma 0.673 on the matched studies).

Duplicates (A.6). 316 test reports have exact text matches in training (mostly short OpenI normal reports); excluding them gives 0.893. Grouping the split by subject, study and identical normalised text gives 0.910 ± 0.009 over 30 runs. We did not apply fuzzy near-duplicate matching; given that exact duplication moves AUROC by at most ≈0.03, near-duplicates are unlikely to explain the headline.

Small deltas (Table 4). Over 30 re-trained splits: Exp. 6 Δ 0.001 [−0.002, 0.004]; Exp. 8 Δ 0.001 [−0.003, 0.006]. Both intervals include zero; the text now says "no detectable effect" rather than "not implicated".

Framing. Agreed; TF-IDF is now presented as a leakage probe, not a competing detector. Your broader warning about lexical-substitution benchmarks also holds beyond our construction: on ReXErr-v1, an independent LLM-built benchmark, the same probe reaches 0.876 on same-report pairs and zero-shot MedGemma with no image reaches 0.768 (Exp. 13, §6.6).

Reviewer_eoi1OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper builds SAFER, an 8,294-sample hallucination detection benchmark for chest radiograph VLMs (OpenI + MIMIC-CXR, with hallucinations injected through single-word substitution from a closed vocabulary), and evaluates 8 VLMs and 5 baselines on it. The paper finds that a TF-IDF text classifier with no image access reaches an AUROC of 0.918, far above the best VLM (MedGemma-27B, 0.700), which suggests that the benchmark's label information leaks heavily into surface text features. Through nine failure-analysis experiments, the authors trace the source of this leakage and find that the closed 50-word vocabulary explains about 55% of the above-chance gap.

Critical strengths
  1. The topic concerns real risks in clinical VLM evaluation, so the perspective has practical value.

  2. The analysis of eliminative attribution chain is informative. By controlling different variables, the authors rule out factors such as grammar and report length, and successfully identify the closed 50-word vocabulary as a major factor, which explains 55% of the performance gap.

Critical weaknesses
  1. I think the core finding of this paper lacks generalizability. The paper mainly discusses perturbation-based benchmark construction using single-word lexical substitution over a closed vocabulary, but works that actually use this method are not mainstream. The two related papers cited in the article are not dedicated hallucination detection benchmarks: RadFM and MedVersa are both general-purpose models, and hallucination detection is only one of their tasks. Since the paper's observations mainly come from its own constructed setting (image–report pairs + closed-vocabulary single-word substitution + binary classification), the conclusions are hard to generalize directly to other radiology hallucination detection work (e.g., RadFlag[1] and ReXTrust[2]).

  2. The paper's claims are somewhat overstated relative to its experimental design. The text-only classifier baseline is trained first and then tested to reach the very strong AUROC of 0.918, while the VLMs are evaluated zero-shot. I therefore suspect that a large part of the text classifier's information leakage actually comes from training, and the true text leakage may be much smaller. For a more rigorous experiment, a more appropriate baseline would be a text LLM tested zero-shot. In addition, the test sets of the baseline and the VLMs are not identical (2K vs. 8K), which further reduces the reliability of the results.

  3. Even for the relatively narrow task of "image–report pairs + closed-vocabulary single-word substitution + binary classification," the paper does not fully explain the performance gap between the text-only classifier baseline and the VLMs (the authors account for about 55% of the gap, leaving around 45% unexplained). I encourage the authors to dig deeper so that this finding becomes more complete and comprehensive.

References:

[1] Zhang, Serena, et al. "Radflag: A black-box hallucination detection method for medical vision language models." arXiv preprint arXiv:2411.00299 (2024).

[2] Hardy, Romain, et al. "Rextrust: A model for fine-grained hallucination detection in ai-generated radiology reports." arXiv preprint arXiv:2412.15264 (2024).

Questions for authors

Please see weaknesses.

2 comments

Thank you for the review. We ran the experiments you suggested, including the test of generalisation on an independent benchmark.

Supervised classifier vs. zero-shot VLMs (§6.4; Table 3). We ran the zero-shot text-only control you proposed: MedGemma-27B prompted with the report alone (no image, no training).

  • Matched 2,042 studies: no image 0.695 [0.675, 0.716]; true image 0.707; blank 0.701; shuffled 0.703.
  • All 8,166 studies: no image 0.685 vs. 0.700 with the image (paired Δ 0.015 [0.007, 0.022]).

The label is largely readable from text without training: the best VLM retains ≈93% of its above-chance signal with no image. We agree the TF-IDF comparison is not a contest between detectors and now present it as a leakage probe.

Test-set mismatch (Table 3; A.7). All systems are compared on the identical 2,042 studies: TF-IDF − MedGemma = 0.212 [0.190, 0.234].

The remaining ≈45% (§6.3). Now explained: a substring-injection artifact. 28.5% of injections fell inside longer words, creating tokens absent from clean reports. Clean vs. injected versions of the same report reproduce the residual exactly (0.675 vs. 0.674), ruling out selection; word-boundary injection reduces it to 0.552. Less than 2% of the gap remains unexplained.

Generalisability (new Exp. 13; §6.6, Table 5; A.9.12). We tested this directly on ReXErr-v1 (Rao et al., PSB 2025; same group as ReXTrust and RadFlag), which is built with the opposite design to SAFER: GPT-4o injects three clinically motivated errors per MIMIC-CXR report from 12 categories, with an open vocabulary, and each edited report comes with its original.

  • Text-only probe on clean vs. edited versions of the same reports (split by patient, 20,000 pairs): 0.876 [0.869, 0.880]; report length alone 0.567.
  • Not only linguistic errors leak: at sentence level, false negations reach 0.840 and false predictions 0.831 from text alone; location changes are hardest (0.624), mirroring SAFER's relational type.
  • No-image control: zero-shot MedGemma with no image reaches 0.768 [0.757, 0.778] (blank image 0.790) on 2,708 test pairs.

So the leak is not specific to closed-vocabulary swaps: an LLM-built, open-vocabulary benchmark leaks comparably, and a VLM's score on it is likewise largely obtainable without the image. The true-image condition could not be run (ReXErr's MIMIC-CXR test images were not available to us), and benchmarks with human-authored errors remain untested; both are stated in the Limitations. RadFlag and ReXTrust are discussed in §2; they detect hallucinations in model-generated reports rather than building benchmarks by perturbation, so they are outside the scope of this audit. The RadFM/MedVersa "widely used" framing is removed.

The protocol is constructive. It located a concrete bug in our own injector, quantified it, and led to a corrected injector (SAFER v1.1). Re-evaluating all seven VLMs on v1.1 (Exp. 12, §6.5) shows the finding survives: on the matched 2,042 studies TF-IDF reaches 0.907 vs. MedGemma 0.673. On all 8,166 studies MedGemma scores 0.661, and without the image (0.643) still retains 89% of its above-chance AUROC.

I appreciate the response from the authors.

The new experiments convincingly address my main concerns: the zero-shot text-only control and the matched 2,042-study evaluation resolve the training asymmetry and test-set mismatch, and the substring-injection artifact explains the previously unexplained residual. The external validation on ReXErr-v1 shows the leakage finding extends beyond closed-vocabulary substitution, which largely addresses my generalizability concern, and the remaining limitations are clearly acknowledged. I am therefore raising my score.

Comments on the paper

2 comments

We thank the reviewers. All three accept that a report-only classifier recovers much of SAFER's label, and raise the same rigor concerns. We ran every requested control. Matched = the identical 2,042 held-out test studies; full = all 8,166 unique studies.

1. Matched evaluation (§6.1, Table 3; A.7). TF-IDF 0.918 [0.906, 0.930] vs. MedGemma 0.707 [0.687, 0.726]; paired Δ 0.212 [0.190, 0.234] (DeLong p<10−79p < 10^{-79}). Main results are recomputed on the full set (MedGemma 0.700 [0.690, 0.710]). We removed the former A.7 argument, which, as Reviewer 4Ydb noted, used a balanced-accuracy gap to bound an AUROC difference.

2. Zero-shot, no-image control (§6.4). Full set: MedGemma prompted with the report alone (no image, no training) reaches 0.685 [0.675, 0.695] vs. 0.700 with the image; paired Δ 0.015 [0.007, 0.022]. Blank image 0.699; shuffled 0.694. The image adds a small but reliable signal, while ≈93% of the best VLM's above-chance AUROC is recoverable from text by a zero-shot model. The leak does not depend on supervised training.

3. Uncertainty (Table 4). Over 30 re-split, re-trained runs:

  • Grammar repair: Δ 0.001 [−0.002, 0.004]
  • Swap vocabulary: Δ 0.239 [0.223, 0.260] (57.9% ± 2.2% of the above-chance gap)
  • Broader vocabulary: Δ 0.001 [−0.003, 0.006]
  • 50 random words instead: unchanged (0.914)

4. The residual is explained (§6.3). Our injector matched substrings, so 1,088 of 3,818 injections (28.5%) landed inside longer words, creating tokens absent from clean reports (lungs→livers, pleural→diaphragml, enlarged→ensmalld).

  • Clean vs. injected versions of the same reports give 0.675 with the vocabulary excluded, matching the benchmark's 0.674. Selection cannot produce this.
  • Excluding perturbation-site tokens gives 0.540; also restricting clean reports to eligible ones gives 0.508.
  • Word-boundary re-injection gives 0.912 (full text) and 0.552 (vocabulary excluded).
  • The gap decomposes as ≈58% vocabulary, ≈33% substring artifacts, ≈8% eligibility, <2% unexplained.

5. Patients and duplicates (A.6). Every MIMIC-CXR study in our subset is from a distinct subject; a subject-grouped split gives 0.887 vs. 0.885. Excluding 316 test reports with exact text matches in training gives 0.893; grouping by subject, study and identical text gives 0.910.

6. Presentation. Appendices A.9.5–A.9.13 are added; related work covers RadFlag, ReXTrust, SHOVIR and ReXErr; the paper follows a single evaluation chain.

7. Corrected benchmark (Exp. 12; §6.5; Table 12). All seven VLMs, baselines and image ablations are re-evaluated on SAFER v1.1 (word-boundary injection; changed studies run afresh, unchanged ones reused after a 200-study anchor check). The substring residual is removed, but the vocabulary leak remains: matched TF-IDF 0.907 vs. MedGemma 0.673 (gap 0.234 [0.211, 0.256]). On the full set MedGemma falls from 0.700 to 0.661; with no image (0.643) it retains 89% of its above-chance AUROC.

8. Generalisation to an independent benchmark (Exp. 13; §6.6, Table 5; A.9.12). On ReXErr-v1 — built by a different group with GPT-4o injecting three open-vocabulary errors per report — a text-only probe separates clean and edited versions of the same reports with AUROC 0.876 (patient-grouped, 20,000 pairs), and zero-shot MedGemma with no image reaches 0.768 [0.757, 0.778]. Even errors that in principle require the image leak from text (false negation 0.840, false prediction 0.831). The finding is not specific to SAFER's construction.

Revised PDF uploaded (third revision). All requested controls are now in the paper; the central claim is unchanged. Appendix A.10 lists every change. Results are on SAFER v1 unless marked v1.1. Matched = the identical 2,042 held-out test studies; full = all 8,166 unique studies.

Matched evaluation (§6.1, Table 3; Appendix A.7). TF-IDF 0.918 [0.906, 0.930] vs. MedGemma 0.707 [0.687, 0.726]; paired Δ 0.212 [0.190, 0.234] (DeLong p<10−79p < 10^{-79}). Every other system trails by at least as much. The former A.7 bound is removed.

One study set (Table 2; A.4). All systems are evaluated on the same 8,166 studies with bootstrap CIs; using all 8,294 rows changes no AUROC by more than 0.002.

30-run elimination (Table 4; A.9.5–A.9.8). The swap vocabulary explains 57.9% ± 2.2% of the above-chance gap (Δ 0.239 [0.223, 0.260]); grammar and length/broader-vocabulary intervals include zero; a 50-random-word control leaves AUROC at 0.914.

Residual attributed (Exp. 10; §6.3; A.9.9). Substring matching placed 28.5% of injections inside longer words, creating tokens absent from clean reports (lungs→livers, pleural→diaphragml). Same-report pairs give 0.675 with the vocabulary excluded vs. the benchmark's 0.674, so selection cannot explain it. Gap decomposition: vocabulary 57.9%, substring artifacts 32.5%, eligibility 7.8%, unexplained 1.8%.

Zero-shot image ablation (Exp. 11; §6.4). Full set: MedGemma 0.685 with no image vs. 0.700 with it (paired Δ 0.015 [0.007, 0.022]); blank 0.699, shuffled 0.694. ≈93% of its above-chance AUROC is recoverable from text alone.

Corrected benchmark, SAFER v1.1 (Exp. 12; §6.5; A.9.11, Table 12). All seven VLMs, baselines and image ablations are re-evaluated. The 1,417 changed studies are run afresh; unchanged-study outputs are reused after a 200-study anchor re-run (verdict agreement ≥0.995); image ablations are re-run on all studies. The substring residual disappears (same-report pairs, vocabulary excluded: 0.500), but the leak does not: matched TF-IDF 0.907 vs. MedGemma 0.673, gap 0.234 [0.211, 0.256]. On the full set MedGemma falls from 0.700 to 0.661 (it had also used the artifacts); with no image it scores 0.643, retaining 89% of its above-chance AUROC.

New: external benchmark, ReXErr-v1 (Exp. 13; §6.6, Table 5; A.9.12). The same controls applied to an independent benchmark built by a different group with the opposite design (GPT-4o injects three errors per MIMIC-CXR report from 12 categories, open vocabulary). A text-only probe on clean vs. edited versions of the same reports (patient-grouped, 20,000 pairs) reaches 0.876 [0.869, 0.880]; length alone 0.567. Clinical-content errors leak too (false negation 0.840, false prediction 0.831). Zero-shot MedGemma with no image reaches 0.768 [0.757, 0.778] (blank 0.790; 2,708 test pairs). The leak is not specific to our construction.

HalluScore (A.9.13). Against six degenerate classifiers on both versions, every variant (original; MCC-gated with τ fixed on v1, tested on v1.1; threshold-free) lets a degenerate classifier outscore at least two real VLMs. We recommend reporting AUROC and MCC with intervals, and calibration separately.

Patients and duplicates (A.6). Our MIMIC-CXR subset has 4,500 distinct subjects (subject-grouped 0.887 vs. 0.885). Grouping by subject, study and identical text gives 0.910; excluding 316 exact-duplicate test reports gives 0.893.

BioViL-T removed (A.10). Its loaded checkpoint exposed no image encoder and our wrapper fell back to a text-only score, so we removed it; no other result depended on it.

Scoring descriptions corrected (§3.2; A.5). Code and outputs are unchanged; the text now states that MedGemma, Qwen2.5-VL, LLaVA-1.5 and Phi-3-Vision are prompted, CheXagent is scored by report comparison, and BLIP-2/BiomedCLIP by an image–text matching margin.

Framing. The contribution is a modality-dependence audit protocol. Beyond a text-only baseline (hypothesis-only NLI; VQA language priors), it adds counterfactual same-report pairs and a no-image control, which decompose a leak into construction choices; on SAFER this found an injector bug a text-only baseline alone would not, and on ReXErr it shows the same leak in an independently built benchmark. v1.1 is released as an audit artifact, not a leaderboard. Related work now covers RadFlag, ReXTrust, SHOVIR and ReXErr. Benchmarks with human-authored errors remain untested (Limitations).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.