- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper investigates the effectiveness of RAG for medical VQA. The authors evaluate four vision-language models across three medical VQA benchmarks using five RAG configurations. The results show that RAG consistently degrades performance under the evaluated settings. The authors attribute this phenomenon, termed the "Retrieval Paradox," to "Semantic Overpowering," where retrieved text distracts models from visual information.
Critical strengths
-
The paper studies a practically important problem concerning the reliability of RAG in medical VQA.
-
The authors conduct several ablation experiments, including oracle retrieval, random-text controls, reranking, and attention analysis, to investigate the underlying failure mechanisms.
Critical weaknesses
W1. Poor presentation and insufficient methodological clarity.
The paper is difficult to follow, as many experimental settings are introduced without sufficient explanation or motivation. For example, the five RAG configurations and the reranker are presented without adequately explaining their design choices or why they are considered. Furthermore, almost ALL mathematical notations are introduced without proper definitions, and the results in Sec. 4.2 are poorly organized. Overall, the paper reads more like an experimental report than a systematic research study.
W2. Limitations of the experimental setting.
Although the paper investigates RAG for medical VQA, none of the evaluated models is a medical-domain VLM. This substantially limits the generalizability of the findings to medical VLMs.
**W3. The RAG pipeline is insufficiently justified. **
In particular, the retrieval corpus consists of medical image-QA pairs, but the retrieved snippets are injected only as plain-text prefixes ("Similar case: Q: [question] A: [answer]"), without including the corresponding retrieved images. The paper does not explain this design choice or investigate how excluding the retrieved visual information affects the results. Consequently, it is unclear whether the findings reflect general limitations of multimodal RAG or simply the particular retrieval strategy adopted.
W4. Concerns about the reliability of the experimental results.
The reported 96.0% zero-shot accuracy of BLIP-2 on VQA-RAD is unexpectedly high. For reference, the SoTA medical VLM Lingshu-7B reports 67.9% on VQA-RAD, although the evaluation protocols may differ. Such a substantial discrepancy requires further explanation.
Additionally, there are numerical inconsistencies in the reported results. For example, Table 7 reports 45.0% baseline accuracy and 40.0% random-text accuracy but assigns a 0.0 percentage-point difference. These issues raise concerns about the reliability of the evaluation and the conclusions drawn from the experimental results.
W5. Limited novelty relative to existing research.
The paper does not adequately distinguish its proposed Semantic Overpowering phenomenon from closely related prior research. In particular, Zhao et al., When RAG Hurts: Diagnosing and Mitigating Attention Distraction in Retrieval-Augmented LVLMs (ICML 2026), already investigate how retrieved text suppresses visual attention and distracts models from question-relevant image regions. The authors should clarify what new insights or findings their work establishes beyond this existing study.
Questions for authors
Please see the weakness.