acceptodds
Under review as a conference paper at ICLR 2027

VIGOR: Enhancing Reliability and Reasoning in Medical Report Generation through Reinforcement Learning

Abstract

Multimodal Large Language Models have shown strong performance in vision–language tasks, yet their application to medical report generation remains limited by insufficient visual grounding, opaque reasoning, and reliance on supervised fine-tuning, which primarily learns to imitate reference reports rather than explicitly optimizing clinically grounded objective. We propose VIGOR, a two-stage reinforcement learning framework for radiology report generation. In the first stage, we introduce a verifiable IoU-based reward and optimize the model using GRPO, enabling the trained model to explicitly align textual descriptions with region-level visual evidence, thereby enforcing accurate localization of abnormalities and improving interpretability. In the second stage, we extend GRPO to report generation by incorporating disease label and LLM-as-a-Judge reward to promote diagnostic accuracy and coherent reasoning. This two-stage optimization enables the generation of reports that are visually faithful, diagnostically consistent, and grounded in interpretable reasoning. Extensive experiments on the resulting model, VIGOR-4B, demonstrate reduced hallucinations, improved disease identification, and stronger alignment with verifiable evidence. Our results highlight the importance of explicit visual grounding and reward-guided optimization for developing reliable and faithful medical report generation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Medical Report Generation, Vision Language Model

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_dcZ3OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes VIGOR, a two-stage reinforcement learning framework for chest X-ray report generation. In the first stage, the model is warmed up with SFT and then trained with GRPO and a verifiable IoU reward to localize abnormal regions. In the second stage, the predicted regions are cropped and fed into the model together with the global image to generate reports. Experiments on multiple public chest X-ray datasets show that VIGOR-4B outperforms open models of similar size and is competitive with closed models. The authors also use ablation studies, a cross-judge evaluation, and a blinded physician study to support the effectiveness of each component. Overall, I think this is a well-motivated and solid paper.

Critical strengths
  1. The two-stage design, which grounds abnormal regions first and then generates reports conditioned on the cropped regions, is intuitive and well motivated. Feeding localized visual evidence directly into report generation is a reasonable way to strengthen the LLM report generation.
  2. VIGOR-4B outperforms open models of similar or larger size on most metrics. It also remains competitive with strong closed models such as GPT-5 and Gemini-2.5-Pro, despite having only 4B parameters.
  3. The cross-judge evaluation and the blinded physician study add credibility to the results.
Critical weaknesses
  1. The authors should report the length of the generated reports and the number of findings per report, to rule out the possibility that VIGOR's advantage comes from generating less text.
  2. How are the predicted boxes matched to the ground-truth boxes when their numbers differ? Is there any penalty when the model predicts the wrong number of boxes?
  3. The paper does not describe the data or the training settings used in the SFT stage.
  4. VIGOR's Disease score is inconsistent between Table 4 (3.318) and Table 2 (3.138).
Questions for authors

Please see the Weaknesses above.

3 comments

W1. The length and number of findings per report should be reported.

A. Thank you for the suggestion. Please see the below table and Appendix Table. 8 of our revised manuscript. VIGOR generates longer reports (46.8 words) with more findings (3.22) than both MedGemma-4B (39.2 words, 2.84 findings) and MedGemma-4B + SFT (42.3 words, 2.92 findings). Importantly, these results address the concern that VIGOR's performance improvements might simply result from generating shorter reports or omitting clinical findings. Instead, VIGOR achieves superior performance while generating more detailed reports containing a greater number of findings. This suggests that its improvements are not merely attributable to reduced report length or fewer reported findings.

Table 4. Average Report Length and Number of Findings per Report

MethodLength of Generated ReportsNumber of Findings per Report
LLaVA-Med-1.5-7B30.21.43
HuluMed-7B37.43.25
Med-Gemma-VL-4B39.22.84
Lingshu-7B40.62.90
Med-Gemma-VL-4B + SFT42.32.92
VIGOR (Ours)46.83.22
GPT-550.53.98
LLaVA-1.5-7B51.51.63
HuluMed-4B55.93.98
MAIRA-260.42.66
Gemini 2.5 Pro75.64.35
Qwen3-VL-4B-Instruct77.44.71
Gemini 2.5 Flash86.24.12

W2. Box matching and penalties when the number of predicted boxes differs from the ground-truth boxes.

A. Thank you for raising this concern. Although we do not explicitly penalize the number of predicted bounding boxes, we focus on spatial alignment between predicted and ground-truth regions through IoU. Thus, missing or inaccurately localized ground-truth regions result in lower IoU rewards. On the test set, the average number of ground-truth bounding boxes was 1.90, while the average number of predicted bounding boxes was 1.79. Nevertheless, it would be an interesting direction for future research to explicitly optimize the number of predicted bounding boxes.

W3. Data and training settings used in the SFT stage.

A. In Stage 1 SFT of VIGOR, we used 5,839 abnormal cases with ground-truth bounding-box annotations from MS-CXR, VinDr-CXR, and NIH Chest X-ray14, randomly split into training, validation, and test sets at an 8:1:1 ratio. In SFT baseline, we used chest X-rays from MIMIC-CXR and Open-I, with their ground-truth FINDINGS sections serving as the target text for report generation. We randomly split the data into training, validation, and test sets at an 8:1:1 ratio. For both SFT experiments, we fine-tuned the model for three epochs by using cross-entropy loss and the AdamW optimizer with a learning rate of 5 × 10⁻⁶.

W4. Typo.

A. Thank you for pointing this out. The correct score is 3.138, as reported in Table 2 of the manuscript.

Thanks for the author's effort. I think the rebuttal has addressed my concerns, and I will maintain my score.

We would like to thank reviewer dcZ3 for the positive and constructive comments. We are glad that we resolved the concerns.

Reviewer_oX9JOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a two-stage RL framework for improving visual grounding and clinical reliability in chest X-ray report generation. In Stage 1, it trains the model to localize abnormal regions through SFT and then GRPO with an IoU-based reward. In Stage 2, the model generates radiology reports conditioned on the image and localized visual regions, using a combination of disease-label consistency and LLM-as-a-judge rewards. The model is built upon MedGemma 4B. The experiments demonstrate improvements in visual grounding, with mean IoU increasing from 0.114 to 0.29 through SFT and from 0.29 to 0.406 through Stage 1 RL. The entire VIGOR-4B framework improves CheXbert F1 from 0.464 to 0.503 and RadGraph-XL from 0.333 to 0.352. The authors also provide component ablations, cross-judge and physician evaluations, and supplementary material showing the benefits of visual grounding and improvements across different model backbones.

Critical strengths
  1. The proposed method addresses an important limitation of medical report generation—the difficulty of generating clinical statements grounded in visual evidence. It effectively combines region-level localization and report generation quality as verifiable RL training rewards. The improvement in visual grounding is significant: 0.29 -> 0.406 in mean IoU. Supplementary Table 1 provides additional evidence that visual grounding contributes to downstream performance. Removing visual grounding during inference decreases CheXbert F1 from 0.503 to 0.471 and the hallucination-avoidance score from 3.942 to 3.050.
  2. VIGOR-4B improves upon MedGemma-4B on multiple report generation metrics: disease-label consistency, localization accuracy, hallucination avoidance, CheXbert F1, and RadGraph-XL. The authors evaluate the method using different LLM judges and conduct a blinded physician comparison on 100 cases. The supplementary material also shows improvements on other backbones, such as LLaVA-Med-7B and Qwen3-VL-4B.
Critical weaknesses
  1. The evaluation protocol and dataset partitions are under-specified. The authors use several chest X-ray datasets for training, such as MIMIC-CXR, MS-CXR, and VinDR-CXR. However, the paper doesn't show the exact evaluation datasets, sample sizes, or train/test splits.
  2. Although the main paper includes a MedGemma-4B+SFT baseline, it is unclear whether this baseline uses the same data and visual conditioning as the full model. Comparing report generation with SFT vs. Stage 2 RL under identical visual inputs and training data would help directly establish the benefit of reward-guided optimization.
  3. The technical novelty is limited. Prior work has explored RL with clinically motivated rewards for medical image report generation, e.g., Miura et al., NAACL 2021; Delbrouck et al., EMNLP 2022.
Questions for authors
  1. Regarding Weakness 1, what are the evaluation datasets, sample sizes, and train/test splits?
  2. Can you compare SFT-only vs. Stage 2 RL under identical data and visual inputs? How does report generation differ when using SFT-trained vs. RL-trained grounding boxes with the same report-generation model?
  3. Does the Stage 2 training judge receive ground-truth FINDINGS? Could the authors clarify the differences between the training and evaluation judge protocols?
3 comments

W1(Q1). Clarifying the exact evaluation protocol and dataset partitions.

A. For Stage 1 (visual grounding), we used 5,839 abnormal cases with ground-truth bounding-box annotations from MS-CXR, VinDr-CXR, and NIH Chest X-ray14, randomly split into training, validation, and test sets at an 8:1:1 ratio. For Stage 2 (report generation), we used 309,077 cases from MIMIC-CXR, MS-CXR, VinDr-CXR, NIH Chest X-ray14, and Open-I, also randomly split into training, validation, and test sets at an 8:1:1 ratio.

W2(Q2). Adding an SFT baseline trained on the same data and cropped regions would make the contribution of RL clearer.

A. Thank you for the suggestion. Please see "general response" and below table. (Appendix Table. 6)

Table 1. Comparison of SFT and RL under identical training data and cropped regions.

MethodDisease ↑Local ↑Severe ↑Halluc ↑
Med-Gemma-VL-4B + SFT (Same training data and cropped regions)2.7722.5302.0172.663
VIGOR (Ours)3.2363.3382.7273.886
MethodF1 ↑Precision ↑Recall ↑RadGraph-XL ↑
Med-Gemma-VL-4B + SFT (Same training data and cropped regions)0.4730.4780.4700.237
VIGOR (Ours)0.5060.5050.5070.349

To clarify the contribution of reinforcement learning, we compare VIGOR with an SFT baseline using the same training data (MIMIC-CXR and Open-I, FINDINGS is available) identical cropped regions. With the cropped-region inputs held fixed, VIGOR outperforms the SFT baseline across all eight metrics. In particular, its gains in localization accuracy (2.530 to 3.338) and hallucination avoidance (2.663 to 3.886) suggest that reward-guided optimization helps the model describe the anatomical locations of findings more accurately and avoid unsupported findings given the same visual evidence.

W3. Technical novelty.

A.Thank you for pointing us to these relevant works. We cite both papers in our Related Works (Reinforcement Fine-Tuning paragraph) in revised manuscript. Miura et al. [1] and Delbrouck et al. [2] have explored reinforcement learning with clinically motivated rewards for radiology report generation. However, their reward functions rely on ground-truth radiology reports to evaluate factual consistency. In contrast, VIGOR introduces a framework that combines verifiable IoU-based visual grounding with clinical reward-guided report generation. Importantly, our Stage 2 disease-label and LLM-as-a-Judge rewards can be applied without ground-truth radiology reports, enabling reward-based optimization even when reference reports are unavailable. This is particularly valuable in medical imaging, where obtaining expert-written reports is costly and time-consuming. Therefore, our contribution goes beyond applying clinically motivated RL by integrating explicit region-level visual grounding with clinical reward optimization that can operate without reference reports, providing a more flexible framework for clinically grounded radiology report generation.

References

[1] “Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generation.”, NAACL 2021.

[2] “Improving the Factual Correctness of Radiology Report Generation with Semantic Rewards.”, EMNLP 2022.

Q3. Clarifying the Stage 2 training and judge protocol.

A. Thank you for the question. Please see Appendix Sec. B, when "ground-truth FINDINGS" are available, such as in MIMIC-CXR and Open-I, they are provided to the Stage 2 training judge. For datasets where "ground-truth FINDINGS" are unavailable, the judge operates without them. We use the same judge protocol for both training and evaluation.

Q4. Ethics Concerns.

A. Thank you for raising this important concern. In PhysioNet’s guidance [1] on the use of MIMIC data with LLMs and online services, PhysioNet states that researchers using external APIs should verify that the service settings ensure zero data retention, no use of the data for model training, and no human review. We obtained approval for Zero Data Retention (ZDR) for both OpenAI GPT-5 and Gemini-2.5-Pro prior to conducting the experiments. Both APIs were used under ZDR configurations designed to prevent data retention, provider-side model training, and routine human review.

References

[1] https://physionet.org/news/post/llm-responsible-use/

Thanks for clarifying the dataset sizes and splits. Could you confirm whether the random 8:1 splits were performed at the patient level or the individual level? Since MS-CXR is derived from MIMIC-CXR, how did you ensure that overlapping images or studies do not appear across training and evaluation partitions, including across stage 1 and stage 2?

Thanks for adding the matched SFT baseline, which helps clarify the contribution of stage 2 RL. one part of my original question remains, could you compare report generation using stage 1 sft-trained boxes versus stage 1 rl-trained boxes, while keeping the stage 2 report-generation model fixed? This would help isolate whether improved grounding through stage 1 rl directly contributes to better report generation.

Thanks for clarifying the method contribution. However, I remain slightly concerned about the novelty. For example,

  • Visual-RFT (ICCV 2025) already explores GRPO with verifiable IoU rewards for visual grounding.
  • MedLoc-R1 (CVPR 2026) applies GRPO with IoU-based rewards to medical visual grounding.
  • BoxMed-RL (Medical Image Analysis, 2026) combines spatially verifiable reinforcement learning with chest X-ray report generation.
  • RadVLM-GRPO (MIDL 2026) applies GRPO to both chest X-ray report generation and visual grounding. Given these closely related approaches, could the authors clarify what the key novelty of VIGOR is beyond integrating existing grounding and report generation RL techniques? In pariticular, how does VIGOR differ from BoxMed-RL and RadVLM-GRPO?

We thank Reviewer oX9J for the follow-up questions.

Q1. Clarifying dataset partition.

A. We performed the random splits at the patient level. We used MS-CXR for visual grounding only in Stage 1 and corrected its erroneous inclusion in the Stage 2 dataset list in the revised manuscript. Because MS-CXR images are drawn from MIMIC-CXR, we constructed the training and evaluation partitions to be disjoint at both the image and study levels, within each stage and across the two stages. We verified this separation by comparing image and study identifiers across the partitions.

Q2. Effect of visual grounding on report generation with a fixed Stage 2.

A. We evaluate using a fixed Stage 2 report generation. The only change was whether the crops came from boxes predicted by the Stage 1 SFT or the Stage 1 RFT.

Table 5. Effect of Stage 1 grounding on report generation with a fixed Stage 2

MethodDisease ↑Local ↑Severe ↑Halluc ↑
SFT2.8303.0422.4200.354
RFT (Ours)3.1383.4362.7243.942
MethodF1 ↑Prec ↑Rec ↑Rad ↑
SFT0.4750.4800.4720.339
RFT (Ours)0.5030.4900.5170.352

With the Stage 2 held fixed, using boxes predicted by Stage 1 RFT yielded higher scores on all eight metrics than using boxes predicted by Stage 1 SFT. The report localization score increased from 3.042 to 3.436, and CheXbert F1 increased from 0.475 to 0.503, suggesting that crops derived from Stage 1 RFT predictions can improve report generation performance.

Q3. Clarifying the method contribution.

A. Thank you for pointing us to these closely related studies. We cite all four works and revise our Related Works (Reinforcement Fine-Tuning paragraph).

Visual-RFT [1] and MedLoc-R1 [2]. We agree that both studies establish precedents for training visual grounding with GRPO and IoU-based rewards. VIGOR uses the predictions of the learned grounding policy to construct the visual inputs for report generation. Specifically, we crop the regions identified by Stage 1 and feed them alongside the full radiograph to the Stage 2 report generator during both training and inference.

BoxMed-RL [3]. BoxMed-RL and VIGOR use IoU-based RL for spatial grounding. After learning spatial grounding, BoxMed-RL freezes the backbone and trains a report-generation adapter on reference reports. VIGOR uses the bounding boxes predicted in Stage 1 to crop the radiograph and supplies those crops alongside the full image to the Stage 2 report generator. Then, stage 2 combines LLM-based reward with a disease-label consistency reward, enabling RL training on cases without reference findings.

RadVLM-GRPO [4]. RadVLM-GRPO and VIGOR apply GRPO to visual grounding and report generation. RadVLM-GRPO optimizes the two tasks separately and uses reference-based RadCliQ as its main report-generation reward. VIGOR directly connects the two tasks by using the predicted bounding boxes to crop localized regions and feeding those crops alongside the full radiograph to the report generator. Stage 2 combines LLM-based reward with a disease-label consistency reward, allowing the report generator to be optimized even when reference findings are unavailable.

Overall, VIGOR uses localized image evidence to guide report generation and enables reinforcement learning even when reference findings are unavailable.

References

[1] “Visual-rft: Visual reinforcement fine-tuning.”, ICCV 2025.

[2] “MedLoc-R1: Performance-Aware Curriculum Reward Scheduling for GRPO-Based Medical Visual Grounding.”, CVPR 2026.

[3] “Reason Like a Radiologist: Chain-of-Thought and Reinforcement Learning for Verifiable Report Generation.”, Medical Image Analysis 2026.

[4] “RadVLM-GRPO: Enhancing Chest X-ray Report Generation and Visual Grounding via Reinforcement Learning.”, MIDL 2026.

Reviewer_5YvyOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a two-stage RL framework for chest X-ray report generation using MedGemma-4B. In the first stage, the model is trained to localize abnormalities using IoU-based rewards. In the second stage, the predicted regions are cropped and used along with the original image to generate reports, with disease-label and LLM-based rewards. The paper evaluates the method using localization and report generation metrics, and also conducts a small blinded clinical evaluation.

Critical strengths
  1. I like the idea of using an IoU-based reward to directly optimize abnormality localization. The paper presented in Table 1 that the mean IoU improves from 0.290 with SFT to 0.406 with RL. I also find it useful that the localized regions are provided as additional inputs during report generation, rather than relying only on the full image.

  2. The cross-judge evaluation technique looks promising. In Table 3, the paper showed improvements with Gemini-2.5-Flash and GPT-5, even though Gemini-2.5-Pro was used during the training phase. This distinction helps address the concern that the model may be optimizing only for the preferences of the training judge.

Critical weaknesses
  1. The paper mainly uses CheXbert F1 and LLM-based scores, which are closely related to the training rewards. It is unclear whether the improvements reflect better clinical accuracy. I think including recent metrics such as GREEN, CRIMSON, and ICARE would be useful, as these have shown better agreement with clinician evaluations.

  2. Figure 3 shows that VIGOR is preferred in 48% of cases, compared to 6% for MedGemma-4B. However, 46% of cases are ties. Although the results favor VIGOR, the high tie rate raises questions about the extent of clinical improvement. The evaluation is also limited to 100 samples rated by a single physician. I believe including more clinicians and reporting inter-rater agreement and statistical significance would make these results more convincing.

  3. Stage 2 introduces RL and the cropped regions at the same time, so it is unclear which of the two drives the improvement. The SFT baseline in Table 2 does not help here, since it is trained without the cropped regions. I think adding an SFT baseline trained on the same data with the cropped regions would make the contribution of RL clearer.

Questions for authors

I would be open to raising my rating if the authors can adequately address the concerns mentioned above.

3 comments

W1. Additional recent metrics (GREEN, CRIMSON, ICARE).

A. Thank you for your suggestion. Please see the below table and Appendix Table. 5 of our revised manuscript. To further assess clinical accuracy beyond metrics related to the training rewards, we additionally evaluate VIGOR using CRIMSON [1], GREEN [2], and ICARE [3]. Notably, VIGOR consistently outperforms both MedGemma-4B and MedGemma-4B + SFT across all four metrics, achieving the highest scores on both ICARE-GT [3] and ICARE-GEN [3] among all evaluated models. VIGOR also demonstrates competitive performance on CRIMSON [1] and GREEN [2], which assess clinically relevant errors in generated reports. These consistent improvements across different evaluation frameworks suggest that VIGOR's gains are not merely driven by optimization toward the training rewards, but also reflect improved clinical fidelity in generated reports.

Table 2. Evaluation Using CRIMSON [1], GREEN [2], and ICARE [3] Metrics (↑)

MethodCRIMSON ↑GREEN ↑ICARE‑GT ↑ICARE‑GEN ↑
Closed Models
Gemini 2.5 Flash0.1010.2770.4210.504
Gemini 2.5 Pro0.1050.2890.4240.510
GPT-50.1070.3090.4390.522
Open Models
LLaVA-1.5-7B-0.0830.0300.2470.238
LLaVA-Med-1.5-7B0.0340.0450.2840.408
MAIRA-20.1010.2470.4060.511
Qwen3-VL-4B-Instruct-0.1540.1030.3300.370
HuluMed-4B0.0890.2780.4160.507
HuluMed-7B0.1140.2970.4280.526
Lingshu-7B0.0910.3060.4400.520
Med-Gemma-VL-4B0.1010.2610.4250.529
Med-Gemma-VL-4B + SFT0.1040.2890.4270.527
VIGOR (Ours)0.1130.3010.4540.547

References

[1] “CRIMSON: A Clinically-Grounded LLM-Based Metric for Generative Radiology Report Evaluation.”, MICCAI 2026.

[2] “GREEN: Generative Radiology Report Evaluation and Error Notation.”, EMNLP 2024.

[3] “Clinically Grounded Agent-based Report Evaluation: An Interpretable Metric for Radiology Report Generation.”, 2025.

W2. Including more clinicians and reporting inter-rater agreement and statistical significance would make the results more convincing.

A. Thank you for your suggestion. Please see the below table and Appendix Table. 7 of our revised manuscript. We expand our clinical evaluation (Fig.3) by including two additional physicians, with all three independently assessing the same 100 cases. In below table, VIGOR is preferred in 65.0% of evaluations, compared to 9.0% for MedGemma-4B, with 26.0% ties. Furthermore, all three physicians agree in 45 out of 100 cases (45%), including 41 cases where all three prefer VIGOR and 4 cases where all three report a tie. Notably, there are no cases where all three physicians unanimously prefer MedGemma-4B. Inter-rater agreement is assessed using Fleiss' kappa (kappa = 0.223). A two-sided exact binomial test on majority-vote preferences, excluding ties and cases without a majority (65 for VIGOR versus 7 for MedGemma-4B), shows a statistically significant preference for VIGOR (p < 0.001). These additional results provide further evidence supporting the clinical preference for VIGOR.

Table 3.Clinical evaluation by three board-certified physicians.

EvaluatorVIGOR PreferredTieMedGemma-4B Preferred
Physician 168%24%8%
Physician 279%8%13%
Physician 348%46%6%
Overall65.0%26.0%9.0%

W3. Adding an SFT baseline trained on the same data and cropped regions would make the contribution of RL clearer.

A. Thank you for the suggestion. Please see "general response" and below table. (Appendix Table. 6)

Table 1. Comparison of SFT and RL under identical training data and cropped regions.

MethodDisease ↑Local ↑Severe ↑Halluc ↑
Med-Gemma-VL-4B + SFT (Same training data and cropped regions)2.7722.5302.0172.663
VIGOR (Ours)3.2363.3382.7273.886
MethodF1 ↑Precision ↑Recall ↑RadGraph-XL ↑
Med-Gemma-VL-4B + SFT (Same training data and cropped regions)0.4730.4780.4700.237
VIGOR (Ours)0.5060.5050.5070.349

To clarify the contribution of reinforcement learning, we compare VIGOR with an SFT baseline using the same training data (MIMIC-CXR and Open-I, FINDINGS is available) identical cropped regions. With the cropped-region inputs held fixed, VIGOR outperforms the SFT baseline across all eight metrics. In particular, its gains in localization accuracy (2.530 to 3.338) and hallucination avoidance (2.663 to 3.886) suggest that reward-guided optimization helps the model describe the anatomical locations of findings more accurately and avoid unsupported findings given the same visual evidence.

Reviewer_5YvyOpenReview ↗

Response to the rebuttal: rating raised to 3

Thank you for the detailed responses. My main concerns have been addressed, and I have therefore increased my score. One remaining concern is the relatively low inter-rater agreement (κ = 0.223). It would be useful to briefly discuss this in the paper.

We would like to thank Reviewer 5Yvy for the positive and constructive comments. We are glad that our responses addressed the reviewer’s concerns. We agree that the low inter-rater agreement (Fleiss’ κ = 0.223) warrants discussion. The proportion of cases rated as ties ranged from 8% to 46% across physicians. The three physicians sometimes differed in their judgments of the same cases, so the overall preference for VIGOR should be interpreted with this variability in mind. We have added a brief discussion of this limitation to the Appendix Sec.A of the revised manuscript.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

We thank the reviewers for their constructive, positive, and thorough reviews. We are happy that the reviewers think that our paper is easy to understand (5Yvy, oX9J, dcZ3), well-grounded (oX9J,dcZ3), addresses an important limitation (oX9J), and is well-motivated (dcZ3). For point-to-point response, please see our responses below.

Here, we address the shared feedback on the “Adding an SFT baseline trained on the same data and cropped regions would make the contribution of RL clearer" raised by reviewers 5Yvy and oX9J.

Table 1. Comparison of SFT and RL under identical training data and cropped regions. LLM-as-a-Judge evaluation and clinical metrics (↑).

MethodDisease ↑Local ↑Severe ↑Halluc ↑
Med-Gemma-VL-4B + SFT (Same training data and cropped regions)2.7722.5302.0172.663
VIGOR (Ours)3.2363.3382.7273.886
MethodF1 ↑Precision ↑Recall ↑RadGraph-XL ↑
Med-Gemma-VL-4B + SFT (Same training data and cropped regions)0.4730.4780.4700.237
VIGOR (Ours)0.5060.5050.5070.349

To clarify the contribution of reinforcement learning, we compare VIGOR with an SFT baseline using the same training data (MIMIC-CXR and Open-I, FINDINGS is available) identical cropped regions. With the cropped-region inputs held fixed, VIGOR outperforms the SFT baseline across all eight metrics. In particular, its gains in localization accuracy (2.530 to 3.338) and hallucination avoidance (2.663 to 3.886) suggest that reward-guided optimization helps the model describe the anatomical locations of findings more accurately and avoid unsupported findings given the same visual evidence.

Dear Reviewers,

The authors have responded to your original comments. Could you please take a look and indicate whether the response resolves the concerns, and whether your score changes? Thank you very much.

Best,
Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.