- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
In this paper, the authors propose ClinOPD, an on-policy self-distillation framework for radiology report generation using medical multimodal LLMs. It introduces clinical hints identifying organs with positive findings to guide a teacher model, whose behavior is distilled into a hint-free student using KL-based objectives. The method does not require hints at inference. Authors evaluated ClinOPD on IU-Xray and CT-RATE against SFT, GRPO, DAPO, and OPSD baselines. ClinOPD improves clinical F1 scores over the baselines, with additional experiments validating the proposed metric and analyzing the effects of regularization and clinical hints.
Critical strengths
-
Well-motivated approach: The paper introduces a simple and effective self-distillation framework that does not require clinical hints at inference.
-
Strong empirical evidence: Clinical hints improve F1 by 26.4 and 28.1 points on IU-Xray and CT-RATE, respectively, using the same checkpoint.
-
Detailed analysis: Additional experiments demonstrate the importance of organ-specific hints, and ablations highlight the role of regularization.
Critical weaknesses
-
Tables 1–3 report results from single runs without standard deviations or confidence intervals. The small improvement over GRPO on IU-Xray (+0.62 F1) may not be statistically significant.
-
The evaluation relies entirely on Clin-F1, an LLM-based metric. Established clinical metrics such as RadGraph-F1, CheXbert-F1, and GREEN are not reported.
-
ClinOPD recovers only approximately 19% and 23% of the hint-conditioned improvements on IU-Xray and CT-RATE, respectively. This limits the strength of the claimed distillation benefits.
-
The paper does not sufficiently evaluate against existing trained radiology report generation systems. A standard image-only SFT baseline is also missing, making it difficult to assess the benefits of mixed-SFT training.
Questions for authors
-
What are the mean ± standard deviation results over multiple seeds for all methods, and what are the test-set sizes and checkpoint selection criteria?
-
What proportion of test cases are normal, and how does a constant normal-report baseline perform under Clin-F1?
-
How does ClinOPD perform under established clinical metrics such as RadGraph-F1, CheXbert-F1, or GREEN? Are the method rankings consistent across these metrics?
-
How does mixed-SFT compare against standard image-only SFT, and were the competing methods tuned under comparable settings?