- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes RHDP to detect and mitigate reward hacking in chest X-ray report generation. Its central statistic, LVoG, measures the temporal variance of spatially averaged absolute input gradients of the joint cross-entropy and reinforcement-learning loss. Reliability scores derived from LVoG modulate rewards and forward KL regularization. Experiments on IU-Xray and MIMIC-CXR subsets examine report quality, gradient dynamics, and individual component contributions.
Critical strengths
- The paper studies an important problem, since optimizing automated rewards does not necessarily improve clinically grounded report generation. Its decomposition of the joint gradient provides a useful basis for investigating which training signals are associated with reward exploitation.
Critical weaknesses
-
The empirical justification for the composite detector is insufficient. Appendix A.2 claims that joint LVoG has the strongest association with proxy–gold alignment, but Figure 9 reports higher correlations for advantage variance in all three displayed windows. The corresponding values are 0.305, 0.286, and 0.288 for advantage variance, compared with 0.206, 0.226, and 0.221 for LVoG. Although this does not establish which statistic produces a better intervention, the paper provides no advantage-variance-based mitigation baseline to demonstrate the additional benefit of computing input gradients.
-
The controlled ablations do not consistently support the claim in Section 4.4 that the two components jointly improve clinical accuracy. On MIMIC-CXR-10k, the complete method reduces CheXbert F1 from 0.377 to 0.369 and RadGraph F1 from 0.264 to 0.251 relative to Base, which already uses the improved RadCliQ reward. On IU-Xray, the complete method improves over Base but remains below KL-only on both clinical metrics. The trade-off acknowledged in Section 4.3 may explain some of these differences, but the additional value of combining the components remains insufficiently established. Furthermore, the detector evaluation uses the original RadCliQ as the gold reward despite identifying exploitable behavior under that reward, limiting its interpretation as independent evidence of reduced clinically harmful reward hacking.
Questions for authors
-
How should the claimed superiority of joint LVoG be reconciled with Figure 9? Does LVoG-based modulation outperform advantage-variance-based modulation when average reward scaling and KL settings are matched?
-
Given the trade-off acknowledged in Section 4.3, what evidence establishes that combining LVoG and forward KL provides additional value over either component alone? Does combining LVoG and forward KL reduce clinical errors in generated reports compared with using either component alone?
-
The comparison with recent methods appears limited: VSANet (Neural Networks, 2026) is the only 2026 baseline evaluated on IU-Xray, where RHDP trails it on two of the three reported metrics (BLEU-4 and ROUGE-L), and no 2026 baselines are included on MIMIC-CXR-10k. Could the authors clarify the rationale for baseline selection and provide comparisons with recent strong methods under a consistent evaluation protocol, including clinical metrics, to substantiate the claimed clinical-accuracy–fluency trade-off?