- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper presents a framework for radiology report generation. The authors introduce four training-only alignment losses: global contrastive alignment, token-level alignment, multi-scale prototype alignment using Banzhaf values, and disease-level alignment using CheXpert labels. The proposed framework builds on a Swin and LLaMA-2 architecture and aims to improve visual-textual alignment without introducing additional inference overhead. The authors evaluate their method on IU-Xray and CheXpert Plus using NLG and clinical metrics. They also conduct ablation studies to assess the contributions of individual alignment losses and different prototype scales.
Critical strengths
-
The alignment module is used only in training, so inference cost and architecture are identical to the base generator. This makes the idea easy to adopt.
-
Scoring prototypes by Banzhaf values under a non-additive (smooth-max) worth is a sensible way to stop redundant prototypes from double-counting, and the Monte Carlo estimator is unbiased and cheap for at most 28 players.
-
The method is detailed with all hyperparameters, initializations, decoding and checkpoint-selection rule. Also, both language and clinical metrics are reported with ablations on two datasets.
-
The conclusion clearly mentions the limitations of this work(single run, non-standard split, in-batch negatives from 4 samples, noisy rule-based labels).
Critical weaknesses
-
Missing baseline comparison: The paper does not report results for the base model without alignment losses, despite mentioning it in Section 4.1. This makes it difficult to assess the overall contribution of the proposed alignment framework.
-
Inconsistent dataset splitting: The IU-Xray splitting strategy is described inconsistently across sections. The image-level split also raises concerns about potential data leakage, particularly given the unusually high BLEU-4 score compared to prior work.
-
Inconsistent methodology: Figure 1 introduces components such as Stackelberg and Shapley games and temporal consistency that are absent from the methodology section. This creates confusion about the actual proposed framework.
-
Lack of statistical validation: All experiments are reported using a single random seed, without standard deviations or confidence intervals. Consequently, the reliability of the reported improvements remains unclear.
-
Insufficient dataset details: The CheXpert Plus subset is not adequately described in terms of size, selection criteria, or report sections. The lack of comparable baselines further limits the evaluation.
-
Limited validation of novelty: The ablation studies do not isolate the contribution of Banzhaf-based prototype scoring against simpler alternatives. Similarly, the token-level alignment resembles existing softmax-based approaches, leaving its methodological novelty unclear.
Questions for authors
- What are the results of Base (all alignment weights zero) on both datasets under the Table 3 settings?
- How was the IU-Xray split constructed? Which statement is correct, Sec. 4.1 (standard split) or Table 1 (image-level split)? Can two images from the same study fall into different splits?
- Why are B-3 and B-4 on IU-Xray almost equal (0.366 / 0.340), when every published method in Table 1 has B-4 well below B-3?
- What is the CheXpert Plus subset (number of studies, how selected), and which report section is generated on each dataset?
- Why does Figure 1 show section-level, Shapley disease-level and temporal-consistency components that Sec. 3 does not describe, and what is "RRG MLGT" in Sec. 4.1?
- Why use a keyword-based extractor for the disease labels when CheXpert Plus already provides CheXpert labels (as stated in Sec. 4.1)?
- What happens if the Banzhaf values are replaced by plain softmax prototype similarities (MGCA-style) with everything else fixed?