acceptodds
Desk rejected at ICLR 2027

RRG-MLVA: Radiology Report Generation with Multi-Level Vision–Language Alignment

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Automated radiology report generation can reduce clinical workload, yet existing methods often struggle with fine-grained vision–language alignment and limited multi-level semantic reasoning. We propose RRG-MLVA, a framework that couples a Swin Transformer encoder and a LoRA-adapted LLaMA-2 decoder with alignment objectives operating at three semantic levels: token-level alignment for fine-grained grounding, multi-scale prototype alignment for hierarchical disease modelling, and disease-level alignment for diagnostic consistency, together with a global contrastive objective. At the core of the framework is a multi-scale Banzhaf formulation that treats learnable disease prototypes as players in a cooperative game and quantifies each prototype's contribution as its expected marginal contribution over sampled coalitions. Because the underlying coalition value function is non-additive, these contributions do not reduce to the individual prototype similarities, and the resulting distributions provide a principled signal for aligning visual and textual evidence. Prototypes are maintained at coarse (7), intermediate (14) and fine (28) granularities, allowing disease semantics to be matched across levels of specificity. Experiments on CheXpert Plus and IU-Xray evaluate the framework with standard language-generation metrics alongside CheXbert and RadGraph clinical-efficacy measures, with ablations isolating the contribution of each objective and of the prototype granularity. Parameter-efficient LoRA fine-tuning keeps the number of trainable parameters small. Results indicate that multi-level alignment improves report accuracy, coherence, and clinical relevance.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Radiology Report Generation, Agent-Based Learning, Large Language Models, Chest X-Ray

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_WjycOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents a framework for radiology report generation. The authors introduce four training-only alignment losses: global contrastive alignment, token-level alignment, multi-scale prototype alignment using Banzhaf values, and disease-level alignment using CheXpert labels. The proposed framework builds on a Swin and LLaMA-2 architecture and aims to improve visual-textual alignment without introducing additional inference overhead. The authors evaluate their method on IU-Xray and CheXpert Plus using NLG and clinical metrics. They also conduct ablation studies to assess the contributions of individual alignment losses and different prototype scales.

Critical strengths
  1. The alignment module is used only in training, so inference cost and architecture are identical to the base generator. This makes the idea easy to adopt.

  2. Scoring prototypes by Banzhaf values under a non-additive (smooth-max) worth is a sensible way to stop redundant prototypes from double-counting, and the Monte Carlo estimator is unbiased and cheap for at most 28 players.

  3. The method is detailed with all hyperparameters, initializations, decoding and checkpoint-selection rule. Also, both language and clinical metrics are reported with ablations on two datasets.

  4. The conclusion clearly mentions the limitations of this work(single run, non-standard split, in-batch negatives from 4 samples, noisy rule-based labels).

Critical weaknesses
  1. Missing baseline comparison: The paper does not report results for the base model without alignment losses, despite mentioning it in Section 4.1. This makes it difficult to assess the overall contribution of the proposed alignment framework.

  2. Inconsistent dataset splitting: The IU-Xray splitting strategy is described inconsistently across sections. The image-level split also raises concerns about potential data leakage, particularly given the unusually high BLEU-4 score compared to prior work.

  3. Inconsistent methodology: Figure 1 introduces components such as Stackelberg and Shapley games and temporal consistency that are absent from the methodology section. This creates confusion about the actual proposed framework.

  4. Lack of statistical validation: All experiments are reported using a single random seed, without standard deviations or confidence intervals. Consequently, the reliability of the reported improvements remains unclear.

  5. Insufficient dataset details: The CheXpert Plus subset is not adequately described in terms of size, selection criteria, or report sections. The lack of comparable baselines further limits the evaluation.

  6. Limited validation of novelty: The ablation studies do not isolate the contribution of Banzhaf-based prototype scoring against simpler alternatives. Similarly, the token-level alignment resembles existing softmax-based approaches, leaving its methodological novelty unclear.

Questions for authors
  1. What are the results of Base (all alignment weights zero) on both datasets under the Table 3 settings?
  2. How was the IU-Xray split constructed? Which statement is correct, Sec. 4.1 (standard split) or Table 1 (image-level split)? Can two images from the same study fall into different splits?
  3. Why are B-3 and B-4 on IU-Xray almost equal (0.366 / 0.340), when every published method in Table 1 has B-4 well below B-3?
  4. What is the CheXpert Plus subset (number of studies, how selected), and which report section is generated on each dataset?
  5. Why does Figure 1 show section-level, Shapley disease-level and temporal-consistency components that Sec. 3 does not describe, and what is "RRG MLGT" in Sec. 4.1?
  6. Why use a keyword-based extractor for the disease labels when CheXpert Plus already provides CheXpert labels (as stated in Sec. 4.1)?
  7. What happens if the Banzhaf values are replaced by plain softmax prototype similarities (MGCA-style) with everything else fixed?
Reviewer_WgZaOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

RRG-MLVA is a VLM with a Swin Transformer encoder and a LoRA-adapted LLaMA-2. A set of alignment objectives drive the training of RRG-MLVA. First fine-grained grounding then multi-scale prototype followed by disease-level alignment, and finally a global contrastive objective. The learnable disease prototypes are treated as players in a cooperative game model, and an approximated Banzhaf formulation quantifies each prototype's contribution as its expected marginal contribution over sampled coalitions. Experiments are reported on CheXpert Plus and IU-Xray.

Critical strengths

None

Critical weaknesses

This paper report results in a non standard split of CheXpert Plus, and evaluated in a set of IU-Xray which is incompatible with the baselines. None of the results outlined in tables 1,2 and 3 represent a direct and fair comparison with any of the baseline methods.This poor experimental design invalidates any empirical claim established in this paper.

Without a correct baseline comparison, it is not possible to draw any conclusions. There are a few interesting elements in the proposed method, but without empirical evidence of their usefulness or limitations, they remain strictly rhetoric.

Questions for authors

My assessment won't change unless direct and fair comparison against the state of the art is provided. Moreover, a full ablation experiment with hyper-parameter sensitivity must also be performed in the same datasets. Once the correct empirical validation is presented I can elaborate meaningful questions about this paper.

Reviewer_kW3hOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes RRG-MLVA, a radiology report generation framework combining a LoRA-adapted Swin Transformer encoder and a LoRA-adapted LLaMA-2-7B decoder connected via a gated visual projection. To address the limitation of implicit vision-language alignment in next-token prediction, the authors introduce four auxiliary, training-only objectives: (1) a global contrastive objective using symmetric InfoNCE, (2) a token-level payoff alignment modeling token responses via quantal response / late interaction, (3) a multi-scale prototype alignment evaluating prototype contributions using Banzhaf values over a smooth-maximum coalition game, and (4) a disease-level alignment supervised by 14 CheXpert label categories extracted via rule-based text matching. Experiments are conducted on IU-Xray (using an image-level split) and an unspecified subset of CheXpert Plus, evaluating standard language generation metrics (BLEU, ROUGE-L, CIDEr) and clinical efficacy metrics (CheXbert-14 F1, RadGraph F1).

Critical strengths

Nothing in particular. Just the general intention of incorporating training-only multi-granularity alignment objectives without adding inference overhead.

Critical weaknesses

There is a severe disconnect between the central architectural diagram and the methodological formulation. Figure 1 depicts a 5-step framework: Step 1 (Ternary Nash Game), Step 2 (Banzhaf Game), Step 3 (Section-level Stackelberg Game), Step 4 (Disease-Level Shapley Game), and Step 5 (Temporal Consistency). In contrast, Section 3 formulates 4 completely different objectives: Global Contrastive, Token-Level Payoff, Multi-Scale Banzhaf Prototypes, and Disease-Level MSE Distillation. It is not clear how they reconcile.

The Stackelberg Game (Step 3) and Temporal Consistency (Step 5) shown in Figure 1 are never defined, formulated mathematically, trained, or ablated anywhere in the text. Furthermore, the disease-level objective in Section 3.3.4 is merely an MSE loss against rule-based labels (it contains no game theory, coalitions, or Shapley values whatsoever, directly contradicting the "Shapley Game" claim in Figure 1.)

In Section 4.1 (line 351), the authors refer to their model as "RRG MLGT" instead of "RRG-MLVA". This and the above mentioned discrepancies suggests that the submission combines some unvetted, conflicting pieces of different drafts without internal coherence.

Concerning experiments:

  1. The authors state in Section 4.2 (lines 373–375, 379–381) that they evaluated on an image-level split rather than the benchmark-standard patient- or study-level split. In chest radiography, different images from the same patient study (e.g., frontal and lateral projections) exhibit nearly identical pathology and anatomy. Splitting at the image level inevitably places images from the same patient across both training and test sets, resulting in severe data leakage. This completely invalidates the comparisons against prior work in Table 1 (where the reported BLEU-4 of 0.340 is an artifact of test leakage).

  2. The authors evaluate on a "CheXpert Plus" dataset without defining the sample size, subset selection criteria, patient distribution, or filtering methodology. Moreover, zero comparative baseline models are trained or evaluated on this subset, rendering the reported numbers uncontextualized.

  3. In Table 3, the prototype scale ablations ("One scale" and "Two scales") for IU-Xray are left completely blank without explanation.

  4. The contrastive objectives are trained with micro-batches B=4, which provide only 3 in-batch negatives. Contrastive InfoNCE estimation with 3 negative pairs is statistically degenerate, highly prone to variance, and insufficient for learning robust representations.

Morever, the paper frames standard vision-language operations in game-theoretic terminology without a theoretical justificsation or shown algorithmic benefits.

Questions for authors
  • Why does Figure 1 illustrate a 5-step framework containing a Section-level Stackelberg game, a Disease-level Shapley game, and a Temporal Consistency objective, while Section 3 formulates 4 completely different losses that omit these steps entirely? Where in the codebase and text are Steps 3 and 5 implemented and evaluated?

-Why was an image-level split adopted instead of the established patient-level split? How does RRG-MLVA perform on the standard, official patient-level splits of IU-Xray where patient leakage between train and test sets is strictly prevented?

  • What exact cohort definition and selection criteria were used to form the subset CheXpert Plus? How many studies and patients are included, and why were existing state-of-the-art baselines not evaluated on this exact benchmark?

  • Why are the multi-scale prototype ablation rows for IU-Xray left blank in Table 3?

  • What is the difference between the "Token-Level Payoff Alignment" (Section 3.3.2) and the standard temperature-scaled cross-attention / late-interaction matching?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.