- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents VEE-RG, a region-level longitudinal radiology report generation framework that stabilizes anatomical correspondences across visits, factorizes regional changes into exchange-symmetric and exchange-antisymmetric components, and aligns longitudinal visual and textual representations through anonymized contrastive learning; experiments on Chest ImaGenome and MIMIC-CXR report improved clinical efficacy and better stability under visit-order exchange.
Critical strengths
1.The paper addresses a limitation of existing longitudinal report generation methods. Modeling disease evolution at the anatomical-region level is clinically meaningful and potentially more interpretable than relying only on global image representations.
2.The exchange-equivariant formulation is intuitively appealing. Decomposing longitudinal information into a symmetric component that remains unchanged under visit exchange and an antisymmetric component whose sign reverses provides a reasonable inductive bias for separating change existence from change direction.
3.The experimental analysis is relatively comprehensive. The paper evaluates two public datasets and includes structural ablations, auxiliary-task ablations, interventions on the antisymmetric component, region-level change and direction analysis, visit-order exchange experiments, and qualitative attention visualizations. These experiments generally support the contributions of PRS and ECF.
Critical weaknesses
1.The report from the earlier visit is not necessarily a valid reference for the report that should be generated after exchanging the two visits, because it was written before the later examination existed and may compare with another, even earlier study. For example, if the current report states that “pleural effusion has increased,” a correct reversed description should indicate a decrease; however, the original prior report may only state “small pleural effusion” without describing this reversed change. Therefore, I am not fully convinced that the reported Absolute Gap and TRD results directly demonstrate correct temporal-direction reasoning. This is an important evaluation concern that could be addressed by clarifying the construction of reversed references and providing label-based or human evaluation of the predicted change direction.
2.In PRS, a region is considered valid only when it is detected in both visits. If it is missed in either image, the bounding-box prior and ROI feature are disabled for both visits, even when the detection in the other image is correct. I understand that this design preserves symmetry, but it unnecessarily removes useful localized evidence in exactly the difficult cases PRS is intended to handle. Moreover, the paper does not report how frequently the regional boxes and ROI features become unavailable; if this rate is high, the proposed regional comparison would largely fall back to query-conditioned global C5 features, weakening the claimed benefit of explicit regional correspondence. This limitation could be addressed by reporting the failure rate and comparing one-sided masks or confidence-weighted regional features.
3.On MIMIC-CXR, VEE-RG obtains a lower Tempo F1 than PriorRG, while also underperforming it on BLEU-4, METEOR, and ROUGE-L. Its clinical F1 improvement is mainly associated with higher recall, accompanied by lower precision. I therefore find the evidence for improved temporal reporting less convincing than the evidence for increased finding coverage.
Questions for authors
1.The MIMIC-CXR dataset contains AP/PA frontal images as well as lateral-view images, while VEE-RG relies heavily on anatomical bounding boxes and regional correspondence. Since region-grounding models designed for frontal radiographs may be less reliable on lateral views, could the authors clarify whether lateral images were excluded and only AP/PA images were retained? If lateral images were included, how were their bounding boxes and cross-visit regional correspondences handled, and was the effect of view position evaluated? 2.RGRG is an important early baseline for region-level radiology report generation. Why are the RGRG results not reported on the MIMIC-CXR split? Is this due to differences in regional annotations, data splits, or experimental settings?
3 comments
W2: PRS may lose useful unilateral evidence by completely discarding bilateral local features when one side is missed; frequency statistics and a comparison with a unilateral gating strategy should be provided.
Thank you for your analysis of the use of unilateral local evidence. PRS employs bilateral gating to ensure symmetric feature extraction, as mixing detection-guided local features with query-guided global features would confound true pathological changes with asymmetric reading biases. Therefore, when unilateral detections fail, both visits uniformly fall back to shared-query global attention, ensuring unbiased longitudinal comparisons while preserving essential spatial and identity cues.
To clarify the scope of this design, we added detection-availability statistics under the same definition as the gating comparison over all 2,260 pairs:
| Detection status | All 65,540 region slots | 3,880 region slots with explicit improvement/worsening |
|---|---|---|
| Detected bilaterally | 76.70% | 80.85% |
| Detected on one side only | 11.20% | 8.09% |
| Undetected bilaterally | 12.11% | 11.06% |
| Triggering PRS fallback | 23.30% | 19.15% |
Thus, 80.85% of explicitly changed regions still utilize bilateral local evidence rather than the fallback. We also evaluated the suggested unilateral-gating variant—where each visit independently uses its own detection flag for local features or query fallback—under identical parameters, supervision, and training budgets:
| Test scope and metric | Bilateral gating | Unilateral gating | Relative improvement (%) | Patient-level paired 95% CI for absolute improvement |
|---|---|---|---|---|
| Natural unilateral-detection group, State macro-F1 | 0.5658 | 0.5343 | +5.90 | [0.69, 5.64] |
| Natural unilateral-detection group, Direction macro-F1 | 0.7456 | 0.7090 | +5.16 | [−0.03, 7.78] |
“Natural unilateral detection” refers to cases in which the actual detection outcome successfully identifies a specific region on only one side, without any artificial masking intervention. Bilateral gating consistently outperforms the unilateral variant, yielding relative improvements of 5.9% in State macro-F1 and 5.2% in Direction macro-F1. We have added the results and analysis to Appendix A.3 of the paper.
Weakness 3
W3.1: VEE-RG lags behind PriorRG on Tempo F1 and some NLG metrics.
Thank you for your attention to the evaluation metrics. VEE-RG's lower BLEU-4, METEOR, ROUGE, and Tempo F1 scores than PriorRG may be because PriorRG additionally introduces patient-specific clinical-context text, CC (Indication and Medical History), during both training and inference. These texts naturally contain highly condensed diagnostic intent and medical-history clues, providing a textual advantage for report generation. To verify this point, during inference on the MIMIC test set, we removed the clinical-context (CC) input from PriorRG and retained only the current chest radiograph, the most recent prior chest radiograph (when available), and the acquisition position of each image. The results are as follows:
| PriorRG input setting | B-1 | B-4 | MTR | R-L | P-14 | R-14 | F1-14 | Tempo F1 | GFLOPs |
|---|---|---|---|---|---|---|---|---|---|
| VEE-RG (Ours) | 0.412 | 0.116 | 0.164 | 0.275 | 0.508 | 0.606 | 0.553 | 0.268 | 1447.03 |
| PriorRG w/o CC | 0.261 | 0.045 | 0.120 | 0.198 | 0.516 | 0.555 | 0.535 | 0.189 | 1877.62 |
| PriorRG w CC | 0.395 | 0.167 | 0.185 | 0.324 | 0.576 | 0.501 | 0.536 | 0.282 | 2021.21 |
The results show that PriorRG's NLG and Tempo metrics drop substantially without CC, while its CE metrics show increased Recall and decreased Precision, with overall F1 remaining relatively stable. This indicates that PriorRG's lexical overlap and temporal-state prediction rely heavily on the CC input. Indeed, the original ablation study of PriorRG also showed that this additional text can substantially improve NLG metrics. In contrast, VEE-RG does not rely on additional clinical text and has lower training and inference overhead.
In addition, NLG metrics alone cannot verify whether the longitudinal model has learned a positional shortcut—generating temporal descriptions based on input slots rather than performing a genuine visual comparison of anatomical regions. We therefore introduced AG and TRD, which evaluate visit-order swap stability and target-time-point discrimination ability, respectively. We further conducted a swap evaluation on the 1,074 examination pairs shared by PriorRG and VEE-RG. The results are shown below:
| AG ↓ | BLEU-1 | BLEU-4 | F1-14 | F1-5 | T-F1 |
|---|---|---|---|---|---|
| VEE-RG (ours) | 0.840 | 0.419 | 0.245 | 0.494 | 3.621 |
| PriorRG | 5.497 | 6.794 | 9.680 | 6.395 | 7.366 |
| Paired-difference 95% CI (pp) | [−6.034, −3.360] | [−7.271, −5.440] | [−11.770, −5.981] | [−8.645, −2.367] | [−7.207, −0.157] |
| TRD ↑ (%) | P-14 | R-14 | F1-14 | F1-5 | T-F1 |
|---|---|---|---|---|---|
| VEE-RG (Ours) | 30.26 | 34.59 | 39.39 | 33.29 | 44.93 |
| PriorRG | 24.02 | 32.54 | 35.06 | 27.56 | 43.81 |
| Relative improvement (%) | +26.0 | +6.3 | +12.4 | +20.8 | +2.6 |
| 95% CI for absolute improvement (pp) | [4.14, 8.20] | [−0.59, 4.53] | [1.74, 6.85] | [3.29, 8.07] | [−1.59, 3.62] |
Relative TRD improvements use PriorRG as the baseline. All confidence intervals are patient-level paired bootstrap 95% CIs for the signed differences VEE-RG − PriorRG, expressed in percentage points. VEE-RG's forward and swapped F1-14 scores are 0.569 and 0.567, respectively, while PriorRG's are 0.544 and 0.447; the corresponding AG values are 0.245 and 9.680 percentage points. This shows that VEE-RG not only obtains higher clinical accuracy under both input orders, but also exhibits greater swap stability. At the same time, VEE-RG consistently outperforms PriorRG on both CE-TRD and Tempo-TRD. Taken together, these results show that VEE-RG has lower computational cost than PriorRG and provides more stable visit-order swapping and target-time-point discrimination. We have added the clinical and temporal TRD comparison with PriorRG to Table 15.
W3.2: The clinical F1 gain mainly comes from higher Recall and lower Precision.
Thank you for your attention to the clinical metrics. VEE-RG achieves relative improvements in clinical F1 of 3.2% over PriorRG on MIMIC-CXR and 8.2% over DDGIP on Chest ImaGenome. A more granular analysis shows that, under the MIMIC-CXR split, its CE-5 Recall achieves a relative improvement of 32.8% over PriorRG (0.737 vs. 0.555) and a relative improvement of 17.5% over DDGIP (0.737 vs. 0.627). In actual clinical diagnosis, the potential risk of false negatives is usually much greater than that of false positives. Recall measures the coverage of positive clinical labels in the reference reports. This substantial gain reflects the model's ability to retain more clinical findings and reduce omissions at the label level, which is particularly important in clinical practice.
Main Questions and Some Weakness
We thank Reviewer QBEq for the careful review and constructive feedback. We also appreciate your recognition of the clinical relevance of modeling disease evolution at the anatomical-region level, the motivation for our exchange-equivariant formulation, and the experimental evidence supporting the contributions of PRS and ECF. Your comments helped us clarify how the original reports are used in the visit-order exchange evaluation and explain what the clinical and temporal metrics capture. We now clarify and address the weaknesses and questions raised by the reviewer.
Questions
Q1: Are lateral-view images excluded?
Thank you for your attention to image views and the applicability of regional localization. The inputs used in this study are all AP or PA frontal chest radiographs; lateral-view images are not included. We further summarized the static report-generation set under the official MIMIC split:
The longitudinal training, validation, and test sets contain 108,207, 934, and 2,260 examination pairs, respectively, and the images at both endpoints are all AP or PA. Consequently, no frontal anatomical boxes are applied to lateral-view images, and cross-visit comparisons strictly occur between aligned canonical frontal regions.
Q2: Why are RGRG baseline results missing on the MIMIC split?
Thank you for pointing this out. We have now added the complete evaluation results under this setting:
VEE-RG outperforms the baseline model on all metrics. We have added the RGRG results to Tables 1 and 2 of the paper.
Major Weaknesses
W1: Using the original early report, which lacks a description of the reverse comparison, directly as the evaluation reference for the swap experiment may make the AG and TRD metrics difficult to accurately reflect the model's temporal-direction reasoning ability; a label-based or manual direction evaluation should be added.
Thank you for pointing out that the reference context of the early report may differ. After swapping the order of the two examinations, we designate the early examination as the new generation target and directly score against its original report at that time. For example, suppose that the later image indeed shows more pleural effusion than the earlier image. Viewed in the reverse direction, the earlier image shows less pleural effusion than the later image. The phrase “small amount of pleural effusion” in the original early report and the phrase “decreased pleural effusion” generated after the swap can both be valid, because the former describes the state at the early time point, whereas the latter describes the change between the two examinations.
This also clarifies what TRD can answer: it evaluates whether a generated report better matches the correct target time point. While CE solely measures clinical finding overlap, Tempo F1 explicitly captures directional temporal words. The forward-generated “increased” can match “increased” in the later reference, whereas a reasonable swapped-direction expression such as “decreased” has no corresponding word in the early reference when that reference only states “small amount of pleural effusion.” Therefore, the basis for TRD discrimination comes from the content actually recorded in the reference reports. In the Official Comment, we provide further analysis of the TRD evaluation, including additional examples of CE and Tempo F1, to validate the reasonableness of the swap evaluation.
AG evaluates whether the overall scores are similar under the two input orders. On the 1,074 examination pairs shared by the two methods, VEE-RG achieves F1-14 scores of 0.569 and 0.567 in the forward and swapped settings, respectively, whereas PriorRG achieves 0.544 and 0.447. The corresponding AG values are 0.245 and 9.680 percentage points. VEE-RG obtains higher clinical scores under both orders, and its scores are also closer across the two directions.
We further performed a direction-intervention experiment on 1,537 pairs by fixing all inputs and only negating ECF's direction component (O). Using our lexical rules, we matched generated directions against ground-truth change labels. Among the 2,266 region slots with unambiguous labels, the number of direction-consistent regions decreased from 201 to 35, while the number of direction-opposite regions increased from 26 to 245. This shows that changing the model's internal direction signal systematically changes the directional expression in the generated text.