acceptodds
Under review as a conference paper at ICLR 2027

ReCER: Recurrent Temporal Relation Refinement for Longitudinal Radiology Report Generation

Abstract

Longitudinal radiology report generation (LRRG) aims to describe both current findings and their temporal evolution relative to prior examinations. When subsequent finding-specific interpretation relies only on a fixed temporal summary, local cues underrepresented during initial aggregation may remain difficult to incorporate. This motivates refining the interpretation of each prior finding against retained local current–prior comparisons and using the resulting representation to guide both temporal description and historical reuse. To this end, we propose Recurrent Experts for Controlled Evidence Reuse (ReCER), a relation-refinement framework that preserves local current–prior comparison evidence and adaptively refines finding-specific temporal representations, enabling historical reuse to draw on local cues underrepresented in fixed summaries. For each previously reported positive finding, prior-report semantics guide the reading of current–prior visual relations, grounding its reinterpretation in evidence from both examinations. Recurrent relation experts then refine finding-specific temporal representations through state-guided attention, allowing each updated state to guide subsequent evidence aggregation. The refined representations are used to characterize temporal changes, construct temporal content, and derive finding-specific usage scores that control this content's contribution during report generation. ReCER also preserves an independent current-image pathway and accounts for newly emerging findings through a separate branch, so that report generation is not restricted to previously reported positive findings. Experiments on MIMIC-CXR, CheXpert Plus, and IU X-Ray demonstrate the effectiveness of ReCER across longitudinal and current-image-only settings. Our code is available at the anonymous link https://anonymous.4open.science/r/ReCERcode-6DB5/.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Longitudinal radiology report generation, Temporal representation refinement, Recurrent relation experts, Controlled evidence reuse, Local visual representations

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_GdX6OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes ReCER for longitudinal radiology report generation. Recurrent experts refine representations of individual findings by repeatedly reading local relations between current and prior images. These representations determine temporal content and its contribution during decoding. Experiments on MIMIC-CXR, CheXpert Plus, and IU X-Ray evaluate report quality and component contributions.

Critical strengths
  1. ReCER connects recurrent refinement of individual findings with controlled historical reuse. Updated finding representations guide repeated attention to local temporal relations and subsequently determine both temporal content and its contribution during decoding. This coupling is a meaningful architectural distinction from using progression predictions solely as generation guidance.
Critical weaknesses
  1. The contribution beyond established mechanisms is insufficiently demonstrated. Diff-RRG already performs disease-specific local temporal comparison, while RECAP and DDaTR control the incorporation of historical information. Iterative attention also provides an established mechanism for rereading evidence. ReCER may offer additional value through their combination, but Appendix D.1 simultaneously removes expert MLPs, state projections, branch identities, and recurrent updates. The CE-F1 difference of 0.515 versus 0.523 therefore does not isolate the benefit of rereading local evidence. The paper provides neither a control with equal depth that reuses the initial readout nor an isolated ablation of OLD usage weighting.

  2. The longitudinal evaluation remains inconclusive. Appendix F acknowledges that the reported metrics do not establish the correctness of temporal direction, emergence, or resolution. The CheXpert Plus input protocol is also unclear. The method describes aggregation across study images, while Appendix A.1 retains benchmark partitions whose released annotation can assign different views of the same study to different splits. Assembling complete studies could therefore introduce images from another split. The paper does not explain how these assignments are reconciled, leaving this concern unresolved.

Questions for authors
  1. Does ReCER outperform a control with equal depth that reuses the initial relation readout? What additional benefit does OLD usage weighting provide when temporal content construction is retained?

  2. Does ReCER use multiple views on CheXpert Plus? If so, how are studies assembled when their views belong to different benchmark splits, and do the resulting training and test inputs share any images?

  3. How are transition and NEW labels constructed and validated against the selected prior study? Is there evidence that ReCER improves the correctness of temporal direction, emergence, and resolution beyond overall report similarity and clinical label agreement?

Reviewer_epxnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes ReCER for longitudinal radiology report generation. For each positive finding in the prior report, the model compares current and prior images and iteratively refines its estimate of how the finding has changed, and uses this to control historical reuse during generation. The authors also introduce a visual front end built on a pretrained encoder.

Critical strengths
  1. Reasonable motivation. A finding in the prior report may persist, change, or resolve in the current examination. The paper assesses the change of each prior finding individually and then decides how to use it, which is reasonable.
  2. Broad experimental coverage. The method is evaluated on two longitudinal datasets and one dataset without prior examinations.
Critical weaknesses
  1. The core claim is not directly validated. The paper's core idea is to assess the change of each prior finding individually and use this to control historical reuse, but the experiments do not directly verify this. (1) Insufficient ablations. All ablation variants retain per-finding processing, including "w/o experts", which still reads the image differences separately for each prior finding and only removes the multiple branches and iterative updates. Comparisons against using the history as a whole, or not using it at all, are missing. (2) No targeted evaluation. The paper does not evaluate whether the temporal descriptions in the generated reports are correct, nor does it report the accuracy of the transition head.
  2. Lack of a unified protocol in comparisons with existing methods. The baseline results in Table 1 are taken directly from the original papers, and the authors acknowledge that the evaluation cohorts, label extraction, and CE averaging differ from those of the baselines, which limits direct comparison with existing methods. Several recent longitudinal methods are evaluated on Longitudinal-MIMIC (e.g., TIM (CVPR 2026), MARE (AAAI 2026), BiOTPrompt (CVPR 2026)); additional experiments under this setting, or re-running the main baselines under the same protocol, would be helpful.
Questions for authors
  1. Relation between VRE and the core claim. On both longitudinal datasets, removing VRE causes a larger CE-F1 drop than removing the experts (MIMIC-CXR: 0.012 vs. 0.008; CheXpert Plus: 0.026 vs. 0.013), and VRE is also effective on IU X-Ray, which has no prior examinations. Does this suggest that the improvement mainly comes from better recognition of the current image rather than from finding-level change modeling?
  2. Motivation for multiple experts and recurrent reading. The paper does not explain why multiple experts are needed, and the authors acknowledge that the experts have no explicitly assigned roles. For recurrent reading, there is no evidence that later rounds read different evidence. What are the motivation and actual effect of these two designs?
  3. The NEW decision uses only globally averaged features, without the local current-prior relations emphasized in the paper. Why is it designed this way?
  4. The paper does not analyze what the usage score g contributes to controlled reuse. Could the authors report results with g fixed to 1 for OLD records, or show the distribution of g across transition categories?
  5. What is the "w/o Stage 1 & Stage 2" variant in Figure 4? It has no corresponding result in Table 2.
Reviewer_1t1GOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how historical information should be utilized in longitudinal radiology report generation, particularly how changes between current and previous examinations can guide the reuse of previously reported findings. The authors propose ReCER, a framework that models local visual relations between current and prior examinations and iteratively refines finding-specific temporal representations through Recurrent Relation Experts. The resulting representations are used to characterize temporal changes and control the contribution of historical information during report generation. The framework also maintains an independent current-image pathway and includes a separate branch for newly emerging findings. Experiments on MIMIC-CXR, CheXpert Plus, and IU X-Ray demonstrate improvements in several text generation and clinical evaluation metrics.

Critical strengths
  1. this paper is that it addresses an important problem in longitudinal radiology report generation. Unlike conventional medical image report generation, longitudinal reporting requires not only describing current findings but also identifying whether previously observed abnormalities persist, improve, worsen, or resolve. Existing methods generally incorporate historical information through feature fusion or image difference modeling. However, directly reusing information from previous reports may cause resolved abnormalities to be incorrectly mentioned in current reports. The authors approach this problem from the perspective of finding-level historical evidence reuse and attempt to connect temporal interpretation with report generation. This is a meaningful research direction with clear clinical relevance.

  2. the proposed framework considers different types of temporal changes. ReCER preserves local comparison features between current and previous examinations, allowing finding-specific representations to repeatedly access relevant visual evidence instead of relying only on a fixed initial summary. This design enables the model to adjust its interpretation of individual findings through recurrent refinement. In addition, the authors separately handle previously reported findings and newly emerging findings while maintaining an independent pathway for current-image information. This structure may help reduce excessive dependence on historical reports and is reasonably aligned with the requirements of longitudinal report generation.

  3. The paper provides a relatively extensive experimental evaluation and ablation analysis. The authors evaluate ReCER on MIMIC-CXR, CheXpert Plus, and IU X-Ray, covering both longitudinal and current-image-only settings. On MIMIC-CXR, ReCER achieves a CE-F1 score of 0.523, outperforming the reported results of PriorRG and MLRG. Table 2 further investigates the effects of removing or replacing the expert module, visual adaptation components, and NEW branch, with the complete model achieving the best overall performance. In addition, the qualitative cases in Figure 4, including newly emerging pleural effusion and resolved pulmonary edema, illustrate the potential benefits of incorporating temporal information into report generation.

Critical weaknesses
  1. the evaluation metrics do not directly measure the main capability that the paper aims to improve. The central contribution of ReCER is to model temporal changes in individual findings, including stable, improved, worsened, and resolved conditions. However, the primary experiments still rely on conventional report generation metrics, such as BLEU, METEOR, ROUGE-L, and CE-F1. These metrics measure textual similarity or agreement on clinical labels, but they do not directly evaluate whether temporal changes are correctly described. For example, a model may correctly identify a particular abnormality but incorrectly describe a resolved condition as persistent. Such an error may not be adequately reflected by conventional clinical label metrics. The authors also acknowledge in the appendix that the current evaluation does not independently establish the correctness of temporal change predictions. This limitation directly affects the validation of the paper's central contribution.

  2. the reported performance improvements are relatively small, while some cross-paper comparisons involve differences in experimental protocols. On MIMIC-CXR, ReCER achieves a CE-F1 of 0.523 compared with 0.511 for PriorRG, which represents a modest improvement. In Table 2, replacing the expert module reduces CE-F1 from 0.523 to 0.515, indicating that the additional benefit of the recurrent expert design is relatively limited. Furthermore, the authors acknowledge that different studies may use different dataset splits, label extraction procedures, and micro- or macro-averaging strategies. Therefore, the results in Table 1 should not necessarily be interpreted as fully controlled comparisons. Without sufficient results across multiple random seeds or statistical significance analysis, it is difficult to determine whether the improvements are stable.

  3. the relatively high complexity of the proposed framework and the need to better justify its novelty. ReCER combines several components, including local spatial adaptation, contralateral reference modulation, frozen semantic refinement, finding selection, recurrent relation experts, and historical evidence usage weighting. Many of these operations are related to existing attention, feature fusion, and iterative refinement techniques. Although the combined framework achieves some improvements, the current experiments do not sufficiently demonstrate that repeatedly reading local comparison features through recurrent experts is essential for the reported performance gains. Since the model also needs to process current images, prior images, and previous reports, further analysis of parameter counts, inference costs, and the contributions of individual components would help clarify the novelty and practical value of the proposed approach.

Questions for authors

Could the authors provide direct evaluation metrics for temporal states, such as stable, improved, worsened, resolved, and newly emerging findings? Classification accuracy, F1 scores, or expert clinical assessment would help determine whether ReCER genuinely improves temporal understanding rather than only conventional report generation performance.

Could the authors reproduce major baselines, such as PriorRG, using identical dataset splits and evaluation protocols, and report results across multiple random seeds? In addition, would replacing Recurrent Relation Experts with a standard multi-layer attention module of comparable parameter size achieve similar performance?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.