- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper addresses an important problem in longitudinal radiology report generation: how to represent changes between medical imaging visits while preserving uncertainty associated with individual observations and disagreements among historical reports. The authors propose Dist-RRG, a distributional framework in which imaging features, clinical indications, temporal differences, and prior reports are represented using diagonal Gaussian distributions. The paper presents substantial theoretical and experimental material, including Gaussian geometry, an exact variance-preservation result, matched baseline reruns, component ablations, calibration experiments, selective prediction, and a direct intervention on the variance pathway.
Critical strengths
The paper has several clear strengths. Its mathematical treatment of Gaussian aggregation is coherent, its comparison methodology explicitly separates published results from matched reruns, and the variance-shuffling intervention provides evidence that patient-specific variance features contribute to prediction performance.
The paper is technically coherent, presents a well-motivated distributional framework, provides useful analytical properties, conducts matched baseline comparisons, and evaluates the variance pathway more carefully than a standard accuracy-only report-generation study. The authors also acknowledge limitations in the geometric interpretation, automated supervision, and clinical meaning of variance.
Critical weaknesses
The frozen variational encoder produces learned diagonal Gaussian parameters, but their variances are not necessarily calibrated uncertainties about radiographic findings. The paper's experiments demonstrate association with prediction difficulty and usefulness as conditioning features, rather than validated epistemic or aleatoric uncertainty.
The title emphasizes statistical manifolds, but the deployed DPG attention uses a trajectory-dependent reweighting of a Gaussian dissimilarity rather than a single globally defined Fisher–Rao distance. The authors acknowledge this in Sections 3.4 and Appendix B, which is good. The distinction should also be conveyed in the abstract.
The sentence: Most sentences in a chest radiograph report are comparative. is stronger than necessary and is not established by the paper's dataset analysis.
The reported ECE is calculated from the auxiliary progression-classification head, not directly from the probability that a generated sentence or clinical finding is correct. The abstract should call it progression-head calibration rather than leaving readers to infer that it measures calibration of generated reports.
The architecture figure combines Gaussian encoders, report history, distributional differences, attention, mixture aggregation, FiLM, decoder fusion, and the training objective into a compact diagram. The font sizes are small, making it difficult to distinguish the essential mathematical contributions.
The paper correctly warns that published scores and matched scores use potentially different evaluation conventions. However, presenting all CE-F1 numbers in the same columns invites direct comparison. For example, TIM reports 0.511 CE-F1 under its original protocol, whereas Dist-RRG reports 0.509 under the matched P1 setting. These numbers cannot establish which model is better.
The P2 model obtains New F1 of 0.412 and Resolved F1 of 0.730 (Table 3, page 9). This asymmetry matters because detecting new pathology is especially relevant to longitudinal reporting.
Appendix Figure 3 (page 23) provides an example in which using five priors avoids an unsupported assertion of change. The example illustrates the value of additional historical context but does not isolate the effect of variance propagation.
The central contribution is spread across several interacting architectural components, which sometimes makes it difficult to identify what is essential to the method. DDM, DPG, MMDA, RTAM, random-Fourier correction, FiLM, the progression head, the KL penalty, and HSIC regularization all enter the final model. The cumulative presentation is coherent, but the relative scientific importance of these components is not immediately clear. Additionally, the strongest uncertainty experiments are located after the generation benchmark results, despite uncertainty preservation being the paper's main motivating claim.
Equation (6) uses the law of total variance: \hat{\sigma}^2=\sum_t \alpha_t\sigma_t^2+\sum_t\alpha_t(\mu_t-\hat{\mu})^2. The associated forward-KL characterization is a known property of exponential-family projection. The authors acknowledge this, but the novelty should be attributed to the application and end-to-end integration rather than the projection itself.
Proposition 1 connects the proposed Gaussian dissimilarity to a local metric. The Fisher–Rao match requires , whereas experiments use . Although the Euclidean-attention ablation supports the usefulness of distribution-aware scoring, it does not determine whether the proposed geometric formulation is better than simpler variance-normalized distances.
The model includes additional attention, FiLM transformations, latent distributions, and auxiliary losses. Consequently, performance improvement may partly arise from the larger or more expressive temporal pathway.
Questions for authors
Sections 3.6, 4.4, and 5; Appendix Tables 9–12. The paper demonstrates that aggregated latent variance contains useful patient-specific information. For instance, shuffling the variance vectors decreases New F1 from 0.412 to 0.389 and increases ECE from 0.041 to 0.056. However, this intervention modifies only the log-variance input to FiLM while preserving the aggregated means and attention weights. It shows that the decoder uses these variance-conditioned features, but does not demonstrate that their values faithfully represent clinical uncertainty. A learned latent variance could encode image complexity, disease prevalence, observation quality, or other patient characteristics without being a well-calibrated uncertainty estimate.
Sections 3.7 and 4.1–4.5; Appendix E. The progression-consistency head is supervised using CheXbert-derived transition labels, and CheXbert is also used to calculate clinical efficacy and progression-transition metrics. The paper appropriately includes RadGraph-F1 as an additional metric. This reduces dependence on a single extraction system but does not eliminate the problem that both automated metrics judge generated reports against reference-report text rather than independently adjudicated clinical facts. A particularly important detail appears in Appendix E: CheXbert's Negative, Uncertain, and Blank categories are all mapped to absence. Consequently: Uncertain → Positive is classified as New. Positive → Uncertain is classified as Resolved. Negative → Uncertain is classified as Stable. These transitions can represent changes in reporting certainty rather than genuine clinical disease progression.
Sections 4.1–4.3; Tables 1 and 2. The strongest matched longitudinal baseline is Diff-RRG, with CE-F1 0.482 versus 0.509 for Dist-RRG under P1. The paired interval for the improvement is [0.012, 0.041], excluding zero. However, the baseline reruns have only one training seed, and they do not receive equivalent auxiliary CheXbert progression supervision. This makes it difficult to attribute the difference specifically to distributional change modeling. Furthermore, the concurrent TIM model reports CE-F1 0.511 under its own protocol, but is not evaluated using the matched procedure. The authors correctly avoid presenting this as a comparable result.
Section 3.2; Appendix B; Table 8. The distributional difference assumes independent visit-level Gaussian representations: \mathrm{Var}(Z_T-Z_t)=\sigma_T^2+\sigma_t^2. However, longitudinal observations of the same patient are correlated. The more general expression is \mathrm{Var}(Z_T-Z_t)=\sigma_T^2+\sigma_t^2-2\mathrm{Cov}(Z_T,Z_t). The paper recognizes this and provides a sensitivity sweep. In Table 8, increasing assumed cross-visit correlation from 0 to 0.75 decreases CE-F1 from 0.516 to 0.504. This establishes some sensitivity but does not show whether the independence approximation is realistic. Moreover, overstating variance under positive covariance is not necessarily conservative with respect to downstream report decisions.
Section 3.7; Appendix D, Equation (12). Under P2, the temporal representation aggregates up to five prior examinations. However, the progression-consistency target is defined relative to the most recent prior. This creates a potential mismatch: the auxiliary head receives an attention-weighted summary of multiple visits but must predict a label referring to a specific pair of visits. The authors acknowledge this design choice, but do not experimentally establish whether it is beneficial.
Section 5; Appendices H and I. All reported quantitative evaluations use Longitudinal MIMIC-CXR. The test split contains 266 patients and 2,058 longitudinal targets. The manuscript includes a proposed reader-study protocol, but no radiologist evaluation has been conducted. This limits conclusions about the clinical correctness and safety of generated comparative statements, particularly unsupported claims of new findings or disease resolution.