- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper suggests a new framework for recognizing temporal change in multi-modal large language models (MLLMs) with adaptation, called TEMA. Following the previous work, TILA, they adapted the basic concept which is the model's confidence or outputs are logically correct when the historical views are reversed to the MLLMs with adding visual evidence consistency and task-adaptive temporal weighting. They showed TEMA can improve the performance of MLLMs for temporal change both in VQA and report generation tasks than other adaptation framework in CheXTemporal and TemMedBench.
Critical strengths
- The paper is well written and easy to follow.
- The authors evaluate the proposed method on VQA and report generation to demonstrate that its applicability is not limited to a single task.
Critical weaknesses
- A central reference for this work, TILA, reports not only retrieval scores and macro accuracy but also the temporal entity matching (TEM) score introduced in BioVIL-T [1]. TEM utilized a broad range of temporal keywords when they calculated the score, including "persistent," "grow," "expand," "elevate," "constant," "unfolded," etc.) Thus, it seems well suited to capturing temporal changes in this paper. However, the authors do not use this metric and instead limit their evaluation to forward balanced accuracy, reverse balanced accuracy, and accuracy excluding stable cases.
- On the CheXTemporal gold set, the proposed method shows lower accuracy when stable cases are included (both FB Acc and RB Acc are lower than their respective counterparts excluding stable cases.) This suggests that the method may classify unchanged cases as changed. Such result appears inconsistent with both clinical needs and the paper's claims.
- All results are based on internal evaluations. For temporal VQA, the authors train on the CheXTemporal training data and evaluate on its test data. For report generation, they partition the TemMed-Bench data for training and validation and evaluate only on a held-out test set from the same dataset.
- The paper also lacks a comparison with ALTA [2], a prior study that focuses on temporal changes in medical vision-language alignment which is the similar approach with this paper.
- The paper does not include any qualitative results to understand the work and check the results.
[1] Bannur, Shruthi, et al. "Learning to exploit temporal structure for biomedical vision-language processing." 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2023.
[2] Lian, Chenyu, et al. "Efficient medical vision-language alignment through adapting masked vision models." IEEE Transactions on Medical Imaging 44.11 (2025): 4499-4510.
Questions for authors
- At the beginning of Section 3.3, the authors explain that Visual Evidence Consistency (VEC) is introduced to make image use observable, since reverse supervision alone does not establish whether the model appropriately uses visual evidence. However, they also caution that VEC should not be interpreted as faithful clinical grounding. Although attention maps or Grad-CAM would not establish faithful clinical grounding either, it would still be useful to determine whether the model attends to, interprets, and uses image regions near the changing lesions. (1) Are there any qualitative evaluations or analyses addressing this? (2) Does the method also work with general-domain models such as Qwen3-VL which is not pretrained on medical imagining?
- How do the authors interpret the differences between accuracy with and without stable cases? What the model's performance specifically on stable cases?