- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
Medical-4D fully leverages the complete spatiotemporal information from cardiac cine MRI, significantly improving visual question answering performance for cardiac structure and motion function. This study holds substantial clinical value. Its factorized encoder separately models spatial, temporal, and inter-slice dependencies, enabling joint spatiotemporal analysis of cardiac function. Furthermore, the paper introduces continuous structured visual masking to capture dependencies between adjacent cardiac phases and neighboring slices. The authors constructed a three-center clinical dataset, and Medical-4D, trained on CMR 4D clinical reports and QA pairs, substantially outperforms previous state-of-the-art methods.
Critical strengths
On the technical level, the authors claim to be the first to achieve joint spatiotemporal modeling of complete 4D CMR data, avoiding information loss. In addition, they introduce continuous structured visual masking and bidirectional cross-modal recovery to achieve fine-grained phase-slice alignment.
Critical weaknesses
Although this paper attempts to fill the gap in 4D image-text pretraining within the medical domain, it does not introduce any fundamentally new learning mechanism from an innovation perspective. Moreover, the QA data were automatically generated by Qwen and only underwent rough manual filtering, making it difficult to adequately verify the accuracy and validity of such a data construction approach.
Questions for authors
- Performing Transformer attention sequentially along the temporal and slice dimensions is essentially an intuitive factorized spatiotemporal scheme. The paper fails to clearly articulate what new learning mechanism this approach introduces compared to existing methods such as axial attention, spatiotemporal decomposition, and medical vision-language pretraining.
- The QA data were automatically generated by Qwen from clinical reports, with answers derived from the same reports. The paper does not sufficiently demonstrate the clinical validity of the questions and answers, whether patient-level isolation is achieved in the independent test set, nor the relationship between open-ended QA metrics and true clinical correctness.
- The performance gains on some metrics are modest, and the proposed method does not lead across all external-center structure-related closed-ended QA tasks. No confidence intervals, repeated experiments, or statistical significance tests are reported. Therefore, the tabulated results alone are insufficient to claim that the method significantly outperforms all baselines.
2 comments
Q3
We thank the reviewer for this important comment. To verify the reproducibility and validity of the experiments, in the revised paper we trained and tested each experimental setting five times using different random seeds, and we report the mean and standard deviation over the five runs. The updated results are shown below.
Comparative analysis on the center 1 and center 2 datasets. Token recall for open-ended questions; accuracy for close-ended questions.
| Method | Input | Close- Structure | Close- Wall Motion | Open-Structure | Open-Wall Motion |
|---|---|---|---|---|---|
| Med-flamingo~\cite{pmlr-v225-moor23a} | 2D | 72.94$\pm$0.64 | 35.90$\pm$0.70 | 58.92$\pm$0.45 | 56.60$\pm$1.12 |
| Llava-med~\cite{li2023llava} | 2D | 79.49$\pm$0.63 | 27.88$\pm$0.98 | 59.87$\pm$0.59 | 53.13$\pm$0.44 |
| HealthGPT-Pro-8B | 3D | 77.57$\pm$0.70 | 36.39$\pm$0.33 | 41.45$\pm$1.05 | 50.21$\pm$0.54 |
| CineMA~\cite{fu2026development} | 3D | 91.61$\pm$0.69 | 61.90$\pm$0.63 | 65.25$\pm$1.82 | 55.14$\pm$0.79 |
| Qwen 3.8-27B~\cite{qwen38} | 2D+T | 54.91$\pm$0.36 | 27.39$\pm$0.97 | 50.49$\pm$2.93 | 43.78$\pm$2.91 |
| CMR-CLIP~\cite{nakashima2026contrastive} | 2D+T | 87.97$\pm$0.41 | 84.86$\pm$0.50 | 62.97$\pm$0.34 | 64.86$\pm$0.45 |
| CMR-Transformer~\cite{shad2026generalizable} | 2D+T | 89.82$\pm$0.49 | 85.68$\pm$0.47 | 64.77$\pm$0.66 | 64.87$\pm$0.72 |
| Ours (Medical-4D) | 4D | 91.93$\pm$0.28 | 90.38$\pm$0.30 | 66.40$\pm$0.31 | 65.87$\pm$0.37 |
Evaluation on Center 3 (unseen during training). Token recall for open-ended questions; accuracy for close-ended questions.
| Method | Input | Close- Structure | Close- Wall Motion | Open- Structure | Open- Wall Motion |
|---|---|---|---|---|---|
| Med-flamingo\cite{pmlr-v225-moor23a} | 2D | 73.36$\pm$0.53 | 27.12$\pm$0.80 | 60.95$\pm$0.36 | 57.64$\pm$1.25 |
| Llava-med \cite{li2023llava} | 2D | 80.47$\pm$0.71 | 28.71$\pm$0.88 | 61.19$\pm$0.68 | 54.82$\pm$0.33 |
| HealthGPT-Pro-8B | 3D | 78.11$\pm$0.55 | 47.45$\pm$0.50 | 42.05$\pm$1.22 | 41.53$\pm$0.38 |
| CineMA \cite{fu2026development} | 3D | 90.48$\pm$0.84 | 63.37$\pm$0.55 | 65.62$\pm$1.65 | 57.31$\pm$0.89 |
| Qwen 3.8-27B \cite{qwen38} | 2D+T | 55.34$\pm$1.21 | 36.58$\pm$1.14 | 45.97$\pm$2.75 | 43.62$\pm$3.02 |
| CMR-CLIP \cite{nakashima2026contrastive} | 2D+T | 88.39$\pm$0.57 | 89.00$\pm$0.43 | 64.27$\pm$0.44 | 46.51$\pm$0.61 |
| CMR-Transformer \cite{shad2026generalizable} | 2D+T | 91.30 $\pm$0.51 | 88.72$\pm$0.58 | 65.17$\pm$0.49 | 63.34$\pm$0.83 |
| Ours(Medical-4D) | 4D | 91.58$\pm$0.44 | 90.11$\pm$0.36 | 66.13$\pm$0.47 | 64.92$\pm$0.22 |
We agree that the improvement in close-structure performance is modest. However, on the wall motion-related close and open tasks, our method achieves substantial improvements over existing CMR vision-language models. This demonstrates the effectiveness of using full CMR-4D data for cardiac functional motion diagnosis, outperforming existing CMR methods (which use only the two temporal frames ED/ES or extract only a subset of temporal frames). For example, the following diagnostic evidence for cardiac functional motion disease depends on continuous and fine-grained temporal frame change information: "reduced and dyssynchronous wall motion"; "reduced motion at the left ventricular apex"; "globally reduced systolic function"; "thinning of the left ventricular wall with regional dyssynchronous motion"; "markedly reduced left ventricular systolic function"; "impaired diastolic function"; and "reduced motion amplitude of the interventricular septum and the left ventricular anterior wall." These cardiac functional motion disease expressions all come directly from physician clinical reports and are also cardiac function phenomena that clinicians focus on in diagnosis; these cardiac function findings strictly depend on temporal changes across the entire cardiac cycle, and thus cannot be derived from subsampled frames or from ED/ES frames alone. In addition, we have revised the claim in the paper that the method significantly outperforms all baselines.
Q1
We thank the reviewer for asking us to clarify the learning mechanism. We agree that applying attention sequentially along separate dimensions, as in axial attention and divided space-time attention, is an established strategy. We do not claim that temporal attention or slice attention, by themselves, constitute a new attention algorithm.
Our contribution is to adapt factorized modeling to the distinct dimensions of full 4D short-axis cine CMR and the computational challenge of its large token volume. To the best of our knowledge, existing medical vision-language pretraining methods do not jointly process the complete multi-slice cine sequence; existing CMR approaches commonly reduce the input through temporal subsampling, slice selection, or use of only ED and ES frames. Medical-4D instead encodes each phase–slice image spatially, adds cardiac-phase and anatomical-slice identity embeddings together with coordinate encodings that preserve temporal order and anatomical position, and then applies temporal attention within each slice followed by slice attention within each phase. This factorization makes it possible to model the full phase–slice grid without discarding its phases or slices. It also preserves a contextual feature for each phase–slice pair before study-level pooling.
The learning objective then uses these phase–slice features for fine-grained cross-modal supervision. In report-assisted visual recovery, clinically structured masks cover contiguous slices or phases, and the paired report helps recover the masked visual features. In image-assisted text recovery, masked clinical phrases are recovered using the phase–slice visual features. These objectives add local report–image interaction to global image–report alignment, while retaining the full 4D representation.
Accordingly, our contribution is the CMR-specific application of full-4D representation learning and report-guided cross-modal recovery to cardiac-function understanding and QA, addressing both the large token volume and the need to preserve phase and slice information. We will revise the manuscript to clarify this scope and avoid implying that the factorized attention operation itself is a fundamentally new learning mechanism.
Q2
We thank the reviewer for this important question.
Clinical grounding and Qwen’s role. The questions are based on clinical questions that physicians address when assessing cardiac structure and function. Qwen was used to rephrase or vary the wording of these questions, rather than independently define new clinical questions from its prior knowledge. For answer construction, Qwen extracted the corresponding structural or functional information from each patient’s physician-authored report according to our predefined templates. Thus, the reference answers originate from the patient’s clinical report, rather than clinical findings independently generated by Qwen.
Throughout dataset construction, specialist physicians from two different hospitals provided ongoing training, guidance, and clarification to the three master’s/doctoral students involved in the process. The physicians also performed random spot checks of the QA pairs and corrected identified errors. We acknowledge that the phrase “rough manual screening” does not adequately describe this process. We will expand the manuscript to explain the physician-supervised construction and quality-control procedures, and provide a detailed breakdown of the question types and their corresponding cardiac-function categories.
Patient-level separation. We confirm that the training and test sets are separated at the patient level, not the QA-pair level. Specifically, the 80/20 split of the Center 1 and Center 2 datasets described in Section 5.3 was performed by patient: the images, reports, and QA pairs belonging to the same patient remain in the same partition. In addition, no patients from Center 3 participated in training; this center was used exclusively for external evaluation. Therefore, the training and test sets contain no overlapping patients. We will explicitly state this in the revised manuscript to remove the ambiguity in the current description.