Medical-4D: 4D Cardiac MRI Vision-Language Models for Cardiac Function Understanding

Cardiac magnetic resonance (CMR) cine imaging provides rich 4D information for assessing cardiac function, underpinning clinical diagnosis. However, jointly modeling spatial, temporal, and inter-slice dependencies over full 4D CMR data is computationally prohibitive. Existing CMR-based diagnostic methods simplify the data via temporal downsampling or single-slice videos, discarding fine-grained cardiac motion and inter-slice anatomical dependencies essential for dynamic analysis. To address this, we propose factorized 4D representation learning with cardiac-phase and anatomical-slice positional encoding. Specifically, the factorized encoder separately performs spatial, temporal, and inter-slice modeling to enable joint spatial-temporal analysis of cardiac function. We further introduce continuous structured visual masking to capture dependencies across adjacent cardiac phases and neighboring slices. Together, these components align visual representations with clinical cardiac function assessment reports. To our knowledge, this is the first cardiac model to exploit the full 4D CMR data for continuous and complete vision-language alignment. Based on this model, we propose Medical-4D, the first cardiac function QA framework. Trained on CMR 4D clinical reports and QA pairs from three data centers, Medical-4D significantly outperforms previous state-of-the-art methods.