- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies spatial reasoning in medical multimodal large language models (MLLMs), focusing on the distinction between image-space coordinates, which describe where an object appears on the displayed image, and patient-space coordinates, which describe its location relative to the patient’s anatomical axes. The authors first show, using layer-wise linear probes and hidden-state interventions, that existing medical MLLMs contain linearly accessible image-coordinate representations, but that these representations are not reliably connected to patient-centered spatial reasoning. To address this gap, the paper proposes Patient-Space Coordinate Learning (PS-Coord), a training framework that jointly supervises image and patient coordinates through two auxiliary prediction heads attached to an intermediate hidden state, together with standard answer supervision. Experiments on eight backbones ranging from 7B to 34B parameters report consistent improvements on CT spatial-reasoning benchmarks, transfer to radiographic tasks and unseen rotations, and gains on medical VQA benchmarks. Representation probing further shows that patient coordinates become substantially more linearly accessible after training, while counterfactual experiments suggest that PS-Coord-trained models use anatomical context from the current image when making patient-relative spatial judgments.
Critical strengths
- Clear and clinically meaningful formulation of spatial reasoning
The paper makes an important conceptual distinction between image-space and patient-space coordinates. This formulation captures a genuine challenge in medical imaging: the screen position of an anatomical structure changes under rotation or reflection, whereas its location relative to the patient should remain invariant. By decomposing patient-centered spatial reasoning into visual grounding, orientation inference, and coordinate-frame conversion, the paper provides a precise explanation of why a model may correctly identify where an object appears in an image yet still fail to reason about its anatomical location. This dual-frame perspective is intuitive, mathematically well specified, and potentially useful beyond the particular training method proposed here.
- Strong and complementary empirical evidence
The evaluation goes beyond reporting end-task accuracy. The authors combine layer-wise linear probing, target-binding analysis, prompt-transfer experiments, hidden-state interventions, held-out rotations, cross-modal transfer, and counterfactual removal or replacement of anatomical context. In particular, patient-coordinate accuracy falls to approximately chance when the anatomical background is removed or replaced, while image-coordinate accuracy remains high. This provides meaningful evidence that the trained models use image-specific anatomical context rather than relying only on marker position, question wording, or memorized anatomical priors. The substantial improvement on rotations excluded from training also supports the claim that the model learns a transferable frame-conversion rule rather than simply memorizing the training orientations.
Critical weaknesses
- The main results do not isolate the contribution of coordinate supervision from spatial instruction tuning
The primary comparison is between the original checkpoint and a model trained with both spatial question-answer supervision and the proposed coordinate losses. Consequently, the large improvements in the main table cannot be attributed specifically to the dual-coordinate objective. The appendix includes an answer-only condition with (\lambda_c=0), but this ablation is limited to two 7B backbones and shows relatively modest additional gains from coordinate supervision. For example, the MIRP improvement is 2.9 percentage points for Qwen2.5-VL-7B and only 0.05 points for Lingshu-7B. The paper would be substantially stronger if the main experiments included matched answer-only baselines for all eight backbones and all six benchmarks, together with statistical variation across multiple seeds. This would clarify how much of the reported improvement comes from the proposed representation-level supervision rather than targeted spatial instruction tuning more generally.
- The experimental setting is highly controlled
The training setup relies primarily on selected two-dimensional axial CT slices, artificial rotation and reflection transformations, explicit circular target markers, and slices filtered to contain recognizable orientation cues. This makes the coordinate labels and counterfactual tests clean, but it also leaves a considerable gap between the benchmark setting and realistic clinical spatial reasoning. The current experiments do not evaluate volumetric localization, oblique or non-axial views, unmarked abnormalities, truncated fields of view, acquisition artifacts, postoperative anatomy, or uncommon anatomical configurations such as situs inversus. These cases are precisely where memorized anatomical priors and standard orientation assumptions may fail.
Questions for authors
See weaknesses.