- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents DK-VLM, a geometry-aware vision-language model for spatial reasoning. Its Dual-Kernel Attention retains native semantic attention and positional encoding while introducing a separate geometric kernel conditioned on estimated 3D points and camera poses. Learned gates combine the two outputs. Training additionally incorporates pairwise geometric supervision and Reference-Frame Composition, which constructs stepwise turn and heading targets from existing annotations. Experiments on two Qwen backbones evaluate spatial reasoning, multi-step composition, and general video understanding.
Critical strengths
-
Well-motivated geometric integration. The paper addresses a meaningful distinction between a visual token’s input position and its physical location across views. The parallel attention design offers a coherent way to incorporate point displacements and camera transformations while retaining the backbone’s native positional computation. Reference-Frame Composition provides complementary supervision for spatial judgments involving successive changes in perspective.
-
Broad evaluation with useful controls. The experiments cover seven spatial benchmarks and two general video benchmarks. Beyond competitive benchmark results, the same-data SFT comparison and capacity-preserving identity-transform ablation provide useful evidence for the proposed geometric modeling: DK-VLM achieves 74.33 on VSI-Bench, versus 69.20 for the SFT baseline and 70.02 without geometric transforms. RC also improves several public route-planning and perspective-dependent tasks, supporting its value beyond the constructed chain evaluation.
Critical weaknesses
-
Incremental methodological novelty. The geometric kernel builds directly on GTA, with Point GTA using patch-level scene points as translation anchors and Camera GTA using camera poses. The parallel branch, learned gating, and auxiliary geometric prediction objective provide a sensible adaptation to pretrained VLMs, but the underlying geometric operation is inherited. RC similarly reorganizes existing angular annotations into short supervision chains. The contribution therefore lies primarily in architectural integration and supervision construction, with a relatively modest conceptual advance over existing geometric attention.
-
Unverified alignment premise. A central motivation is that reconstruction-feature fusion suffers from semantic–geometric representation misalignment, yet this limitation is not directly demonstrated. The benchmark comparisons and geometric ablations establish the usefulness of DK-VLM, but do not isolate its advantage over feature fusion under matched conditions. A comparison using the same backbone, geometry frontend, training data, and adaptation budget would better substantiate the proposed integration strategy. The existing evidence leaves the claimed alignment advantage insufficiently supported.
-
Unisolated compositional benefits. RC produces encouraging improvements on public spatial tasks, but its specific contribution remains difficult to separate from additional task-relevant supervision. Adding 150K examples changes both training volume and exposure to sequential spatial questions. The backbone comparison does not isolate the importance of intermediate reference-frame targets. Controls using matched additional single-step data or the same chains with final-target supervision would clarify this distinction. Furthermore, the four-step evaluation follows the same construction procedure, providing limited evidence of compositional generalization beyond short turn and heading sequences.
-
Unquantified computational cost. The small trainable-parameter fraction provides an incomplete picture of the method’s practical requirements. DK-VLM requires a geometry frontend and adds full attention over geometry-valid visual tokens at five decoder layers. These operations introduce costs that depend on the number of frames and visual tokens. Without end-to-end latency, peak memory, or inference FLOPs relative to competing approaches, it remains unclear how the spatial accuracy gains trade off against computational overhead and whether the approach scales practically to longer videos.
Questions for authors
See the weaknesses.
3 comments
W3: Isolating the compositional benefit. We ran the single-step control you suggested. On the same base data, we add the same number of single-step questions or RC chains, built from the same relations with the same answer format; the setup and tables are in our reply to 8Sow, W2/Q2. RC reaches 10.9% chain accuracy, against 2.3% for single-step data. The single-step model often gives the wrong number of steps, but even among answers of the correct length, RC is right twice as often. RC also scores highest on VSI-Bench and on route planning, which requires a sequence of turns.
Four steps is a length not seen in training. In the paper, RC reaches 20.6% chain accuracy on four-step chains, against 0.0% without RC; in the controlled experiment, only RC answers any four-step chain correctly, 4.4% against 0% for the other two arms. The four-step set uses the same construction as training and covers turn and heading targets, which we now state in Appendix A.8.
W4: Cost. One H100, each method's official protocol, 32 frames (Spatial-MLLM: 128 frames to VGGT, 16 selected for the LLM); reported VSI-Bench scores:
| Model | VSI-Bench | Latency (s) | Memory (GiB) | Memory growth, 8→64 frames (GiB) |
|---|---|---|---|---|
| Qwen3-VL-8B | 57.90 | 3.01 | 19.2 | +4.9 |
| DK-VLM-8B | 74.33 | 3.14 | 22.3 | +4.5 |
| GeoThinker-8B | 72.60 | 3.09 | 31.7 | +17.4 |
| VG-LLM-8B | 62.20 | 3.83 | 29.5 | +21.4 |
| VLM-3R-7B | 60.90 | 9.03 | 23.4 | +10.8 |
| Spatial-MLLM-4B | 48.4 | 12.11 | 44.9 | – |
DK-VLM gains 16.4 points over Qwen3-VL-8B at nearly the same latency and 3.1 GiB more memory, and it is more accurate than GeoThinker with similar latency and less memory. Of the two costs you point out, the geometry frontend runs in parallel with tokenization, and the full attention at five layers adds only 2.6% to the prefill compute. For longer videos, from 8 to 64 frames DK-VLM's memory grows no faster than Qwen's, while the memory increase of the other geometry-aware methods is 2–5× larger. At 64 frames its latency is 5.61 s against 4.80 s for Qwen: VGGT-Ω's global attention grows quadratically with the number of frames, so the frontend is no longer fully hidden behind tokenization.
Thank you again for your constructive feedback on DK-VLM. We have added the matched feature-fusion comparison, the single-step control for RC, and inference-cost measurements to the revised manuscript, and clarified our contributions relative to prior work in our response.
As the discussion period draws to a close, we would appreciate any further feedback on our responses and additional results. We would be happy to clarify any remaining questions.
W1: Novelty relative to GTA. GTA encodes camera transforms in attention; we keep this as Camera GTA and add Point GTA, which uses each patch's estimated 3D point as the translation anchor. Removing Point GTA costs 2.5 points on VSI-Bench (Table 6). Our main contribution is the adaptation you mention: GTA was designed for training from scratch, and in a pretrained VLM the same operator scores 39.14 when it replaces mRoPE inside the native attention, far below SFT at 59.48, but 62.78 as a separate branch beside it. The dual kernel injects geometry as positions in attention rather than through the feature fusion common in spatial VLMs, and in the matched comparison under W2 it outperforms feature fusion. Taking explicit points and poses as input also lets the geometry frontend be swapped without retraining.
RC does build on existing annotations and needs no relabeling. What it adds is a different target: each step is defined relative to the heading reached at the previous step, so a chain supervises successive reference-frame updates, which single-step questions do not. In the controlled experiment under W3, the same relations as single-step questions reach 2.3% chain accuracy, and RC raises it to 10.9%.
W2: Alignment premise. We ran the comparison you suggested, with the same backbone, frontend, data and training steps: DK-VLM scores 62.78 on VSI-Bench, feature fusion 61.33 and SFT 59.48 (table in our reply to 8Sow, W1/Q1).