- Rating
- 2
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes to add eight learnable "geometry query tokens" at the end of each user turn of a streaming Video-LLM navigation policy. During training, the last-layer hidden states of these tokens are decoded by small heads to reconstruct three targets for the current frame, and binary connectivity and traversability masks. Query tokens attend bidirectionally within their group. After training, the heads and tokenizer are discarded, so inference uses only RGB and text, with the eight tokens left in the prompt. On R2R-CE / RxR-CE Val-Unseen the 4B model reaches 64.8 / 59.6 SR, above 7B geometry-enhanced methods such as JanusVLN and GA-VLN, at 81 ms per frame versus 750–1,160 ms.
Critical strengths
- The paper is clear and the goal is practical. The design is simple and backbone-agnostic.
- Controlled comparisons are good: same backbone with/without GeoScaffold on three model families (Table 2), and a "query tokens without reconstruction" control showing the gain is not from extra tokens.
- The ablations are informative: which hidden states to supervise (image tokens hurt badly, action tokens hurt mildly), discrete vs continuous vs pixel depth targets, attention mask, token budget, and blocking the latents at test time to show that action decoding reads them.
Critical weaknesses
- The novelty is incremental. Auxiliary reconstruction through learnable query/perception tokens is well established (ROSS, 3DRS, Spatial Forcing, DreamVLA, Perception Tokens, LISA-style readouts). The new part is the specific target set for VLN.
- The headline Table 1 comparison is confounded by backbone. GeoScaffold-4B uses Qwen3-VL, while most baselines use older 7B backbones. The same backbone trained with the full data recipe but without GeoScaffold is missing from Table 1.
- The latency comparison in Fig. 4 contrasts a 4B RGB-only model with 7B models that run VGGT or BEV builders. Any RGB-only policy of the same size would have the same latency, so this is not evidence for GeoScaffold specifically.
- All simulation numbers are single runs. Many ablation gaps are 1–2 SR on ~1.8K episodes, which is near the sampling noise of SR.
Questions for authors
- Full-data baseline. Please add "Qwen3-VL-4B w/o GeoScaffold" trained with the identical full-data recipe (ScaleVLN, DAgger, LLaVA-Video, ScanQA, MMC4) to Table 1. Without it, the 64.8 SR cannot be attributed to GeoScaffold rather than to the newer backbone.
- Comparison with training-only alternatives. Spatial Forcing (aligning hidden states to VGGT features during training only) and 3DRS also add no inference cost. How does GeoScaffold compare to them on the same backbone and data?
- Why does image-token supervision collapse to 43.2 SR? That is 5.6 points below the policy with no geometry at all. Was the loss weight or head re-tuned for that variant? A drop this large suggests an implementation or weighting issue rather than an inherent property.
Thank you for your constructive review and for recognizing the controlled comparisons and ablations.
A1: Spatial Forcing and 3DRS also use training-only supervision, so removing auxiliary modules at inference is a shared advantage. Our contribution combines navigation-specific targets with persistent geometric query memory, retaining spatial cues across viewpoint changes, rooms, and corridors for later action decisions.
Navigation targets. Depth provides distance cues, connectivity identifies route continuations, and traversability identifies walkable surfaces. The submitted target ablation shows that removing any target lowers SR from 55.3 to 51.1–53.1 and SPL from 51.0 to 47.1–49.2 (Table 3), supporting their complementary roles.
VGGT feature alignment. Aligning the VLN model's final-layer image tokens with VGGT's final-layer features yielded only a modest SR gain. Retaining dense image states for direct access to geometric history would require many more cached tokens, so we chose compact query memory with eight queries per frame.
A2: We now report this comparison below. The results are also included in Appendix A, Table 7 (p. 14). The baseline uses Qwen3-VL-4B and the same full-data recipe, with geometry queries, reconstruction heads, and geometric losses removed. GeoScaffold improves SR/SPL by 6.4/9.2 points on R2R-CE and SR/SPL/nDTW by 3.6/6.5/3.2 points on RxR-CE.
A3: The comparison measures complete-system deployment cost. With Qwen2.5-VL-7B, GeoScaffold runs at 121 ms/frame on an RTX 4090, compared with 750 ms for GA-VLN and 1,160 ms for JanusVLN, showing low latency also at 7B. At inference, we remove the tokenizer and reconstruction heads. The retained queries add attention computation and KV storage.
A4: All simulation results are averages over three random seeds. We apologize for not stating this clearly.
A5: The submitted ablation (Table 5) keeps the three targets, loss weights, head parameter count, and training hyperparameters unchanged. For depth reconstruction, it mean-pools each frame's final-layer image states into a 3,584-dimensional vector and predicts depth tokens using LayerNorm and MLP. Pooling compresses spatial information into one vector, and the image states support both navigation and depth reconstruction. Dedicated queries provide separate geometric readout positions. Supervising a subset of image tokens raises SR from 43.2 to 48.7, but remains below the 55.3 achieved with dedicated queries.