- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper uses controlled reference-heading reversals in COMFORT scenes, together with probes, activation replacement, phase analysis, and targeted fine-tuning, to argue that VLMs encode but do not read out object-centered heading information and instead rely on camera-relative cues.
Critical strengths
The paired heading-reversal design is a clear behavioral diagnostic, and the paper combines it with probes, interventions, phase measurements, and fine-tuning splits supported by substantial implementation details and controls.
Critical weaknesses
-
The heading probe is not shown to capture an abstract heading variable. Held-out-object heading decoding is only 0.481--0.537 (Table 16), and phase scrambling preserves part of the signal. The proposed representation may therefore reflect object- or rendering-specific correlates rather than semantic object heading.
-
The camera-versus-heading patch comparison is not semantically matched. Camera reversal changes the entire scene projection, while heading reversal mainly changes one object. Equal residual norms do not make them interventions of equal visual scope or information content, so the result establishes sensitivity to these particular patches, not a clean relative readout preference.
-
The fine-tuning conclusion is too broad. The experiments show that the tested fixed-heading LoRA/full-tuning recipes fail to extrapolate to a reversed heading. But they do not show that fine-tuning with rotation augmentation, object-coordinate supervision, or reference-frame-aware data cannot work.
-
The novelty is inadequately positioned. The paper should distinguish its heading-specific analysis from recent work on perspective consistency, allocentric/egocentric disambiguation, and causal transport from spatial representations to answers. Without this distinction, the remaining contribution appears incremental for ICLR.
Questions for authors
-
Given the held-out-object heading decoding and phase-scrambling result, what evidence shows that the probed signal is semantic reference-object heading rather than object- or rendering-specific correlates? Would decoding and intervention effects transfer across objects, textures, and renderers?
-
How can full-scene camera-reversal replacement and mainly local heading-reversal replacement be treated as matched tests of reliance on camera versus heading? Can the authors provide a semantically matched counterfactual, or limit the claim to sensitivity to these particular interventions?
-
Which conclusion remains after distinguishing failure of the tested fixed-heading recipes from failure of fine-tuning more broadly? How does the heading-specific causal analysis differ from prior causal-transport work?