- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper asks whether Cosmos3-Nano-Policy-DROID transfers its Qwen3-VL reasoner's object grounding to action generation. It concludes that the generator follows object pixels and nouns, but location text does not bridge the reasoner's grounding to actions.
Critical strengths
The paper demonstrates substantial empirical effort, including a broad set of probes, paired interventions, controls, and closed-loop evaluations.
Critical weaknesses
-
The central question is mismatched to the deployed architecture. The paper shows that there is no visual-reasoner-to-action pathway through which the reasoner's grounding of the current scene could be transferred. But this is the architecture and training setup, not an accidental missing pathway. Under this design, the generator is the component that is expected to ground visual observations and produce actions. Q2 provides evidence for exactly this explanation: the generator responds causally to target-pixel changes and re-targets after object movement. The policy may simply not need visual information from the reasoner. A model's not using a particular capability, or one pathway's not carrying information, does not imply that the system lacks that capability or information. Therefore, the premise that the reasoner's visual grounding should transfer to actions is absent for this model; even if this transfer question is meaningful for another architecture, Cosmos is not an appropriate model on which to test it. This is consequential because the paper treats an architectural property as a failed transfer.
-
The Q3 location-text intervention is confounded by prompt distribution shift. The paper acknowledges that the appended sentences and relative-clause prompts are rare or absent in policy-training instructions. Its null result may therefore show that the policy does not interpret these prompt formats, rather than that it cannot use location language. The same-class-distractor test does not rule this out: it uses the same constructions, and its verified subset is small. A convincing test would use naturally occurring, in-distribution spatial referring expressions in scenes with genuinely ambiguous same-class instances.
-
Q1--Q3 do not establish the paper's central claim. Q1 evaluates the reasoner in VLM mode, Q2 instead shows that the generator responds to target pixels and re-targets after movement, and Q3 constructs a text bridge. Thus these experiments can be explained by the architecture itself, but do not show that the reasoner's visual grounding was expected to transfer to actions and failed to do so. The presentation reinforces this overstatement. The abstract is overly dense with numerical results and the contribution list largely repeats result-level statistics. The paper does not clearly separate high-level contributions, empirical results, and the central claim. It should state its precise, supportable claim first and distinguish it from broader architectural implications.
Questions for authors
-
Is there evidence from the Cosmos design or documentation that the reasoner is intended to process visual observations during policy inference, rather than functioning only as a text-conditioning component? If not, why is this architectural property framed as a failed transfer?
-
How do Q1, which is off-policy, Q2, which shows direct generator grounding, and Q3, which adds an untrained text interface, jointly test the claimed failure of reasoner-to-action grounding transfer?
-
Can the authors evaluate spatial referring expressions that occur naturally in the policy-training distribution, with genuinely ambiguous same-class instances where the expression is necessary to identify the target?