- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes TokenGazer, which conditions a visuomotor policy on a small subset of frozen DINOv2 patch tokens rather than on all tokens. In Stage 1, a lightweight transformer gaze policy scores each token and samples K of them. It is trained with GRPO, using as reward how accurately a frozen action-prediction head predicts the expert action from the selected tokens alone, and a Gumbel-softmax variant is also compared. In Stage 2, a diffusion policy is trained on an even mix of the gaze’s top-K tokens and random subsets. Only the top-K tokens are used at deployment. On PushT, PushT-Distract, OGBench Cube, and LIBERO-Goal, TokenGazer with 8 to 64 tokens outperforms the dense policy (256 or 512 tokens) on PushT (0.85 vs. 0.75) and LIBERO (0.82 vs. 0.72) and matches it on Cube (0.88 vs. 0.89 cubes placed). Stage 2 training is also cheaper than dense policy training.
Critical strengths
-
The problem is clearly formulated. The paper keeps the dense tokens of a frozen encoder and frames the task as selecting only the tokens the policy needs. Selecting tokens by action-prediction accuracy rather than reconstruction quality is well suited to robot learning.
-
The method matches or exceeds the dense policy with far fewer tokens. Without modifying the encoder, the policy receives only 8 to 64 of the 256 tokens (512 on LIBERO), and it outperforms the dense policy on PushT and LIBERO and matches it on Cube. Because the policy’s sequence length scales with K, Stage 2 training time also drops by a factor of 1.46 to 3.2.
Critical weaknesses
-
The performance gains are small and statistically uncertain. TokenGazer only matches the dense policy on Cube (0.88 vs. 0.89) and Cube-Distract (0.53 vs. 0.51).
-
Inference efficiency, a central motivation of the paper, is not measured. Only training time is reported (Table 3), although at inference DINOv2 still encodes the full image and the gaze policy attends to all N tokens. With Stage 1 included, end-to-end training on PushT takes longer than for the dense policy.
-
The experimental scope is limited. All experiments are in simulation, and only DINOv2 is tested as the encoder. Removing proprioception for all methods also places the dense baseline well below the published Patch Policy results (LIBERO 0.72 vs. 0.98), which raises concerns about the reported dense policy results.
-
The manuscript would also benefit from experiments at VLA/WAM scale and from a comparison with the VLA token pruning method it cites (Pei et al., 2026).
-
Several results are incomplete, and the text is inconsistent in places. Pending cells remain (Table 16), and Gumbel-softmax is not run on Cube or LIBERO. The abstract describes alternating training, whereas the method trains in two stages with a frozen reward head.
Questions for authors
-
Does the same trend hold with encoders other than DINOv2, such as SigLIP, CLIP, or WebSSL?
-
Could the authors provide quantitative results comparing reconstruction-based and action-based rewards?
-
Could the authors report per-step inference latency broken down into the encoder, the gaze policy, and the diffusion policy? Since the gaze policy attends to all N tokens, is its cost smaller than the savings in the diffusion policy?
-
Can the method be applied at VLA scale? Results on a small VLA such as SmolVLA, together with a comparison with Pei et al. (2026), would strengthen the paper.
-
Why are the dense results lower than the published Patch Policy numbers? Could the authors compare TokenGazer and the dense policy under the Patch Policy protocol, with proprioception and the original action horizon?
-
With proprioception, TokenGazer drops to 0.31 at K = 8 on LIBERO, well below the dense policy (0.70) and random selection (0.48). Why is a policy conditioned on selected tokens more prone to the proprioception shortcut, and does removing proprioception favor selection methods?