acceptodds
Desk rejected at ICLR 2027

GlimpseBC: Learning Saccadic Visual Policies from Global View Demonstrations

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Visual policies often process entire camera frames at every step, despite only small regions being relevant to control. In this paper, we study behavioral cloning subject to a hard pixel constraint. We introduce GlimpseBC, a simple learning algorithm in which a model jointly learns where to look and what to do from offline global view trajectories. Our model uses a differentiable crop to learn where to look from full-frame trajectories during training and a hard crop during inference, ensuring that it cannot access full-frame observations. On RoboMimic and Push-T, our models are competitive with full-frame policies (with comparable architecture) while reducing the computational cost in both training and inference. We additionally use the cluttered LIBERO-Spatial benchmark to study where narrow glimpses struggle.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Computer Vision, Saccades, Behavioral Cloning

Related papers

Discussion (0)

Sign in to comment.