PGOT: Physically Grounded Optimal Transport for Distilling Vision-Language-Action Models

Vision–language–action (VLA) models bring large-scale visual and semantic knowledge to embodied control, but their computational demands motivate distillation into compact counterparts for responsive deployment. Existing distillation objectives, however, fall short in capturing the physical semantics and decision structure of embodied control: categorical matching of token distributions treats physical commands as unstructured labels, while unselective supervision dilutes learning at critical decision points and passes imperfect teacher decisions directly to students. We introduce **Physically Grounded Optimal Transport (PGOT)**, a distillation framework that addresses these gaps through three complementary mechanisms. **Geometry** aligns action distributions through optimal transport of probability mass, grounding teacher–student discrepancies in physical action space. **Criticality** exploits the temporal structure of expert trajectories to identify pivotal decision points and concentrate learning on them. **Reliability** adjusts teacher guidance according to its physical agreement with expert demonstrations, favoring trustworthy supervision across states and action dimensions. Across UAV navigation and robotic manipulation, PGOT distills 7B/9B teachers into 0.8B students that retain teacher-level success with approximately 9–11$\times$ fewer parameters. Over the strongest distillation baselines, it improves success rate by 2.67 and 2.40 percentage points and three-out-of-three completion by 7.33 and 6.00 points, respectively, exceeding even the teachers on the latter. Analyses of learning dynamics and closed-loop behavior further reveal that physical transport supplies optimization directions largely distinct from categorical matching, while selective supervision improves outcomes at critical maneuvers and limits the inheritance of teacher errors, with the largest gains where the teacher itself is unreliable.