acceptodds
Desk rejected at ICLR 2027

Where to Look Matters: On-Policy Self-Distillation for Long-Video Understanding

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-language models (VLMs) have made substantial progress in long-video understanding, with standard backbone models typically answering questions based on frames sampled across the full video. However, as videos become longer, the full-video context inevitably contains more question-irrelevant temporal content, which can distract the model from the evidence needed to answer a specific question. We empirically find that focusing the visual input on short annotated clue intervals containing question-relevant evidence consistently improves prediction accuracy across model scales compared with using the corresponding full videos, while requiring fewer input frames. Based on this finding, we introduce Clue-OPSD, a clue-privileged on-policy self-distillation framework for long-video understanding. During training, a full-video student learns from a self-teacher conditioned on the corresponding clue interval by aligning their next-token distributions along student-generated trajectories. Clue-OPSD thus uses clue intervals as privileged supervision without relying on ground-truth answer labels, while requiring no clue annotations or additional modules at inference time. Extensive experiments across multiple long-video understanding benchmarks and Qwen3.5 model scales show consistent gains over vanilla backbones while achieving competitive performance against several supervised post-training methods.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
On-Policy Self-Distillation, Long Video Understanding, Temporal Clue

Related papers

Discussion (0)

Sign in to comment.