WatchWeave: Concurrent Proactive Perception and Transactional Action over Streaming Video
Abstract
Streaming video assistants must sustain user requests across ongoing observations, intervening questions, and instruction changes. Timely responses require tracking both visual events and the changing requests and reminder histories associated with them. We introduce WatchWeave, a runtime for concurrent proactive perception over a shared video stream. It maintains independently editable, versioned per-request records (Watches), combines event-phase and transition evidence with reminder history, and recovers weak event cues through relative changes in the model’s binary prediction scores. It checks proposed updates against current state and revalidates requests before releasing generated responses. On all 3,738 IPIBench episodes, WatchWeave achieves Overall scores of 30.58% and 34.54% with Qwen3-VL-8B and Qwen3-VL-32B, respectively. The 8B configuration exceeds published same-backbone IPI-Agent by 2.93 percentage points, with gains in repeated monitoring and multi-task management. On all 250 OmniInteract samples, native speech achieves an overall Interaction-Aware Quality-Timeliness F1 (IA-QTF1) score of 0.420 under video-time evaluation, extending the assessment to continuous multimodal interaction.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.