SyncPro: Synchronous Reasoning and Responding for Proactive Streaming Video Understanding
Abstract
Proactive streaming video understanding requires assistants to track unfolding events and respond when a user's requested condition is met. To make these decisions, assistants must accumulate relevant evidence and reassess response conditions as new observations arrive. Existing work offers complementary capabilities for this process: proactive video assistants learn when to respond, while streaming video reasoning methods generate intermediate text to organize observations and support subsequent answers. Together, these capabilities provide a foundation for proactive reasoning, but how to coordinate ongoing reasoning with timely responses remains underexplored. In particular, common reasoning formulations schedule updates within a single generation stream, so updates may be redundant or lag behind relevant state changes. The shared generation path also has to accommodate the distinct objectives of internal monitoring and user-facing task execution, while responses placed after reasoning must await its completion. To address these challenges, we introduce SyncPro, a synchronous dual-stream framework that separates internal monitoring from user-facing responses. Its Inner and Response branches share causal history and activate independently, allowing reasoning to follow relevant evidence and responses to proceed concurrently. We construct 189K dual-stream examples to train this framework. Experiments across three proactive benchmarks show improved response timing and task performance in event monitoring, repetition counting, sequential step tracking, and anomaly-triggered responses.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.