- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents SyncPro, a dual-stream framework for proactive streaming video understanding. It separates continuous internal reasoning from user-facing responses through two specialized branches that share causal history while activating independently and decoding concurrently. The authors construct 189K dual-stream trajectories and employ supervised fine-tuning followed by trajectory-level reinforcement learning. Experiments demonstrate improvements in response timing and accuracy across several benchmarks. However, concerns remain regarding the methodological novelty, generalizability of Inner reasoning, insufficient validation of key components, and computational overhead.
Critical strengths
S1. Well-Motivated Problem Formulation.
Decoupling continuous internal reasoning from user-facing responses into two streams is a reasonable and promising direction for streaming video understanding. The proposed framework allows both streams to share causal history while activating independently and decoding concurrently.
S2. Dataset Construction.
The paper constructs a substantial dataset of 189K trajectories, providing supervision for proactive reasoning and responding. The data construction pipeline incorporates response reconstruction to improve the quality of the generated trajectories.
Critical weaknesses
W1. Unclear Research Gap and Novelty.
The paper does not clearly distinguish its contributions from existing multi-stream reasoning frameworks, such as [1] [2]. While several challenges are identified, the connection between these challenges and the proposed design could be better articulated. As a result, the novelty and necessity of the synchronous dual-stream framework remain unclear.
W2. Limited Evidence of Generalizable Inner Reasoning.
The Inner stream mainly focuses on query-specific observations, event counts, and intermediate states. It is unclear whether the model develops generalizable continuous reasoning capabilities or primarily learns to generate structured state summaries for predefined tasks. In addition, the accuracy, complexity, and long-term consistency of Inner reasoning receive limited evaluation, particularly for complex or unseen monitoring tasks.
W3. Insufficient Validation of Dual-Stream Coordination and Training Objectives.
The framework involves several architectural components and training objectives, yet their individual contributions are not fully examined. Notably, only 8.6K out of 13.52M training micro-turns (0.064%) involve simultaneous activation of both streams, raising questions about how effectively concurrent reasoning and responding are learned and evaluated. Moreover, the contribution of Action Temporal Ranking (ATR) is not directly assessed through ablation experiments.
W4. Limited Analysis of Computational Overhead.
While the paper demonstrates speedups from concurrent decoding, the additional computational cost of Inner reasoning relative to single-stream approaches is not fully explored. A more detailed analysis of the trade-off between reasoning benefits, computational overhead, and real-time efficiency would strengthen the evaluation.
Questions for authors
Q1. Quality of Synthetic Training Data.
Given that most training trajectories are synthetically generated, how do the authors ensure the factual correctness and diversity of Inner reasoning annotations? Beyond response reconstruction, is there any independent evaluation of the quality of the generated intermediate reasoning states?
Q2. Generalization and Performance Limitations.
Why does SyncPro underperform existing methods on certain ProactiveVideoQA subsets, particularly EGO, while also exhibiting degradation in general video understanding? Could these limitations result from task-specific specialization or information loss during state updates? Additionally, would similar performance trends be observed with other backbone models?
[1]Multi-Stream LLMs: Unblocking Language Models with Parallel Streams of Thoughts, Inputs and Outputs.
[2]Think-as-You-See: Streaming Chain-of-Thought Reasoning for Large Vision-Language Models. CVPR 2026.
11 comments
Response to Reviewer YV3X (2)
Response to W3
Thank you for the attention to the architecture design. Each component is supported by a single-factor control, concurrent activation is learned from full-trajectory supervision rather than from joint-activation turns alone, and removing ATR lowers both StreamingBench-PO and OVO-FAR.
- Every component has a single-factor control that favors the full design. All controls use the same backbone and source trajectories.
| Component | Control | Control vs. SyncPro |
|---|---|---|
| Internal-update target | Response-only | PO Joint@2s 10.4 vs. 30.8 |
| Independent branch decisions | Interleaved (one shared activation sequence) | PO Joint@2s 17.2 vs. 30.8, Exact Content Acc.@Response 8.98 vs. 45.24 |
| Role-specialized routes | Swap the Inner and Response LoRA routes | Reference-token NLL rises by 0.706 (Inner) and 1.578 (Response) |
| State-merge weights | Learned per branch vs. equal (γ = 0.5) | OVO-FAR mean F1 30.71 vs. 32.37 |
| ATR | λ_ATR = 0 | PO Acc@2s 36.40 vs. 38.80, OVO-FAR F1 (CRR/SSR/REC) 25.66/10.22/34.08 vs. 28.06/21.36/47.69 |
| Trajectory-level RL | SFT only | PO Joint@2s 30.8 vs. 33.6 |
| Execution schedule | Inner-first serial decoding | Real-time Joint@2s 21.2 vs. 24.0, 615 vs. 498 ms/frame |
-
Concurrent activation is learned from action supervision on every micro-turn and is effective at the level of action timing, Inner content and final responses. Both branches receive action supervision (⟨silent⟩ or active) at all 13.52M micro-turns and learn content generation separately, over 409K Inner entries and 377K Response outputs, so learning does not depend on the joint-activation turns alone. On action timing, the mean per-case action-event F1 on 50 rollouts is 0.71 for Inner and 0.61 for Response. On Inner content, an audit of 50 OVO-Bench questions gives a direct query-relevance rate of 90.24% (SFT) and 91.41% (RL). On final responses, SyncPro-SFT reaches 30.8% Joint@2s versus 17.2% for Interleaved.
-
To address your concern, we ablated ATR, and it helps the model learn trigger timing. ATR resolves a trade-off in silent weighting: raising the silent weight from 0.1 to 0.2 lowers the false-trigger rate on repeated-silent steps from 17.28% to 9.76%, but also lowers recall to 12%, since suppressing repeated speech also suppresses the positions where the model should speak. Adding ATR improves every metric:
| Setting (SyncPro-SFT) | PO Acc@2s | OVO F1 CRR | OVO F1 SSR | OVO F1 REC |
|---|---|---|---|---|
| w/ ATR (2.5/0.2/3) | 38.80 | 28.06 | 21.36 | 47.69 |
| w/o ATR (2.5/0.2/3) | 36.40 | 25.66 | 10.22 | 34.08 |
| Δ | +2.40 | +2.40 | +11.14 | +13.61 |
Response to W4
Thank you for the suggestion. Inner adds about 20% per-frame latency (414.2 517.4 ms), and in return it improves content accuracy by 7.6 points, response rate by 8.4 points and Joint@2s by 7.2 points (all statistically significant), while 91.5% of frames still complete within 1 second.
-
The cost of Inner is small relative to its gains. To address your concern, we ablated Inner directly. Disabling it saves 103.3 ms per frame and raises the share of frames completed within 1 s from 91.5% to 99.9%, but lowers content accuracy by 7.6 points, response rate by 8.4 points and input-frame-anchored Joint@2s by 7.2 points. Because Inner is activated adaptively rather than at every step, this cost is paid only when the evidence changes.
-
Concurrent execution offsets part of the cost. Compared with running Inner first and then Response, running the two branches concurrently finishes 114.74 ms earlier (1.37× faster) and delivers the first response token 199.52 ms earlier (2.22× faster) (Table 6). The two branches produce identical outputs on all 1,870 held-out examples, so the speedup does not change the results.
-
Memory stays stable on long streams, and latency grows gradually with retained context. On a 480-s stream at 1 fps, persistent GPU allocation changes by only 1.1 MiB, and the per-frame peak stays at 11.75 GiB while retained KV grows from 2,126 to 11,220 tokens (Figure 6). Dual-branch decoding latency grows by 15.6 ms per 1k retained KV tokens when only Response is active and 32.4 ms when both branches are active, and 74.9% of frames meet the 1-s budget over the full stream. Evicting stale context is a straightforward way to flatten this growth.
Revision: Appendix I reports the quality and runtime cost of Inner updates (Table 13).
Response to Reviewer YV3X (3)
Response to Q1
Thank you for the concern. Inner annotations go through quality control beyond response reconstruction, and an independent audit of 100 trajectories directly measures their factuality (88.34% correct among verifiable facts) and cross-turn consistency (94.68%).
-
Inner annotations are generated without future information and then revised in several rounds. Each Inner entry is generated from the video truncated to the current moment. Multi-round revision then reviews content, timing and event coverage, removes redundant entries and adds missed state changes. Response reconstruction further checks that the retained internal states suffice to support later answers.
-
SyncPro-189K covers diverse scenarios and task types rather than a single state-summary template. It combines proactive query synthesis with adaptation of existing temporal annotations, spanning 12 video sources, 3 interaction functions and 8 task subtypes. The supervision includes factual QA, counting, state monitoring and continuous assistance.
-
Human review and an independent audit confirm quality. After three rounds of revision, we manually checked 5,000 synthesized trajectories, and human judgments agree with the LLM quality judgments in about 89.7% of cases. To address your concern about intermediate states, we audited 100 randomly sampled trajectories:
| Metric (100 trajectories, 803 atomic facts) | Result |
|---|---|
| Atomic facts supported by the visuals | 84.31% |
| Correct among verifiable facts | 88.34% |
| Cross-turn consistency | 94.68% |
Response to Q2
SyncPro stays within 2.0 points of its base model on general video understanding, and the lower subsets on ProactiveVideoQA trace to the focus of our supervision, with a clear path for improvement.
-
On WEB, SyncPro ranks second (48.28), and EGO has clear room for improvement. To address your concern, we audited 50 low-scoring response windows on each subset. The main WEB issue is incomplete key points (38%), and the main EGO issue is missed or late triggering (76%). Our annotations emphasize concise answers after an explicit trigger, while WEB's multi-point descriptions and EGO's long-horizon step tracking require broader content coverage and continuous updating. These are the directions for improvement.
-
General video understanding is largely preserved without mixing in general VQA data. Averaged over MVBench, EgoSchema and Video-MME, SyncPro-RL is 1.68 points below the base model, and only 0.06 points below SFT.
-
Other streaming adaptation work on different backbones also reports small drops on general benchmarks. StreamBridge reports a 2.6-point drop on MVBench (Qwen2-VL-7B) and a 2.0-point drop on LongVideoBench (LLaVA-OneVision-7B) [1], and MMDuet2 shows a small drop on LongVideoBench with Qwen2.5-VL-3B [2]. Our goal is continuous state updating with timely, correct responses: under the same backbone and data, SyncPro-SFT raises Joint@2s from 17.2% (Interleaved) to 30.8%, while keeping general ability close to the base model.
[1] StreamBridge: Turning Your Offline Video Large Language Model into a Proactive Streaming Assistant.
[2] MMDuet2: Enhancing Proactive Interaction of Video MLLMs with Multi-Turn Reinforcement Learning.
Response to Reviewer YV3X (1)
Response to W1
Response to W2
Inner maintains query-relevant state that supports later decisions. Measurements show it is factual, integrates multi-frame evidence, holds up on long streams, and transfers to a new benchmark without extra training.