- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Live-OPSD trains proactive streaming models on their own response continuations at recorded contexts. The same model, conditioned on privileged hints, supplies distillation targets, while Decision-Aware Selection prioritizes training points. Evaluation on two Qwen-Omni backbones shows improved average LiveProBench scores over continued SFT.
Critical strengths
- Privileged-context distillation provides token-level feedback on current-model continuations without an additional inference-time teacher or gate.
- Matched-backbone comparisons and divergence/selection ablations support the approach.
Critical weaknesses
-
The source of the gains is insufficiently isolated. Live-OPSD receives externally annotated hints unavailable to the SFT baselines. Without hint and rollout controls, the contribution of on-policy distillation cannot be separated from additional privileged supervision. Hint generation also uses full-clip access, and the annotation-model identity is not reported.
-
The on-policy scope and baseline coverage are limited.
- Only the current continuation is generated on policy; earlier assistant turns remain recorded. This leaves deployment-history errors outside the training distribution.
- Comparisons omit timing-oriented RL, including StreamPro's GRPO stage. CB-SFT includes only its supervised objective, so the results do not establish an advantage over policy-optimization alternatives.
Questions for authors
- Can you compare no-hint distillation, hinted distillation on reference prefixes, and full Live-OPSD to isolate the effects of hints and on-policy generation? How are hints checked for future-information leakage?
- How does performance change when earlier assistant turns are model-generated, particularly after premature or missed responses?
- Can you compare against a timing-reward RL baseline using the same backbone, data, initialization, and compute budget?
3 comments
Responce to Reviewer qmWh (Part 2/3)
2. Earlier assistant turns generated by the model (Weakness 2, Question 2)
The effect of self-generated preceding turns is already evaluated by the “- Self-generated prefix” control in Table R1. Compared with Live-OPSD using preconstructed histories, actor-generated histories reduce the LiveProBench average from 59.64 to 57.20. The Duplicate Counting (DC) score also decreases from 34.33 to 32.84, indicating a drop in the model’s ability to handle tasks requiring repeated proactive responses. This control includes histories in which earlier turns may contain premature or missed responses, and shows how using such histories affects subsequent performance.
3. Comparison with timing-reward RL (Weakness 2, Question 3)
We compare Live-OPSD with GRPO using the same Qwen3-Omni backbone, initialization, and training data. For GRPO, each annotated response point defines a timing window extending from that point through the following two seconds; responses within this window receive an IoU-based timing reward.
Live-OPSD uses one rollout per example naturally, while GRPO uses eight as it need to estimate group-relative advantages. Both runs use 121 update steps and a learning rate of 5e-6. We report the optimization settings so that the scope of the comparison is clear.
Table R3. Comparison on all LiveProBench metrics (Qwen3-Omni-30B-A3B, %).
| Method | ES Resp. | ES Sil. | ES Geo. | OR | WR | AA | SS | DC | Avg. |
|---|---|---|---|---|---|---|---|---|---|
| Live-OPSD | 40.28 | 99.59 | 63.34 | 61.93 | 70.00 | 68.49 | 59.78 | 34.33 | 59.64 |
| GRPO | 38.43 | 97.97 | 61.36 | 57.80 | 75.00 | 70.59 | 42.39 | 10.45 | 52.93 |
Table R4. Training settings.
| Method | Rollouts per example | Update steps | Learning rate |
|---|---|---|---|
| Live-OPSD | 1 | 121 | 5e-6 |
| GRPO | 8 | 121 | 5e-6 |
This comparison illustrates the different training signals used by the two methods. GRPO relies on a trajectory-level timing reward. Designing this reward requires distinguishing premature responses from responses made within the annotated window and delayed responses outside it. Since the reward is assigned at the trajectory level, it also provides limited credit assignment when a multi-turn trajectory contains several response decisions with different outcomes. In addition, GRPO’s group-based update requires multiple rollouts of a multi-turn video stream, while Live-OPSD only needs one actor rollout per example.
Responce to Reviewer qmWh (Part 3/3)
4. How GRPO/RLVR and Live-OPSD differ, and why we consider OPSD may especially suited to proactive streaming video models (Question 3)
The considerations below motivated our choice of Live-OPSD but were not discussed in sufficient detail in the paper due to space constraints. We are grateful that the reviewer recognized this underlying motivation and prompted us to articulate it more clearly.
GRPO/RLVR:
- Requires a timing-sensitive reward that distinguishes timely, evidence-supported responses from premature responses that may be hallucinated and delayed responses that miss the appropriate interaction window. Designing such a reward is challenging.
- Standard policy-gradient updates often distribute the same advantage across all tokens in a trajectory. Yet a trajectory may contain multiple proactive responses, some appropriate and others not, so a single outcome reward provides weak credit assignment to the individual decisions.
- Tasks requiring repeated proactive responses, such as counting push-ups, make this problem harder: the reward must assess multiple response events and their timing. More elaborate rewards may increase the risk of reward hacking, while sparse rewards and coarse credit assignment can still limit learning.
- RLVR requires multiple sampled rollouts to estimate group-relative advantages. In multimodal streaming video, each rollout must process a multi-turn video stream, making this a particularly time-consuming part of training.
Live-OPSD:
- Uses hinted token-level teacher targets instead of a manually designed outcome reward, provided the teacher supplies a useful target that improves on the unhinted student’s output.
- Provides token-level distillation targets, allowing training to focus on tokens relevant to proactive response decisions and content, rather than assigning one trajectory-level advantage to every token.
- This fine-grained supervision naturally supports examples requiring multiple proactive responses within one video, since it can guide the relevant response tokens at each decision point.
- Does not require a group of sampled rollouts to estimate relative rewards. In our training setup, it uses one on-policy actor rollout per example, reducing the heavy rollout cost(especially for multimodal examples) compared with multi-sample GRPO.
- Real-time proactive streaming video models may introduce model-specific control tokens (speak/slience token), making it difficult to find a stronger external teacher with a compatible vocabulary and output structure. Live-OPSD uses the same model as a teacher, avoiding the need for a separate and stronger teacher model.
We thank the reviewer again and we hope these analyses make the source of the gains and the scope of on-policy training explicit. We are looking forward to your response.
Responce to Reviewer qmWh (Part 1/3)
We thank the reviewer for the constructive suggestions. We respond to each concern below.
1. Privileged hints and prefix source (Weakness 1, Question 1)
In our current rollout setup, preceding assistant turns come from preconstructed interaction histories, while the current policy generates the continuation at the target decision point. To isolate the effect of prefix source, we compare hinted distillation on preconstructed histories with the same hinted distillation on self-generated preceding histories. Both settings use the same point-specific hints and dynamically updated actor as the teacher, with the remaining training settings matched as closely as possible.
We do not include a no-hint distillation control. In our current setup, removing the hint while using the current policy as both teacher and student would produce no meaningful teacher–student learning signal. We also do not have a stronger external proactive streaming model with a compatible response protocol to use as a no-hint teacher (See Point 4). We therefore do not attribute the full gain over continued SFT to prefix source alone. The prefix comparison below isolates the effect of preceding-history source, while the hint-component ablation examines which information in the hint is useful.
The use of preconstructed preceding turns is intentional: it lets us study the target-turn distillation signal without earlier actor errors changing the context for later turns. This design was inspired by SOD [1], which reports that errors in multi-step agent trajectories can cascade, increase student–teacher divergence, and make later token-level supervision less reliable. This motivates our controlled-history design.
Table R1. Prefix-source comparison on LiveProBench (Qwen3-Omni-30B-A3B, %).
The “- Self-generated prefix” control keeps the same point-specific hints and dynamically updated actor as the teacher, but replaces the preconstructed preceding turns with actor-generated turns. The target continuation is generated by the current policy in both settings. The LiveProBench average is 2.44 points lower with self-generated preceding turns.
Our hints describe the currently available evidence and indicate whether the assistant should respond or remain silent. To examine the contribution of these hint components, we compare the full hint with variants that remove the response/silence recommendation or the stream description.
Table R2. Hint-component ablations on LiveProBench (Qwen3-Omni-30B-A3B, %).
Details of data generation. The external annotator is Qwen3.5-Omni. When generating a hint for decision point t, its video input contains only the stream available up to t; it is not given future frames or the complete clip. Full-clip access is used only in a separate post-generation review of hint quality and temporal appropriateness. The student remains restricted to the current streaming context, and the deployed model receives no hints. The continued-SFT baselines use the same 3,712 training examples and their reference responses.
The 5,000 candidate examples for Live-OPSD were randomly sampled from the SFT data. We retained 3,712 examples for OPSD training, reserved 50 for testing, and discarded 1,238 after example-level quality filtering. If any turn in an example contained erroneous or unsuitable data or a hint, we discarded the entire example.
Live-OPSD data filtering and train/test split.
[1] Q. Zhong, M. Zheng, M. Song, X. Lin, J. Sun, H. Jiang, X. Wang, and J. Fang. “SOD: Step-wise On-policy Distillation for Small Language Model Agents.” arXiv:2605.07725, 2026.