- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes StreamPro-Duplex, a full-duplex multimodal agent framework for always-on streaming interaction. The system decouples real-time interaction control from understanding through a plug-and-play Duplex Controller, introduces persistent multi-intent management, and trains a unified Intent Monitor that jointly monitors multiple proactive intents. The work further constructs StreamInteract-138K, covering full-duplex and multi-intent interaction scenarios.
Critical strengths
- The paper presents a practical framework for integrating full-duplex control with persistent multi-intent streaming interaction. The separation between the Duplex Controller, intent management, and downstream monitoring/generation produces a relatively clean modular system.
- The paper constructs StreamInteract-138K, a dataset covers diverse full-duplex interaction behaviors as well as concurrent and evolving proactive intents, providing large-scale supervision for aspects of always-on interaction.
Critical weaknesses
- The paper is motivated by always-on interaction, where multiple real-time and proactive intents may coexist, persist, and evolve, and addresses this challenge through decoupled full-duplex control and persistent multi-intent management. However, the Duplex Controller appears conceptually close to turn-taking works [1] while the Intent Manager resembles persistent task-memory or task-state management in recent agent systems [2, 3, 4]. Given these conceptual overlaps, the paper could more clearly discuss in the Related Work what fundamentally distinguishes the design.
- The current experiments do not clearly disentangle the contributions of the backbone models, task-specific training, and the proposed system design. For example, in Table 1, the Duplex Controller is initialized from Qwen3-Omni-30B-A3B, but the unadapted backbone is not included as a baseline on Full-Duplex-Bench. Table 2 further shows that the adapted controller has a higher WER than the original Qwen3-Omni backbone. Similarly, the strong “training-free” results in Table 4 use Qwen3.6-35B-A3B as the backbone for the Memory Manager and Real-Time Generator, while the compared methods use heterogeneous backbones and training regimes. Without a same-backbone comparison or ablations, it remains unclear whether the gains primarily arise from the strong backbone or from the proposed framework.
- The experiments do not directly validate some of the paper’s central multi-intent claims. The claimed efficiency advantage of the unified Intent Monitor is not evaluated against a per-intent monitoring baseline as the number of active intents increases. There is also limited quantitative evaluation of persistent intent management under nested or heavily interleaved interactions.
[1] A Survey of Recent Advances on Turn-taking Modeling in Spoken Dialogue Systems, ACL 2025
[2] IPIBench: Evaluating Interactive Proactive Intelligence of MLLMs under Continuous Streams, to be accepted NIPS 2026
[3] Proactive Assistant Dialogue Generation from Streaming Egocentric Videos, EMNLP 2025
[4] Memento: Toward an All-Day Proactive Assistant for Ultra-Long Streaming Video, ICLR2026
Questions for authors
- Could the authors clarify the key methodological distinctions between the proposed Duplex Controller / Intent Manager and closely related full-duplex control and persistent task-state management paradigms? In particular, which capabilities are fundamentally enabled by the proposed design and how they influence always-on tasks?
- Could the authors provide more controlled experiments to disentangle the contributions of the backbone models, task-specific training, and the proposed framework?
- Could the authors provide additional experimental evidence or analysis to more directly substantiate the central multi-intent claims, particularly regarding persistent intent management and the effectiveness and efficiency of unified multi-intent monitoring under complex interleaved interactions?
- Could the authors provide the complete category-level results for MOSS-VL-Realtime and Streamo in Table 4, if available, or clarify why some results are not reported?
2 comments
Thanks for the responses. The additional concurrent-monitoring experiments with Intent Monitor address some of my concerns about its effectiveness. However, my main concerns remain the methodological distinctions and the attribution of the reported gains:
-
Methodological distinctions. Decoupling control from generation is a common idea in turn-taking. Might the novelty instead lie in more specific differences in state representation, decision-making, and training methods? The characterization of IPI-Agent may also need clarification. It already supports task-state management through Proactive Memory, Interaction Memory, and an Intent Router, and processes different tasks independently and in parallel.
-
Attribution of gains remains unclear. For Table 4, the additional matched-backbone comparison quantifies positive gains of +2.1/+1.3 from the framework, but also highlights that the backbone itself is already strong. The claims about the framework's advantages in Table 4 caption and main text should therefore be more carefully qualified. A similar attribution issue arises for Intent Monitor: Table 3 compares models such as Qwen3-VL-8b, but does not include its original Qwen3.6-35B-A3B backbone. This makes it difficult to disentangle the contributions of the backbone, training, and framework.
-
Could you please list the per-category scores for MOSS-VL-Realtime?
Dear Reviewer WpJp,
Thank you again for your valuable feedback. We would like to kindly check whether our previous response has addressed your concerns, or if there are any remaining questions or issues that we could further clarify.
We would greatly appreciate any additional feedback.