- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper presents ConfabMuse, a diffusion Transformer framework for controllable two-person conversational video generation. Given a reference image, two audio streams, and optional text guidance, the framework aims to generate realistic talking avatars with appropriate speech synchronization and coordinated interactions. The proposed method consists of three components: a Turn-Context Audio Adapter that models speech activity and conversational turns, an Audio-Vision Binding module that associates audio streams with their corresponding avatars, and a Scene-Grounded Interaction Planner that uses structured interaction graphs to control evolving actions and relations. The authors conduct extensive experiments across single-person and multi-person talking-avatar benchmarks. The results demonstrate competitive visual quality and audio-visual synchronization, along with improvements in conversational interactivity and structured interaction control.
Critical strengths
- Clear motivation and coherent framework design. The paper addresses an important challenge in multi-person conversational video generation: generating natural interactions requires more than simply associating audio streams with visible avatars. The decomposition into conversational timing, audio-avatar binding, and evolving interaction control is intuitive and well motivated. I particularly appreciate the Scene-Grounded Interaction Planner, which explicitly models avatar actions and relations across conversational turns while preserving ongoing behaviors.
- Comprehensive experimental evaluation and promising performance. The paper provides extensive evaluations across multiple single-person and multi-person benchmarks, covering generation quality, audio-visual synchronization, motion coordination, and interaction controllability. The proposed framework achieves strong results across most reported metrics. The ablation studies further demonstrate the effectiveness of the proposed components, particularly in reducing unintended listener activation and improving interaction transitions. The experimental section is a significant strength of the paper.
Critical weaknesses
- The contribution of the structured interaction planner could be more clearly isolated. Although the Scene-Grounded Interaction Planner is an interesting component, its advantages over simpler forms of temporally structured conditioning are not fully established. In Table 6, the authors compare the proposed planner with simple and expanded text prompts. However, it remains unclear whether these baselines receive equivalent information about avatar-specific actions and transition timing. A comparison with turn-aligned text guidance containing the same interaction information would better clarify the benefit of the proposed graph representation and planning mechanism.
- The proposed interaction-control metrics lack human validation. The paper introduces several MLLM-based metrics to evaluate interaction states, transitions, gaze, and temporal consistency. While these metrics are well motivated and provide more targeted measurements than conventional video-generation metrics, their reliability remains somewhat uncertain. The authors appropriately include a cross-evaluator comparison with InternVL3.5, which produces consistent method rankings. Nevertheless, agreement between automated evaluators does not fully establish their correspondence with human judgments, particularly for subtle behaviors such as gaze and conversational responses. A small human evaluation would provide additional confidence in the reported improvements.
Questions for authors
Q1. Could the authors provide a more controlled comparison to clarify the contribution of the Scene-Grounded Interaction Planner? For example, how would the framework perform with turn-aligned text instructions containing the same avatar actions and transition timing, but without the structured graph representation? Q2. The paper introduces several MLLM-based metrics for interaction controllability. Although the cross-evaluator analysis is encouraging, have the authors considered validating these metrics against human judgments? In particular, how reliably do the proposed metrics capture subtle conversational behaviors such as gaze direction, listener responses, and interaction transitions?