- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DuplexAvatar, a unified speech–language–motion framework for controllable full-duplex conversational avatars. Its central interface is an Executable Behavior Plan (EBP), which specifies behavioral intent, spatial and temporal support, and preservation constraints. A typed executor realizes precise geometric operations, while a masked residual-flow module handles open-ended semantic edits over a speech-driven motion base. Motion-only readback and geometric verification support bounded local correction before commitment. At runtime, the system separates committed motion from pending future output, allowing interventions to revise uncommitted behavior while re-encoding executed motion into the shared conversational state. To train and evaluate this framework, the authors compile SPEAR, a multi-source, geometry-grounded supervision suite and intervention-based benchmark, evaluating the system across motion generation, motion understanding, and behavioral control.
Critical strengths
1.Well-motivated control architecture. Separating exact geometric operations from learned semantic realization is a sensible way to combine precise control with expressive co-speech motion. Explicit preservation constraints and commitment boundaries address concrete requirements of interactive avatars. Integrating these mechanisms alongside executed-state feedback within a shared causal conversational backbone is a well-considered system design.
2. Good offline behavioral-control results. Table 1 reports 78.95% constraint success, compared with 59.13% for SimMotionEdit and 51.68% for MotionReFit. These improvements are accompanied by substantially lower preservation drift (8.64 mm) and target error (18.73 mm), indicating that the method effectively localizes requested edits while preserving unrelated motion outside the specified support.
3. Ablations and dataset organization support component utility. Removing EBP, the typed executor, masked residual flow, or readback correction reduces constraint success to 31.58%, 52.63%, 60.53%, and 68.42%, respectively. These results provide substantive evidence that the proposed planning, realization, and verification modules work complementarily. In addition, organizing heterogeneous datasets into unified plan views under a speaker- and recording-disjoint splitting protocol is a sound experimental design.
Critical weaknesses
- The central full-duplex claim lacks quantitative validation. The paragraph on page 9 ("Full-duplex interaction") introduces time to first audio (TTFA), time to first motion (TTFM), real-time factor (RTF), and barge-in recovery as the evaluation metrics for online interaction. However, neither the main paper nor the supplementary material reports any numerical results, tables, or curves for these measures.
- Generation evaluation and SPEAR documentation leave key details underspecified. For speech-conditioned motion generation, Table 1 reports only upper-body smoothness, beat consistency, and human preference. While useful, these metrics do not separately characterize distributional realism (e.g., FGD), generation diversity, or facial quality. Smoothness can favor damped, low-amplitude motion, while beat consistency measures rhythmic alignment rather than semantic appropriateness.
- The comparisons do not sufficiently isolate the incremental value of the conversational control interface. EMAGE and SemTalk are reasonable co-speech generation baselines, and StreamTalk provides a relevant streaming reference. However, these comparisons do not isolate the benefit of EBP, typed/residual execution, verification, and commitment over a compatible conversational architecture. The paper itself discusses closely related full-duplex and conversational behavior systems such as DuplexSLA, ViBES, Motion-Omni, and DyaPlex. While differences in input modalities and body coverage make an unadapted end-to-end comparison across all of them impractical, comparing on shared capabilities would clarify what the structured EBP and verification loop add beyond backbone scaling and multi-view SPEAR supervision.
Questions for authors
- Can you provide the missing quantitative results for the online full-duplex evaluation (TTFA, TTFM, RTF, and barge-in recovery), along with hardware specifications, precise timing definitions, and an evaluation of motion continuity at the commitment boundary?
- What are the test-set sizes for generation, understanding, and control? What participant counts, trial counts, and comparison protocol produced the 73% human-preference result?