- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes EIP, a system that takes streaming user audio, dialogue history, and recent robot motion as input and generates spoken responses and upper-body gestures in real time.
The system has two main components:
- A frozen audio interaction model processes the user’s audio and dialogue history to generate a spoken response.
- A flow-matching motion model reads the interaction model’s hidden states, recent robot state, and speaking/intensity conditions to generate upper-body motion in 400 ms chunks.
The paper also constructs EI-Data from existing human interaction and gesture datasets and demonstrates the system on an Astribot S1 robot.
Critical strengths
- The problem is practical. A conversational robot needs to generate motion continuously and respond to changes between listening, speaking, and interruption.
- The paper demonstrates the complete system on a physical robot. This is useful evidence that the proposed integration can operate beyond offline motion generation.
Critical weaknesses
-
The motion conditioning could be better justified. The model does not explicitly use acoustic features of the robot’s generated speech. For lively, speech-aligned gestures, the speaking state and coarse intensity conditions seem insufficient. The gestures in the supplementary videos also do not look well aligned with the speech rhythm.
-
The paper reports better FGD and comparable BC. It would be helpful to add subjective comparisons with baselines to show whether these results also reflect how natural the gestures look and how well they align with speech.
Questions for authors
- Could you clarify whether any acoustic or prosodic features of the robot’s generated speech reach the motion model?
- Could you provide subjective comparisons with external baselines on gesture naturalness and speech alignment?