- Rating
- 3
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes GDPO-Listener, a two-stage framework for expressive listener head-motion generation. It combines autoregressive flow matching with reinforcement learning post-training, using group-wise reward normalization to improve motion expressiveness and temporal dynamics. Experiments demonstrate improvements over existing methods.
Critical strengths
-
Relevant problem: Addresses the challenge of generating expressive and natural listener reactions in conversations.
-
Effective framework: Combines flow matching with reward-driven post-training in a coherent pipeline.
-
Promising results: Experiments and ablations demonstrate improvements over existing methods, including standard GRPO.
Critical weaknesses
- Limited methodological novelty: GRPO-based post-training has become increasingly common in generative modeling. Applying it to expressive head-motion generation, even with task-specific rewards, may represent a relatively incremental contribution. The methodological novelty of GDPO over existing policy optimization approaches could be better established.
- The necessity of reinforcement learning is not fully established. The proposed rewards are primarily based on explicitly defined motion statistics and reconstruction errors. It remains unclear whether the observed improvements require policy optimization or could be achieved through direct supervised fine-tuning with corresponding auxiliary losses. A comparison against such an alternative would help establish the specific value of the reinforcement learning stage.
- Limited validation of GDPO: The contribution of group-wise reward normalization and its sensitivity to design choices could be investigated further.
Questions for authors
- How does GDPO compare with supervised fine-tuning using the same reward components as auxiliary losses?
- How do you ensure that the reward design improves naturalness and conversational relevance rather than merely increasing motion variability?
- How sensitive is GDPO to the choice of parameter groups and reward weights?
- Why do speaking metrics improve if reinforcement learning is applied exclusively to listener frames?
2 comments
Thank you for the response. I believe a score of 3 is appropriate, and I will maintain my positive rating.
Response to Reviewer Tj9k
We thank the reviewer for their time and valuable feedback. Please find the responses to the raised concerns below:
1. Novelty (W1)
We respectfully clarify that our contribution goes beyond applying GRPO-based post-training. First, to our knowledge, this is the first work to introduce RL post-training to dyadic listening head generation, treating listener expressiveness as an alignment problem. Second, unlike previous autoregressive methods that rely on discrete motion tokens, we build a continuous Auto-Regressive Flow Matching model, and adapt RL to it with SDE rollouts. Third, we design dynamics-aware rewards with group-wise normalization over an expanded FLAME space. With these designs, we demonstrate clear improvements in addressing the problem of static listening head reactions, as shown in Tab. 1, Tab. 5, Tab. 9, Tab. 10 and Tab. 12.
2. Necessity of reinforcement learning (W2 & Q1)
Thank you for the suggestion. First, regarding the necessity of RL, we compare with supervised alternatives in Tab. 8, including the supervised AR-Flow model and AR-Flow with SFT on an expressive subset. Both perform worse than our GDPO post-training.
Second, regarding auxiliary losses, our rewards cannot be directly used as auxiliary losses in supervised training. They are sequence-level statistics computed on the full decoded motion (200 frames), which requires a complete autoregressive rollout. In contrast, AR-Flow is trained with teacher forcing, where each step only predicts a single latent token given the reference history, so no full sequence is generated during training. Instead, RL post-training naturally optimizes such sequence-level rewards on complete rollouts.
3. Validation and sensitivity of GDPO (W3 & Q3)
Thank you for the question. These analyses are provided in the appendix. For the contribution of group-wise reward normalization, we compare GDPO with standard GRPO in Tab. 12 and Fig. 6 (Appendix C.6). For the sensitivity to parameter groups, we conduct leave-one-out ablations in Tab. 10 (Appendix C.3), where the reward weight of each FLAME group is set to zero in turn. For the reward design, we further ablate the reward components in Tab. 9 (Appendix C.2). These results show that group-wise normalization and each reward design contribute to the final performance.
4. Naturalness beyond motion variability (Q2)
This is a fair concern. Our rewards not only encourage expressive motion dynamics, but also preserve naturalness and conversational relevance. Among the four reward terms, the variance and velocity-variance terms are responsible for expressive dynamics, while the mean and MSE terms serve as regularization to keep the motion natural and grounded in the conversational context. This is further verified by our user study in Tab. 5, where participants prefer our method on naturalness, liveliness and diversity. We also provide video demos in the supplementary materials for visual comparison.
5. Scope of RL post-training (Q4)
Thank you for pointing this out. The statement in Sec. 3 that GDPO is “applied exclusively to listener frames” is a writing mistake, and we have corrected it in the revised paper. GDPO post-training is applied to all frames, both speaking and listening, consistent with the reward description in Sec. 3.3.
Since our rewards are designed for motion dynamics, the speaking dynamic metrics also improve substantially (Tab. 7: FDD 39.54 → 22.45, PDD 3.50 → 2.71, JDD 1.81 → 1.26). In contrast, the lip-sync metric remains largely unchanged (Tab. 7: LVE 3.07 → 2.95), as lip motion is largely determined by the audio and is largely unaffected by RL post-training.