- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes RePO to address the conflict between survival and motion fidelity in PPO-based humanoid motion imitation. It combines critic-based GAE with step-wise group-relative advantages, survival gating, and kinematic pretraining. Experiments on AMASS and OMOMO demonstrate improvements over the selected PPO baselines.
Critical strengths
-
Interesting optimization perspective. Comparing multiple executions of the same reference motion provides a potentially useful learning signal for improving motion fidelity.
-
Empirical improvements. RePO improves both tracking accuracy and success rate over the selected PPO baselines, with ablations supporting the effectiveness of its components.
Critical weaknesses
-
Insufficient validation of the central hypothesis. The paper attributes tracking failures to survival-biased optimization but does not establish that this is a major bottleneck. More importantly, NAE uses the same survival-dependent rewards as GAE, and the survival gate does not explicitly eliminate this bias. The experiments do not demonstrate that NAE actually provides fidelity-specific credit assignment.
-
Outdated benchmarks and weak baselines. PHC+, MaskedMimic, and FARM have already reported near-perfect success rates on AMASS, whereas RePO achieves only 96.0% on training and 89.9% on testing. Although evaluation protocols may differ, the absence of comparisons with these stronger methods makes the significance of the improvements unclear.
Questions for authors
-
Mechanism clarification: Since NAE is computed from the same cumulative rewards as GAE, why should it specifically encourage fidelity rather than survival? Can the authors provide evidence that NAE better correlates with tracking accuracy after controlling for survival length?
-
Experimental validation: Can RePO improve stronger existing controllers such as PHC+ or MaskedMimic, or demonstrate substantial benefits on more challenging motion distributions? This would help establish whether the method addresses a meaningful remaining limitation in humanoid motion tracking.
3 comments
Response to jeXz (Part 2)
Ⅲ. Clarification of the Benchmark [W2, Q2]
We would like to clarify that our experimental setting differs from existing SOTA methods. We aim to train with single-stage RL, whereas these methods rely on complex multi-stage pipelines.
We would like to clarify that our work is to explore RL policy optimization for humanoid motion imitation. To ensure a fair comparison, we adopt vanilla PPO as the baseline, excluding additional training stages and specialized control mechanisms to isolate the benefits of RePO. We have highlighted these differences in Related Work.
For the challenging motions, Figure 4 of the main text shows that RePO substantially outperforms PPO on challenging datasets, suggesting that improving RL policy optimization is a promising direction for learning complex motions.
Furthermore, we refine the advantage estimation for different reward components, substantially improving RePO (+)'s performance, as shown in the table below. These results demonstrate that even minor modifications to our method can yield significant gains. We hope these findings can encourage further exploration of policy optimization as a valuable direction for humanoid motion imitation.
| Methods | Succ | Time | ||
|---|---|---|---|---|
| RePO | 22.7% | 93.29 | 0.113 | 0.172 |
| RePO(+) | 51.1% | 126.05 | 0.130 | 0.179 |
Look forward to your feedback
Dear Reviewer jeXz,
Thank you again for your valuable feedback and constructive suggestions.
As the discussion period is coming to an end, we would like to kindly ask whether our responses and additional experiments have adequately addressed your concerns, or if there are any remaining questions we could clarify. We would greatly appreciate any further feedback.
Thank you for your time and consideration!
Response to jeXz (Part 1)
We thank you for recognizing our proposed method as "interesting" and "potentially useful". Your suggestions are also valuable in improving this paper.
Ⅰ. Clarifying Fidelity as a Major Bottleneck [W1]
Our central claim is that survival-biased optimization prioritizes tracking success at the cost of motion fidelity, rather than causing tracking failures.
In our visualization results, red spheres represent the reference motion, and yellow bodies show the policy execution. Better alignment between them indicates higher motion fidelity.
For example, in Figure 1, PPO fails to accurately track the leg motion during takeoff, leading to insufficient jump height and early termination, whereas RePO achieves higher tracking fidelity throughout the jump. Similarly, in the last two rows of Figure 3, RePO more closely follows the reference leg trajectories during dancing, while PPO shows substantial deviations.
Beyond the qualitative results from our visualization results, we further provide quantitative results of RePO's fidelity advantage. Specifically, we adjust the tracking-error termination threshold to compare the success rates of RePO and PPO at comparable fidelity levels. Tracking errors are computed only over successfully tracked motions. As shown in the table below, RePO achieves a substantially higher success rate under matched fidelity levels. This result suggests that PPO’s tracking success comes at the expense of motion fidelity.
Ⅱ. Clarifying NAE's Optimization Preference Toward Fidelity [W1, Q1]
Notably, NAE favors motion fidelity over survival, despite using survival-dependent rewards. This is because the advantage estimates we use have little survival length bias.
Specifically, Eqs. (6)–(8) estimate cumulative tracking fidelity advantages, influenced by survival length and per-step fidelity. When survival lengths are comparable within each group (ensured by survival gate (SG)), NAE favors fidelity-oriented optimization. In contrast, GAE estimates advantages independently for each environment. Longer trajectories tend to accumulate higher returns, leading to a strong survival preference. We also apply SG to GAE in RePO to exclude the influence of SG. As shown in the second row of the table below, this substantially reduces the success rate.
For the other experimental evidences, the table below shows that removing SG substantially degrades RePO's performance. SG restricts NAE comparisons to trajectories with similar survival lengths, reducing survival bias and allowing NAE to better emphasize motion fidelity.
As shown in the table below, GRPO achieves lower tracking errors than PPO across all four fidelity metrics on successfully tracked motions, despite its lower success rate. This suggests that GAE favors survival length, while NAE places greater emphasis on motion fidelity.
In addition, the ablation study in Figure 3 shows that reducing the weight of NAE leads to increased tracking errors. Together, these results support NAE's preference for optimizing motion fidelity.
Furthermore, we replace the return-to-go (Eq. 6) in NAE with single-step rewards (fidelity without survival information) to examine the role of future steps in fidelity estimation. As shown in the table below, this modification substantially reduces success rates and increases tracking errors, indicating that incorporating future steps leads to more accurate advantage estimates.