acceptodds
Under review as a conference paper at ICLR 2027

RePO: Reference-Relative Policy Optimization for High-Fidelity Humanoid Motion Imitation

Abstract

Physics-based humanoid controllers trained with Proximal Policy Optimization (PPO) have achieved high success rates in tracking diverse reference motions. However, successful execution does not necessarily imply faithful imitation: policies can increase cumulative rewards by surviving longer while adopting safer behaviors that deviate from the reference. To address this mismatch, we propose Reference-Relative Policy Optimization (RePO), which combines a critic-based viability advantage to support robust, long-horizon physical execution with a group-relative fidelity advantage to encourage faithful reference tracking. Instead of assigning a coarse trajectory-level score to each rollout, RePO normalizes returns-to-go across executions of the same motion at each aligned reference timestep, yielding dense, step-wise group-relative advantages that are blended with the viability advantage for policy updates. To reduce interference from survival differences, we introduce a survival gate that activates the fidelity advantage only when group-level execution is sufficiently reliable. We further adopt a kinematic cold start to initialize the policy from motion data, providing a reference-following prior for early group comparisons. Experiments on AMASS demonstrate improved motion fidelity and execution success without PHC's progressive training strategy. Furthermore, we demonstrate that our approach can facilitate the learning of complex motions. We validate this capability through challenging human–object interaction experiments on OMOMO, where RePO enables direct full-dataset training while the InterMimic fails to learn an effective policy.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Humanoid Imitation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_jeXzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RePO to address the conflict between survival and motion fidelity in PPO-based humanoid motion imitation. It combines critic-based GAE with step-wise group-relative advantages, survival gating, and kinematic pretraining. Experiments on AMASS and OMOMO demonstrate improvements over the selected PPO baselines.

Critical strengths
  1. Interesting optimization perspective. Comparing multiple executions of the same reference motion provides a potentially useful learning signal for improving motion fidelity.

  2. Empirical improvements. RePO improves both tracking accuracy and success rate over the selected PPO baselines, with ablations supporting the effectiveness of its components.

Critical weaknesses
  1. Insufficient validation of the central hypothesis. The paper attributes tracking failures to survival-biased optimization but does not establish that this is a major bottleneck. More importantly, NAE uses the same survival-dependent rewards as GAE, and the survival gate does not explicitly eliminate this bias. The experiments do not demonstrate that NAE actually provides fidelity-specific credit assignment.

  2. Outdated benchmarks and weak baselines. PHC+, MaskedMimic, and FARM have already reported near-perfect success rates on AMASS, whereas RePO achieves only 96.0% on training and 89.9% on testing. Although evaluation protocols may differ, the absence of comparisons with these stronger methods makes the significance of the improvements unclear.

Questions for authors
  1. Mechanism clarification: Since NAE is computed from the same cumulative rewards as GAE, why should it specifically encourage fidelity rather than survival? Can the authors provide evidence that NAE better correlates with tracking accuracy after controlling for survival length?

  2. Experimental validation: Can RePO improve stronger existing controllers such as PHC+ or MaskedMimic, or demonstrate substantial benefits on more challenging motion distributions? This would help establish whether the method addresses a meaningful remaining limitation in humanoid motion tracking.

3 comments
AuthorsOpenReview ↗

Response to jeXz (Part 1)

We thank you for recognizing our proposed method as "interesting" and "potentially useful". Your suggestions are also valuable in improving this paper.

Ⅰ. Clarifying Fidelity as a Major Bottleneck [W1]

Our central claim is that survival-biased optimization prioritizes tracking success at the cost of motion fidelity, rather than causing tracking failures.

In our visualization results, red spheres represent the reference motion, and yellow bodies show the policy execution. Better alignment between them indicates higher motion fidelity.

For example, in Figure 1, PPO fails to accurately track the leg motion during takeoff, leading to insufficient jump height and early termination, whereas RePO achieves higher tracking fidelity throughout the jump. Similarly, in the last two rows of Figure 3, RePO more closely follows the reference leg trajectories during dancing, while PPO shows substantial deviations.

Beyond the qualitative results from our visualization results, we further provide quantitative results of RePO's fidelity advantage. Specifically, we adjust the tracking-error termination threshold to compare the success rates of RePO and PPO at comparable fidelity levels. Tracking errors are computed only over successfully tracked motions. As shown in the table below, RePO achieves a substantially higher success rate under matched fidelity levels. This result suggests that PPO’s tracking success comes at the expense of motion fidelity.

MethodsEg↓E_g \downarrowEl↓E_l \downarrowEacc↓E_{acc} \downarrowEvel↓E_{vel} \downarrowSucc ↑\uparrow
RePO31.924.22.84.196.0%
PPO32.326.32.84.184.1%

Ⅱ. Clarifying NAE's Optimization Preference Toward Fidelity [W1, Q1]

Notably, NAE favors motion fidelity over survival, despite using survival-dependent rewards. This is because the advantage estimates we use have little survival length bias.

Specifically, Eqs. (6)–(8) estimate cumulative tracking fidelity advantages, influenced by survival length and per-step fidelity. When survival lengths are comparable within each group (ensured by survival gate (SG)), NAE favors fidelity-oriented optimization. In contrast, GAE estimates advantages independently for each environment. Longer trajectories tend to accumulate higher returns, leading to a strong survival preference. We also apply SG to GAE in RePO to exclude the influence of SG. As shown in the second row of the table below, this substantially reduces the success rate.

MethodsSucc ↑\uparrowEg↓E_g \downarrowEl↓E_l \downarrowEacc↓E_{acc} \downarrowEvel↓E_{vel} \downarrow
RePO96.0%34.525.93.24.6
RePO (w/ SG for GAE)79.1%42.032.74.45.8

For the other experimental evidences, the table below shows that removing SG substantially degrades RePO's performance. SG restricts NAE comparisons to trajectories with similar survival lengths, reducing survival bias and allowing NAE to better emphasize motion fidelity.

MethodsSucc ↑\uparrowEg↓E_g \downarrowEl↓E_l \downarrowEacc↓E_{acc} \downarrowEvel↓E_{vel} \downarrow
RePO96.0%34.525.93.24.6
RePO (w/o SG)94.6%34.326.23.34.6

As shown in the table below, GRPO achieves lower tracking errors than PPO across all four fidelity metrics on successfully tracked motions, despite its lower success rate. This suggests that GAE favors survival length, while NAE places greater emphasis on motion fidelity.

MethodsSucc ↑\uparrowEg↓E_g \downarrowEl↓E_l \downarrowEacc↓E_{acc} \downarrowEvel↓E_{vel} \downarrow
PPO (GAE)94.5%35.527.93.24.6
GRPO (NAE)72.3%31.227.12.74.0
RePO96.0%31.924.22.84.1

In addition, the ablation study in Figure 3 shows that reducing the weight of NAE leads to increased tracking errors. Together, these results support NAE's preference for optimizing motion fidelity.

Furthermore, we replace the return-to-go (Eq. 6) in NAE with single-step rewards (fidelity without survival information) to examine the role of future steps in fidelity estimation. As shown in the table below, this modification substantially reduces success rates and increases tracking errors, indicating that incorporating future steps leads to more accurate advantage estimates.

MethodsSucc ↑\uparrowEg↓E_g \downarrowEl↓E_l \downarrowEacc↓E_{acc} \downarrowEvel↓E_{vel} \downarrow
RePO96.0%34.525.93.24.6
RePO (w/ Step Reward)86.4%57.239.84.57.5
AuthorsOpenReview ↗

Response to jeXz (Part 2)

Ⅲ. Clarification of the Benchmark [W2, Q2]

We would like to clarify that our experimental setting differs from existing SOTA methods. We aim to train with single-stage RL, whereas these methods rely on complex multi-stage pipelines.

We would like to clarify that our work is to explore RL policy optimization for humanoid motion imitation. To ensure a fair comparison, we adopt vanilla PPO as the baseline, excluding additional training stages and specialized control mechanisms to isolate the benefits of RePO. We have highlighted these differences in Related Work.

For the challenging motions, Figure 4 of the main text shows that RePO substantially outperforms PPO on challenging datasets, suggesting that improving RL policy optimization is a promising direction for learning complex motions.

Furthermore, we refine the advantage estimation for different reward components, substantially improving RePO (+)'s performance, as shown in the table below. These results demonstrate that even minor modifications to our method can yield significant gains. We hope these findings can encourage further exploration of policy optimization as a valuable direction for humanoid motion imitation.

MethodsSucc ↑\uparrowTime ↑\uparrowEh↓E_h \downarrowEo↓E_o \downarrow
RePO22.7%93.290.1130.172
RePO(+)51.1%126.050.1300.179
AuthorsOpenReview ↗

Look forward to your feedback

Dear Reviewer jeXz,

Thank you again for your valuable feedback and constructive suggestions.

As the discussion period is coming to an end, we would like to kindly ask whether our responses and additional experiments have adequately addressed your concerns, or if there are any remaining questions we could clarify. We would greatly appreciate any further feedback.

Thank you for your time and consideration!

Reviewer_75GYOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that PPO-based physics humanoid imitation can favor policies that survive longer while deviating from the reference motion because of cumulative rewards and early termination. It proposes RePO, which combines critic-based GAE as a physical-viability signal with a group-relative return signal computed across multiple rollouts of the same reference motion at aligned time steps. A survival gate activates the fidelity signal only when group execution is sufficiently reliable. The method also uses kinematic cold-start initialization and an EMA-updated reference policy for regularization. On AMASS, RePO achieves an 89.9% test success rate versus 87.0% for PHC, while reducing several tracking errors. On OMOMO, direct full-dataset training reaches 22.7% success, whereas the reported InterMimic plus vanilla PPO baseline fails to converge.

Critical strengths
  1. The problem diagnosis is clear and the method is well aligned with it. The paper explicitly separates physical viability from motion fidelity and addresses them with GAE, a step-wise group-relative advantage, and a survival gate rather than relying on a single opaque reward modification.
  2. The evaluation covers both standard imitation and contact-rich interaction. RePO improves success and tracking metrics on AMASS and shows some ability to handle complex human-object interaction without PHC's progressive training strategy.
Critical weaknesses
  1. The OMOMO comparison uses a relatively weak baseline. The main comparison is against InterMimic trained with vanilla PPO, which has 0% success. Stronger and compute-matched imitation or contact-control baselines are missing, so the 22.7% success rate does not establish a broad advantage on this benchmark.
  2. The method introduces additional parallel-rollout and hyperparameter requirements. It needs multiple rollouts for each reference motion, group statistics, a survival threshold τ, and a fidelity weight λ. Although some K and λ ablations are reported, the paper does not systematically quantify the added compute, variance, or behavior under different motion lengths and dataset scales.
Questions for authors

See weakness

2 comments
AuthorsOpenReview ↗

Response to 75GY

We thank the reviewer for recognizing that “The problem diagnosis is clear and the method is well aligned with it”. Your suggestions are also valuable for improving this work.

Ⅰ. Clarification of Benchmark Selection [W1]

We would like to clarify that our OMOMO setting differs from existing SOTA methods, such as InterMimic. We aim to train directly on the full dataset using single-stage RL, whereas these methods rely on complex multi-stage pipelines. Although RePO could be integrated into their RL stages, reproducing these pipelines is computationally expensive (17 teacher models, each requiring approximately four days on a single NVIDIA L40S GPU, plus an additional distillation stage). Our limited computational resources make it difficult to conduct such comparisons within the rebuttal period.

Moreover, our goal on OMOMO is to explore the potential of RL policy optimization for complex humanoid imitation. To ensure a fair comparison, we adopt InterMimic with vanilla PPO as the baseline, excluding additional training stages and specialized control mechanisms to isolate the benefits of RePO.

Furthermore, we refine the advantage estimation for different reward components, substantially improving RePO (+)'s performance, as shown in the table below. These results demonstrate that even minor modifications to our method can yield significant gains. We hope these findings can encourage further exploration of policy optimization as a valuable direction for humanoid motion imitation.

MethodsSucc ↑\uparrowTime ↑\uparrowEh↓E_h \downarrowEo↓E_o \downarrow
RePO22.7%93.290.1130.172
RePO(+)51.1%126.050.1300.179

Ⅱ. Computational Cost [W2]

Notably, RePO introduces only modest computational overhead. The table below compares the key parameters affecting the computational cost of PPO and RePO. For a fair comparison, we keep these parameters identical, with RePO increasing the average epoch time by only 8.9%. For group-based rollouts, we group existing parallel simulation environments rather than performing multiple rollouts in each environment.

Different motion lengths and dataset scales do not introduce additional computational overhead, as the number of environments and rollout horizon are fixed for each epoch.

MetricsRePOPPORePO/PPO
num envs30723072-
horizon length3232-
mini-batch size16,38416,384-
mini-epochs per epoch66-
training epochs2000020000-
mean epoch time (s/epoch)3.5883.2961.089x

Ⅲ. Statistical reliability of RePO [W2]

Regarding statistical reliability, we conduct five independent training runs with different seeds. As shown in the table below, RePO consistently achieves stable performance across these runs, demonstrating the robustness of our methods.

MethodsSucc ↑\uparrowEg↓E_g \downarrowEl↓E_l \downarrowEacc↓E_{acc} \downarrowEvel↓E_{vel} \downarrow
RePO (seed=0)96.0%34.525.93.24.6
RePO (seed=1)95.7%33.525.83.24.5
RePO (seed=2)96.0%34.626.13.24.5
RePO (seed=3)96.4%33.325.53.14.4
RePO (seed=4)96.7%32.925.83.04.4
RePO96.16 ±\pm 0.39%33.76 ±\pm 0.7525.82 ±\pm 0.223.14 ±\pm 0.094.48 ±\pm 0.08

Furthermore, we evaluate the training stability of RePO by reporting its performance on AMASS-Test at different training epochs. As shown in the table below, RePO shows consistent improvements across all metrics while maintaining stable training.

MethodsSucc ↑\uparrowEg↓E_g \downarrowEl↓E_l \downarrowEacc↓E_{acc} \downarrowEvel↓E_{vel} \downarrow
250037.97% ±\pm 2.44%69.82 ±\pm 0.8946.24 ±\pm 1.6110.53 ±\pm 0.2413.86 ±\pm 0.14
500058.26% ±\pm 1.89%62.82 ±\pm 2.4041.16 ±\pm 1.418.76 ±\pm 0.3011.95 ±\pm 0.40
750067.25% ±\pm 3.67%58.56 ±\pm 0.9638.23 ±\pm 0.418.12 ±\pm 0.3211.15 ±\pm 0.30
1000072.61% ±\pm 3.49%56.02 ±\pm 2.3436.26 ±\pm 1.767.73 ±\pm 0.3010.57 ±\pm 0.40
1250076.96% ±\pm 1.94%53.24 ±\pm 1.2735.32 ±\pm 1.237.28 ±\pm 0.2210.05 ±\pm 0.13
1500080.14% ±\pm 1.96%52.38 ±\pm 1.5234.64 ±\pm 0.737.04 ±\pm 0.199.79 ±\pm 0.24
1750081.01% ±\pm 4.08%51.47 ±\pm 2.0334.17 ±\pm 1.086.93 ±\pm 0.419.62 ±\pm 0.40
2000086.38% ±\pm 2.01%49.02 ±\pm 0.5632.88 ±\pm 0.266.44 ±\pm 0.179.10 ±\pm 0.13
AuthorsOpenReview ↗

Look forward to your feedback

Dear Reviewer 75GY,

Thank you again for your valuable feedback and constructive suggestions.

As the discussion period is coming to an end, we would like to kindly ask whether our responses and additional experiments have adequately addressed your concerns, or if there are any remaining questions we could clarify. We would greatly appreciate any further feedback.

Thank you for your time and consideration!

Reviewer_a3DKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies physics-based humanoid motion imitation and proposes Reference-Relative Policy Optimization (RePO). In addition to PPO with GAE, RePO introduces a group-relative advantage by normalizing returns-to-go at corresponding timesteps across rollouts that track the same reference motion from a shared initial phase. A gate based on the average survival ratio controls the additional advantage. The method also uses kinematic pretraining and KL regularization against an EMA reference policy. Experiments on AMASS and OMOMO report improvements in success rate and tracking error over the implemented PPO baselines, with ablations on group size, mixing weight, and selected components.

Critical strengths

The complete training setup improves both success rate and tracking error in the reported results. In Table 1, AMASS-Train success increases from 94.5% to 96.0%, while global tracking error decreases from 38.8 to 34.5. On AMASS-Test, success increases from 87.0% to 89.9%, while global tracking error decreases from 55.3 to 51.2. These results provide preliminary support for the practical value of the approach. Removing the step-wise advantage in Table 5 reduces success to 91.6%, suggesting that this temporal treatment merits further investigation.

Critical weaknesses
  1. The method may be effective, but the interpretation of GAE and NAE as viability and fidelity advantages is insufficiently established. The rationale for SG is that return differences better reflect tracking accuracy when execution lengths are similar. However, this condition also applies to PPO with GAE using the same rewards and does not establish fidelity guidance specific to NAE. Moreover, group normalization in Eqs. (6)-(8) preserves return rankings at a given timestep and does not itself remove survival preferences in the original returns. Different baselines or normalization may improve optimization, but the evidence does not distinguish such general benefits from the claimed division of roles. The performance gains therefore do not establish the central mechanistic explanation.
  2. The experiments do not adequately isolate NAE. RePO also introduces kinematic pretraining and KL regularization; removing pretraining in Table 5 reduces success from 96.0% to 94.0%, showing that other training components affect the results. A PPO control should match pretraining, grouped sampling, regularization, and interaction and optimization budgets while removing only NAE. The tables do not clarify whether the results are averages over independent training runs, and omit seed counts, standard deviations, or confidence intervals. For example, AMASS-Test success in Table 1 increases from 87.0% to 89.9%, but without variability across training runs, I cannot assess whether this difference is stable. Matched controls and means and standard deviations across independent seeds are needed.
  3. The demos do not clearly demonstrate the claimed tracking-fidelity advantage over the baseline or substantiate the proposed mechanism. The manuscript also contains an unresolved citation (“?” on page 3, line 121) and author-name errors, for example Jun-Mei Song instead of Junxiao Song. These presentation and citation-quality issues should be corrected.
Questions for authors
  1. When execution lengths are similar, GAE can also use differences in tracking accuracy. What specific deficiency in GAE does NAE address, and what evidence distinguishes the proposed viability/fidelity roles from general estimation or normalization benefits?
  2. Does PPO use the same pretraining, grouped sampling, regularization, and training budgets? Can the authors provide a full-dataset control matching these conditions and removing only NAE?
  3. How many independent training seeds underlie the main comparisons and key ablations? Please report means and standard deviations across seeds and uncertainty in the principal performance differences.
3 comments
AuthorsOpenReview ↗

Response to a3DK (Part 1)

Thank you for acknowledging that our method is "effective" and "practical value". We also appreciate your constructive feedback on improving this paper.

Ⅰ. Optimization Preferences of GAE and NAE [W1, Q1]

We thank the reviewer for pointing out this imprecise statement. We have revised Lines 77–84 and 227-230 of the paper accordingly.

GAE and NAE exhibit viability- and fidelity-oriented advantages by different credit assignment mechanisms, rather than implying a strict separation of viability and fidelity rewards.

We would like to clarify that the group normalization in Eqs. (6)–(8) should be understood under comparable survival lengths within each group. We use Survival Gate (SG) to ensure this comparability, allowing NAE to primarily reflect cumulative future tracking fidelity rather than survival. In contrast, GAE estimates advantages independently for each environment. Longer trajectories tend to accumulate higher returns, leading to a strong survival preference, as illustrated in Figure 1 and described in Lines 50–54 of the paper.

Furthermore, we replace the return-to-go (Eq. 6) in NAE with single-step rewards (fidelity without survival information) to examine the role of future steps in fidelity estimation. As shown in table below, this modification substantially reduces success rates and increases tracking errors, indicating that incorporating future steps leads to more accurate advantage estimates.

MethodsSucc ↑\uparrowEg↓E_g \downarrowEl↓E_l \downarrowEacc↓E_{acc} \downarrowEvel↓E_{vel} \downarrow
RePO96.0%34.525.93.24.6
RePO (w/ Step Reward)86.4%57.239.84.57.5

Ⅱ. Applicability of Survival Gate (SG) [W1, Q1]

Notably, SG ensures that NAE compares trajectories with similar survival lengths.

We also apply SG to GAE in RePO. As shown in the second row of the table below, this substantially reduces the success rate. The results suggest that SG is not appropriate for GAE, as it excludes early-terminated trajectories from training.

Furthermore, we train PPO with SG to investigate its applicability to GAE. As shown in the table below, PPO with SG achieves a success rate of only 0.3%. This indicates that SG filters out too many early-terminated trajectories, leaving insufficient samples for effective policy updates. Consequently, training fails to converge.

MethodsSucc ↑\uparrowEg↓E_g \downarrowEl↓E_l \downarrowEacc↓E_{acc} \downarrowEvel↓E_{vel} \downarrow
RePO96.0%34.525.93.24.6
RePO (w/ SG for GAE)79.1%42.032.74.45.8
PPO (w/ SG)0.3%186.6148.949.646.3

Ⅲ. Isolating the Effect of NAE [W2, Q2]

We would like to emphasize that PPO serves as our primary baseline. Although removing pretraining reduces RePO's success rate from 96.0% to 94.0%, it still achieves consistently lower tracking errors, as shown in the table below, while maintaining comparable success rates with PPO (94.0% vs. 94.5%). This demonstrates that RePO promotes more accurate motion tracking without compromising execution success.

MethodsSucc ↑\uparrowEg↓E_g \downarrowEl↓E_l \downarrowEacc↓E_{acc} \downarrowEvel↓E_{vel} \downarrow
PPO94.5%38.830.13.85.1
RePO(w/o pretraining)94.0%36.527.83.55.0

Furthermore, we conduct an ablation to assess NAE's contribution by removing it. As shown in the table below, removing only NAE significantly degrades fidelity, especially in EaccE_{acc} and EvelE_{vel}, resulting in performance similar to PPO. This result demonstrates that NAE is a key contributor to RePO's gains.

MethodsSucc ↑\uparrowEg↓E_g \downarrowEl↓E_l \downarrowEacc↓E_{acc} \downarrowEvel↓E_{vel} \downarrow
PPO94.5%38.830.13.85.1
RePO96.0%34.525.93.24.6
RePO (-NAE)94.4%36.827.53.85.1

Ⅳ. Computational Cost [Q2]

Notably, RePO introduces only modest computational overhead. The table below compares the key parameters affecting the computational cost of PPO and RePO. For a fair comparison, we keep these parameters identical, with RePO increasing the average epoch time by only 8.9%. For group-based rollouts, we group existing parallel simulation environments rather than performing multiple rollouts in each environment.

MetricsRePOPPORePO/PPO
num envs30723072-
horizon length3232-
mini-batch size16,38416,384-
mini-epochs per epoch66-
training epochs2000020000-
mean epoch time (s/epoch)3.5883.2961.089x
AuthorsOpenReview ↗

Response to a3DK (Part 2)

Ⅴ. Statistical Reliability of RePO [W2,Q3]

Regarding statistical reliability, we conduct five independent training runs with different seeds. As shown in the table below, RePO consistently achieves stable performance across these runs, demonstrating the robustness of our methods.

MethodsSucc ↑\uparrowEg↓E_g \downarrowEl↓E_l \downarrowEacc↓E_{acc} \downarrowEvel↓E_{vel} \downarrow
RePO (seed=0)96.0%34.525.93.24.6
RePO (seed=1)95.7%33.525.83.24.5
RePO (seed=2)96.0%34.626.13.24.5
RePO (seed=3)96.4%33.325.53.14.4
RePO (seed=4)96.7%32.925.83.04.4
RePO96.16 ±\pm 0.39%33.76 ±\pm 0.7525.82 ±\pm 0.223.14 ±\pm 0.094.48 ±\pm 0.08

Furthermore, we evaluate the training stability of RePO by reporting its performance on AMASS-Test at different training epochs. As shown in the table below, RePO shows consistent improvements across all metrics while maintaining stable training.

MethodsSucc ↑\uparrowEg↓E_g \downarrowEl↓E_l \downarrowEacc↓E_{acc} \downarrowEvel↓E_{vel} \downarrow
250037.97% ±\pm 2.44%69.82 ±\pm 0.8946.24 ±\pm 1.6110.53 ±\pm 0.2413.86 ±\pm 0.14
500058.26% ±\pm 1.89%62.82 ±\pm 2.4041.16 ±\pm 1.418.76 ±\pm 0.3011.95 ±\pm 0.40
750067.25% ±\pm 3.67%58.56 ±\pm 0.9638.23 ±\pm 0.418.12 ±\pm 0.3211.15 ±\pm 0.30
1000072.61% ±\pm 3.49%56.02 ±\pm 2.3436.26 ±\pm 1.767.73 ±\pm 0.3010.57 ±\pm 0.40
1250076.96% ±\pm 1.94%53.24 ±\pm 1.2735.32 ±\pm 1.237.28 ±\pm 0.2210.05 ±\pm 0.13
1500080.14% ±\pm 1.96%52.38 ±\pm 1.5234.64 ±\pm 0.737.04 ±\pm 0.199.79 ±\pm 0.24
1750081.01% ±\pm 4.08%51.47 ±\pm 2.0334.17 ±\pm 1.086.93 ±\pm 0.419.62 ±\pm 0.40
2000086.38% ±\pm 2.01%49.02 ±\pm 0.5632.88 ±\pm 0.266.44 ±\pm 0.179.10 ±\pm 0.13

VI. Clarification of Tracking Fidelity Demos [W3]

In our visualization results, red spheres represent the reference motion, and yellow bodies show the policy execution. Better alignment between them indicates higher motion fidelity.

For example, in Figure 1, PPO fails to accurately track the leg motion during takeoff, leading to insufficient jump height and early termination, whereas RePO achieves higher tracking fidelity throughout the jump. Similarly, in the last two rows of Figure 3, RePO more closely follows the reference leg trajectories during dancing, while PPO shows substantial deviations.

Beyond the qualitative results from our visualization results, we further provide quantitative results of RePO's fidelity advantage. Specifically, we adjust the tracking-error termination threshold to compare the success rates of RePO and PPO at comparable fidelity levels. Tracking errors are computed only over successfully tracked motions. As shown in the table below, RePO achieves a substantially higher success rate under matched fidelity levels. This result suggests that PPO’s tracking success comes at the expense of motion fidelity.

MethodsEg↓E_g \downarrowEl↓E_l \downarrowEacc↓E_{acc} \downarrowEvel↓E_{vel} \downarrowSucc ↑\uparrow
RePO31.924.22.84.196.0%
PPO32.326.32.84.184.1%

Ⅶ. Correction of Manuscript Errors [W3]

We thank the reviewer for pointing out these errors. We have corrected the missing citation and author names in the revised manuscript.

AuthorsOpenReview ↗

Look forward to your feedback

Dear Reviewer a3DK,

Thank you again for your valuable feedback and constructive suggestions.

As the discussion period is coming to an end, we would like to kindly ask whether our responses and additional experiments have adequately addressed your concerns, or if there are any remaining questions we could clarify. We would greatly appreciate any further feedback.

Thank you for your time and consideration!

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revision

Dear Chairs and Reviewers,

We sincerely thank all reviewers for their careful and constructive feedback. We are pleased that our work was recognized as effective(a3DK,jeXz), "interesting"(jeXz), "potentially useful" (jeXz), "The problem diagnosis is clear and the method is well aligned with it" (75GY), and having "practical value" (a3DK).

In accordance with the reviewers’ comments and suggestions, we have conducted approximately 11 new experiments and revised the manuscript accordingly. All revisions are highlighted in blue. The main changes are summarized below:

Main Text

  • Section 1: Revised the imprecise description and clarified that GAE and NAE have viability- and fidelity-oriented preferences.
  • Section 2: Further clarified the differences between our work and existing state-of-the-art methods.
  • Section 3: Added explanations to clarify why NAE is fidelity-oriented.
  • Section 4: Added ablation studies and computational efficiency evaluations.

Appendices

  • Appendix A: Statistical reliability of RePO.
  • Appendix B: Quantifying the impact of tracking fidelity on PPO.
  • Appendix C: Applicability of survival gate (SG).
  • Appendix D: Further improvements with an enhanced RePO variant.

Thank you once again for your contributions to refining our work and polishing this paper.

Yours sincerely,

Authors of Paper 12910

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.