acceptodds
Withdrawn from ICLR 2027

NeCR: Neural Contact-Consistent Retargeting for Whole-Body Humanoid Tracking

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Retargeting human motion to a humanoid becomes particularly challenging for multi-contact motions, in which the knees, shins, and forearms support the body alongside the feet. Crawling, kneeling, and getting up from the floor depend on such whole-body contacts, and a robot with different proportions may be unable to reproduce contacts as the source human made them. Learning-based retargeters preserve motion semantics but cannot guarantee physical validity at inference, so a contact may be reproduced while penetrating the ground or sliding. Optimization- based retargeters enforce physical constraints, either frame by frame, where they can misplace support, or over the whole sequence, at a cost that grows with its length. To address this challenge, we present NeCR, a hybrid neural–optimization framework in which a learned contact model specifies where, when, and how strongly the optimizer corrects the motion. A neural retargeter first produces a semantically faithful nominal trajectory. Then a temporal model predicts continuous whole-body contact probabilities for the optimizer, which adjusts only the degrees of freedom of regions in robot-feasible contact. Therefore, the resulting problem is sparse and block-banded and can be solved in three Gauss–Newton sweeps. Evaluated on public human motion datasets retargeted to a Unitree G1, NeCR removes penetration and cuts support-phase sliding from 20.6% to 4.4% at a cost of only 0.16 cm MPKPE, and runs at 5.9 ms per frame. Policies trained on its references track more reliably on two independently implemented trackers and transfer zero-shot to a real G1 for crawling and fall recovery. Code, retargeted datasets, and trained policies will be released.

Primary area:
applications to robotics, autonomy, planning
Keywords:
humanoid robots, motion retargeting, contact consistency, whole-body motion tracking, trajectory optimization, learning-based control

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_j4ZAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents a hybrid neural–optimization framework for contact-aware motion retargeting. Whole-body multi-contact retargeting is an important problem, and the results demonstrate that the proposed projection reduces penetration and sliding.

Critical strengths
  1. The paper addresses a meaningful problem in humanoid motion retargeting: preserving whole-body contacts involving the knees, shins, and forearms, which are important for behaviors such as crawling and getting up.

  2. The hybrid neural–optimization design is well motivated. Relative to its own nominal output, CAP substantially reduces contact artifacts while introducing only a small fidelity cost: skating decreases from 20.60% to 4.39%, with an MPKPE increase of 0.16 cm.

  3. The evaluation connects reference quality to downstream control through two tracking implementations, policy-by-reference cross-evaluation, and real-robot demonstrations. These provide useful evidence beyond purely kinematic metrics.

Critical weaknesses
  1. The evaluation does not sufficiently test tasks whose success depends on precise contact placement and timing.

The main demonstrations involve crawling, kneeling, and getting up, with contacts primarily occurring on flat ground. These are meaningful multi-contact behaviors, but a large planar surface provides relatively permissive contact locations. Consequently, the experiments mainly demonstrate reductions in penetration and sliding, rather than showing how precise contact reconstruction enables successful task execution.

Whole-body loco-manipulation tasks, such as carrying boxes, and parkour involving constrained footholds and contact transitions would more directly demonstrate the value of contact-aware retargeting.

  1. The source of the nominal retargeter’s advantage over NMR is unclear, confounding attribution of the overall gains.

In Table 1, NeCR w/o CAP already achieves an MPKPE of 6.86 cm, compared with 8.46 cm for NMR. However, the paper does not clearly identify a methodological contribution in the nominal retargeting stage relative to NMR. The models use different training data and supervision targets. Although the appendix discusses differences in data coverage and potential biases introduced by RL refinement, these explanations are not verified through controlled comparisons.

Moreover, the advantage of NeCR w/o CAP primarily concerns pose fidelity: its penetration and skating rates are higher than those of NMR. These results may therefore reflect different trade-offs between fidelity and contact quality under different supervision targets.

Crucially, the downstream policy experiments do not include NeCR w/o CAP. It is therefore difficult to determine how much of the improvement in tracking success comes from the nominal retargeter versus the proposed contact prediction and CAP components. I would like to see comparisons of the base models under matched training data, as well as downstream evaluations using the same nominal retargeter with no projection, a simple contact-correction baseline, and the full NeCR method.

  1. The motion-quality improvements over OmniRetarget are mixed.

In Table 1(b), OmniRetarget achieves a lower skating rate than NeCR: 1.63% versus 4.39%. Both methods report zero penetration and joint-limit violation rates, while their floating rates are nearly identical. NeCR achieves a smaller peak penetration depth and approximately 2.4 percentage points higher contact preservation, but also exhibits a higher peak floating height. These results indicate trade-offs across metrics rather than consistent superiority in motion quality.

The skating reduction highlighted in the abstract, from 20.6% to 4.4%, is measured against the method’s own uncorrected nominal output. This demonstrates the effectiveness of CAP in correcting that output, but does not directly establish better contact quality than OmniRetarget.

The substantial contact-preservation improvement on LAFAN1 is encouraging. However, results should be broken down by motion category and contacting body part to clarify where these benefits arise and whether they translate into improved task completion.

Questions for authors
  1. What accounts for the nominal retargeter’s fidelity advantage over NMR, given that no distinct methodological contribution is identified for this stage? Could matched-data comparisons clarify the respective effects of model design, training-data coverage, and supervision targets?

  2. Could the downstream policy evaluation include NeCR w/o CAP and a heuristic-contact CAP variant using the same nominal retargeter? These comparisons would help isolate the control benefits of the proposed contact components from differences in the underlying reference generator.

  3. Could the method be evaluated on tasks where precise contact placement and timing directly affect success, such as narrow footholds or designated contact regions within static environments? Could contact quality also be reported separately for non-foot body parts, particularly the knees and forearms?

  4. How should the motion-quality advantage over OmniRetarget be interpreted, given its lower skating rate and comparable penetration and floating rates in Table 1(b)? Could the authors identify which motion categories and contacting body parts account for NeCR’s contact-preservation gains, particularly on LAFAN1, and whether these gains improve task completion?

  5. How does NeCR advance beyond existing continuously weighted, spatially and temporally sparse contact optimization, such as ReConForM? In particular, could the authors isolate the benefit of contact-dependent DOF selection by comparing against a full-DOF projection with the same contact evidence, constraints, and solver?

Reviewer_abEEOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses contact inconsistencies in human-to-humanoid motion retargeting, including ground penetration, floating, and foot sliding. It focuses on the difficulty of preserving whole-body contacts across different embodiments while avoiding unnecessary changes to the transferred motion. The proposed NeCR framework first generates a nominal robot trajectory using a learned retargeter. A temporal model then predicts contact, support, stick, and continuation probabilities, together with per-site contact feasibility, to identify plausible contacts and their temporal persistence. These predictions are compiled into contact events that specify where, when, and how strongly corrections should be applied. A structured projector subsequently performs localized, confidence-weighted optimization to improve contact consistency while remaining close to the nominal trajectory.

Critical strengths

The paper translates its analysis of existing contact-handling limitations into a coherent framework that combines learned contact prediction with structured optimization. The predicted contact information guides where and when corrections are applied, as well as their strength, while the optimization preserves proximity to the nominal trajectory. Table 1 shows that this correction substantially reduces penetration and sliding with only a small increase in pose error. This trade-off is a substantive strength because it improves contact consistency without substantially compromising motion fidelity.

Critical weaknesses
  • The rationale for preserving human-derived contacts under the embodiment gap is insufficiently validated. The method derives contact supervision from human motion, although differences in morphology may make alternative contact patterns or motion trajectories more appropriate for the robot. Robot-side feasibility prediction partially addresses whether a source contact can be realized, but does not establish whether preserving it is preferable to adopting a different strategy. This distinction matters because improved agreement with human contacts does not necessarily imply more effective or efficient robot execution. The evaluation would be strengthened by evidence connecting source-contact preservation to practical execution benefits.

  • The evaluation does not sufficiently separate the contribution of the nominal retargeter from that of CAP. Table 1 shows that the nominal-only variant already achieves strong motion fidelity and that CAP improves contact metrics at a small fidelity cost. However, the nominal-only variant is absent from the downstream policy comparisons and the LAFAN1 evaluation. Consequently, it remains unclear whether the reported execution gains arise primarily from the nominal predictions or from the proposed contact-guided correction. A controlled comparison with and without CAP, ideally separated into foot-supported and whole-body contact motions, is needed to establish the practical contribution of the central correction mechanism.

Questions for authors
  • The method accounts for whether human-derived contacts are feasible for the robot. What evidence shows that preserving these contacts is preferable to allowing alternative robot-appropriate contact patterns?

  • Are there w/o CAP results for the downstream policy and LAFAN1 evaluations to distinguish gains from nominal prediction from those introduced by CAP?

  • How does downstream policy performance vary across motion categories?

  • The paper emphasizes knee, shin, and forearm contacts, but reliable foot contact is also essential in ordinary locomotion. Can the results in Tables 1 and 2 be broken down by body part to show whether CAP also improves foot-contact performance?

  • The contact predictor is trained on the proposed nominal retargeter’s outputs. Can CAP also improve other retargeters’ outputs, either directly or after retraining the contact predictor?

  • The temporal mode includes four components, but their individual contributions are not evaluated. Which components contribute most to performance, and are all four necessary?

  • How does the full pipeline’s runtime compare with the baselines under the same hardware and timing protocol?

Reviewer_1KJFOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper works on human-to-humanoid motion retargeting with non-foot contacts, such as crawling and getting up. To this end, the authors train two networks: a motion retargeting network f and a contact prediction network g, which predicts continuous contact modes and robot-feasible surface patches. The outputs of these two networks are then further refined through constrained optimization to improve contact consistency while preserving motion fidelity. Experiments compare retargeting quality and downstream tracking performance across two policy-training frameworks and include hardware demonstrations, where the learned tracking policy successfully executes crawling and fall-recovery motions on a real Unitree G1 without fine-tuning.

Critical strengths
  • Important and emerging problem: The paper addresses the emerging problem of human-to-humanoid motion retargeting with non-foot contacts, such as crawling, kneeling, and getting up from the ground. Compared with conventional retargeting methods that primarily focus on foot-ground contacts, these motions involve more complex whole-body contact patterns and introduce additional challenges due to human–robot embodiment differences. The problem is practically relevant to enabling more diverse and expressive humanoid behaviors.

  • Comprehensive ablation studies: The ablation studies are well designed and executed to demonstrate the effectiveness of the proposed components. In particular, the authors compare learned contact predictions against heuristic alternatives, investigate continuous versus binary contact weights, and evaluate the contribution of individual optimization terms. The paper also studies the effect of the number of Gauss–Newton iterations on optimization quality and computational efficiency. These experiments provide useful evidence supporting the design choices and effectiveness of the proposed framework.

Critical weaknesses
  • Insufficient motivation for the method design: While the paper motivates combining learning-based retargeting with constrained optimization, the rationale behind several key design choices is not sufficiently clear.
    • First, it is unclear why a separately trained neural retargeter fθf_\theta is necessary instead of using existing optimization-based retargeting methods, especially given that the BONES-SEED training dataset itself is constructed using optimization-based retargeting. Since the proposed Contact-Aware Projection (CAP) primarily performs local corrections, it would be valuable to clarify whether its effectiveness depends on the specific neural retargeter or whether it can serve as a general refinement framework for other retargeting methods.
    • Second, the motivation for introducing a relatively complex contact prediction network gϕg_\phi could be better justified. Since the contact supervision is derived from geometric cues and robot-side feasibility analysis, it is unclear why these cues cannot be directly used to guide the optimization. Although the ablation studies partially address this question by comparing learned contact predictions with heuristic alternatives, a clearer explanation of the necessity and advantages of the learned formulation would strengthen the methodological motivation.
    • Furthermore, the paper emphasizes that the predicted contact modes determine which degrees of freedom are unlocked during optimization, but the specific benefits of this design choice are not isolated experimentally. An additional ablation comparing the proposed selective optimization with an optimization over all degrees of freedom would help justify this design choice.
  • The presentation and notation are difficult to follow: The method introduces numerous variables, intermediate representations, and optimization components without sufficiently explaining their relationships, making the overall pipeline difficult to understand.
    • Inconsistency between the notation in the text and figures: The notation in the main text does not align precisely with Figure 2. For example, Figure 2 uses q0q^0 to denote the nominal trajectory, while Equation (1) uses Qˉ\bar Q. Furthermore, CC in Figure 2 appears to represent the compiled contact phases, whereas Equation (1) defines CC as the predicted contact modes, including continuous probabilities and per-node feasibility. The inputs to gϕg_\phi also differ in presentation between Equations (1) and (2), without clearly explaining how the additional motion features and geometric descriptors are constructed. These inconsistencies make it difficult to establish a clear correspondence between the mathematical formulation and the proposed pipeline.
    • Essential methodological details are deferred to the appendix: Several important components necessary to understand the proposed method are only described in the appendix. For example, Section 3.3 introduces four contact probabilities in Equation (2), but the additional five-way contact-role classifier, which influences contact-event compilation and constraint activation, is introduced only in Appendix B.3. Similarly, the selection of active degrees of freedom, the construction of contact patches, the detailed formulations of the tangential contact objectives, and the implementation of the constrained optimization solver are largely deferred to Appendix B.4. Since these components are central to the claimed contributions, their absence from the main text makes it difficult to understand how the predicted contact modes are actually translated into optimization constraints.
    • Overcomplicated presentation of a relatively standard optimization formulation: The optimization in Equations (3) and (6) is essentially a constrained nonlinear least-squares problem, solved through Gauss–Newton linearization and constrained quadratic subproblems. Such formulations are well established and can, in principle, be implemented using existing optimization libraries. However, the paper devotes considerable attention to the mathematical formulation and specialized solver implementation without sufficiently distinguishing standard optimization techniques from the actual methodological contributions. In particular, the necessity and advantages of the proposed local–global solver over existing constrained optimization solvers are not clearly justified. A more concise formulation, accompanied by a clear explanation of how the learned contact predictions determine the active variables, residuals, and constraints, would make the technical contribution easier to understand.
Questions for authors
  1. Motivation and necessity of the two neural networks: Both networks are trained using existing supervision: fθf_\theta learns from paired human–robot retargeted trajectories, while gϕg_\phi learns contact modes and feasibility from labels constructed using explicit geometric formulations and robot-side feasibility analysis. Could the authors clarify why two separately trained neural networks are necessary instead of directly applying existing optimization-based retargeting methods and geometric contact estimation? In particular, what advantages does learning provide beyond approximating these explicitly constructed targets, such as computational efficiency, temporal consistency, or robustness? Although Appendix E.1 compares learned contact predictions with heuristic alternatives, it remains unclear whether directly using the full geometric contact and feasibility estimation procedures could achieve comparable results. Clarifying the necessity of each learned component and whether CAP can improve trajectories from other retargeting methods would help establish the significance and generality of the proposed framework.

  2. Technical contribution of the optimization in Section 3.4: The optimization in Equation (3) is essentially a regularized constrained nonlinear least-squares problem, while Equation (6) follows a standard Gauss–Newton linearization and can be approached using existing constrained optimization solvers. Could the authors clarify what constitutes the main technical novelty of CAP beyond this conventional optimization formulation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.