acceptodds
Under review as a conference paper at ICLR 2027

HumanoidPose:Monocular 3D Pose Estimation for Humanoid Robots

Abstract

As humanoid robots increasingly enter real-world environments, recovering their 3D whole-body pose from monocular imagery is becoming increasingly important for external state perception, motion analysis, and robot learning. However, Humanoid Robot Pose Estimation (HRPE) lies at the intersection of conventional Robot Pose Estimation (RPE) and Human Pose Estimation (HPE), while not being fully addressed by either paradigm: RPE primarily focuses on non-humanoid robots, particularly fixed-base manipulators, whereas HPE is built upon human-specific skeletal topologies, body models, and motion priors. Despite recent progress in humanoid 2D keypoint detection and HPE adaptation, humanoid-native 3D pose recovery remains insufficiently studied, and a dedicated benchmark with suitable training data and evaluation protocols is still lacking. To address this gap, we introduce Humanoids, a benchmark for monocular 3D humanoid robot pose estimation, and propose HumanoidPose, a framework for this task. Humanoids provides synthetic 3D supervision and real-world evaluation across diverse humanoid embodiments, motions, and visual conditions, while HumanoidPose combines human-inspired visual perception with robot articulation modeling and explicit kinematic grounding. Experiments on Humanoids and the DHRP-H1 benchmark show that HumanoidPose outperforms representative HPE and RPE approaches in articulation recovery, 3D pose estimation, and real-world evaluation. Ablation studies further demonstrate the contribution of robot-native articulation modeling and explicit kinematic grounding. The dataset and code will be publicly released to support future research in the community.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
humanoid pose estimation, human pose estimation, humanoid motion

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ex7cOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces Humanoids, a benchmark with synthetic RGB and robot-state annotations for 12 humanoid embodiments, plus real videos annotated with 2D keypoints. HumanoidPose estimates joint configurations and camera-relative root poses using frozen visual features, keypoint-conditioned queries, separate root prediction, and FK-based residual refinement. Models are trained separately for each robot. Evaluation reports improvements over adapted human/robot pose estimators on synthetic 3D metrics and real-image 2D metrics.

Critical strengths
  1. The benchmark pairs RGB with robot states across 12 embodiments and uses motion-disjoint splits. This provides a common setting for comparing humanoid pose estimators beyond individual robot platforms.
  2. The method improves synthetic MPJPE from approximately 75.6 mm for the strongest adapted baseline to 60.0 mm (Table 5). Component ablations support the usefulness of the proposed combination.
Critical weaknesses
  1. The methodological and dataset contributions appear incremental. Joint-angle and camera-relative pose estimation already appear in RoboPose (Labbe et al., 2021), while HoRoPose (Ban et al., 2024) already separates joint states, rotation and root depth. Section 4 does not clearly establish what new mechanism is required for humanoids beyond adapting these designs with keypoint-conditioned decoding and geometric refinement. Likewise, Humanoids-Sim uses existing AMASS motions, GMR and simulator rendering; related retargeting workflows are established in SONIC (Luo et al., 2026). Covering 12 robots broadens the benchmark, but each robot has a separate model. More controlled comparisons are needed to establish the contribution beyond this engineering adaptation.
  2. The evaluation leaves the real-world significance of the contribution uncertain. Humanoids-Real contains only 31.5 minutes across 12 embodiments, excludes severely occluded videos, and provides no metric 3D or joint-state ground truth (Section 3.2.2; Appendix B). Consequently, the central claim of recovering 3D robot states is validated only in simulation; improved 2D projection does not establish accurate depth or articulation.
  3. The synthetic training split is also limited to 5 h 19 min, with no data-scaling analysis to separate architectural gains from data limitations.
  4. No downstream experiment demonstrates whether the recovered states support the robot-learning or skill-transfer applications motivating the paper.
Questions for authors
  1. Do the adapted baselines use comparable visual features, 2D-keypoint inputs and training budgets? What evidence isolates the benefit of keypoint-conditioned queries and bounded refinement beyond a standard robot-state regressor with FK supervision?
  2. Is there any real-video joint-angle or metric 3D evaluation, or downstream validation showing that the recovered states are usable beyond matching 2D projections?
  3. How do the proposed method and the strongest adapted baseline compare as the number of source motions or rendered views increases? This would help assess both the dataset's contribution and the persistence of the reported method advantage.
Reviewer_uZPxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the problem of monocular 3D pose estimation for humanoid robots. Existing methods are deigned either for non-humanoid robots or using human-specific 2d-keypoint/skeleton/motion. In this paper, authors introduce a benchmark for monocular 3D pose estimation for humanoid robots. Further, a framework is proposed to estimate the robotic pose. Experiments demonstrate good performance.

Critical strengths
  • This paper introduces a new benchmark which can help the study of humanoid robots.
  • Experiments show good performance.
Critical weaknesses

I have the following concerns:

  • Could the author please clarify the application background? Why is it necessary to estimate the pose of a humanoid robot? “In such settings, understanding their whole-body state from external visual observations becomes important for making robot behavior observable and interpretable to people and surrounding systems. ” In fact, for humans, estimating the robot’s pose is not necessary, as we can easily observe and predict the robot’s behavior.

  • The 3D information is provided only in the synthetic dataset, whereas in the real dataset, the ground truth annotations consist solely of 2D poses. This is unreasonable for a dataset, as 3D information appears to be more valuable for humanoid robots. Due to this issue, the authors are unable to evaluate the performance of 3D estimation on the real dataset and can only evaluate it based on 2D projections. Consequently, this also fails to reflect the actual performance of 3D parameter estimation.

  • Training a separate model for each type of humanoid robot is expensive. How can this be implemented in a more general way?

  • How does the proposed framework address challenges specific to humanoid robots, such as high degrees of freedom and floating bases?

  • What are the criteria for annotating key points?

Questions for authors
  • Please clarify the real-world application.
  • How to address the 3D evaluation/annotation in real data?
  • How to train a general model for different humanoid robot ?
  • What the specific challenges this framework addressed for humanoid robot?
Reviewer_ig1qOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Humanoids, a benchmark for monocular humanoid robot pose estimation, and HumanoidPose, a framework that estimates robot-native joint configurations and camera-relative root pose. The benchmark covers 12 humanoid embodiments and includes 6,885 synthetic clips with robot-state and 3D supervision, together with 365 real-world clips with manually verified 2D annotations. The method combines frozen visual features and canonical 2D keypoints with separate articulation and root-pose prediction branches, followed by forward-kinematics-based grounding and bounded residual refinement. A separate model is trained for each embodiment. The reported results improve aggregate synthetic 3D metrics and real-world image-space metrics over the evaluated baselines. Real-world metric 3D accuracy is not directly evaluated.

Critical strengths

Separating articulation prediction from root-pose estimation, followed by embodiment-specific forward kinematics and bounded refinement, is a coherent design for this task. In Table 5, MPJPE decreases from 75.574 mm for the strongest baseline on that metric to 59.954 mm, while G-MPJPE decreases from 320.149 mm to 183.218 mm.

Critical weaknesses

1.The paper would benefit from a clearer comparison with HoRoPose, which already separates joint-state and root-pose estimation and uses forward kinematics. The humanoid setting is valuable, but the architectural advance needs stronger support. In Table 7, kinematic grounding and refinement are removed together, so the improvement cannot be attributed to either component separately. The replacement used in “w/o Robot Artic.” is also unclear. 2.The real-world evaluation measures only projected 2D accuracy. A direct comparison with the 2D frontend is therefore important to show what the 3D stages add. The assumed camera intrinsics in Appendix H are another source of uncertainty: all methods use the same setting, but their sensitivity to inaccurate intrinsics may differ. This is relevant when interpreting the real-world ranking. 3.Some formatting and writing issues need to be fixed:Table 2 labels RoboNet’s frame count as clips; Figure 4 contains “Viusal Tokens”; Eq. (19) mixes up the keypoint count and joint-angle dimension; and the unit of synthetic [email protected] is unspecified;Tables 6–8 should also use larger text.

Questions for authors

1.In Table 7, kinematic grounding and refinement are removed together. Could you separate their effects while keeping the refinement network comparable? Please also clarify what replaces the articulation branch in “w/o Robot Artic.” and how MPJAE is computed for that variant. 2.Since Table 6 evaluates projected 2D accuracy, could you also report the raw 2D frontend results under the same protocol? Comparing the frontend predictions, initial 3D projections, and final refined projections would help show what the 3D stages contribute on real data. 3.Appendix E.1 mentions “DAPT pre-training.” What does this refer to, and which data were used? Also, are the 2,629 clips in E.2 sampled windows from the source clips listed in Table 18? Please explain how this training count is obtained.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.