- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces LDBODY, a benchmark for 2D/3D human pose estimation and mesh recovery for individuals with limb deficiencies and prosthesis users. The dataset combines in-the-wild images with newly collected multi-view motion data captured using eight synchronized RGB cameras and a Vicon system. It provides annotations for body joints, residual-limb endpoints, and prosthesis-supported landmarks, along with fitted mesh references. The authors also evaluate several existing pose estimation and mesh recovery methods and analyze their performance across different movement and prosthesis conditions.
Critical strengths
The authors created a dataset for 2D and 3D pose estimation in prosthesis-related scenarios and also provided fitted mesh reference data. They provide both an in-the-wild dataset and a calibrated dataset, which complement each other to some extent. Specifically, the in-the-wild images are based on the existing LDPose dataset with additional annotations, while the calibrated motion data were newly collected by the authors. The evaluation metrics are generally standard and widely recognized, relatively comprehensive, and reasonably adapted to prosthesis-related scenarios.
Critical weaknesses
My main concern is the limited number of subjects in LDBODY Motion. The dataset only includes five participants, which is insufficient for a benchmark, especially one intended to evaluate generalization across different body structures and prosthesis configurations. Furthermore, all five participants are male, and four are active Para athletes. This raises concerns about the lack of diversity and how representative the benchmark really is. I consider this a major limitation of the dataset.
More importantly, with such a small number of subjects, it is difficult to know whether the reported results reflect the general performance of the evaluated methods or are strongly influenced by the characteristics of these particular individuals. The large number of recorded frames does not really address this issue, since they all come from the same five participants. While the in-the-wild image dataset provides additional diversity, it is mainly an extension of the existing LDPose dataset and does not compensate for the limited diversity of the calibrated 3D data.
I think the dataset is a useful contribution, but its current scale makes it difficult to establish a sufficiently representative and reliable benchmark for broader evaluation.
Questions for authors
How do the authors see the benchmark supporting generalization across different populations and prosthesis configurations since there are only 5 subjects? Are there any plans to expand the dataset to include more diverse participants, particularly female and non-athlete prosthesis users?