- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes HER, a monocular human–environment reconstruction framework that leverages reconstructed human point clouds to assist global human localization and metric scene scale estimation. Through geometry-guided spatial and temporal reasoning, HER recovers human motion in a unified world coordinate system. Experiments demonstrate that HER improves world-space human reconstruction accuracy and scene scale estimation compared to existing methods.
Critical strengths
- The paper achieves competitive performance on multiple benchmarks, demonstrating the effectiveness of the proposed framework.
- The paper provides relatively comprehensive evaluations and ablation studies, covering multiple metrics and key components of the proposed method.
Critical weaknesses
- Limited evaluation of 4D reconstruction quality. The existing static qualitative results are insufficient to adequately assess 4D reconstruction quality. Additional video demonstrations would help better evaluate human–scene alignment, contact consistency, and temporal coherence.
- Although HER outperforms VGMR Off* (GT Scale), the comparison does not fully isolate the localization benefit. I suggest fixing camera trajectories, metric scale, human pose, and post-processing, while varying only the human localization method.
- The paper lacks a direct comparison of Pi3X-based scene/camera reconstruction before and after applying the predicted scale. The authors should also clarify whether Camera ATE uses Sim(3) or SE(3) alignment.
Questions for authors
- Could the authors report the end-to-end inference speed (FPS) and runtime of the complete pipeline?
- While the separately predicted metric skeleton slightly improves world-space joint accuracy, does it provide more accurate body height or bone lengths than SMPL-X-derived joints? A direct evaluation of skeletal scale accuracy would be helpful.
- Despite the strong results on several benchmarks, what are the limitations of the proposed method? More in-the-wild internet video examples would help illustrate its generalization capabilities and potential failure cases.
We have also provided video demonstrations of HER via an anonymous link:
https://anonymous.4open.science/w/HER-anonymous-submitted/ for convenient viewing.
The high-resolution videos remain available in the supplementary material.
W1/Q3: Continuous 4D visualizations
Thank you for this suggestion. We have updated the supplementary material and our anonymous link with continuous reconstruction videos covering 3 RICH & 3 EMDB2 and 3 in-the-wild videos.
Each sequence is shown from 2 viewpoints to facilitate inspection of body motion, human–scene spatial relationships, foot contact behavior, and temporal consistency.
W2: Isolating VGMR's localization benefit
Thank you for this suggestion. We fixed camera-trajs, human pose and shape, and geometric inputs, and disable post-processing in both methods. Only localization changes between Points Align and VGMR. We test both a shared predicted scale and a shared GT-derived scale fitted from camera trajectories.
Results on all 25 EMDB2 sequences use common valid frames:
VGMR consistently improves under both shared-scale settings. GT-derived scale improves RTE but does not necessarily reduce Sim(3)-aligned joint errors, since it is fitted to camera trajectories rather than human joints.
W3: Scale recovery and the camera ATE alignment protocol
Thank you for highlighting this ambiguity. The camera ATE in Table 2 uses Sim(3) alignment, which evaluates camera-traj accuracy after removing global scale differences. We have explicitly labeled the alignment protocol in the table in our updated paper.
To directly assess scale recovery, we additionally evaluate camera ATE using SE(3) alignment, which preserves scale errors. We evaluate all 25 EMDB2 sequences, comprising 43,205 camera frames, and report the arithmetic mean of per-sequence ATE RMSE: | Camera trajectory | SE(3) ATE ↓ | Sim(3) ATE ↓ | |---|---:|---:| | Before scale recovery | 2.086 m | 0.297 m | | After applying HER's predicted scale | 0.674 m | 0.297 m |
The predicted scale is applied without GT scale correction. The reduction in SE(3) ATE prove that HER improves metric camera trajectory accuracy. Sim(3) ATE remains unchanged because similarity alignment removes the global scale difference.
Q1: Computational efficiency
Thank you for asking about runtime. HER achieves 4.84 FPS on a single NVIDIA RTX 4090. We compute throughput as
FPS=Tseg+SLAM+Thuman+VGMR+TrefinementN
where N is the total number of processed frames. The denominator includes segmentation and SLAM, human reconstruction and VGMR, and final world space refinement. We measure computation time after warm-up with CUDA synchronization at stage boundaries.
For reference, reported speeds of related methods are summarized below:
† UniSH's speed is reported by UniCon3R. Baseline speeds are taken from published reports.
Q2: Skeletal scale accuracy
Thank you for suggesting this direct evaluation. We compare our directly predicted joints with SMPL-X-derived joints across all EMDB2 sequences. We preserve the original metric dimensions without scale fitting and evaluate against GT.
Our predicted joints achieves lower bone-length errors. This evaluation focuses on bone-length consistency rather than overall body height, which is not explicitly measured.
Q3: In-the-wild examples and Limitations
Thank you for this suggestion. We have updated the supplementary material with in-the-wild videos and failure cases.
We also updated the Limitation Section in our paper, HER would produce inaccurate results when the person is far from the camera, occupies a small image region, or is heavily occluded, due to limited visual and geometric evidence.