Major Weaknesses
1. The paper overlooks HAC, a directly relevant prior work that introduced optimization-free human-guided metric scale calibration.
The authors present reconstructed human meshes as natural metric references to calibrate monocular scene geometry. However, Humans as Checkerboards (HAC, ICCV 2025) has already proposed an optimization-free framework that utilizes metric human reconstruction to calibrate monocular SLAM scale. HAC combines independently estimated human and scene information, uses human–scene contact joints as metric depth references, and recovers scaled camera trajectories and global human motion. Despite this close relationship, COHERE4D neither cites HAC nor compares its results against it.
The overlap is not limited to motivation; both methods use pretrained human reconstructions to resolve scene-scale ambiguity without training a joint reconstruction model. COHERE4D differs in that it uses dense pixel-aligned body-surface correspondences instead of contact-joint references and jointly estimates scene scale and human translation through robust geometric fitting. These differences may be significant, but the paper fails to explain them in relation to HAC.
HAC is also absent from Table 4, despite reporting the same global human motion metrics on EMDB-2. HAC achieves 71.0 mm WA-MPJPE, 197.2 mm W-MPJPE, and 1.2% RTE, while COHERE4D reports its best results as 68.1 mm, 152.9 mm, and 1.15%. Although I recognize that the two systems utilize different human and scene backbones, making cross-comparison of their published metrics challenging, HAC remains a directly relevant baseline. Its complete omission makes it difficult to assess the novelty and empirical progress of COHERE4D.
2. The choice of dense pixel-aligned correspondences is not adequately justified, especially considering prior evidence that the specific human reference points used for scale calibration significantly impact results.
COHERE4D generates correspondences by rendering the predicted human mesh at its initial camera-space translation and retrieving scene points at the same image pixels. This method avoids 3D nearest-neighbor matching and provides many candidate correspondences. However, it does not ensure that paired points represent the same physical surface.
If the initial human translation is incorrect, the rendered body can overlap background geometry, nearby objects, or even another person. Consequently, the resulting correspondences may originate from unrelated surfaces. While RANSAC can aid in identifying correct correspondences when enough valid ones are present, it cannot recover the correct locations if the initial projection is significantly displaced. Moreover, Appendix A.2 constructs translation-refinement correspondences using the original human placement without an explicit correspondence-update step after refinement.
The choice of dense matching is particularly crucial given HAC's findings regarding reference-point ablations. HAC reports a WA-MPJPE of 372.86 mm when using the head for scale calibration, 289.76 mm with the pelvis, 71.94 mm with the feet, and 70.95 mm using contact joints. These results demonstrate that having more human geometry does not necessarily equate to more reliable geometry for scale estimation.
COHERE4D does not compare all visible body-surface points with contact or support-region correspondences, nor does it evaluate how alignment behaves when the initial human translation is perturbed. I would expect these controls to be included since dense surface matching is a primary methodological difference between COHERE4D and earlier human-based scale calibration approaches. The current experiments show that dense fitting works well with robust scene backbones, but they do not establish why it is preferable to selecting fewer, more physically meaningful correspondences.
3. The reliability of the metric scale estimator depends on the reconstructed human's dimensions and the first scene chunk; however, neither of these dependencies has been sufficiently examined.
Equation 8 estimates the scene scale based on the relative spatial distribution of human mesh points and scene points. This estimation relies on the assumption that the reconstructed human provides an accurate metric reference. If the human backbone consistently predicts a body size that is too large or too small, the estimated scene scale will inherit the same error. RANSAC cannot eliminate this bias, as correspondences may agree with one another while still sharing the same incorrect metric reference.
Moreover, the method selects a shared scale from the first reconstructed chunk and keeps it fixed throughout the sequence, making initialization particularly crucial. If the video starts with distant humans, significant occlusions, inaccuracies in body dimensions, or poor foreground depth, the initial metric scale may be unreliable. Although later frames might contain more accurate human observations, the method does not utilize them to reassess the initial scale.
There is also an unexplained scenario in which the first chunk contains no valid human reconstruction. Appendix A.1 discusses how unreliable scale hypotheses are filtered and what occurs when none meets the quality thresholds, but it does not clearly explain how initialization proceeds when no valid correspondence or scale candidate is available.
HAC explores the temporal aggregation of human-based scale estimates, although its full-sequence approach is not directly suitable for causal inference. Instead, COHERE4D could assess a running robust estimate or use delayed initialization based solely on past observations. Since the paper claims reliable long-video reconstruction, the stability of its metric initialization should be evaluated rather than simply assumed.
4. The RANSAC ablation demonstrates the importance of robust fitting but fails to establish why the proposed estimator is preferable to other robust alternatives.
The authors employ two-point RANSAC hypotheses, body-size-dependent inlier thresholds, and cross-frame consensus to estimate scene scale. Appendix Table 10 compares the proposed estimator against ordinary least squares and reports significant improvements. These results are beneficial; however, ordinary least squares is known to be sensitive to outliers, particularly when correspondences involve occluders or incorrect depth.
Thus, the comparison shows that robust estimation is necessary rather than demonstrating that the specific RANSAC and scale-consensus procedure is the best option available.
This distinction is even more evident because COHERE4D already employs weighted medians, median absolute deviation, confidence-weighted Procrustes, and IRLS for inter-chunk alignment. These methods present plausible alternatives for parts of the main scale-calibration procedure. Additionally, the selected scene backbones can provide geometric confidence estimates, yet the paper does not clarify whether confidence-weighted robust fitting offers any advantage over treating all candidate correspondences equally before inlier selection.
The authors do not need to compare against every robust estimator. However, given that robust geometric alignment is the central methodological contribution, a controlled comparison against confidence-weighted IRLS, trimmed fitting, or another robust scale estimator would be more enlightening than comparing RANSAC with non-robust least squares. This comparison would help clarify whether the observed gains stem from the specific consensus formulation or simply from utilizing standard outlier rejection methods.
5. Scene-grounded translation refinement is not consistently beneficial, and the method does not clarify when to trust the scene over human predictions.
The paper introduces human-anchored scale calibration and scene-grounded translation refinement as complementary components of bidirectional alignment. However, Appendix Table 9 indicates that translation refinement can actually worsen global human motion.
In the EMDB-2 dataset, adding translation refinement after scale calibration increases the G-MPJPE in seven out of the nine human/scene-backbone combinations. For example, with the 3DB + VGGT-Ω configuration, the G-MPJPE rises from 1203.9 mm to 1208.6 mm. On the RICH dataset, the same configuration causes an increase from 138.9 mm to 184.9 mm.
The authors acknowledge this limitation and retain the original 3DB translation in their main results. While this is a reasonable engineering decision, it raises questions about the generality of the refinement procedure.
The effectiveness of the current choice depends heavily on the specific human backbone used. However, the relative reliability of the human and scene predictions can fluctuate across frames, even within the same sequence. A strong human backbone may occasionally make translation errors, while a reliable scene backbone might fail near occlusion boundaries.
The method lacks a mechanism to determine when to apply translation refinement based on the current predictions' reliability. Although the RANSAC inlier ratio, geometric residuals, and scene-model confidence could potentially provide useful signals, this has not been explored. Therefore, the experiments demonstrate that scene-grounded translation refinement is beneficial under certain conditions, but not that it is a reliably corrective method in all cases.
6. The paper claims to present more plausible human–scene interactions, but its quantitative evaluation focuses on geometric agreement rather than physical grounding.
The introduction and qualitative comparisons repeatedly link improved alignment to more plausible contact and interaction scenarios. Figures 7, 8, and 15 provide examples of humans interacting with benches, supporting surfaces, and other individuals.
However, the primary alignment evaluation employs HS-CF, which measures the agreement between the visible human mesh surface and scene points within the human region. These scene points can actually represent the human's surface rather than the static surfaces that support the body.
As a result, a reconstruction could yield a low HS-CF value while the person still appears to float above the ground, penetrate nearby objects, or slide during contact. Aligning two reconstructed representations does not inherently establish physically correct interactions with the surrounding environment.
This is a significant oversight, as JOSH and UniCon3R—both of which are included in COHERE4D's global-motion comparison—already evaluate relevant physical grounding errors. JOSH reports issues such as foot sliding and floating, while UniCon3R assesses collision ratios, penetration depths, floating distances, and maximum penetration on the RICH dataset.
I believe that a geometric alignment method does not necessarily need to predict contact forces or perform physical optimization. However, improved physical interaction is presented as an outcome of COHERE4D, yet the reported alignment metric fails to substantiate this claim. At a minimum, one quantitative comparison of physical grounding is necessary to support this assertion. Otherwise, conclusions should be limited to improved geometric alignment between the predicted human and scene surfaces.
7. Baseline Results Require Further Explanation
Several baseline results are not adequately explained, particularly the substantial difference between COHERE4D's reproduced UniSH depth results and those reported in the original paper.
COHERE4D reevaluates Human3R and UniSH on Bonn, focusing on scene depth and camera reconstruction under a protocol that maintains the predicted metric scale. This approach is appropriate for evaluating metric calibration.
However, the reproduced UniSH results show a significant discrepancy when compared to the original publication. UniSH reports an AbsRel of 0.035 and a threshold accuracy of 0.980 on Bonn, while COHERE4D reports an AbsRel of 0.318 and a threshold accuracy of 0.5541. This represents approximately a ninefold increase in AbsRel and a drop of over 42 percentage points in threshold accuracy. In contrast, the reproduced Human3R results are much closer to the original publication.
I acknowledge that this discrepancy may stem from differences in scale-alignment conventions, checkpoints, preprocessing, or evaluation subsets. I am not asserting that COHERE4D's results are necessarily incorrect; however, such a significant difference warrants a clearer explanation, especially as these results are used to illustrate the limitations of learned metric-scale reconstruction.
The authors should specify precisely which evaluation settings differ from UniSH's original benchmark to ensure reproducibility. Without this information, it is challenging to ascertain whether COHERE4D is highlighting a genuine metric-scale generalization failure or comparing methods under significantly different evaluation conventions.
Minor Weaknesses
-
Discussion of HumanBA: The relationship to HumanBA should be addressed. HumanBA (CVPR 2026) utilizes human motion to enhance camera pose estimation through motion decoupling and reliability-weighted bundle adjustment. Its optimization-based framework differs from COHERE4D's approach, so a direct comparison in the main feed-forward reconstruction table is not necessary. However, a brief discussion would help delineate COHERE4D's human-guided scale calibration from human-aware camera-motion refinement.
-
Implementation Details: Implementation details should be documented in greater detail. The method introduces chunk length L and overlap O, but does not provide their actual values. These choices significantly impact latency, correspondence availability, and cumulative alignment error. The authors should also specify the exact DA3, Pi3X, and VGGT-Ω checkpoints used, along with preprocessing and confidence settings. This information is particularly crucial as some backbone repositories contain multiple model variants and updated checkpoints.