- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper investigates the failure of compositional world-space human mesh recovery (HMR) methods under high camera dynamics, using a curated set of K-Pop broadcast dance videos as a “K-Pop stress test.” The authors attribute severe world-space trajectory drift to a single, constant sequence-level scalar scale mismatch between independently estimated visual odometry (VO) translations and camera-space HMR translations. They propose a closed-form weighted least-squares estimator for this scale, fitted over detected stationary foot-contact intervals, and integrate it into a multi-stage pipeline that includes Physically Grounded Optimization (PGO).
The application scenario is relevant, and the integrated system produces visually coherent multi-person trajectories. However, the central diagnostic claim is not adequately established. The constant-scale assumption is questionable under broadcast cameras with continuously changing focal lengths; clips that fail the VO time-reversal shuttle test are discarded, undermining the claim that VO remains self-consistent under high camera dynamics; and the paired-camera evaluation uses monocular PromptHMR predictions as pseudo-ground truth. The evaluation also relies heavily on 2D reprojection and smoothness/contact metrics that do not establish world-space accuracy and are directly related to terms optimized by PGO. On EMDB-2, the proposed method degrades accuracy relative to the PromptHMR baseline. Finally, the extensive preprocessing and 3,000-iteration PGO stage make it difficult to isolate the contribution of the claimed closed-form scale estimator, while the reported ablations suggest that PGO can absorb much of the estimator’s contribution.
Critical strengths
-
Practical Problem Scenario: Highlighting the vulnerability of compositional HMR pipelines under aggressive camera movements (e.g., dolly zooms and rapid panning) and dense multi-person choreography is relevant for in-the-wild media applications.
-
Visually Coherent Qualitative Outputs: The integrated system produces visually pleasing multi-person trajectories and reduces inter-person collisions in complex dance sequences.
Critical weaknesses
-
The core diagnostic claim and evaluation do not establish reliable world-space accuracy. The proposed single, constant sequence-level scale mismatch is theoretically questionable for broadcast videos with continuous optical zooming and time-varying perspective effects. The paper also discards clips that fail the VO time-reversal shuttle test, uses monocular PromptHMR predictions as pseudo-ground truth in the paired-camera reference set, and evaluates the large K-Pop set mainly with proxy metrics that cannot establish world-space accuracy and are directly affected by the optimization objectives. On EMDB-2, the proposed method degrades W-MPJPE from 388.66 mm to 415.03 mm and WA-MPJPE from 139.65 mm to 158.50 mm.
-
The contribution of the closed-form scale estimator is not isolated from preprocessing and PGO. The pipeline combines ViPE, custom dynamic-foreground prompts, filtering, SAM2.1/YOLO11x tracking, RANSAC floor fitting, heuristic gap filling and contact blending, and 3,000 iterations of PGO that itself includes a learned residual scale. Appendix ablations suggest that PGO can absorb substantial differences in the closed-form initialization and can improve trajectory smoothness even without the proposed closed-form correction. The main baseline comparisons do not apply the same preprocessing and PGO, making the contribution of the proposed estimator difficult to determine.
Questions for authors
-
Unsubstantiated Core Claim and Oversimplified Problem Attribution (Scalar Scale Mismatch). The central claim is that world-space drift under high camera dynamics can be explained by a single, constant sequence-level scalar scale mismatch, (Eq. 4). This modeling is theoretically questionable for broadcast K-Pop videos, where continuous optical zooming (time-varying focal length ) and nonlinear perspective distortions are common. As the authors acknowledge in Appendix B (line 962) and D.3, focal-length error directly shifts camera-space depth estimation proportionally. When a camera zooms continuously during a shot, the effective scale bias between HMR depth—which assumes a fixed per-clip focal length in Step 2—and VO translation may be time-varying () and nonlinear rather than a constant scalar. How do the authors justify modeling a potentially dynamic focal-length and perspective-distortion problem as a static scalar mismatch?
-
Contradiction in Time-Varying Scale Analysis (Table 7). In Table 7, the authors justify using a constant scale by showing that a time-varying scale severely degrades Jitter (26.87 to 131.57 mm) and Foot Skating (17.94 to 32.58 mm). However, this does not establish that the true physical mismatch is constant; it may instead indicate that the stationary-interval estimator is too ill-conditioned and noisy to estimate local scales without injecting discontinuous step changes into the camera trajectory. How can the authors conclude that the true error is a constant scalar when broadcast cameras actively zoom within shots?
-
Selection Bias from Pre-Filtering Clips with the Shuttle Test. The paper claims in Section 1 and Section 5 that “camera estimation remains self-consistent under high camera dynamics, so the failure must lie in composition.” However, Section 4 (line 199) and Appendix F reveal that all K-Pop clips that failed the ViPE time-reversal shuttle test were explicitly discarded. The dataset is therefore filtered to retain clips where VO succeeds before the paper concludes that VO does not fail under high camera dynamics. How do the authors rule out this survivorship bias, and how many clips were excluded by the shuttle test?
-
Flawed “Ground Truth” in the Paired-Camera Reference Set (Table 1). The paper evaluates metric accuracy (WA-MPJPE, W-MPJPE, and Scale Ratio) on 11 video pairs (989 frames) using PromptHMR predictions from a fixed venue camera as “Ground Truth” (Section 4, line 206). PromptHMR is a monocular estimation method, not a calibrated multi-view motion-capture or LiDAR system. On wide-angle venue shots where 6–12 performers are far from the camera, monocular HMR can suffer from severe depth ambiguity, scale bias, and inter-person depth-ordering errors. Using one monocular model’s predictions as ground truth to benchmark other models—and claiming superiority over PromptHMR itself—is scientifically questionable. Can the authors validate these results using independent calibrated multi-view or sensor-based ground truth?
-
Circular Reasoning in Stationary-Interval Detection. In Section 6 (line 365), the authors detect stationary performers for estimating using a directional gate that rejects intervals if , arguing that “a stationary performer drifts only along the camera’s path (Eq. 4).” This assumes Eq. 4 to filter the data and then uses the filtered data to support Eq. 4 and fit . In dance choreography, performers may take small steps or sway parallel to the camera axis, passing both the partial foot-contact threshold () and the cosine gate and thereby corrupting . How do the authors quantify or rule out this source of bias?
-
Severe Entanglement of the Closed-Form Scale Correction with PGO and Preprocessing. The paper presents the closed-form scale correction (Eq. 6) as its core technical contribution, but Appendices B and C describe a heavily engineered six-step pipeline: ViPE with custom dynamic-foreground text prompts, One Euro filtering, two-pass SAM2.1/YOLO11x tracking, RANSAC floor fitting, and 3,000 iterations of Adam-based PGO with seven loss terms (). PGO also includes a learned residual scale . How much of the reported improvement is attributable to the closed-form estimator rather than the preprocessing and PGO stages?
-
Appendix Ablations Suggest That the Closed-Form Estimator May Be Largely Redundant. Appendix Tables 6 and 9 raise concerns about the necessity of the closed-form estimator.
- In Table 6, reducing the pooled stationary intervals from 125 to 3 changes the closed-form estimate from 0.310 to 0.443, yet PGO absorbs almost the entire difference through its learned , producing nearly identical final scales (0.318 versus 0.320). If PGO’s learned can recover essentially the same scale regardless of the closed-form initialization, what is the actual contribution of the closed-form estimator?
- In Table 9, on stationary clips, raw displacement is 2.334 m. PGO alone, without closed-form drift correction, reduces displacement by 75% to 0.581 m and reduces Jitter from 105.1 mm to 12.46 mm. Adding the closed-form drift correction leaves Jitter essentially unchanged at 12.38 mm.
- In the main text (Tables 2 and 3), “Ours” is compared against PromptHMR, TRAM, and GVHMR without applying the authors’ PGO, SAM2.1 tracking, or ViPE preprocessing to the baselines. Please disentangle (a) backbone replacement (ViPE + SAM2.1), (b) closed-form alone without PGO, (c) PGO alone with learned initialized at zero and without closed-form , and (d) closed-form + PGO. These controlled comparisons should be reported consistently across the relevant tables.
-
Questionable Validity and Value of Proxy Metrics (Tables 2 and 3). Evaluation on the 616 K-Pop clips relies entirely on 2D reprojection (PCK against YOLO11n-Pose and Mask IoU) and smoothness heuristics (Jitter and Foot Skating).
- As the authors acknowledge, 2D reprojection is blind to world-space scale errors: TRAM and HAC achieve higher PCK and Mask IoU than “Ours” in Table 3 despite severe world-space drift.
- In Appendix C, PGO explicitly optimizes 3D joints against ViTPose 2D keypoints through , and directly penalizes acceleration/velocity through (weight 1) and foot-contact velocity/stickiness through (weight 500) and (weight 100). Evaluating the method with Jitter and Foot Skating when these exact finite-difference terms are heavily penalized during test-time optimization risks being trivially gamed: setting world velocity to zero would achieve zero Jitter and Foot Skating. Thus, Tables 2 and 3 do not establish world-space trajectory accuracy. What independent metric or ground truth can validate the recovered world-space trajectories?
-
Performance Degradation on the Standard Ground-Truth Benchmark (EMDB-2). On the only benchmark with real sensor ground truth evaluated in the paper (EMDB-2, Section 7.1, lines 454–459), the proposed method degrades world-space trajectory accuracy relative to PromptHMR: W-MPJPE worsens from 388.66 mm to 415.03 mm (+26.4 mm error), and WA-MPJPE worsens from 139.65 mm to 158.50 mm (+18.9 mm error). Furthermore, the PromptHMR reproduction on EMDB-2 reportedly obtains 388.66 mm W-MPJPE and 139.65 mm WA-MPJPE at line 456, whereas Appendix Table 4 reports PromptHMR-Vid achieving 216.5 mm W-MPJPE and 71.0 mm WA-MPJPE on EMDB-2. Please explain this substantial discrepancy and why a general drift-correction method harms accuracy on data with real ground truth.
-
Missing Evaluation on Existing Multi-Person, High-Dynamic Ground-Truth Datasets (WorldPose/DCM). Table 5 lists WorldPose, which contains in-the-wild moving and zooming camera motion with LiDAR multi-camera ground truth for multiple people, and DCM, which contains moving and zooming dance videos with ground truth. Why is the method not evaluated on WorldPose or DCM instead of relying on 11 K-Pop clips with pseudo-ground truth? Alternatively, can the authors evaluate on synthetic high-camera-dynamic sequences, for example by rendering BEDLAM/AMASS dances with aggressive virtual broadcast-camera trajectories and focal zooms, where exact 3D ground truth is available?
-
Ad Hoc Heuristics in Gap Filling and Contact Blending. Appendix B (lines 1014–1018) states that missing detections are handled by linearly interpolating 3D translations, damping boundary velocities by 0.8 per frame, and artificially forcing contact logits toward 1.0 after 15 frames so that PGO plants the feet. How much of the stability reported in Table 3 comes from these hand-crafted track-filling heuristics rather than the proposed scale correction?