- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
CAST addresses monocular motion capture for arbitrary target skeletons. It combines learned local-rotation prediction with global bone-direction constraints and introduces GRACE, which corrects hierarchical rotation errors according to topology-dependent rotational observability and predicted directional reliability. Experiments cover matched-rig, unseen-topology, cross-skeleton, causal, and sparse-observation settings, with substantial improvements over the evaluated baselines.
Critical strengths
- Correcting local rotations with global bone-direction constraints offers a well-motivated approach to mitigating error propagation through the skeletal hierarchy.
- The method achieves consistent performance improvements in both matched-rig and cross-skeleton motion recovery while supporting real-time causal inference.
Critical weaknesses
The technical distinction from existing methods needs to be clarified. Adaptive HybrIK already addresses error accumulation through adaptive inverse kinematics, while TopoCap uses global orientation prediction. The paper should explain how GRACE differs from these approaches and what additional contribution its confidence weighting and learned gating provide.
Questions for authors
- How does GRACE differ from Adaptive HybrIK and TopoCap in handling error propagation? In particular, what do confidence weighting and learned gating add beyond these existing approaches?
- Could the authors ablate direction supervision, confidence weighting, learned gating, and hierarchical correction separately? Two useful controls would be to retain direction supervision without applying correction and to compare hierarchical correction with correction applied after full forward kinematics (FK).
- What input resolution, numerical precision, and compilation settings were used for each baseline? Were comparable optimizations applied to all methods? Since the methods use different temporal window lengths, please report online latency and offline throughput separately.
3 comments
I thank the authors for their response, which has addressed my concerns. I have no further questions.
Thank you for your follow-up and for confirming that our response has addressed your concerns.
In the revised manuscript, we will expand Table 3 with additional component ablations and update Table 5 with remeasured inference times using the optimized baseline implementations. The corresponding implementation and timing details, including input resolutions, numerical precision, and compilation settings, will be provided in the appendix.
Thank you again for your time and constructive feedback.
Thank you for the constructive feedback and helpful suggestions. Below, we clarify the differences from prior methods, provide component ablations, and report updated inference timings with optimized baselines.
Q1. Differences from Adaptive HybrIK and TopoCap. Adaptive HybrIK recovers rotations from predicted human joints through adaptive inverse kinematics, addressing bone-length mismatch with a body template. Our target skeletons have fixed bone lengths, and GRACE addresses rotation errors accumulated during FK. TopoCap first predicts global rotations, then recovers each child's local rotation by multiplying the inverse of its parent's global rotation by the child's global rotation. This conversion couples rotation prediction errors across parent–child relationships. GRACE instead starts from predicted local rotations and uses independently predicted global bone directions to correct composed orientations during FK. Each corrected orientation is passed to the next level.
The learned gate limits interference from inaccurate bone-direction predictions early in training. Confidence weights account for bone-specific occlusion and skeletal structure, giving more reliable directions greater influence. Ablations are reported in Q2.
Q2. Component ablations. We evaluate six CAST-B DINOv3 variants. CAST-B w/o GRACE removes direction supervision and correction. Direction-only retains direction supervision without correction. Uniform weighting gives valid directions equal solver weights, retaining the learned gate. Fixed gate sets g=1, retaining predicted confidence weights. Post-FK applies corrections after a complete, uncorrected FK pass. Full GRACE corrects during FK, propagating corrected orientations to children. Entries report MPJPE / PA-MPJPE (mm, lower is better).
Full GRACE performs best on Zoo and seen, while Post-FK performs best on unseen.
Q3. Baseline optimization and updated timings. Original Table 5 used official TopoCap and MCA V2 implementations without additional inference optimization. We now optimize both baselines.
DINO, T5 where used, and motion networks use BF16 with FlashAttention acceleration. CAST and MCA V2 use torch.compile. TopoCap is faster uncompiled. MCA V2 processes only actual joints, eliminating default padding to 150 joints.
We remeasured all methods on another NVIDIA A100-SXM4-80GB machine with PyTorch 2.7.0+cu128. CAST's timing setup is unchanged, with minor runtime variations between machines. TopoCap and MCA V2 use 224 × 224 inputs with DINOv3 and DINOv2, respectively. CAST uses 256 × 256 with DINOv3. Timing includes image encoding and motion prediction.
Each entry reports time (ms) / FPS. Offline time is per non-overlapping window of the listed length, with FPS = 1000 × Frames / Time. Causal time is per incoming frame, with online FPS = 1000 / Time.
We will update Table 5 values and supplement the appendix with baseline timing details.