- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents HumanSkill-4D, a training-free framework for language-based querying of animatable Gaussian avatars. A frozen MLLM interprets language queries and composes geometric operators to identify target regions and motion intervals. Detailed experiments demonstrate improvements in 3D and 4D referring segmentation over existing methods.
Critical strengths
-
Clear and well-motivated methodology. The decomposition of 4D language querying into spatial and temporal grounding is intuitive and effective. The overall framework is clearly structured and easy to follow.
-
Detailed and strong experimental results. The method gets substantial improvements over existing baselines on both 3D and 4D referring segmentation tasks. The ablation studies and experiments support the proposed design.
Critical weaknesses
-
Limited generalization across avatar reconstruction models. The proposed framework is intended to be training-free and reusable. However, all experiments are conducted on avatars reconstructed using 3DGS-Avatar. Evaluating more frontier 3D avatar reconstruction models, such as LHM++, would help establish whether the method generalizes beyond the specific avatar representation used in the paper.
-
Potentially unfair baseline comparisons. HumanSkill-4D relies on 3DGS-Avatar, which already uses SMPL-based geometry and pose-driven animation. However, the main 4D baselines rely primarily on RGB/video features, while HumanSkill-4D directly exploits the available SMPL joint trajectories for temporal grounding. The substantial performance gains may therefore largely reflect the use of existing pose information rather than the proposed methodology.
-
Limited robustness evaluation. Obtaining accurate SMPL trajectories can be challenging in real-world scenarios, especially under occlusions. In practical scenarios, estimated poses may contain noise, temporal jitter, or tracking errors. Since the proposed method heavily relies on SMPL poses for temporal grounding, errors in pose estimation may affect its performance. Therefore, an evaluation of the framework's robustness to inaccurate SMPL estimates is needed.
Questions for authors
- The evaluation contains only 40 queries, primarily involving fine-grained motions supported by the predefined operators. Could the authors provide more failure cases and discuss the limitations of the framework?
8 comments
Response to Reviewer drd9 (2/4)
W2. Potentially unfair baseline comparisons. HumanSkill-4D relies on 3DGS-Avatar, which already uses SMPL-based geometry and pose-driven animation. However, the main 4D baselines rely primarily on RGB/video features, while HumanSkill-4D directly exploits the available SMPL joint trajectories for temporal grounding. The substantial performance gains may therefore largely reflect the use of existing pose information rather than the proposed methodology.
Answer: Thank you for raising this concern. We would respectfully note that the main 4D baselines are the currently available methods closest to our task, language-based 4D Gaussian referring segmentation. Still, to clarify that the performance gains come from the proposed methodology rather than from the use of existing pose information, we have added two kinds of baselines that use the same SMPL joint trajectories but replace our temporal grounding. First, we use the intervals that skeleton-based temporal grounding models (MG-MotionLLM [1], MoChat-R [2], and TMR [3]) predict for each query from these SMPL trajectories. Second, we simply provide the SMPL trajectories to naive MLLMs (GPT-6 Sol and Qwen3.8-27B) in their prompts and let them predict the intervals for each query by themselves. In both cases, the regions are selected as in HumanSkill-4D, so only the temporal grounding differs from ours.
| Input | MLLM | Temporal grounding | vIoU | tIoU |
|---|---|---|---|---|
| SMPL | Qwen3.8-27B | MG-MotionLLM | 10.3 | 12.4 |
| SMPL | Qwen3.8-27B | MoChat-R | 14.0 | 18.2 |
| SMPL | Qwen3.8-27B | TMR | 13.9 | 19.6 |
| SMPL | GPT-6 Sol | Prompt | 21.7 | 28.7 |
| SMPL | Qwen3.8-27B | Prompt | 16.2 | 21.4 |
| SMPL | Qwen3.8-27B | HumanSkill-4D | 47.3 | 63.1 |
All of these baselines use the same SMPL input, yet they fall far below HumanSkill-4D, whether our temporal grounding is replaced with the skeleton-based models or with the naive MLLMs, even the commercial GPT-6 Sol. In contrast, once HumanSkill-4D is applied, the same Qwen3.8-27B nearly triples both scores and far exceeds the commercial MLLM. Therefore, the substantial performance gains largely reflect the proposed methodology, namely how it uses the existing pose information, rather than the use of this information itself.
Location in the revised paper: We have added these two comparisons to Sec. 5.4 (L472–501) with Tabs. 6 and 7 (p. 10).
Ref:
[1] Wu et al. MG-MotionLLM: A Unified Framework for Motion Comprehension and Generation across Multiple Granularities. In CVPR, 2025.
[2] Mo et al. MoChat: Joints-Grouped Spatio-Temporal Grounding Multimodal Large Language Model for Multi-Turn Motion Comprehension and Description. IEEE JBHI, 2026.
[3] Petrovich et al. TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesis. In ICCV, 2023.
Response to Reviewer drd9 (3/4)
W3. Limited robustness evaluation. Obtaining accurate SMPL trajectories can be challenging in real-world scenarios, especially under occlusions. In practical scenarios, estimated poses may contain noise, temporal jitter, or tracking errors. Since the proposed method heavily relies on SMPL poses for temporal grounding, errors in pose estimation may affect its performance. Therefore, an evaluation of the framework's robustness to inaccurate SMPL estimates is needed.
Answer: Thank you for raising this point. Following your suggestion, we have evaluated the robustness of our HumanSkill-4D to more realistic SMPL estimation errors. Specifically, we replace the provided SMPL trajectories with those estimated directly from single-view RGB videos using two recent pose estimators, GVHMR [1] and Human3R [2]. Unlike the provided SMPL trajectories, these estimated trajectories naturally contain pose inaccuracies, temporal jitter, and potential tracking errors, reflecting the practical challenges highlighted in your concern.
| Method | vIoU | tIoU |
|---|---|---|
| 4-LEGS | 1.8 | 14.1 |
| 4D LangSplat | 5.8 | 19.7 |
| OpenHuman4D | 7.0 | 24.5 |
| HumanSkill-4D with SMPL estimated by Human3R | 32.9 | 45.9 |
| HumanSkill-4D with SMPL estimated by GVHMR | 37.7 | 51.0 |
| HumanSkill-4D with provided SMPL (GT-like) | 47.3 | 63.1 |
While using estimated SMPL trajectories incurs slight performance degradation, HumanSkill-4D maintains strong temporal grounding performance despite realistic pose estimation errors. Notably, even with these imperfect trajectories, our method substantially outperforms the compared methods. These results highlight the practical effectiveness of HumanSkill-4D even when accurate SMPL trajectories are unavailable.
Location in the revised paper: We have added this evaluation to Sec. 5.4 (L502–512) with Tab. 8 (p. 10). The compared methods in the table above are from Tab. 3 (p. 7).
Ref:
[1] Shen et al. World-Grounded Human Motion Recovery via Gravity-View Coordinates. In SIGGRAPH Asia, 2024.
[2] Chen et al. Human3R: Everyone Everywhere All at Once. In ICLR, 2026.
Response to Reviewer drd9 (1/4)
Thank you for your insightful review. We appreciate your recognition of our well-motivated and intuitive design and of our detailed and strong experimental results.
Answer:Thank you for this suggestion. As you advised, we have added results with LHM++ [1] and two further avatar reconstruction models, SeqAvatar [2] and GauHuman [3], in place of 3DGS-Avatar. For each model, we only rebuild the semantic memory on its Gaussians and keep the target categories and motion programs unchanged.Overall, even the lowest results stay far above the best compared methods in the main paper, LangSplat† (61.9 mIoU in Tab. 2) and OpenHuman4D (7.0 vIoU in Tab. 3). Therefore, these results show that HumanSkill-4D, as a training-free and reusable framework, generalizes beyond the specific avatar representation used in the paper.
Location in the revised paper:We have added these results to Supp. D.3 (L1341–1362) with Tab. G (p. 25) and Fig. B (p. 26).Ref:[1] Qiu et al. LHM++: An Efficient Large Human Reconstruction Model for Pose-free Images to 3D. arXiv:2506.13766, 2025.
[2] Xu et al. Sequential Gaussian Avatars with Hierarchical Motion Context. In ICCV, 2025.
[3] Hu et al. GauHuman: Articulated Gaussian Splatting from Monocular Human Videos. In CVPR, 2024.