- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
4DHOISolver reconstructs whole-body human-object motion from monocular video in three stages. It starts with agent-assisted initialization and geometric optimization, uses sparse point-pair annotations to repair difficult sequences, and then refines the human motion in simulation. The evaluation covers InterCAP and 200 in-the-wild sequences, with a 20-case annotation study and physical-refinement experiments on 45 motions.
Critical strengths
- For me, the repair tool is the strongest contribution. The point-pair interface gives users a direct way to correct contact and alignment, and the 20-case study shows less hands-on annotation time, lower penetration, and better ratings from blinded evaluators. This is concrete evidence that the tool can improve a practical reconstruction workflow.
- I appreciate the breadth of the in-the-wild evaluation: 200 sequences across 92 object categories. It tests the workflow across varied appearances and interactions, which makes its potential usefulness more convincing than a few selected demonstrations would.
Critical weaknesses
- I'm less convinced by the automatic stage. On the wild set, its penetration is higher than CARI4D's (33.99 versus 20.14 mm). The large contact-distance improvement after repair is measured against the same annotations used to guide it. This leaves limited independent evidence that the automatic reconstruction improves on prior methods, which matters because it is presented as a main contribution.
- The contribution of physical refinement is also unclear. Object trajectories are prescribed, object gravity is disabled, and a separate PPO controller is trained for each sequence. Without a contact-reward ablation, I cannot tell what the proposed term adds beyond the underlying tracker. The results support a narrower tracking setup than a general claim about physically coupled human-object interaction.
Missing references: AGILE: Hand-Object Interaction Reconstruction from Video via Agentic Generation (SIGGRAPH 2026), HSImul3R: Physics-in-the-Loop Reconstruction of Simulation-Ready Human-Scene Interactions (ECCV 2026).
Questions for authors
- Is there an independent pose or contact reference for evaluating the automatic stage on wild videos? Could you also clarify how the 20 repair-study cases were selected, since that affects how broadly I can interpret the workflow results?
- Do you have a contact-reward ablation on the same 45 motions with other settings fixed? Please also report how many motions contribute to each penetration metric after unreliable meshes are excluded. These details would determine how much support I place on the physical-refinement result.
- Could you give the processing time and manual effort for each stage, including per-sequence PPO training? The current hands-on timing is useful, but the full cost would affect my assessment of the claimed efficiency.
2 comments
Thank you for the response and the additional experiments. The InterCAP repair results and contact-reward ablation address some of my concerns.
However, the automatic result still has higher penetration than CARI4D on the wild set. The gains from the contact reward are also modest, and penetration is evaluated on only 20 of the 45 motions. Please make these limitations, the prescribed object motion, and the full processing cost clear in the paper.
For now, I’ll keep my score at 2.
Automatic Reconstruction, Contact-Reward Ablation, and Evaluation Costs
1. Automatic reconstruction and contact evaluation
We agree that our automatic result has higher penetration than CARI4D (33.99 vs. 20.14 mm). Contact and penetration should be considered together: severe contact errors, defined as a per-video median contact distance above 20 cm, occur in 24/200 videos (12%) for our method and 48/200 (24%) for CARI4D. On the 94 videos where both methods achieve contact distance at most 20 cm and penetration p95 at most 5 cm, our median contact distance improves from 66.32 to 60.24 mm, while penetration increases from 16.14 to 18.45 mm. We will report the full-set results alongside this subset analysis and state the selection criteria explicitly.
The contact annotations are not inputs to either automatic method, so they provide an independent measure of automatic contact alignment. After point-pair repair, the same annotations become optimization targets; we interpret the larger improvement at that stage as constraint satisfaction, not independent reconstruction accuracy.
2. Wild-video reference and repair-study selection
We do not have independent 3D pose or contact ground truth for the wild videos. Table 2 therefore evaluates contact and physical consistency rather than absolute reconstruction accuracy. The 20 repair-study cases were selected from internet and self-recorded videos with visible errors in the automatic results. They cover 20 different object categories and varied interactions. We will clarify that this is a targeted repair study and describe its selection procedure.
3. Contact-reward ablation and penetration sample counts
We completed the contact-reward ablation on the same 45-motion test set with all other training settings fixed, using the original evaluation protocol. Adding the reward reduces reference-tracking MPJPE from 95.25 to 94.62 mm, angular jerk from 195.60 to 183.38 rad/s³, penetration depth from 3.15 to 2.79 mm, and penetrating frames from 19.39% to 17.96%. These results support the contribution of the contact reward beyond the base tracker under prescribed object motion. Regarding the effective sample counts, some object meshes are non-watertight and do not support reliable signed-distance evaluation, so these motions were excluded from the penetration metrics as described in Appendix C.4. Both penetration metrics use 20 motions. This mesh-validity filtering applies only to penetration evaluation, not to MPJPE or angular jerk.
4. Processing time and manual effort
Our efficiency claim concerns annotation and geometric optimization. For a 273-frame clip matching the benchmark’s median automatic-input length, VLM-guided automatic optimization takes 2.26 min, including Qwen and frame preparation and 1,500 Adam steps. In the 20-case study, hands-on annotation takes 40.63 s per video, while point-guided geometric optimization takes approximately 1.82 min for a typical-length clip. Physics-based PPO is a separate, optional refinement stage. Its training costs 7.38 GPU-hours per sequence for our method versus 18.09 GPU-hours for the adapted RePHO baseline, accounting for both GPUs used in training. We will report the timing protocol and hardware in the supplement.
5. Related work
Thank you for identifying the missing references. We will add and discuss AGILE and HSImul3R in the related-work section.