- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
SportCraft reconstructs editable 4D sports scenes from monocular videos. It combines pretrained estimates of human motion, scene geometry, and object locations in a shared coordinate frame. Codex supervises a refinement loop that inspects rendered feedback, selects editable variables and sports constraints, and calls numerical tools to update cameras, human motion, and equipment trajectories. Candidate updates are checked before being retained. The framework also records repair experience and proposes reusable geometric tools for subsequent videos. Experiments on 13 videos across 12 sports evaluate source-view reconstruction consistency against OVOW and RHINO, with additional qualitative results and an ice-hockey ablation.
Critical strengths
1. Combining sports knowledge with feedback-guided refinement is a useful approach to scene reconstruction. Camera, human, and equipment estimates can disagree even when each looks plausible on its own. The framework addresses this by selecting which variables to refine and applying constraints such as venue dimensions, support, grasping, and flight only when the observations and interaction phase support them. Checking candidate updates before retaining them also helps avoid fixing one component at the expense of another. I find this a promising system design for bringing complementary predictions into a coherent sports scene.
2. The results show improved source-view consistency across a range of sports. In Table 1, human reprojection error decreases from 180.5 px for OVOW to 28.2 px for SportCraft, while human and object IoU also improve. The evaluation covers 12 sports, and the qualitative results show reconstructions of seven of them, including racket sports, ball games, and close human interactions. These results support the usefulness of the overall system beyond a single sports setting.
Critical weaknesses
1. The benefit of agent-guided refinement needs a quantitative ablation. Table 1 compares complete systems, while Figure 4 only provides a qualitative ablation on one ice-hockey example. I would like to see a quantitative comparison with a fixed refinement pipeline using the same predictors, constraints, and numerical tools, to understand how much the agent’s diagnosis and refinement decisions improve reconstruction quality.
2. The evaluation leaves 3D geometry and interaction accuracy unclear. The reported metrics measure agreement with source-view keypoints and masks. They do not directly establish whether depth, contact, and motion trajectories are recovered accurately. Evaluation on a subset with 3D ground truth or additional views would help assess these aspects.
Questions for authors
- With the same predictors, constraints, and numerical tools, how much does agent-guided refinement improve reconstruction quality over a fixed pipeline? Please provide a quantitative ablation.
- Some supplementary videos contain more than two people, but the reconstructions appear to include only two. Does the method support reconstructing all visible people, and how are participants selected?
- How much manual intervention is needed, including reviewing venue features and masks? What are the typical agent-call count and end-to-end runtime per video?