SportCraft: Agentic 4D Reconstruction of Sports Scenes from Monocular Video

Monocular sports reconstruction must align venue geometry, human motion and equipment interactions despite ambiguous scale, rapid motion and changing contacts. Corrections tailored to one action or view can also fail when reused across videos. We propose SportCraft, an agentic framework for editable 4D sports reconstruction organized around two coupled feedback loops, with Codex serving as the supervisory agent for scene diagnosis, code proposals and update review. Agent-directed model fusion registers complementary pretrained predictions in a shared scene with consistent coordinates and timing. Knowledge-guided scene refinement forms the inner loop, using rendered feedback, venue references and phase-specific sports constraints to refine cameras, human motion, object trajectories and contacts. Visual, geometric and contact checks govern the acceptance of scene updates. Agent-driven tool evolution forms the outer loop: the agent records repair experience, generates and revises parameterized geometric functions with explicit applicability conditions, and uses replay and regression checks to review candidate revisions before reuse. Pretrained model weights remain fixed.