Measuring Frontier Code Agents at Digital-Twin Scene Construction

Simulation-based robot learning is gated by a manual step that repeats at every deployment site: constructing a physical digital twin of the scene. This paper measures how far unmodified, off-the-shelf code agents can automate that step, a setting referred to as Frontier Agent as Scene Constructor (FASC). Frontier code agents such as Claude Code and Codex, each used as shipped, rebuild eight BEHAVIOR-1K rooms from one photograph, an asset catalog and the simulator. Each reconstruction is scored against the room's known 3D layout on ACDC's sim-to-sim protocol, with memorisation and file-access contamination controlled. The strongest agent on the rooms, Codex with GPT-6 Astra, retrieves the exact asset model for 66% of the scored objects and places objects within 14 cm of ground truth across repeated runs. Claude Code with Opus 5, released a little over one month earlier, retrieves the exact asset model for 52% of the scored objects, against 6% for the same agent with Opus 4.6, released under six months earlier, and places objects within 39 cm against 153 cm, a 4× reduction in placement error over that predecessor. A gap to ACDC's curated pipeline, at 6 cm, remains. Next, the two strongest agents on the rooms both extend to outdoor farm scenes, where no calibrated camera or 3D ground truth exists, using a procedural plant model and common asset libraries. Validated against their photographs in DINOv2 embedding space, 24 of the 46 GPT-6 Astra twins and 22 of the 46 Opus 5 twins rank their own photograph first among the 46 reference photographs in cosine similarity. Together with prior results on trajectory generation, these findings indicate that a generic code-agent harness covers a growing share of embodied tasks, and that its capabilities improve with each frontier-model generation while requiring no embodiment-specific harness engineering.