PGM: Agentic Spatial Reasoning through Programmable Geometric Model

Spatial reasoning from images requires measuring object relations and evaluating how they change under hypothetical scene modifications. Connecting these computations requires a scene representation that preserves object identity and keeps geometry consistent across successive operations. We introduce the Programmable Geometric Model (PGM), which couples reconstructed scene geometry with executable operations in a structured Python object. PGM organizes object point clouds, oriented bounding boxes, semantic orientations, and camera poses in a shared coordinate system. Its editable object states allow measurements and rendered views to reflect the same scene configuration after each modification. A Program Agent generates Python programs to query, inspect, and modify PGM, then uses execution results and diagnostics to guide subsequent programs. The scene state persists across executions, allowing dependent computations to build on earlier operations. This formulation supports quantitative queries and hypothetical reasoning through explicit geometric computation, with no task-specific training. Experiments on MMSI-Bench, MindCube-1K, and Omni3D-Bench show improvements across multiple agent backbones. Ablation studies further support the contributions of scene editing and iterative program interaction.