When Visual Programs Fail Silently: An Execution-Guided Repair Agent for Spatial Reasoning

Visual program synthesis makes spatial reasoning inspectable, but an incorrect perception output can pass through a well-formed program without raising an exception. We present PitCrew, an execution-guided repair framework that turns failed perception calls and inconsistent intermediate results into actionable program revisions. A question-conditioned Scene Captioner provides supplementary visual context for synthesis. The Program Executor records tool calls, and a deterministic Consistency Checker organizes answer-validity failures and disagreements with the scene context into feedback for a Repair Agent. The Repair Agent uses the failed program, execution trace, and scene context to localize the issue and revise perception queries or program logic within a bounded budget. Mask-aware depth and size operators support both initial and repaired executions. PitCrew achieves 49.8% on Omni3D-Bench and 89.4% on CLEVR, improving on the original VADAR configuration by 9.4 and 35.8 percentage points without task-specific training. Matched-backbone controls and component ablations support the utility of the combined system.