acceptodds
Under review as a conference paper at ICLR 2027

VETR: Execution-Grounded Self-Improving Agent for Symbolic Graphics Reasoning

Abstract

Visual agents for symbolic graphics reasoning depend on executable world representations, yet continued improvement remains bottlenecked by ambiguous supervision from final renders. Recent agentic systems refine scenes through repeated interaction, but they lack mechanisms to recursively extract actionable evidence from execution traces, overlooking which proposed symbolic edits survived, disappeared, or interfered during program execution. To address this, we propose VETR (Verified Execution Trajectories for Refinement), a framework that enables *agentic recursive self-improvement* over executable world representations for symbolic graphics reasoning. Within each task, VETR retains aligned program/render records with bounded runtime observations and introduces an *execution refiner* that compiles observed writes and state changes into constrained alternative programs. The refiner validates their endpoints and preservation conditions against captured runtime state; the ordinary visual verifier then assesses goal relevance. Repair construction makes no additional language-model calls, but can require additional executions. Evaluations on BlenderGym (280 tasks) and BlenderBench (27 released tasks) with GPT-5.6-luna and Qwen3.8-27B show that VETR consistently outperforms existing refinement strategies, including BlenderAlchemy and VIGA. Controlled ablations evaluate the contributions of execution refinement and role-asymmetric evidence organization under fixed generator-interaction caps.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
program synthesis, 3D scene editing, LLM agents, self-refinement, execution traces, symbolic graphics reasoning, Blender

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_rBX7OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes VETR, an execution-grounded agent for 3D scene editing in Blender. It combines candidate generation (width N), rendering-based visual verification, and iterative refinement. It is evaluated on BlenderGym (280 tasks) and BlenderBench (27 tasks) with GPT-5.6-luna and Qwen3.8-27B backbones. The paper claims VETR consistently outperforms VIGA, BlenderAlchemy and One-Shot baselines under a fixed generator-interaction budget.

Critical strengths

The paper presents an interesting perspective and a novel, engaging idea.

Critical weaknesses

From an experimental setup perspective, the claimed contributions are not clearly validated. In particular, the so-called recursive self-improvement is not reflected in downstream code repair, as the current experiments only show task-internal modifications, which can hardly be categorized as genuine recursive self-improvement. Furthermore, what is actually demonstrated in the experimental section does not align well with the main contributions claimed in the paper. This disconnect between claims and empirical evidence makes the paper somewhat difficult to follow.

Questions for authors
  1. The VIGA runner saves successful scene updates, whereas the VETR runner reloads the base scene for every execution. In Appendix C.1, rebuilding alone reduces N-CLIP by 33.9%. This is close to the 36.3% reduction of VETR (N=1) over VIGA in Table 11. Was execution refinement enabled in both arms of C.1? Can you report VETR vs. VIGA on BlenderGym with the same execution origin? Can you also report a refiner on/off ablation on BlenderGym?
  2. How often does the refiner find displaced writes, and how often is an alternative accepted and then delivered? Please break this down by channel. Appendix N indicates that each arm is a single run. The same GPT N=1 default yields 2.603/0.996 in the context cohort and 2.616/1.149 in the systems cohort. What is the run-to-run variance across seeds?
  3. Which N does the VETR row in Table 1 use? How many candidates per interaction does VIGA use? Against VIGA, the comparisons fail the paper's own significance criterion: GPT N=4 on BlenderGym (sign-test p = .185 for N-CLIP and .151 for PL, Table 11) and BlenderBench (p = .052 and .701, Table 13). For BlenderGym, the main text reports only the point estimates. How does this fit with the claims that VETR "consistently outperforms … VIGA" and that "its advantage remains substantial"?
Reviewer_WcntOpenReview ↗
Rating
2(−1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

VETR is an execution-grounded refinement system for program-based graphics editing. Existing agents normally judge progress from rendered images alone. VETR additionally records program execution and uses observed writes, bindings, and state changes to construct alternative programs. These alternatives are executed and checked against preservation constraints before being evaluated visually.

The method is tested on BlenderGym and BlenderBench with GPT-5.6-luna and Qwen3.8-27B. It consistently improves over one-shot generation, VIGA, and BlenderAlchemy. The paper also includes paired statistical tests and ablations of execution refinement, context separation, visual verification, and history length.

Critical strengths

The core idea is strong. A render does not tell the agent whether an intended program edit was overwritten, attached to the wrong resource, or lost during execution. Using the execution trace to construct and validate alternative programs directly addresses this ambiguity.

The evaluation is also convincing. The authors use two different backbones, two benchmarks, paired bootstrap intervals, sign tests, several complementary metrics, and targeted ablations. Disabling execution refinement causes a substantial drop, especially on the more compositional BlenderBench tasks.

Critical weaknesses

The main comparison fixes the number of generator interactions rather than total execution cost. VETR can perform as many as six additional repair executions for a submitted program. These repairs do not require another language-model call, but they still provide additional search and rendering opportunities. An equal-render comparison would make the source of the improvement clearer.

The current evidence is also specific to Blender and to tasks with target renders. The supported write types and preservation rules are manually tied to Blender’s execution model. It is not yet clear how well the method transfers to SVG, CAD, or tasks where several visually different scenes satisfy the request.

Questions for authors
  1. What happens when VETR and the baselines receive the same total number of Blender executions and renders?

  2. How often does a proposed change fall outside the supported write types or preservation contract?

  3. Does a semantic or human evaluation preserve the ranking when multiple outputs can satisfy the task?

  4. Can the authors demonstrate the same idea on a small SVG or CAD benchmark?

Reviewer_2wRPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

VETR augments iterative Blender editing with a deterministic execution refiner. The refiner uses observed writes, captured endpoints, and resource bindings to construct alternative programs, validates their effects and preservation conditions through execution, and returns a delivered result for ordinary visual assessment. The paper evaluates this stage together with history and role-context policies on BlenderGym and BlenderBench.

Critical strengths

Converting execution records into alternative programs is a concrete contribution beyond retaining logs. The separation between runtime acceptance and visual goal assessment is useful, and the paper limits preservation guarantees to captured state. This offers a meaningful interface for repairing iterative graphics edits.

The refinement on/off experiment provides direct component evidence. On 27 GPT BlenderBench tasks, refinement reduces mean photometric loss by 41.8%, with 24 wins and 3 losses (Table 12). Evaluation on another backbone and the persistent-scene replay further help establish practical relevance. The photometric ablation is the strongest evidence in favor of acceptance.

Critical weaknesses

The benefit of trace-derived repair is not separated from extra execution compute. Each submission can trigger up to six additional executions. Holding generator interactions fixed does not establish an advantage at a comparable execution budget; the width-1 versus width-4 study addresses a different intervention. Complete-system comparisons also change execution origin and role instructions. The controlled photometric result is convincing, but the N-CLIP on/off sign test is p = .122, limiting claims of uniform improvement across primary measures (Tables 12–13).

Important parts of the central repair policy remain implementation-dependent. Appendix B identifies the supported-property registry, owner matching, support grouping, projection scope, light reconstruction rules, numerical tolerances, and shortcut predicates as necessary for exact reproduction. These choices determine the generated and accepted candidates, so a precise mapping to the evaluated implementation is important for assessing the contribution. This concern is based on the manuscript; separately supplied code has not been inspected.

Questions for authors

What are the observed additional execution counts and wall-clock costs for refinement enabled versus disabled on the same 27 tasks? Can the benefit be placed on a quality-versus-execution-cost curve using the existing logs?

Where are the implementation-dependent selectors and tolerances listed in Appendix B specified in the supplied artifacts? Please give the exact rules or code mapping and state how often each repair channel is accepted.

Can the main comparison distinguish complete-system effects from the controlled refiner effect, and clearly separate mean reductions from comparisons satisfying both statistical criteria?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.