- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper proposes VETR, an execution-grounded agent for 3D scene editing in Blender. It combines candidate generation (width N), rendering-based visual verification, and iterative refinement. It is evaluated on BlenderGym (280 tasks) and BlenderBench (27 tasks) with GPT-5.6-luna and Qwen3.8-27B backbones. The paper claims VETR consistently outperforms VIGA, BlenderAlchemy and One-Shot baselines under a fixed generator-interaction budget.
Critical strengths
The paper presents an interesting perspective and a novel, engaging idea.
Critical weaknesses
From an experimental setup perspective, the claimed contributions are not clearly validated. In particular, the so-called recursive self-improvement is not reflected in downstream code repair, as the current experiments only show task-internal modifications, which can hardly be categorized as genuine recursive self-improvement. Furthermore, what is actually demonstrated in the experimental section does not align well with the main contributions claimed in the paper. This disconnect between claims and empirical evidence makes the paper somewhat difficult to follow.
Questions for authors
- The VIGA runner saves successful scene updates, whereas the VETR runner reloads the base scene for every execution. In Appendix C.1, rebuilding alone reduces N-CLIP by 33.9%. This is close to the 36.3% reduction of VETR (N=1) over VIGA in Table 11. Was execution refinement enabled in both arms of C.1? Can you report VETR vs. VIGA on BlenderGym with the same execution origin? Can you also report a refiner on/off ablation on BlenderGym?
- How often does the refiner find displaced writes, and how often is an alternative accepted and then delivered? Please break this down by channel. Appendix N indicates that each arm is a single run. The same GPT N=1 default yields 2.603/0.996 in the context cohort and 2.616/1.149 in the systems cohort. What is the run-to-run variance across seeds?
- Which N does the VETR row in Table 1 use? How many candidates per interaction does VIGA use? Against VIGA, the comparisons fail the paper's own significance criterion: GPT N=4 on BlenderGym (sign-test p = .185 for N-CLIP and .151 for PL, Table 11) and BlenderBench (p = .052 and .701, Table 13). For BlenderGym, the main text reports only the point estimates. How does this fit with the claims that VETR "consistently outperforms … VIGA" and that "its advantage remains substantial"?