- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces BlendCUA, a self-evolving computer-use harness for completing Blender tasks specified through natural-language instructions and reference images. Rather than mining successful trajectories, the method diagnoses defects in completed artifacts: an independent checker identifies failures, a curator attributes each failure to planning, execution, review, or generated views, and the system proposes a targeted harness repair. A repair is retained only when a resumed rebuild resolves the defect without introducing an attributable major regression. The harness is evolved on 20 curriculum tasks and evaluated on 10 held-out tasks, with additional analyses of the acceptance gate, defect attribution, component composition and reuse, transfer, and retraining.
Critical strengths
-
The outcome-driven defect-attribution mechanism is a clear and useful contribution for creative-software tasks, where an apparently plausible interaction trajectory can still yield a materially flawed artifact. Connecting observable artifact defects to specific harness components enables targeted repairs and is more diagnostically meaningful than indiscriminately compiling successful traces.
-
The empirical analysis is unusually informative for a harness-evolution study. The closest baseline is controlled under a comparable budget, and the gate, attribution, composition, reuse, transfer, and retraining experiments collectively probe the proposed mechanism. In particular, restoring rejected remedies substantially degrades judged quality and increases major-defect counts, providing direct evidence that the acceptance gate is consequential rather than incidental.
Critical weaknesses
-
The held-out evaluation contains only 10 tasks and a single generated build for each system-task cell. This design leaves many moderate differences statistically underpowered and does not characterize generation variability. Repeated builds across independent seeds, together with evaluation by Blender experts, would materially strengthen the evidence behind the reported gains.
-
The headline performance depends substantially on a strong, handcrafted initial harness, and the evolutionary loop is not shown to discover major conceptual workflow changes from a minimal interface. Moreover, Blender provides resumable and inspectable artifacts that are especially compatible with the proposed diagnosis-and-repair loop, so generalization to less structured computer-use environments remains uncertain.
Questions for authors
-
Can the authors repeat the held-out evaluation with multiple independent generation seeds and obtain an assessment from Blender experts covering instruction compliance, form fidelity, editability, and production readiness?
-
How much of the reported performance remains when BlendCUA starts from only the scripting interface, without the strong handcrafted H0 harness? This comparison would help separate the contribution of the evolutionary procedure from that of the initial harness.