acceptodds
Withdrawn from ICLR 2027

BlendCUA: Self-Evolving Computer-Use Harness for Blender Use

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

This paper introduces BlendCUA, a self-evolving method for harness generation for an LLM agent carrying out Blender tasks given in natural language with reference imagery, covering modelling, materials, physics simulation and animation. Such an agent depends on its harness, the tools it can use and the skills that show it procedures, and building one by hand for every new family of tasks costs as much as the expert demonstrations it was meant to replace. Self-improving agents grow a harness by compiling successful trajectories into skills, which captures what the agent did rather than what its result got wrong. We instead evolve the harness from the defects in the agent's own finished work, through outcome-driven credit assignment. Starting from a hand-built harness, BlendCUA builds each task, has an independent checker name the defects in the finished model, traces each defect to the part of the harness responsible, and keeps a repair to that part, whether a plan correction, a skill or a tool, only if a rebuild shows the defect closed. Evaluated on held-out Blender tasks with judged and computed measures of quality, BlendCUA outperforms a harness learned from transcripts on every measure under the same agent and curriculum, reducing major defects by 29%. These results establish that the harness an agent needs for professional creative software can be grown from its own finished work rather than authored by an expert.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Computer-Use Agents, LLM Agents, Self-improving Agents, Agents, GenAI

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_JSteOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces BlendCUA, a self-evolving computer-use harness for completing Blender tasks specified through natural-language instructions and reference images. Rather than mining successful trajectories, the method diagnoses defects in completed artifacts: an independent checker identifies failures, a curator attributes each failure to planning, execution, review, or generated views, and the system proposes a targeted harness repair. A repair is retained only when a resumed rebuild resolves the defect without introducing an attributable major regression. The harness is evolved on 20 curriculum tasks and evaluated on 10 held-out tasks, with additional analyses of the acceptance gate, defect attribution, component composition and reuse, transfer, and retraining.

Critical strengths
  1. The outcome-driven defect-attribution mechanism is a clear and useful contribution for creative-software tasks, where an apparently plausible interaction trajectory can still yield a materially flawed artifact. Connecting observable artifact defects to specific harness components enables targeted repairs and is more diagnostically meaningful than indiscriminately compiling successful traces.

  2. The empirical analysis is unusually informative for a harness-evolution study. The closest baseline is controlled under a comparable budget, and the gate, attribution, composition, reuse, transfer, and retraining experiments collectively probe the proposed mechanism. In particular, restoring rejected remedies substantially degrades judged quality and increases major-defect counts, providing direct evidence that the acceptance gate is consequential rather than incidental.

Critical weaknesses
  1. The held-out evaluation contains only 10 tasks and a single generated build for each system-task cell. This design leaves many moderate differences statistically underpowered and does not characterize generation variability. Repeated builds across independent seeds, together with evaluation by Blender experts, would materially strengthen the evidence behind the reported gains.

  2. The headline performance depends substantially on a strong, handcrafted initial harness, and the evolutionary loop is not shown to discover major conceptual workflow changes from a minimal interface. Moreover, Blender provides resumable and inspectable artifacts that are especially compatible with the proposed diagnosis-and-repair loop, so generalization to less structured computer-use environments remains uncertain.

Questions for authors
  1. Can the authors repeat the held-out evaluation with multiple independent generation seeds and obtain an assessment from Blender experts covering instruction compliance, form fidelity, editability, and production readiness?

  2. How much of the reported performance remains when BlendCUA starts from only the scripting interface, without the strong handcrafted H0 harness? This comparison would help separate the contribution of the evolutionary procedure from that of the initial harness.

Reviewer_oH65OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

BlendCUA evolves a Blender agent's tools and skills from defects in its finished artefacts. It attributes defects to planning, execution, review or reference views, proposes targeted changes, and retains them after a rebuild verifies the repair. The model remains frozen. Evaluation covers held-out tasks, reuse of learned components, and transfer between models.

Critical strengths
  1. Turning artefact defects into reusable improvements addresses a practical need for agents in creative software. Stable defect identifiers, saved checkpoints and partial rebuilds make the proposed loop concrete and usable.

  2. The evidence extends beyond repairing the current task: H20 outranks H0 on nine of ten held-out tasks, and the paper documents later reuse of specific tools and skills. Cross-model transfer and cost reporting further support the system's practical relevance.

Critical weaknesses
  1. The novelty claim is too broad, and the comparison does not isolate the new contribution. Voyager already uses state-based critique and success verification to improve and retain skills; ACE accumulates reusable knowledge through reflection and curation. Prior work does more than record successful trajectories or check that code runs. The more specific contribution here is component-level attribution followed by defect-closure testing. However, B3 removes artefact checking, attribution and rebuilds together. Could the authors compare against a feedback-driven baseline using the same H0, artefact feedback and rebuild budget, without explicit component attribution? The existing attribution ablation is encouraging, but its seven-to-three task split provides limited evidence.

  2. The validation does not yet establish reliable repair attribution or the benefit of the acceptance gate. Restoring a checkpoint does not control stochastic generation in subsequent steps; an unchanged-harness replay is needed to distinguish repair effects from retry effects. Adding every rejected edit to H20 is also different from running the learning process without a gate. Evaluation uses one build per system on ten tasks, with model judges providing the main quality assessments. The comparison with human defect lists is useful, but does not validate the final system ranking; mesh quality can improve even when simpler outputs have worse form. Could repeated builds and a small blinded human comparison confirm the central gains?

Questions for authors

See Weaknesses.

Reviewer_WJa9OpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents BlendCUA, a Blender agent that improves its tools and skills based on defects found in finished 3D assets. Starting from a handcrafted harness, a checker identifies problems, a curator attributes them to parts of the system, and proposed changes are tested through rebuilding before being retained. The paper reports improvements on held-out tasks, but I am not convinced that it establishes a sufficiently distinct contribution over existing work on reflective agent improvement.

Critical strengths
  • The paper is clearly written, and the figures make the workflow and individual repair examples easy to follow.

  • Checking the finished asset without exposing the plan is a sensible way to catch requirements that the planner and step-level reviewer both missed.

  • The reuse examples, held-out evaluation, and experiments with a second model provide some evidence that the accumulated tools and skills help beyond the tasks that produced them.

Critical weaknesses
  1. The evaluation contains only ten held-out tasks with one build per setting, and there is no expert assessment of the final held-out outputs. The starting harness already contains 57 tools and 44 skills, while the additional compliance and form-fidelity gains over this starting point are small. Some reported numbers also need clarification: Table 2 says all six generated-view remedies were accepted, whereas Appendix K says only one of six was kept.
  2. GEPA already uses execution traces and final outcomes for credit assignment, proposes targeted updates to prompt components, and validates those updates, yet it is neither cited nor compared against. BlendCUA adds visual defect descriptions, executable tool edits, and checkpoint-based rebuilding, but the paper does not establish what these additions contribute beyond adapting reflective component optimization to Blender. The broad claim around “outcome-driven credit assignment” needs a much clearer distinction from GEPA.
  3. Section 2 describes prior skill-learning methods as accepting updates largely when the code runs, which is not an accurate account of Voyager. Voyager uses a critic to check actual task completion, provides feedback on failures, and stores skills after success verification. The paper therefore draws too sharp a line between prior trajectory learning and its own outcome-based learning; the meaningful differences need to be explained through what gets updated and how each update is validated.

Reference

  1. GEPA: REFLECTIVE PROMPT EVOLUTION CAN OUTPERFORM REINFORCEMENT LEARNING
  2. Voyager: An Open-Ended Embodied Agent with Large Language Models
Questions for authors
  1. The authors should explain precisely how BlendCUA differs from GEPA’s outcome-informed component updates and ACE’s failure-driven context curation, identifying which parts constitute a new mechanism and which are adaptations for Blender.

  2. Please compare against GEPA and ACE adapted to the same Blender setting, with the same starting harness, access to outcome feedback, and adaptation budget, and document any differences in what each method is allowed to modify.

  3. Please revise the discussion of Voyager’s verification process and clarify what the proposed acceptance gate adds beyond checking task success before retaining a reusable skill.

  4. I would like to see the durable harness updates tested without directly patching the current task’s plan, together with an unchanged-harness rebuild control, to separate reusable improvement from direct correction and retry effects.

Reference

  1. Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.