acceptodds
Withdrawn from ICLR 2027

VEvo-Harness: Causally-Guarded Self-Evolving Harness for Agentic Image Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

10 views

Abstract

The central bottleneck in visual generation and editing has migrated from achieving pixel-level fidelity to satisfying densely constrained user specifications through increasingly agentic workflows. Prevailing agentic generation pipelines remain shallow generate-verify-rewrite loops treating VLM judgments as coarse visual feedback, either discarding repair experience or admitting ungated updates that poison skill library. With memorize and govern periods left open, they degrade into ungoverned iterators of repeated trial-and-error rather than continual self-evolving harness. Towards this issue, we present VEvo-Harness, the first causally-guarded continual self-evolving harness system for agentic visual generation. VEvo-Harness introduces four coupled innovations: (1) a visual traceback that densifies sparse judgments into root-cause attributions with a hierarchical verification tree; (2) a visual continual governance that admits only causally valid skill patches through counterfactual dual-arm image generation replay; (3) a lifecycle-managed harness state that hierarchically distills root-cause-anchored episodes across working, experience, and pattern tiers into a persistent skill library of routable and causally verified capabilities; (4) a heterogeneous sandbox policy that orchestrates a diverse action space of repair primitives, dynamically routing cost-aware interventions based on root causes to select the most effective strategy. VEvo-Harness surpasses state-of-the-art agentic methods and rivals proprietary generators (e.g., GPT-Image, Nano-Pro) while maintaining the continual harness evolution and generalization.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Agentic Image Generation, Self-Evolving Agents, Agent Harness

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_R6dLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VEvo-Harness, a self-evolving framework for image generation and editing with frozen generative backbones. The method decomposes failed visual constraints into diagnostic predicates, selects among rewriting, resampling, and parameter tuning, and consolidates repair experience into persistent memory and skills. Candidate skill updates are evaluated through paired image-generation replay before admission. Experiments with Z-Image-Turbo and Qwen-Image backends report improvements across generation and editing benchmarks, with additional studies of frozen-harness transfer, intervention selection, and harmful-update suppression.

Critical strengths
  • The evolve–freeze–evaluate experiment is the right test of persistent learning rather than additional adaptation during evaluation. Table VIII disables memory writing, skill promotion, consolidation, and policy updates before testing each checkpoint on the same unseen probe set. GenEval2 improves from 77.6 to 86.1 while average repair iterations decrease from 4.18 to 2.63. If this trajectory is confirmed (see the consistency concern below), it directly supports the practical value of accumulated harness state.

  • The paper includes useful controls for both update evaluation and experience reuse. Table VII compares paired and unpaired replay at the same stated replay size, reporting lower estimation variance and better downstream performance with pairing. Figure VII further compares root-cause-matched skills against random, shuffled, and prompt-matched skills while fixing the remaining system. These experiments help distinguish useful repair experience from simply supplying more context or drawing more candidates.

Critical weaknesses
  • The claimed causal guarantee is substantially stronger than what the admission procedure establishes. Eq. 4 estimates the effect of a patch on a particular prompt and visual contract, averaging over generation seeds. A positive estimate from four paired rollouts does not establish that the patch improves other requests, preserves previously acquired capabilities, or remains beneficial after subsequent updates. Pairing reduces sampling variance; it does not remove estimation error or establish cross-task validity. Moreover, each patch is scored on the same prompt and contract from which it was proposed (Algorithm 1, line 18), so a positive estimate is partly selected in rather than independently confirmed. The reported 2.1% harmful admission in Table VII and nonzero forgetting in Table IX support risk reduction, not guaranteed monotonic evolution. Likewise, increasing checkpoint averages across nine runs do not establish monotonicity within individual runs. This distinction matters because reliable capability accumulation, rather than ordinary local repair, is the paper’s central claim.

  • The governance mechanism does not cover the full persistent state described by the method. In Algorithm 1, a rejected trajectory still updates episodic memory, pattern memory, and the routing policy, all of which influence subsequent generation. Accepted patches are also merged into existing skills after verification, but the resulting merged skill is not explicitly re-evaluated, and neither are the 25-run consolidation passes or the per-skill merges from 8 to 4 entries (Appendix A.2.1); repeated rewriting of accumulated context is known to erode stored knowledge [R1]. Consequently, the deployed harness can differ from the intervention that passed the gate. Eq. 2 further describes this merge as an EMA of strategy embeddings without specifying how the result becomes an executable instruction. Learning from rejected experience can be useful, but these additional update paths prevent the patch-level test from certifying the complete harness update.

  • The principal governance metric cannot be interpreted consistently from the reported definitions. Table VII defines harmful admission as the fraction of admitted patches with negative held-out gain. The harmful-injection analysis instead defines it as Nadmitted∩harmful/NharmfulN_{\mathrm{admitted}\cap\mathrm{harmful}}/N_{\mathrm{harmful}}. These measure different quantities. Under the latter definition, a genuine Admit-All control must score 100%, yet the reported GenEval2 value at 50% corruption is 48.1%. This requires either a different denominator or additional filtering that is not specified. Other headline quantities raise the same question. Tables III and VIII report the same metrics at checkpoints of the same 300-episode evolution and coincide exactly at 0 and 300 episodes in all six shared metrics, yet differ at every common intermediate checkpoint (GenEval2 at 50/100/200 episodes: 79.12/80.85/83.74 vs. 81.5/83.4/85.2). Table XII reports 87.96/37.3/80.4 on DPG-Bench/RISE/WiseEdit for the stated default depth of 2, whereas Tables VIII, XIII, and XV report the default configuration as 88.14/37.6/80.6, which are the depth-3 values. Since harmful-admission rates provide the main evidence that governance prevents persistent contamination, the underlying counts and selection procedure need to be reconciled before this conclusion can be assessed.

  • The methodological advance over existing skill-based agents is narrower than the paper suggests. GEMS already combines atomic visual verification, hierarchical trajectory memory, and reusable skills. SkillOpt already optimizes skills around a frozen model using bounded edits, held-out acceptance, a rejected-edit buffer, and slow updates; VEvo-Harness itself cites this work for its edit budget. Outside visual generation, reflective, trace-driven evolution of textual components [R2] and empirically validated self-modification [R3] already establish governed self-improvement. The distinctive contribution is therefore the combination of visual failure-conditioned reuse and paired visual admission, not governed skill evolution itself. Table VII supports the benefit of pairing within the proposed system, but does not establish what the full framework adds over an existing visual agent equipped with validation-gated skill updates. Such a comparison would be more informative about novelty than comparisons against stateless or Admit-All variants. The headline comparisons are also sensitive to protocol: under the paper’s own shared-resource protocol (Table II), the GenEval2 margin over GEMS on Qwen-Image-2512 is 4.2 points rather than the 15.7 in Table 1, and GEMS itself rises from 70.4 to 79.8 between the two tables.

References

[R1] (ICLR 2026) Q. Zhang et al. Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models. arXiv:2510.04618.

[R2] (ICLR 2026, Oral) L. A. Agrawal et al. GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning. arXiv:2507.19457.

[R3] (ICLR 2026) J. Zhang et al. Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents. arXiv:2505.22954.

[R4] (ICML 2025) S. Zhang et al. Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems. arXiv:2505.00212.

[R5] (CVPR 2025) N. Ma et al. Scaling Inference Time Compute for Diffusion Models. arXiv:2501.09732.

Questions for authors
  • Can the authors compare against a GEMS-based agent equipped with SkillOpt-style bounded updates and a held-out visual acceptance gate, using the same generator, reasoning model, verifier, and resource budget? This would clarify how much improvement comes specifically from adaptive traceback and paired admission rather than adding persistent, validated skill learning.

  • Does the independent replay in Table VII use new seeds for the same prompt, genuinely unseen prompts, or both? Please report how admission-time gain predicts cross-task gain and regression on previously solved requests. Which admission rule produced the main results: the positive-effect rule in Eq. 8 or the margin of 0.02 selected in Table XIII?

  • Which persistent updates are actually subject to governance? Please provide a complete accepted and rejected update trace showing what enters episodic memory, pattern memory, the skill library, and the routing policy. How is the EMA operation in Eq. 2 implemented for executable skills, and is the final merged artifact re-evaluated before reuse?

  • Please provide the total candidate count, harmful candidate count, admitted count, and harmful admitted count for Table VII and Figure VI. Which denominator is used in each experiment, and what explains an Admit-All rate below 100% under the definition in the harmful-injection analysis?

  • How are the reference diagnoses in Table XI and the independent assessments in Figure VIII obtained, given that automated failure attribution remains unreliable even for text-only agents [R4]? Please specify the diagnostic-set size, reference-label construction, independent verifier, and human-rating protocol. These results are important evidence against verifier self-confirmation, but the manuscript also states that the study involves no human subjects; the scope of the reported human evaluation needs clarification. Relatedly, are the main-table scores computed with judges disjoint from the in-loop verifier? The loop optimizes a VLM’s answers to atomic constraint questions, and several benchmarks (e.g., DPG-Bench, WISE) are themselves scored by VQA- or VLM-based judges, so part of the gain may reflect verifier hacking [R5].

Reviewer_iaSzOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces VEvo-Harness, a framework that aims to improve image generation over time by evolving the agentic harness around a frozen image generator rather than updating the generator’s parameters. The harness combines four components. The authors evaluate the framework on image-generation and image-editing benchmarks, compare it with existing agentic pipelines, and conduct component ablations, continual evolution experiments, and generalization tests. The results suggest that persistent memory and controlled skill updates can improve performance over several tasks. But, the paper leaves open important questions about how unsuccessful experiences enter persistent memory, how locally verified updates remain useful across tasks, and how intervention specific causal effects are estimated.

Critical strengths
  • The paper addresses a real weakness of current agentic generation pipelines, the repair experience can be discarded or reused without sufficient screening. The idea of comparing proposed skill updates against an unpatched alternative before persistent admission is a meaningful way to address this problem.

  • Additionally, the component ablations and evolve- freeze-evaluate protocol address whether memory, updates, and intervention routing help the harness accumulate reusable capabilities. This is more convincing than demonstrating only a gain from additional generation attempts. Overall this is a relevant problem with a good design direction.

Critical weaknesses
  • The method explicitly gates admission into the skill library using counterfactual utility, but Algorithm 1 subsequently updates experience memory and distills patterns without making these operations conditional on the candidate passing the same gate. This appears inconsistent with the description of experience memory as storing successful repair trajectories. Since experiences and patterns can subsequently influence future generations, the paper should clarify how rejected or uncertain experiences are represented and prevented from becoming positively reinforced reusable strategies .

  • The paper does not clarify whether each new skill is checked for negative effects on previous tasks before being saved. It also does not clearly explain how it determines which action caused an improvement when multiple actions are used in one repair process.

Questions for authors
  • Algorithm 1 (Appendix A.2.1) appears to insert every trajectory into experience memory E and distill pattern memory P, even when its counterfactual utility fails the skill admission threshold. How are rejected or uncertain trajectories represented in these memories, and how are they prevented from influencing future generations as positively reinforced repair strategies ? Could the authors clarify the distinction between retaining a failed trajectory as negative evidence and promoting its associated strategy into a reusable pattern? (Suggestion: an ablation that prevents unverified trajectories from influencing pattern induction or retrieval would help isolate the effect of this design )
  • Algorithm 1 computes one counterfactual utility per trajectory and uses it to update action values, although a trajectory can contain several interventions. How is improvement attributed to individual actions?

Clarifications:

  • Section 4.1 describes Qwen3.5-27B as the core reasoning backend and Gemini-3.7-Flash as the judge for Visual Traceback sub-questions, while Appendix A.2.1 states that a single VLM backend serves both traceback diagnosis and counterfactual judging. Which model performs each role? Section A.5 (p. 38) is an empty heading. Please add this content or remove the references to it.
Reviewer_tp6fOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VEvo-Harness, a self-evolving harness for agentic image generation and editing. The framework combines visual traceback, persistent experience and skill memory, counterfactual verification, and adaptive intervention routing. The experimental results are promising, but I have several concerns about the novelty, causal claims, and experimental evidence.

Critical strengths
  1. The proposed framework provides a reasonably complete system for continual harness evolution. I particularly appreciate the attempt to verify skill updates before adding them to persistent memory, since unreliable accumulated experience can affect future tasks.

  2. The experiments cover a broad range of image generation and editing benchmarks. The authors also provide component-level ablations, comparisons between paired and unpaired replay, and evaluations of frozen evolved harnesses, which help understand the effects of different components.

Critical weaknesses
  1. The novelty relative to existing agent harness research is unclear. The paper introduces memory, experience accumulation, reusable skills, and evolving harness states as its main contributions. However, these ideas have already been studied extensively in LLM agents, including memory systems, experience learning, skill-based agents, and self-evolving harnesses. Although the paper mentions some related work, it does not sufficiently explain how VEvo-Harness differs from these existing approaches. This makes it difficult to identify the actual methodological contribution beyond integrating existing ideas into image generation.

  2. Some experimental results need further justification, and the presentation is poor. I have concerns about the selection of qualitative comparisons in Figure 3, particularly the croissant example involving Nano-Banana-Pro. The evolution curve in Figure 4(c) also continues to increase near the end of the experiment, making it difficult to understand whether the method has converged. Separately, the paper has serious formatting issues. Pages 7 and 8 are especially difficult to read, with almost no spacing between tables, figures, captions, and the surrounding text. These issues make it unnecessarily difficult to examine the experimental evidence.

Questions for authors
  1. Could the authors explain more concretely how VEvo-Harness differs from existing memory, experience learning, skill learning, and self-evolving harness methods? A more detailed comparison with closely related approaches, especially those already using persistent memory and reusable skills, would help establish the novelty.

  2. What exactly makes the proposed method causal? In particular, how does Visual Traceback distinguish a true root cause from a descriptive failure category identified by a VLM? I understand that paired replay estimates the effect of a skill patch, but I am less convinced that it establishes the causal mechanism behind the visual failure.

  3. How were the qualitative examples in Figure 3 selected? For the croissant example, I independently tested the prompt through the Gemini 3.1 Pro web interface and obtained a satisfactory result. I understand that this is not a controlled comparison with the exact Nano-Banana-Pro setup used in the paper. Still, it raises questions about the representativeness of the selected examples. Could the authors clarify the generation settings, sampling procedure, and criteria used to select the displayed outputs?

  4. In Figure 4(c), the performance of VEvo-Harness appears to be still improving at 300 episodes rather than reaching a clear plateau. What would happen if the harness continued evolving for more episodes? Would performance eventually saturate, continue improving, or potentially degrade due to accumulated experience? A longer evolution experiment or further discussion would help support the claim of sustained capability growth.

Reviewer_DFKHOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes VEvo-Harness, a self-evolving agent framework for image generation and editing with frozen underlying models. It combines visual failure tracing, update validation through same-seed paired replay, hierarchical persistent memory, and heterogeneous repair-action routing to consolidate useful interaction experience into reusable skills. The authors evaluate the framework across generation, editing, and downstream benchmarks, supported by ablations, frozen-checkpoint evaluations, and cross-benchmark transfer experiments.

Although the approach is potentially useful, the submission has substantive problems concerning numerical consistency, the accuracy of comparative claims, and the reproducibility of key evaluation protocols. The current evidence does not adequately support its broad superiority and causal-guarantee claims. I therefore lean toward rejection.

Critical strengths
  1. The research problem is worthwhile. Improving future performance through reusable experience while keeping the generator frozen is a meaningful direction. The paper extends the focus beyond per-query prompt optimization to the reliable updating of persistent memory.

  2. The paired validation mechanism is reasonably motivated. Comparing patched and unpatched outputs under identical random seeds can reduce sampling noise in estimated update effects. Table VII includes random admission, single-arm verification, Best-of-K, and unpaired two-arm verification, providing useful evidence for this design choice.

  3. The evaluation is broad and includes meaningful controls. The appendix provides shared-resource-ceiling comparisons, multiple task orders and random seeds, evolve–freeze–evaluate experiments, forgetting analyses, and independent-verifier and human evaluations. These are useful attempts to distinguish accumulated capability from per-query inference-time optimization.

Critical weaknesses
  1. Unexplained numerical inconsistencies undermine confidence in the experimental conclusions. In Table 2, after placing all five metrics on a percentage scale, the arithmetic mean for VEvo-Harness on Z-Image-Turbo is 67.97, whereas the reported average is 75.62. On Qwen-Image-2512, the corresponding values are 68.59 and 77.93. If a different normalization or weighting scheme is intended, its formula must be specified; otherwise, the aggregate improvements emphasized in the text cannot be reproduced. Moreover, Tables XIII and XVI both describe admission-margin sweeps with other settings fixed, yet at (ϵ=0.01\epsilon=0.01), they report GenEval2 scores of 85.4 versus 84.9 and KRIS scores of 83.91 versus 83.58, without explaining any protocol difference. These issues affect central results and hyperparameter conclusions.

  2. The comparative narrative emphasizes favorable metrics and overstates the breadth of superiority. Page 7 claims state-of-the-art performance among community models and leadership across knowledge-intensive metrics. However, Table I reports an overall RISE score of 37.8, below Know-Thinker’s 40.0, with lower Causal and Spatial scores as well: 47.7/37.0 versus 52.2/46.0. The claim of outperforming proprietary systems on Temporal reasoning also requires qualification: GPT-Image-1.5 achieves 57.6, compared with 45.8 for VEvo-Harness. Although the appendix acknowledges some weaknesses, the main text retains overly broad claims. The paper should present the complete performance trade-offs and establish statistical support for narrow margins.

  3. The budget controls do not establish comprehensive comparison fairness, and coverage of closely related methods is incomplete. Table II is a useful shared-resource-ceiling experiment, but it covers only one generator, three benchmarks, and one resource setting. Compared with GenRouter, VEvo-Harness uses more VLM calls (3,720 versus 3,300), more tokens (3.966M versus 3.780M), and higher latency. Consequently, the statements on pages 30–31 that it operates within GenRouter’s resource envelope and rules out a compute-only explanation are too strong. Accuracy–cost curves or comparisons under an explicit common cost measure are needed. Furthermore, the bibliography includes GenEvolve, which also addresses self-evolving image-generation agents, but the paper provides neither a quantitative comparison nor a sufficient explanation for its exclusion.

  4. The causal-guarantee and monotonic-evolution claims exceed what the experiments establish. Same-seed paired replay supports controlled estimation of an update’s effect under particular inputs, verifiers, and execution conditions. It does not automatically guarantee persistent benefits on future tasks. Under the default configuration, Tables VII and XVI still report a 2.1% harmful-admission rate, while Table IX reports nonzero forgetting of 0.74. These results support reduced harmful updating and forgetting, rather than absolute reliability or strict monotonic improvement. Monotonically increasing averages at sampled checkpoints in Table III also do not establish non-degradation for every update or individual run. The main claims should reflect the locality of verification evidence already acknowledged in the limitations.

  5. The independent-verifier and human evaluations lack sufficient methodological detail. Page 33 and Figure VIII provide an important additional evaluation, but the independent verifier’s identity, sample sizes and sampling procedure, number of annotators, blinding, disagreement resolution, and annotation agreement are not adequately specified. “Human Correctness” and “Accepted-Patch Gain” also require clearer operational definitions and uncertainty estimates. Because update admission directly depends on the verifier, these details are essential for assessing whether the framework improves genuine task performance or preferentially optimizes a particular evaluator.

  6. The manuscript shows conspicuous formulaic writing and insufficient editorial verification alongside its disclosed AI-assisted editing. Strong expressions such as “new paradigm,” “absolute reliability,” and “monotonic capability growth” recur without proportionate attention to qualifications and negative results. More concretely, the caption of Table VI on page 28 calls the proposed method “our KnowThinker,” although Know-Thinker is a baseline. The same page reports a KRIS overall score of 84.57, whereas Table V reports 84.83. These inconsistencies indicate inadequate checking of terminology, numerical results, and narrative claims. The authors already disclose AI-assisted editing; the concern is insufficient scientific and editorial verification, rather than AI use itself.

Questions for authors
  1. What normalization and weighting formulas define the averages in Tables 1 and 2? Can the authors provide the calculation scripts and reconcile the differing results in Tables XIII and XVI?
  2. How should the broad superiority claims be interpreted given the lower RISE overall score than Know-Thinker and lower Temporal score than some proprietary models? Have the key baseline differences been assessed using paired statistical tests?
  3. What are the exact resource ceilings and common cost measure in Table II? Does accounting include paired replay, verification, memory consolidation, and evolution? Can the authors provide multiple budget points and a comparison with GenEvolve, or justify its exclusion?
  4. What precisely is guaranteed by “causal” admission and “monotonic” evolution? Given nonzero harmful admission and forgetting, will the authors narrow these claims and report degradation statistics for individual runs?
  5. Which independent verifier was used, and what were the sample sizes, annotator counts, blinding procedures, disagreement-resolution rules, and confidence-interval estimators?
  6. Can the authors comprehensively reconcile method names, tables, numerical statements, and overstated claims, particularly “our KnowThinker” and the inconsistent KRIS overall scores?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.