- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
- Method. The paper proposes EvoSkillGen, a self-evolving framework for agentic text-to-image generation and image editing, which represents workflows as executable Skills composed of meta-operations.
- Two-level learning. (1) L1 selects or creates a Skill for each task. (2) L2 uses checklist feedback and execution traces to revise, distill, and consolidate Skills, retaining updates only if they win pairwise image comparisons.
- Evaluation. The paper introduces GenAgentEval, built from six existing benchmarks, and reports consistent gains over agentic baselines with relatively low workflow-time overhead.
Critical strengths
- Interpretable workflow representation. Skills are explicit, reusable, and human-readable, and they evolve without updating model weights.
- Consistent improvements. EvoSkillGen achieves the best average results across both agents, both judges, and both modalities, and the ablations confirm the contribution of L2.
Critical weaknesses
- Insufficient ablations. (1) The contribution of the Prompt Compiler is not isolated. (2) The selective acceptance mechanism in L2 is also not ablated. It remains unclear how the method would perform if proposed updates were accepted without pairwise comparison.
- Incomplete cost analysis. (1) The reported workflow-time overhead excludes image generation, which does not reflect the actual cost of each method. The end-to-end time to obtain the final image should be reported instead. (2) The computational cost, such as the number of API calls and token consumption, is not reported.
- Missing intermediate baselines. The current baselines are either Direct, with no agent overhead, or full agentic pipelines. Simpler alternatives in between are not considered, for example: (1) single-call prompt rewriting with the same agent, with or without thinking mode, before passing the prompt to Seedream-4; and (2) direct generation followed only by RefineLoop, which iteratively checks the generated image against the request, produces a corrected candidate, and keeps it only if it wins a pairwise comparison, for up to five rounds, without using any Skill.
Questions for authors
See weaknesses 1-3.
5 comments
Author Response (2/2)
W2. Complete computational cost
Thank you for highlighting the need for end-to-end latency and API usage measurements.
Tables R14 (T2I) and R15 (editing) add end-to-end time, including image generation or editing, together with API and token usage. The measurements use the main-experiment settings: 600 test tasks per modality (200 per benchmark), GPT-5.5 (Strong) or Doubao-Seed-2.0-pro (Weak), and Seedream-4 at 1K with adaptive aspect ratios. Each method retains its original workflow and settings; EvoSkillGen uses its final frozen banks. The original Time column measures workflow overhead, excluding image synthesis and independent evaluation.
1. End-to-end time
Time runs from method invocation to final-image delivery, including task analysis, routing, preparation, generation/editing, internal checks, selection and retries where used. We average per-request wall-clock time over all 600 tasks. Skill-bank evolution costs are reported in R12; independent test-image scoring is excluded from these measurements.
In Table R14, EvoSkillGen takes 52.3/62.3 s (Strong/Weak), second-fastest among agent methods in both settings, after GenRouter. In Table R15, it is fastest among agent methods with both agents (90.7/86.2 s), reducing time by 44.8%/27.3% relative to Mind-Brush and 67.6%/58.8% relative to RS-Gen.
2. API calls and token usage
Tables R14–R15 sum external calls and LLM usage per request, including internal checks and retries, then average over the same 600 tasks. Image-generation/editing calls are listed separately; other API calls include LLM, search and external tool requests, excluding local functions. Tokens are summed from API usage records, with reasoning tokens included in output. Total tokens are input plus output. Independent benchmark scoring is excluded.
EvoSkillGen uses fewer image calls than RS-Gen: 1.52/1.53 versus 2.68/2.67 for T2I, and 1.42/1.47 versus 4.27/3.30 for editing. Other API calls and total LLM tokens are also lower in all four settings. Total token usage decreases by 57.0%/57.4% for T2I and 86.5%/72.5% for editing (Strong/Weak).
Table R14. T2I time and API usage per request; cells Strong / Weak.
| Method | End-to-end time (s) | Other API calls | Image-generation API calls | LLM input tokens | LLM output tokens | Total LLM tokens |
|---|---|---|---|---|---|---|
| Direct | 5.5 / 5.5 | 0.00 / 0.00 | 1.00 / 1.00 | 0 / 0 | 0 / 0 | 0 / 0 |
| GenRouter | 38.3 / 46.4 | 4.87 / 4.81 | 1.00 / 1.00 | 2,105 / 2,182 | 641 / 444 | 2,746 / 2,626 |
| GEMS | 262.1 / 220.1 | 32.85 / 44.35 | 2.46 / 3.79 | 35,017 / 55,048 | 1,918 / 881 | 36,935 / 55,929 |
| Mind-Brush | 96.8 / 84.6 | 2.66 / 2.09 | 1.00 / 1.00 | 4,109 / 3,979 | 1,340 / 1,152 | 5,449 / 5,131 |
| RS-Gen | 120.5 / 123.6 | 8.31 / 7.95 | 2.68 / 2.67 | 8,849 / 10,736 | 2,307 / 2,261 | 11,156 / 12,997 |
| EvoSkillGen | 52.3 / 62.3 | 4.76 / 6.59 | 1.52 / 1.53 | 3,125 / 4,255 | 1,669 / 1,287 | 4,794 / 5,542 |
WISE, DPG-Bench and SpatialGenEval: 200 test tasks each. All columns use the same 600 tasks and report per-request means.
Table R15. Editing time and API usage per request; cells Strong / Weak.
| Method | End-to-end time (s) | Other API calls | Image-editing API calls | LLM input tokens | LLM output tokens | Total LLM tokens |
|---|---|---|---|---|---|---|
| Direct-Edit | 11.7 / 11.7 | 0.00 / 0.00 | 1.00 / 1.00 | 0 / 0 | 0 / 0 | 0 / 0 |
| Mind-Brush | 164.3 / 118.5 | 4.84 / 3.16 | 1.00 / 1.00 | 6,421 / 8,813 | 1,197 / 1,214 | 7,618 / 10,027 |
| RS-Gen | 279.8 / 209.2 | 11.69 / 9.58 | 4.27 / 3.30 | 26,184 / 23,718 | 8,127 / 3,084 | 34,311 / 26,802 |
| EvoSkillGen | 90.7 / 86.2 | 4.63 / 3.08 | 1.42 / 1.47 | 3,184 / 4,592 | 1,438 / 2,781 | 4,622 / 7,373 |
GEdit, ImgEdit and RISEBench: 200 test tasks each. All columns use the same 600 tasks and report per-request means.
Thank you for the detailed response and the additional experiments. However, two concerns remain, and I will keep my original score.
-
Limited contribution of key components. The new ablation in Table R7 shows that removing the Prompt Compiler lowers performance by only 0.8/0.5 points in the Strong T2I/Edit settings, suggesting that this component brings limited gains and is not clearly essential.
-
Limited novelty. I agree with Reviewer g7dg that the paper's core ideas have already been explored in prior work and are not novel.
Author Response (1/2)
Thank you for these suggestions. Tables R7–R8 equally weight three benchmarks and both Gemini/GPT judges, with equal Basic/UGE weights within ImgEdit. Strong/Weak denote GPT-5.5/Doubao-Seed-2.0-pro.
Prompt Compiler. With fixed instructions, the agent LLM extracts necessary additions from preparation-node outputs (retrieved information, layouts and sketches) and selects references, retaining the original request and source images. The bypass fixes the bank and node outputs, appends text artifacts to the unchanged request in a fixed order, and uses the same image interface for source images and eligible references, with unchanged generator settings. Removing Compiler lowers Strong T2I/editing by 0.8/0.5 points and Weak by 1.8/2.3: both benefit, especially Weak.
Selective acceptance. We compare Full's pairwise acceptance with two alternatives:
All three start from Direct-only banks with the same evolution tasks, order and proposal limits, retaining revision replay and task-local RefineLoop. Full leads in all four settings, by 3.8–6.1 points over removing bank-level image comparison and 0.4–1.9 over rubric scoring.
Table R7. Prompt Compiler and selective-acceptance ablations.
Full scores are from Table 3.
R8 uses Full's test tasks, generator and scoring protocol. Both agents used thinking during Full's Skill-bank evolution in the main experiments. Direct passes original inputs to Seedream-4 (Direct-Edit for editing). L1 only creates/executes a temporary Skill per request without bank updates. Full learns a bank through L2, freezes it for testing and executes applicable Skills, using Direct when none applies.
Table R8. Intermediate baselines.
Direct averages Image scores in Tables 1–2; L1 only and Full are from Table 3.
Single-call rewriting. The same agent receives the original request and editing source images, rewrites the prompt in one LLM call, then invokes Seedream-4 without Skill routing, retrieval or bank updates. Within each agent, off/on share instructions and generation settings (1K, adaptive aspect ratios), differing only in thinking:
reasoning_effort="none"; on usesreasoning_effort="medium".thinking={"type":"disabled"}; on usesthinking={"type":"enabled"}withreasoning_effort="medium".Both use
max_completion_tokens=8192for reasoning plus answer tokens; temperature is unset. Thinking improves rewriting by 0.8–1.5 points; Full remains 2.8–7.3 points ahead.Direct + RefineLoop. Check and correct Direct's image against the request for at most five rounds, without creating, retrieving or evolving Skills. Each round generates at most one candidate, kept only if it wins anonymous comparisons with the current image in both presentation orders. Editing must also preserve required source content at least as well as the current image. Correction and stopping follow Full's RefineLoop settings. This improves over Direct by 1.9–4.1 points; Full remains 2.6–5.3 points ahead.