- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents OmniHarness, an agent framework that improves visual generation by accumulating experience in an external library. Before downstream tasks arrive, the system autonomously generates and selects practice tasks, abstracts verified execution workflows into reusable symbolic policies, and stores failure experience and corrective strategies. The authors evaluate the framework through multiple visual generation benchmarks, component ablations, and experience-library transfer experiments.
Critical strengths
- The experiments cover a broad range of benchmarks and include extensive ablation studies, providing good support for the system's effectiveness.
- The paper is clearly written and easy to follow.
Critical weaknesses
- Some claims about the contributions are overstated. The paper primarily integrates existing mechanisms into a system and validates it through extensive experiments, making the contribution incremental overall. Symbolic workflow abstraction, experience accumulation, and feedback-driven recovery have already been explored in prior work.
- The paper lacks a complete cross-task example showing how experience is distilled and subsequently applied to another task.
- The performance gains across task categories on ComfyBench are uneven. Complex tasks mainly involve composing Vanilla workflows and would theoretically be expected to be easier to learn. However, why does experience accumulation substantially improve Creative tasks without demonstrating an advantage on Complex tasks under the same underlying model?
- Although practice for GenEval, GenEval2, and WISE does not access their evaluation tasks, the paper does not sufficiently clarify whether the experience library continues to be updated during evaluation or whether it is reset between benchmarks.
Questions for authors
See Weaknesses
We Would Be Grateful to Hear from You Before the Discussion Closes
Dear Reviewer R7xY,
As the discussion deadline approaches, we sincerely hope to hear from you while there is still time to address any remaining concerns. Your feedback would mean a great deal to us, and we would be grateful to know whether our response has answered your questions. If anything remains unclear, we are ready to provide further clarification or additional information promptly.
We deeply appreciate the time and care you have devoted to reviewing our work and helping us improve it.