- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces VisualEvolve, a framework that improves visual reasoning by updating an external skill library while keeping the VLM frozen. It uses task outcomes and execution traces to revise tool-use instructions and, when needed, executable visual operations. Each round starts from the best historical library on a fixed validation set, and the agent inspects feedback through reports, summaries, and raw traces. Across four backbones and three benchmarks, the evolved library improves over the initial library in 23 of 24 settings. Ablations support both design choices, and transfer experiments show that evolved skills remain useful across backbones.
Critical strengths
-
The improvements are consistent across backbones. The evolved library outperforms the initial library in 23 of 24 settings. This provides useful evidence that updating external skills can improve visual reasoning without changing model parameters.
-
The evolved skills transfer across models. A library developed with Kimi-K2.5 improves all three other backbones without modification. This suggests that the skills capture useful procedures beyond the model used to develop them.
Critical weaknesses
-
The added value over existing skill-evolution methods is unclear. Dynamo [1] already evolves visual skills and tools, while Meta-Harness [2] uses historical code, scores, and traces to guide updates. The paper compares mainly against an initial skill library and training-based systems, leaving its advantage over these closer approaches unresolved.
-
The experiments do not directly test the evidence-overload claim. The ablations show that PEA improves accuracy, but do not show whether it helps the agent find relevant information as more history accumulates. This supports PEA’s usefulness, while leaving its stated motivation only partly tested.
[1] Sun et al. Dynamo: Dynamic Skill-Tool Evolution for Vision-Language Agents. 2026. https://arxiv.org/abs/2606.30185
[2] Lee et al. Meta-Harness: End-to-End Optimization of Model Harnesses. 2026. https://arxiv.org/abs/2603.28052
Questions for authors
-
What evidence shows that VisualEvolve improves over existing feedback-driven skill-refinement approaches when using the same backbone, data, and comparable optimization budget?
-
What evidence shows that PEA helps the agent find relevant evidence more effectively as the evolution history grows?
4 comments
Response to Reviewer veV1 (2/3): Q1
- Q1: What evidence shows that VisualEvolve improves over existing feedback-driven skill-refinement approaches when using the same backbone, data, and comparable optimization budget?
Thank you for this question. As clarified in our response to W1, Dynamo and Meta-Harness were published after the ICLR submission deadline, and we found no open-source implementation of Dynamo. Nevertheless, to address your concerns, we additionally conducted experiments with Meta-Harness and SkillHone (another self-evolution method discussed in Sec. 2) under the same backbone, the same data, and the same optimization budget (new, Appendix B, Table 3), and with three backbones without a budget cap (new, Table 1). In all experiments, each method uses one backbone for all agents, and the three methods share the same Initial Skill Library, evolution data, and 200-example validation set. Each method is tested with the snapshot that has the highest validation accuracy in that method's own run.
Same optimization budget (Table 3, Qwen3.5-Plus, 200M tokens counting all input and output tokens). Accuracy (%).
| Method | Tokens (M) | Rounds | V*Bench | HRBench 4K | HRBench 8K | VisualProbe Easy | VisualProbe Medium | VisualProbe Hard | Macro Avg. |
|---|---|---|---|---|---|---|---|---|---|
| Meta-Harness | 200 | 7 | 89.5 | 88.4 | 87.0 | 70.2 | 48.5 | 46.2 | 71.6 |
| SkillHone | 200 | 39 | 86.4 | 87.0 | 85.5 | 68.8 | 48.1 | 42.5 | 69.7 |
| VisualEvolve | 192.73 | 16 | 92.1 | 91.0 | 90.0 | 77.3 | 60.4 | 54.7 | 77.6 |
Three backbones, without a budget cap (Table 1, where Meta-Harness and SkillHone run 20 rounds and VisualEvolve runs 16 rounds). Accuracy (%).
| Backbone | Method | V*Bench | HRBench 4K | HRBench 8K | VisualProbe Easy | VisualProbe Medium | VisualProbe Hard | Macro Avg. |
|---|---|---|---|---|---|---|---|---|
| Qwen3.5-Plus | Initial Skill Library | 86.4 | 87.0 | 83.3 | 68.1 | 48.1 | 39.6 | 68.8 |
| Qwen3.5-Plus | SkillHone | 85.9 | 88.0 | 86.9 | 67.4 | 47.0 | 41.5 | 69.5 |
| Qwen3.5-Plus | Meta-Harness | 90.1 | 89.0 | 88.4 | 72.3 | 51.9 | 49.1 | 73.5 |
| Qwen3.5-Plus | VisualEvolve | 92.1 | 91.0 | 90.0 | 77.3 | 60.4 | 54.7 | 77.6 |
| Qwen3.6-Plus | Initial Skill Library | 90.6 | 88.5 | 87.4 | 66.7 | 53.4 | 36.8 | 70.6 |
| Qwen3.6-Plus | SkillHone | 89.5 | 89.0 | 88.5 | 67.4 | 48.5 | 38.7 | 70.3 |
| Qwen3.6-Plus | Meta-Harness | 90.6 | 91.5 | 90.4 | 71.6 | 55.6 | 46.2 | 74.3 |
| Qwen3.6-Plus | VisualEvolve | 91.1 | 92.0 | 90.9 | 73.8 | 58.2 | 50.0 | 76.0 |
| Doubao-Seed-2.0-Pro | Initial Skill Library | 89.5 | 86.8 | 86.0 | 75.9 | 59.0 | 42.5 | 73.3 |
| Doubao-Seed-2.0-Pro | SkillHone | 89.0 | 87.5 | 87.0 | 77.3 | 60.4 | 45.3 | 74.4 |
| Doubao-Seed-2.0-Pro | Meta-Harness | 89.5 | 87.9 | 87.5 | 78.0 | 61.6 | 48.1 | 75.4 |
| Doubao-Seed-2.0-Pro | VisualEvolve | 90.6 | 89.4 | 89.0 | 79.4 | 63.8 | 50.9 | 77.2 |
-
Under the same backbone, data, and budget, VisualEvolve achieves the highest accuracy on all six evaluation sets (Table 3). In macro-average accuracy, VisualEvolve exceeds Meta-Harness by 6.0 points and SkillHone by 7.9 points.
-
The gains are largest on the VisualProbe sets (Table 3). VisualEvolve exceeds Meta-Harness by 11.9 points on VisualProbe Medium, and exceeds SkillHone by 12.3 points on VisualProbe Medium and by 12.2 points on VisualProbe Hard.
-
Meta-Harness and SkillHone still trail VisualEvolve without the 200M-token budget. Meta-Harness completes only 7 rounds within 200M tokens because Meta-Harness generates three candidate revisions per round (Sec. 4.1, L313–316). Without a budget cap, the 20-round run of Meta-Harness with Qwen3.5-Plus uses 594M tokens, about three times the 192.73M tokens of VisualEvolve, and Meta-Harness still trails VisualEvolve by 4.1 points. The 20-round run of SkillHone uses 115M tokens and reaches 69.5%, close to the 69.7% that SkillHone reaches under 200M tokens (Table 1, Appendix B).
-
Without a budget cap, VisualEvolve achieves the highest accuracy on all 18 combinations of three backbones and six evaluation sets (Table 1).
We have added the comparison across three backbones to Sec. 4.1–4.2 and Table 1, and the comparison under a matched token budget to Appendix B (Table 3).
Response to Reviewer veV1 (3/3): W2 & Q2
- W2 & Q2: The experiments do not directly test the evidence-overload claim. The ablations show that PEA improves accuracy, but do not show whether it helps the agent find relevant information as more history accumulates. What evidence shows that PEA helps the agent find relevant evidence more effectively as the evolution history grows?
Thank you for this question. To address your concern, we tested the evidence-overload claim more directly. We additionally analyzed what the Self-Evolving Agent ("the agent" below) reads, and how the updates of the agent turn out, as more history accumulates, with Progressive Evidence Access (PEA) and without PEA (new, Appendix C, Table 6). The results show that PEA helps the agent find relevant evidence more effectively as the evolution history grows.
This analysis uses Qwen3.5-Plus as the backbone of all agents, as in our ablation studies (Sec. 4.3). Over 16 rounds, the archived evolution evidence grows nearly 40×, from ≈71K to ≈2.83M tokens (Appendix C, Table 4). Table 6 compares rounds 1–8 with rounds 9–16. The columns "VisualEvolve" are the run with PEA. In the columns "w/o PEA", the agent has access to the same validation-accuracy history, skill-change history, and raw trajectories, but accesses these evolution records directly (Sec. 4.3, L414–416).
| Metric (Table 6, Qwen3.5-Plus) | w/o PEA, rounds 1–8 | w/o PEA, rounds 9–16 | VisualEvolve, rounds 1–8 | VisualEvolve, rounds 9–16 |
|---|---|---|---|---|
| Mean context length (tokens) | 47,512 | 48,294 | 41,084 | 42,768 |
| Raw trajectory segments accessed per round | 7.3 | 7.1 | 4.4 | 3.4 |
| Historical skill diffs inspected per round | 1.3 | 1.9 | 1.8 | 4.4 |
| Trajectory summaries accessed per round | – | – | 10.4 | 9.5 |
| Updates improving on their starting library | 3 of 8 | 1 of 8 | 3 of 8 | 3 of 8 |
| Best-so-far validation accuracy at the end (%) | 64.0 | 65.0 | 68.0 | 71.5 |
A historical skill diff is a recorded historical modification of a skill. An update improves on its starting library if the updated skill library has a higher validation accuracy than the snapshot from which the update starts. "–" denotes unavailable evidence.
-
With PEA, the agent inspects more historical skill diffs as more history accumulates. From rounds 1–8 to rounds 9–16, the number of historical skill diffs inspected per round rises from 1.8 to 4.4 with PEA, but only from 1.3 to 1.9 without PEA.
-
With PEA, the agent reads Trajectory Summaries and accesses fewer raw trajectory segments. With PEA, the agent accesses about 10 trajectory summaries per round, and accesses 4.4 raw trajectory segments per round in rounds 1–8 and 3.4 in rounds 9–16. Without PEA, the agent accesses about 7 raw trajectory segments per round in both halves. This matches the coarse-to-fine access of PEA (Sec. 3.5, L218–221).
-
PEA lowers the context burden. The mean context length is 41,084 and 42,768 tokens with PEA, compared with 47,512 and 48,294 tokens without PEA, although the agent with PEA inspects more historical skill diffs.
-
The evidence that the agent finds with PEA is relevant, as judged by the updates. With PEA, the number of improving updates does not drop as more history accumulates. We judge the relevance of the evidence by the outcome of the updates. With PEA, 3 of the 8 updates improve on their starting library in both halves. Without PEA, this number drops from 3 of 8 in rounds 1–8 to 1 of 8 in rounds 9–16.
-
The advantage of PEA grows as more history accumulates. The advantage of PEA in best-so-far validation accuracy is 4.0 points after round 8 (68.0% versus 64.0%) and 6.5 points after round 16 (71.5% versus 65.0%). In rounds 9–16, best-so-far validation accuracy rises by 3.5 points with PEA and by 1.0 point without PEA, at comparable total token consumption (192.7M tokens with PEA and 190.8M tokens without PEA, Appendix C).
In summary, without PEA, both consequences of evidence overload that we state in Sec. 1 (L077–079) appear, namely a higher context burden (point 3) and capability gaps that are harder to diagnose as more history accumulates (point 4). With PEA, the agent inspects more historical skill diffs and accesses fewer raw trajectory segments as more history accumulates (points 1 and 2), and the evidence that the agent finds leads to better updates (points 4 and 5).
We have added this analysis to Appendix C (Tables 4–6). Thank you again for your suggestions. We would be glad to discuss any further questions during the discussion period.
Dear Reviewer veV1,
Thank you very much for the time and effort you have devoted to reviewing our paper. We sincerely appreciate your helpful comments and suggestions.
We have carefully responded to each of your comments. In particular, we compared VisualEvolve with Meta-Harness and SkillHone under the same backbone, data, and optimization budget (W1 & Q1; Tables 1 and 3 of the revised PDF), and we directly tested the evidence-overload claim by analyzing what the Self-Evolving Agent reads as the evolution history grows, with and without PEA (W2 & Q2; Appendix C, Tables 4–6).
As the discussion deadline approaches, we would be very grateful if you could review our responses and let us know whether any concerns remain. We would welcome the opportunity to discuss them further and provide any additional clarification that may be helpful.
Thank you again for your time and consideration.
Best regards,
The authors
Response to Reviewer veV1 (1/3): W1
We sincerely thank you for the careful review and for recognizing the consistent gains across backbones and the cross-backbone transfer of the evolved skills. Line numbers refer to the revised PDF, where new content is marked in blue.
Thank you for this comment. We would like to clarify two points. The comparison with these closer approaches is reported in our response to Q1 (next comment).
Dynamo [1] and Meta-Harness [2] were published after the ICLR submission deadline. The two works that you cite as arXiv preprints were formally published in October 2026, Dynamo at EMNLP 2026 and Meta-Harness at COLM 2026. In addition, we found no open-source implementation of Dynamo, so we cannot reproduce Dynamo. However, to address your concerns, we additionally conducted two visual skill evolution experiments with Meta-Harness and SkillHone under the same backbone, data, and optimization budget. We report the results in our response to Q1 (next comment).
The added value of VisualEvolve lies in two mechanisms that address two important challenges of visual skill evolution. Skill updates can introduce performance regressions, and the accumulated trajectories and evolution history can cause evolution evidence overload (Sec. 1, L053–079). To address these two challenges, we propose Skill-Library Snapshot Selection and Progressive Evidence Access (PEA) (Sec. 1, contributions, L108–111). Below, we contrast each mechanism with Meta-Harness and Dynamo.
For Skill-Library Snapshot Selection, we would like to clarify three points.
Snapshot Selection does not select the final skill library at the end of evolution, which every variant in our ablation studies (Table 2) already does. Snapshot Selection selects the starting skill library of every round. At the beginning of each round, VisualEvolve selects the archived snapshot with the highest validation accuracy, instead of always continuing from the latest snapshot (Sec. 3.3, L194–196). In contrast, Meta-Harness lets its proposer decide which prior candidate to start from.
Snapshots whose validation accuracy drops are not discarded. These snapshots stay in the archive and serve as diagnostic evidence. In contrast, Dynamo keeps a candidate update only if the candidate improves accuracy on Dynamo's training cases, and discards every other candidate.
Snapshot Selection is necessary in multi-round evolution and complementary to PEA. Removing Snapshot Selection lowers macro-average accuracy from 77.6% to 72.2%. Without the Skill-Performance Report and the Evolution Log of PEA, Snapshot Selection reaches only 71.8% (Table 2, row "w/o SPR + EL"), close to the 71.2% of the variant with neither Snapshot Selection nor these two components (row "w/o Evaluate").
For PEA, we would like to clarify three points.
Meta-Harness gives its proposer access to the source code, scores, and traces of prior harness candidates (Sec. 2, L138–139), and the proposer accesses this history directly. PEA targets the drawback of accessing the accumulated evolution history directly. As the evolution records accumulate, inspecting all the records at once can cause evolution evidence overload (Sec. 1, L077–079).
To address this drawback, PEA organizes the evidence into a coarse-to-fine hierarchy. The Self-Evolving Agent first reads the Skill-Performance Report, then the Evolution Log, then the Trajectory Summaries, and accesses raw trajectory segments only when finer-grained evidence is needed (Sec. 3.5, L218–221).
PEA is necessary and complementary to Snapshot Selection. Removing PEA lowers macro-average accuracy from 77.6% to 71.3% at nearly the same token consumption, and PEA without Snapshot Selection reaches only 72.2% (Table 2, row "w/o Selection"). Our response to W2 & Q2 shows what the Self-Evolving Agent reads as the evolution history grows, with PEA and without PEA.
We have added a discussion of Dynamo to Sec. 2 (L135–137).