- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces MetaEvoForge, a framework for learning transferable agent-improvement strategies through meta-evolution. It maintains a shared MetaAgent that proposes modifications to TaskAgents across different environments, and updates its own improvement policy using accumulated execution evidences.
The authors train MetaAgents on 62 task pools and evaluate their transferability across seven held out pools. Experiments with two MetaAgent backbone models demonstrate improvements over their unevolved counterparts.
Critical strengths
The authors construct 62 training pools and 7 evaluation pools, covering diverse task categories. This provides a useful experimental setting for investigating cross-task transfer rather than improvements limited to a single benchmark. They also show that meta-evolution can improve downstream agent optimization.
Critical weaknesses
-
The proposed method consists of several components, but lacks ablations to establish what drives the improvements, such as cross-pool experience sharing and MetaAgent self-evolution.
-
The proposed MetaAgent receives additional cross-task training that the external baselines do not receive. In particular, Hyperagents also supports meta-level improvement and cross-domain transfer, making the comparison less conclusive.
Questions for authors
-
Could the authors provide ablations to isolate the effects of different components in the proposed framework?
-
Could the authors compare against Hyperagents with comparable cross-task training experience to better establish the advantages of the proposed method?
2 comments
Thank you for the additional analyses, which help clarify the roles of meta evolution and source experience. However, without comparable cross task training for Hyperagents, it remains difficult to separate the method’s advantage from the benefit of additional training experience. After careful consideration, I have decided to maintain my original score.
Thank you for highlighting the value of evaluating transfer across diverse task pools and for asking which components contribute to the gains.
W1 / Q1. MetaAgent self-evolution and cross-pool experience
Self-evolution. We compared the initial MetaAgent M0 with trained checkpoints under the same held-out search protocol. Both start from the same TaskAgent and can edit it within the same search budget. Across three independent search seeds, M8 achieved the highest mean for both the GLM and DeepSeek backbones. See our response to kW3D (W2) for the full checkpoint table.
Cross-pool experience. We compared the pooled DeepSeek M8 with four single-source M8 checkpoints using the same downstream TaskAgent and 32 search attempts per pool. At seed 42, the pooled checkpoint scored 50.57% across six common targets. It exceeded AEGIS2 (47.12%), ARC-AGI-1 (44.49%), and OptiBench (44.18%). APPS was the strongest single source at 52.17%. We excluded FormalMath from this comparison because validation failures affected candidate selection in some single-pool runs. The results show that the choice of source experience affects what the MetaAgent can find on held-out pools.
Our response to kW3D (W1) gives the trace ablation, which keeps the patch and paired scores on the same tasks. Our response to eFLE (W1 / Q1) traces a learned rule from the MetaAgent’s skill file to a concrete API-Bank patch and a subsequent LawBench proposal.
W2 / Q2. Hyperagents and comparable source training
Hyperagents is a relevant comparison because it also evolves the improver. Our adapter preserves archive-based parent selection and lets both agent packages evolve. When a child is selected as a parent, its updated MetaAgent generates the next proposal.
The three-seed Hyperagents results in our response to kW3D (W2 / W3) measure adaptation from fresh packages on each target pool under the shared evaluation protocol. The source-training comparison you propose would give Hyperagents the same 62 source pools and a comparable training budget, then evaluate the resulting improver under the same held-out protocol.