- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes EvoGap, a framework for improving frozen LLM agents through weakness-guided skill evolution. It maintains a graph of diagnosed behavioral weaknesses, selects targeted practice through breadth- or depth-oriented task generation, and updates an external skill library using environment feedback and validated outcomes. Experiments across five benchmark domains and two frozen backbones demonstrate consistent improvements over existing memory- and skill-based methods.
Critical strengths
-
The paper addresses a practical problem of improving frozen LLM agents by explicitly identifying and targeting behavioral weaknesses rather than relying on passive experience accumulation.
-
The framework integrates weakness-guided practice generation, environment-grounded skill distillation, and validation-based skill updates into a coherent closed-loop process.
-
Experiments across multiple benchmarks and model backbones demonstrate promising performance gains, supported by ablations investigating several key design components.
Critical weaknesses
- Sections 2 and 4.1–4.2 assume access to machine-checkable weakness triggers, error signatures, correct action sequences, and executable verifiers, but do not explain how these are obtained or what supervision they require.
- Table 3 shows only a two-task improvement over simpler informed selection policies on 40 Telecom tasks, without statistical significance analysis. Moreover, the prerequisite edges introduced in Section 4.1 are never explicitly evaluated or ablated.
- Equation (2) imposes , yet Section 4.3 treats unrelated canary tasks as warnings rather than hard admission constraints. No HARM results are reported, and improvements in aggregate success rates cannot rule out regressions on previously solved tasks.
Questions for authors
see wekaness.