- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Adaptive Multi-Agent Pool Evolution (AMAPE), an online framework for dynamically generating, selecting, and reusing task-specific LLM agents. The proposed approach maintains an evolving pool of sub-agents and employs the Wilson confidence lower bound of the global task success rate to determine when a new agent should be generated. New agents are constructed using the current task description, historical successful experiences, and failure information. When reusing existing agents, a similarity-weighted Upper Confidence Bound (UCB) criterion is adopted to balance task relevance and historical performance. The framework is evaluated on RealFin-Bench and MATH-Bench using three LLM backbones, with comparisons against several baseline approaches. The reported results demonstrate improvements in task success rates and, in several settings, token efficiency, particularly on RealFin-Bench. An additional sensitivity analysis investigates the influence of the confidence threshold on performance and agent pool size.
Critical strengths
-
The paper addresses a relevant and practically meaningful problem concerning the online adaptation of LLM-based agents to heterogeneous and evolving tasks. The proposed agent pool provides a straightforward mechanism for reusing previously acquired task-specific capabilities, potentially reducing the need for repeated agent optimization.
-
The experimental results are promising, particularly on RealFin-Bench, where the proposed method achieves substantial improvements in task success rates across three backbone models. The consideration of token efficiency also provides a useful perspective on the practical effectiveness of the framework.
Critical weaknesses
-
The methodological novelty and technical contributions are not sufficiently established. The proposed framework primarily relies on established techniques, including Wilson confidence intervals, UCB-based selection, task similarity, and experience-guided skill generation. Although their integration is reasonable, the paper does not adequately explain why this particular combination is necessary or what fundamental limitations of existing self-evolving agent frameworks it overcomes. In particular, the claimed exploration-exploitation mechanism lacks sufficient theoretical or empirical justification.
-
The experimental validation is insufficient to establish the effectiveness of the proposed mechanisms. The ablation studies focus mainly on the confidence threshold, without independently examining the contributions of Wilson-based generation, similarity-weighted UCB selection, and historical success/failure information. Furthermore, the paper does not provide convincing evidence of online adaptation under changing task distributions, despite this being a central motivation. These limitations make it difficult to determine whether the reported improvements arise from the proposed framework or from specific implementation choices.
Questions for authors
-
What is the fundamental methodological contribution of the proposed Wilson-UCB mechanism beyond combining existing statistical confidence estimation and bandit-based selection? Could the authors provide a clearer explanation of why these mechanisms are particularly suitable for online agent evolution?
-
The Wilson confidence lower bound is computed from the cumulative global success rate. How can this mechanism respond effectively to changes in task distributions, particularly when substantial historical observations may dominate the statistics? Could the authors provide experiments under explicit distribution shifts?
-
In Eq. (11), the selection score combines the maximum similarity to historically successful tasks with an agent's UCB score. What is the rationale for this multiplicative formulation? How does the method handle agents with no successful execution history, and how sensitive is the selection performance to the similarity function?
-
Could the authors provide component-wise ablation studies to isolate the effects of Wilson-based generation, UCB selection, task similarity, successful experience retrieval, and failure information? Comparisons against simpler alternatives, such as similarity-only selection or fixed-frequency agent generation, would be particularly informative.
-
How are the reported results affected by task ordering, random seeds, and the choice of threshold? Since the framework operates sequentially, repeated experiments with different task orders and statistical variation would be important for establishing robustness.
-
How is the computational cost of generating new agents accounted for? The reported efficiency primarily considers token consumption, but additional generation calls, retrieval operations, and execution latency may influence the overall efficiency of the framework.
4 comments
Response to Reviewer 1G4u
3 About agent selection
- Agents without a history of successful executions. For an agent with no successful executions, n > 0 and its UCB is finite; however, because it has no history of successful executions, it will not be included in the candidate set for selection. This is consistent with the semantics of "no successful cases available for reuse."
- Sensitivity of the similarity function. We currently adopt Jaccard similarity because it is lightweight, interpretable, and requires no model calls, making it suitable for online scenarios. To address the sensitivity of the similarity function, we supplemented ablation experiments with two alternative similarity measures, edit distance and LCS, as shown in the table below. The results indicate that our algorithm is sensitive to the choice of similarity computation, and that the currently adopted Jaccard achieves the best performance on this task.
Table 1: Similarity function ablation study on RealFin-Bench.
| Model | Similarity function | SR (%) | InTok | OutTok | TotalTok | EFF |
|---|---|---|---|---|---|---|
| GLM-5.2 | Edit Distance | 70 | 1.47M | 139k | 1.61M | 17.40 |
| LCS | 78 | 1.40M | 134k | 1.53M | 20.23 | |
| Jaccard (ours) | 85 | 1.55M | 152k | 1.70M | 20.01 |
4 Ablation study
In response to yours concerns, we provide three categories of ablation studies: generation guidelines, generation components, and agent selection.
- Generation guidelines: We compare three settings—acc point estimation, acc sliding-window estimation, and the Wilson lower confidence bound of acc. The sliding-window length is set to 5, and the threshold is set to 0.4 for all three. The experimental results using GLM-5.2 on the RealFin dataset are shown below. The results verify the superiority of the Wilson lower confidence bound over point estimation and sliding-window estimation. In summary, point estimation ignores sampling uncertainty, and the sliding window only addresses timeliness; the Wilson lower bound directly models the reliability of estimates under finite samples, making skill-evolution decisions statistically robust to sampling randomness and sample size.
Table 2: Generation Guidelines ablation study on RealFin-Bench.
| Model | Method | SR (%) | InTok | OutTok | TotalTok | EFF |
|---|---|---|---|---|---|---|
| GLM-5.2 | acc | 72 | 1.51M | 154k | 1.66M | 17.42 |
| sliding-window estimate | 72 | 1.40M | 151k | 1.55M | 18.73 | |
| Wilson lower bound (ours) | 85 | 1.55M | 152k | 1.70M | 20.01 |
- Generation components: We conduct ablation experiments on generation components, comparing the use of the current task only, the current task + failure experience, the current task + successful examples, and our full proposed algorithm; the experiments cover two tasks, RealFin and MATH, on GLM-5.2 and Qwen3.8-27B, as shown in the table below. As can be seen: on the RealFin task, adding only successful or failure experience affects the generated skills and thus reduces SR to some extent; only by combining the current task, successful examples, and failure experience can skill generation be better guided, thereby improving task SR. On the MATH task, the effects of failure and successful experience differ across models: successful experience better helps GLM-5.2 generate task-adapted skills, whereas for Qwen3.8-27B the inclusion of successful experience leads to a decrease in SR. Overall, however, only when all components are included can task SR be improved.
Table 3: Component ablation study on RealFin-Bench.
| Model | Component | SR (%) | InTok | OutTok | TotalTok | EFF |
|---|---|---|---|---|---|---|
| GLM-5.2 | q_i | 80 | 1.42M | 156k | 1.57M | 20.33 |
| (q_i, F_fail) | 68 | 1.48M | 161k | 1.64M | 16.48 | |
| (q_i, R(q_i)) | 72 | 1.58M | 134k | 1.71M | 16.93 | |
| (q_i, R(q_i), F_fail) (ours) | 85 | 1.55M | 152k | 1.70M | 20.01 | |
| Qwen3.8-27B | q_i | 65 | 1.65M | 120k | 1.77M | 14.70 |
| (q_i, F_fail) | 60 | 1.72M | 140k | 1.86M | 12.90 | |
| (q_i, R(q_i)) | 60 | 1.67M | 106k | 1.78M | 13.48 | |
| (q_i, R(q_i), F_fail) (ours) | 68 | 1.59M | 104k | 1.69M | 15.94 |
Table 4: Component ablation study on MATH-Bench.
| Model | Method | SR (%) | InTok | OutTok | TotalTok | EFF |
|---|---|---|---|---|---|---|
| GLM-5.2 | q_i | 88 | 640k | 68k | 708k | 124.24 |
| (q_i, F_fail) | 86 | 675k | 70k | 745k | 115.48 | |
| (q_i, R(q_i)) | 91 | 637k | 80k | 717k | 126.86 | |
| (q_i, R(q_i), F_fail) (ours) | 91 | 624k | 73k | 697k | 130.5 | |
| Qwen3.8-27B | q_i | 78 | 958k | 100k | 1.06M | 73.70 |
| (q_i, F_fail) | 78 | 959k | 100k | 1.06M | 73.66 | |
| (q_i, R(q_i)) | 75 | 1.08M | 126k | 1.21M | 62.06 | |
| (q_i, R(q_i), F_fail) (ours) | 82 | 947k | 113k | 1.06M | 77.33 |
Response to Reviewer 1G4u
4 Ablation study
- Agent selection: In addition, we supplement ablation experiments on agent selection, comparing three selection strategies: similarity only, UCB only, and the product of similarity and UCB. The experiments cover two tasks, RealFin and MATH, on GLM-5.2 and Qwen3.8-27B, as shown in the table below. As can be seen: on both the RealFin and MATH datasets, measured by SR and EFF, the product of similarity and UCB performs best among the three strategies, followed by similarity only, and UCB only last.
Table 5: Agent selection ablation study on RealFin-Bench.
| Model | Selection method | SR (%) | InTok | OutTok | TotalTok | EFF |
|---|---|---|---|---|---|---|
| GLM-5.2 | sim | 80 | 1.51M | 175k | 1.69M | 18.95 |
| ucb | 70 | 1.44M | 154k | 1.60M | 17.55 | |
| sim × ucb (ours) | 85 | 1.55M | 152k | 1.70M | 20.01 | |
| Qwen3.8-27B | sim | 68 | 1.72M | 152k | 1.87M | 14.41 |
| ucb | 48 | 1.72M | 123k | 1.84M | 10.31 | |
| sim × ucb (ours) | 68 | 1.59M | 104k | 1.69M | 15.94 |
Table 6: Agent selection ablation study on MATH-Bench.
| Model | Method | SR (%) | InTok | OutTok | TotalTok | EFF |
|---|---|---|---|---|---|---|
| GLM-5.2 | sim | 80 | 1.51M | 175k | 1.69M | 18.95 |
| ucb | 88 | 732k | 84k | 816k | 107.90 | |
| sim × ucb (ours) | 91 | 624k | 73k | 697k | 130.5 | |
| Qwen3.8-27B | sim | 75 | 1.08M | 111k | 1.19M | 62.97 |
| ucb | 69 | 1.01M | 118k | 1.13M | 60.89 | |
| sim × ucb (ours) | 82 | 947k | 113k | 1.06M | 77.33 |
5 About task order,random seeds and threshold selection
We thank the reviewer for this suggestion. We clarify three points.
- Inter-task order. Tasks in both RealFin and MATH are mutually independent and start from an initial state; there is no state propagation or ordering dependency across tasks. Hence "task order" is not a variable affecting results in this setting, which is why we did not construct repeated experiments with different inter-task orders.
- Intra-task order and random seeds. The factor that could introduce ordering/sampling variability is the execution order and sampling randomness within a dataset. For RealFin, we follow the intra-task order used in Bayesian-Agent without modification, to ensure comparability with prior work; for MATH, we use random seed 42 to determine the intra-task order. To further test statistical robustness, we additionally conducted experiments with the random seed set to 3, and the results are shown in the table below.
Table7: Random seed ablation study on MATH-Bench.
| Model | Random seed | SR (%) | InTok | OutTok | TotalTok | EFF |
|---|---|---|---|---|---|---|
| GLM-5.2 | 3 | 91 | 605k | 73k | 678k | 134.25 |
| 42 (ours) | 91 | 624k | 73k | 697k | 130.5 |
The experimental results are largely consistent with those obtained with the random seed set to 42, indicating that our conclusions do not depend on a particular random seed.
- Threshold selection. The threshold is not tuned on the test set. It is determined using the Wilson confidence lower bound of SR (i.e., acc) as a conservative reference: the threshold is placed below the point estimate of the best baseline's SR and follows the conservative direction indicated by its Wilson lower bound, so that the success rate corresponding to the threshold carries statistical credibility; the specific value is then determined in conjunction with sensitivity experiments on AMAPE (e.g., on RealFin, where GA+'s SR is about 0.65, AMAPE is optimal with a threshold of 0.3–0.4; on MATH, where SR is about 0.8–0.9, a threshold of 0.5–0.6 is optimal). The threshold was never tuned on the test set.
Response to Reviewer 1G4u
6 Computational cost and reported efficiency
- The generation of new agents and task execution both invoke large models, and their input, output, and total tokens are fully counted and entirely included in the efficiency metric. The retrieval side is implemented based on Jaccard similarity and is essentially a lightweight, interpretable similarity-case retriever with graceful degradation; it does not consume any LLM tokens and is therefore not included in the efficiency metric.
- To address the execution-latency concern, we introduced an additional metric—the average execution time per successful task, defined as the total task execution time divided by the number of successfully executed tasks—and conducted supplementary experiments on the RealFin tasks, with results shown below. This metric already covers the latency of generating new agents and the retrieval time. We note that this metric measures the cost per successful task rather than the total investment; since the numerator includes the time of failed tasks while the denominator counts only successful ones, it varies with the success rate. On GLM5.2, our method achieves an average execution time of 210.4 s per successful task, higher than the shortest 157.8 s; however, the latter corresponds to an SR of only 60 and an EFF of only 13.41, whereas our method reaches an SR of 85 and an EFF of 20.01. That is, our method spends more time per successful task while achieving substantially higher success rate and efficiency—approximately 1.33× the per-task time in exchange for a 25-percentage-point gain in SR and about a 49% gain in EFF. On DeepSeek V4 Flash and Qwen3.8-27B, our method not only has the shortest execution time but also the highest EFF. Overall, even when successful tasks involve the generation of new agents, on DeepSeek V4 Flash and Qwen3.8-27B the average execution time per task still exhibits a downward trend compared with other algorithms, whereas on GLM5.2 our method trades a higher per-task cost for substantially higher SR and EFF.
Table 8: Performance comparison on RealFin-Bench.
| Model | Method | SR (%) | Total (s) | Per successful task (s) | TotalTok | EFF |
|---|---|---|---|---|---|---|
| GLM-5.2 | Vanilla | 30 | 2738.3 | 228.2 | 1.79M | 6.71 |
| CoT | 22 | 1785.83 | 198.4 | 1.64M | 5.48 | |
| GA | 52 | 6702.53 | 319.2 | 1.71M | 12.27 | |
| BA | 60 | 3786.79 | 157.8 | 1.79M | 13.41 | |
| GA+ | 62 | 7828.65 | 313.1 | 2.58M | 9.69 | |
| AMAPE (ours) | 85 | 7155.02 | 210.4 | 1.70M | 20.01 | |
| DeepSeek-V4-Flash | Vanilla | 5 | 762.01 | 762.01 | 1.43M | 1.40 |
| CoT | 8 | 1097.07 | 548.5 | 1.44M | 2.09 | |
| GA | 5 | 1236.4 | 618.2 | 1.43M | 1.40 | |
| BA | 12 | 1028.26 | 205.7 | 1.56M | 3.21 | |
| GA+ | 18 | 2796.25 | 399.5 | 1.48M | 3.38 | |
| AMAPE (ours) | 65 | 2543.59 | 106.0 | 1.83M | 14.19 | |
| Qwen3.8-27B | Vanilla | 28 | 1187.29 | 108.0 | 1.54M | 7.15 |
| CoT | 8 | 3309.63 | 1103.2 | 1.78M | 1.69 | |
| GA | 45 | 999.76 | 55.5 | 1.59M | 11.29 | |
| BA | 52 | 1070.18 | 51.0 | 1.69M | 12.41 | |
| GA+ | 65 | 1467.25 | 56.4 | 2.57M | 10.12 | |
| AMAPE (ours) | 68 | 902.85 | 33.4 | 1.69M | 15.94 |
Response to Reviewer 1G4u
Dear Reviewer 1G4u, We sincerely thank you for the detailed and constructive review, and we appreciate your recognition of the practically meaningful problem studied in this paper, the straightforward agent-pool reuse mechanism, the experimental results on RealFin-Bench showing substantial improvements in success rates, and the practical-effectiveness perspective offered by our consideration of token efficiency. Below, we address your main questions and describe the additional analyses and revisions we will incorporate.
1 Fundamental methodological contribution
The methodological contribution of Wilson-UCB is not the standalone use of a Wilson bound or UCB, but a a decoupled decision framework of "globally conservative triggering and locally optimistic selection" for online agent evolution. The Wilson lower bound serves as admission control for generation: it provides a conservative success-rate bound under finite samples and triggers generation only when the pool's overall capability is genuinely insufficient, suppressing spurious generation from early fluctuations—crucial when generation is expensive and the pool is append-only. UCB serves skill selection: it performs optimistic selection over the same success/failure statistics, exploiting verified skills while using an exploration term to avoid starving new ones. This decoupling is why the mechanism suits online evolution: generation is low-frequency, expensive, and cautious, whereas selection is high-frequency, reversible, and timely, and the two have opposite needs for conservatism versus optimism. Both share the same online evidence and require no offline validation set, so that both triggering and selection are driven by statistical signals rather than manual scheduling.
2 Wilson Lower Bound under Distribution Shifts
We thank the reviewer for this question. We clarify that the cumulative Wilson lower bound is indeed computed from the global cumulative success rate, and its response to distribution change is therefore lagged and smooth—the more historical observations, the smaller the influence of any single new task. This is a deliberate conservative trade-off: in an online setting where generation is expensive and the pool is append-only, we prefer delayed response over spurious generation caused by short-term fluctuations.
Importantly, the first line of response to distribution change is not the global Wilson bound but the UCB selection layer. UCB performs local, immediate selection based on each skill's own history and task similarity, without relying on global cumulative statistics; when the task distribution shifts, this first manifests in which skill is selected, and the resulting outcomes then propagate gradually to the global statistics, triggering generation only when capability is genuinely insufficient. The global bound thus serves as conservative admission control, not as the shift detector itself.
Regarding an explicit distribution-shift experiment: in our online setting, shift manifests as dynamic variation in task difficulty within the stream rather than an explicit cross-domain switch (RealFin itself contains tasks of varying difficulty, and for MATH we selected problems of varying difficulty). We therefore did not construct an artificial explicit-shift experiment. We acknowledge this as a boundary of the current evaluation and will clarify the lagged nature of the cumulative statistic and its division of labor with UCB-based local response in the revised manuscript.
3 About agent selection