acceptodds
Under review as a conference paper at ICLR 2027

Adaptive Multi-Agent Pool Evolution: An Online Selection and Generation Framework

Abstract

Existing single-agent systems typically rely on fixed policies or Bayesian posterior updates to handle diverse tasks during execution. However, fixed policies lack flexibility, while Bayesian methods incur high computational overhead and converge slowly under frequent task switching. To overcome these limitations, this paper proposes AMAPE (Adaptive Multi-Agent Pool Evolution)—an online selection and generation framework that dynamically maintains an evolvable pool of sub-agents. For each incoming task, the system generates sub-agents tailored to its characteristics and caches them in the pool for direct reuse by similar tasks in the future. The framework employs the Wilson confidence interval to monitor the global execution success rate online: when the confidence lower bound falls below a preset threshold, the system triggers the generation of a new agent by synthesizing the current task description, prior experience from similar successful tasks, and failure avoidance strategies distilled from historical failures. For agent selection, we design a composite metric that integrates task similarity with each agent's Upper Confidence Bound (UCB) score, effectively balancing the exploration of newly generated agents and the exploitation of proven high-performing ones. Experimental results demonstrate that AMAPE dynamically expands and updates the agent pool in response to evolving task distributions, improving task success rates without introducing additional execution overhead.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multi-Agent Systems,Adaptive Agent Pool,Online Selection,Agent Evolution

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_1G4uOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Adaptive Multi-Agent Pool Evolution (AMAPE), an online framework for dynamically generating, selecting, and reusing task-specific LLM agents. The proposed approach maintains an evolving pool of sub-agents and employs the Wilson confidence lower bound of the global task success rate to determine when a new agent should be generated. New agents are constructed using the current task description, historical successful experiences, and failure information. When reusing existing agents, a similarity-weighted Upper Confidence Bound (UCB) criterion is adopted to balance task relevance and historical performance. The framework is evaluated on RealFin-Bench and MATH-Bench using three LLM backbones, with comparisons against several baseline approaches. The reported results demonstrate improvements in task success rates and, in several settings, token efficiency, particularly on RealFin-Bench. An additional sensitivity analysis investigates the influence of the confidence threshold on performance and agent pool size.

Critical strengths
  1. The paper addresses a relevant and practically meaningful problem concerning the online adaptation of LLM-based agents to heterogeneous and evolving tasks. The proposed agent pool provides a straightforward mechanism for reusing previously acquired task-specific capabilities, potentially reducing the need for repeated agent optimization.

  2. The experimental results are promising, particularly on RealFin-Bench, where the proposed method achieves substantial improvements in task success rates across three backbone models. The consideration of token efficiency also provides a useful perspective on the practical effectiveness of the framework.

Critical weaknesses
  1. The methodological novelty and technical contributions are not sufficiently established. The proposed framework primarily relies on established techniques, including Wilson confidence intervals, UCB-based selection, task similarity, and experience-guided skill generation. Although their integration is reasonable, the paper does not adequately explain why this particular combination is necessary or what fundamental limitations of existing self-evolving agent frameworks it overcomes. In particular, the claimed exploration-exploitation mechanism lacks sufficient theoretical or empirical justification.

  2. The experimental validation is insufficient to establish the effectiveness of the proposed mechanisms. The ablation studies focus mainly on the confidence threshold, without independently examining the contributions of Wilson-based generation, similarity-weighted UCB selection, and historical success/failure information. Furthermore, the paper does not provide convincing evidence of online adaptation under changing task distributions, despite this being a central motivation. These limitations make it difficult to determine whether the reported improvements arise from the proposed framework or from specific implementation choices.

Questions for authors
  1. What is the fundamental methodological contribution of the proposed Wilson-UCB mechanism beyond combining existing statistical confidence estimation and bandit-based selection? Could the authors provide a clearer explanation of why these mechanisms are particularly suitable for online agent evolution?

  2. The Wilson confidence lower bound is computed from the cumulative global success rate. How can this mechanism respond effectively to changes in task distributions, particularly when substantial historical observations may dominate the statistics? Could the authors provide experiments under explicit distribution shifts?

  3. In Eq. (11), the selection score combines the maximum similarity to historically successful tasks with an agent's UCB score. What is the rationale for this multiplicative formulation? How does the method handle agents with no successful execution history, and how sensitive is the selection performance to the similarity function?

  4. Could the authors provide component-wise ablation studies to isolate the effects of Wilson-based generation, UCB selection, task similarity, successful experience retrieval, and failure information? Comparisons against simpler alternatives, such as similarity-only selection or fixed-frequency agent generation, would be particularly informative.

  5. How are the reported results affected by task ordering, random seeds, and the choice of threshold? Since the framework operates sequentially, repeated experiments with different task orders and statistical variation would be important for establishing robustness.

  6. How is the computational cost of generating new agents accounted for? The reported efficiency primarily considers token consumption, but additional generation calls, retrieval operations, and execution latency may influence the overall efficiency of the framework.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 1G4u

Dear Reviewer 1G4u, We sincerely thank you for the detailed and constructive review, and we appreciate your recognition of the practically meaningful problem studied in this paper, the straightforward agent-pool reuse mechanism, the experimental results on RealFin-Bench showing substantial improvements in success rates, and the practical-effectiveness perspective offered by our consideration of token efficiency. Below, we address your main questions and describe the additional analyses and revisions we will incorporate.

1 Fundamental methodological contribution

The methodological contribution of Wilson-UCB is not the standalone use of a Wilson bound or UCB, but a a decoupled decision framework of "globally conservative triggering and locally optimistic selection" for online agent evolution. The Wilson lower bound serves as admission control for generation: it provides a conservative success-rate bound under finite samples and triggers generation only when the pool's overall capability is genuinely insufficient, suppressing spurious generation from early fluctuations—crucial when generation is expensive and the pool is append-only. UCB serves skill selection: it performs optimistic selection over the same success/failure statistics, exploiting verified skills while using an exploration term to avoid starving new ones. This decoupling is why the mechanism suits online evolution: generation is low-frequency, expensive, and cautious, whereas selection is high-frequency, reversible, and timely, and the two have opposite needs for conservatism versus optimism. Both share the same online evidence and require no offline validation set, so that both triggering and selection are driven by statistical signals rather than manual scheduling.

2 Wilson Lower Bound under Distribution Shifts

We thank the reviewer for this question. We clarify that the cumulative Wilson lower bound is indeed computed from the global cumulative success rate, and its response to distribution change is therefore lagged and smooth—the more historical observations, the smaller the influence of any single new task. This is a deliberate conservative trade-off: in an online setting where generation is expensive and the pool is append-only, we prefer delayed response over spurious generation caused by short-term fluctuations.

Importantly, the first line of response to distribution change is not the global Wilson bound but the UCB selection layer. UCB performs local, immediate selection based on each skill's own history and task similarity, without relying on global cumulative statistics; when the task distribution shifts, this first manifests in which skill is selected, and the resulting outcomes then propagate gradually to the global statistics, triggering generation only when capability is genuinely insufficient. The global bound thus serves as conservative admission control, not as the shift detector itself.

Regarding an explicit distribution-shift experiment: in our online setting, shift manifests as dynamic variation in task difficulty within the stream rather than an explicit cross-domain switch (RealFin itself contains tasks of varying difficulty, and for MATH we selected problems of varying difficulty). We therefore did not construct an artificial explicit-shift experiment. We acknowledge this as a boundary of the current evaluation and will clarify the lagged nature of the cumulative statistic and its division of labor with UCB-based local response in the revised manuscript.

3 About agent selection

  • Rationale for the multiplicative form. In Eq. (11), sim × ucb treats similarity as a gating and scaling factor for the UCB rather than as a term to be additively weighted with the UCB. There are three reasons. First, zero similarity means a zero score: if an agent has no successfully similar cases to the current task, it will not be preferentially selected regardless of how high its UCB is, ensuring that reuse occurs primarily among agents that have been empirically validated on "similar tasks." Second, similarity amplifies differences when UCB values are close, giving agents with higher similarity a stronger tendency toward reuse, while the UCB still governs the exploration–exploitation balance when similarities are comparable. Third, the multiplicative form requires no additional weighting coefficient, avoiding the tuning of the weight α in the additive form, and the semantics of "zero similarity constitutes a veto" cannot be expressed by an additive form. The cost is that when similarity is very small, the exploration term of the UCB is suppressed, which may inhibit exploration of agents that are "dissimilar but potentially promising"—a deliberate conservative trade-off on our part.
AuthorsOpenReview ↗

Response to Reviewer 1G4u

3 About agent selection

  • Agents without a history of successful executions. For an agent with no successful executions, n > 0 and its UCB is finite; however, because it has no history of successful executions, it will not be included in the candidate set for selection. This is consistent with the semantics of "no successful cases available for reuse."
  • Sensitivity of the similarity function. We currently adopt Jaccard similarity because it is lightweight, interpretable, and requires no model calls, making it suitable for online scenarios. To address the sensitivity of the similarity function, we supplemented ablation experiments with two alternative similarity measures, edit distance and LCS, as shown in the table below. The results indicate that our algorithm is sensitive to the choice of similarity computation, and that the currently adopted Jaccard achieves the best performance on this task.

Table 1: Similarity function ablation study on RealFin-Bench.

ModelSimilarity functionSR (%)InTokOutTokTotalTokEFF
GLM-5.2Edit Distance701.47M139k1.61M17.40
LCS781.40M134k1.53M20.23
Jaccard (ours)851.55M152k1.70M20.01

4 Ablation study

In response to yours concerns, we provide three categories of ablation studies: generation guidelines, generation components, and agent selection.

  • Generation guidelines: We compare three settings—acc point estimation, acc sliding-window estimation, and the Wilson lower confidence bound of acc. The sliding-window length is set to 5, and the threshold is set to 0.4 for all three. The experimental results using GLM-5.2 on the RealFin dataset are shown below. The results verify the superiority of the Wilson lower confidence bound over point estimation and sliding-window estimation. In summary, point estimation ignores sampling uncertainty, and the sliding window only addresses timeliness; the Wilson lower bound directly models the reliability of estimates under finite samples, making skill-evolution decisions statistically robust to sampling randomness and sample size.

Table 2: Generation Guidelines ablation study on RealFin-Bench.

ModelMethodSR (%)InTokOutTokTotalTokEFF
GLM-5.2acc721.51M154k1.66M17.42
sliding-window estimate721.40M151k1.55M18.73
Wilson lower bound (ours)851.55M152k1.70M20.01
  • Generation components: We conduct ablation experiments on generation components, comparing the use of the current task only, the current task + failure experience, the current task + successful examples, and our full proposed algorithm; the experiments cover two tasks, RealFin and MATH, on GLM-5.2 and Qwen3.8-27B, as shown in the table below. As can be seen: on the RealFin task, adding only successful or failure experience affects the generated skills and thus reduces SR to some extent; only by combining the current task, successful examples, and failure experience can skill generation be better guided, thereby improving task SR. On the MATH task, the effects of failure and successful experience differ across models: successful experience better helps GLM-5.2 generate task-adapted skills, whereas for Qwen3.8-27B the inclusion of successful experience leads to a decrease in SR. Overall, however, only when all components are included can task SR be improved.

Table 3: Component ablation study on RealFin-Bench.

ModelComponentSR (%)InTokOutTokTotalTokEFF
GLM-5.2q_i801.42M156k1.57M20.33
(q_i, F_fail)681.48M161k1.64M16.48
(q_i, R(q_i))721.58M134k1.71M16.93
(q_i, R(q_i), F_fail) (ours)851.55M152k1.70M20.01
Qwen3.8-27Bq_i651.65M120k1.77M14.70
(q_i, F_fail)601.72M140k1.86M12.90
(q_i, R(q_i))601.67M106k1.78M13.48
(q_i, R(q_i), F_fail) (ours)681.59M104k1.69M15.94

Table 4: Component ablation study on MATH-Bench.

ModelMethodSR (%)InTokOutTokTotalTokEFF
GLM-5.2q_i88640k68k708k124.24
(q_i, F_fail)86675k70k745k115.48
(q_i, R(q_i))91637k80k717k126.86
(q_i, R(q_i), F_fail) (ours)91624k73k697k130.5
Qwen3.8-27Bq_i78958k100k1.06M73.70
(q_i, F_fail)78959k100k1.06M73.66
(q_i, R(q_i))751.08M126k1.21M62.06
(q_i, R(q_i), F_fail) (ours)82947k113k1.06M77.33
AuthorsOpenReview ↗

Response to Reviewer 1G4u

4 Ablation study

  • Agent selection: In addition, we supplement ablation experiments on agent selection, comparing three selection strategies: similarity only, UCB only, and the product of similarity and UCB. The experiments cover two tasks, RealFin and MATH, on GLM-5.2 and Qwen3.8-27B, as shown in the table below. As can be seen: on both the RealFin and MATH datasets, measured by SR and EFF, the product of similarity and UCB performs best among the three strategies, followed by similarity only, and UCB only last.

Table 5: Agent selection ablation study on RealFin-Bench.

ModelSelection methodSR (%)InTokOutTokTotalTokEFF
GLM-5.2sim801.51M175k1.69M18.95
ucb701.44M154k1.60M17.55
sim × ucb (ours)851.55M152k1.70M20.01
Qwen3.8-27Bsim681.72M152k1.87M14.41
ucb481.72M123k1.84M10.31
sim × ucb (ours)681.59M104k1.69M15.94

Table 6: Agent selection ablation study on MATH-Bench.

ModelMethodSR (%)InTokOutTokTotalTokEFF
GLM-5.2sim801.51M175k1.69M18.95
ucb88732k84k816k107.90
sim × ucb (ours)91624k73k697k130.5
Qwen3.8-27Bsim751.08M111k1.19M62.97
ucb691.01M118k1.13M60.89
sim × ucb (ours)82947k113k1.06M77.33

5 About task order,random seeds and threshold selection

We thank the reviewer for this suggestion. We clarify three points.

  • Inter-task order. Tasks in both RealFin and MATH are mutually independent and start from an initial state; there is no state propagation or ordering dependency across tasks. Hence "task order" is not a variable affecting results in this setting, which is why we did not construct repeated experiments with different inter-task orders.
  • Intra-task order and random seeds. The factor that could introduce ordering/sampling variability is the execution order and sampling randomness within a dataset. For RealFin, we follow the intra-task order used in Bayesian-Agent without modification, to ensure comparability with prior work; for MATH, we use random seed 42 to determine the intra-task order. To further test statistical robustness, we additionally conducted experiments with the random seed set to 3, and the results are shown in the table below.

Table7: Random seed ablation study on MATH-Bench.

ModelRandom seedSR (%)InTokOutTokTotalTokEFF
GLM-5.2391605k73k678k134.25
42 (ours)91624k73k697k130.5

The experimental results are largely consistent with those obtained with the random seed set to 42, indicating that our conclusions do not depend on a particular random seed.

  • Threshold selection. The threshold is not tuned on the test set. It is determined using the Wilson confidence lower bound of SR (i.e., acc) as a conservative reference: the threshold is placed below the point estimate of the best baseline's SR and follows the conservative direction indicated by its Wilson lower bound, so that the success rate corresponding to the threshold carries statistical credibility; the specific value is then determined in conjunction with sensitivity experiments on AMAPE (e.g., on RealFin, where GA+'s SR is about 0.65, AMAPE is optimal with a threshold of 0.3–0.4; on MATH, where SR is about 0.8–0.9, a threshold of 0.5–0.6 is optimal). The threshold was never tuned on the test set.
AuthorsOpenReview ↗

Response to Reviewer 1G4u

6 Computational cost and reported efficiency

  • The generation of new agents and task execution both invoke large models, and their input, output, and total tokens are fully counted and entirely included in the efficiency metric. The retrieval side is implemented based on Jaccard similarity and is essentially a lightweight, interpretable similarity-case retriever with graceful degradation; it does not consume any LLM tokens and is therefore not included in the efficiency metric.
  • To address the execution-latency concern, we introduced an additional metric—the average execution time per successful task, defined as the total task execution time divided by the number of successfully executed tasks—and conducted supplementary experiments on the RealFin tasks, with results shown below. This metric already covers the latency of generating new agents and the retrieval time. We note that this metric measures the cost per successful task rather than the total investment; since the numerator includes the time of failed tasks while the denominator counts only successful ones, it varies with the success rate. On GLM5.2, our method achieves an average execution time of 210.4 s per successful task, higher than the shortest 157.8 s; however, the latter corresponds to an SR of only 60 and an EFF of only 13.41, whereas our method reaches an SR of 85 and an EFF of 20.01. That is, our method spends more time per successful task while achieving substantially higher success rate and efficiency—approximately 1.33× the per-task time in exchange for a 25-percentage-point gain in SR and about a 49% gain in EFF. On DeepSeek V4 Flash and Qwen3.8-27B, our method not only has the shortest execution time but also the highest EFF. Overall, even when successful tasks involve the generation of new agents, on DeepSeek V4 Flash and Qwen3.8-27B the average execution time per task still exhibits a downward trend compared with other algorithms, whereas on GLM5.2 our method trades a higher per-task cost for substantially higher SR and EFF.

Table 8: Performance comparison on RealFin-Bench.

ModelMethodSR (%)Total (s)Per successful task (s)TotalTokEFF
GLM-5.2Vanilla302738.3228.21.79M6.71
CoT221785.83198.41.64M5.48
GA526702.53319.21.71M12.27
BA603786.79157.81.79M13.41
GA+627828.65313.12.58M9.69
AMAPE (ours)857155.02210.41.70M20.01
DeepSeek-V4-FlashVanilla5762.01762.011.43M1.40
CoT81097.07548.51.44M2.09
GA51236.4618.21.43M1.40
BA121028.26205.71.56M3.21
GA+182796.25399.51.48M3.38
AMAPE (ours)652543.59106.01.83M14.19
Qwen3.8-27BVanilla281187.29108.01.54M7.15
CoT83309.631103.21.78M1.69
GA45999.7655.51.59M11.29
BA521070.1851.01.69M12.41
GA+651467.2556.42.57M10.12
AMAPE (ours)68902.8533.41.69M15.94
Reviewer_RndiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes AMAPE, an online framework that maintains a pool of task-specific LLM sub-agents. For each incoming task, it either generates a new skill prompt or selects an existing sub-agent. Generation is triggered when the Wilson lower bound of cumulative task success falls below a threshold; selection combines similarity to previously successful tasks with a UCB score. Experiments on RealFin-Bench and a subset of MATH show improvements in success rate across several backbone models, with particularly large gains on RealFin-Bench.

Critical strengths
  1. The paper addresses a practical question: how to reuse task-specific experience while processing tasks sequentially, without an extensive offline search.
  2. The framework is relatively simple to implement. Its generated skills and selection decisions have clear operational roles.
  3. Evaluation covers two different task domains and three backbone models. The RealFin-Bench results suggest that concise, reusable task guidance can substantially improve performance in the tested setting.
Critical weaknesses
  1. The evidence does not establish adaptation to genuinely new or shifting tasks. The central motivation is online adaptation to changing task distributions, but new-agent generation depends on a global cumulative Wilson lower bound, not on whether the current task falls outside the expertise represented in the pool. After a long period of strong performance, a novel task may therefore be assigned to an unsuitable existing agent without triggering generation. Even when generation occurs, the new sub-agent primarily differs by an LLM-generated skill prompt; the framework provides no clear pre-execution validation or quality control for that skill. Results aggregated over two benchmarks do not show how the method behaves at a distribution shift, how well it handles the first task in a new domain, or how quickly subsequent performance recovers. The concern is not the absence of a universal performance guarantee, but the lack of direct evidence for the paper’s main adaptability claim.
  2. The efficiency evaluation does not clearly account for the framework’s additional costs. Skill generation requires an extra LLM call, while retrieval, similarity computation, and pool maintenance add work beyond task execution. The reported token accounting appears centered on executed tasks, and the paper does not clearly state whether all generation calls are included in TotalTok. It also does not report end-to-end latency, generation frequency, or how storage and retrieval costs grow over a longer task stream. Consequently, the reported success-per-million-token metric does not by itself substantiate the claim of improved performance without additional execution overhead.
Questions for authors
  1. How does AMAPE identify a genuinely novel task when the cumulative Wilson lower bound remains above the generation threshold? Have the authors considered a task-level novelty or low-similarity trigger?
  2. Can the authors evaluate a chronological task stream with explicit domain shifts? Please report first-task performance in a new domain, the number of tasks required to adapt, and performance after the shift.
  3. How often are generated skills ineffective or harmful? Is there a validation, rejection, replacement, or rollback mechanism before a new skill is retained in the pool?
  4. Do the reported Input Tokens, Output Tokens, and TotalTok include every LLM call used to generate skills and process historical successes and failures? If not, how do the efficiency rankings change when these calls are included?
  5. What are the end-to-end latency and resource costs per task, separated into generation, retrieval and selection, and execution? How do these costs scale as the pool and its task histories grow?
2 comments
AuthorsOpenReview ↗

Response to Reviewer Rndi

Dear Reviewer Rndi, We sincerely thank you for the detailed and constructive review. We appreciate your positive assessment of the practical problem addressed, the simple implementation, and the clear operational roles of generated skills and selection decisions. We address your main questions below and describe the additional analyses and revisions we will incorporate.

1 Novel-Task Identification

When the cumulative Wilson lower bound remains above the generation threshold, AMAPE does not generate a new skill; instead, it retrieves the best-matching skill from the existing pool and executes the task with it. This design follows the reinforcement-learning principle of "exploit when capable, explore/update when not": a lower bound above the threshold corresponds to "currently capable," in which case verified skills are preferentially reused; only when the lower bound falls below the threshold, indicating insufficient capability, is a new skill generated. It should be noted that the cumulative Wilson lower bound is a global trigger signal, used to decide whether the skill pool as a whole still needs expansion, whereas in-pool similarity retrieval is a local judgment, used to decide which skill covers the current task. The two are complementary: the former decides "whether to expand the pool," the latter decides "which skill to use for the current task."

We clarify, however, that we have not implemented an explicit task-level low-similarity trigger. When the cumulative lower bound remains above the threshold, the system does not actively judge a task as "novel"; it retrieves the most similar skill and executes the task under it. Even if the match quality is low, the task is still executed under that skill and its outcome is recorded in the skill's statistics, thereby lowering the skill's UCB and the global Wilson lower bound and indirectly and with delay driving subsequent generation through statistical propagation. Thus, the match quality of low-similarity retrieval can serve as a supplementary indicator of novelty, but does not constitute an independent generation trigger. We will clarify this boundary in the revised manuscript and list "explicit low-similarity-based novelty triggering" as future work.

2 About Domain Shifts

We thank the reviewer for this question. We clarify that the online setting of this study does not contain a "temporally ordered task stream with explicit domain shift," and therefore the three metrics requested by the reviewer—the performance on the first task in the new domain, the number of tasks required for adaptation, and the post-shift performance—have no corresponding objects of definition in our task stream, and we are unable to report them in the form of a "domain switch point." The reason is that the distribution shift in this study refers to the dynamic variation of task difficulty within an online task stream, rather than an explicit cross-domain switch. RealFin itself exhibits natural difficulty fluctuations over time; the MATH Level-5 subset covers four categories (Counting & Probability, Number Theory, Prealgebra, and Precalculus), introducing heterogeneity in problem-solving patterns across sequentially arriving tasks rather than a localizable "domain switch point." AMAPE's Wilson-lower-bound triggering mechanism generates task-specific sub-agents precisely when capability is insufficient, in response to such difficulty variations. Therefore, the main experiments on RealFin and MATH already evaluate online adaptive capability on task streams with dynamically varying difficulty , rather than under an artificially constructed explicit domain shift. Should the reviewer deem it necessary, we can clarify this setting difference and its implications for the choice of evaluation metrics in the discussion.

3 Validation of Generated Skills

We thank the reviewer for this question. We clarify the current design: a newly generated skill is executed directly on the current task without a separate validation stage; the skill pool is an append-only structure with no explicit rejection, replacement, or rollback mechanisms. Instead, we rely on UCB scores for implicit elimination: once admitted, a new skill immediately participates in UCB-based competition, and if its observed success rate is low, its UCB score decreases accordingly, reducing its probability of being selected and thus marginalizing it in practice—functionally equivalent to a "soft rollback," although the pool entry itself is not removed.

AuthorsOpenReview ↗

Response to Reviewer Rndi

4 About token and efficiency

The reported input tokens, output tokens, and total tokens already cover all LLM tokens incurred in skill generation and in processing every task, whether it succeeds or fails. There are therefore no uncounted LLM calls; re-attributing these calls would not change the accounting basis, and the efficiency ranking remains unchanged.

5 End-to-end latency and resource costs

We thank the reviewer for this question. To address it directly, we conducted a supplementary experiment and report the average execution time per successful task on RealFin as a measure of end-to-end latency.

Table 1: Performance comparison on RealFin-Bench.

ModelMethodSR (%)Total (s)Per successful task (s)TotalTokEFF
GLM-5.2Vanilla302738.3228.21.79M6.71
CoT221785.83198.41.64M5.48
GA526702.53319.21.71M12.27
BA603786.79157.81.79M13.41
GA+627828.65313.12.58M9.69
AMAPE (ours)857155.02210.41.70M20.01
DeepSeek-V4-FlashVanilla5762.01762.011.43M1.40
CoT81097.07548.51.44M2.09
GA51236.4618.21.43M1.40
BA121028.26205.71.56M3.21
GA+182796.25399.51.48M3.38
AMAPE (ours)652543.59106.01.83M14.19
Qwen3.8-27BVanilla281187.29108.01.54M7.15
CoT83309.631103.21.78M1.69
GA45999.7655.51.59M11.29
BA521070.1851.01.69M12.41
GA+651467.2556.42.57M10.12
AMAPE (ours)68902.8533.41.69M15.94
  • Metric definition: Per Successful Task is defined as the total end-to-end execution time divided by the number of successfully executed tasks, where the end-to-end execution time is the complete duration of a single task from start to finish, already including generating new agents, retrieval and selection, and invoking the LLM to execute the task with the selected agent. We note that the numerator includes failed-task time while the denominator counts only successful tasks, so the metric varies with the success rate and is a conservative estimate for successful tasks.
  • Breakdown: the end-to-end time consists of three parts—(i) generation, which occurs only when a new skill is triggered and involves LLM calls; (ii) retrieval and selection, where Jaccard-based similar-case retrieval and UCB scoring are local computations with no LLM calls; and (iii) execution, where each task invokes the LLM. Since our logs record the end-to-end duration per task rather than separately timing the three parts, we report the end-to-end total rather than per-part values; retrieval is local and negligible relative to LLM calls, and generation occurs only upon triggering, so the end-to-end time is dominated by execution.
  • Results: on GLM5.2, our method achieves an average end-to-end time of 210.4 s per successful task, higher than the shortest 157.8 s; however, the latter corresponds to an SR of only 60 and an EFF of only 13.41, whereas our method reaches an SR of 85 and an EFF of 20.01. On DeepSeek V4 Flash and Qwen3.8-27B, our method has both the shortest end-to-end time and the highest EFF.Overall, our method consistently leads in success rate and EFF, achieves the shortest end-to-end time on two of the three models, and on GLM-5.2 trades a slightly higher execution time for substantially higher SR and EFF.
  • Scaling: as the skill pool and task history grow, the component of end-to-end time affected by pool size is mainly retrieval and selection—its local computation grows with the number of candidates (one similarity and UCB computation per arm), but this overhead is local and far below LLM-call cost; generation cost grows only with the number of generations and is independent of pool size; execution cost is independent of pool size. The growth in end-to-end latency due to pool expansion therefore comes mainly from local retrieval computation and is negligible relative to LLM calls.
Reviewer_A76kOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AMAPE, an online framework for adaptive multi-agent pool evolution. AMAPE uses the Wilson confidence lower bound to determine when to generate new agents, synthesizes task-specific skills from historical successes and failures, and employs similarity-weighted UCB scores for agent selection. Experiments on RealFin-Bench and MATH-Bench across three backbone models demonstrate improvements in task success rates and token efficiency, particularly on complex financial tasks.

Critical strengths

Practical and unified framework. The paper presents a complete online agent generation and selection framework that combines historical experience, performance monitoring, and skill reuse. The overall design is intuitive and potentially useful for practical agent systems.

Promising empirical performance. The experiments cover two benchmarks and three backbone models, with comparisons against recent methods such as GenericAgent and Bayesian-Agent. The improvements on RealFin-Bench are particularly encouraging.

Critical weaknesses

Limited methodological novelty and heuristic design choices. The main components rely on existing techniques, including Wilson confidence intervals, UCB, similarity-based retrieval, and LLM-based skill generation. Their integration appears largely heuristic, without sufficient theoretical justification or empirical evidence demonstrating why these particular designs are preferable to simpler alternatives.

Insufficient ablation studies and validation of online adaptation. The ablation study primarily examines the sensitivity to the Wilson threshold, without isolating the contributions of the generation criterion, success/failure experience, or similarity-weighted UCB selection. Moreover, the claimed adaptability to changing task distributions is not directly evaluated through distribution-shift experiments.

Questions for authors

Generation criterion: What motivates using the Wilson confidence lower bound rather than a simple success-rate threshold or a sliding-window estimate? Although threshold sensitivity is evaluated, can the authors demonstrate that Wilson-based triggering provides a meaningful advantage? How is the threshold selected without tuning on the test set?

Skill generation: How much does each component of the skill-generation prompt contribute to performance? Could the authors provide ablations using only the current task, adding successful examples, and adding failure experiences?

Agent selection: Why is the selection score defined as the product of task similarity and UCB? Can the authors compare it against simpler alternatives, such as similarity-only selection, UCB-only selection, or additive combinations?

Online adaptation and efficiency: Can the authors evaluate performance under explicit task-distribution shifts to substantiate the online adaptation claim? Additionally, are the token costs of skill generation and retrieval fully included in the reported efficiency metrics?

3 comments
AuthorsOpenReview ↗

Response to Reviewer A76k

Dear Reviewer A76k, we sincerely thank you for the detailed and constructive review. We appreciate your positive assessment of Practical and unified framework and Promising empirical performance. We address your main questions below and describe the additional analyses and revisions we will incorporate.

1 Generation criterion:

  • Motivation for using the Wilson lower confidence bound: Using the point estimate of acc as the decision criterion fails to distinguish estimates with vastly different statistical credibility: 3/3 (acc = 1.0) and 210/300 (acc = 0.7) differ substantially in value, yet under a single threshold both may be accepted equally. The former has only 3 samples and thus enormous variance; its high estimate may stem from chance fluctuation, yet it crosses the threshold and erroneously suppresses evolution. The Wilson lower bound penalizes sample size (LCB ≈ 0.44 for 3/3 vs. ≈ 0.65 for 210/300), shifting the decision from "whether the empirical mean exceeds the threshold" to "whether the lower bound of the true accuracy exceeds the threshold," thereby reducing the risk of false acceptance caused by sampling variance. Sliding-window acc addresses temporal non-stationarity but remains a point estimate and does not handle the uncertainty introduced by insufficient samples within the window: a small window yields high variance and spurious triggers, while a large window responds sluggishly, and the window length itself constitutes a bias–variance trade-off. In summary, point estimates ignore sampling uncertainty, and sliding windows address only recency; the Wilson lower bound directly models the credibility of estimates under finite samples, rendering skill-evolution decisions statistically robust to sampling randomness and sample size.

  • Ablation studies: Accordingly, we supplement three ablation studies— acc point estimation, acc sliding window (window length = 5), and the Wilson lower confidence bound—with the sliding-window threshold kept consistent with the main experiment at 0.4. As shown in the table below, the experimental results corroborate the theoretical analysis above.

Table 1: Generation Guidelines ablation study on RealFin-Bench.

ModelMethodSR (%)InTokOutTokTotalTokEFF
GLM-5.2acc721.51M154k1.66M17.42
GLM-5.2sliding-window estimate721.40M151k1.55M18.73
GLM-5.2Wilson lower bound (ours)851.55M152k1.70M20.01
  • Threshold selection:The threshold is not set purely empirically; rather, it is determined using the Wilson confidence lower bound of SR (i.e., acc) as a conservative reference: the threshold is placed below the point estimate of the best baseline's SR and follows the conservative direction indicated by its Wilson lower bound, so that the success rate corresponding to the threshold carries statistical credibility; the specific value is then determined in conjunction with the sensitivity of AMAPE to the threshold. For example, on the RealFin dataset, where GA+'s SR is approximately 0.65, AMAPE reaches its optimum when the threshold is set to 0.3–0.4; on the Math dataset, where GA+'s SR is approximately 0.8–0.9, AMAPE reaches its optimum when the threshold is set to 0.5–0.6. The offset of the threshold relative to SR is roughly around 0.3 on both datasets; this offset is an interval determined by taking the lower bound as a conservative reference and combining it with sensitivity experiments, rather than the shrinkage of the lower bound itself.
AuthorsOpenReview ↗

Response to Reviewer A76k

2 Skill generation:

To address this question, we conducted an ablation study on skill generation, comparing four settings: current task only, current task + failure experience, current task + successful examples, and our full algorithm. The experiments cover RealFin and MATH on GLM5.2 and Qwen3.8-27B, as shown in the table below. As can be seen from the table: on RealFin, adding either successful examples or failure experience alone affects the generated skills and to some extent leads to a decrease in SR; only by combining the current task, successful examples, and failure experience can skill generation be better guided, thereby improving task SR. On MATH, the effects of failure experience and successful experience differ across models: successful experience better helps GLM5.2 generate task-adapted skills, whereas for Qwen3.8-27B the inclusion of successful experience leads to a decrease in SR. Overall, however, only the inclusion of all components improves task SR.

Table 2: Component ablation study on RealFin-Bench.

ModelComponentSR (%)InTokOutTokTotalTokEFF
GLM-5.2q_i801.42M156k1.57M20.33
(q_i, F_fail)681.48M161k1.64M16.48
(q_i, R(q_i))721.58M134k1.71M16.93
(q_i, R(q_i), F_fail) (ours)851.55M152k1.70M20.01
Qwen3.8-27Bq_i651.65M120k1.77M14.70
(q_i, F_fail)601.72M140k1.86M12.90
(q_i, R(q_i))601.67M106k1.78M13.48
(q_i, R(q_i), F_fail) (ours)681.59M104k1.69M15.94

Table 3: Component ablation study on MATH-Bench.

ModelMethodSR (%)InTokOutTokTotalTokEFF
GLM-5.2q_i88640k68k708k124.24
(q_i, F_fail)86675k70k745k115.48
(q_i, R(q_i))91637k80k717k126.86
(q_i, R(q_i), F_fail) (ours)91624k73k697k130.5
Qwen3.8-27Bq_i78958k100k1.06M73.70
(q_i, F_fail)78959k100k1.06M73.66
(q_i, R(q_i))751.08M126k1.21M62.06
(q_i, R(q_i), F_fail) (ours)82947k113k1.06M77.33
AuthorsOpenReview ↗

Response to Reviewer A76k

3 Agent selection:

We define the selection score as the product of task similarity and UCB, i.e., score = sim × ucb, for three reasons.

First, the product implements similarity gating. If an agent has no successful case similar to the current task (sim = 0), its score is 0 regardless of its UCB, so it is not preferentially selected. This ensures that reuse is biased toward agents already validated on similar tasks, rather than selecting an irrelevant agent merely due to the exploration term. This "zero-similarity veto" cannot be expressed by an additive form, where an agent with sim = 0 could still win on UCB alone.

Second, the product makes similarity a scaling factor on UCB. When two agents have close UCB scores, the similarity difference directly separates their scores, giving more similar agents a stronger reuse preference; when similarities are close, UCB still dominates selection and continues to balance exploration and exploitation. Similarity thus determines among which agents to choose, while UCB determines which one to prefer among them.

Third, the product requires no additional weighting coefficient. An additive form α·sim + (1−α)·ucb requires setting α and is sensitive to it; the product introduces no extra hyperparameter and matches the intuition of a zero-similarity veto. The cost is that a small similarity suppresses the UCB exploration term, which may under-explore dissimilar-but-promising agents. This is a deliberate conservative trade-off: in an online setting where generation is expensive and the pool is append-only, we prefer reusing verified similar skills over selecting irrelevant ones merely for exploration.

To address this question, we conducted an ablation study on agent selection , comparing three selection methods: similarity only, UCB only, and the product of similarity and UCB. The experiments cover RealFin and MATH on GLM5.2 and Qwen3.8-27B, as shown in the table below. As can be seen from the table: on both RealFin and MATH, measured by SR and EFF, the product of similarity and UCB performs best among the three methods, followed by similarity only, and UCB only ranks last.

Table 4: Agent selection ablation study on RealFin-Bench.

ModelSelection methodSR (%)InTokOutTokTotalTokEFF
GLM-5.2sim801.51M175k1.69M18.95
ucb701.44M154k1.60M17.55
sim × ucb (ours)851.55M152k1.70M20.01
Qwen3.8-27Bsim681.72M152k1.87M14.41
ucb481.72M123k1.84M10.31
sim × ucb (ours)681.59M104k1.69M15.94

Table 5: Agent selection ablation study on MATH-Bench.

ModelMethodSR (%)InTokOutTokTotalTokEFF
GLM-5.2sim801.51M175k1.69M18.95
ucb88732k84k816k107.90
sim × ucb (ours)91624k73k697k130.5
Qwen3.8-27Bsim751.08M111k1.19M62.97
ucb691.01M118k1.13M60.89
sim × ucb (ours)82947k113k1.06M77.33

4 Online adaptation and efficiency:

  • Online adaptation: We thank the reviewer for this question. We clarify that the task distribution shift in this work refers to the dynamic variation of task difficulty within an online task stream, rather than a classical train–test distribution mismatch, which is precisely the scenario AMAPE targets. RealFin itself exhibits natural difficulty fluctuations over time; the MATH Level-5 subset covers four categories (Counting & Probability, Number Theory, Prealgebra, and Precalculus), introducing heterogeneity in problem-solving patterns across sequentially arriving tasks. AMAPE's Wilson-lower-bound-based trigger mechanism responds to such variations by generating task-specific sub-agents when the lower bound falls below the threshold. Therefore, the main experiments on RealFin and MATH already evaluate online adaptive capability on task streams with dynamically varying difficulty, rather than offline evaluation under a static distribution.
  • efficiency: Skill generation invokes the LLM, and its input, output, and total tokens are all measured and fully accounted for in the efficiency metric. Retrieval is implemented via Jaccard similarity and is essentially a lightweight, interpretable, gracefully-degrading similar-case retriever that produces no LLM tokens, and is therefore not included in the efficiency metric.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.