acceptodds
Under review as a conference paper at ICLR 2027

AgentPSO: Evolving Agent Reasoning Skill via Multi-agent Particle Swarm Optimization

Abstract

Multi-agent reasoning has shown promise for improving the problem-solving ability of large language models by allowing multiple agents to explore diverse reasoning paths. However, most existing multi-agent methods rely on inference-time debate or aggregation, which can be vulnerable to incorrect peer influence and biased consensus. Moreover, the agents' reasoning procedures remain static, as they do not accumulate reusable behaviors across tasks. In this paper, we introduce AgentPSO, a particle-swarm-inspired framework for evolving multi-agent reasoning skills. AgentPSO treats each agent as a particle-like reasoner whose state is a natural-language skill and whose velocity is a semantic update direction, iteratively guiding agents toward higher-performing skill configurations. Across training iterations, each agent updates its skill by combining its previous velocity, personal-best skill, global-best skill, and a self-reflective direction derived from peer reasoning trajectories. This enables agents to learn reusable reasoning behaviors by drawing on their own experience and on the strongest skills found by the population, without updating the parameters of the backbone language model. Experiments on mathematical and general reasoning benchmarks show that AgentPSO improves over static single-agent skills and test-time-only multi-agent reasoning baselines. The evolved skills further transfer across benchmarks and to another backbone model, suggesting that AgentPSO captures reusable reasoning procedures rather than merely optimizing benchmark-specific prompts.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Multi-Agent Reasoning, Self-Evolving, Reasoning Skill Optimization

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_8CnkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes to use the particle swarm optimization algorithm to achieve a self-evolving agent reasoning improvement. To be specific, they treat each agent as a particle-like reasoner and iteratively update its inherent skills after communicating with other agents. Through the training iterations, each agent will update its skills by combining its previous velocity and various skills. The authors conduct a set of extensive experiments to validate the effectiveness of the proposed designs. As shown, the proposed method improves over static single-agent skills and test-time-only multi-agent reasoning baselines.

Critical strengths
  1. The paper explores using particle swarm intelligence to update collective intelligence, offering a novel alternative to conventional approaches based on static message passing.
  2. The proposed method is evaluated on a wide range of benchmarks and outperforms conventional baselines on most datasets, demonstrating its effectiveness.
Critical weaknesses
  1. I think the main weakness of the paper is that its presentation is too high-level, making it difficult to understand. The authors combine collective intelligence with particle swarm intelligence to update agents’ skills. However, skill updating is a rather abstract concept, and the paper does not make it easy for readers to understand the implementation details, why the method works, or which components contribute to its effectiveness. The paper needs more technical detail to explain how particle swarm intelligence specifically influences agents’ skills and, in turn, their reasoning processes and outcomes.
Questions for authors
  1. I could not find explanations of the factors examined in the ablation study in Table 6. The definitions of these skills are unclear to me. If I have missed them, please point me to the relevant section.
  2. I would like to see a comparison of token consumption between the proposed particle swarm intelligence algorithm and other collective intelligence algorithms, so that its efficiency can be assessed.
  3. The current results are limited to text-based data. I am interested in the method’s potential for multimodal reasoning. Could it be extended to multimodal reasoning in the future?
2 comments

[W1] Presentation of the Method

We thank the reviewer for pointing out that the connection between the update mechanism and the agents’ reasoning behavior needs clearer explanation.

In AgentPSO, a skill sits_i^t is a natural-language instruction included in the agent’s problem-solving prompt, specifying procedures such as strategy selection and verification. This follows the broader use of language-based skill representations in prior work [1,2].

We clarify the implementation, component contributions, and resulting skill changes below.

1. Implementation details: Eqs. (1)–(3) use three prompted LLM calls: comparing the agent’s own and peer trajectories to derive a reflective direction; combining it with the current skill, previous velocity, and personal/global-best skills to form a revision plan; and rewriting the skill for subsequent problem solving. Figures 8–11 provide the prompts.

2. Which components contribute: Table 6 compares different combinations of velocity-update inputs. Removing ditd_i^t while retaining the other inputs reduces DeepMath accuracy from 79.50% to 72.50%. We explain the input definitions in Q1.

3. How the skill instructions evolve : Figures 6 and 7 trace the evolution of the velocity and skill, with annotations identifying their sources. The Self-Refine agent progressively incorporates problem-type classification, method selection, and verification procedures. Table 20 complements these examples by classifying accepted updates into formatting, verification, strategy selection, and generic changes.

[1] Skill Set Optimization: Reinforcing Language Model Behavior via Transferable Skills, ICML 2024

[2] SkillAxe: Sharpening LLM-Authored Agent Skills Through Evaluation-Guided Self-Refinement, arXiv 2026

[Q1] Definitions in Table 6

Each row of Table 6 specifies the inputs supplied to the velocity-update operator in Eq. (2). Here, viv_i denotes the previous velocity, sis_i the current skill, did_i the self-reflective direction, pip_i the personal-best skill, and gg the global-best skill. The symbol ⊕\oplus indicates that the components are included together in the prompt, rather than numerically added.

For example, vi⊕si⊕pi⊕gv_i \oplus s_i \oplus p_i \oplus g removes the reflective direction while retaining the other inputs. The variant vi⊕si⊕div_i \oplus s_i \oplus d_i retains trajectory-derived reflection but removes personal/global-best guidance. Since did_i uses both the agent’s own and peer trajectories, this latter variant is not a self-only reflection baseline. We add these definitions to the caption and Section 6.4.

[Q2] Token Consumption

We thank the reviewer for this suggestion. We compare token consumption across AgentPSO, MAD, and GEPA, reporting both optimization and test-time inference costs. For GEPA, we match AgentPSO's optimization API budget (approximately $4.45).

MethodOpt. TokensTest Tokens/QCalls/QAcc. (%)Pass@4 (%)
AgentPSO1,771,1553,517.3479.5087.00
MAD–5,878.1869.0070.50
GEPA1,534,863823.4176.50–

Test Tokens/Q denotes the average total input and output tokens per test question. Compared with MAD, AgentPSO reduces inference token consumption by approximately 40.2% (3,517.3 vs. 5,878.1 tokens/question) while achieving higher accuracy (79.5% vs. 69.0%). This reduction is primarily attributed to avoiding multi-round debate, requiring four rather than eight LLM calls per question.

Under matched optimization API costs, GEPA uses fewer tokens than AgentPSO but achieves lower accuracy (76.5% vs. 79.5%). Overall, these results highlight AgentPSO's reduced inference overhead compared with multi-round debate methods and its competitive performance relative to GEPA.

[Q3] Multimodal Reasoning

AgentPSO could potentially be extended to multimodal reasoning using a multimodal backbone while retaining textual skills and the direction–velocity–skill update structure. Peer observations would need to include relevant visual evidence and task-specific feedback. We leave empirical validation of this extension to future work.

Thank you for your response. After reading, some of my concerns have been addressed. However, I think the main concern still lies in the contribution and presentation. I think this work is still not in good shape at this stage. In this case, I lean to maintaining my previous score.

Reviewer_wgB5OpenReview ↗
Rating
2
Confidence
2(−2)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents AgentPSO, a multi-agent framework that uses particle swarm optimization (PSO) principles to iteratively improve the reasoning skills of LLM agents without updating model parameters. Each agent maintains a natural-language skill instruction and refines it based on its own reasoning experience, feedback from other agents, and personal-best and global-best skill states. The framework introduces a self-reflective direction and a semantic velocity update to guide skill evolution, while the optimized agents independently solve problems and aggregate their answers through majority voting at inference time. Experiments on mathematical and general reasoning benchmarks compare AgentPSO with multiple baselines, along with ablation studies and cross-benchmark and cross-model transfer evaluations.

Critical strengths
  1. I like the paper presents a clear and easy-to-follow framework for evolving agent reasoning skills through multi-agent collaboration. The overall pipeline is well organized, and the integration of peer feedback, personal-best, and global-best skills makes the approach straightforward to understand and implement.

  2. The experimental evaluation is fairly comprehensive in terms of baseline comparisons and transferability analysis. I particularly like the cross-benchmark and cross-model transfer experiments, which provide some encouraging evidence that the optimized skills can be reused beyond their original training settings.

Critical weaknesses
  1. I find it difficult to identify a substantial technical contribution in this work. Population-based prompt optimization, reflective evolution, and PSO-inspired optimization have already been explored in prior studies. The proposed semantic velocity is essentially an LLM-generated prompt revision, and I do not see any particularly novel or sophisticated mechanism that clearly distinguishes this method from existing approaches.

  2. The experimental evidence is not very convincing to me, especially given the relatively modest improvements over strong baselines and the substantial additional computational cost. Most experiments focus on static reasoning benchmarks, which limits the conclusions about generalizable agent skills. I also wonder how much of the improvement actually comes from multi-agent collaboration, since the paper does not adequately isolate peer learning from ordinary self-reflection.

Questions for authors
  1. I am curious what the PSO-inspired formulation actually brings beyond existing reflective prompt optimization methods such as GEPA or population-based evolution approaches.

  2. Would the performance gains still hold if each agent only learned from its own reasoning trajectories and correctness feedback, without observing other agents? I would like to see this comparison.

  3. The evaluation mostly focuses on static mathematical and general reasoning tasks, and I noticed that AgentPSO actually underperforms MAD in majority-vote accuracy on the additional medical benchmarks. Could you provide stronger evidence that the learned skills generalize to more diverse or interactive agentic environments?

  4. I noticed that AgentPSO requires substantially more optimization time and inference cost than GEPA. How would the results change under a strictly matched computational budget, particularly when accounting for actual token consumption and API costs rather than just the number of LLM calls?

2 comments

[W1,Q1] Contribution Beyond Iterative Prompt Rewriting

We thank the reviewer for the opportunity to clarify this distinction. Our main research questions lie in the optimization structure rather than the textual representation.

AgentPSO maintains an individual search history for each agent: the personal-best skill preserves successful behaviors, the global-best skill communicates discoveries across the population, and the previous semantic velocity retains earlier revision directions. These states are combined with a reflective direction derived from the agents’ reasoning trajectories and correctness feedback. Each agent therefore updates its own instruction using both accumulated search history and concrete behavioral evidence from peers.

This motivates the PSO-inspired structure: it combines individual continuity with population-level learning. AgentPSO does not claim numerical equivalence to standard PSO or inherit its convergence guarantees. Its effectiveness is evaluated through the component ablations and the additional evolutionary baseline below.

We evaluated an elitist selection-and-mutation baseline initialized with the same four skills: CoT, Step-Back, Self-Refine, and Reflection. The highest-scoring skill is preserved as an elite and independently mutated three times to form the next population. After 10 iterations, the final population is evaluated on 200 DeepMath test examples.

MethodAvg@4MajorityPass@4
Selection+Mutation60.50%69.50%76.50%
AgentPSO72.75%79.50%87.00%

AgentPSO improves Avg@4 by 12.25% and majority-vote accuracy by 10.00%. These results show that the tested selection-and-mutation procedure does not recover AgentPSO’s performance. Since the baseline also omits trajectory-level feedback, this comparison supports the complete update design rather than isolating personal/global-best memory alone.

[W2,Q2] Peer Learning versus Self-Only Reflection

We evaluate a self-only, no-global variant where each agent reflects only on its own trajectories and feedback. Peer trajectories and global-best guidance are removed, while the current skill, previous velocity, and personal-best skill are retained.

MethodAvg@4MajorityPass@4
Self-only,no-global62.50%72.00%80.00%
AgentPSO72.75%79.50%87.00%

Removing cross-agent information reduces Avg@4 by 10.25%, majority accuracy by 7.50%, and Pass@4 by 7%. All four agents show lower individual accuracy, indicating that the benefit extends beyond majority-voting.

The self-only variant retains individual experience but cannot learn from peer trajectories or global-best guidance. These results support the combined contribution of these two cross-agent information sources to skill evolution (Appendix D.10, Table 16).

[W2,Q3] Generalization beyond Static Reasoning Benchmarks

To evaluate generalization beyond static reasoning, we conduct experiments on ALFWorld [1], an interactive environment requiring multi-step planning and sequential actions.

We use 100 training games, 100 validation games, and 140 valid_seen test games with GPT-6-luna. AgentPSO retains the same four initial skills, four-agent population, and 10 optimization iterations. We adapt peer observations to include action-observation trajectories, task success, and step counts, while preserving the original skill-evolution mechanism.

MethodSuccess Rate(%)↑Avg.Steps↓Avg.Steps(Success)↓
CoT79.2923.5016.58
Step-Back79.2922.7615.64
Self-Refine81.4322.8716.68
Reflection79.2923.3416.38
MAD79.2923.2516.26
GEPA80.0023.6317.04
AgentPSO82.1419.1614.25

AgentPSO achieves the highest success rate (82.14%) with 17.6% fewer steps than MAD and 18.9% fewer than GEPA. These results provide preliminary evidence that AgentPSO can be applied to interactive multi-step tasks, with particularly promising improvements in action efficiency.

[1] ALFWorld: Aligning Text and Embodied Environments for Interactive Learning (ICLR 2021).

[W2,Q4] Comparison under Matched API Costs

To account for differences in token consumption and API costs, we compare AgentPSO with GEPA, the strongest self-evolving baseline, under matched optimization API costs.

AgentPSO consumes \4.4496foroptimization(1,400callsand1.77Mtokens).WeallocateGEPAthesameoptimizationbudget,resultingin4.4496 for optimization (1,400 calls and 1.77M tokens). We allocate GEPA the same optimization budget, resulting in \\4.4488 (1,507 calls and 1.53M tokens).

Under this budget, AgentPSO achieves higher DeepMath accuracy (79.50% vs. 76.50%). These results show that AgentPSO maintains competitive performance under comparable optimization costs. These results are included in Appendix D.5.

Dear authors,

Thanks for these additional experiments and your hard work. I think these results are helpful and provide stronger evidence, but I'm still worries about the novelty and technical contribution, so I decided to keep my original rating but reduce my confidence score from 4 to 2. The new selection-and-mutation baseline is useful, but it also removes trajectory-level feedback, so actually it is difficult to tell whether the improvement comes from the PSO-inspired optimization structure, or just come from richer feedback and information sharing. The self-only ablation demonstrates that cross-agent information is beneficial, but I think this is somehow different from demonstrating the necessity of PSO formulation. As for ALFWorld, I like your extension to an interactive environment. But, the success-rate improvement over GEPA is relatively small (82.14 vs. 80). The step-efficiency results are more encouraging, but I would be cautious about draw strong generalization conclusions from this environment. Besides, the matched optimization-cost comparison addresses an important part of my concern, but I am still want to know the overall inference cost, especially with multiple agents used for answer aggregation. So, my concern is mainly whether the PSO-inspired mechanism provides a sufficiently distinct technical contribution.

Reviewer_Lb3MOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces AgentPSO, a particle-swarm-inspired framework for optimizing the natural-language reasoning skills of LLM agents without updating model parameters. Each agent independently solves training problems and reflects on its own and other agents’ reasoning trajectories to derive an update direction. This direction is combined with the previous update direction, the agent’s personal-best skill, and the population’s global-best skill to refine its skill instructions. At inference time, agents solve problems independently and aggregate their answers through majority voting. Experimental results on mathematical and general reasoning benchmarks show that AgentPSO improves reasoning performance and learns skills that transfer across benchmarks and models.

Critical strengths
  1. The framework converts experience gained during optimization into reusable reasoning instructions without updating model parameters, making it suitable for models accessed through APIs.

  2. The method is presented and evaluated against single-agent, multi-agent, and self-evolving baselines. Cross-benchmark and cross-model evaluations assess transferability, while results over three random seeds help assess performance consistency.

  3. Ablation studies show that removing the self-reflective direction reduces performance, indicating its contribution to reasoning accuracy. Transfer experiments further support the reuse of learned skills across benchmarks and models.

Critical weaknesses

Insufficient differentiation from related work. SwarmAgentic already combines language-based updates, failure feedback, and personal-best and global-best guidance[1]. The paper should more establish the additional contribution of optimizing individual reasoning skills through peer trajectories.

Limited isolation of peer learning. The current ablation does not distinguish the contribution of peer trajectories from that of the agent’s own experience, leaving the benefit of peer learning unclear.

The interpretation of transfer results requires stronger support. Cross-benchmark and cross-model results support the reuse of learned instructions, but do not distinguish improved reasoning from better output-format compliance. Controlling output requirements and clarifying whether inference budgets are matched would strengthen the conclusions.

[1] SwarmAgentic: Towards Fully Automated Agentic System Generation via Swarm Intelligence. EMNLP, 2025.

Questions for authors
  1. How does AgentPSO differ methodologically from SwarmAgentic [1]? Which differences constitute the main contribution, and what experimental evidence supports their benefits?

  2. Does using peer trajectories improve performance over self-only reflection? Could the authors compare these settings while keeping the other update components unchanged?

  3. How is the no-velocity variant implemented, particularly the input to Eq. (3) after removing Eq. (2)? Have the authors considered retaining the two-stage update while omitting only the previous velocity? Are the ablation results consistent across independent runs?

  4. Could the authors clarify the relationship between this submission and the non-archival workshop paper [2], and summarize any differences in methodology or experimental evaluation?

[1] SwarmAgentic: Towards Fully Automated Agentic System Generation via Swarm Intelligence. EMNLP, 2025.

[2] AgentPSO: Evolving Agent Reasoning Skill via Multi-agent Particle Swarm Optimization. The 3rd AI for Math Workshop at ICML, 2026.

2 comments

[W1,Q1] Methodological Difference from SwarmAgentic

We thank the reviewer for highlighting this closely related work. We agree that SwarmAgentic already incorporates language-based PSO updates and personal/global-best guidance. Our contribution lies in the optimization unit and the trajectory-level feedback.

SwarmAgentic treats each particle as a complete agentic system and generates failure-aware updates from diagnosed system flaws and personal/global-best configurations. AgentPSO treats each particle as an individual reasoning instruction and derives its reflective direction by comparing the agent’s own reasoning with peer trajectories on the same problems. This provides direct evidence of alternative solution strategies and their outcomes, beyond identifying deficiencies in the current configuration. The reflective direction is then combined with persistent velocity and personal/global-best guidance to update reusable reasoning skills. We clarify this distinction in Section 2.

Empirically, removing the reflective direction reduces DeepMath accuracy from 79.50% to 72.50% (Table 6). Separately, removing both peer trajectories and global-best guidance reduces accuracy to 72.00% (Table 16). These results support the contributions of trajectory-based reflection and the combined use of peer trajectories and global-best guidance to skill evolution.

[W2,Q2] Peer Learning versus Self-Only Reflection

We conducted a self-only, no-global experiment in which each agent generates its reflective direction using only its own trajectories and feedback. We also remove global-best guidance while retaining the current skill, previous velocity, and personal-best skill.

MethodAvg@4MajorityPass@4
Self-only, no-global62.50%72.00%80.00%
AgentPSO72.75%79.50%87.00%

This reduces Avg@4 by 10.25%, majority accuracy by 7.50%, and Pass@4 by 7% points. All four agents show lower individual accuracy, so the benefit extends beyond voting.

In the self-only variant, agents retain their own experience but cannot directly learn from peer trajectories or the best skill discovered by another agent. These results support the combined contribution of these two cross-agent information sources to skill evolution. We have added this analysis to Appendix D.10 (Table 16).

[W3] Transfer Gains Beyond Output-Format Improvements

We analyze errors on three transfer benchmarks (MATH500, AIME25, and Minerva) to distinguish reasoning improvements from output-format compliance. Errors are aggregated across the four individual-agent responses for each of the test questions. Errors are classified as format-related only if the answer can be deterministically recovered and verified by the original benchmark grader; all others are classified as reasoning/content errors.

Across the three transfer benchmarks, reasoning/content errors decrease by 37, compared with 10 format-related errors. Thus, reasoning/content errors account for 37 of the 47 fewer errors (78.7%), suggesting that the transfer improvements extend beyond output formatting (Appendix D.11, Table 17).

For all transfer evaluations, initial and evolved skill populations use the same inference budget of four LLM calls per problem, ensuring that performance differences are not due to additional inference sampling.

[Q3] Implementation and Robustness of the No-Velocity Ablation

We thank the reviewer for pointing out the ambiguity in our description. Clarification of the no-velocity variant. In this variant, we remove the velocity-generation step in Eq. (2) and directly revise the current skill using the self-reflective direction (ditd_i^t), personal-best skill (pitp_i^t), and global-best skill (gtg^t), without generating or maintaining a semantic velocity. Thus, Eq. (3) is replaced by a direct skill update rather than used in its original form. We have revised Section 6.3 to clarify the definitions. Additional ablation without previous velocity. We also remove only the previous velocity (vitv_i^t) from Eq. (2), retaining all other inputs and Eq. (3).

MethodAcc.(%)↑Mean ↓Median ↓Max ↓
w/o previous velocity77.50.09420.06470.4894
w/ velocity (Full)79.50.08680.04880.4153

Consistent with the no-velocity ablation, removing the previous velocity decreases accuracy from 79.5% to 77.5% and increases mean semantic drift from 0.0868 to 0.0942. These results suggest that retaining velocity history helps preserve accumulated update directions and promotes more stable skill evolution (Section 6.3, Table 5).

[Q4] Relationship to the Non-Archival Workshop Paper

Regarding [2], the core method is shared, while the present submission includes additional evaluations and analyses. The present submission retains the core method and adds total LLM-call budget comparisons, a static multi-agent voting control, and a functional analysis of skill updates.

Thank you for the detailed response and additional experiments, which address many of my earlier concerns. However, I still have two concerns.

First, I remain uncertain about the substantive contribution beyond the non-archival workshop paper [2]. I would appreciate a clearer explanation of the new insights from the expanded evaluation. Second, I remain concerned about the overall computational cost. The added LLM-call comparisons are helpful. I would also appreciate comparisons under matched budgets that report accuracy, token usage, and runtime for both optimization and inference.

I will therefore maintain my original score.

Reviewer_UHjyOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper represents agent reasoning skills as natural-language states and uses a PSO-inspired optimization process combining personal-best, global-best, and peer-reflection signals. The backbone model remains frozen.

Critical strengths

The method is intuitive and avoids parameter updates. It evaluates against several single-agent, multi-agent, and self-evolving baselines. On five benchmarks, the reported average accuracy is 71.56%, compared with 68.31% for DMAD and 68.10% for GEPA. It also includes cross-benchmark and cross-model transfer experiments.

Critical weaknesses

The central novelty is uncertain. Natural-language velocity updates resemble iterative prompt rewriting with population-based selection. The paper needs to establish why PSO specifically is responsible for the improvement.

Comparisons should be normalized for total optimization calls, tokens, and inference-time sampling.

The primary experiments use four agents, 100 optimization examples, and 10 iterations. This is relatively narrow evidence for generalizable skill evolution.

Questions for authors

Does AgentPSO outperform a population-based evolutionary prompt optimizer with the same compute budget? What happens when the personal-best/global-best mechanism is replaced with ordinary selection and mutation? How stable are the improvements across substantially different backbone models?

1 comment

[W1] Contribution Beyond Iterative Prompt Rewriting

We thank the reviewer for the opportunity to clarify this distinction. We define “reasoning skills” as reusable procedures encoded in natural-language instructions (e.g., decomposition, strategy selection, verification). Our contribution lies in the optimization structure, not the textual representation.

AgentPSO maintains an individual search history for each agent: the personal-best skill preserves successful behaviors, the global-best skill communicates discoveries across the population, and the previous semantic velocity retains earlier revision directions. These states are combined with a reflective direction derived from the agents’ reasoning trajectories and correctness feedback. Each agent therefore updates its own instruction using both accumulated search history and concrete behavioral evidence from peers.

This motivates the PSO-inspired structure: it combines individual search history with population-level guidance. On DeepMath, removing previous velocity reduces accuracy from 79.50% to 77.50% (Section 6.3, Table 5); removing personal/global-best guidance yields 76.00%, and removing reflective direction yields 72.50% (Section 6.4, Table 6). These ablations support the roles of search history, best-state guidance, and trajectory-based reflection. We clarified the settings in Sections 6.3--6.4.

[W2, Q1] Comparison under Matched Optimization Budgets

To account for differences in token consumption and API costs, we compare AgentPSO with GEPA, the strongest self-evolving baseline, under matched optimization API costs.

AgentPSO consumes \4.4496foroptimization(1,400callsand1.77Mtokens).WeallocateGEPAthesameoptimizationbudget,resultingin4.4496 for optimization (1,400 calls and 1.77M tokens). We allocate GEPA the same optimization budget, resulting in \\4.4488 (1,507 calls and 1.53M tokens).

Under this budget, AgentPSO achieves higher DeepMath accuracy (79.50% vs. 76.50%). These results show that AgentPSO maintains competitive performance under comparable optimization costs. These results are included in Appendix D.5.

[W3] Population Size, Optimization Length, and Generalization

Figure 3 shows that DeepMath performance improves from two to four agents but slightly decreases with five, indicating that increasing the population does not necessarily improve performance.

We additionally extended optimization from 10 to 30 iterations and observed no further performance improvement in the tested setting. This supports the default optimization length for this experiment, without implying that the same schedule is sufficient for every task.

Table 3 provides complementary evidence of reuse across benchmarks. Skills optimized on BBH achieve 75.00% majority-vote accuracy on DeepMath, while DeepMath-optimized skills achieve 85.50% on BBH, without additional optimization. These results support cross-benchmark transfer, although sensitivity to the number of optimization examples remains a separate question.

[Q2] Comparison with Selection and Mutation

We evaluated an elitist selection-and-mutation baseline initialized with the same four skills: CoT, Step-Back, Self-Refine, and Reflection. The highest-scoring skill is preserved as an elite and independently mutated three times to form the next population. After 10 iterations, the final population is evaluated on DeepMath test examples.

MethodAvg@4MajorityPass@4
Selection+Mutation60.50%69.50%76.50%
AgentPSO72.75%79.50%87.00%

AgentPSO improves Avg@4 by 12.25 points and majority accuracy by 10 points, showing that simple selection and mutation do not recover its performance (Appendix D.10, Table 16).

[Q3] Robustness Across Different Backbone Models

We thank the reviewer for this question. To further examine AgentPSO's robustness across backbone models, we conduct additional experiments using Qwen3-32B.

MethodDeepMathMATHAIME25MinervaBBHAvg.
CoT60.0077.2010.0037.5071.5051.24
MAD71.0086.0026.6740.4481.5061.12
GEPA72.0086.0023.3338.9777.5059.56
AgentPSO (Qwen-only)79.5087.0023.3335.6682.5061.60
AgentPSO (Hybrid GPT/Qwen)81.0087.2026.6737.1383.0063.00

AgentPSO achieves an average accuracy of 61.60% in the Qwen-only setting, remaining competitive with Qwen-based multi-agent baselines and outperforming GEPA.

Furthermore, in the hybrid setting, Qwen3-32B serves as the problem-solving model during optimization, while GPT-5.4-mini performs reflection and skill updates. This setting achieves the highest average accuracy of 63.00% among the evaluated Qwen settings. This suggests that the framework is applicable to open-weight backbones (Appendix D.6, Table 12).

In addition, Table 4 and Appendix D.7 (Table 13) report direct transfer of GPT-optimized skills to Claude and Qwen3-32B without additional optimization, providing further evidence of cross-model skill reuse.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.