- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
AsyncSwarm addresses delays in LLM swarms where a lead agent delegates information-seeking tasks to sub-agents. At inference time, the lead can act on a sub-agent’s response as soon as it arrives, without waiting for the others. During reinforcement learning, the method retrieves stored sub-agent responses instead of repeatedly running sub-agents, and updates only the lead model.
The authors evaluate the method on WideSearch, BrowseComp-ZH, and GAIA. On WideSearch, asynchronous inference is 1.62× faster than blocking inference with similar performance. Trace retrieval makes RL steps 5.84× faster than online RL, excluding the one-time cost of collecting traces. Under the same 22-hour RL budget, it yields larger performance gains than online RL on all three benchmarks. The evaluation uses real sub-agents at test time and includes analyses of retrieval and learned delegation behavior.
Critical strengths
-
A key strength is that the lead agent learns from stored responses but improves on held-out tasks when evaluated with real sub-agents and live tools. This shows that the learned behavior carries over to real execution, which is the central concern with this training approach.
-
The method also addresses a practical bottleneck in multi-agent reinforcement learning. Reusing a trace corpus makes training substantially faster without training the sub-agents or adding a complex retrieval system. Its simplicity and measured efficiency make it useful beyond the specific swarm studied here.
Critical weaknesses
-
The RL comparison is not step-matched. Both methods train for 22 hours, but online RL completes roughly 39 GRPO steps (22 h at 33.49 min per step), while trace-retrieval RL completes 237. The larger gains may therefore come from doing more updates, rather than from better learning per update. A held-out evaluation of the trace-retrieval checkpoint around step 40, together with online RL reward curves if available, would help separate these explanations.
-
A matched single-agent baseline is missing. The initial swarm scores 57.6 on WideSearch, below the reported 61.3 for its lead model alone, although those results use different settings. Please evaluate the single-agent lead with the same tools, context limit, and protocol to show whether the swarm itself helps before RL.
Questions for authors
Could you report held-out results for the trace-retrieval checkpoint at about step 40 and for the single-agent lead under the same setup?
2 comments
W2 — Initial swarm performance and learned orchestration
We agree that the initial swarm score needs explanation: a swarm also needs an effective orchestration policy. Table A separates the external standalone result from our before/after-RL evaluations.
Table A. System scores (%); standalone and swarm results use different evaluation settings.
| System | Result source | WideSearch Item-F1 (%) | BC-ZH accuracy (%) |
|---|---|---|---|
| GLM-4.5-Air standalone agent | External report | 61.3 | 53.1 |
| Initial swarm (No RL) | Our evaluation | 57.6 | 55.4 |
| Swarm + Trace-Retrieval Swarm RL | Our evaluation | 68.2 | 63.0 |
Within the same swarm architecture, sub-agent model, and evaluation settings, training only the lead agent raises WideSearch from 57.6 to 68.2 and BC-ZH from 55.4 to 63.0. This supports the value of learning orchestration within the same swarm.
The initial policy may not yet coordinate WideSearch's multi-entity searches well. Beyond finding individual facts, the lead agent must combine replies, identify missing entries, and arrange checks. Table B examines delegation and verification, which matter for this coverage requirement.
Table B. Changes in orchestration behavior on WideSearch (Appendix D).
| Behavior | Initial swarm | After RL | Change |
|---|---|---|---|
| Trajectories with verification (%) | 87.2 | 100.0 | +12.8 pp |
| Share of verification tasks (%) | 30.5 | 38.6 | +8.1 pp |
| Tasks per question | 7.37 | 8.50 | +1.13 |
| Sub-agents per question | 6.28 | 2.60 | −3.68 |
| Task-prompt length (tokens) | 254.8 | 321.4 | +66.6 |
After RL, the lead agent assigns more tasks to fewer sub-agents, uses more verification, and gives longer prompts. These shifts are consistent with more follow-up work and checking of missing information, helping explain how the same swarm can improve through training its lead agent.
We have clarified repeated assignment in Section 3.4 and Appendix D.
We appreciate your helpful comments on our evaluation.
Your question separates learning at the same step count from the benefit of completing more updates. We compare both 40-step checkpoints on identical test questions, tools, and inference settings with real sub-agents.
Table A. Performance after 40 steps (%, differences in percentage points).
Both methods improve after 40 steps, with Trace-Retrieval Swarm RL scoring higher on both benchmarks. Useful policy learning is therefore already visible with steps matched. With 22 hours of RL (excluding collection), faster interaction permits more steps and gains of 9.2 / 6.3 percentage points over Online RL on WideSearch / BC-ZH (Table 1). We have included the 40-step results in Section 3.2 and Appendix H.1 (Table 9). The added comparison is highlighted in blue in the updated PDF.