acceptodds
Under review as a conference paper at ICLR 2027

CollabFlow: Recursive Self-Improvement of Agent Collaboration

Abstract

Recursive self-improvement (RSI) lets a system improve from its own outcomes; in LLM-based multi-agent systems, Agents refine one another within a task, and outcomes improve how they collaborate across tasks. However, existing multi-agent collaboration leaves this loop open: collaboration is pre-defined at the operator level, topology-only learning keeps verbatim exchange that propagates errors, and reward maximization on a system's own outcomes concentrates on a few teams. To address these challenges, we propose CollabFlow, an RSI system of Learned Agent Collaboration: a trainable Collab-Director constructs teams of complete Agents, a frozen executor runs them, and each round's outcomes retrain the director. Within each round, the edges of a collaboration graph carry protocols of Evidence-Conditioned Communication: a receiver adopts a differing answer only when the sender's evidence is stronger by a margin, so the director learns who communicates and how. Across rounds, we further propose Collaborative Trajectory Balance (CTB), a flow-based objective that credits each team once across its construction orders and targets a reward-proportional distribution over teams, so several good teams stay in play. We also bound how far this self-generated target moves between rounds, which shrinks as records accumulate. On twelve datasets, CollabFlow outperforms all baselines and keeps improving across rounds. Code is available at https://anonymous.4open.science/r/CollabFlow-631E.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
LLM Agents, Multi-Agent Systems

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_eBPxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

CollabFlow trains a director to assemble teams of agents, assign skills, and choose communication paths and protocols. A frozen executor runs each team, and an evidence gate determines whether a receiver keeps, adopts, or revises an answer. The director is trained with Collaborative Trajectory Balance (CTB), while a skill library evolves across rounds. The experiments cover QA, reasoning, coding, and interactive tasks. CollabFlow reports the best average result for each row in its five-run main comparison and evaluates six additional frozen executors.

Critical strengths

S1. The communication design is worth studying. The director chooses who communicates and how, while the evidence gate determines the receiver's response. This gives the paper a framework for examining communication alongside team structure.

S2. The evaluation spans different tasks and includes useful controls. The main results report five runs. Table 2 compares CollabFlow with a single agent, voting, a fixed team, and simpler communication rules. The fixed-team control is especially helpful for judging what the learned director adds.

Critical weaknesses

W1. The benefit of learning the team policy is smaller than the headline gain. Section 5 emphasizes a 33.3-point IID gain over direct inference, but the Fixed team control is much stronger than direct inference. In Table 2, which reports a single run, CollabFlow exceeds Fixed team by 2.68 points on average across the IID tasks and 3.39 points on OOD tasks. Some gaps amount to only one or two correct items. Please run the Fixed team under the same five-run protocol as the main comparison and report uncertainty for the difference. Its skills, tools, and budget also need to be clear.

W2. The CTB comparison is too short to establish its contribution. Figure 5b compares training methods for only eight steps, without run-to-run uncertainty. The diversity plot uses other systems and an untrained director rather than the same director trained with a different objective. A full training comparison of held-out accuracy and diversity would show whether CTB's early advantage persists. Please also define what replaces CTB in the “w/o CTB” ablation.

W3. The QA gate ablation is hard to square with the stated rule. With κ = 1, Eq. 7 adopts a differing answer only when the evidence-score difference exceeds 1 and keeps the current answer only when it is below −1. If QA tasks have no executable checks, scores can only be 0 or 1, so every disagreement appears to trigger revision. Yet Table 2 shows large gaps between CollabFlow and “Always revise”: 92.97 versus 68.75 on NQ-Open and 46.88 versus 24.22 on MuSiQue. Which checks are available, and how often does each gate branch run?

W4. Key settings are still hard to recover. The paper needs a compact account of the tools and retrieval available to each agent, the training and execution budgets, and the settings used to compare executors. These choices may affect the gains as much as the team policy does, especially since QA performance varies markedly across executors (Table H.1b). A single setup table would make the main comparisons easier to interpret and reproduce.

W5. The skill-evolution evidence is selective. The method describes binding and editing skills, but it is unclear how changes to skills affect team identity or past success records. The persistence plots also show the twenty steps with the largest immediate gains rather than the full sequence. Please explain how edited skills enter later training rounds and show the complete evolution trace. That would help assess whether the library improves reliably over time.

W6. The paper is hard to follow. Details about skill binding, library edits, and training are scattered, and the relationship among evolution steps, training rounds, and the steps in Figures 4c and 5b is unclear. Please define SR in Figure 5a and better justify the reward in Eq. 13. A simpler overview and one worked example would make the method easier to assess.

Questions for authors
  1. How does the Fixed team compare under the main five-run protocol with matched tools, skills, and budgets?
  2. Does CTB still outperform a matched alternative after the full training budget? What exactly is “w/o CTB”?
  3. Which executable checks and retrieval sources are available on the QA tasks? How frequently does each gate branch occur, and what does “Always revise” change when agents agree?
  4. Can you summarize the tools, budgets, and executor settings in one place?
  5. How do skill edits affect team identity and records, and what does the full evolution trace show?
Reviewer_g1EJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents a framework for learning collaboration among LLM-based agents. CollabFlow trains a director to construct agent teams and select communication protocols, while a frozen executor runs them. It combines evidence-conditioned answer updates with Collaborative Trajectory Balance, which learns a reward-proportional distribution over teams using task performance, historical reliability, and a structural prior. The paper evaluates the system on 12 benchmarks for question answering, reasoning, code generation, and interactive tasks.

Critical strengths
  • The paper is well-written and easy to follow.
  • The RSI topic is actual and well addressed in the paper.
  • The experimental evaluation is broad, demonstrating the consistently strong results on 12 benchmarks.
Critical weaknesses
  • Equation 6 and its accompanying discussion attribute changes across rounds entirely to the team distribution under a fixed execution kernel. But Appendix A.2 and RQ6 state that the system adds, modifies, and deletes textual skills between evolution steps. Skill binding appears in the implementation interface, but Appendix B.1 explicitly defines the theoretical construction process using four actions that omit skill binding.
  • Results in Figure 5c do not establish reliable improvement across the full evolution process, as only 20 evolution steps with the largest immediate gains. The reported improvement from 34.4% to 75.0%, and the persistence of gains for 15 of those steps, therefore characterize a favorably selected subset.
  • Ablations in Table 2 report one run and do not clearly state whether each ablation is retrained. This makes it difficult to separate a component’s intrinsic value from the effect of changing the execution conditions of an already trained director.
Questions for authors

Please address weaknesses above.

Reviewer_aA7xOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposed a new multi-agent architecture: CollabFlow, which has a trainable Collab-Director that builds a team as an annotated graph. The paper reported a big improvement over many baselines.

Critical strengths

The problem statement is clear. It tests again many benchmark, which is good. The code and data are public.

Critical weaknesses

The major concern is the lack of careful design of the experiments. Please see the questions for all the related weaknesses.

Since there is no author rebuttal, so I suggest the author revise based on the questions and resubmit for the next cycle.

Questions for authors

What tasks are used to train the director, and which 30 were evaluated?

Why do the six OOD benchmarks have "training splits" in the release, and what were they used for? Could you give more detailed explanations

On which items did skill-library evolution run in Fig. 5c–e? Was the skill library frozen before any evaluation was performed?

Does the token count in Table H.2 include director construction (scoring every legal action), revisions, and integration?

What is the retrieval tool?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.