- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how competition and cooperation affect the self-evolution of LLM agents. It compares both the solutions discovered during interaction and the memories transferred to fresh agents, while keeping the memory updater fixed. Competition encourages exploration, whereas cooperation produces procedures that are easier for later agents to reuse. The paper also evaluates schedules that combine the two regimes.
Critical strengths
- I find the connection between diversity and memory inheritance interesting. A population can lose behavioral diversity while learning a procedure that improves subsequent agents. The paper examines this distinction through transfer, which gives more information than diversity or task accuracy alone.
- Evaluating fresh solvers with frozen memory separates inherited experience from the original interaction history. Section 6 provides memory controls and results across evolution lengths. The scheduling results also show that the preferred order depends on the model and task, rather than one schedule working best throughout.
Critical weaknesses
- The main claim concerns how diversity affects the experience inherited by later agents. The transfer experiments show that cooperative conventions can be reused, but say less about whether successors can depart from them when the conventions become unsuitable. This leaves a limit on how much the results tell us about avoiding diversity collapse. It does not undermine successful reuse in the tested tasks, and I would treat it as a boundary of the contribution rather than require diversity to remain high throughout.
- The experimental explanation based on terminal credit also needs clarification. Tables 2 and 5 give different AIME and HotpotQA contrasts. Table 5 distinguishes any-agent success from joint success that receives credit, but its relationship to Table 2 remains unclear. Reconciling the tables would help determine how much of the effect reflects improved task solving and how much reflects which successes receive credit.
Questions for authors
- Can successors recover when an inherited cooperative convention no longer fits the task?
- Can the terminal-credit effects in Tables 2 and 5 be reconciled under the same definition of task success?
4 comments
Response to Reviewer e5zo - Part (2/2) (Q1b - Q2)
Q1 (continued)
Most backbones achieve their highest completion rate under COMP→COOP.
-
Frozen cooperative conventions are followed faithfully, even when they become misleading: in the emergent-convention experiment, swapping cyan and yellow inside an inherited cooperative memory drops completion from 0.814 to 0.367 , showing that successors execute the stored convention rather than correcting it. The new experiment clarifies the conditions under which agents can still break free from it.
-
However, conventions can still be revised online during active interaction: When the color rule is inverted during the final 4 of 12 rounds, cooperative pairs realign within a single round and solve 65% of the post-reversal rounds (62% overall), whereas competitive pairs solve only 13%.
Q2: The experimental explanation based on terminal credit also needs clarification. Tables 2 and 5 give different AIME and HotpotQA contrasts. Table 5 distinguishes any-agent success from joint success that receives credit, but its relationship to Table 2 remains unclear. Reconciling the tables would help determine how much of the effect reflects improved task solving and how much reflects which successes receive credit.
R2: Thank you for your careful check. Table 2 reports credit assignment, whereas Table 5 reports task success; Reading the two tables together is essential to disentangle task solving from credit attribution.
-
Table 2 reports Credit: The fraction of episodes in which an agent receives a positive terminal reward under a given credit assignment rule.
-
Table 5 reports Task Success: The fraction of episodes achieving any-agent success and joint success.
-
Joint Interpretation: The gap in Table 2 reflects how much additional credit is assigned under individual credit, whereas Table 5 reveals how much of that gap corresponds to genuinely solving the task.
Revision Plan: We will explicitly state the exact metric reported in each table's caption and place the credit and task success views side by side for direct comparison.
To further quantify run-to-run variance, we replicated the experiment across 16 random seeds :
| Task | Benchmark Channel | Any-Agent (Individual) | Joint (Individual) | Any-Agent (Shared) | Joint (Shared) |
|---|---|---|---|---|---|
| AIME | Broadcast | 0.93 | 0.63 | 0.58 | 0.45 |
| AIME | Isolated | 1.00 | 1.00 | 0.90 | 0.84 |
| HotpotQA | Broadcast | 1.00 | 0.77 | 0.41 | 0.36 |
| HotpotQA | Isolated | 1.00 | 0.95 | 0.84 | 0.81 |
| ToolUse | Broadcast | 1.00 | 1.00 | 1.00 | 1.00 |
| ToolUse | Isolated | 1.00 | 1.00 | 1.00 | 1.00 |
Two key findings emerge from this replication:
-
First, the effect of credit assignment depends on the degree of trajectory coupling. Measured by any-agent accuracy, the gain of individual over shared credit varies substantially across communication channels: under a shared problem stream , the gains are +0.35 (AIME) and +0.59 (HotpotQA), whereas under independent answering , they drop to +0.10 and +0.16—an approximately difference. Reward structure begins to shape outcomes only when both agents operate along the same trajectory.
-
Second, shared credit suffers from a final-step bottleneck under a shared problem stream . Performance is weakest in this setting and rebounds to 0.90 and 0.84 under independent answering . In a shared problem stream, the team's outcome is dictated by the last-acting agent, so a mistake by either party penalizes the entire team—the tighter the coupling, the higher the coordination demand placed on cooperation.
Dear Reviewer e5zo,
Thank you for your review. Since the discussion time is less than a day, we would appreciate hearing whether our response addresses your concerns, and are happy to clarify anything further.
Best regards,
The Authors
Thank you for the clarifications and additional experiments. The rule-reversal experiment addresses my question about outdated conventions. It shows that competing first and cooperating after the reversal helps agents pass the updated rule on to their successors. The distinction between receiving positive reward in Table 2 and answering correctly in Table 5, together with the 16-seed results, also clarifies how credit assignment affects task performance. I maintain my Weak Accept rating.
Response to Reviewer e5zo - Part (1/2) (Q1a)
Thank you very much for accurately summarizing our core thesis and for your thoughtful assessment of the scope of our contribution. Your intuition is spot on: whether successor agents can depart from obsolete conventions was indeed a key boundary of our initial analysis. Motivated by your comment, we conducted a dedicated set of experiments showing that this boundary can in fact be crossed—provided the training stages are ordered properly.
R1: We agree that this is an important question. Our new experiments demonstrate that as long as exploration precedes consolidation, successor agents can successfully depart from outdated conventions.
New Experiment: Can successors adopt a new convention after a convention reversal? Across 160 seeds in the emergent convention setup, we invert the color semantics in the second half of training and compare the frozen memories produced by five training schedules in terms of completion rate on 10 held-out circuits governed by the new rule (where yellow denotes the true breakpoint) versus the original rule:
- Exploring before consolidating enables escaping obsolete conventions: COMP→COOP (compete first, then cooperate after reversal) outperforms COOP-only by +0.261 (95% CI:
+01˙95,+03˙28; outperforming in 111 of 160 seeds and underperforming in 32).
- Ordering is critical: The reverse schedule, COOP→COMP (cooperate first, then compete after reversal), offers no benefit and drops by −0.159 relative to COOP-only (
−02˙33,−00˙82). The gap between the two staged schedules reaches 0.420, with COMP→COOP prevailing in 121 out of 160 seeds.
This finding turns what was previously a boundary condition into a positive takeaway that aligns directly with Proposition C.5: exploration must precede consolidation; once a convention is crystallized into memory, switching to competition afterward cannot undo it.
Cross-Model Replication (Harder Circuits). To rule out the possibility that our conclusions depend on a specific circuit design or a single model, we constructed a harder set of circuits: decoys are placed along equally plausible alternative paths, and each marker is assigned a distinct color, driving the no-convention completion rate down to near-chance levels. We then replicated the convention-reversal experiment across 3 backbones. Averaged with equal weight across the 3 backbones, COMP→COOP competefirst,thencooperatepost−reversal achieves a completion rate of 0.523, outperforming both COOP-only at 0.384 (+0.139, 95% CI:
00˙52,02˙24) and COOP→COMP cooperatefirst,thencompetepost−reversal at 0.396 (+0.127,
00˙33,02˙20. The per-backbone results completionrateonnew−rulecircuits are as follows: