- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies recursive social improvement in populations of self-interested LLM agents. Rather than jointly solving one task, agents pursue individual rewards while deciding whether to revise their own skills, observe and copy a peer's skill, or execute a skill under a shared token budget. The authors formalize recursive social improvement as both recursive transmission of revised skills and a positive performance advantage over independent learners at matched population-wide token cost. They first investigate social learning in controlled multi-armed bandit environments and then study editable skill files on IFBench using OpenEvolve.
Critical strengths
- The paper asks whether knowledge exchange among self-improving agents produces gains beyond equally resourced independent search, rather than assuming that more agents or more copying necessarily improve outcomes.
- Careful evaluation perspective. The matched-cost formulation and distinction between efficiency and effectiveness address a meaningful confound in multi-agent comparisons. Including algorithmic policies in the controlled environment helps establish that useful social information is available.
- Useful behavioral diagnostics. The execution-budget, exploration, observation-timing, and skill-lineage analyses provide insights into how copying changes agent behavior. The authors also report negative results transparently instead of overstating small numerical gains.
Critical weaknesses
- Limited representativeness of the main experimental setting. The central IFBench experiments use only two LLMs on one benchmark, with homogeneous agents trained on identical examples and evaluated using the same objectives. This is a legitimate controlled setting, but it does not adequately cover the settings in which social learning would be most compelling: agents with complementary knowledge, different tasks, or heterogeneous capabilities. The breadth of the conclusions is therefore not commensurate with the breadth of the evidence.
- Insufficient support for the central negative conclusion. The absence of a statistically significant matched-cost gain is not evidence that social improvement cannot occur or that social and solo learning are effectively equivalent. With only six population seeds in the principal skill-evolution comparisons, the uncertainty remains consequential, especially for GLM-5.3-Flash. A more convincing negative result would require stronger uncertainty analysis and broader replication across tasks and conditions.
- The failure mechanisms are not yet convincingly established. Under-exploration, token-management failures, and convergence onto fewer skill lineages are plausible observations, but the targeted interventions do not explain the full social-versus-solo gap. In particular, the lineage analysis shows association, not that diversity loss causes lower population performance. The paper does not yet identify when these effects dominate or what conditions would reverse the outcome.
Questions for authors
- What conclusions do the authors believe are justified specifically for homogeneous agents sharing the same training examples, and which claims are intended to generalize to heterogeneous populations or different task distributions?
- Can the authors provide a power or equivalence analysis for the matched-cost comparisons, particularly the GLM-5.3-Flash result, to clarify what magnitude of social advantage the current six-seed experiments can actually exclude?
- How can the proposed failure mechanisms be causally distinguished? For example, can independent exploration or skill-lineage diversity be controlled while holding total token expenditure fixed, to determine whether lineage collapse itself explains the lack of gains?
- How robust are the findings outside instruction following? Is there evidence from a second skill-evolution benchmark, or from agents with differing initial skills, capabilities, or training data, that supports the broader conclusion?
3 comments
Response to reviewer feedback pt. 2
3. Do the proposed mechanisms explain the outcome?
Each intervention alone fixes its target problem but gives no reliable social advantage, so we ran both together for all three bandit models (eight matched seeds), within the original 4,096-token allowance: 2,048 tokens are reserved for executing a pull, and independent search is forced on 15% of agent-rounds.
| Model | Social − solo cumulative reward | Social − solo reward/round at matched tokens | Pulls completed, solo / social (%) | Arms found independently, solo / social (of 25) |
|---|---|---|---|---|
| Qwen3-14B | +2.7 ± 7.9 | −0.25 ± 0.27 | 97.14 ± 0.38 / 98.49 ± 0.23 | 22.13 ± 0.72 / 22.75 ± 0.70 |
| Ministral-3-14B | −43.9 ± 17.8 | −0.96 ± 0.16 | 96.38 ± 0.81 / 87.88 ± 1.05 | 23.13 ± 0.40 / 14.13 ± 1.62 |
| GPT-OSS-20B | −26.3 ± 24.6 | −0.65 ± 0.35 | 99.79 ± 0.06 / 99.70 ± 0.07 | 25.00 ± 0.00 / 25.00 ± 0.00 |
Paired differences ± SE over eight seeds; cumulative reward is per agent over 100 rounds. No cumulative difference is significant after Bonferroni correction; Ministral's matched-token disadvantage is significant (p < 0.01). Qwen and GPT-OSS complete nearly every pull and find nearly every arm in both conditions, yet social agents earn no more. Ministral's social agents still complete fewer pulls and find fewer arms. We will add to Section 4:
"Combining protected execution with forced independent search, applied equally to solo and social populations, gives no detectable social advantage. Qwen and GPT-OSS complete nearly all pulls and find nearly all arms in both conditions; Ministral's social agents still complete fewer pulls and find fewer arms. Repairing both failures is therefore not sufficient for these models to benefit from peers."
To test whether lineage concentration means agents use similar procedures, we measured embedding distance between skill files and how often agent pairs disagree on which held-out questions they solve.
| Model | Skill-text distance, solo | Skill-text distance, social | Social − solo success disagreement (pp) |
|---|---|---|---|
| GPT-OSS-120B | 0.285 ± 0.033 | 0.016 ± 0.016 | −0.42 ± 0.65 |
| GLM-5.3-Flash | 0.114 ± 0.004 | 0.078 ± 0.006 | −0.62 ± 0.63 |
Social agents' skill files are much more similar, especially for GPT-OSS, but which questions they solve barely changes. We will add the methods to Appendix D.1 and to the lineage discussion:
"Social populations converge onto fewer lineages and more similar skill files, but differ little from solo populations in which questions they solve. Future work can test whether preserving distinct lineages improves learning at matched token expenditure."
Thank you again. The different-initial-skill experiment, power analysis, and combined intervention let us state more precisely what our results do and do not rule out.
Thank you again for your thoughtful feedback! As the rebuttal deadline approaches, we wanted to check whether our responses and additional experiments have helped address your concerns. We would be happy to answer any remaining questions, and would be grateful if you could take these updates into account in your final assessment.
Response to reviewer feedback pt. 1
Thank you for recognizing our matched-cost evaluation and behavioral diagnostics, and for the concrete suggestions on heterogeneous agents, statistical power, and causal tests. We ran each suggested analysis; results and the corresponding paper changes follow.
1. What happens when agents start with different skills?
We ran this experiment. Each five-agent population starts with five different skill files, focused on formatting, counting limits (words, sentences, list lengths), required and forbidden content, ordering, and a final audit against every instruction. All agents share the same model and training examples. Matched solo and social populations receive the same file assignment, rotated across seeds. Each model ran for 50 rounds with six matched seeds (24 populations total), using the same 100 training and 200 held-out examples.
Means ± SE across six seeds. Matched-token columns are paired differences, interpolated at the smaller final learning-token expenditure of each pair, as in the paper. Neither difference is significant (two-sided paired t-tests, Bonferroni-corrected over two models and two metrics). Subtracting initial accuracy gives +1.56 ± 1.17 pp (GPT-OSS) and +2.12 ± 0.57 pp (GLM). Social populations also spend fewer learning tokens: 34.49 ± 1.82M vs. 50.18 ± 1.27M (GPT-OSS) and 57.89 ± 1.60M vs. 69.53 ± 2.57M (GLM).
We will add to Section 5:
And to the Discussion:
2. What can we conclude from the current matched-cost results?
We computed the smallest social advantage our six-seed experiment could reliably detect, and how much the estimate changes when test questions are also resampled (using the same questions for both conditions).
For GPT-OSS, the 90% interval rules out advantages above about 0.85 pp. For GLM, the estimate is about 1 pp, but six seeds can only reliably detect about 3 pp, so a moderate benefit remains possible. We will add this to Appendix H and replace the abstract's negative conclusion with:
We also compared accuracy at four token budgets per model that every run reached, chosen without looking at accuracy. Tokens count learning prompts and completions across all five agents.
Social − solo whole-answer accuracy (pp), mean ± seed SE. Social advantages are largest early. GPT-OSS's gap mostly closes; GLM's shrinks slowly. We will add the curves to Appendix E and to Section 5: