acceptodds
Under review as a conference paper at ICLR 2027

From Solo to Social Learning: Characterizing Recursive Social Improvement in LLMs

Abstract

Large language models (LLMs) can now improve themselves by revising the instructions they follow, and LLM agents are increasingly orchestrated to work together on complex problems. However, self-improvement methods typically optimize one system at a time, and multi-agent frameworks often have every model work toward a shared goal. We ask a different question. When each agent pursues its own reward, can self-improving LLMs learn from one another well enough to improve the whole population? We call this capability *recursive social improvement*. We study populations that revise skill files and choose whether, when, and whom to copy from. Independent search, learning from peers, and acting all share one token budget. In controlled environments, established social-learning algorithms benefit from peers, but three LLMs do not. They earn less reward per token than solo learners, and explore too narrowly or run out of tokens before acting. We then let the models write and revise their own skills. Observing peers changes how they improve, helping one model find useful skills sooner and another spend less on private search. Neither, however, outperforms independent learners at the same cost. Skills are copied, revised, and passed on, so one discovery can seed further search. Yet these exchanges concentrate the population around fewer independent discoveries. Together, these results show that LLMs can make learning more efficient by copying from peers, but not yet more effective.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multi-agent, social learning, self-improvement, skill evolution

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_55pVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies recursive social improvement in populations of self-interested LLM agents. Rather than jointly solving one task, agents pursue individual rewards while deciding whether to revise their own skills, observe and copy a peer's skill, or execute a skill under a shared token budget. The authors formalize recursive social improvement as both recursive transmission of revised skills and a positive performance advantage over independent learners at matched population-wide token cost. They first investigate social learning in controlled multi-armed bandit environments and then study editable skill files on IFBench using OpenEvolve.

Critical strengths
  1. The paper asks whether knowledge exchange among self-improving agents produces gains beyond equally resourced independent search, rather than assuming that more agents or more copying necessarily improve outcomes.
  2. Careful evaluation perspective. The matched-cost formulation and distinction between efficiency and effectiveness address a meaningful confound in multi-agent comparisons. Including algorithmic policies in the controlled environment helps establish that useful social information is available.
  3. Useful behavioral diagnostics. The execution-budget, exploration, observation-timing, and skill-lineage analyses provide insights into how copying changes agent behavior. The authors also report negative results transparently instead of overstating small numerical gains.
Critical weaknesses
  1. Limited representativeness of the main experimental setting. The central IFBench experiments use only two LLMs on one benchmark, with homogeneous agents trained on identical examples and evaluated using the same objectives. This is a legitimate controlled setting, but it does not adequately cover the settings in which social learning would be most compelling: agents with complementary knowledge, different tasks, or heterogeneous capabilities. The breadth of the conclusions is therefore not commensurate with the breadth of the evidence.
  2. Insufficient support for the central negative conclusion. The absence of a statistically significant matched-cost gain is not evidence that social improvement cannot occur or that social and solo learning are effectively equivalent. With only six population seeds in the principal skill-evolution comparisons, the uncertainty remains consequential, especially for GLM-5.3-Flash. A more convincing negative result would require stronger uncertainty analysis and broader replication across tasks and conditions.
  3. The failure mechanisms are not yet convincingly established. Under-exploration, token-management failures, and convergence onto fewer skill lineages are plausible observations, but the targeted interventions do not explain the full social-versus-solo gap. In particular, the lineage analysis shows association, not that diversity loss causes lower population performance. The paper does not yet identify when these effects dominate or what conditions would reverse the outcome.
Questions for authors
  1. What conclusions do the authors believe are justified specifically for homogeneous agents sharing the same training examples, and which claims are intended to generalize to heterogeneous populations or different task distributions?
  2. Can the authors provide a power or equivalence analysis for the matched-cost comparisons, particularly the GLM-5.3-Flash result, to clarify what magnitude of social advantage the current six-seed experiments can actually exclude?
  3. How can the proposed failure mechanisms be causally distinguished? For example, can independent exploration or skill-lineage diversity be controlled while holding total token expenditure fixed, to determine whether lineage collapse itself explains the lack of gains?
  4. How robust are the findings outside instruction following? Is there evidence from a second skill-evolution benchmark, or from agents with differing initial skills, capabilities, or training data, that supports the broader conclusion?
3 comments
AuthorsOpenReview ↗

Response to reviewer feedback pt. 1

Thank you for recognizing our matched-cost evaluation and behavioral diagnostics, and for the concrete suggestions on heterogeneous agents, statistical power, and causal tests. We ran each suggested analysis; results and the corresponding paper changes follow.

1. What happens when agents start with different skills?

We ran this experiment. Each five-agent population starts with five different skill files, focused on formatting, counting limits (words, sentences, list lengths), required and forbidden content, ordering, and a final audit against every instruction. All agents share the same model and training examples. Matched solo and social populations receive the same file assignment, rotated across seeds. Each model ran for 50 rounds with six matched seeds (24 populations total), using the same 100 training and 200 held-out examples.

ModelFinal whole-answer acc., solo (%)Final whole-answer acc., social (%)Social − solo whole-answer acc. at matched tokens (pp)Social − solo constraint acc. at matched tokens (pp)
GPT-OSS-120B67.63 ± 0.3069.93 ± 0.81+2.18 ± 0.93+2.17 ± 1.01
GLM-5.3-Flash60.57 ± 0.5762.03 ± 0.46+1.49 ± 0.65+1.53 ± 0.70

Means ± SE across six seeds. Matched-token columns are paired differences, interpolated at the smaller final learning-token expenditure of each pair, as in the paper. Neither difference is significant (two-sided paired t-tests, Bonferroni-corrected over two models and two metrics). Subtracting initial accuracy gives +1.56 ± 1.17 pp (GPT-OSS) and +2.12 ± 0.57 pp (GLM). Social populations also spend fewer learning tokens: 34.49 ± 1.82M vs. 50.18 ± 1.27M (GPT-OSS) and 57.89 ± 1.60M vs. 69.53 ± 2.57M (GLM).

We will add to Section 5:

"With five different initial procedures, social-minus-solo whole-answer accuracy at matched learning tokens is +2.18 ± 0.93 pp for GPT-OSS and +1.49 ± 0.65 pp for GLM (six seeds each). Both favor social learning but are not significant."

And to the Discussion:

"All agents share a model and training examples, including in the different-initial-skill populations. Agents with different models or training data may have more to learn from one another, but may also find each other's procedures harder to use."

2. What can we conclude from the current matched-cost results?

We computed the smallest social advantage our six-seed experiment could reliably detect, and how much the estimate changes when test questions are also resampled (using the same questions for both conditions).

ModelSocial − solo at matched tokens (pp)90% paired interval (pp)Detectable with 80% power (pp)SE incl. question resampling (pp)95% bootstrap interval (pp)
GPT-OSS-120B+0.14 ± 0.35[−0.56, +0.85]1.220.82[−1.50, +1.71]
GLM-5.3-Flash+0.98 ± 0.85[−0.73, +2.69]2.981.12[−1.27, +3.10]

For GPT-OSS, the 90% interval rules out advantages above about 0.85 pp. For GLM, the estimate is about 1 pp, but six seeds can only reliably detect about 3 pp, so a moderate benefit remains possible. We will add this to Appendix H and replace the abstract's negative conclusion with:

"It remains uncertain whether observing peers improves accuracy at matched token expenditure."

We also compared accuracy at four token budgets per model that every run reached, chosen without looking at accuracy. Tokens count learning prompts and completions across all five agents.

ModelBudget 1Budget 2Budget 3Budget 4
GPT-OSS-120B9.61M: +1.48 ± 0.2516.37M: +0.56 ± 0.5023.14M: +0.19 ± 0.4729.90M: +0.07 ± 0.35
GLM-5.3-Flash19.21M: +1.81 ± 0.3134.43M: +1.63 ± 0.3649.65M: +1.42 ± 0.4964.88M: +1.18 ± 0.72

Social − solo whole-answer accuracy (pp), mean ± seed SE. Social advantages are largest early. GPT-OSS's gap mostly closes; GLM's shrinks slowly. We will add the curves to Appendix E and to Section 5:

"Social advantages at matched tokens are largest early in learning. Solo GPT-OSS agents catch up, while GLM's advantage shrinks more slowly, suggesting that copying speeds access to useful skills even when a lasting advantage is uncertain."

AuthorsOpenReview ↗

Response to reviewer feedback pt. 2

3. Do the proposed mechanisms explain the outcome?

Each intervention alone fixes its target problem but gives no reliable social advantage, so we ran both together for all three bandit models (eight matched seeds), within the original 4,096-token allowance: 2,048 tokens are reserved for executing a pull, and independent search is forced on 15% of agent-rounds.

ModelSocial − solo cumulative rewardSocial − solo reward/round at matched tokensPulls completed, solo / social (%)Arms found independently, solo / social (of 25)
Qwen3-14B+2.7 ± 7.9−0.25 ± 0.2797.14 ± 0.38 / 98.49 ± 0.2322.13 ± 0.72 / 22.75 ± 0.70
Ministral-3-14B−43.9 ± 17.8−0.96 ± 0.1696.38 ± 0.81 / 87.88 ± 1.0523.13 ± 0.40 / 14.13 ± 1.62
GPT-OSS-20B−26.3 ± 24.6−0.65 ± 0.3599.79 ± 0.06 / 99.70 ± 0.0725.00 ± 0.00 / 25.00 ± 0.00

Paired differences ± SE over eight seeds; cumulative reward is per agent over 100 rounds. No cumulative difference is significant after Bonferroni correction; Ministral's matched-token disadvantage is significant (p < 0.01). Qwen and GPT-OSS complete nearly every pull and find nearly every arm in both conditions, yet social agents earn no more. Ministral's social agents still complete fewer pulls and find fewer arms. We will add to Section 4:

"Combining protected execution with forced independent search, applied equally to solo and social populations, gives no detectable social advantage. Qwen and GPT-OSS complete nearly all pulls and find nearly all arms in both conditions; Ministral's social agents still complete fewer pulls and find fewer arms. Repairing both failures is therefore not sufficient for these models to benefit from peers."

To test whether lineage concentration means agents use similar procedures, we measured embedding distance between skill files and how often agent pairs disagree on which held-out questions they solve.

ModelSkill-text distance, soloSkill-text distance, socialSocial − solo success disagreement (pp)
GPT-OSS-120B0.285 ± 0.0330.016 ± 0.016−0.42 ± 0.65
GLM-5.3-Flash0.114 ± 0.0040.078 ± 0.006−0.62 ± 0.63

Social agents' skill files are much more similar, especially for GPT-OSS, but which questions they solve barely changes. We will add the methods to Appendix D.1 and to the lineage discussion:

"Social populations converge onto fewer lineages and more similar skill files, but differ little from solo populations in which questions they solve. Future work can test whether preserving distinct lineages improves learning at matched token expenditure."

Thank you again. The different-initial-skill experiment, power analysis, and combined intervention let us state more precisely what our results do and do not rule out.

Thank you again for your thoughtful feedback! As the rebuttal deadline approaches, we wanted to check whether our responses and additional experiments have helped address your concerns. We would be happy to answer any remaining questions, and would be grateful if you could take these updates into account in your final assessment.

Reviewer_ZbNnOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether a population of independent LLM agents can improve by learning and conducting new kinds of investigations into one another’s procedures. Each agent can conduct a private search where they observe another agent’s skill or reuse what is already known. Self improvement of the agent is conducted by revising skill files at inference time. The paper covers three improvements. In recursive transmission, a skill is revised, copied by another agent, and revised again. This helps with efficiency where the population obtains better performance per token. And the paper measures the matched cost where a social population performs better than an equally sized population of independent earners. The paper uses GPT-OSS and GLM-5.3 in a 25-armed bandit setting with ten agents, 100 rounds, and a budget of 4096 tokens. They find the GPT-OSS benefits from reduced spending while GLM shows an earlier accuracy advantage.

Critical strengths
  • Presentation: I think the paper is very readable and easy to follow. Figure 1 also reads quite strongly. I like the writing in this paper.
  • Meaningful research question: This is an interesting setting for self-improvement and the paper also investigates the comparison thoroughly to address sources of misleading gains. For example, they count work by all agents to understand and compare equally sized social and solo populations. This makes the question more informative than asking whether an agent improves after receiving a useful artifact which I appreciate.
  • Controlled experiments: The social learner demonstrates that the environment contains exploitable peer information. Without this control, failures in the LLMs could reflect an environment in which social learning has no opportunity.
  • Statistical reliability: Very careful statistical reporting which has been rare in other self-improvement papers.
Critical weaknesses
  • Uneven improvements: Unfortunately, the improvement given by social learning is mixed. The conclusion supports the idea that there is no reliable advantage. The primary matched-cost skill results are positive but uncertain. I think the appendix recognizes this limitation but the conclusion I see is that social policies do not demonstrate a reliable accuracy advantage over their solo counterparts. The current writing makes this a blanket statement on no improvement but I think this has to be more carefully described because improvements are observed with GLM. Could the authors characterize what the takeaway of these uneven results should be?
  • Matched-cost comparison: One choice that the authors make is to choose the smaller of the social and solo runs’ final expenditures. The paper interpolates performance at that expenditure. This is weaker than evaluating both policies at budgets fixed in advance. If the claim could be supported by a set of small predefined token budgets, that would better support the paper.
  • Causal explanation: The paper identifies causal explanations of results but experiments do not establish that the problems explain the outcomes. For example, protected execution and independent search are tested separately. But exploration and execution interact? Fixing either one alone is likely insufficient. Furthermore, forced search can worse execution completion.
  • (Nit) Negative descriptions: the stronger negative wording in the abstract and diffusion turns an inconclusive estimate into an apparent absence of benefit. I think this should be tweaked.
Questions for authors
  • What distinguishes recursive improvement from communicating a good discovery? Could I just copy without any revision or show held-out gains from a copied parent skill?
  • Could cost reduction just come from sharing useful knowledge vs simply performing fewer evals?
4 comments
AuthorsOpenReview ↗

Response to reviewer feedback pt. 1

Thank you for recognizing the clarity of the paper, the population-wide cost accounting, and the careful statistical reporting. We ran the experiments you suggested; results and the corresponding paper changes follow.

1. How should we interpret the uneven social-learning gains?

We will describe the matched-cost results as inconclusive rather than negative. To quantify this, we computed the smallest social advantage our six-seed experiment could reliably detect, and how much the estimate changes when test questions are also resampled (using the same questions for both conditions).

ModelSocial − solo at matched tokens (pp)90% paired interval (pp)Detectable with 80% power (pp)SE incl. question resampling (pp)95% bootstrap interval (pp)
GPT-OSS-120B+0.14 ± 0.35[−0.56, +0.85]1.220.82[−1.50, +1.71]
GLM-5.3-Flash+0.98 ± 0.85[−0.73, +2.69]2.981.12[−1.27, +3.10]

For GPT-OSS, the 90% interval rules out advantages above about 0.85 pp. For GLM, the estimate is about 1 pp, but six seeds can only reliably detect about 3 pp, so a moderate benefit remains possible. We will add this to Appendix H and replace the abstract's negative conclusion with:

"It remains uncertain whether observing peers improves accuracy at matched token expenditure."

2. Does the result depend on comparing agents at their final expenditure?

Using the saved learning curves, we compared accuracy at four token budgets per model that every run reached, chosen without looking at accuracy. Tokens count learning prompts and completions across all five agents, excluding held-out evaluation.

ModelBudget 1Budget 2Budget 3Budget 4
GPT-OSS-120B9.61M: +1.48 ± 0.2516.37M: +0.56 ± 0.5023.14M: +0.19 ± 0.4729.90M: +0.07 ± 0.35
GLM-5.3-Flash19.21M: +1.81 ± 0.3134.43M: +1.63 ± 0.3649.65M: +1.42 ± 0.4964.88M: +1.18 ± 0.72

Social − solo whole-answer accuracy (pp), mean ± seed SE. Social advantages are largest early. GPT-OSS's gap mostly closes; GLM's shrinks slowly. We will add the curves to Appendix E and to Section 5:

"Social advantages at matched tokens are largest early in learning. Solo GPT-OSS agents catch up, while GLM's advantage shrinks more slowly, suggesting that copying speeds access to useful skills even when a lasting advantage is uncertain."

3. Could exploration and execution failures interact?

We ran both interventions together for all three bandit models (eight matched seeds), within the original 4,096-token allowance: 2,048 tokens are reserved for executing a pull, and independent search is forced on 15% of agent-rounds. The reserved tokens keep forced search from crowding out pulls.

ModelSocial − solo cumulative rewardSocial − solo reward/round at matched tokensPulls completed, solo / social (%)Arms found independently, solo / social (of 25)
Qwen3-14B+2.7 ± 7.9−0.25 ± 0.2797.14 ± 0.38 / 98.49 ± 0.2322.13 ± 0.72 / 22.75 ± 0.70
Ministral-3-14B−43.9 ± 17.8−0.96 ± 0.1696.38 ± 0.81 / 87.88 ± 1.0523.13 ± 0.40 / 14.13 ± 1.62
GPT-OSS-20B−26.3 ± 24.6−0.65 ± 0.3599.79 ± 0.06 / 99.70 ± 0.0725.00 ± 0.00 / 25.00 ± 0.00

Paired differences ± SE over eight seeds; cumulative reward is per agent over 100 rounds. No cumulative difference is significant after Bonferroni correction; Ministral's matched-token disadvantage is significant (p < 0.01). Qwen and GPT-OSS complete nearly every pull and find nearly every arm in both conditions, yet social agents earn no more. Ministral's social agents still complete fewer pulls and find fewer arms. We will add to Section 4:

"Combining protected execution with forced independent search, applied equally to solo and social populations, gives no detectable social advantage. Qwen and GPT-OSS complete nearly all pulls and find nearly all arms in both conditions; Ministral's social agents still complete fewer pulls and find fewer arms. Repairing both failures is therefore not sufficient for these models to benefit from peers."

AuthorsOpenReview ↗

Response to reviewer feedback pt. 2

4. What distinguishes recursive improvement from sharing a good discovery?

Copying spreads a skill unchanged; recursive transmission means the recipient then revises it. To measure whether those revisions help, we took every revision of a copied skill that the recipient went on to deploy, and evaluated it and its copied parent on the same 200 held-out examples with the recipient's model and identical sampling settings.

ModelParent–revision pairsRevised − copied whole-answer acc. (pp)Revised − copied constraint acc. (pp)
GPT-OSS-120B24+0.53 ± 1.02+0.24 ± 1.04
GLM-5.3-Flash22+0.30 ± 0.78+0.36 ± 0.75

Differences are averaged within each population, then mean ± SE across six seeds. Revisions score slightly higher than their copied parents on average, but neither difference is significant. Agents do build on copied skills, but we cannot show that these revisions improve performance. We will add the methods to Appendix D.1 and to Section 3:

"Copying spreads a skill unchanged; recursive transmission occurs when the recipient revises it. Deployed revisions change whole-answer accuracy relative to their copied parents by +0.53 ± 1.02 pp for GPT-OSS and +0.30 ± 0.78 pp for GLM (six seeds), so recursive transmission occurs but its performance benefit is not established."

5. Does copying save computation mainly by avoiding evaluations?

Yes. Evaluating candidate skills dominates private learning cost, and observing a peer instead of revising skips some of those evaluations. Deciding whom to copy and which skill to deploy costs far less than evaluating a revision on 100 training examples. We will add to Section 5:

"Most private-learning computation is spent evaluating candidate skills, and copying saves tokens mainly by avoiding some of these evaluations. We therefore also compare accuracy at equal total learning expenditure."

Thank you again. Your questions led us to compare agents at fixed token budgets, combine the two interventions, and test whether revisions of copied skills improve on their parents, which lets us describe both the positive and the null findings more accurately.

I thank the authors for their response. This has clarified many things and removed many concerns. I think the core issue remaining is that how to interpret this paper and whether the inconclusive results are not due to a factor of implementation/model choices.

For example, could the social learning gains be weak because the models authors can run are weak? I would appreciate a discussion in the paper and from the authors as well as maybe one or two experiments with a stronger model if the authors have the capacity to load a larger model. After seeing this discussion, I think the contribution of the paper will be cleaner and I will raise my score.

AuthorsOpenReview ↗

Follow-up to reviewer feedback

Follow-up: Could the findings reflect model size or capability?

Thank you for this suggestion. We agree that the paper should address whether our results depend on the models we tested. To check this, we repeated the bandit experiment with a stronger model, GPT-6.1 Sol, run at low reasoning effort.

Everything else matches the original bandit experiment. Each population has ten agents choosing among 25 arms for 100 rounds. Each agent has 4,096 completion tokens per round, and unused tokens expire. We used the same prompts and did not add the exploration or execution interventions. In the social condition, agents could observe the arms their peers pulled in the previous round and the payoffs they received. Because of cost, we ran four environment seeds per condition, for eight populations in total.

MeasureSoloSocialSocial − Solo
Cumulative reward per agent over 100 rounds362.8 ± 70.7412.0 ± 71.7+49.3 ± 1.5
Reward per round at matched total tokens4.57 ± 0.773.39 ± 0.80−1.18 ± 0.43
Total tokens per population (millions)0.82 ± 0.013.19 ± 0.68+2.37 ± 0.68

Each value is the mean across four populations, with its standard error. For each population, we first average over its ten agents. The last column compares solo and social populations that share the same environment seed. The matched-token row follows the method in the paper. For each pair, we take the smaller of the two populations' total token use, which counts both prompt and completion tokens. We then compare the two populations' reward per round at that level of spending. A missed pull earns zero reward.

With access to peers, GPT-6.1 Sol earned more reward in all four seeds. Over 100 rounds, the social advantage is 49.3 ± 1.5 reward per agent (p < 0.001, two-sided paired t-test, Bonferroni-corrected across the two reward comparisons). Both conditions completed more than 99% of their pulls, so the advantage does not come from social agents completing more actions. This result matters for interpreting the paper: with a more capable model, the same setup and prompts produce a clear social advantage.

This advantage came at a much higher token cost. Social populations used about 3.9 times as many tokens as solo populations, mostly because reading peers' choices adds input tokens. When we compare the two conditions at equal token spending, the social estimate is lower than the solo estimate, but the difference is not significant after correction.

To answer your question about interpretation: the stronger model benefits from social learning when the number of rounds is held fixed, but we do not find a benefit when total computation is held fixed. Whether social learning helps therefore depends both on the model and on which resource is held fixed. We will add this experiment to Section 4 and the appendix, and add the following paragraph to the Discussion:

"Model capability can change the outcome of social learning. In an additional bandit experiment, social access raises GPT-6.1 Sol's cumulative reward by 49.3 ± 1.5 per agent over 100 rounds. Because the social agents also use more tokens, we do not find an advantage at matched computation. Our results with smaller models should therefore not be read as a limit on what stronger agents can achieve. Because of cost, we have not repeated the full skill-evolution experiment with this model. Stronger models might write more useful revisions, judge their peers' procedures more accurately, or spend fewer tokens deciding what to copy. They might also improve faster on their own, which would leave less room for a social advantage. Testing these possibilities is an important step toward understanding whether our findings hold as models become more capable."

Reviewer_TLa3OpenReview ↗
Rating
2
Confidence
2(−2)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies recursive social improvement in populations of self-improving LLM agents, examining whether agents can benefit from observing, copying, and further revising each other's skills. The authors formulate a social learning setting where each agent independently optimizes its own task rewards while deciding how to allocate a shared token budget between private learning, peer observation, and task execution. The study first uses controlled multi-armed bandit environments to compare social and independent learning strategies, then extends the evaluation to natural-language skill evolution on IFBench using OpenEvolve. Experiments across several LLM backbones investigate learning efficiency and performance. The results show that peer observation can reduce learning costs or accelerate early improvement in some settings, but the tested LLM-controlled social populations do not demonstrate a statistically significant performance advantage over independent learners at matched computational costs.

Critical strengths
  1. For this paper, I like the paper explores an interesting and relevant question about whether self-improving LLM agents can benefit from sharing and building on each other's discoveries. The distinction between learning efficiency and effectiveness under matched computational budgets is particularly useful for evaluating the actual benefits of social learning.

  2. The paper provides a fairly detailed empirical analysis, covering controlled bandit environments and several intervention studies. I particularly like the investigations into exploration behavior, token consumption and skill propagation, which offer some useful insights into why social learning does not always lead to better collective performance.

Critical weaknesses
  1. For this paper, I found the overall methodological and analytical depth of this work rather limited, despite the interesting motivation. The framework mainly relies on existing skill evolution methods, simple peer copying, and standard learning strategies, without introducing particularly novel mechanisms for studying recursive social improvement. I was also hoping for a deeper investigation of how cross-agent skill revisions actually contribute to collective learning.

  2. Besides, I found the experimental setting seems too restricted to support broader conclusions about social learning in LLM agents. The main skill evolution experiments only use IFBench with homogeneous agents, which does not capture the diversity or complementary expertise that often motivates multi-agent learning. The improvements at matched computational cost are also small and statistically inconclusive.

Questions for authors
  1. Whether you have directly examined the benefits of recursively revising skills across different agents, rather than simply tracking their transmission histories?

  2. How would the conclusions change if agents had different capabilities, specialized skills, or access to different training experiences?

  3. I am not fully convinced that shared ancestry necessarily implies reduced functional diversity. Have you considered measuring the semantic or behavioral diversity of evolved skills, and examining how these differences relate to actual task performance?

  4. Also I noticed that some experimental settings show positive signals for social learning, while the main matched-cost comparisons remain statistically inconclusive, so could you provide additional experiments across more realistic agentic tasks.

4 comments
AuthorsOpenReview ↗

Response to reviewer feedback pt.1

Thank you for recognizing the distinction between learning efficiency and effectiveness under matched budgets, and our analyses of exploration, token use, and skill propagation. We ran the experiments you suggested; results and the corresponding paper changes follow.

1. Do agents benefit from revising skills they acquire from others?

Our lineage analysis shows that agents revise skills they copy, but not whether those revisions help. To measure this, we took every revision of a copied skill that the recipient went on to deploy, and evaluated it and its copied parent on the same 200 held-out examples with the recipient's model and identical sampling settings.

ModelParent–revision pairsRevised − copied whole-answer acc. (pp)Revised − copied constraint acc. (pp)
GPT-OSS-120B24+0.53 ± 1.02+0.24 ± 1.04
GLM-5.3-Flash22+0.30 ± 0.78+0.36 ± 0.75

Differences are averaged within each population, then mean ± SE across six seeds. Revisions score slightly higher than their copied parents on average, but neither difference is significant. Agents do build on copied skills, but we cannot show that these revisions improve performance. We will add the methods to Appendix D.1 and to Section 5:

"Deployed revisions change whole-answer accuracy relative to their copied parents by +0.53 ± 1.02 pp for GPT-OSS and +0.30 ± 0.78 pp for GLM (six seeds), so recursive transmission occurs but its performance benefit is not established."

On novelty: we deliberately use existing skill evolution (OpenEvolve) and simple copying, so differences between conditions come from how agents split computation between private search and observing peers, not from a new search method. We will state this in the Introduction:

"We study whether agents can use one another's evolving procedures to learn more effectively than equally resourced independent learners, tracing how skills are copied and revised and comparing social and independent learning at matched population-wide token expenditure."

2. How would the conclusions change with different skills, capabilities, or training experiences?

We ran a new experiment varying initial skills. Each five-agent population starts with five different skill files, focused on formatting, counting limits (words, sentences, list lengths), required and forbidden content, ordering, and a final audit against every instruction. All agents share the same model and training examples. Matched solo and social populations receive the same file assignment, rotated across seeds. Each model ran for 50 rounds with six matched seeds (24 populations total), using the same 100 training and 200 held-out examples.

ModelFinal whole-answer acc., solo (%)Final whole-answer acc., social (%)Social − solo whole-answer acc. at matched tokens (pp)Social − solo constraint acc. at matched tokens (pp)
GPT-OSS-120B67.63 ± 0.3069.93 ± 0.81+2.18 ± 0.93+2.17 ± 1.01
GLM-5.3-Flash60.57 ± 0.5762.03 ± 0.46+1.49 ± 0.65+1.53 ± 0.70

Means ± SE across six seeds. Matched-token columns are paired differences, interpolated at the smaller final learning-token expenditure of each pair, as in the paper. Neither difference is significant (two-sided paired t-tests, Bonferroni-corrected over two models and two metrics). Subtracting initial accuracy gives +1.56 ± 1.17 pp (GPT-OSS) and +2.12 ± 0.57 pp (GLM). Social populations also spend fewer learning tokens: 34.49 ± 1.82M vs. 50.18 ± 1.27M (GPT-OSS) and 57.89 ± 1.60M vs. 69.53 ± 2.57M (GLM).

We will add to Section 5:

"With five different initial procedures, social-minus-solo whole-answer accuracy at matched learning tokens is +2.18 ± 0.93 pp for GPT-OSS and +1.49 ± 0.65 pp for GLM (six seeds each). Both favor social learning but are not significant."

We did not vary model capability or training data, and will state this in the Discussion:

"All agents share a model and training examples, including in the different-initial-skill populations. Agents with different models or training data may have more to learn from one another, but may also find each other's procedures harder to use."

AuthorsOpenReview ↗

Response to reviewer feedback pt. 2

3. Does shared ancestry imply less functional diversity?

We measured both: embedding distance between skill files (text diversity), and how often agent pairs disagree on which held-out questions they solve (behavioral diversity).

ModelSkill-text distance, soloSkill-text distance, socialSocial − solo success disagreement (pp)
GPT-OSS-120B0.285 ± 0.0330.016 ± 0.016−0.42 ± 0.65
GLM-5.3-Flash0.114 ± 0.0040.078 ± 0.006−0.62 ± 0.63

Social agents' skill files are much more similar, especially for GPT-OSS, but which questions they solve barely changes. Shared ancestry therefore reduces text diversity without a detectable loss of behavioral diversity. We will add the methods to Appendix D.1 and to the lineage discussion:

"Social populations converge onto fewer lineages and more similar skill files, but differ little from solo populations in which questions they solve. Future work can test whether preserving distinct lineages improves learning at matched token expenditure."

4. How should we interpret the positive but uncertain results?

We will describe these results as inconclusive rather than negative. Using the saved learning curves, we compared accuracy at four token budgets per model that every run reached, chosen without looking at accuracy. Tokens count learning prompts and completions across all five agents.

ModelBudget 1Budget 2Budget 3Budget 4
GPT-OSS-120B9.61M: +1.48 ± 0.2516.37M: +0.56 ± 0.5023.14M: +0.19 ± 0.4729.90M: +0.07 ± 0.35
GLM-5.3-Flash19.21M: +1.81 ± 0.3134.43M: +1.63 ± 0.3649.65M: +1.42 ± 0.4964.88M: +1.18 ± 0.72

Social − solo whole-answer accuracy (pp), mean ± seed SE. Social advantages are largest early. GPT-OSS's gap mostly closes; GLM's shrinks slowly. We will add the curves to Appendix E and to Section 5:

"Social advantages at matched tokens are largest early in learning. Solo GPT-OSS agents catch up, while GLM's advantage shrinks more slowly, suggesting that copying speeds access to useful skills even when a lasting advantage is uncertain."

We will also replace the abstract's negative conclusion with:

"It remains uncertain whether observing peers improves accuracy at matched token expenditure."

On additional tasks: the different-initial-skill experiment broadens the conditions tested within IFBench, but we use one skill-evolution benchmark. We will list this as a limitation in the Discussion and not claim our findings generalize to agentic tasks broadly.

Thank you again. Your questions led us to measure what revising a copied skill adds, how different starting skills change the comparison, and whether shared ancestry reduces behavioral diversity.

Dear Author,

Thank you very much for your response and additional results. I think the new results are useful, like the analysis on skill revisions and behavioral diversity. However, I am still not very convinced about the really benefit of recursive skill improvement. The results show that agents can revise skills copied from others, but these revisions do not bring significant improvements. So I feel the benefit may come from sharing existing good skills, because this is a bit different from what the paper aims to study. And, I think the experiments are still need to be improved. Because the evaluation is still only on IFBench. It would be more convincing to see whether the findings hold with different agent capabilities or more diverse tasks. So I appreciate you efforts and the new experiments. These helped clarify some of my questions, but I still feel the main contribution and the experiments are not strong enough to fully address my concerns, but I will reduce my confidence score to 2.

AuthorsOpenReview ↗

Follow-up to Reviewer Feedback

Thanks so much for engaging with us on this, and for the suggestions that led to the new analyses. We want to clarify that the aim of our paper is not to propose a technique or prove that LLMs are already capable of recursive social improvement. Rather, our results show that LLMs can be surprisingly bad at benefiting from social information in a multi-agent setting, even underperforming simple algorithms. In the bandit environment, an algorithmic social learner (hierarchical UCB) gains from observing its peers, while none of the three LLMs do (Figure 2), and repairing their exploration and execution problems does not close this gap (Figure 4). In skill evolution, simple algorithmic rules for when to copy reach accuracy similar to independent search while using 76 to 78% fewer learning tokens (Figure 7).

This pattern holds across the four settings we study, which range from simple choices to open-ended skill revision: a bandit with a fixed set of arms (Section 4), a bandit with effectively infinite arms (Appendix B), selection among a fixed set of skill files for a scheduling task (Appendix C), and full skill evolution on IFBench (Section 5). In each setting, access to peers changes how LLM agents learn, for example by reducing independent discovery (Figure 3b, Appendix Figure 22b), but it does not give a reliable advantage over independent learners with the same resources. In the fixed-skill setting, for instance, most of the social advantage disappears once solo agents can access the same skills (Appendix Figure 29).

So the new results, which also show that LLMs struggle to improve the skills they copy from peers, are consistent with the overall message of the paper, that more work is needed to enable LLMs to effectively learn in social settings. We agree with the reviewer that in our experiments, the benefit of copying appears to come mainly from sharing existing good skills. We see this as one of our findings, and we will state it directly in Section 5 and the Discussion.

On different agent capabilities, our response to Reviewer ZbNn reports an additional bandit experiment with a stronger model, GPT-6.1 Sol. Social access improves its cumulative reward over 100 rounds, although not at matched computation. This suggests that stronger models may close part of the gap we identify, and we will add this result and discussion to the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.