Competition Discovers, Cooperation Consolidates: Social Rewards Shape Self-Evolving LLM Agents

Self-evolving LLM agents improve by converting interaction traces into persistent external memory. Yet while substantial effort has been devoted to refining downstream memory updaters, the upstream social environment that generates and filters these trajectories remains largely unexamined. In multi-agent self-evolution, social reward structures govern the evolutionary loop at two foundational stages: during interaction, process rewards dictate how agents search, communicate, and commit; after interaction, social credit assigns marginal value to outcomes, selecting which experiences are retained in persistent memory. To systematically explore this mechanism, we investigate two canonical social regimes: competition driven by relative individual credit, and cooperation driven by shared team outcomes. Our results reveal a fundamental division of labor: competitive pressure penalizes hesitation and incentivizes divergent exploration, accelerating environment-verified discovery under finite budgets and yielding decisive heuristics that elevate downstream speed-sensitive task completion. Cooperation instead aligns agents toward shared objectives and favors partner-legible proceduralization, producing structured routines that prove essential when downstream deployment requires inheriting complex collaborative conventions. Rather than revealing a universally superior paradigm, these findings show that competitive environments drive expansive frontier discovery, whereas cooperative environments synthesize inheritable order. Overall, social reward structures thus act not as incidental scaffolds, but as primary evolutionary drivers that determine both what multi-agent systems can discover and what persistent capabilities they ultimately inherit.