acceptodds
Withdrawn from ICLR 2027

ARTIFICIAL SELF-DOMESTICATION: HOW SOCIAL SE- LECTION SHAPES PROSOCIAL AGENT POPULATIONS

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

We study Artificial Self-Domestication (ASD): whether autonomous agent populations can reduce socially costly behavior through endogenous social dynamics, without externally imposed norms. We formalize ASD and introduce a Self-Domestication Index (SDI), evaluating it in multi-agent learning populations and LLM-agent societies. We find that ASD does not emerge from random initialization: populations converge to universal predation or, when encounters can be declined, to universal refusal. However, prosocial behavior becomes stable once sufficiently established, revealing ASD as a bistable basin-of-attraction problem. Seeding prosocial and discriminating agents produces a critical-mass transition. Partner choice is necessary, while punishment and social learning progressively lower the critical mass, with social learning additionally converting non-prosocial agents. ASD also requires agents to represent their own social standing; outcome-only learning fails to produce domestication. In LLM societies, prosocial behavior is sustained under reputation but collapses under anonymity, and agents consistently fail to exercise partner refusal even against partners with highly negative histories. These results show that social structure can sustain and propagate prosocial behavior but does not create it from scratch, identifying partner exclusion—not information alone—as a key mechanism making prosociality viable.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Artificial Self-Domestication, Multi-Agent Systems, LLM Agents, Social Dynamics, Prosocial Behavior, Partner Choice, Reputation, Social Learning, Punishment, Cooperation, Emergent Behavior, Critical Mass, Bistability, Social Evolution, AI Societies, Multi-Agent Learning, Agent Alignment, Social Norms, Artificial Intelligence

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_zFkHOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes the Self-Domestication (SSD) framework for multi-agent LLM societies to evolve social norms endogenously. It identifies alignment faking: agents cooperate when observed but defect privately, showing public behaviours are poor proxies for intrinsic values. The validation includes theoretical analysis, simulated agent experiments with ablation/adversarial tests, and contextual LLM experiments. Key findings include the crowding-out effect of external enforcement on intrinsic cooperation.

Critical strengths

1) The research direction is valuable: the distinction between superficial cooperation and internalization of genuine behaviors within large-model multi-agent societies constitutes a topic worthy of attention for AI alignment and multi-agent safety.

2) The work constructs an SSD (Social Self-Domestication) simulation environment, conducts a suite of ablation studies, parameter sweeps, and adversarial-invasion control experiments, demonstrating a solid experimental workload.

Critical weaknesses
  1. The manuscript suffers from excessive concept stacking, with dense jargon borrowed from evolutionary game theory and sociology. These concepts are loosely connected and lack concise explanatory text in the main body. Existing literature and components are frequently enumerated, yet the authors provide little critical judgment or selection rationale of their own.

  2. The theoretical chapter (Chapter 4) has a confusing structure. Theorem 1 and Theorem 2 are corollaries of known results in evolutionary game theory and only hold under specific simplified settings. Nevertheless, they are placed at the beginning of the theoretical exposition, while their critical assumptions are buried in the appendix. Proposition 5, the paper’s core claim regarding observability conditions, has unclear connections to the preceding two theorems and lacks the necessary derivations to establish such links.

Overall, this work applies evolutionary game ideas to study the social evolution of agents. The research direction is promising, yet the manuscript suffers from poor readability and reads like a rigid patchwork of disparate studies. For instance, the Related Work section alone covers seven subtopics without conveying the authors’ own perspectives or reflections.

Questions for authors
  1. Does masking itself cause the reported improvement? Appendix C states that masked distillation both removes recording labels and explicitly instructs agents not to condition their commitments on whether they are observed. What happens when only the labels are removed, without the additional instruction? This targeted control is necessary to distinguish the effect of withholding observability information from compliance with an explicit behavioral constraint. The disappearance of conditional commitments alone cannot distinguish these explanations.

  2. Does endogenous norm formation outperform a matched fixed-norm baseline? The fixed-constitution baseline disables internalisation, whereas SSD enables it. What evidence separates the benefit of endogenous norm evolution from the benefit of Phase 5? Is there a fixed-constitution condition with the same internalisation procedure? Also, if the fixed baseline already contains the full norm corpus, which additional norms can SSD generate that this baseline cannot express?

  3. What assumptions delimit Proposition 5 and Corollary 6? Does the hidden-branch result require independently parameterised public and hidden policies with no transfer between them? For a shared-parameter policy, conditioning on observability does not necessarily prevent public examples from updating hidden behavior. Likewise, why should withholding the cue guarantee that an update applies to the whole policy? Please distinguish conclusions established by the specified update rule from claims intended to hold for general LLM policies.

  4. What evidence establishes internalisation in the LLM experiments? Were agents evaluated with learned commitments retained but reputational rewards, discovery risk, and sanctions removed? If not, how is internalisation distinguished from residual deterrence and context-dependent compliance? Given that masked distillation raises private cooperation only to 0.117 while public cooperation remains 0.927, what operational criterion supports describing this result as restored internalisation rather than partial mitigation?

Reviewer_v6HiOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies whether a population of language-model agents can develop cooperative behavior without an externally defined reward model or fixed set of rules

Critical strengths

The original proposed question is interesting.

Critical weaknesses

This paper is very low quality. I can tell this paper was directly generated by AI. The author did not respect reviewers' time and effort for reviewing the paper, and this kind of low quality work should not be submitted! The most funny thing is that on every page it writes "Published as a conference paper at ICLR 2025". The experiments are very limited, only one model is used. The paper is obviously in an unfinished mode. Many cross-reference left with ?? in the paper.

Questions for authors

Could you take the paper submission more seriously than submitting your own student project? Could you please not waste reviewers' time on reading this kind of low quality AI generated stuff?

Reviewer_3UxpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether societies of LLM agents can develop cooperative behavior through social learning without external reward models. The authors introduce SSD, a framework involving reputation, peer evaluation, punishment, norm evolution, behavioral distillation, and generational turnover. The paper distinguishes cooperation under observation from cooperation when agents are unobserved. Through agent-based simulations and experiments with Claude agents, the authors find that agents can exhibit high levels of public cooperation while continuing to defect in private. They further show that masking observability information during distillation can reduce this gap. The paper also examines the effects of enforcement strength and adversarial agents.

Critical strengths

The distinction between observable cooperation and internalized cooperation is interesting and relevant to multi-agent alignment. The paper raises an important question about whether social learning produces genuine behavioral changes or merely encourages agents to behave cooperatively when monitored.

The combination of theoretical analysis, agent-based simulations, and LLM experiments is also a strength. The results provide some initial evidence that observability plays an important role in how agents learn social norms.

Critical weaknesses

First of all, most of the theoretical analysis and experimental results rely on a simplified agent-based model with predefined behavioral dispositions and learning rules. In the LLM experiments, internalization is implemented by adding self-generated commitments to the system prompt rather than updating model parameters. These are substantially different learning mechanisms. It is unclear whether the conclusions about internalized cooperation in the mathematical model extend to LLM agents. The current experiments primarily demonstrate changes in prompted behavior rather than changes in learned dispositions.

In addition, the LLM evaluation uses only one model, eight agents, ten interaction rounds, and four runs per condition. The public-private cooperation gap is striking, but the evidence is limited to a small and highly controlled setting. The observability intervention also introduces a confound. According to Appendix C, the masked condition not only removes observability labels but also explicitly instructs agents not to condition their commitments on whether actions are observed. The improvement therefore cannot be attributed solely to masking observability information. Furthermore, private cooperation remains low even after the intervention, increasing only from 0% to 11.7%.

Questions for authors

1.How do the authors justify using prompt-level commitments as a proxy for norm internalization? Can the reported effects be distinguished from ordinary instruction following?

2.Does the public-private cooperation gap generalize to other LLMs, larger populations, or longer interactions? The current LLM experiments use only Claude Sonnet 4.6 with four runs per condition.

3.Have the authors tested observability masking without the additional instruction prohibiting observation-dependent commitments? The current intervention changes both the information available to agents and the constraints on their generated commitments.

4.Why does private cooperation remain at only 11.7% after masking? To what extent does this result support the claim that the intervention restores internalization?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.