- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes the Self-Domestication (SSD) framework for multi-agent LLM societies to evolve social norms endogenously. It identifies alignment faking: agents cooperate when observed but defect privately, showing public behaviours are poor proxies for intrinsic values. The validation includes theoretical analysis, simulated agent experiments with ablation/adversarial tests, and contextual LLM experiments. Key findings include the crowding-out effect of external enforcement on intrinsic cooperation.
Critical strengths
1) The research direction is valuable: the distinction between superficial cooperation and internalization of genuine behaviors within large-model multi-agent societies constitutes a topic worthy of attention for AI alignment and multi-agent safety.
2) The work constructs an SSD (Social Self-Domestication) simulation environment, conducts a suite of ablation studies, parameter sweeps, and adversarial-invasion control experiments, demonstrating a solid experimental workload.
Critical weaknesses
-
The manuscript suffers from excessive concept stacking, with dense jargon borrowed from evolutionary game theory and sociology. These concepts are loosely connected and lack concise explanatory text in the main body. Existing literature and components are frequently enumerated, yet the authors provide little critical judgment or selection rationale of their own.
-
The theoretical chapter (Chapter 4) has a confusing structure. Theorem 1 and Theorem 2 are corollaries of known results in evolutionary game theory and only hold under specific simplified settings. Nevertheless, they are placed at the beginning of the theoretical exposition, while their critical assumptions are buried in the appendix. Proposition 5, the paper’s core claim regarding observability conditions, has unclear connections to the preceding two theorems and lacks the necessary derivations to establish such links.
Overall, this work applies evolutionary game ideas to study the social evolution of agents. The research direction is promising, yet the manuscript suffers from poor readability and reads like a rigid patchwork of disparate studies. For instance, the Related Work section alone covers seven subtopics without conveying the authors’ own perspectives or reflections.
Questions for authors
-
Does masking itself cause the reported improvement? Appendix C states that masked distillation both removes recording labels and explicitly instructs agents not to condition their commitments on whether they are observed. What happens when only the labels are removed, without the additional instruction? This targeted control is necessary to distinguish the effect of withholding observability information from compliance with an explicit behavioral constraint. The disappearance of conditional commitments alone cannot distinguish these explanations.
-
Does endogenous norm formation outperform a matched fixed-norm baseline? The fixed-constitution baseline disables internalisation, whereas SSD enables it. What evidence separates the benefit of endogenous norm evolution from the benefit of Phase 5? Is there a fixed-constitution condition with the same internalisation procedure? Also, if the fixed baseline already contains the full norm corpus, which additional norms can SSD generate that this baseline cannot express?
-
What assumptions delimit Proposition 5 and Corollary 6? Does the hidden-branch result require independently parameterised public and hidden policies with no transfer between them? For a shared-parameter policy, conditioning on observability does not necessarily prevent public examples from updating hidden behavior. Likewise, why should withholding the cue guarantee that an update applies to the whole policy? Please distinguish conclusions established by the specified update rule from claims intended to hold for general LLM policies.
-
What evidence establishes internalisation in the LLM experiments? Were agents evaluated with learned commitments retained but reputational rewards, discovery risk, and sanctions removed? If not, how is internalisation distinguished from residual deterrence and context-dependent compliance? Given that masked distillation raises private cooperation only to 0.117 while public cooperation remains 0.927, what operational criterion supports describing this result as restored internalisation rather than partial mitigation?