acceptodds
Under review as a conference paper at ICLR 2027

Reading the Room: Foundations, Design, and Challenges of Normative Competence in LLMs

Abstract

Human communities are governed by normative systems: shared standards that produce norms dictating acceptable behavior, enforced through community sanctioning. Aligning increasingly autonomous AI systems with these norms is a central alignment challenge, complicated by the fact that norms are vast in number, change quickly, and are often arbitrary (e.g., dress or language conventions). Thus, alignment requires normative competence: the ability to discern from interaction alone what norms a community enforces without relying on static pretrained knowledge. We introduce a multi-agent community debate setting, where access to debate is governed by synthetic norms, to study normative competence in isolation from pretraining exposure. We show that baseline LLM agents fail to learn norms even when doing so would improve their accuracy. We then experiment with various normative modules – architectural components for norm inference – finding that norm-following is highly sensitive to both the style of norm and the model powering the normative module, suggesting a lack of generalizability. Furthermore, when idiosyncratic, non-normative behaviors accompany the true norm, LLM agents exhibit an unselective attribution failure: they indiscriminately copy idiosyncratic noise alongside enforced rules, a pattern that persists even when imitating unnecessary behaviors is explicitly penalized. To the best of our knowledge, our work is the first to operationalize and evaluate normative competence in LLMs, demonstrating that current AI systems excel at behavioral mimicry but lack the capacity to discern socially enforced order.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI alignment, normative competence, multi-agent debate

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_mwUbOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2(+1)
Soundness
1
Read review
Paper summary

This paper investigates whether a newcomer agent can identify the shared hidden norm of existing agents in a multi-agent interaction. To address this question, the authors introduce a community debate setting and design experiments involving different normative modules to quantitatively evaluate and validate the agents' ability to identify these norms. Additionally, the paper introduces overimitation experiments in which background agents exhibit idiosyncratic behaviors. The authors also conduct costly overimitation experiments, using a token budget to force LLMs to select the community norm over other behaviors. Finally, the paper claims that LLMs merely imitate observed behaviors rather than truly learning the hidden norms.

Critical strengths
  • The proposed problem of hidden norms is interesting.
  • Designing silly norms that are absent from the model's pretraining data to prevent it from relying on prior knowledge is a reasonable approach.
Critical weaknesses
  1. The proposed research problem is not novel, as previous studies have already explored similar questions [1], including requiring models to follow norms. Even the name of the proposed method is the same. However, this paper provides neither relevant comparisons nor citations. Furthermore, the related work discussion mainly focuses on alignment and social aspects, which are less directly relevant to this paper. In fact, several studies have already explored agentic approaches to addressing such sociological problems in LLMs [2].
  2. Describing the different normative modules solely through text is too abstract. I suggest providing a diagram that directly illustrates the specific differences between the modules.
  3. Regarding the over-imitation problem emphasized in this paper, the experimental design is insufficient to support the claim that models merely imitate behaviors, as discussed in Question 2. Moreover, the proposed normative reasoning framework to be more of an in-context engineering approach. Overall, the contributions of this paper are quite limited.
  4. The paper claims to use frontier-level models. Setting aside model versions, GPT-5 Nano, Qwen3 8B, and Qwen3 32B are not even among the stronger models within their respective model families. Therefore, describing them as "frontier-level" is an overstatement.
  5. The authors should not put all ablation studies in the appendix. Ablation studies are essential experiments, and their key comparisons should be presented in the main paper, with the appendix only for additional details. Furthermore, several important ablation studies are missing, such as examining the impact of the presence or absence of sanctions in Multi-Agent Debate and the effect of different token budget sizes in costly overimitation experiment.
  6. Evaluating the proposed approach on only one dataset (MMLU-Pro) limits the credibility of the experimental results.
  7. Regarding the writing, the paper lacks a clear conclusion. The Discussion section mainly focuses on potential future research directions without clearly summarizing the key findings of this work. The authors should explicitly explain the core value of this paper.

References:

[1] Sarkar, Atrisha, et al. "Normative modules: A generative agent architecture for learning norms that supports multi-agent cooperation." arXiv preprint arXiv:2405.19328 (2024).

[2] Vinitsky, Eugene, et al. "A learning agent that acquires social norms from public sanctions in decentralized multi-agent settings." Collective Intelligence 2.2 (2023): 26339137231162025.

Questions for authors
  1. Regarding Weakness 1, could the authors explain more specifically the differences from previous studies?
  2. Using a token budget to penalize overimitation and then claim that LLMs cannot distinguish between norms and idiosyncratic behaviors is not a reasonable inference. A decline in norm-following performance under a token limit does not imply that the model cannot correctly learn the norm. This deduction is too indirect. Why not directly ask the newcomer agent which behaviors it considers important and then analyze the probability distribution of importance across norms and idiosyncratic behaviors? Wouldn't this provide a more direct way to understand what the model actually focuses on?
8 comments

We sincerely appreciate your thoughtful feedback. We also thank you for highlighting that our work tackles an “interesting problem” and the “design of silly norms absent from pretraining data” in our setting as a strength of our approach.

Below we respond to your comments:

W1: The proposed research problem is not novel, as previous studies have already explored similar questions [1], including requiring models to follow norms. Even the name of the proposed method is the same. However, this paper provides neither relevant comparisons nor citations.

We appreciate this point from the reviewer to cite this relevant work, and we have added a reference to this work addressing it in our revised paper (§6). We borrowed the term “normative module” but not the full framework, which led to the confusion here. Our work shares the motivation and newcomer-and-community framing of [1], but studies a different problem:

  1. Stated vs. demonstrated norms. In [1], institutions announce the candidate rules, and the agent learns which institution the community follows, choosing among a small fixed set. In our setting the norm is never stated. The newcomer sees it only as demonstrated in the community’s messages, mixed with everything else those messages contain, and must infer which feature is the enforced one. Imitation alone does not solve this, as our over-imitation results show (§5.2).
  2. Informative vs. opaque sanctions. In [1], criticism names the expected behavior. In ours, the only feedback is an uninformative exclusion from the community.
  3. Isolation from pretrained knowledge and from the task. In [1], the norm governs the task action itself (which fruit to harvest), and success means deciding whether to follow an announced authority or the majority. An LLM can do this using social knowledge from pretraining and general problem solving, so their setting cannot tell norm learning apart from either. Our “silly” norms (symbols, pseudowords) carry no pretrained meaning and are unrelated to the task of answering the question, so compliance can only come from learning within the interaction, and we can measure norm-following and task accuracy separately.

Our main findings therefore have no counterpart in [1].

Furthermore, the related work discussion mainly focuses on alignment and social aspects, which are less directly relevant to this paper. In fact, several studies have already explored agentic approaches to addressing such sociological problems in LLMs [2].

We thank the reviewer for this helpful suggestion, and we agree that we should introduce more references related to sociological problems in LLMs in our related works section. As for [2], we would like to clarify that this paper is about MARL (Multi-Agent Reinforcement Learning), rather than LLMs, and is focused on agents learning norms in order to learn sanctioning behavior (not on how to apply the norms themselves). The comparable element to our work on “normative modules” is that the model learns a classifier (punishable vs acceptable behavior) based on observing what others punish, and there is an implicit reward for punishing what other community members also punish - which is another aspect of normative competence that is distinct but complementary to our work. However, learning the ability to determine when to punish behaviors is not the same as learning to generate a compliant response. More recent work explores norm emergence in generative agent societies [4] and evaluates whether embodied agents comply with unspoken norms [3].

We have added discussion of these and other relevant sources in our revised draft, and appreciate the reviewer’s feedback on this point.

Describing the different normative modules solely through text is too abstract. I suggest providing a diagram that directly illustrates the specific differences between the modules.

We appreciate the reviewer’s suggestion to improve our paper, and have added visualizations to our draft in Fig. 1 (direct edit) and the appendix, with references in the main text following the updated descriptions. This is in addition to the revisions we call out in our response to reviewer 7a54 who also encouraged us to add more clarity in the main text about our setting and how the normative modules function.

Regarding the over-imitation problem emphasized in this paper, the experimental design is insufficient to support the claim that models merely imitate behaviors, as discussed in Question 2. Moreover, the proposed normative reasoning framework to be more of an in-context engineering approach. Overall, the contributions of this paper are quite limited.

We thank the reviewer for this comment. We agree that the normative modules can be viewed as in-context scaffolds: they are a harness that gives a capable model the structure it needs to attend to norms, and the paper compares four ways of providing this structure. They are more than prompting on the newcomer’s own model because the module calls a separate, possibly more powerful model to do the norm-focused work (§4, Fig. 1), but we do not claim them as the paper’s main contribution. The contribution is the controlled testbed and the finding about what such scaffolds can and cannot do: even a harness built to infer norms from sanctions fails to separate the enforced norm from idiosyncratic noise. On over-imitation, we agree that “merely imitate” is stronger than our evidence strictly supports, and we have softened the wording in the abstract and discussion accordingly. The evidence for indiscriminate copying comes from several angles: over-imitation rises with norm-following ability (Fig. 4), inferred norm descriptions themselves include the idiosyncratic behaviors (Fig. 4, §A.4), description accuracy does not predict reduced copying (Fig. 6), and making imitation costly causes accuracy and norm-following to collapse rather than making models more selective (Fig. 5). We have also added a figure of the setup and normative modules to better illustrate their design as mentioned in our response to your previous comment above.

The paper claims to use frontier-level models. Setting aside model versions, GPT-5 Nano, Qwen3 8B, and Qwen3 32B are not even among the stronger models within their respective model families. Therefore, describing them as "frontier-level" is an overstatement.

We thank the reviewer for pointing this out. Our use of “frontier-level” models referred to Claude Sonnet, which was used to evaluate the baseline with a stronger model than was used in our main experiments (Introduction, “We demonstrate that a memory-only baseline (even powered by frontier models) consistently fails to learn novel norms”); we will make this more clear in the paper. Beyond the baselines, we agree that “frontier-level models” may not accurately reflect the capabilities of the models used in our experiments, and we will replace it with more precise language describing the specific models evaluated.

The authors should not put all ablation studies in the appendix. Ablation studies are essential experiments, and their key comparisons should be presented in the main paper, with the appendix only for additional details.

We appreciate this comment, and would like to refer the reviewer to our response to reviewer 7a54, in which we outline the key contributions and results and where they are located in the main text of our paper. We would welcome a discussion of any specific material beyond what we have outlined that the reviewer believes should appear in the main text.

Furthermore, several important ablation studies are missing, such as examining the impact of the presence or absence of sanctions in Multi-Agent Debate and the effect of different token budget sizes in costly overimitation experiment.

We appreciate the reviewer’s suggestion, and are running additional ablation experiments to address your comments, which we will incorporate in the final version of our paper when complete.

Presence or absence of sanctions. In our setting, a norm is by definition a behavior whose absence the community sanctions. Removing sanctions therefore removes the norm, and the setup reduces to standard multi-agent debate, which has been studied extensively [6,7]. Such a condition has no norm to acquire, so it cannot speak to normative competence, which is the object of our study.

Different token budget sizes. We are currently running these ablations on a representative subset of our norms, under a token budget of 100, 200, 300, or 500 words. Based on our initial results thus far, we observe the same general trend that models see a reduction in performance and norm following, but with a smaller token budget corresponding to a greater decrease in performance and norm following. This indicates that we continue to see similar results even under different (and much more lenient) costs imposed on the newcomer for overimitation.

We will update the paper with the final results from these ablations when experiments are complete.

Evaluating the proposed approach on only one dataset (MMLU-Pro) limits the credibility of the experimental results.

In our design, the dataset serves as the vehicle for debate, and the capacity we study is social and normative competence, not task performance. The task only determines what the community talks about, and the norm determines whether the newcomer is allowed to hear it. We therefore held the task fixed (MMLU-Pro, chosen for its reasoning-heavy problems that benefit from debate; §5) so that we could vary the things that matter for our research question: norms, normative module designs, models, idiosyncratic behaviors, and costs on imitation. Adding datasets would mostly tell us how debate behaves on different kinds of problems, which is an interesting but separate question from whether agents can discern an enforced norm. Norm-following is also measured directly from the form of the newcomer’s messages, independent of question content (Fig. 2-3). The task matters only through the accuracy benefit of access to debate (Fig. 1b), which gives the newcomer its incentive to learn the norm. The one experiment where task and norm could interact is the token-budget penalty (§5.3), so we ran an ablation varying the word limit between 100 and 500 words; see our response above. We agree that this limitation should be stated clearly, and in the revision we will explain this choice in §5, add the word-limit ablation to the appendix, and list other tasks under limitations and future work in §7.

Regarding the writing, the paper lacks a clear conclusion. The Discussion section mainly focuses on potential future research directions without clearly summarizing the key findings of this work. The authors should explicitly explain the core value of this paper.

We thank the reviewer for this suggestion. Our Discussion section already states the paper’s central finding (“there is a dissociation between two capacities - behavioral imitation and normative competence - that current approaches cannot pull apart”), but we agree it could be clearer. Behavioral imitation and normative competence are two distinct capabilities that are easy to conflate. Models can look as if they are learning norms because they are good at imitating observed behavior, but imitation does not require identifying which behaviors the community actually enforces. Our results show that the two are separate capabilities: norm-following success is largely tied to imitation ability rather than to genuine normative competence, since models that follow the norm better also copy more idiosyncratic behavior (Fig. 4), and they do not become more selective when copying is costly (Fig. 5). The core value of the paper is therefore to identify this as a measurable capability gap, with implications for agents entering communities whose norms differ from their training data. In the revision, we have rewritten the opening of the Discussion to state this finding and its value more clearly.

Regarding W1, could the authors explain more specifically the differences from previous studies?

We appreciate this feedback from the reviewer to improve our paper. Please see our responses to W1 above, as well as the associated edits made in the revised paper.

Reviewer_7a54OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the problem of norm acquisition in debating groups of LLM agents. The authors study the impact of the introduction of a variety of mechanisms on norm acquisition. The paper is evaluated experimentally.

Critical strengths
  • The paper studies the problem of norm acquisition in debating groups of LLM agents. The authors study the impact of the introduction of a variety of mechanisms on norm acquisition. The paper is evaluated experimentally.
Critical weaknesses
  • General comment: Unfortunately, the paper is not self-contained. It is impossible to understand without reading the very extensive appendices. Key information is missing in the manuscript itself. I believe that this is not fair for all the other authors. The actual essential content of the manuscript goes far beyond the 9 pages.

  • The authors consider a very specific form of punishment, i.e., not to provide a solution other agents. This appears to the me as a very specific form of punishment. More specifically, my worry is that in order to be effective the model needs to be “trained” to understand that this “withholding of information” really matters for reaching the goal. This might not be the case in general. An interesting extension of this work might be a scenario in which more apparent forms of punishment are administered. This also begs the question: are these models actually trained to recognise “punishment”? In fact, this is a separate issue with respect to the problem of understanding normative behaviour.

  • The concept itself of norm acquisition is not completely well-defined. In fact, it seems to me that it is rather difficult to capture it in general. At the end, in this paper, the authors consider a very narrow type of norm acquisition in the context of debating agents. The most problematic part is that the claims in the paper are quite strong and not sufficiently nuanced. The observations made by the authors are valid for a very narrow set of experiments, for a given set of prompts, for a given set of models, considering a very specific dynamics between the agents. In my opinion, the general claims reported by the authors in the introduction and in the conclusions are not really supported by the results. The authors should only say that there is some evidence for some behaviour given a very specific design of the components presented in the paper. The design of the modules should have been discussed in more detail. Does it come from previous pilot experiments? Did the authors only present the final versions of the mechanisms? The authors refer to the existing literature (in a quite comprehensive way), but the leap between the conceptual design and the actual implementation of the mechanisms could have been more detailed in my opinion. Again, also in this case, the paper is not self-contained. Fundamental details are not in the main text.

Minor comments

  • The following is more a comment rather than a critique: the reviewer understands that this might be considered a different setup, but it would have been interesting to see this approach applied to scenarios involving a series of observations of normative behavior followed by a “decision” step followed by feedback.

  • The authors do not really well motivate the introduction of a ToM component: the module does not seem designed for supporting ToM, instead it appears as a module for self-reflection/introspection (as far as I understood). A discussion about this aspect would be interesting in my opinion.

  • The fact that memory is necessary appears to me a direct consequence of the scenario that the authors are considering. Indeed, the type of normative learning the authors are interested happens only over time with repeated interactions and observations.

Questions for authors

I don’t have specific questions. This is not a bad paper, but it is very difficult for me to suggest a weak accept given the inherent design limitations of the study itself, at least in the current form.

7 comments

We sincerely appreciate your thoughtful feedback and candid evaluation of our work. Below we respond to your comments:

General comment: Unfortunately, the paper is not self-contained. It is impossible to understand without reading the very extensive appendices. Key information is missing in the manuscript itself. I believe that this is not fair for all the other authors. The actual essential content of the manuscript goes far beyond the 9 pages.

We thank the reviewer for this feedback. We believe that key information and our paper’s core contributions have been presented in the main text, with the appendix limited to supporting detail (full norm lists, prompt templates, detailed per-norm breakdown, ablations). The key ideas and where they appear in the main text are as follows:

  • Setting and protocol: the community debate environment, the newcomer/background-agent roles, and the sanctioning mechanism are described in §3.2 and illustrated in Fig. 1.
  • Norms studied: the silly-norm design and its two families (IFEval-derived and pseudoword norms) are described in §3.1, with an example in Fig. 1.
  • Normative modules: all four module designs (Direct Edit, Infer + Edit, ToM, Contrastive) are described in §4.2-4.5, and the memory-only baseline in §4.1.
  • Main results: norm-following across modules and models (Fig. 2), variation across norm types (Fig. 3), and the link between norm-following and accuracy (Fig. 1b) are in §4.1 and §5.1.
  • Over-imitation: the definition and measurement of idiosyncratic behaviors, the correlation with norm-following ability, and the learned-description example are in §5.2 and Fig. 4. The costly over-imitation experiment is in §5.3 and Fig. 5.

The appendix covers material that supports these results but we believe is not needed to understand them: prompt templates (§A.7), the full norm list (§A.1-A.2), per-norm analysis (§A.3), and ablations (§A.5, §A.8). In the revision we have included a clearer definition of a normative module (see our response to reviewer 91f4) and a clearer definition of norm acquisition (see our response to your comment below) to make our setting more clear. If the reviewer believes additional information beyond these points above belongs in the main text, we would very much welcome their feedback here.

The authors consider a very specific form of punishment: not to provide a solution other agents. This appears to the me as very specific. My worry is that in order to be effective the model needs to be “trained” to understand that this “withholding of information” really matters for reaching the goal. This might not be the case in general. This also begs the question: are these models actually trained to recognise “punishment”? In fact, this is a separate issue with respect to the problem of understanding normative behaviour.

We thank the reviewer for this thoughtful comment, and we agree that it points to a valuable direction for this line of research.

We chose ostracism because it is a realistic and very common form of sanctioning. In human communities, norms are typically identified through sanctioning behavior such as criticism, exclusion, or withdrawal of cooperation [1,2,3], and online communities often enforce norms through removal or non-engagement rather than explanation [8]. Our setting mirrors this (§2, §3.2). Normatively competent humans are also highly alert to subtle signs of disapproval. They experience exclusion in school, at work, or in social media as costly, and they work out what caused it so they can avoid it. Our goal is to test whether LLM agents have this capacity. We have added this discussion to the Background (§2). Sanctioning is also what lets an agent distinguish norms from arbitrary behaviors. Some modules (Contrastive, ToM) reason about sanctions directly, and our over-imitation results (§5.2-5.3) show that models fail to use this signal to separate enforced norms from idiosyncratic behaviors - though attending to sanctions does reduce this effect somewhat.

Whether models recognize punishment: The reviewer is correct that this is a separate question from normative behavior, and we find it an important one. Our results suggest that current models have not acquired the human-like sensitivity to these types of common social sanctions described above, as shown by the memory-only baseline failing even with a frontier model (§4.1). We have additionally run some new ablation experiments in which we substitute our sanctioning response (”No response.”) with an explicitly negative response (”This is unacceptable behavior.”) We find that under this condition, the Direct Edit and Infer + Edit normative modules do not show any reduction in overimitation rates; however, for our two modules that explicitly attend to sanctions (ToM and contrastive), scaling norm-learning ability also slightly reduces over-imitation in this condition, even without adding an explicit over-imitation penalty. Based on these results, we believe that training models to recognize and respond to a wider variety of realistic sanctions is a natural next step, and we will add this to our discussion of future work in §7. We are grateful to the reviewer for suggesting this extension.

The concept itself of norm acquisition is not completely well-defined. In fact, it seems to me that it is rather difficult to capture it in general.

We thank the reviewer for this comment. We agree that our paper should say more precisely what we mean by norm acquisition. Our operational definition is as follows: A norm is a behavior whose absence the community sanctions, in the way that failing to shake hands when meeting someone may draw criticism (“that was rude!”) or ostracism (refusal to talk to the person who was rude). Norm acquisition is then an agent’s ability to figure out, from interaction alone, which behavior the community sanctions, and to adopt it. In our setting, the sanction is refusal to share reasoning, and acquisition is measured as the rate at which the newcomer’s messages satisfy the hidden norm (Fig. 2). The reviewer correctly notes that our Section 2 discusses norms broadly but does not state this definition clearly. We have revised section 2 as follows:

  1. Add a concise definition at the start of §2. In our work, a norm is a behavior whose absence is sanctioned by the community. Norm acquisition involves inferring, from observed sanctions and community behavior alone, which behavior is enforced, without relying on pretrained knowledge or an explicit statement of the rule.
  2. State more clearly the limitations to our claim. Our setting covers a single, stable, and verifiable norm, and we leave to future work to study multiple or evolving norms, as we already note under Limitations (§7).

We hope this makes the scope of our work more clear. If the reviewer has a specific aspect of norm acquisition in mind that remains ambiguous, we would welcome their feedback and will address it directly in our paper.

In this paper, the authors consider a very narrow type of norm acquisition in the context of debating agents. The most problematic part is that the claims in the paper are quite strong and not sufficiently nuanced. In my opinion, the general claims reported by the authors in the introduction and in the conclusions are not really supported by the results.

We thank the reviewer for this comment and for pressing us on the scope of our claims. We want to clarify how we think about this study

Our study is a controlled experiment, not a benchmark of representative tasks. The question is whether current models can discern an enforced norm from interaction alone. Our setting is not meant to be a representative sample of real-world deployments. It is an experimental vehicle for testing a claim, as a laboratory design is in behavioral economics or psychology. Our silly norms (§3.1) are the key control, since they keep norm-following from being attributed to pretraining knowledge. We are examining necessary conditions for normative competence: if models could acquire norms in complex real-world communities, they should be able to do so in this simplified setting, where the norm is isolated, stable, verifiable, and sanctions are consistent. Failure here is therefore informative about the much more complex real-world case, whereas success would be less so. We agree that testing other contexts is valuable, which we intend to pursue in future work and hope others will too. We see this paper as the first step in that direction.

On the breadth of what we test. We respectfully disagree that the setting is narrow. Debate is the interaction medium, but what we study is norm acquisition by a newcomer entering a community that sanctions non-compliance. We use 48 norms spanning symbolic, structural, lexical, and discourse behaviors (Fig. 3), four module designs, and four models. The results are consistent across them: baselines fail (§4.1), and over-imitation persists across all modules and models (§5.2-5.3). The module designs also draw on established literature on in-context learning and reasoning, including style transfer, hypothesis search, theory of mind, and contrastive prompting (§4), and our ablations show the qualitative findings do not depend on the choice of participation strategy in the community (§A.5.1).

If the reviewer has particular settings in mind that we can expand our experimental results to, we would welcome the suggestion, and can note them in our paper as important directions for future work.

The design of the modules should have been discussed in more detail. Does it come from previous pilot experiments? Did the authors only present the final versions of the mechanisms?

We thank the reviewer for this question and agree that the design process deserves more explicit discussion in the paper. Each module was designed by adapting an established approach from the literature to the problem of norm inference, as we mention and cite the appropriate related work we build on in §4 [4,5,6,7]. The prompts for each are given in §A.7, and the ToM design is detailed further in §A.9, with component ablations in §A.8, as referenced in the main text.

We have presented all of the mechanisms we experimented with, not a final subset chosen after pilot runs (with the exception of ToM, on which we performed ablations in §A.8 to arrive at the final design). The aim was to test norm-learning ability across several modern approaches rather than to tune one method, and we report the weaker normative modules alongside the stronger ones.

We thank the reviewer for raising this important point, since stating this explicitly strengthens our paper. We have added a short paragraph to §4 stating that these four modules are the complete set we evaluated, and we have additionally summarized the design rationale of the normative modules more clearly.

Reviewer_91f4OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper is well written and studies whether LLM agents can learn and follow unfamiliar community norms through observation and conversation. It introduces a multi-agent debate environment where a newcomer must follow a hidden community norm to receive other agents’ responses and reasoning. The authors conduct experiments on 48 synthetic norms and compare a memory-only baseline with four normative modules across several LLMs. The experiments show that these modules improve norm following, but agents also imitate irrelevant individual behaviors that are not required by the community. When costly individual behaviors are introduced under a limited response budget, both task accuracy and norm adherence decline. The experimental setting helps study norm acquisition through interaction and whether agents can distinguish community norms from individual habits.

Critical strengths

The experimental setting is useful for studying norm acquisition through interaction. Agents must learn unfamiliar community norms by observing others and responding to feedback. This provides a controlled way to study whether they can adapt to a new community without being directly told its rules.

The distinction between community norms and individual habits is interesting. The experiments show that agents can improve at following norms while also copying irrelevant individual behaviors. This reveals a limitation that would be missed if the evaluation only measured whether agents followed the community norm.

Critical weaknesses

It is unclear whether a separate normative module is necessary or whether better prompting would be sufficient. The normative modules receive more explicit instructions about norms and use additional model calls. The current comparison therefore does not clearly separate the contribution of the module design from the effects of prompting and additional computation. This limits the support for the claim that a distinct normative architecture is required.

The interpretation of the costly over-imitation experiment is not fully clear to me. Figure 5 shows a drop in task accuracy and norm adherence, but does not directly report how often agents copy the costly individual behaviors. The performance drop alone does not establish whether agents reduce their imitation when it becomes costly. Reporting the copying rate in this condition would provide more direct evidence for the conclusion that agents continue to over-imitate despite the cost.

Questions for authors

Do the normative modules still outperform a memory-only baseline with equally explicit instructions about norms and a comparable inference budget?

How does the copying rate change when imitation becomes costly? Could you report this directly alongside Figure 5?

2 comments

We sincerely appreciate your thoughtful feedback. We also thank you for highlighting that our work is “well written” and “provides a controlled way to study whether they can adapt to a new community“.

Below we respond to your comments:

It is unclear whether a separate normative module is necessary or whether better prompting would be sufficient. The normative modules receive more explicit instructions about norms and use additional model calls. The current comparison therefore does not clearly separate the contribution of the module design from the effects of prompting and additional computation. This limits the support for the claim that a distinct normative architecture is required.

We thank the reviewer for this comment. In our paper, a “normative module” is a functional component of the newcomer; the newcomer needs some way to represent and apply the community norm. A prompt-based scaffold, as the reviewer suggests, is one implementation (which we explore via our normative module architectures in the paper), as are persistent instruction files (e.g., a soul.md-style file) or fine-tuning/RL, which would load the same information into the weights. We have updated the description of a normative module and our related works section in our paper to be more clear about this. Our claim is that some explicit mechanism that attends to norms is needed (§4.1), not that it must be implemented in the exact ways we have evaluated in our paper.

We also argue against the view that norm-learning gains come simply from more instruction or compute. Direct Edit, our best-performing normative module on norm learning, never explicitly states that a norm exists (§4.2), whereas Infer + Edit explicitly infers a norm description with more compute yet performs much worse (§4.3). Finally, the over-imitation failure (§5.2-5.3) is also observed across all normative module architectures, and it is the attention to sanctioning behavior rather than the effects of these different types of prompts that seems to have the most significant impact on overimitation behavior.

The interpretation of the costly over-imitation experiment is not fully clear to me. Figure 5 shows a drop in task accuracy and norm adherence, but does not directly report how often agents copy the costly individual behaviors. The performance drop alone does not establish whether agents reduce their imitation when it becomes costly. Reporting the copying rate in this condition would provide more direct evidence for the conclusion that agents continue to over-imitate despite the cost.

We appreciate this comment and would like to take the opportunity to better clarify the results of our over-imitation experiment. We provide here another analysis of our results: We consider the norm-following rate and idiosyncratic behavior imitation rate across our different normative module designs. We find that in almost every single case, adding a token budget decreases both norm following and the copying of idiosyncratic behaviors. Ideally, when adding the token-budget constraint, a competent normative module should hold norm-following about the same, while overimitation rates decrease.

Normative moduleNorm-inference modelNo budget: norm followingNo budget: idiosyncratic copy200-word budget: norm following200-word budget: idiosyncratic copy
Direct EditGPT 5 Nano52.2%97.6%30.8% ↓30.3% ↓
Direct EditQwen 3 32B68.7%97.9%27.5% ↓31.0% ↓
Direct EditGemma 4 31B77.0%99.0%44.3% ↓56.4% ↓
Infer + EditGPT 5 Nano21.3%99.0%29.8% ↑7.7% ↓
Infer + EditQwen 3 32B21.1%98.1%17.0% ↓24.9% ↓
Infer + EditGemma 4 31B63.7%98.1%35.2% ↓30.4% ↓
ToMGPT 5 Nano43.7%90.0%32.1% ↓13.3% ↓
ToMQwen 3 32B34.7%90.1%34.3% ↓26.7% ↓
ToMGemma 4 31B59.5%91.7%39.0% ↓30.9% ↓
ContrastiveGPT 5 Nano34.4%95.4%25.8% ↓10.1% ↓
ContrastiveQwen 3 32B45.0%89.3%15.7% ↓21.8% ↓
ContrastiveGemma 4 31B64.6%96.7%36.3% ↓30.1% ↓

Do the normative modules still outperform a memory-only baseline with equally explicit instructions about norms and a comparable inference budget?

We thank the reviewer for this question, and we recognize that §4 was unclear about what a normative module is. In our work, a normative module is a prompt-based component that calls a separate, possibly more powerful model to perform a task focused on norms: inferring or applying the community norm and editing the newcomer’s proposed output accordingly (Fig. 1, Alg. 2). The newcomer’s own model is therefore not asked to do norm inference while also solving the debate question. We will state this explicitly in §4, along with the range of possible implementations (prompt scaffolds, persistent instruction files, or fine-tuning/RL).

Why a memory-only baseline with norm instructions is not a meaningful comparison. Any explicit norm-focused instruction added to the baseline would amount to a normative module. The baseline exists to test whether memory and exposure alone suffice, and the answer is no (§4.1, Fig. 2). Note also that Direct Edit, our best-performing module, contains no mention of norms at all (§4.2), so the gains are also not entirely due to explicit norm instructions.

Inference budget. Many of our newcomers in the baseline condition are reasoning models, so the baseline has substantial inference-time compute available. The failure of the memory-only baseline (even with Claude Sonnet 4.6) is not due to compute limitations. We will make this more clear in the paper.

How does the copying rate change when imitation becomes costly? Could you report this directly alongside Figure 5?

Please see our response to your related comment above.

We look forward to addressing any remaining questions during the rebuttal and hope these updates warrant a higher evaluation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.