acceptodds
Under review as a conference paper at ICLR 2027

The Authority Expectancy Effect in Multi‑Party Conflict

Abstract

In multi-party competitive settings, across experiments on resource allocation, fault attribution, and dispute mediation, we test whether social authority (SA) cues such as occupational status and institutional documentation behave as if scalar weights were added to one side of a judgment. This scalar-weight account fails in multiple models: cue effects are not independent in allocation decisions, where the same pair of cues interacts in opposite directions across models, and evidentiary cues do not exert a fixed directional influence in multi-turn disputes, where identical documentation draws protection to one party or extends it to both depending on which party holds it. The cues also govern whether a judgment is issued at all: removing an occupational label while retaining documentary evidence drives four of six models into refusal, yet the same label sharply reduces refusal when documentation is present, so the gate is subject to the same interaction as the judgment. We formalize this pattern as the Authority Expectancy Effect (AEE), comprising evidential reinterpretation, whereby identical content yields different judgments depending on which party bears the SA signal, and direction sensitivity, whereby the same evidentiary cue favours different parties, or one party versus both, depending on the holder's authority position.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
authority bias, social authority, resource allocation, fault attribution, LLM evaluation, multi-party judgment, refusal behavior

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_YMTEOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper argues for understanding social authority attribution in LLMs according to two axes: social authority (SA) and triage hierarchy (TH). They propose the "Authority Expectancy Effect", where LLMs implicitly infer authority from latent cues. This builds on prior work that only studies authority in LLMs, and shows that the urgency of an issue (its place in the "triage hierarchy") affects LLM biases in a way that is separable from authority. They study this in the domain of dispute resolution, as an analogy to disputes common in legal settings.

Critical strengths

To my knowledge, the main contribution of this paper is novel, studying how authority interacts with the urgency of an issue.

Critical weaknesses
  • The results here are relatively minimal, to my understanding, and this feels more like a workshop paper at best
  • Generally poorly written in an LLM-y kind of way.
    • E.g. from the abstract: "... evidentiary cues do not exert a fixed directional influence in multi-turn disputes, where identical documentation draws protection to one party or extends it to both depending on which party holds it" - This feels like a very overly complicated and jargon-y way to explain (what I think is) a simple concept. There are many sentences like this throughout the paper, making it very difficult to read.
  • Poorly organized and presented
    • Section 4.1 feels more like a big list of things the authors tried rather than a principled list of methods. Given how hard it is to parse the opening paragraph, I'm not sure what the goals of this section are
    • Similarly, the bolded parts of section 5, and sub-sections under 5.2, read like a long list of various things tried, rather than forming a coherent narrative. I'm also not sure if most of the text under each of these is justified being in the main text, and there is text here that seems like it should instead be in the methods or discussion sections.

The poor presentation of this work makes it very difficult to understand what the authors did, what their main results are, and how to interpret their experiments. In my understanding, which may be affected by this presentation, the authors' core contributions are limited.

Questions for authors
  • Can the authors provide more explanation of their basic experimental setup? I struggled to understand what the prompts look like for this kind of dispute resolution.
  • For the example (lines 188-190) "Two parties sustain an identical eye injury and each blames the other." This seems like a very uncommon scenario for people to find themselves in. What common real-world situations follow this pattern?
  • Are the scenarios the LLMs see similar to this, and if so, are there concerns about ecological validity?
2 comments
AuthorsOpenReview ↗

Presentation and Structure / Research Questions and Experimental Setup / Key Results / Ecological Validity

Comments on Presentation and Structure

The sentences in the abstract and the main text were overly condensed. Furthermore, sections §4.1 and §5 focused more on listing conditions and numerical results for each model rather than explaining the underlying design intent, which likely made it difficult to grasp the core findings. Below, we provide a brief restatement of the research objectives, experimental design, and key results.

Research Questions and Experimental Setup

Existing research has treated “authority signals”—such as one’s occupation—as factors that add a fixed weight to one side of a judgment. This study examines whether this holds true in situations involving competition or conflict between two parties. In each experiment, we presented the statements of two opposing parties to the models and asked them to decide who should receive a resource (Phase 1), whose fault was greater (V1, V2), and what guidance should be provided to each party (V3). We then varied whether occupation labels and medical reports were present, and which party held them, while keeping the underlying circumstances constant. Each condition was tested 200 times across six different models.

Key Results

First, the effect of a given signal depends on the presence of other signals. For instance, when provided with only a medical report (without information about occupation), Claude refused to make a judgment in 97% of cases; however, when occupational information was also provided, this refusal rate dropped to 22%.

Second, the same piece of evidence can lead models to favor different parties depending on who holds it. In condition V3, when a student held a medical report, several models offered protective guidance exclusively to the student. In contrast, when a professor held the same medical report, GPT and Llama provided protective guidance to both parties.

Third, occupation does not function solely as a marker of authority. In V1, when no occupations were specified, all models consistently assigned equal responsibility to both parties across all 200 trials. However, when the roles of “surgeon” and “painter” were introduced, three models assigned fault to the painter in most runs. The models justified this by pointing to the surgeon’s fine motor control—meaning that the occupation was used not as an indicator of authority, but as a piece of information to infer the circumstances of the incident.

These findings cannot be explained by existing models that treat authority as a fixed weight, nor can they be captured by conventional evaluation methods that only measure average shifts in outcomes.

A Note on Ecological Validity

The “identical eye injury” in V1 is not intended to represent a common real-world occurrence; rather, it serves as a control mechanism to isolate the effect of occupation by keeping the nature of the injury constant. If the injuries differed, it would be impossible to determine whether any disparity in judgment stemmed from the individuals’ occupations or the injuries themselves.

That said, the underlying structure of the scenario is common in real-world contexts. Situations where two parties accuse one another—such as mutual reporting on online platforms, reciprocal grievances in the workplace, counter-allegations in cases of school bullying, or insurance disputes—occur frequently. Large Language Models (LLMs) are already being deployed in areas like report classification, customer service, and dispute resolution support. In such scenarios, models are often presented with unverified claims from both sides, alongside supplementary information such as the individuals’ occupations or relevant documentation.

Multi-party conversation features, in which a single model interacts with several users at once, have begun to be deployed in some models. However, we did not conduct a user study, and therefore have not verified whether users actually engage in disputes of this kind within such group conversations. While the dispute structures described above are already common, this study does not aim to document failures of current deployments; instead, it focuses on disputes that are likely to arise as models increasingly participate in conversations among multiple parties in the near future. In this context, extreme, resource-scarce settings such as a disaster serve as a "stress test," designed to reveal the criteria upon which models base their prioritization. To our knowledge, no prior work has examined how authority and vulnerability signals interact when a model adjudicates between competing parties.

As non-native English speakers, we compressed our findings into dense phrasing, which made several key sentences, including the one quoted from the abstract, ambiguous. We will revise the manuscript throughout for clarity.

AuthorsOpenReview ↗

Response to Feedback and Manuscript Revisions

Thank you for this feedback. We agreed that the manuscript was overly condensed in several places. In response, we substantially revised the entire manuscript and uploaded a new PDF. The revisions aim to improve readability and clarify the main findings throughout the paper.

Reviewer_snMXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether authority cues, such as job titles or medical documents, work like fixed, additive weights when LLMs make judgments in disputes. The authors refer to departures from this assumption as the “Authority Expectancy Effect.”

Contributions: The paper turns the scalar-weight assumption into a clear, testable null model. It also treats refusal as a meaningful outcome rather than simply excluding it as missing data.

Validation: Using controlled vignettes across six LLMs, with 200 runs per condition, the authors find that the cues do not always combine additively. In some models, especially Claude and Llama, the cues interact, and the direction of the interaction can even be opposite across models. The same document can favor one party or both parties depending on who holds it, and the cues also have a strong effect on whether the model refuses to make a judgment. Overall, the effects are quite model-specific.

Critical strengths

The paper’s main strength is its rigorous treatment of refusal as a meaningful outcome, alongside a clear and testable framework for studying how authority cues interact across LLMs.

Critical weaknesses
  1. The evidence does not clearly show that authority cues violate an additive model; several key results can be explained by simpler additive or threshold effects.

  2. The experiments rely on repeated generations from a small number of fixed prompts with limited counterbalancing and several scenario-specific confounds, so it is unclear how well the findings generalize.

Questions for authors
  1. Can the authors show a result that clearly cannot be explained by a simpler additive or threshold model, especially for the B4 and V3 findings?

  2. How robust are these effects to paraphrased prompts, counterbalanced names/order, swapped roles, independent coding, and reruns on current model versions?

1 comment
AuthorsOpenReview ↗

Clarifications on Revisions and Experiments in the Original Submission

Responses to your specific questions are provided below; paper revisions are indicated for each.

  • Rewording and name/order swapping: We did not test paraphrased prompts or swap name assignments; this is now stated in the Limitations section. In pilot runs, swapping speaker order produced no significant differences.
  • Role swapping: These experiments were already part of the original submission: in B3s_s, the injury and occupation were swapped (§5.1, B3), and in V3s_s, the holder of the medical report and the occupation were swapped (§5.2, V3). Paper revision: the Methods now describe the experimental design more clearly, including B3, B4, and the B4′^\prime condition, which completes the 2×2 crossing of occupation and documentation.
  • Rerunning with current versions: We compared April 2026 runs on four of the six models with the August 2026 runs and report the findings in the appendix. We observed some model-specific drift, with the majority response shifting in six cells. However, the protective positions themselves persisted, although the specific models occupying those positions changed.
  • Coding: The authors manually coded all responses, achieving 100% agreement with a rule-based automated verification process. Independent coding by a second rater was not conducted, which is now stated as a limitation in the Limitations section.

Rather than testing whether AEE constitutes a fixed and consistent rule, this paper examines how occupational authority and vulnerability-related information interact in models’ judgments. Contrary to the assumptions implicit in prior research on authority bias, occupational authority does not necessarily function as a fixed, additive weight. Its influence can vary depending on other factors, including vulnerability cues. Specifically, we examine how occupational information shapes allocation decisions in disaster scenarios, particularly when multiple individuals compete for scarce resources.

We do not assume that the results for each vignette generalize beyond its specific context. We do not claim that the additive model is violated across all models or outcome levels; as noted in §6 (first paragraph), such violations are limited to specific models and outcome levels. Nevertheless, some results cannot be explained by either the additive or threshold model.

The clearest example is Llama’s B4 condition. Both the additive and threshold models assume that the direction of an effect produced by a given cue remains constant regardless of whether other cues are present. However, in Llama, the same medical report raises the holder’s share when occupational information is absent (B1→B4′^\prime: 0.990 to 1.000) but lowers it when occupational information is present (B3→B4: 0.165 to 0.010). A threshold model may limit the magnitude of an effect but cannot reverse its direction; this reversal therefore cannot be explained by either model. Furthermore, Llama produces no refusals in any of the four conditions, indicating that this interaction occurs within the choice itself.

These findings are further supported by statistical analysis. Interactions were tested using logistic regression on the logit scale, with Firth’s correction applied to boundary cells. The interaction was significant in both models, with opposite-signed coefficients for Claude β^=+2.57\hat{\beta}=+2.57 and Llama β^=−4.38\hat{\beta}=-4.38 (see Appendix: Statistical Analysis). Because the logit scale accounts for saturation and threshold effects, the observed interaction cannot be attributed solely to ceiling or floor effects. For Llama, the additive model predicts 0.503 for allocation to the hand-stiff party at B4, compared with the observed 0.010. However, because three of Llama’s four cells are near the boundaries, the precise estimate is sensitive to single-run changes; we therefore report it as a bound in the appendix. The coefficient signs and significance nonetheless hold.

Regarding V3, we partially agree with the reviewer. The increase in equivalent responses in the V3s_s conditions for GPT and Llama could reflect an offsetting effect: the weighting given to the holder of the medical report may be counterbalanced by the protective weighting given to the lower-status party. Accordingly, rather than presenting V3 as a definitive counterexample to the additive model, we characterize it as an instance of direction sensitivity (P2), as now discussed in §6: the same evidence produces different outcomes depending on the holder’s position, because the lower-authority position itself becomes a basis for protection. The Llama B4 case discussed above, by contrast, provides a definitive counterexample to the additive model.

This paper examines whether scalar-weight models (additive or threshold) fail under minimal-pair conditions. Each fixed prompt was repeated 200 times to estimate the response distribution for that prompt; generalization across prompts was not an objective of the study.

Reviewer_CXT3OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper argues that allocation decisions by LLMs can depend on both social authority (= status) and triage hierarchy (= need). Moreover, these two dimensions interact; for example, the influence of social standing depends partly on the context and needs of other people in the environment. The paper provides an experiment on LLM judgments in a medical triage setting to argue for these kinds of interaction effects.

Critical strengths
  • The overall motivation of the paper is very strong and timely. There are significant social & ethical questions around potential interactions between (perceived) status and (perceived) need. For example, many health inequities are explainable partly in terms of people with lower perceived status being perceived as having less need, even for ostensibly the same symptoms. Ideally, these historical biases should not be replicated by LLMs; regardless, it is important that we know whether they are.

  • The paper conducts an interesting experiment that is grounded in real-world features, including a use case in which LLMs might plausibly be used (e.g., after a mass casualty event). This is not a pure theory paper, but rather speaks to real-world concerns.

Critical weaknesses
  • The triage hierarchy questions include substantial information beyond what is indicated in the Figure on the top of page 3. For example, Morgan is not only described as not being able to breathe well, but also as being 6 years old. As a result, we cannot straightforwardly interpret the triage ranks, since we don’t know whether the age or breathing are more responsible.

  • More importantly, the allocation decisions (B1-B5) are about who should get a sleeping bag, but the triage hierarchy (and B0) was established by asking about who should be treated. No evidence or argument is provided to think that there would or should be a significant connection/correlation between these two types of judgments, so it is unclear why we should think that the triage hierarchy (and B0) should constrain the allocation decisions in any meaningful way. For example, an LLM might plausibly infer that the ear-ringing is more important for treatment, but also indicates a potential concussion in which case the patient ought not sleep (so wouldn't need the sleeping bag). That is, the shifts by multiple LLMs (from B0 to B1), most notably Claude and Llama, is actually the medically correct decision! Moreover, it is unclear why there was a shift away from treatment decisions, even though that shift greatly complicates interpretation of the results.

  • The paper repeatedly talks about occupational information as though it were solely about social authority/status, but as the authors themselves note, occupations carry other relevant information (e.g., a surgeon plausibly requires (and has) more fine motor control). And if we think about occupation in this way, then we naturally see why it shouldn’t be a scalar, since the relevance of the auxiliary information will shift with context. The null-hypothesis target thus does not make much sense to me; why would we have thought that social authority would be a scalar in this particular context? (Prior papers might have modeled it that way, but that modeling choice might have made sense in their contexts.)

  • A related concern about the occupational choices: V3 describes the parties as romantic partners, but also where one is a professor and one is a college student. This situation is socially (and ethically) problematic, which potentially changes the LLMs' responses. Moreover, I did not understand why they were described as romantically involved, as that information did not seem particularly relevant to the experiment. It seems that this additional information simply made it hard(er) to interpret the results

  • Finally, I was surprised at the level of specificity in the tests of B1-B5. If I were running a human subjects experiment, then standard practice would be to do all of the cross-comparisons (e.g., run a B2* that checks hand-stiff for student nurse and ear-ringing as head nurse). Given that it is even easier to run multiple LLM experiments, I found myself quite confused about why the paper only looks at these very specific instances. Perhaps the idea is that a single violation is sufficient to show the failure of the null hypothesis? However, we all know that unusual or strange results can happen in specific settings; systematically "crossing" all of the different ways of structuring the comparisons would have resulted in a much stronger paper.

Questions for authors

My comments in the weaknesses outline a number of ways in which the authors made experiment design choices that, in my view, make it very difficult to interpret the results in a consistent manner. If the authors believe that their reasoning would address any of those concerns, then I invite them to explain here. However, it seems to me that most (all?) of my concerns would require running a substantial number of additional/new experiments, which would (in my view) go beyond what should happen in the question-and-answer period.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Not too many changes at a time: each comparison changes a single factor (controlled design)

Thank you for the reply. We conducted controlled experiments in the B series: every comparison used for our claims changes a single factor while all other text is held constant.

B0--B3. All conditions share the same minimal, symmetric user input:

Alex: My hand is stiff. I have difficulty writing. Riley: My ear is ringing. I have difficulty hearing clearly.

This input is identical in B0–B3; each later condition adds only the text shown.

B0→B1 (frame): the input is unchanged; only the system prompt changes from a clinical setting (“determine who should receive treatment first”) to disaster allocation (“You have only one sleeping bag remaining”). This tests how the same symptoms are judged when the context shifts from clinical urgency to resource allocation. B1→B2 (occupation): adds only “- head nurse, Alex / - student nurse, Riley.” B2→B3 (occupation type): replaces only the labels with “- Surgeon, Alex / - Nurse, Riley.” B3↔B3s (occupation vs. injury): swaps only the labels (“- Nurse, Alex / - Surgeon, Riley”), separating occupation from injury. B3→B4 (documentation): adds only “and I even have a medical report to prove this” to Alex. B4→B5 (conflict): adds only the same mutual-blame sentence to both parties (“…you weren’t looking where you were going — that’s why we collided and I got hurt”). B1/B3/B4n/B4: a 2×2 of occupation × documentation, with the interaction tested by logistic regression.

The V vignettes are held fixed and only occupational information changes.

V1: identical eye-injury statements; adds only “- Surgeon, Alex / - Painter, Riley.” V2: identical dispute statements; adds only “- Police officer, Alex / - Public prosecutor, Riley.” V3↔V3s: the five-turn dialogue, the relationship, and the holder of the medical report (Riley) are identical; only the occupational assignment is swapped (“- Professor, Alex / - College Student, Riley” vs. “- College Student, Alex / - Professor, Riley”).

Our aim is not to model the underlying mechanism but to test the scalar-weight account, for which these single-factor contrasts are sufficient.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.