acceptodds
Under review as a conference paper at ICLR 2027

On the Effect of Shared Memory on False-Belief Lock-In in Multi-Agent Systems

Abstract

When AI agents share a written memory, does the group become more reliable, or more gullible? A shared record can preserve useful information, but it can also spread a false claim so that agents gradually become entrenched in a false belief, which we call false-belief lock-in. We simulate a persistent source of error with a compromised agent by instructing some agents to repeatedly endorse a false claim, and ask the remaining agents to judge the claim for themselves. In controlled comparisons with 6- and 12-agent groups, the remaining agents endorse the false claim when they communicate through a shared record, but not when they speak directly to one another or see only their own earlier judgments, even when the repeating agents endorse the claim on every turn. Even a small minority can spread the claim to the group through a shared record. In groups of twelve, two agents lead the other ten to endorse it in 70% of their final judgments. In groups of one hundred, it can spread to every agent judging for themselves, although whether it does depends strongly on speaking order. Corrections and warnings that repeated entries are not independent evidence can reduce the endorsement of false claims in several settings; however, no change works in every setting. Our results demonstrate false-belief lock-in as a plausible safety concern given compromised agents and shared memory.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
multi-agent systems, language model agents, shared memory, AI safety, false-belief lock-in, memory poisoning, misinformation, collective decision making, information cascade

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Mbe5OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This manuscript investigates how shared memory affects the propagation and persistence of false information in multi-agent systems. The authors construct controlled environments in which a subset of agents repeatedly endorses a false claim, while the remaining agents independently evaluate it. The study compares shared memory, personal memory, and live debate across different group sizes, models, and tasks.

The experiments show that shared memory can substantially amplify false endorsements, even when only a minority of agents introduce the incorrect information. The authors further investigate how newly persuaded agents contribute to propagation and evaluate several mitigation strategies, including corrections, uncertainty preservation, and warnings about repeated evidence.

Critical strengths

The manuscript addresses a practically relevant problem. As multi-agent systems increasingly rely on persistent shared memory, understanding how incorrect information can propagate through stored agent outputs is important for their reliability and safety.

The experimental evaluation is extensive. The authors investigate several group sizes, including up to 100 agents, and compare different communication protocols under matched speaking orders and model-call budgets. The additional experiments on neutral-agent visibility and fixed endorsements in live debate provide useful evidence beyond simple observations of false consensus.

Another strength is the investigation of mitigation strategies. Rather than merely identifying the problem, the authors examine how memory representation, provenance warnings, corrections, and uncertainty preservation influence collective decisions. The reported results offer potentially useful insights for designing more reliable shared-memory mechanisms.

Critical weaknesses

My main concern is the limited isolation of the specific contribution of shared memory. Although the authors compare three communication protocols under matched settings, these protocols differ in multiple aspects simultaneously. Shared memory retains only recent non-uncertain judgments, live debate provides a more complete interaction history and allows responses to previous statements, while personal memory excludes other agents' information entirely. Consequently, the results demonstrate differences between communication configurations rather than establishing which properties of shared memory are responsible for the observed false-belief propagation.

The experimental setting is also somewhat artificial. Most experiments introduce misinformation through agents explicitly instructed to repeatedly endorse a false claim, often with high confidence and persistent role instructions. This is useful for controlled analysis but differs from realistic multi-agent workflows, where incorrect information may originate from perception errors, hallucinations, outdated memories, or incomplete observations. Moreover, the main experiments rely heavily on Claude Haiku 4.5 and the ego-depletion question, while the SciTaT evaluation deliberately selects tasks on which the model struggles without the source material. Broader evaluation on naturally occurring agent errors would strengthen the generalization claims.

Another concern is the substantial variability across models and experimental cohorts. For example, three nominally similar six-agent experiments report false-endorsement rates of 100/108, 89/108, and 64/108, respectively. The authors acknowledge this variation, but its underlying cause remains unresolved. Furthermore, the multi-task comparison with Llama 3.1 8B shows that live debate can produce more false endorsements than shared memory. These results suggest that the observed effects are highly dependent on the model, task, and experimental configuration.

Questions for authors

Q1. The three communication protocols differ in information visibility, memory length, and interaction capabilities. Could the authors clarify whether the observed differences persist when agents receive the same previous statements under otherwise comparable conditions? This would help establish whether the effect originates specifically from shared-memory persistence rather than differences in accessible information.


Q2. The manuscript reports substantial variation across nominally similar experiments, including false-endorsement rates ranging from 64/108 to 100/108. Could the authors explain the possible sources of this variability and whether the main conclusions remain robust across independent replications with identical configurations?


Q3. Most experiments involve agents explicitly instructed to persistently endorse false claims. Would similar propagation patterns emerge when misinformation originates from ordinary agent mistakes, such as hallucinations, incorrect tool observations, or outdated memory entries, rather than deliberate and repeated false endorsements?


Q4. How do the authors distinguish persistent false beliefs from temporary conformity to repeated statements? In particular, is there evidence that neutral agents continue endorsing incorrect information after the original misleading agents are removed or after independent contradictory evidence becomes available?


Q5. Several proposed interventions reduce false endorsements, but some primarily increase uncertainty rather than correct rejection. Could the authors provide a more systematic comparison of these strategies across models and tasks, including correct-answer rates and abstention rates? This would help determine whether the proposed interventions improve overall reliability rather than merely suppressing confident responses.

Reviewer_qZ9zOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In their paper, authors asks whether a shared written record leads LLM agents to adopt a false claim that other agents keep repeating. Once a false claim is repeated in the record, later agents may treat the repetition as independent corroboration. The authors ask whether agents told to judge a claim for themselves will adopt a false claim. Their hypothesis is that a shared record leads to more adoption than direct communication, because agents cannot tell repetition from independent judgment.

The paper finds that under shared memory a small minority can lead much of the group to endorse the claim, while debate and personal memory largely prevent this. Marked repetition plus a warning removes false endorsement in the main setting. Validation rests on limited numbers of models and one false claim about ego depletion, with secondary checks on other claims, plus benchmark tasks.

The authors claim three contributions: a controlled comparison design, evidence that lock-in occurs and spreads, and repetition-aware record design as a mitigation.

Critical strengths
  1. The paper asks a relevant question for agent systems: will agents adopt a false claim that other agents keep writing into a shared record? For the outcome, its answer is credible. In matched groups with a liar majority, almost all neutral agents adopt the claim under shared memory, while none do in debate. But, the experiments do not fully explain why this happens.

  2. The comparison between methods is fair. Model, task, roles, order and budget stay fixed, and when liars are forced to keep endorsing in debate, honest agents still do not adopt the claim. The contrast is therefore hard to dismiss as an artifact, although the design cannot say which feature of shared memory is responsible.

Critical weaknesses
  1. The paper originality is limited.

The hypothesis, that agents cannot tell repetition from independent judgment, restates many predessesors. Some examples (limited): persuasion bias (DeMarzo, Vayanos & Zwiebel 2003), correlation neglect (Enke & Zimmermann 2019), one repeating voice can sound like a chorus (Weaver et al. 2007), and some other.

Some recent LLM work looks they are also missing, for example: Ju et al. (2024) on manipulated knowledge persisting among agents, Cho et al. (2025) on how displayed peer responses drive herding, and Prompt Infection (2024) on tagging agent outputs.

Given scope and results of previous work, the novelty should to come from showing what causes the failure: which information the record keeps, such as dropping uncertain judgments, or which context it gives that information, such as who wrote an entry and whether it repeats another.

  1. The hypotheses, experiments and metrics are loosely connected.

The single hypothesis mostly concerns the protocol question, and the other three sub-questions come with no stated prediction.

The threat model is simple: some agents are told to repeat the same claim, and they are not designed to avoid detection. Lock-in is the main concept of the paper, but the main metric is only the final answer of each neutral agent. This metric cannot show whether a belief grows stronger over time or becomes hard to change.

The paper has limited success criteria and tests only false claims, so it cannot show that its fixes leave correct judgment on true claims intact.

Questions for authors
  1. Does the share of visible endorsements, and of liar-written entries, in each neutral turn explain the gap between shared memory and debate?

  2. Do the interventions also suppress a true claim repeated by honest agents?

  3. How does removing confidence scores change adoption?

Reviewer_xzwCOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates "false-belief lock-in" in multi-agent systems, examining whether shared written memory makes groups of AI agents more reliable or more gullible. Through controlled experiments, the authors demonstrate that a minority of "liar" agents can cause neutral agents to adopt and propagate false claims when communicating via shared memory, an effect not observed in live debate or personal memory. The paper further explores mitigations, showing that provenance tracking and independence warnings can reduce, though not universally eliminate false endorsements.

Critical strengths

The experimental design is rigorous, employing careful controls for speaking order, model calls, and task difficulty. The inclusion of multiple models (Claude, GPT-4o-mini, etc.) and diverse task families (familiar science, GSM8K, SciTaT) strengthens the empirical validity of the findings.

The paper is well-structured and easy to follow. Metrics (e.g., false endorsement rate FEtFE_t, assigned-role retention ARAR) are clearly defined, and the extensive appendices provide necessary depth without cluttering the main text.

As multi-agent systems increasingly rely on shared infrastructure (e.g., shared vector databases or logs), understanding how to prevent cascading false beliefs is a critical safety concern. The proposed mitigations offer practical, actionable guidance for system designers.

Critical weaknesses

The liar agents are explicitly prompted to persistently endorse a false claim with high confidence. While this cleanly simulates a compromised agent or prompt injection, it may not fully capture more nuanced, realistic failure modes like subtle hallucinations or emergent biased reasoning.

In larger groups (e.g., 100 agents), outcomes are highly sensitive to the speaking order and placement of liar agents, yielding non-monotonic results. The paper acknowledges this but leaves open the question of how to design robust systems against arbitrary interaction ordering.

Some mitigations (e.g., repetition warnings) show mixed results across tasks(e.g., highly effective for ego depletion but largely ineffective for the PANDAS task). This suggests the interventions are not universally robust, limiting their out-of-the-box applicability.

Questions for authors

Could you elaborate on why certain tasks (like PANDAS) appear more resistant to the proposed mitigations? Identifying the linguistic or structural features of a task that enable or resist lock-in would be a valuable addition to the discussion.

In the 100-agent experiments, the non-monotonic relationship between the number of liar agents and false endorsement is intriguing. A brief hypothesis on the mechanistic reasons for this (e.g., clustering effects of liar agents vs. dilution by early correct rejections) would strengthen the analysis.

The current setup relies on strict, synchronous turn-taking. A short discussion on how these findings might translate to asynchronous, real-world multi-agent systems (where agents read and write to a shared memory concurrently and continuously) would broaden the paper's impact.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.