- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This manuscript investigates how shared memory affects the propagation and persistence of false information in multi-agent systems. The authors construct controlled environments in which a subset of agents repeatedly endorses a false claim, while the remaining agents independently evaluate it. The study compares shared memory, personal memory, and live debate across different group sizes, models, and tasks.
The experiments show that shared memory can substantially amplify false endorsements, even when only a minority of agents introduce the incorrect information. The authors further investigate how newly persuaded agents contribute to propagation and evaluate several mitigation strategies, including corrections, uncertainty preservation, and warnings about repeated evidence.
Critical strengths
The manuscript addresses a practically relevant problem. As multi-agent systems increasingly rely on persistent shared memory, understanding how incorrect information can propagate through stored agent outputs is important for their reliability and safety.
The experimental evaluation is extensive. The authors investigate several group sizes, including up to 100 agents, and compare different communication protocols under matched speaking orders and model-call budgets. The additional experiments on neutral-agent visibility and fixed endorsements in live debate provide useful evidence beyond simple observations of false consensus.
Another strength is the investigation of mitigation strategies. Rather than merely identifying the problem, the authors examine how memory representation, provenance warnings, corrections, and uncertainty preservation influence collective decisions. The reported results offer potentially useful insights for designing more reliable shared-memory mechanisms.
Critical weaknesses
My main concern is the limited isolation of the specific contribution of shared memory. Although the authors compare three communication protocols under matched settings, these protocols differ in multiple aspects simultaneously. Shared memory retains only recent non-uncertain judgments, live debate provides a more complete interaction history and allows responses to previous statements, while personal memory excludes other agents' information entirely. Consequently, the results demonstrate differences between communication configurations rather than establishing which properties of shared memory are responsible for the observed false-belief propagation.
The experimental setting is also somewhat artificial. Most experiments introduce misinformation through agents explicitly instructed to repeatedly endorse a false claim, often with high confidence and persistent role instructions. This is useful for controlled analysis but differs from realistic multi-agent workflows, where incorrect information may originate from perception errors, hallucinations, outdated memories, or incomplete observations. Moreover, the main experiments rely heavily on Claude Haiku 4.5 and the ego-depletion question, while the SciTaT evaluation deliberately selects tasks on which the model struggles without the source material. Broader evaluation on naturally occurring agent errors would strengthen the generalization claims.
Another concern is the substantial variability across models and experimental cohorts. For example, three nominally similar six-agent experiments report false-endorsement rates of 100/108, 89/108, and 64/108, respectively. The authors acknowledge this variation, but its underlying cause remains unresolved. Furthermore, the multi-task comparison with Llama 3.1 8B shows that live debate can produce more false endorsements than shared memory. These results suggest that the observed effects are highly dependent on the model, task, and experimental configuration.
Questions for authors
Q1. The three communication protocols differ in information visibility, memory length, and interaction capabilities. Could the authors clarify whether the observed differences persist when agents receive the same previous statements under otherwise comparable conditions? This would help establish whether the effect originates specifically from shared-memory persistence rather than differences in accessible information.
Q2. The manuscript reports substantial variation across nominally similar experiments, including false-endorsement rates ranging from 64/108 to 100/108. Could the authors explain the possible sources of this variability and whether the main conclusions remain robust across independent replications with identical configurations?
Q3. Most experiments involve agents explicitly instructed to persistently endorse false claims. Would similar propagation patterns emerge when misinformation originates from ordinary agent mistakes, such as hallucinations, incorrect tool observations, or outdated memory entries, rather than deliberate and repeated false endorsements?
Q4. How do the authors distinguish persistent false beliefs from temporary conformity to repeated statements? In particular, is there evidence that neutral agents continue endorsing incorrect information after the original misleading agents are removed or after independent contradictory evidence becomes available?
Q5. Several proposed interventions reduce false endorsements, but some primarily increase uncertainty rather than correct rejection. Could the authors provide a more systematic comparison of these strategies across models and tasks, including correct-answer rates and abstention rates? This would help determine whether the proposed interventions improve overall reliability rather than merely suppressing confident responses.