acceptodds
Under review as a conference paper at ICLR 2027

MemSwarm: A Distributed Memory Architecture for Persistent Context Sharing Across Multi-Agent Systems

Abstract

Agent memory systems are built for one agent's stream. When a team of large language model (LLM) agents shares memory across hosts, nothing specifies what a reader on another host sees, and a deployed agent store used centrally silently loses one side of every concurrent contradictory write. We treat shared agent memory as a replication problem and adopt the classical contract of causal consistency plus session guarantees (read-your-writes, monotonic reads, writes-follow-reads), under which a handoff is sound once the delegation carries the delegator's causal frontier and the delegate waits for it. MemSwarm implements the contract with no coordination on writes: each agent host keeps a local replica of an event-sourced log, hybrid logical clocks version the records, conflict-free replicated data types merge them, and a concurrent contradiction survives as an explicit multi-value for upstream policy to resolve. Multi-process experiments on one machine over loopback show zero lost writes in 23,200 visibility samples, conflicts surfaced on every replica where last-write-wins destroys them without a signal, and monotonic-read violations in 26% of degraded roaming-reader trials that frontier-enforced reads remove by blocking 0.4% of reads. On a controlled handoff task, LLM readers shown both values and told to report conflicts flag them in 91% of trials; without that instruction haiku picked one side in 20/20 trials, re-implementing last-write-wins in the prompt. A central node with the same records loses the conflict as last-write-wins does, since detection uses what the writer's store had applied, which only a per-host replica tracks for its agent. Under a single failure, replicas serve a cut-off agent 600/600 operations where a central store serves 0/600, at the price of staleness until heal. We release a Model Context Protocol server, the benchmark suite, and raw logs.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
multi-agent systems, LLM agents, shared memory, agent memory, distributed systems, causal consistency, session guarantees, CRDTs, Model Context Protocol, benchmark

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_psxFOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper's starting point is a gap that is easy to miss: agent memory systems are built for a single agent's stream, so when a team of LLM agents shares one store across hosts, nothing actually specifies what a reader on another host sees — and a central store used this way silently drops one side of every concurrent contradictory write, because the database underneath resolves it by last-write-wins with no signal. The authors reframe shared agent memory as a classical replicated-data problem and adopt the matching contract from distributed systems: causal consistency plus the three session guarantees (read-your-writes, monotonic reads, writes-follow-reads). They map each guarantee onto a concrete agent-team failure mode — unstable self-knowledge, mid-task regression, and unsound handoff — and show that a delegation is sound exactly when the handoff carries the delegator's causal frontier and the delegate waits until its replica dominates it (Proposition 1).

MemSwarm is the implementation. Each agent host runs a full local replica of an append-only event log; records are immutable and bi-temporal, stamped with hybrid logical clocks; replicas merge by CRDT (a remove-wins OR-Set over records, with a multi-value register per superseded id) via anti-entropy with acknowledged cursors; and — the design choice that carries the paper — a concurrent contradiction is not resolved but survives as an explicit multi-value for an upstream policy to settle. The memory verbs are exposed as an MCP server.

The evaluation asks three things and runs entirely in real OS processes on one machine over loopback. On staleness: 200ms anti-entropy gives roughly 100ms median cross-replica visibility, percentiles tune with the sync interval, and no write is ever lost across 23,200 samples. On conflict surfacing: against an LWW ablation on identical replicated state, the multi-value register surfaces the contradiction on every replica 100/100 while LWW destroys the losing side 100/100. On session guarantees: read-your-writes and the frontier-waited handoff hold in all trials, and the only violation is a roaming reader under degradation (26%), which pinning or frontier-enforced reads eliminate at the cost of blocking about 0.4% of reads. A controlled handoff task then connects the systems property to an LLM outcome: shown both values and told to flag conflicts, haiku flags 91% of the time; drop that instruction and it picks one side in 20/20, re-implementing LWW; and a central store with the same records flags none. Under a single failure, a replica crash leaves the other agents at 600/600 where a central-store crash fails everyone 0/600, at the price of staleness until heal. The authors release the MCP server, a nine-benchmark suite, and raw logs.

Critical strengths

The reframing is clean, correct, and under-served. The paper's real contribution is a lens: shared multi-agent memory is a replicated-data problem, a central store used naively is a last-write-wins register that silently discards concurrent contradictions, and the classical session guarantees map one-to-one onto agent-team failure modes. The fourteen-system survey makes a credible case that no existing agent-memory system treats cross-agent consistency as a first-class contract — they stop at one store or inherit whatever the backing database does. This is the kind of consolidation I'd cite and build on, and it is genuinely useful even though the authors are upfront that the underlying mechanics are not new.

An honest, well-controlled demonstration that the design choice matters. The paper does not just assert that multi-values beat last-write-wins; it shows it on identical replicated state (100/100 surfaced versus 100/100 destroyed), then carries the point all the way to an LLM outcome and shows the harder truth — that surfacing a conflict only helps if the consumer's contract asks it to keep the conflict, and that a central store with the same records cannot even detect it because detection needs each writer's applied context, which only a per-host replica tracks. The candor throughout (co-load disclosure, the ordering confound, conceding the central-datacenter case outright) makes the directional claims trustworthy rather than oversold.

Critical weaknesses

The technical core is imported distributed-systems machinery, and the paper says so. The replication layer — hybrid logical clocks, a remove-wins OR-Set, multi-value registers, anti-entropy with acked cursors, the Bayou/Dynamo/COPS lineage — is textbook, and the Appendix C proofs are standard session-guarantee arguments. The authors are explicit that the replication mechanics and the contract are established, and that what is new is their agent-facing framing, substrate, and benchmark. I think that framing-plus-substrate-plus-benchmark is a legitimate contribution, but it does mean there is no new algorithm or theorem here, and for a reader weighing conceptual depth this is the main thing keeping the paper from a clearer accept. The value is in the mapping and the demonstration, not in the method.

The evidence that it matters for LLM agents is thin and partly confounded. The one tier that makes this an agents paper rather than a systems paper — the controlled handoff task — is also the weakest empirically. It runs essentially on one model family, the per-cell counts are small, and the authors themselves note that the field agent's record is last in 133 of 140 trials, so recency and list position are confounded with the conflict signal that is supposed to drive the 91% flagging rate. The scaling claim is similarly bounded: eight nodes on a single laptop over loopback tells us about protocol fan-out, not about the WAN or cross-organization fleets the motivation invokes. The direction is believable and consistently reported, but the magnitude and the generality to real deployments remain open.

Questions for authors
  1. On the handoff task's confound: since the field agent's record lands last in 133 of 140 trials, how much of haiku's 91% flagging is the conflict signal versus recency or list position? A re-run with the two conflicting records shuffled in the prompt — or even a split of the runs you already have by where the field-agent record landed — would tell us whether the headline effect survives once ordering is decoupled. This is the result I'd most want pinned down.
  2. The scaling result (flat p99 from two to eight nodes) is single-machine over loopback and, as you note, bounds protocol fan-out rather than network cost. Could you state plainly in the main text what a reader should and should not take from it — that it is not evidence about WAN or geo-distributed behavior — so the robustness claim is not over-read? No new experiment needed if none is feasible.
  3. The session guarantees hold for clients that route reads through the thin session layer, while the store serves unconstrained reads to anything that bypasses it and store-side enforcement is left as future work. In a heterogeneous fleet, how realistic is it that every agent routes through the layer, and does a single bypassing agent reintroduce the regressions the contract is meant to remove? A sentence on this deployment precondition would help scope the guarantee.
1 comment

As the deadline is approaching and no any response from the authors so far, I decide to maintain my original score and assessment.

Reviewer_4Ep3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces a problem about multi-agent’s system memory where there is a shared memory, and each agent holds a local replica copy of this shared memory without any central memory. In such a system, there are 2 types of failures: a) When the writer agent has completed writing a new memory to the local replica, but it has not been synced with the other local copies, this is stale propagation. b) When multiple concurrent writes on the same data of the shared memory takes place then it can lead to loss of data if only one of them is chosen, this is a silent contradiction.

The distributed memory is based on three principles:

  • Read Your Writes (RYW): A read by an agent that comes after its own write must see that or a newer write. Memswarm writes onto a local replica of the memory first so that it is guaranteed.
  • Monotonic Reads (MR): A read operation will always see at least the previous version of a data.
  • Writes follow Reads (WFR): A write operation should be causally ordered after the reads that ordered it. That means that if an agent A first reads task state, then writes findings m1 …mk, those writes must be ordered after everything ‘A’ reads.

The paper uses ideas in classical database theory of database replication and proposes a 5-layer approach to distributed memory problems in LLM agent systems. The 5-layer ideas are:

  • Layer 1: Each data record has 2 identifiers: a. A hybrid logical clock with a timestamp in ms, a logical counter, and a node ID. When the data is being written. b. An interval showing when the data is valid from and valid until. Some operation rules decide how this is operated. Supersede closes the interval’s [valid until] and adds a new record with the above new identifiers. So records are immutable.
  • Layer 2: A log file is maintained where data is only appended with 3 operations: observe, supersede, and forget. The current memory state is observed by performing some function called fold over this log.
  • Layer 3: Data is pushed to peers (anti-entropy), and also multiple concurrent versions of data are stored. A function called conflicts() is used to find conflicts of these multiple versions and leaves resolution to an upstream policy (how it is not mentioned).
  • Layer 4: Functions that track per-writer write counts called the frontier, which is used to decide consistency.
  • Layer 5: Interface used to expose the functions put, get, supersede, etc. This is exposed as newline JSON over TCP or MCP tools.

The authors conduct the following experiments:

  • They test for staleness in memory in an emulated scenario where replication across all the local replica memories is synced every 200 ms and polling is done among the agents every few ms. They find that, on 23200 writes, the data is delayed but never lost because each replicated batch must be acknowledged before the sender moves the cursor.
  • They test for write conflicts by multiple agents when it is being synced. They found that Memswarm can detect conflict 100/100 times that conflicting write signals were written, while Last Write Wins (LWW) is unable to track any of the 100 conflicts.
  • They test if the change in write changes can check for conflicting memory. They write 8 facts into the memory and then change 3. Now, with a second agent during a network partition, they write a conflicting fact and do this experiment 600 times. The system must answer correctly, or if there is conflicting information, the agent must say there is a conflict. This is where an LLM is used in smaller separate runs. - The scripted runs detect the conflict in all 600 runs while the LLM system detects 91 % of the times only when instructed.
Critical strengths

S1. The paper successfully identified an important problem: most memory systems are built for a single agent’s interaction stream. Multi-agent memory is often stored as a single centralized memory that many agents read from or write to. The idea of keeping a local memory workspace for each agent, which is then synced across all other agents’ local memories, is an old idea in distributed databases being applied to multi-agent LLM systems. The paper is explicit about what is borrowed and what they are introducing.

S2. They have performed multiple experiments showcasing how the claims are tested and show that the design bounds staleness and catches conflicts that LWW loses.

Critical weaknesses

W1. The paper uses classical distributed database concepts for LLM multi-agent memory. Their semantic memory retrieval is left unevaluated, which makes this paper of questionable interest to the machine learning community. This reads more like an engineering paper rather than a machine learning research paper.

W2. The LLM system where the prompt says to say conflict in case there are conflicting writes of the same record, it is just an LLM with instruction following prompt. The results vary quite a lot by model (Haiku is 91 % but Sonnet is much less). I see this more as a prompt engineering failure, and I don’t see the point of it while a deterministic system is able to surface conflict much more robustly.

W3. The paper mentions leakage of data due to missing access control, one of the failures of distributed memory systems, but I don’t see any experiment testing or evaluating leakage of the data.

W4. The paper is very dense in its writing, and it is hard to follow. Simpler writing could lead to a better understanding of the paper’s hypothesis as well as a better understanding of why this is novel. I think this paper can be substantially improved with better writing and motivation, and a stronger scientific question and submitted for the next conference.

Questions for authors

Q1. The paper states that there is a plan to release the MCP server and logs. Is there any plan to release this memory system as part of any coding harness or existing system and show end-to-end results against the harness’s default memory? That way, we can see real benefits (like wall clock time, real concurrency issues) on a larger-scale task rather than just the small controlled task shown here.

Q2. Can you please present a real use case where such a system might be implemented? Currently, this system reads like a hypothetical scenario. Although the paper does mention all the other memory systems and why they fall short in comparison to memswarm, I would like to know what other systems can be used to implement the same features and why they are inefficient, incapable, or too complex (e.g., Redis).

Q3. LLM inferences are typically measured in seconds, but the write conflict presented here looks like a much smaller timescale. Can you present a comparison of end-to-end time taken to complete memory reads/writes with the LLM inferences as well? Also, were the conflicts written with real LLM writers or simulated writers ?

Reviewer_ZnK2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MemSwarm, which is a local-first multi-agent memory architecture, in order to address silent data loss and stale context across distributed agent fleets. The key ideas include (1) enforcing causal consistency across agent handoffs using causal frontiers, (2) coordinating asynchronous log merges via conflict-free replicated data types, and (3) exposing concurrent conflicting writes as multi-value siblings for agentic resolution. Evaluation shows that MemSwarm eliminates lost writes, prevents stale read regressions, preserves contradictory updates, and guarantees partition availability.

Critical strengths

Prior multi-agent memory systems treat context sharing as an ad-hoc centrelized database problem. That can suffering from stale reads, silent update loss, and partition vulnerability. The proposed MemSwarm reframes fleet-scale agent memory as a relpicated data problem. By enforcing causal consistency and session gaurantees coordination-free via causal frontiers during delegations, it provides a mathematically sound, local-first base for distributed agent architectures.

In concurrent agent workflows, contradictory updates are typically broken by naive Last-Write-Wins hueristics without a signal to the application. The paper preserves concurrent conflicting writes as explicit multi-value siblings using conflict-free replicated data types. This seems to be quite effective.

Critical weaknesses

However, I also have a few concerns.

First, the evaluation seems to confeined to a single physical machine over the local loopback interface. While simulated fault injection demonstrates basic correctness, it can lead to much performance trade-offs of causal consistency across distributed networks. In production spanning cross-region cloud instances or edge devices, network latency synchranization may lag could force delegate hosts to suffer lots of head-of-line blocking while waiting for causal frontiers. That can undermine coordination-free execution.

Second, the proposed design defers conflict resolution to the downstream LLM by storing contredictory writes as multi-value siblings, which may create unbounded context inflation and increasing token costs during sustained write contention. Also the paper shows that unless explicitly prompted to detect contradictions, LLMs arbitrarily choose one branch anyway. Without background garbage collection or semantic compaction mechanisms, the framework may just relocate the Last-Write-Wins patholagy from the database into the prompt window.

Third, the architecture does not consider access control, authorization, and data isolation. MemSwarm's design relies on replicating the full event-sourced log to every local agent host. In real-world multi-agent deployments, particularly enterprise environments or heterogeneous swarms with tiered agent permissions, agents possess different privilege boundaries and private contexts. Because the whole log is synchronized across all hosts, compromised or low-privilege worker node can obtain unrestricted visibility into the complete operational history and private reasoning traces of the entire fleet. As such, the design can create a critical data-leakage and multi-tenant vulnerability.

Fourth, there is a fundamental semantic mismatch between syntactic CRDTs and unstructured natural language memory. MemSwarm detects concurrency and divergence at the level of discrete keys, multi-value registers, and JSON fields. However, agent memory is inherently relational and semantic. Two agents writing the exact same factual observation using slightly different phrasing will be flagged as a structural conflict and generate redundant multi-value siblings, inflating the state space unnecessarily. As such, if two agents write logically contradictory statements across different memory keys or graph attributes, the syntactic CRDT layer will be blind to the semantic inconsistency, allowing latent domain contradictions to propagate through the system undetected.

Questions for authors
  1. How does the blocking latency of causal frontier handoffs behave when deployed across high-latency wide-area networks?
  2. How does the architecture prevent unbounded prompt context bloat and garbage collect multi-value siblings during sustained write contention?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.