- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper's starting point is a gap that is easy to miss: agent memory systems are built for a single agent's stream, so when a team of LLM agents shares one store across hosts, nothing actually specifies what a reader on another host sees — and a central store used this way silently drops one side of every concurrent contradictory write, because the database underneath resolves it by last-write-wins with no signal. The authors reframe shared agent memory as a classical replicated-data problem and adopt the matching contract from distributed systems: causal consistency plus the three session guarantees (read-your-writes, monotonic reads, writes-follow-reads). They map each guarantee onto a concrete agent-team failure mode — unstable self-knowledge, mid-task regression, and unsound handoff — and show that a delegation is sound exactly when the handoff carries the delegator's causal frontier and the delegate waits until its replica dominates it (Proposition 1).
MemSwarm is the implementation. Each agent host runs a full local replica of an append-only event log; records are immutable and bi-temporal, stamped with hybrid logical clocks; replicas merge by CRDT (a remove-wins OR-Set over records, with a multi-value register per superseded id) via anti-entropy with acknowledged cursors; and — the design choice that carries the paper — a concurrent contradiction is not resolved but survives as an explicit multi-value for an upstream policy to settle. The memory verbs are exposed as an MCP server.
The evaluation asks three things and runs entirely in real OS processes on one machine over loopback. On staleness: 200ms anti-entropy gives roughly 100ms median cross-replica visibility, percentiles tune with the sync interval, and no write is ever lost across 23,200 samples. On conflict surfacing: against an LWW ablation on identical replicated state, the multi-value register surfaces the contradiction on every replica 100/100 while LWW destroys the losing side 100/100. On session guarantees: read-your-writes and the frontier-waited handoff hold in all trials, and the only violation is a roaming reader under degradation (26%), which pinning or frontier-enforced reads eliminate at the cost of blocking about 0.4% of reads. A controlled handoff task then connects the systems property to an LLM outcome: shown both values and told to flag conflicts, haiku flags 91% of the time; drop that instruction and it picks one side in 20/20, re-implementing LWW; and a central store with the same records flags none. Under a single failure, a replica crash leaves the other agents at 600/600 where a central-store crash fails everyone 0/600, at the price of staleness until heal. The authors release the MCP server, a nine-benchmark suite, and raw logs.
Critical strengths
The reframing is clean, correct, and under-served. The paper's real contribution is a lens: shared multi-agent memory is a replicated-data problem, a central store used naively is a last-write-wins register that silently discards concurrent contradictions, and the classical session guarantees map one-to-one onto agent-team failure modes. The fourteen-system survey makes a credible case that no existing agent-memory system treats cross-agent consistency as a first-class contract — they stop at one store or inherit whatever the backing database does. This is the kind of consolidation I'd cite and build on, and it is genuinely useful even though the authors are upfront that the underlying mechanics are not new.
An honest, well-controlled demonstration that the design choice matters. The paper does not just assert that multi-values beat last-write-wins; it shows it on identical replicated state (100/100 surfaced versus 100/100 destroyed), then carries the point all the way to an LLM outcome and shows the harder truth — that surfacing a conflict only helps if the consumer's contract asks it to keep the conflict, and that a central store with the same records cannot even detect it because detection needs each writer's applied context, which only a per-host replica tracks. The candor throughout (co-load disclosure, the ordering confound, conceding the central-datacenter case outright) makes the directional claims trustworthy rather than oversold.
Critical weaknesses
The technical core is imported distributed-systems machinery, and the paper says so. The replication layer — hybrid logical clocks, a remove-wins OR-Set, multi-value registers, anti-entropy with acked cursors, the Bayou/Dynamo/COPS lineage — is textbook, and the Appendix C proofs are standard session-guarantee arguments. The authors are explicit that the replication mechanics and the contract are established, and that what is new is their agent-facing framing, substrate, and benchmark. I think that framing-plus-substrate-plus-benchmark is a legitimate contribution, but it does mean there is no new algorithm or theorem here, and for a reader weighing conceptual depth this is the main thing keeping the paper from a clearer accept. The value is in the mapping and the demonstration, not in the method.
The evidence that it matters for LLM agents is thin and partly confounded. The one tier that makes this an agents paper rather than a systems paper — the controlled handoff task — is also the weakest empirically. It runs essentially on one model family, the per-cell counts are small, and the authors themselves note that the field agent's record is last in 133 of 140 trials, so recency and list position are confounded with the conflict signal that is supposed to drive the 91% flagging rate. The scaling claim is similarly bounded: eight nodes on a single laptop over loopback tells us about protocol fan-out, not about the WAN or cross-organization fleets the motivation invokes. The direction is believable and consistently reported, but the magnitude and the generality to real deployments remain open.
Questions for authors
- On the handoff task's confound: since the field agent's record lands last in 133 of 140 trials, how much of haiku's 91% flagging is the conflict signal versus recency or list position? A re-run with the two conflicting records shuffled in the prompt — or even a split of the runs you already have by where the field-agent record landed — would tell us whether the headline effect survives once ordering is decoupled. This is the result I'd most want pinned down.
- The scaling result (flat p99 from two to eight nodes) is single-machine over loopback and, as you note, bounds protocol fan-out rather than network cost. Could you state plainly in the main text what a reader should and should not take from it — that it is not evidence about WAN or geo-distributed behavior — so the robustness claim is not over-read? No new experiment needed if none is feasible.
- The session guarantees hold for clients that route reads through the thin session layer, while the store serves unconstrained reads to anything that bypasses it and store-side enforcement is left as future work. In a heterogeneous fleet, how realistic is it that every agent routes through the layer, and does a single bypassing agent reintroduce the regressions the contract is meant to remove? A sentence on this deployment precondition would help scope the guarantee.
As the deadline is approaching and no any response from the authors so far, I decide to maintain my original score and assessment.