- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Selective Confluence, a handoff contract that separates advisory semantic memory from canonical records used to support exact commitments. A selector retrieves a claim-relevant record, and a local Merkle membership check controls whether that record is released to the successor. The paper establishes a conditional evidence-confluence property and evaluates coverage, addressing, integrity, and downstream evidence use through offline reconstruction experiments.
Across 24 tasks and five writer realizations, verified recovery improves the registered score from 21/120 to 109/120 under frozen correct record selection. The paper includes valuable controls showing that recovered information supplies most of the accuracy benefit, while verification enforces a narrower integrity boundary. My main concern is whether this carefully documented decomposition establishes sufficient additional scientific and practical value beyond source revisitation and authenticated storage. The authority policy is also more fully specified than enforced by the evaluated successors, and the registered metric does not consistently capture faithful task outcomes.
Critical strengths
-
Clear separation of distinct failure mechanisms. The distinction between coverage, addressing, membership integrity, and evidence use is useful (Sections 3–5). The paper correctly explains that authentic records may still be irrelevant or false, and that invariant evidence does not automatically imply invariant model decisions.
-
Informative matched controls and negative results. Table 5 is particularly helpful: receipt-free episode traces achieve 52/60, clean exact records achieve 53/60, and verified exact records achieve 54/60. The authors correctly avoid attributing this one-cell difference to improved reasoning from verification. The sufficient-capacity structured control and the Mem0 comparison also establish meaningful limits on when recovery adds information.
-
Transparent experimental and statistical reporting. The confirmation retains all five writer realizations and prospectively extends to the remaining tasks. The analysis reports writer-level gains, task directions, and crossed task–writer resampling rather than treating all 120 cells as independent samples (Appendices C.5 and C.9). Null results and the incomplete second-model natural run are disclosed appropriately.
-
Correctly scoped formal guarantees. I found no substantive error in the two propositions in Appendix A. The assumptions about canonical archives, authenticated roots, selection, and downstream decision rules are explicit.
Critical weaknesses
- The additional contribution beyond existing source recovery is limited
Source revisitation is already an established response to lossy memory; for example, ReadAgent combines gist memories with lookup of original passages. The paper acknowledges this prior work and adds an explicit authority contract and a standard authenticated-record interface.
However, Proposition 2 assumes an order-independent archive and index, the same selected relevant key, unique materialization, and a fixed record-grounded decision rule. Evidence and decision invariance then follow directly from identical inputs. This is useful as a specification, but it does not establish how the difficult selection and decision assumptions are achieved.
The empirical controls reinforce this concern: ordinary receipt-free source recovery supplies nearly all clean utility, while the alteration experiment demonstrates the expected behavior of a correctly implemented membership gate. The paper therefore establishes its component decomposition more strongly than a new capability for agent handoffs. A direct demonstration of the complete authority contract, including its decision rule, would strengthen the contribution. ReadAgent, ICML 2024.
- The headline gain conditions on correct selection of an unusually informative record
The 21/120→109/120 comparison uses frozen correct record selection, and the designated support record is the native evaluator receipt containing the recorded outcome (Appendix B; Appendix C.5). This is a legitimate evidence-sufficiency experiment, but it does not estimate the performance of the complete retrieval-and-decision system.
The additional routing experiments are useful, although their results depend strongly on the available addressing information. Sparse target-derived keys achieve 12/12 selection, whereas the hint-free index achieves 1/12. The local Qwen full stack selects the correct record for 3/12 tasks and achieves 15/60 correct outputs; Mem0 and Mem0 plus recovery both achieve 8/12 under the registered score (Sections 5.5 and Appendix D.7).
These results leave the practical benefit over a competent memory-and-retrieval baseline uncertain. The paper should clarify how routing metadata is constructed independently of future queries and show what operational records substitute for evaluator receipts. This is especially important where a receipt supplies a corrected interpretation absent from the actor trace (Appendix D.5).
- The evaluated system does not fully enforce its commitment-authority rule
The contract declares semantic memory non-authoritative for commitment-bearing claims. Nevertheless, the wrong-but-valid-record experiment produces 12/60 correct answers supported by semantic memory rather than the released record (Section 5.5; Appendix D.6). Four responses in the broader routing panel also cite record IDs unavailable in their condition.
Furthermore, the local full-stack experiment deliberately calls the successor after invalid selection instead of implementing the stipulated hard-DEFER branch (Appendix C.7). These limitations are candidly acknowledged and do not contradict the conditional theorem. They do, however, mean that the experiments demonstrate a record-integrity gate plus prompted adherence, rather than enforcement of the complete authority contract.
For the existing task family, the authors should demonstrate a successor/action rule that satisfies the contract and report evidence-reference compliance alongside answer correctness. Testing decisions across differing memories with the same verified record would directly evaluate the claimed decision invariance.
- The primary metric incompletely measures faithful outcome reconstruction
The registered exact-term or status/action rule can classify correctly described negative outcomes as DEFER. Conversely, exact-term matching can credit an answer that falsely reports task completion. The Mem0 audit illustrates this distinction: both registered scores are 8/12, while outcome fidelity changes from 7/12 to 10/12 (Appendix D.7).
Consequently, “109/120 correct decisions” should be interpreted strictly under the registered scoring rule. Its relationship to faithful task outcomes and authorized commitments is not fully quantified.
The authors should retain the registered score and add a clearly labeled outcome/status audit over the complete primary panel, including failed episodes. This could use the saved outputs and native outcome records without additional model calls. Reporting correct negative answers, unsupported positive commitments, and deferrals separately would clarify the practical meaning of the result.
Questions for authors
What behavior or measured benefit distinguishes the complete contract from the same source-revisitation system using canonical records and ordinary trusted storage?
How are the sparse routing keys generated, and which metadata would be available before the future claim? What operational evidence would replace native evaluator receipts?
Can the tested successor enforce both hard deferral after failed lookup and the rule that semantic memory cannot independently authorize a commitment? What are the primary panel’s evidence-reference compliance rates?
What are the complete 24-task results under a uniform outcome/status audit, particularly for recorded failures and false completion claims?
4 comments
Authors' Response (2/3) for Weakness 2 and Question 2
Thank you again for asking what replaces the evaluator receipt in practice. Below we show archives built only from operational observations and their results.
Weakness 2. [...] The paper should clarify how routing metadata is constructed independently of future queries and show what operational records substitute for evaluator receipts. This is especially important where a receipt supplies a corrected interpretation absent from the actor trace (Appendix D.5).
Question 2. How are the sparse routing keys generated, and which metadata would be available before the future claim? What operational evidence would replace native evaluator receipts?
Our answer: archives built only from operational observations, indexed by metadata recorded during the session, support autonomous recovery.
Routing keys. Appendix D.6 derived its keys from the task instruction. The new archives use no task text.
Operational records. The archives contain native browser and API observations, without evaluator outputs, actor reasoning or actor answers. They replace the receipt:
- GitLab: contributor pages and API responses.
- Shopping Admin: filtered order and review pages.
- Salesforce: creation confirmations and account-contact views.
Metadata available before the claim. The index stores only the observed URL, page title, filter keys and timestamp. All are recorded when the agent visits the page, before any future claim exists.
Corrected interpretations. The archive stores observations, not actor reasoning, so an actor's misreading cannot enter the evidence. The rule reads the observed page value.
Results. Over 120 genuine historical sessions, the learned selector finds the target session in 22/24 queries, under both the original and a paraphrased wording, and in 10/12 on the tasks of Appendix D.6. The full contract then commits on all 22 claims that the native records support, with no unsupported commitment. The two Salesforce creation requests without an observed creation are deferred.
Full answers with the paper's primary model. With GPT-5.6-sol reading the retrieved records, and a strict criterion that requires every stated fact to be grounded in a retrieved record, learned retrieval matches oracle routing (24/48 versus 22/48). It cuts unsupported commitments from 42/48 with index access alone to 14/48. On typed claims, the executable rule makes no unsupported commitment.
Transfer to unseen entities. With all rules frozen, we added 16 new repository entities. The learned selector finds the target in 96/96 present cases and returns NONE in 96/96 withheld cases. Over all 40 tasks and entities, the rule commits on 37 and defers on 3, with no unsupported commitment.
For deployment, an agent needs only the observation log it already produces. We will add these archives and results to Section 5.5 and Appendix D.6.
Authors' Response (3/3) for Question 1 and Question 4
Thank you again for these questions on confluence and positioning. Below we answer both, starting with order effects.
Question 4. What evidence demonstrates order invariance independently of writer variability? In particular, can the authors distinguish the effects of acquisition order on memory coverage, selected records, and final decisions, and clarify how often the proposed guarantees hold when selection is performed by learned components?
Our answer: order changes memory and memory-only decisions, and recovery removes that change. A bounded sequential Qwen3.8-27B writer updates memory one session at a time within the same limits. We crossed forward and reverse order with three seeds and reran each chain unchanged.
- Writer variability. Identical order and seed reproduce byte-identical memories, with no decision change.
- Memory coverage. Order changes which primary fields survive, by up to 12 of the 98 fields for the same seed.
- Selected records. The selector reads only the query and the index, so every order selects the same record.
- Final decisions. Reversing the order flips the memory-only decision on 19 of the 24 tasks. With recovery, it flips on none. The executable rule returns identical outputs across all memories.
- Same record, different memories. With record, query and seed fixed, switching between empty memory and a generic completion claim changes the Qwen3.8-27B decision in 0/48 (ordinary) and 1/48 (strict) clean pairs. The rule's output never changes.
- Learned selection. Proposition 2 requires a stable relevant selection. The learned selector returns the same target across candidate orders and wordings in 96/96 held-out cases, and finds the target in 22/24 queries over 120 genuine sessions.
With recovery, a long-running agent reports a past outcome the same way, however its history was summarized.
Question 1. How does Selective Confluence differ technically from TierMem, MemWeaver, and APEX-MEM [...]? In particular, what new guarantees does the proposed authority contract provide beyond existing provenance-aware retrieval and tiered memory architectures?
Our answer: these systems decide what to retrieve. Our contract decides which representation may authorize a commitment. As the paper states, “The contribution is the interface that connects claim eligibility, evidence authority, and release, using established retrieval and Merkle components.” MemGPT [1] and MemoryOS [2] manage memory tiers. TierMem [3] routes between summaries and raw evidence. MemWeaver [4] links graph, experience and passage memory to sources. APEX-MEM [5] keeps temporal history with structured and hybrid retrieval. The contract adds three guarantees.
- Memory cannot authorize an exact commitment. The executable rule takes only the query and one bound record.
- Evidence must be sufficient, not only authentic and relevant. On authentic records that lack the needed observation, GPT-5.6-sol defers, while Qwen3.8-27B readers still commit in 10/144 (ordinary) and 3/144 (strict) cases. The rule defers on every such record for any reader, without a model call, and keeps the clean accuracy of the prompted readers.
- Failure ends in DEFER before any model call. A missing key, a mismatched entity or a rejected proof never reaches the successor.
Each of these systems can keep its memory manager and add the contract for commitment-bearing claims. We will add this comparison to Section 2.
References
[1] Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, and Joseph E. Gonzalez. MemGPT: Towards LLMs as operating systems. arXiv:2310.08560, 2023. https://arxiv.org/abs/2310.08560
[2] Jiazheng Kang, Mingming Ji, Zhe Zhao, and Ting Bai. Memory OS of AI Agent. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 25961–25970, 2025. https://doi.org/10.18653/v1/2025.emnlp-main.1318
[3] Qiming Zhu, Shunian Chen, Rui Yu, Zhehao Wu, and Benyou Wang. From Lossy to Verified: A Provenance-Aware Tiered Memory for Agents. arXiv:2602.17913, 2026. https://arxiv.org/abs/2602.17913
[4] Juexiang Ye, Xue Li, Yang Xinyu, Chengkai Huang, Lanshun Nie, Lina Yao, and Dechen Zhan. MemWeaver: Weaving Hybrid Memories for Traceable Long-Horizon Agentic Reasoning. In Findings of the Association for Computational Linguistics: ACL 2026, pp. 12928–12956, 2026. https://doi.org/10.18653/v1/2026.findings-acl.630
[5] Pratyay Banerjee, Masud Moshtaghi, Shivashankar Subramanian, Amita Misra, and Ankit Chadha. APEX-MEM: Agentic Semi-Structured Memory with Temporal Reasoning for Long-Term Conversational AI. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 16470–16489, 2026. https://doi.org/10.18653/v1/2026.acl-long.749
Dear Reviewer,
As the deadline for the discussion period approaches, we kindly request your feedback on our response. Your insights would be greatly appreciated.
Thank you for your time and consideration!
Authors' Response(1/3) for Weakness 1, Question 1 and Question 3
Thank you for your careful review and for recognizing the separation of failure mechanisms, the matched controls in Table 5, our transparent statistics and the correctness of both propositions. You asked for a direct demonstration of the complete contract, operational evidence beyond evaluator receipts and a uniform outcome audit. We built an executable commitment rule and ran each requested test.
Our answer: the contract adds an executable commitment rule that makes authority independent of the deployed model. The rule takes a typed query and one bound record. It checks entity and field scope, then outputs only extracted fields with the bound record ID. It has no memory input. Missing, invalid, mismatched or insufficient evidence returns DEFER.
The complete contract end to end. Over 120 genuine historical sessions, the learned selector, membership verification and the rule run together without oracle routing. They commit on all 22 claims that the native records support, with no unsupported commitment and no successor call. When the selector picked another person's session, the rule deferred.
Matched comparison with source revisitation. Prompted readers receive the same canonical record and answer the same typed claims:
The contract therefore differs from source revisitation in three measured ways.
Proposition 2 sets a design target: make the authorized input identical across memories. Our experiments now achieve both of its assumptions. The learned selector returns a stable relevant record (22/24 over 120 sessions, 96/96 across candidate orders and wordings), and the executable rule is the fixed record-grounded decision rule.
Our answer: yes. Both rules are now enforced in code.
Verified recovery turns citations from labels into checkable evidence. With the executable rule, compliance is complete by construction.