acceptodds
Under review as a conference paper at ICLR 2027

CAMEL: Collaborative Agentic Memory via Evidence Linking across Speakers and Time

Abstract

Agents in collaborative settings must answer questions using evidence distributed across speakers and time. Participants use different terms for shared concepts, revise decisions, and contribute complementary information across threads. These challenges motivate evidence assembly: retrieving and organizing related messages while preserving the context needed to interpret them. We introduce CAMEL, a collaborative memory framework combining an offline typed graph with a query-time assembly pipeline. The graph retains attributed messages and records reply, decision, and term relations without generative LLM calls at ingestion. At query time, CAMEL seeds candidates through query decomposition and hybrid retrieval, links related evidence through graph expansion and bidirectional contextual term alignment, grounds messages in speaker and temporal context, and selects an evidence window with chronological ordering where appropriate. Across EverMemBench, GroupMemBench, and SocialMemBench, CAMEL improves answer accuracy over the strongest of ten baselines by , , and percentage points. Analyses of components, answer models, and evidence budgets support jointly considering evidence coverage, attribution, and temporal context across collaborative settings.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agent Memory, Collaborative Memory, Evidence Retrieval

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_qRKWOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

CAMEL is a memory framework for answering questions over long conversations involving multiple speakers. It preserves original messages in a typed graph and combines hybrid retrieval, query decomposition, graph expansion, and speaker and temporal context to assemble evidence. It improves accuracy over ten baselines on three collaborative-memory benchmarks, with further experiments on answer models, evidence budgets, and component ablations.

Critical strengths
  1. The paper addresses a useful problem. Keeping track of who said what, when a decision changed, and which messages need to be read together matters for collaborative agents.
Critical weaknesses
  1. Most importantly, the baseline comparison could be more up to date. The methods compared are mainly from 2023–2025, although the paper cites newer memory work such as LiCoMemory and Wu et al. from 2026. Including recent methods would help establish whether CAMEL still offers the same advantage over current memory systems.

  2. The term-alignment step uses a relevance reranker in both directions to decide whether two terms mean the same thing. Two terms can be closely related without being interchangeable, especially when their meaning changes with the speaker or topic.

Questions for authors

Please see the weaknesses above. I also have the following concerns and questions.

  1. The ablation study could be more detailed. Removing Linking and Grounding disables graph expansion, term alignment, and the use of speaker and time information together, so it is unclear how much each part helps. Could the authors test these components separately?

  2. Multi-hop retrieval remains a notable limitation given the paper's emphasis on linking complementary evidence. On EverMemBench, multi-hop accuracy is 12.0% at k = 5 and only reaches 13.7% at k = 20. The paper acknowledges this, but a closer analysis of where the evidence chain breaks would be helpful. At which stage does the system lose the evidence needed to answer these questions?

  3. The paper reports retrieval time, but the cost of building the memory is still unclear. Although ingestion does not use generative LLM calls, it still requires model inference to build the graph. Could the authors report how long this step takes on the evaluated datasets?

I would be willing to revise my rating if most of my concerns are addressed.

Reviewer_xBMWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CAMEL is a query-time evidence-assembly pipeline for multi-party conversational memory. Offline, it indexes raw messages (BM25 + BGE-M3) and builds a heuristic typed graph over messages and surface-extracted terms, admitting SAME-AS edges only when a cross-encoder scores a term pair above a quantile threshold in both directions. At query time it fuses lexical and dense retrieval with LLM query decomposition, expands via two graph hops and alias lookup, adds asker- and date-restricted candidates, reranks under a k-message budget, annotates superseded decisions, orders chronologically, and injects computed date spans. With a fixed answer model at k=5, it reports +12.5/+11.4/+11.7 points over the best of ten baselines on EverMemBench, GroupMemBench and SocialMemBench.

Critical strengths
  1. The problem framing (attribution, temporal validity, evidence completeness) is clear and the “retain raw messages, assemble at query time” position is well argued against ingestion-time compression.

  2. Three benchmarks of very different scale, ten baselines, sub-task breakdowns, three budgets, three backbones, a judge-reliability check; margins hold in every tested setting.

  3. Candid reporting of failures, and code released.

Critical weaknesses
  1. Limited methodological novelty. RRF hybrid retrieval, decomposition, cross-encoder reranking, graph-neighbourhood expansion and metadata filters are standard advanced-RAG components; the one distinctive idea, bidirectional alias admission, is never isolated.

  2. Ablations are too coarse to support the stated contributions. “Linking and Grounding” bundles graph expansion, aliasing, speaker profiles, asker/date retrieval, supersession and span computation, and that bundle is where essentially all the gain lives.

  3. Baseline fairness is underdocumented. Appendix D.1 lists models only; how Mem0/Zep/HippoRAG/RAPTOR outputs were mapped to the 5-message, 32k-character budget is unstated.

Questions for authors

As in weaknesses

Reviewer_sYWHOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CAMEL addresses memory over multi-party conversations through a typed graph preserving messages, speaker attribution, timestamps, and conversational relations. Its query-time pipeline combines hybrid retrieval, query decomposition, graph expansion, terminology alignment, speaker and temporal grounding, and reranking. At a five-message budget, it reports accuracy improvements of 12.5, 11.4, and 11.7 percentage points over the strongest evaluated baselines on EverMemBench, GroupMemBench, and SocialMemBench.

Critical strengths
  1. A useful treatment of collaborative memory. CAMEL jointly addresses speaker attribution, evolving decisions, terminology differences, and distributed evidence while retaining original messages.
  2. Substantial empirical support. Evaluation spans three benchmarks, ten baselines, alternative answer models, and multiple evidence budgets. Component-group ablations and judge verification further support the reported results.
Critical weaknesses
  1. Comparability of auxiliary information is insufficiently clear. CAMEL supplements retrieved messages with participant context, temporal annotations, and computed date spans. It is unclear whether competing systems receive equivalent assistance. Matching retrieved-message count therefore does not necessarily match the information or processing supplied to the answer model, limiting attribution of gains specifically to evidence selection.
  2. Terminology alignment is not directly validated. Bidirectional cross-encoder relevance is used to infer SAME-AS relations, although mutual relevance does not establish equivalence. The linking-and-grounding ablation supports that component group collectively, but does not determine whether these inferred aliases improve performance or introduce erroneous connections.
Questions for authors
  1. Is the indexing done on the message text only or does it include speaker name, timestamps, etc?
  2. Do baselines receive the same participant context, temporal annotations, and date calculations? How do results change when this assistance is matched across methods?
  3. What is the precision of inferred SAME-AS edges, and how does bidirectional scoring compare with embedding similarity or one-directional scoring?
  4. With grounding retained, what incremental gains come from graph expansion and terminology alignment separately?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.