- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
CAMEL is a memory framework for answering questions over long conversations involving multiple speakers. It preserves original messages in a typed graph and combines hybrid retrieval, query decomposition, graph expansion, and speaker and temporal context to assemble evidence. It improves accuracy over ten baselines on three collaborative-memory benchmarks, with further experiments on answer models, evidence budgets, and component ablations.
Critical strengths
- The paper addresses a useful problem. Keeping track of who said what, when a decision changed, and which messages need to be read together matters for collaborative agents.
Critical weaknesses
-
Most importantly, the baseline comparison could be more up to date. The methods compared are mainly from 2023–2025, although the paper cites newer memory work such as LiCoMemory and Wu et al. from 2026. Including recent methods would help establish whether CAMEL still offers the same advantage over current memory systems.
-
The term-alignment step uses a relevance reranker in both directions to decide whether two terms mean the same thing. Two terms can be closely related without being interchangeable, especially when their meaning changes with the speaker or topic.
Questions for authors
Please see the weaknesses above. I also have the following concerns and questions.
-
The ablation study could be more detailed. Removing Linking and Grounding disables graph expansion, term alignment, and the use of speaker and time information together, so it is unclear how much each part helps. Could the authors test these components separately?
-
Multi-hop retrieval remains a notable limitation given the paper's emphasis on linking complementary evidence. On EverMemBench, multi-hop accuracy is 12.0% at k = 5 and only reaches 13.7% at k = 20. The paper acknowledges this, but a closer analysis of where the evidence chain breaks would be helpful. At which stage does the system lose the evidence needed to answer these questions?
-
The paper reports retrieval time, but the cost of building the memory is still unclear. Although ingestion does not use generative LLM calls, it still requires model inference to build the graph. Could the authors report how long this step takes on the evaluated datasets?
I would be willing to revise my rating if most of my concerns are addressed.