- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes KITE, a vector-free framework for long-term conversational memory. It organizes conversations into structured memories using a governed topic vocabulary, then compiles questions into executable retrieval plans with deterministic lookup and ranking operations. A sparse lexical retrieval complements structured retrieval. Experiments on LoCoMo and LongMemEval-S surpass chosen baselines.
Critical strengths
- Explicit query planning and structured lookup are useful for long-term memory retrieval.
- The method achieves good benchmark results.
Critical weaknesses
- The methodological novelty is not sufficiently convincing. The framework mainly combines established techniques, including structured memory extraction, vocabulary management, executable query planning, deterministic lookup, and lexical retrieval.
- The contribution of vector-free retrieval is insufficiently distinguished from prior work. For example, Beyond Semantic Similarity [1] has already demonstrated effective retrieval without embedding models or vector indexes through direct corpus interaction.
[1] Li, Zhuofeng, et al. "Beyond semantic similarity: Rethinking retrieval for agentic search via direct corpus interaction." arXiv preprint arXiv:2605.05242 (2026).
Questions for authors
- What are the end-to-end token consumption, including memory extraction, vocabulary management, and query compilation? How do these compare with existing systems under consistent evaluation settings?
3 comments
Thank you for the clarifications and additional experiments. My reference to DCI was not intended to request a direct comparison, but to emphasize that vector-free retrieval is not itself novel. KITE's contribution therefore lies mainly in its indexing and vocabulary-governance mechanisms, which I find largely engineering-oriented but not methodologically novel. Thus, I would like to maintain my rating.
KITE’s contributions to conversational memory retrieval
Thank you for clarifying your point. Our methodological contribution is to make it possible to trace why a particular memory was selected as evidence for a question. Three aspects of KITE work together to make this possible.
-
Retrieval as an explicit decision procedure. KITE checks a question’s conditions against stored memories before ranking them. For a fixed index, validated plan, and execution policy, the same records qualify every time. If the initial conditions yield no matches, the executor applies predefined relaxations. If no record qualifies after those steps, the structured path returns an explicit no-match.
-
A governed index that evolves across conversations. KITE admits new topics as they recur across sessions and consolidates redundant ones, redirecting their entries to a common key. This keeps earlier and later mentions of a topic accessible through the same key, while typed fields preserve distinctions such as who participated in an event and when it occurred.
-
A shared index for writing memories and interpreting questions. The query compiler resolves references in a question against the same topic and entity keys used to index the stored facts. Because each fact links back to its source dialogue, we can trace a match from the wording of the question, through the conditions the compiler inferred, to the utterance that supplied the evidence.
We evaluated plan-guided retrieval on LoCoMo by holding the stored memories, reader settings, and reader-context token budget fixed (Table 4). On those same memories, KITE achieved 87.4% evidence recall compared with 79.9% for hybrid retrieval. Answer accuracy was 93.5% and 87.8%, respectively.
Response to V6PF: novelty, vector-free retrieval, and token cost
Thank you for the review. We appreciate the opportunity to clarify the contribution and evaluation scope.
W1: A combination of established techniques?
KITE makes the governed index a shared interface between memory construction and evidence selection (Sections 3.1-3.3). Vocabulary admission and consolidation maintain reusable topic keys. These keys and typed facets organize source-linked facts and ground conditions in executable retrieval plans. The executor checks indexed and temporal conditions before ranking, with explicit relaxation and no-match behavior. The contribution concerns this connection between vocabulary governance, query compilation, and evidence selection, building on established techniques such as structured extraction, vocabulary management, and provenance.
The experiments examine this design at three levels. Table 4 holds the memory store, reader, and reader-context budget fixed: KITE achieves 93.5% LoCoMo accuracy, compared with 85.4% for dense retrieval and 87.8% for hybrid retrieval. Table 5 varies vocabulary-governance policies: fixed vocabulary, immediate expansion, delayed expansion, and full KITE achieve 91.0%, 87.9%, 91.7%, and 93.5%.
In an additional ablation, removing topic/entity labels together with their vocabularies and indexes, while retaining fact text, query-routing logic, and the remaining retrieval rules, reduces accuracy from 93.5% to 88.76%. This measures the contribution of the indexing layer as a whole. Together, these results support the roles of the index, its maintenance, and retrieval over that index.
W2: Beyond Semantic Similarity (DCI)
We agree that being vector-free is not, by itself, a contribution; sparse retrieval such as BM25 has always worked without embeddings. In DCI, an agent searches the raw corpus with terminal tools (grep/rg pipelines, file reads, short scripts). It builds no offline index, enforces exact lexical constraints, and refines its search over many steps. KITE shares the exact matching, and its plans even carry a lexical condition. The differences:
Furthermore, on timing, DCI was posted on arXiv in May 2026 and will appear at NeurIPS 2026 in December, under the ICLR 2027 reviewer guidelines it counts as contemporaneous work. It is clearly relevant, though. We discuss its relationship to KITE in this response.
Q1: End-to-end tokens
For the complete stage-by-stage token and LLM-call breakdown, please see the table under “W3, Q5: Total token cost” in our response to UTnP (2/2). On LoCoMo, KITE uses 8.3k tokens per question: 0.6k for construction, 5.8k for query compilation and search, and 1.9k for answering. EverMemOS uses 12.8k: 6.1k, 2.9k and 3.8k. Both count input plus output tokens. Construction includes extraction, consolidation and vocabulary maintenance, and is divided over the same 1,540 questions.
The settings are only partly consistent. We used the totals EverMemOS published rather than rerunning it, and its counts leave out its 4B embedding and reranking models. EverMemOS is the only compared system that publishes a stage-level breakdown with the same backbone, which is why we compare against it. We have not done this accounting on LongMemEval-S, where construction is not amortized.