acceptodds
Under review as a conference paper at ICLR 2027

SOLM: SELF-ORGANIZING LATTICE MEMORY FOR LLM AGENTS

Abstract

Long-term memory for LLM agents requires imposing structure on an evergrowing history, but several recent memory architectures pay for that structure with real, per-message LLM calls: A-Mem generates a structured note and re-evaluates evolution for a set of neighboring memories on every insertion, MemTree re-aggregates content across an average of over three ancestor nodes per insertion, and Zep extracts entities, relations, and deduplicates edges via multiple LLM calls per episode. This construction cost scales with memory volume, tying a deployment’s throughput directly to model inference capacity. SOLM’s structure-building (bottom-up placement, quantization-error-driven splitting, cross-branch weak links) needs zero LLM calls, computed from embeddings alone. Its only LLM usage: one fixed call per query for generation, and one optional, cheap small-model call per message for importance scoring; this is the sole cost scaling with volume, far lighter than A-Mem, MemTree, or Zep’s perinsertion costs. Weak links directly address what we call aggregation dilution: a cluster’s centroid can fail to resemble a query even when a member buried inside it would match well, causing that whole branch to be wrongly discarded. (A nongenerative reranking step narrows and reorders an already-retrieved pool at query time and is not counted here; see Section 3.5.) On LongMemEval-S, a controlled ablation (mean over 3 independent runs) shows weak links yield a relative accuracy improvement of 7.7% (68.6% to 74.0%) at matched settings; at nearly identical retrieved-context size, SOLM outperforms a flat similarity-retrieval baseline by a relative 13.8% (mean over 3 runs: 76.0% vs. 66.7%); and accuracy scales predictably with the retrieved-context budget: 76.0%, 83.0%, and 86.0% when the top-5, top-10, and top-15 retrieved memories, respectively, are used as answer context. These results show that competitive long-term memory accuracy does not require scaling LLM usage in proportion to memory volume.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
LLM, Long-term Memory, Memory, Gen AI, Agentic AI, Agentic Memory, RAG

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_mowiOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes SOLM, an embedding-based hierarchical memory system combining bottom-up clustering, quantization-error-driven splitting, cross-branch weak links, and reranking. Experiments on LongMemEval-S and a subset of LongMemEval-M investigate retrieval accuracy and the contribution of weak links.

Critical strengths
  1. Reducing LLM-dependent memory construction is practically relevant. The weak-link ablation includes repeated runs and provides evidence of improvement within the proposed pipeline.
Critical weaknesses
  1. The method largely combines familiar ingredients: centroid-based clustering, binary splitting, associative graph links, beam search, and cross-encoder reranking. The paper does not establish what constitutes a substantive algorithmic advance beyond this combination. The related-work section cites only eight references and omits relevant work on self-organizing clustering, hierarchical vector indexing, and efficient memory systems. Consequently, the claimed contribution is inadequately positioned.

  2. The vanilla-RAG baseline explicitly excludes reranking, whereas SOLM uses a cross-encoder (Section 4.2). Thus, Table 3 cannot attribute the improvement specifically to the lattice. SOLM also receives approximately 18% more context tokens, despite the “nearly matched” characterization. Table 4 imports published results obtained with different generation backbones rather than rerunning systems under a common setup. A flat-retrieval baseline with the same reranker and context budget is essential.

  3. Zero LLM calls for structure construction does not establish low computational cost. Embedding inference, cluster maintenance, weak-link construction, and cross-encoder scoring still incur costs. The paper provides no systematic end-to-end latency, ingestion throughput, memory-footprint, or scaling comparison against baselines. Its batch construction of weak links also leaves recurring maintenance costs in continuously operating agents untested. The claimed deployment advantage therefore remains largely unsubstantiated.

  4. Experimental coverage and component validation are incomplete. Evaluation remains within one benchmark family, the LongMemEval-M experiment covers only 120 questions, has no competitive baseline, and disables importance scoring. Although weak links are ablated, cluster splitting is not, and Table 7 shows no aggregate accuracy improvement from the importance shortcut at the reported setting. These omissions leave substantial parts of the proposed architecture unvalidated and provide limited evidence of generalization.

  5. Presentation and technical explanations require substantial revision. Section 4.3 describes primarily intuition-based parameter selection, whereas Appendix A.3 states that parameters were selected through manual exploration against the evaluation set. The embedding checkpoint is unspecified, and Figure 1 is merely schematic, without illustrating splitting or an actual pruning-recovery example. More importantly, Section 6 treats session-level recall as evidence that the correct content reaches generation, although retrieving a message from the correct session does not guarantee retrieval of the answer-bearing evidence.

Questions for authors

N/A

Reviewer_LJkFOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes SOLM, a hierarchical memory structure for LLM agents. Messages are inserted into embedding-based clusters, heterogeneous clusters may be split using a quantization-error criterion, and cross-branch weak links are added to recover memories that might be missed during top-down centroid-based retrieval. The system also includes an optional LLM-based importance score, an expanding beam search, and a cross-encoder reranker.

Critical strengths
  1. The main intuition is easy to understand.

  2. Well organized.

Critical weaknesses
  1. The technical novelty is unclear. Bottom-up online clustering, centroid updates, quantization-error-based splitting, nearest-neighbor graph connections, beam search, and cross-encoder reranking are all familiar ideas.

  2. SOLM uses Gemini Flash 3.5, whereas most of the reported baselines use GPT-4o-mini. Although the paper acknowledges this confound, the comparison is not meaningful for establishing the relative quality of the systems. A controlled comparison against other baselines using the same generation model is essential.

  3. The weak-link ablation is the only relatively clean experiment, but it establishes only that weak links help within the authors’ pipeline. It does not validate the complete SOLM system against a strong alternative.

  4. Cluster splitting is presented as a core component, but no splitting ablation is reported. The authors state that LongMemEval-S may be too small for the mechanism to matter, which raises the question of why it should be considered an experimentally supported contribution.

  5. No measured ingestion time, query latency, peak memory usage, index size, or end-to-end cost comparison is reported. Counting only generative LLM calls while excluding embedding computation and cross-encoder inference is not enough to establish that the complete system is cheaper in practice.

  6. The paper’s main practical claim is that SOLM avoids construction costs that scale with LLM inference. However, the default system still performs an optional Qwen2.5-7B call for every ingested message. The LongMemEval-M experiment disables this mechanism because it would require approximately 2.45 million calls, illustrating that the default mechanism itself does not scale to the larger setting used in the paper.

Questions for authors

Please see the Weaknesses section. My main question is whether the authors can provide controlled evidence that the complete SOLM system, rather than only the weak-link component, outperforms a strong baseline under the same generation model, reranking pipeline, and computational budget.

Reviewer_P9b8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SOLM is a hierarchical memory system for LLM agents that combines embedding-based clustering, quantization-error-triggered splitting, and cross-branch weak links. Retrieval uses beam search followed by cross-encoder reranking. Optional importance scoring creates shortcuts to selected memories. Experiments on LongMemEval-S show improvements from weak links and over a flat cosine-retrieval baseline, with additional preliminary results on LongMemEval-M.

Critical strengths
  1. SOLM offers a practical way to organize agent memories without generative LLM calls during structure construction, when optional importance scoring is disabled.

  2. The controlled weak-link ablation improves accuracy from 68.6% to 74.0%, supported by three runs and a paired significance test.

Critical weaknesses
  1. The main baseline does not isolate the lattice’s contribution. Section 4.2 states that flat retrieval directly selects the top-mm messages without reranking, whereas SOLM applies a cross-encoder to up to rmr m candidates. Therefore, the 76.0% versus 66.7% comparison changes both retrieval structure and reranking. A flat top-(rm)(r m) baseline with the same reranker, generator, and final context budget is necessary to establish the lattice’s incremental value. In addition, Appendix A.5 suggests that weak links substantially enlarge the available candidate pool. Their advantage over simply widening the beam at comparable computational cost remains unresolved.

  2. The efficiency and mechanistic claims exceed the current evidence. Avoiding generative LLM calls during structure construction is valuable, but does not establish end-to-end efficiency without measuring indexing, weak-link construction, embedding, and reranking costs. The LongMemEval-M experiment reports accuracy rather than runtime or resource scaling.

The theoretical justification also needs correction. The limit in Appendix A.6,

cos⁡(q,c^n)→bˉρˉ,\cos(q,\hat{c}_n) \to \frac{\bar{b}}{\sqrt{\bar{\rho}}},

requires a strictly positive limiting pairwise coherence, a condition not stated in the theorem:

ρˉ>0.\bar{\rho} > 0.

Moreover, the splitting criterion,

qenode>τ qeparent,qe_{\mathrm{node}} > \tau \, qe_{\mathrm{parent}},

does not establish a bound on cluster size. These gaps weaken the justification for the proposed architecture beyond its observed accuracy gains.

Questions for authors
  1. Does SOLM retain an advantage over flat top-(rm)(r m) retrieval followed by the identical reranker? At matched latency or candidate-scoring cost, how do weak links compare with a wider beam without links?

  2. Can you report ingestion time, weak-link construction time, retrieval latency, and memory consumption at both S and M scales, together with hardware and indexing details?

  3. Tables 2 and 3 report different category accuracies for the purportedly identical three-run configuration with k=20k=20 and m=5m=5—for example, SS-Pref is 46.7% versus 52.2%. Can you reconcile these results, clarify how abstention questions enter the retrieval metrics, and specify the embedding checkpoint?

  4. How are ancestor centroids updated when an existing child changes or is reassigned? Please clarify the conditions required by Appendix A.6, including ρˉ>0\bar{\rho}>0, and whether the splitting rule provides any actual cluster-size guarantee.

  5. What is the recall of the evidence-bearing messages, or the answer accuracy with oracle evidence? Retrieving a message from the correct session does not establish that all necessary evidence reaches generation, so session-level recall alone cannot support a generation-only explanation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.