acceptodds
Under review as a conference paper at ICLR 2027

S²-Memory: A Zero-Training Sparse Concept Key-Value Memory

Abstract

All AI and cognitive systems rest on a shared and rarely questioned assumption: memory requires storage and training. From Transformer KV caches to the classic "memory trace" theory in cognitive science, this assumption has shaped our understanding of memory. We have identified a simple quantity that governs the retrieval behavior of sparse concept representations. We call it the truncation loss Z: the difference in retrieval quality between a concept activation that is fully retained and one truncated to its top-k entries. For the system we study, Z is large and decisive—retaining the full activation raises MRR from 0.47 to 0.91 (Z = 0.44)—which identifies truncation, not the choice of representation, as the dominant failure mode. To examine this, we built S²-Memory—a 126MB, zero-training, plug-and-play system that represents each sentence as a sparse key over a frozen semantic center space. It retrieves over 156,660 sentences from 54 novels of widely varying genre and style using half the storage of an fp32 dense index (120.3 MB vs. 240.6 MB). Integrated with the 1.5B Qwen model (Qwen Team, 2024), it answers cross-book questions without knowing which book contains the answer. Training cost: 1,000. Examining the system revealed three further results. First, the self-referential state that motivated its design—a decayed accumulation of concept activations used directly as the retrieval query—is not merely unhelpful but harmful, and should be removed (MRR 0.0014 against a random baseline of 0.0021). Second, the concept centers are highly collinear (effective rank 292.7 of 2000; mean pairwise cosine 0.199); ZCA whitening repairs this and raises R@5 from 0.830 to 0.920. Third, the abstract centers can be replaced by content-word centers that preserve retrieval quality (R@5 0.925 vs. 0.930) while making every center nameable: 99.6% of activated centers are semantically related to the sentence, and masking the content-word centers collapses retrieval quality (MRR 0.791 → 0.234) whereas masking function-word centers costs almost nothing (MRR 0.791 → 0.788). On our corpus, BM25 is strongest under lexically overlapping queries (MRR 0.9942) but degrades sharply under non-overlapping queries (MRR 0.2650, median rank 1 → 21), where dense retrieval leads (MRR 0.7240). Our sparse key does not exceed dense retrieval in either condition. Because the intermediate representation is a set of natural-language concept labels rather than a dense vector, it can enter a language model's context directly, without a trained probe or projection layer, and it is not bound to any particular encoder; we do not evaluate the downstream benefit of this interface.

Primary area:
interpretability and explainable AI
Keywords:
sparse retrieval, interpretable retrieval, concept activation, zero-training memory, dense retrieval, top-k truncation, retrieval evaluation, WordNet centers

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_5BjNOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes S²-Memory, a sparse concept key-value memory system that requires no additional training. The system uses a pretrained text encoder to map sentences into a sparse representation in a predefined concept space, and retrieves information through the similarity between concept activation vectors. Through experiments, the authors mainly analyze how concept activation truncation, concept center collinearity, and different concept selection strategies affect retrieval performance. They also discuss the interpretability and storage cost of sparse concept representations.

Critical strengths

None

Critical weaknesses
  1. The authors fail to establish a clear and coherent research objective. The abstract, introduction, stated contributions, and subsequent sections lack clear logical connections and fail to establish a coherent research narrative. This paper reads more like a technical report that gradually debugs and analyzes an experimental system, rather than an academic paper with a coherent technical contribution. The current organization also makes it hard for readers to understand what problem the authors want to solve and what the most important contribution of the paper is.
  2. At the same time, S²-Memory is built from existing text encoding, sparse representation, and similarity retrieval techniques. The paper does not propose a new representation learning mechanism, retrieval algorithm, or training method, and it does not provide theoretical results with sufficient novelty. The experiments mainly provide empirical observations about existing techniques and their parameter choices, rather than offering substantial scientific insights.
Questions for authors

1.What is the core scientific question and the main technical contribution of this paper? 2. What is the substantive novelty of the proposed method compared with existing sparse retrieval and representation learning methods?

Reviewer_isNPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents S²-Memory, a retrieval module that maps frozen Sentence-BERT embeddings into concept activations and stores top-k activations as sparse retrieval keys. The paper studies the effects of activation truncation, concept-center collinearity, whitening, and replacing abstract WordNet centers with content-word centers. It also reports a negative result: an accumulated self-referential state performs poorly when used as the retrieval query. Experiments use sentence retrieval over public-domain novels, with dense SBERT retrieval and BM25 as baselines, and include both lexically overlapping and reduced-overlap queries. The proposed representation offers readable concept labels and reduced key storage relative to fp32 dense vectors, while generally trailing dense retrieval in accuracy.

Critical strengths
  1. The paper reports useful negative results and acknowledges important limitations. It explicitly recognizes that the proposed representation does not outperform dense retrieval, that its storage advantage disappears against fp16 dense vectors, and that the original repeated-filler evaluation was inadequate. The comparison between overlapping and reduced-overlap queries also illustrates how query construction can substantially change retrieval rankings. These disclosures help identify the actual scope of the contribution.
  2. Readable retrieval keys are a potentially useful design objective. Using natural-language concept labels as the intermediate representation could make retrieval decisions easier to inspect. The experiments on truncation, whitening, and center construction provide an initial investigation of the accuracy cost associated with this representation, including evaluation on a corpus of 156,660 sentences.
Critical weaknesses
  1. The abstract and introduction claim that retaining the full activation raises MRR to approximately 0.91–0.92 and identify truncation as the dominant failure mode. However, Section 4.3 reports full-activation MRR of 0.7800, below top-1024 at 0.8341, while Table 1 reports full-activation MRR of 0.9158. These may correspond to different configurations, but the distinction is not adequately specified. Similar ambiguity affects the storage claim: Equation (4) stores the dense embedding and sentence as values, whereas the headline storage comparison appears to count only sparse keys. The paper also omits a sufficiently precise whitening procedure. These issues prevent a reliable assessment of the claimed retrieval and storage trade-off.
  2. The masking experiment removes content-word centers comprising 94.2% of all centers, but compares this with removing the much smaller function-word subset. Without a matched-size random masking control, the performance drop cannot isolate the importance of semantic content from the amount of representation removed. Similarly, 94.0% of the top contributing centers being content words does not demonstrate enrichment over their 94.2% prevalence. The relevance evaluation relies on embedding similarity and author inspection without a clearly specified annotation protocol. Since retrieval accuracy is generally below the dense baseline and storage is neutral against fp16 vectors, stronger evidence of faithful, useful interpretability is important to establishing the contribution.
Questions for authors
  1. What exact configurations produce full-activation MRR values of 0.7800 and 0.9158? Please specify the corpus, query construction, center set, whitening status, and normalization for each. After holding these choices fixed, what evidence supports truncation as the dominant failure mode?
  2. How is ZCA whitening computed and applied to the centers and queries, including covariance estimation and regularization? Does the reported 120.3 MB include dense embeddings, sentence values, center storage, and indexing overhead, or only sparse keys? What is the total deployed memory footprint for each baseline?
  3. Does the masking conclusion hold against random masks removing the same number of centers, or the same activation mass? What criterion and annotation procedure support the reported 99.6% semantic relevance, and does that relevance establish which concepts actually determine a retrieval decision?
1 comment

Dear Reviewer,

This is a kind reminder that the discussion period will end in less than 24 hours. We would appreciate it if you could take a moment to review authors' responses and provide any further comments before the discussion period closes.

Thank you for your time and effort!

Reviewer_yPyNOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

S²-Memory represents sentences as sparse, word-labeled retrieval keys derived from a frozen sentence encoder. Each sentence and query is mapped to activations over a shared dictionary of semantic centers, with the strongest activations retained for dot-product retrieval. The paper studies the effects of truncation, center collinearity, whitening, and replacing abstract concepts with corpus-derived content words. Its central observation is that aggressive truncation discards shared query–sentence activations and reduces retrieval quality. The method can therefore be understood as a sparse re-encoding of dense embeddings, with readability as its proposed additional benefit.

Critical strengths

The method is simple, requires no additional retrieval training, and provides word-labeled coordinates that are easier to inspect than dense vectors. The sparsity and whitening experiments offer useful diagnostics of retrieval degradation. The paper acknowledges that its method does not outperform dense retrieval and that its storage advantage depends on the numerical precision of the dense baseline. Evaluating different query constructions also usefully exposes the influence of lexical overlap.

Critical weaknesses

The central truncation finding resembles the established compression trade-off between representation size and preservation of retrieval rankings. Query and target sentences are not guaranteed to retain shared coordinates, and the paper provides no guarantee that the sparse transformation preserves their original similarity or nearest-neighbor ordering. Comparisons with vector-compression methods and learned sparse retrieval approaches such as SPLADE v2 are needed to establish novelty and practical value. The claimed readability also requires stronger evidence of semantic faithfulness: word-labeled coordinates alone do not demonstrate that they explain retrieval decisions. Storage comparisons should account for the complete stored representation, including the original dense vectors specified in the values. Finally, inconsistent full-activation results and insufficiently controlled ablations make the claim that truncation dominates other representation choices difficult to assess.

Questions for authors
  1. How does the proposed representation differ substantively from existing sparse retrieval and vector-compression approaches beyond using a selected word-labeled dictionary? 2 At matched storage budgets, how does it compare with fp16 embeddings, dimensionality reduction, vector quantization, and pretrained SPLADE v2 without corpus-specific fine-tuning?
  2. Would vocabulary normalization through wordnet in choosing the centroids help in retrieval?
1 comment

Dear Reviewer,

This is a kind reminder that the discussion period will end in less than 24 hours. We would appreciate it if you could take a moment to review authors' responses and provide any further comments before the discussion period closes.

Thank you for your time and effort!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.