acceptodds
Withdrawn from ICLR 2027

Y-RET: A SEMANTIC INVERTED INDEX FOR QUERY-BOUNDED CONTEXTUAL RETRIEVAL

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Neural retrievers typically persist learned corpus state at the granularity of every searchable unit, from one vector per sentence in dense retrieval to many contextual vectors in late-interaction systems. We ask whether contextual state must be stored at the same granularity at which relevance is evaluated. We introduce Y-RET, a semantic inverted index that instead persists reusable lexical-semantic representatives and sparse sentence postings, then materializes full contextual sentence states only for a bounded candidate set. Across six retrieval benchmarks and four embedding backbones, Y-RET matches the corresponding dense retriever in macro effectiveness while reducing persistent retrieval state by 71.7–87.8%. On direct sentence-evidence retrieval, where BM25, dense, and Y-RET candidates are reranked by the same contextual verifier, Y-RET matches dense acquisition on SciFact. At 10.1 million sentence units on Natural Questions, the storage reduction grows to 88.4%, with a measurable effectiveness cost under the current scoring configuration. These results show that reusable semantic state can replace a large fraction of corpus-wide neural persistence while full contextual discrimination is deferred to query time.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
neural information retrieval, late interaction, efficient retrieval, contextual representations, representation compression, lexical retrieval, semantic search

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_WtGjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Y-RET, a semantic inverted index architecture for sentence-level retrieval that aims to reduce the persistent storage overhead of neural retrieval systems. Unlike conventional dense retrieval, which stores an independent embedding for each sentence, Y-RET encodes lexical units into reusable semantic representations and constructs inverted posting lists through semantic consolidation. At query time, the system first retrieves candidate sentences through exact lexical matching and semantic neighbor expansion, and then contextually encodes the top 200 candidates for reranking using IDF-weighted MaxSim. Experiments are conducted on six BEIR datasets with four embedding backbones, along with a large-scale evaluation on Natural Questions. The results show that Y-RET reduces persistent index storage by approximately 72%–88% compared with FP32 dense retrieval while maintaining comparable average retrieval effectiveness on the six BEIR tasks. However, a noticeable effectiveness degradation is observed on the larger Natural Questions dataset.

Critical strengths
  1. The research problem is practically meaningful. This paper revisits the persistence granularity of neural retrieval representations and questions whether independent contextualized representations must be stored for every sentence or token. By sharing reusable lexical-semantic states and deferring full contextual computation to query time, the proposed approach offers an interesting perspective on reducing the storage overhead of neural retrieval systems.

  2. Y-RET combines lexical-semantic consolidation, inverted-index-based candidate acquisition, and contextual reranking into a coherent framework. The authors evaluate the method across six retrieval datasets and four embedding backbones, with ablation studies examining the contributions of individual components. The results demonstrate that the proposed approach can substantially reduce persistent index storage while maintaining retrieval effectiveness comparable to dense retrieval under the evaluated settings.

Critical weaknesses
  1. The paper does not sufficiently demonstrate that the proposed method offers a practical efficiency advantage over existing dense retrieval approaches. The primary contribution of Y-RET is its substantial reduction in index storage, but the experiments mainly compare against uncompressed FP32 dense embeddings. In practice, dense retrieval systems can significantly reduce storage through established compression techniques such as FP16, INT8, and product quantization (PQ). Therefore, it remains unclear whether the reported 72%–88% storage reduction would persist under more competitive comparisons. More importantly, Y-RET shifts part of the computation from offline indexing to online retrieval, requiring contextual encoding of up to 200 candidate sentences for each query. This introduces additional online computational overhead, yet the paper does not provide sufficient comparisons of end-to-end latency, throughput, and computational cost. Consequently, it is difficult to determine whether Y-RET achieves a genuinely better trade-off among retrieval effectiveness, storage, and computational efficiency, or merely reduces storage at the expense of substantially increased online computation.

  2. The necessity and effectiveness of the proposed semantic inverted index over simpler existing retrieval and reranking pipelines remain insufficiently justified. The ablation results suggest that most of Y-RET's effectiveness comes from the second-stage contextual reranking, while the first-stage semantic inverted index alone achieves considerably lower retrieval performance. This raises an important question: why not simply use BM25, dense retrieval, or another established retrieval method to obtain candidates, followed by the same contextual reranker? Although the paper includes a shared-verifier comparison on SciFact, the results show that dense retrieval with the same verifier achieves almost identical effectiveness to Y-RET. Moreover, this comparison is not extended to the other datasets. As a result, the current experiments do not convincingly establish whether the additional complexity of constructing and maintaining the proposed semantic index provides meaningful benefits over conventional two-stage retrieval pipelines. This weakens the motivation for the proposed architecture.

Questions for authors
  1. Regarding the first weakness, could the authors provide comparisons against compressed dense retrieval baselines, such as FP16, INT8, and PQ, and report retrieval effectiveness, total index size, end-to-end latency?

  2. Regarding the second weakness, could the authors extend the shared-verifier experiments to all six datasets and Natural Questions, comparing BM25, dense retrieval, SPLADE, and Y-RET under matched candidate budgets and identical contextual verification?

Reviewer_cMF7OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes Y-RET, a retrieval system for sentence-level units by creating (1) a novel offline index which stores one embedding per lexical type, merges near-duplicate types into “representatives” (cosine ≥ 0.85), and keeps sparse sentence posting lists; then (2) it gathers candidates through IDF-weighted exact postings plus embedding-nearest-neighbor expansion over representatives. This uses a BM25-style length normalization with b = 0.25; finally (3) It encodes the top 200 candidates with the backbone encoder and reranks them with IDF-weighted MaxSim.

The paper evaluates Y-RET on BEIR tasks and matrches or beats dense retrieval. On natural questions, it helps storage but costs performance.

Critical strengths
  1. Persistence granularity as a design axis is novel in literature to my knowledge and general not under consideration while designing information retrieval systems.
  2. The Inverted Index is a novel and interesting approach to improving on the persistence granularity axis.
Critical weaknesses

[W1] The gains might come from the reranker, which is just IDF-weighted MaxSim. Static acquisition performs roughly on par with BM25. The shared verifier largely erases differences between first-stage retrievers. Verification improves BM25 but not dense retrieval. In Table 1, Y-RET includes a reranking stage, whereas all baselines are evaluated using first-stage retrieval alone. The results are therefore consistent with a simpler explanation that BM25 candidate retrieval followed by a reranker can achieve dense-level quality (which is a known result). Yet the paper attributes its results primarily to the index, without adequately isolating its contribution from that of reranking.

[W2] The paper lacks storage-matched comparisons against BM25, compressed dense retrieval, and hierarchical dense retrieval. A BM25 index could be roughly an order of magnitude smaller. If BM25 + verifier performs within confidence intervals of Y-RET, it could dominate the paper’s quality–storage frontier.

[W3] Y-RET’s quality-storage trade-off omit query-time GPU compute. Each query encodes 200 sentences, yet the paper reports only Y-RET’s latency, with no baseline latency or throughput. The paper should report storage savings alongside GPU-seconds per query for every system. Although persistence granularity is important, practical usage necessitates monitoring latency as well.

Questions for authors
  1. Address weaknesses
  2. In reference to W1, can the authors add BM25 + dense RRF + verifier and BM25 top-200 reranked by the verifier?
  3. The initial inverted index would be a new contribution. There is a lot of change from BM25 to this new inverted index, but it is not clear why those specific formulas were chosen. Can the authors explain this more deeply?

More suggestions that are interesting by not needed

  1. An ANN baselines like HNSW, and IVF which are fit once and then the leaves are discarded would be interesting to see in this scenario. Y-RET can be considered and IVF over vocabulary right?
  2. How do int8 and PQ dense indexes compare on quality and storage?
Reviewer_rP3oOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

It is not very clear to me what the paper's method is. It mentions a semantic inverted index and sentence retrieval units, but what exactly these mean are not clearly explained. My best understanding is that each token embedding (like ColBERT's embeddings) is found and based on the token and embedding is stored once and mapped to the original text using an inverted index. The paper calls these sentences, but it is unclear why sentences need to be used and not full passages like in other retrieval techniques. Based on the lexical-semantic representations the query representations can find the most similar candidates and retrieve the sentences from the posting list which are then re-encoded and reranked using MaxSim.

Critical strengths

The idea to trade off storage and compute is interesting especially given that one of the main downsides of late-interaction models is there substantial storage use. There is some work in this area for dense retrieval methods, but not in late-interaction as far as I know.

Critical weaknesses

The paper presents this idea in a way that is much more confusing than necessary in large part because it uses terminology that is not widely used without clear definitions. For example, "a semantic inverted index that instead persists reusable lexical-semantic representatives and sparse sentence postings, then materializes full contextual sentence states only for a bounded candidate set" and "Let the corpus contain deterministic sentence retrieval units". The framing and terminology appears to be in an attempt to appear more novel, but makes it nearly impossible to parse. This writing style seems to clearly point towards AI writing used throughout.

Related to this point the paper does not properly contextualize the work. Methods like PLAID already built inverted indexes from semantic units although they use it with contextualized token embeddings and store the actual embeddings. If PLAID just kept the inverted index and recomputed the embeddings for final candidates it would achieve similar storage compaction I'm sure.

Beyond the presentation and poor contextualization the paper also does not compare things apples-to-apples. All the baseline methods don't have to any documents at retrieval time while Y-RET does. A fairer comparison would be to use cross-encoder reranker for the other methods as that would also take additional compute at query time. I'm guessing that this would remove the benefits of Y-RET. Additionally, BM25 or learned sparse approaches already have a very small index so using these methods would be a more reasonable baseline with a reranker on top.

The method also uses several odd approaches without sufficient justification. For example, it uses dense retrieval model with MaxSim similarity even though they have never been trained to produce meaningful similarities with MaxSim. They also encode terms without context which is unusual and would result in problems with polysemantic words.

Questions for authors

How does your method compare to BM25 using the same reranking you use? How about BM25 with a cross-encoder reranker with a similar size to your encoder?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.