- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Recurrent Reading for Single-Vector Retrieval (RLI) to address the limitations of single-vector representations in reasoning-intensive retrieval. Existing multi-vector retrieval performs fine-grained relevance matching through late interaction among multiple token representations, but incurs high index storage and retrieval costs. The authors analyze the slot locking phenomenon that arises from simply concatenating multiple pooled representations. The problem is that, as each query read is trained to correspond only to the document read at the same position, the benefits of cross-read matching are not fully exploited. RLI uses a GRU-based recurrent read head and a shared projection, and applies cross-read MaxSim during training. At actual retrieval, multiple reads are combined into a single vector and only a single inner product is performed. On BRIGHT, the 4B model achieves 42.42 nDCG@10, 3.62 points higher than the baseline with reads removed under the same training recipe (38.80).
Critical strengths
- RLI uses late interaction during training while retaining single-vector retrieval at inference. Unlike conventional multi-vector retrieval methods such as ColBERT, this design benefits from cross-read matching without requiring a multi-vector index.
- The analysis of slot locking provides a clear motivation for the proposed method. The authors support this observation with off-diagonal argmax rates, read cosine similarity, and scoring function comparisons. The ablation studies further help isolate the effects of the shared projection and training objective.
- The paper identifies a limitation of aligned dot-product scoring in concatenated representations and addresses it through cross-read MaxSim supervision. This is a useful contribution beyond simply introducing another retrieval architecture.
- RLI offers a practical way to improve reasoning-intensive retrieval while preserving the simplicity of single-vector search. The results on BRIGHT suggest that the proposed training strategy can improve retrieval quality without introducing query-time late interaction.
Critical weaknesses
- The independent contribution of recurrent reading is relatively small. A single learned read achieves 42.46 nDCG@10, nearly identical to the 3-read RLI average of 42.42. This suggests that a substantial portion of the core performance gain comes from the additional learned representation itself rather than from the recurrent multi-read interaction. A clearer analysis appears necessary.
- Empirical analysis of single-vector efficiency is lacking. Although RLI maintains a single-vector index, the vector dimension increases from 2,560 to 3,328, by about 30%. Measuring the actual index memory and retrieval latency would make the efficiency claim clearer.
Questions for authors
- Given that a single learned read performs similarly to the 3-read RLI, what is the main advantage of the recurrent multi-read architecture? I am curious about differences depending on read diversity or query complexity.
- Recommend briefly discussing the difference from LEMUR [1] in the Related Work, which would make the contribution clearer.
[1] Elias Jääsaari et al. "LEMUR: Learned Multi-Vector Retrieval." In Proceedings of the 43rd International Conference on Machine Learning (ICML), arXiv preprint arXiv:2601.21853 (2026).
4 comments
Response Part #2
Comment 3 of 5
Q1: the advantage of recurrent multi-read reading; read diversity and query complexity
Main advantage. Recurrence is what makes extra reads useful. Each read sees what the earlier reads attended to, and the cross-read maximum lets a query read match whichever document read holds the same evidence. Without either, three reads fall below one (Comment 1). Most of the benefit comes with the second read (T=2 43.37, T=3 43.41), and T=2 stores 7.7% fewer floats per document.
Read diversity. Table 2 of the paper (Section 4.3) measures how reads are used at inference on BRIGHT. The off-diagonal argmax rate is the share of query reads, over all (query, gold document) pairs, whose best-matching document read is not the aligned one (0% = slot-locked, 66.7% = uniform at T=3):
| Config. | Training objective | Tied W_o | Off-diagonal argmax rate | Read cosine |
|---|---|---|---|---|
| B | aligned dot | no | 0.0% | −0.04 |
| C | aligned dot | no | 0.0% | −0.03 |
| E | cross-read max | no | 0.0% | −0.00 |
| G | aligned dot | yes | 10.3% | 0.84 |
| F (RLI) | cross-read max | yes | 37.1% | 0.88 |
Untied reads only look diverse (cosine ≈ 0) because they sit in unrelated subspaces, so a query read never matches another document read better, even with the cross-read maximum (E). RLI's reads share one space but attend to different parts: 37% of query reads find their best match in a different document read, which one read cannot do. RLI is the only configuration with real cross-read matching, and the only one above a single read.
Query complexity. Using query length as a proxy, we split each domain's queries into three equal-size groups and recompute the gain of RLI over the non-recurrent reads (H; domain-macro, paired bootstrap, seed 44):
| Query length (tercile within each domain) | Median words | Queries | RLI − non-recurrent reads H, nDCG@10 (95% CI) |
|---|---|---|---|
| Short | 50 | 454 | +2.45 [+1.26, +3.65] |
| Medium | 84 | 457 | +1.44 [+0.23, +2.65] |
| Long | 142 | 473 | +1.97 [+0.93, +3.06] |
Comment 4 of 5
W2: index memory and latency (measured)
We measured the deployed RLI-4B (seed 44) on one H100 GPU (CPU on the same node), with exhaustive inner-product search as in the paper. The read head has 3.28M parameters (0.08% of the backbone).
Index memory. A flat index takes N × d × 4 bytes in fp32 (half in fp16); our FAISS measurement on the largest BRIGHT corpus matches this:
| Stored vector | Dims | Bytes / doc (fp32) | Index, 1M docs (fp32) | Index, LeetCode corpus 414k docs: measured (FAISS flat) | BRIGHT nDCG@10 (seed 44) |
|---|---|---|---|---|---|
| Base vector only (reads switched off) | 2,560 | 10,240 | 10.24 GB | 4.24 GB | 38.94 |
| T=1 | 2,816 | 11,264 | 11.26 GB | 4.66 GB | 42.46 |
| T=2 | 3,072 | 12,288 | 12.29 GB | 5.09 GB | 43.37 |
| T=3 (RLI) | 3,328 | 13,312 | 13.31 GB | 5.51 GB | 43.41 |
Encoding. The read head runs on token states the backbone has already computed. On 2,000 BRIGHT documents (267 tokens on average, batch 8), it adds 1.3% to encoding time (87.0 documents/s with the head vs. 88.1 without). One query takes 26.5 ms with the reads vs. 25.6 ms without (median over 195 queries).
Search latency, 1M documents, top-100, median (GPU: fp16 matrix product; CPU: FAISS IndexFlatIP, fp32, 16 threads):
| Stored vector | Dims | GPU, 1 query (ms) | GPU, per query in a batch of 128 (ms) | CPU, 1 query (ms) |
|---|---|---|---|---|
| Base vector only (reads switched off) | 2,560 | 2.61 | 0.030 | 285.4 |
| T=1 | 2,816 | 2.86 | 0.032 | 327.8 |
| T=2 | 3,072 | 3.11 | 0.034 | 356.3 |
| T=3 (RLI) | 3,328 | 3.35 | 0.036 | 387.9 |
Search cost grows with vector size: T=3 takes 20–36% more time than the base vector alone, in line with its 30% larger dimension, while T=2 gives the same accuracy (43.37 vs. 43.41) at +20%. The index is still one vector per document, so standard approximate search and quantization apply. Late interaction, by contrast, stores one vector per token, and these documents average 267 tokens. We will add these numbers to the paper.
Comment 5 of 5
Q2: LEMUR (Jääsaari et al., ICML 2026)
This is a very useful reference to know about, and we agree that discussing it will help clarify our contribution. We will add a brief comparison with LEMUR in the Related Work section.
We hope this addresses your concerns. The revision will add the matched-seed table and its replication, the read-usage and query-length analyses, the efficiency numbers, and the LEMUR discussion.
Dear Reviewer,
we hope you have a chance to check our response above. Please let us know if you need any further clarification or another experiment before the deadline
thank you!
Response
I have received the rebuttal response and thank you for the response and extra experiments. After careful consideration, I keep my score.
Response Part #1
Comment 1 of 5
W1: one learned read vs. the three-read RLI
Thank you for the careful reading and helpful questions. Our T=1 model is a parameter-matched control: the GRU and W_o are shared across reads, so the read head has the same 3.28M parameters for every T. Only the number of read steps changes.
The 42.46 in the review is a single T=1 run (seed 44), while 42.42 is RLI's three-seed mean. At the same seed, data and recipe, RLI scores 43.41, +0.95 nDCG@10 over one read (95% CI [0.32, 1.60]; paired bootstrap over queries), and is better on 8 of 12 domains. The table also shows where the gain comes from: three reads without the cross-read maximum (G) or without recurrence (H) score below one read. BRIGHT nDCG@10, seed 44:
So extra reads do not help just by adding vectors. If either ingredient is missing, they cost about one point (G −0.95, [−1.47, −0.42]; H −0.99, [−1.63, −0.36]). With both, recurrence and training through the cross-read maximum, they help: RLI beats the non-recurrent reads by +1.95 ([1.27, 2.64]), on 11 of 12 domains.
Comment 2 of 5
W1 (continued): same result on a second training mixture
To check that this does not depend on our data, we retrained T=1 and T=3 on a second public mixture: the ReasonEmbed release plus about 7.5k real StackExchange, LeetCode, ReasonRank and NuminaMath questions. No row comes from a BRIGHT query or label. T=3 again beats one read, by +0.74 nDCG@10 (95% CI [0.18, 1.32]), on 8 of 12 domains:
These intervals cover query sampling, not seed noise (about one point on BRIGHT, which is why the main result uses three seeds). What holds on both mixtures is that three recurrent reads beat one. Badly designed extra reads hurt by much more than this noise: three slot-locked reads (aligned loss, untied W_o; configuration C, original mixture) are 3.05 below one read (95% CI [−3.86, −2.26]).