- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work investigates whether retrieval and reranking can share a single precomputed representation without requiring an additional backbone pass over candidate documents. The authors propose BreadBowl-Embed, which independently encodes queries and documents into a fixed set of routing–value slots. The routing vectors support indexed candidate retrieval and also determine attention over cached document value vectors for query-dependent reranking. The model is trained using a contrastive retrieval objective and reranking distillation, with periodic retrieval replay to mitigate interference between the two objectives. Experiments on 15 BEIR tasks show that adding value-based reranking improves macro nDCG@10 from 48.47 to 51.54 over routing-only scoring on the same retrieved candidates.
Critical strengths
-
The paper addresses a practical problem in retrieval systems by exploring how to reuse precomputed document representations for both candidate retrieval and reranking. Avoiding an additional backbone pass over candidate documents is an appealing direction for reducing inference costs.
-
The proposed routing–value architecture is an interesting design. Using the same routing vectors for both indexed retrieval and query-dependent attention provides a unified way to retrieve candidates and refine their relevance without re-encoding the document text.
-
The experiments cover 15 BEIR tasks and demonstrate improvements from value-based reranking on 10 of the 15 tasks. The retrieval replay ablation also supports the usefulness of periodically incorporating retrieval supervision during reranking adaptation.
Critical weaknesses
-
The novelty over existing late-interaction methods remains unclear. ColBERT already supports retrieval and reranking using cached multi-vector representations, while MVA explores attention-based scoring over stored value vectors. Direct comparisons under matched settings would help establish the specific advantages of the proposed unified architecture.
-
The efficiency claims are not sufficiently supported by system-level evaluations. Each document requires storing 16 routing vectors and 16 value vectors, introducing additional memory and indexing costs, while the evaluated implementation also relies on exact flat search. End-to-end latency, throughput, and memory comparisons with existing retrieval and reranking methods would help determine whether the proposed approach offers practical efficiency advantages at scale.
-
The benefits of value-based reranking are inconsistent across datasets, with performance declining on five of the 15 BEIR tasks and a particularly large drop on ArguAna. Moreover, the QK-versus-QKV comparison uses the same trained model rather than separately trained routing-only and combined architectures. Further analysis would help clarify when the value pathway is beneficial and whether its gains justify the additional representation costs.
Questions for authors
see weakness
4 comments
Response to Reviewer yaWv (2/2)
W1: novelty over ColBERT and MVA. We agree with both precedents, and the revision will position the model against them precisely. The routing score is a fixed-budget late-interaction scorer of the ConstBERT kind, and we do not claim it as new. MVA is the closest precedent for the value score; it reranks candidates supplied by a separate retriever. The contribution is the coupling: the vectors indexed for retrieval are the keys of the attention over the stored values, so one encoding, one index and one set of query vectors serve both stages.
The matched comparison you ask for is the control above: the routing-only model is a fixed-budget late-interaction model trained on identical data. Given its result, we no longer present QKV − QK as the advantage of this design over late interaction. The claim we make is the measured one: a single encoder and index whose own reranking is level with a cross-encoder of similar size on the same candidates, at about a fiftieth of its rerank time (comment to all reviewers, sections 1 and 4). We have no same-protocol ColBERT run. Published ColBERTv2 and ConstBERT numbers are in our reply to Reviewer ok2o.
W2: efficiency, storage and exact search. All three are measured in the comment to all reviewers (sections 3 and 4). Approximate search (IVF-PQ, one GPU, 5.7 GB index) replaces exact search at a cost of at most 0.54 nDCG@10 and brings a query to about 43 ms. We did not measure throughput under concurrent queries.
On storage you are right that 32 vectors per passage is the method's price, and it compresses well: 32,768 bytes in float32, 8,256 as int8 with no measurable loss, and 4,160 as int4 with at most 0.12 loss, about the size of one 1,024-dimensional float32 vector. Whether that is justified depends on the alternative:
- Against a retriever plus cross-encoder, it buys about 50 times less rerank time at the quality of a cross-encoder of similar size. A 4B cross-encoder is more accurate on MS MARCO dev and takes about 230 times our rerank time.
- Against a single-vector retriever alone, it is more storage, for ranking that is significantly better than Qwen3-Embedding-0.6B's and level with Jina v5 text-small's on MS MARCO (reply to Reviewer ok2o).
- Against a routing-only model, which needs only the 16 routing vectors, the control above applies: at equal training it is within 0.4 points of the combined model on MS MARCO dev and not distinguishable from it on DL19 and DL20. The revised paper will put this next to the storage numbers.
Thank you for the additional experiments. The efficiency results address my concerns about exact search and representation storage. However, the routing-only control shows that it recovers most of the QK-to-QKV gap, leaving the additional benefit of the value pathway insufficiently demonstrated. The lack of a corresponding BEIR comparison further limits the evidence. While I appreciate the authors’ efforts, I do not find the additional results sufficient to change my assessment and therefore maintain my original score.
Two new experiments: the value pathway is ahead on BEIR and on TREC DL19
Thank you for the follow-up. You named two gaps: the benefit of the value pathway over the control was not demonstrated, and the control had no BEIR comparison. We ran the two experiments that answer them. Both favor the value pathway.
- The control at full length: 1,024 updates, the same as the reported model (512 in our first reply).
- The control on all 14 BEIR tasks.
| nDCG@10 × 100 | Routing-only control | Full model (QKV) | Difference, 95% interval |
|---|---|---|---|
| BEIR, mean of 14 tasks | 55.41 | 56.08 | +0.68 [+0.45, +0.91] |
| TREC DL19 | 72.98 | 74.63 | +1.66 [+0.04, +3.37] |
| TREC DL20 | 72.13 | 73.14 | +1.01 [−0.50, +2.58] |
| MS MARCO dev | 41.63 | 41.71 | +0.07 [−0.24, +0.38] |
- Task by task on BEIR, the full model is significantly ahead on five: Touché-2020 +4.22, ArguAna +1.56, HotpotQA +0.72, Climate-FEVER +0.66, NQ +0.57. The control is significantly ahead on one: Quora, by 0.34.
- Twice the training did not put the control ahead. After 1,024 updates it leads on none of the four sets.
- The comparison is matched. Same starting checkpoint, data, data order, teacher labels, seed and number of updates. Only the objective differs: the control's ranking losses train the routing score alone.
You were right that a routing-only model, not QK, is the yardstick. The revision uses it and labels QK an ablation. Against that yardstick the value pathway is not redundant: for a second set of 16 vectors per passage and 2.9 ms per query, it buys a significant lead on DL19 and on the BEIR mean.
In the revision: Section 4.4, and every BEIR task with the training trend (Figure 6) in Appendix C.4. Scope: one training seed; the intervals are over queries.
Response to Reviewer yaWv (1/2)
Thank you for the careful reading. Your third point led to the most important experiment of this rebuttal, so we answer it first. Numbers are nDCG@10 × 100.
W3, first part: QK and QKV come from one trained model.
We ran exactly the control this asks for. It cuts both ways: a routing-only model recovers most of the QK-to-QKV gap, and it is never significantly ahead of QKV. From the same starting checkpoint, with the same data, order, teacher labels, seed and schedule, we continued training with every ranking loss applied to the routing score alone, so the value vectors receive no ranking gradient. "Combined model" below is the model trained with both scores, as in the paper. Intervals are 95%, from a paired bootstrap.
Development panel (5,000 queries, nine sources, fixed candidate lists, source-macro nDCG@10):
Held-out queries, both models at 512 updates (full MS MARCO corpus, 8.8M passages):
What this shows:
So most of the QK-to-QKV gap is not a gain over a routing-only model: the combined model's routing score is weaker, and its value score makes up the difference. Beyond the 0.35 points on MS MARCO dev, a gain of the value vectors over a model trained for routing alone is not shown. The revised paper will say this: the QK row will be labelled as an ablation of the scoring rule inside one model, and the control will be reported next to it.
Limits: one seed; both models continue from a checkpoint that was trained jointly; 512 of the 1,024 updates of the reported checkpoint; not run on BEIR.
W3, second part: tasks where the value score hurts. With the new checkpoint the value score lowers nDCG@10 on three of the 15 BEIR tasks instead of five: ArguAna (−2.76 [−3.54, −1.98], from −5.20), Quora (−0.63 [−0.82, −0.44], from −2.20) and CQADupStack (−0.25, from −0.08). We rescored our own candidates on ArguAna and Quora with both cross-encoders, and the two tasks turn out to be different cases.
The revision will state this as a limitation: the value score approaches cross-encoder quality on the query-to-passage tasks it was distilled on, and not on argument retrieval, where the queries are whole documents.