- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes RIMU, a document vector editing method for frozen dense retrieval models. The method leverages verified query feedback to directly adjust target document vectors through convex optimization, thereby improving their ranking relative to competing documents. It also employs tangent-space constraints and an ownership mechanism to reduce unnecessary vector changes and inter-query interference. Experiments cover ten retrieval pools and five encoders, and the results show that the method achieves a notable improvement in N@10 compared with frozen indexes.
Critical strengths
The problem has practical value. Without retraining the encoder, retrieval feedback is absorbed directly through vector editing, which offers certain deployment advantages.
The optimization objective is clear. Through ranking constraints and a minimum-displacement objective, it balances retrieval effectiveness and preservation of the original vectors.
The experimental coverage is relatively broad. The method is evaluated on multiple encoders and retrieval pools, and performance changes, ablation results, and update overhead are analyzed.
Critical weaknesses
Weakness 1. There is an obvious connection to the document-vector editing work Embedding Surgery [1], which shares much of RIMU's motivation, yet the paper does not discuss or compare with it. In particular, it is not explained what substantive innovations RIMU offers over this closest work in terms of ranking constraints, continual feedback updates, and optimization mechanisms. A direct comparison with the closest method is lacking, and without it readers cannot judge what RIMU substantively adds.
Weakness 2. The ownership constraint's benefit is confined to guarded queries (training N@10 +0.69, invasions −0.012/query), but the paper does not quantify how often an invasion actually pushes a gold out of the top 10. Appendix C states that "fewer than one invasion in twenty displaces a gold from a top ten," and describes this as the mechanism working as designed on a quantity that costs retrieval almost nothing. This is confusing, because the whole motivation of the ownership constraint is to finely control and reduce the probability of such invasions. If an invasion almost never displaces a gold from the top 10 (fewer than 5%), this conditional probability should be surfaced in the main text, with an honest statement that ownership mainly reduces harmless invasions. Otherwise readers cannot judge whether the module deserves to be on by default.
[1] Amendola M, Mallia A, Perego R. Embedding Surgery: Localized Updates for Adaptive Ranking Correction in Dense Retrieval. arXiv preprint arXiv:2609.05110. 2026 Sep 4.
Questions for authors
See the weakness.
4 comments
Response to Reviewer 8SvY (2/2)
W2: The ownership constraint's benefit is confined to guarded queries (training N@10 +0.69, invasions −0.012/query), but the paper does not quantify how often an invasion actually pushes a gold out of the top 10. App. C.3 mentions "fewer than one invasion in twenty displaces a gold"; if so, this conditional probability should be surfaced in the main text, with an honest statement that ownership mainly reduces harmless invasions. Otherwise readers cannot judge whether the module deserves to be on by default.
Experimental setup
Appendix C.3 counts invasion events. In the revision we also count per test query (Table R12), for RIMU with and without ownership, for ERM and for Embedding Surgery.
Table R12: Ownership and intrusion harm, mean over 10 pools × 5 folds. Intrusion: a rewritten non-gold enters a test query's top-10 above the query's best gold, which it did not outscore before the update; harmful: this lowers the query's N@10; any gold out: at least one gold that was in the query's top-10 before the update is no longer; all golds out: the query had a gold in its top-10 and has none after (% of test queries). RIMU and ERM: same runs as the paper's Table 24; Emb. Surgery: symmetric configuration, same pools and folds.
| Method | N@10 Train | N@10 Test | Intrusion % | Harmful % | Any gold out % | All golds out % |
|---|---|---|---|---|---|---|
| BGE-large | ||||||
| ERM | 91.7 | 46.5 | 36.8 | 17.0 | 9.3 | 7.0 |
| Emb. Surgery | 61.7 | 45.9 | 12.6 | 5.3 | 1.1 | 0.8 |
| RIMU w/o ownership | 77.1 | 51.6 | 21.5 | 7.6 | 2.8 | 1.6 |
| RIMU | 77.9 | 51.6 | 20.7 | 7.1 | 2.5 | 1.5 |
| Qwen3-Emb.-8B | ||||||
| ERM | 91.5 | 50.6 | 33.4 | 17.8 | 8.9 | 6.9 |
| Emb. Surgery | 71.3 | 54.1 | 9.5 | 5.4 | 1.0 | 0.6 |
| RIMU w/o ownership | 77.0 | 55.6 | 18.2 | 8.7 | 2.8 | 1.5 |
| RIMU | 77.3 | 55.6 | 17.8 | 8.5 | 2.7 | 1.4 |
Findings
(1) The conditional probability the reviewer asks for is about 11–12%. Among test queries that suffer an intrusion, at least one gold that was in the top-10 is pushed out in 11.2% with BGE (493 of 4,397) and 12.1% with Qwen3-8B (338 of 2,803), against 21.2% and 18.1% with ERM (pooled over all test queries; dividing the mean shares of Table R12 gives 12.1% and 15.2%). The "fewer than one in twenty" of Appendix C.3 (C.4 in the revision) counts per invasion event, i.e. per rewritten non-gold entering a top-10, most of which stay below the gold; per query the rate is about one in nine. The revised paper reports both, next to Figure 5 and in Appendix H.3.
(2) Ownership's effect is small, but keeping it on by default has no visible downside. It leaves test N@10 unchanged and lowers the intrusions it targets with both encoders: harmful intrusions 7.6 → 7.1% (BGE) and 8.7 → 8.5% (Qwen3-8B), any gold out 2.8 → 2.5% and 2.8 → 2.7%, all golds out 1.6 → 1.5% and 1.5 → 1.4%, training N@10 +0.8 and +0.4 (unrounded 76.95 → 77.31). Of the 0.8-point drop in intrusions with BGE, 0.5 points are harmful ones; under the per-event definition of Appendix C.3, ownership removes 0.77 points of invaded queries, 0.55 of them harmful (0.26 and 0.19 with Qwen3-8B), so it does not mainly remove harmless invasions. Over the 50 (pool, fold) cells the reductions are significant with BGE (paired t-test: harmful t = −4.8, any gold out t = −4.2) and weaker with Qwen3-8B (t = −3.0 and −1.7); test N@10 does not change (t = 0.7 and 0.9). Its update-time cost is small (a few ownership rows per query's QP), so we keep it on by default.
(3) The gap to ERM comes from RIMU's update, not from ownership. Without ownership, RIMU already lowers the share of test queries that lose a gold from about 9% (ERM) to 2.8%, because it ranks a gold above its competitors instead of pulling it onto the query. Ownership's effect is largest where many feedback queries ask for nearby targets (MAIR-Code, ToolRet-Code, MAIR-Medical, SRA) and close to zero on MAIR-Legal, -Web and -Academic.
(4) Embedding Surgery intrudes least because it repairs least. It repairs less of the feedback (training N@10 61.7 vs. 77.9 with BGE, 71.3 vs. 77.3 with Qwen3-8B), so fewer rewritten rows reach other queries' top-10 (harmful 5.3 / 5.4%), and its test N@10 stays lower (45.9 / 54.1 vs. 51.6 / 55.6).
In the revised paper: Figure 5 in the main text and Appendix H.3 (Table 24).
Thank you for your careful and detailed response. In light of the fact that I have already given a positive evaluation, I will maintain my original recommendation.
Thank you for the time and care you have devoted to reviewing our work. Your suggestions on the additional comparison and on the ownership term have helped us strengthen the paper and make it more comprehensive. We sincerely appreciate your continued support.
Response to Reviewer 8SvY (1/2)
We thank the reviewer for the valuable feedback and for bringing this important concurrent work to our attention. We answer the two weaknesses in turn: novelty with respect to Embedding Surgery (W1) and the ownership term (W2). In the revised PDF, changes made in response to your comments are shown in purple.
W1: The argument for novelty needs further supplementation. There is an obvious connection to the document vector editing work Embedding Surgery, and it has not yet been fully explained what substantive innovations RIMU offers in terms of ranking constraints, continual feedback updates, and optimization mechanisms. A direct comparison with the closest method is lacking.
Both methods use ranking-constrained convex editing of corpus vectors; RIMU makes five different design choices (Table R10). We regard Embedding Surgery as concurrent work, and the revised paper now discusses it as such in the Related Work section, next to the direct comparison in Table 1.
Table R10: Design choices of Embedding Surgery and RIMU.
Experimental setup
We reproduced Embedding Surgery in its default symmetric configuration (the gold and its top-20 competitors writable, one feedback query at a time) on the paper's pools, folds and feedback with both encoders. Two variants test single design choices (Table R11): post-update renormalization only, without RIMU's tangent constraint, and Surgery's own Promotion variant, which moves only the relevant documents, as RIMU does.
Table R11: Embedding Surgery vs. RIMU, test N@10 per pool, mean over 5 folds. Surgery: symmetric (default) configuration; Promotion: only the relevant documents move. Best updated method in bold.
Findings
(1) RIMU is better on the mean with both encoders, and on seven of ten pools against all three Surgery variants. The margin is +5.7 N@10 with BGE and +1.5 with Qwen3-8B, and RIMU repairs more of the feedback (training N@10 77.9 vs. 61.7). Surgery, or its renormalized variant, is stronger on MAIR-Medical, -Academic and -Web with both encoders. Post-update renormalization alone does not close the gap; the two variants are reported only in this response, not in the paper's Table 1.
(2) The gain comes from RIMU's choices together, not from the writable set alone. The Promotion variant moves only the relevant documents, as RIMU does, but under hard constraints and without normalization or protection; it is the weakest variant (mean 39.2 / 48.6 N@10) and loses 14.3 / 13.2 N@10 on unseen test queries (BGE / Qwen3-8B), against 1.2 / 0.6 for the symmetric default, which agrees with Surgery's choice of the symmetric form as its default. RIMU's writable set pays off together with its tangent geometry, its soft constraints triggered only when a ranking needs repair, and ownership.
In the revised paper: the Emb. Surgery rows of Table 1 and of the timing table (Table 21, Appendix G.1).