acceptodds
Under review as a conference paper at ICLR 2027

EmbedFlow: Upgrading Legacy Embeddings Without Full Upfront Re-Embedding

Abstract

Upgrading an embedding model usually means rebuilding the document index. This is a natural consequence of independently trained models living in different vector spaces: embeddings from the new model generally cannot be searched directly against vectors produced by the old one. But this does not necessarily make the old index useless. We ask whether a legacy retriever can still return enough useful candidates for a new model to recover its retrieval quality without first re-embedding the full corpus. We call this property candidate compatibility. We study candidate compatibility across 63 source-target migrations. At K = 50, 42 of 63 settings are within 0.01 nDCG@10 of native target retrieval. The same pattern appears in a frozen BRIGHT evaluation and remains stable in a controlled Natural Questions study from 100K to one million documents. At 1M documents, Qwen3-4B → 8B is effectively lossless with only 50 source candidates. Qwen3-0.6B → 8B and MiniLM → 8B require deeper candidate pools, reaching the same stringent near-target criterion at K = 200 and K = 500, respectively. A practical challenge is deciding whether a migration is compatible without first building the target index. To address this, we introduce a frozen residual candidate-tail diagnostic that uses neither relevance labels nor native target retrieval. Across 28 frozen holdout cells, it makes 17 SAFE predictions with zero observed false-safes, and it recovers the same migration ordering in a subsequent blind million-document evaluation. Finally, we study what happens when the legacy index itself is approximate. We show that deployed migration error separates into two terms: an intrinsic source-target compatibility gap and an additional penalty from approximate nearest-neighbor search. Our million-document ANN experiments match this decomposition. Together, these results suggest that embedding migration need not be treated as an all-or-nothing re-indexing operation. For compatible model transitions, a legacy index can remain useful while the new model is introduced progressively.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
embedding migration, dense retrieval, embedding models, vector search, retrieval compatibility, approximate nearest neighbor search, retrieval-augmented generation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_RMmMOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes metrics to assess whether old retrievers can serve as candidate generators for new models.

Critical strengths

None

Critical weaknesses
  1. The writing of this paper shows strong signs of AI-generation and weak logical coherence. There are still AI instructions left in the appendix, such as figure 8.
  2. The experiments do not provide sufficient evidence to establish the effectiveness or generalizability of the proposed methods.
Questions for authors

None

1 comment
AuthorsOpenReview ↗

Response to Reviewer RMmM

We disagree with this interpretation of our paper strongly.

Thank you for pointing out the missing figure 8 in the appendix. The authors used frontier models to generate boilerplate latex code for the appendix, with failsafe methods that allowed the document to compile even with a missing figure. The figure was present during compilation, but in a nested directory, causing the compiler to miss it. The paper has now been checked for all figures. The authors take full responsibility for the contents of the paper, and our use of LLMS in this project is consistent with our ICLR 2027 AI disclosure.

Aditionallly, AI assisted code/text does not imply the work is sub standard. The use of tools to improve research productivity is neither new nor inherently problematic. What matters is if the result is 1)technically correct, 2)reproducible, and 3)both scientifically/practically useful.

Furthermore, we respectfully note that the review doesn't identify a specific error beyond the figure, such as a specific methodological error, incorrect result/interpretation of result, yet claims the " experiments do not provide sufficient evidence to establish the effectiveness or generalizability of the proposed methods". We question if the review provides enough justification for a soundness score of 1.

Regardless, we have made the paper’s main question clearer. Retrieve-then-rerank is standard. Our contribution is a practical way to decide whether an existing embedding index can support a model upgrade before committing to a full re-embedding. Section 2 now introduces T2-v1 before Eq. 4 and explains why it compares the target’s reranked top 10 as the legacy candidate pool grows from 50 to 500. Section 5 turns that prediction into a concrete deployment procedure.

At (K=50), 42 of 63 development settings meet the 0.01 nDCG@10 margin by point estimate, and 26 meet it under the paired-bootstrap criterion. On the separate BRIGHT evaluation of 522 queries across four domains, the pooled gap at (K=50) is 0.0035 for Qwen3-4B→8B and 0.0367 for MiniLM→8B. On the 1M Natural Questions corpus with the same 8B target, the smallest tested depths meeting the paired-bootstrap criterion are 20, 500, and 1,000 for the 4B, 0.6B, and MiniLM sources, respectively. The new 50K comparison in Table 5 also tests BM25, ridge, and a residual MLP on the same queries, both standalone and with the same target reranker.

The probe’s limitations matter for deployment. Across 28 cells from four model pairs held out from rule development, T2-v1 issued 17 SAFE calls with zero observed false-safes under exact retrieval. In the low-recall ANN tests, two SAFE calls for 4B→8B had deployed gaps of 0.06223 and 0.02169, both above the 0.01 margin. That’s why the recipe in Section 5 uses SAFE to choose a starting depth for a pilot and checks the deployed index against exact legacy retrieval before rollout.

Thank you again for your time. It is often the harshest reviews that have the most impact.

Reviewer_TrxROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper explores the relevance of legacy embeddings extracted using the old model when the query embeddings are from a different model. While existing works focus on making old and new model embeddings interoperable, this paper investigates the candidate compatibility of the legacy embeddings which evaluates if the legacy retriever can still select enough useful candidates for the new model. This could eliminate the need to re-compute the embeddings using the new model. The authors evaluate the retrieve-then-rerank approach across 63 development settings, controlled experiments on Natural Questions dataset with varying corpus sizes using a wide range of embedding models.

Critical strengths
  1. The paper is well-written with good motivation and discussion of compatibility between different embedding models.
  2. The paper defines several metrics (candidate gap, migration depth) and provides large-scale experiments to analyze migration compatibility between two frozen retrieval models.
Critical weaknesses
  1. The scope and contributions of the paper are not significant. The paper only provides benchmarks and guidelines for evaluating model compatibility. The paper is limited to analysis of candidate compatibility following the retrieve-then-rerank approach, ignoring the model interoperability approaches with some discussions only.

Existing post-hoc approaches enabling direct compatibility between models [a] could be an interesting addition. Drift compensation based approaches [b] could be another training-free post-hoc way to enable compatibility on frozen models. It would be interesting to compare with these post-hoc methods and also see how the retrieve-then-rerank approach works when combined with these post-hoc methods.

Although the paper provides useful extensive analysis for evaluation, the conclusion that "Embedding-space incompatibility does not necessarily imply retrieval incompatibility" is not a significant claim and depends on the models used and the pre-training data.

  1. Non-semantic lexical approaches like BM25 [c] are not discussed or included in the analysis. Recent work [d] highlights the importance of BM25 over dense embedding retrieval models.

[a] Drift-adapter: A practical approach to near zero-downtime embedding model upgrades in vector databases. EMNLP 2025.

[b] Query Drift Compensation: Enabling Compatibility in Continual Learning of Retrieval Embedding Models. CoLLAs 2025.

[c] The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends in Information Retrieval, 2009.

[d] On the Theoretical Limitations of Embedding-Based Retrieval. ICLR 2026.

Questions for authors
  1. While the paper focus on source-target model pairs (figure 1), it would be interesting to see how the candidate compatibility varies when the source-target model training data change where the model could be same as done in a continual setting [b]. Can the authors reflect on the impact of the training data on the source target model pairs for the analysis?
2 comments
AuthorsOpenReview ↗

Response to Reviewer TrxR

Thank you for your detailed review. Your suggestions have been implemented in the paper, greatly strengthening it.

We believe Retrieve-then-rerank is standard. Our contribution is a practical way to decide whether a deployed embedding index can support a model upgrade and how deep it needs to retrieve before committing to a full re-embedding. On the same 1M Natural Questions corpus with Qwen3-8B as the target, the smallest tested depths meeting the 0.01 nDCG@10 margin under our paired-bootstrap criterion were K=20, 500, and 1,000 for Qwen3-4B, Qwen3-0.6B, and MiniLM, respectively.

Table 5 and Appendix F.2 add a comparison on the same 50K-document Natural Questions subset and 1,000 evaluation queries. We fit ridge and a Drift-Adapter-style residual MLP using the same 10,000 paired documents without relevance labels. Each map sends 8B queries into the legacy space. We test each method on its own and use its candidates for reranking with the same native 8B scores.

Candidate generatorStandalone nDCG@108B rerank, K=508B rerank, K=500
Native Qwen3-8B0.90619——
BM25 (SQLite FTS5)0.608250.778630.84779
Legacy MiniLM-L60.788320.878910.89546
Ridge into MiniLM-L60.735860.880820.90060
Residual MLP into MiniLM-L60.735860.880820.90060
Legacy Qwen3-4B0.892380.904050.90619
Ridge into Qwen3-4B0.890820.905990.90619
Residual MLP into Qwen3-4B0.892980.906120.90619

The MiniLM maps illustrate why testing the combined method matters. Both score below legacy MiniLM on their own and give a higher mean after reranking at K=500. The paired intervals for their gaps to native 8B still cross the 0.01 margin. For Qwen3-4B, the residual MLP’s gains over ridge are small and both paired intervals include zero. BM25 also benefits from reranking, with lower mean scores than either legacy dense candidate pool at the same depths on this subset.

On training data, even with the same backbone, an update can move relevant documents up the target ranking that the legacy retriever rarely surfaces. That could widen the gap at a fixed K. Our checkpoint comparisons capture the combined effect of training data, objectives, architecture, and encoding settings. A controlled test would hold the backbone, training recipe, indexed corpus, and evaluation queries fixed while varying the updated checkpoint’s training data. That would give us a matched setting to test Query Drift Compensation both as a standalone retriever and as a candidate generator for target reranking.

Thank you again for your detailed review.

I thank the authors for the rebuttal responses. The experiments added during the rebuttal improves the paper. After reading the authors response, I still maintain my opinion that the contributions of the paper are primarily limited to benchmarking results for evaluating model compatibility. I maintain the same rating.

Reviewer_SSLAOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The current work investigates embedding migration: how to replace an old embedding model without immediately re-indexing the entire collection of documents. The paper takes the well-known retrieve-and-rerank paradigm and introduces principled metrics to evaluate this candidate compatibility.

First, the authors introduce a metric to measure the performance gap between full native re-indexing (what we try to avoid) and retrieve-and-rerank. As this metric requires the exact computation the paper tries to avoid, they introduce a label-free diagnostic probe. For a small set of unlabelled queries, the probe estimates compatibility by measuring the stability of the reranked top-10 (with the new model) as the old model's candidate pool expands. They also disentangle the error from the model disagreement from the error induced by approximating KNN.

The paper brings extensive support in favour of the method, but they only compare against a simple linear alignment. The conclusion of their empirical investigation is that compatibility is common (67% in their sweep, so not universal) and stable up to 1M documents for selected pairs of embedders.

Critical strengths
  • Clear writing and a very much needed compatibility analysis framework for retrieve-and-rerank.
  • The suggested metrics are practical and disentangle confounders (other sources of error).
Critical weaknesses
  • The lack of comparison with the more sophisticated methods, and maybe an analysis on how the failure or success of retrieve-then-rerank correlate with those methods and/or when such methods could be combined. The question is: when should we use this method against others?

  • The amortization analysis in appendix J (which I find to be important and I would bring it closer to the main content) lacks explicit modeling of how KK impacts the cold-start cache-warming horizon.

Questions for authors

T2-v1 introduced at the bottom of Page 3 is mentioned out of nowhere. It should be formally introduced. The next equation is the one for the core residual-tail statistic. Could you consider rewriting those last lines on page 3?

Could you add to the paper an adoption recipe, consolidatig Section 5 and Appeindx E.3. Would be great to wrap-up the paper with a practical algorithm that puts together T2-v1, ANN recall checks and the selection of depth KK.

1 comment
AuthorsOpenReview ↗

Response to Reviewer SSLA

Thank you for your detailed review. Your suggestions have been implemented in the paper, greatly strengthening it.

  1. Regarding when an adapter is worth fitting: We add a comparison on the same 50k Natural Question subset, with 1000 queries. (Table 5, Appendix F.2). For the upgrade Qwen3-4B to 8B, reranking the legacy top 50 reached 0.90405 nDCG@10, close to native 8B at 0.90619. In that case, we would start with the existing candidate generator. For MiniLM to Qwen3-8B at K=500, using ridge to generate candidates narrowed the gap from 0.01072 to 0.00559 with the same reranker. The 95% interval for the ridge gap, [0.00159, 0.01051], still crosses the 0.01 margin. The nonlinear adapter tied ridge on MiniLM in mean nDCG@10, and the intervals for its gains on 4B included zero. A standalone adapter also makes sense if it meets the quality target on its own and we want to avoid encoding candidate documents at serving time.
  2. We now explain T2-v1 before Eq. 4. It checks how much the target’s reranked top 10 changes when the legacy candidate pool grows from 50 to 500. In Section 5, a SAFE result means K=50 is the first depth to try. We then check the deployed ANN index against exact legacy retrieval and choose the smallest tested depth that meets the quality, latency, and encoding budgets.

3)We also replayed 1,061 saved queries from an empty cache (Appendix I.1). With Qwen3-4B, K=50, 200, and 500 touched 19,395, 31,891, and 34,432 distinct documents. Each replay reached 90% of its own final total at query 844, 777, and 766, respectively. The deeper pools reached that relative threshold sooner while requiring more documents to be encoded.

Thank you again. Retrieve-then-rerank is a standard technique. Our contribution is a practical way to decide whether an existing embedding index can support a model upgrade before committing to a full re-embedding.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.