- The scope and contributions of the paper are not significant. The paper only provides benchmarks and guidelines for evaluating model compatibility. The paper is limited to analysis of candidate compatibility following the retrieve-then-rerank approach, ignoring the model interoperability approaches with some discussions only.
Existing post-hoc approaches enabling direct compatibility between models [a] could be an interesting addition. Drift compensation based approaches [b] could be another training-free post-hoc way to enable compatibility on frozen models. It would be interesting to compare with these post-hoc methods and also see how the retrieve-then-rerank approach works when combined with these post-hoc methods.
Although the paper provides useful extensive analysis for evaluation, the conclusion that "Embedding-space incompatibility does not necessarily imply retrieval incompatibility" is not a significant claim and depends on the models used and the pre-training data.
- Non-semantic lexical approaches like BM25 [c] are not discussed or included in the analysis. Recent work [d] highlights the importance of BM25 over dense embedding retrieval models.
[a] Drift-adapter: A practical approach to near zero-downtime embedding model upgrades in vector databases. EMNLP 2025.
[b] Query Drift Compensation: Enabling Compatibility in Continual Learning of Retrieval Embedding Models. CoLLAs 2025.
[c] The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends in Information Retrieval, 2009.
[d] On the Theoretical Limitations of Embedding-Based Retrieval. ICLR 2026.
Response to Reviewer RMmM
We disagree with this interpretation of our paper strongly.
Thank you for pointing out the missing figure 8 in the appendix. The authors used frontier models to generate boilerplate latex code for the appendix, with failsafe methods that allowed the document to compile even with a missing figure. The figure was present during compilation, but in a nested directory, causing the compiler to miss it. The paper has now been checked for all figures. The authors take full responsibility for the contents of the paper, and our use of LLMS in this project is consistent with our ICLR 2027 AI disclosure.
Aditionallly, AI assisted code/text does not imply the work is sub standard. The use of tools to improve research productivity is neither new nor inherently problematic. What matters is if the result is 1)technically correct, 2)reproducible, and 3)both scientifically/practically useful.
Furthermore, we respectfully note that the review doesn't identify a specific error beyond the figure, such as a specific methodological error, incorrect result/interpretation of result, yet claims the " experiments do not provide sufficient evidence to establish the effectiveness or generalizability of the proposed methods". We question if the review provides enough justification for a soundness score of 1.
Regardless, we have made the paper’s main question clearer. Retrieve-then-rerank is standard. Our contribution is a practical way to decide whether an existing embedding index can support a model upgrade before committing to a full re-embedding. Section 2 now introduces T2-v1 before Eq. 4 and explains why it compares the target’s reranked top 10 as the legacy candidate pool grows from 50 to 500. Section 5 turns that prediction into a concrete deployment procedure.
At (K=50), 42 of 63 development settings meet the 0.01 nDCG@10 margin by point estimate, and 26 meet it under the paired-bootstrap criterion. On the separate BRIGHT evaluation of 522 queries across four domains, the pooled gap at (K=50) is 0.0035 for Qwen3-4B→8B and 0.0367 for MiniLM→8B. On the 1M Natural Questions corpus with the same 8B target, the smallest tested depths meeting the paired-bootstrap criterion are 20, 500, and 1,000 for the 4B, 0.6B, and MiniLM sources, respectively. The new 50K comparison in Table 5 also tests BM25, ridge, and a residual MLP on the same queries, both standalone and with the same target reranker.
The probe’s limitations matter for deployment. Across 28 cells from four model pairs held out from rule development, T2-v1 issued 17 SAFE calls with zero observed false-safes under exact retrieval. In the low-recall ANN tests, two SAFE calls for 4B→8B had deployed gaps of 0.06223 and 0.02169, both above the 0.01 margin. That’s why the recipe in Section 5 uses SAFE to choose a starting depth for a pilot and checks the deployed index against exact legacy retrieval before rollout.
Thank you again for your time. It is often the harshest reviews that have the most impact.