acceptodds
Under review as a conference paper at ICLR 2027

ETBench: Evaluating Embedding Translation for Mixed-Index Retrieval

Abstract

Mixed-index retrieval searches documents encoded by different embedding models, e.g., a legacy model and its replacement. Embedding translation enables this by mapping stored source embeddings into the target space, where queries are also encoded. Translation often uses embedding alignment, as in bilingual lexicon induction, fitting a map to anchor pairs, i.e., the same documents encoded by both models, and applying it to the remaining source-only vectors. We show that better alignment does not necessarily yield better mixed-index retrieval, and prove a sufficient and necessary condition under which alignment maps can preserve the target model's top-k set. We further develop ETbench, a benchmark evaluating alignment methods for embedding translation across 24 directed encoder pairs and four corpora. It tests whether translation makes source-only documents searchable alongside native target embeddings in a mixed index. Our results show that high alignment accuracy can coexist with poor retrieval of translated documents, and common retrieval measures can be insensitive to this.

Primary area:
datasets and benchmarks
Keywords:
embedding translation, cross-model translation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_74vdOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies embedding translation for mixed-index retrieval. Instead of re-encoding a corpus after switching a model, a map fitted on anchor documents that are encoded by both models converts the remaining old vectors into the new space, where they are searched together with the native ones. The main claim is that the usual alignment metrics don't predict well how such a mixed index retrieves. The authors prove a necessary and sufficient condition based on per-document error bounds and query-dependent ranking gaps, and show two equally well-aligned maps that return different results. The authors introduce ETBench to evaluate alignment methods for embedding translation (24 encoder pairs, 4 BEIR corpora, 9 translators). The paper also includes several side studies.

Critical strengths

The evaluation protocol catches failures that aggregate scores hide, which I think is the paper's most useful contribution. Procrustes has a recovery of 90% but keeps nDCG@10 under 30% on translated documents, and the identity control scores a respectable-looking 0.66 overall while its translated documents are essentially invisible (Table 1). Anyone migrating an index should run exactly this kind of check.

The study is broad and transparent, every per-pair result is in the appendix. Fig. 2a has a good practical point: alignment quality is mostly a property of the encoder pair, while retrieval quality depends heavily on the corpus, so good alignment on one dataset tells you little about another.

Critical weaknesses

The experiments do not really follow the theory. Theorem 1(a) is the standard interval-separation argument (the appendix proof says so), its "necessary" direction only holds for an adversarially chosen map rather than a fitted one, and the construction in (b) relies on anchors that don't span the space. In practice the condition seems out of reach. Even the best held-out cosine in Table 1 (0.978, for relative) corresponds to an error of about 12 degrees, while neighboring results around rank 10 are usually less than a degree apart. So the guarantee doesn't apply to any method tested whether it retrieves well or badly. And the benchmark measures relevance retention, not what the theorem is about, projected errors against ranking gaps, or overlap with the native top-k. So the paper shows that procrustes and GW fail, but doesn't show why, and doesn't offer diagnostic for it.

Questions for authors
  1. Do the theorem's own quantities (say, the share of queries satisfying Eq. (2), or projected error relative to the rank-10 gap) predict translated-document retention better than Hits@1 or cosine? A clear yes would make the theory a usable diagnostic and raise my score.
  2. How much of the procrustes and GW loss goes away with a simple per-query score offset estimated on held-out anchors? If most of it does, this is largely a calibration problem and I'd find the paper less informative; if not, the main claim would get stronger
8 comments

Dear reviewer 74vd,

Thank you for the thoughtful review and the helpful suggestions. We are glad that you consider our evaluation protocol's ability to catch failures hidden by aggregate scores the paper's most useful contribution. We address your concerns and questions below.

  • W1 and Q1: Gap between theory and experiments, and Could Eq. (2) better than Hits@1 or cosine?

Thank you for highlighting this gap in the manuscript. Theorem 1(a) was introduced to clarify when embedding alignment suffices for embedding translation. We didn’t include certification in the evaluation since it additionally requires target queries and corpus-wide query-document comparisons, making its computational cost comparable to exhaustive retrieval.

However, we do find strong correlation between the condition of Theorem 1(a) and retention metrics in evaluation. Specifically, we now examine the connection between the certification rate ( the percentage of evaluated queries satisfying Eq. (2) of Theorem 1(a) with k=10k=10) and the real top-10 answer preservation as well as nDCG@10 retention used in the evaluation, as shown in the table blow. We can see that Theorem 1(a) certificate ratio correlates significantly more strongly with nDCG@10 retention (Spearman ρ=0.903\rho=0.903) than held-out alignment metrics such as Hits@1 (0.309) or cosine (0.212), and positively with observated top-10 preservation (0.685). Consistent with Theorem 1(a), these results highlight the importance of query-specific ranking gaps in assessing retrieval quality beyond alignment scores alone.

MethodCertificate Ratio (%)Hits@1 (D0)Hits@1 (D1)Cosine (D0)Cosine (D1)nDCG@10 retention (%)Real Top-10 preservation (%)
Identity0.020.2530.2520.1760.17666.403.52
Relative1.200.9970.5200.9870.97879.601.63
Procrustes0.980.9130.8980.9480.93460.301.03
CCA1.340.9730.9360.9490.91887.101.61
GW0.200.8490.8370.8030.80359.300.64
MLP5.700.9540.9060.8670.83187.405.87
Vec2Vec0.070.0010.0010.5030.50359.902.59
EmbConv8.170.9500.8700.8790.78688.3010.18
LA2M18.720.8930.8940.8310.85294.1020.21
HMoE12.870.8340.8120.8820.85994.5018.57
Spearman (ρ\rho) with retention0.9030.2480.3090.3090.212/0.842
Spearman (ρ\rho) with real Top-10 preservation0.685-0.164-0.067-0.176-0.2240.842/

W2 & Q2 : Why Procrustes, GW fail, and could Procrustes, GW improve after held-out calibration?

Per-query offset calibration does not fully resolve the retrieval failures of Procrustes and GW. For each query, we estimate the offset as the mean translated-minus-native target score difference over held-out anchors, then subtract it from all translated-document scores while leaving native scores unchanged. As shown below, this correction does not consistently improve retrieval across methods. We also evaluate oracle calibration, which uses test relevance labels to select the offset maximizing each query’s full-index nDCG@10. We can see that Procrustes and GW achieve only 68.72% and 67.92% retention, respectively, indicating that a query-specific score bias alone cannot fully explain their retrieval losses.

MethodRetentionHeld-out calibrationOracle calibration
Procrustes59.5%64.51% (↑\uparrow 5.01)68.72% (↑\uparrow 9.22)
GW58.09%64.25% (↑\uparrow 6.16)67.92% (↑\uparrow 9.22)
HMoE92.36%90.21%(↓2.15)98.72% (↑\uparrow 6.36)
LA2M92.77%91.60%(↓1.18)97.37%(↑\uparrow 4.60)

Best regards,

The authors from Submission4958

Reviewer 74vd : do you find your concerns addressed (particularly the gap between the experiments and the theory)?

Thank you for running both analyses.

Q2 is resolved for me, and the result strengthens the main claim. Q1 is only partly resolved. You used full-index retention for the correlation, but my question was about translated documents. After pairing your certificate ratios with the D1 nDCG@10 retention column of Table 1, I got Spearman 0.65, against 0.70 for held-out cosine. So across these 10 methods, the condition doesn’t predict retention of translated documents better than cosine.

Could you compute the same correlation against D1 retention across the 96 pair-corpus cells for each method? I think 10 method averages are too few to settle this.

Reviewer_2BmsOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies embedding translation for mixed-index retrieval: searching a single vector index that stores some documents encoded by a source model (translated into a target model's space via an alignment map fitted on anchor pairs) alongside documents natively encoded by the target model. The central claim of the paper is that better embedding alignment does not necessarily yield better mixed-index retrieval. The contributions are threefold: (1) a theoretical result (Theorem 1) giving a necessary and sufficient condition under which alignment-error bounds preserve the target's top-k set, plus a construction showing arbitrarily small alignment error can produce disjoint retrieval answers; (2) ETBench, a benchmark spanning 18 encoders, 24 directed encoder pairs, and 4 BEIR corpora, evaluating 9 translators plus an identity control and measuring retrieval of translated and native documents separately; and (3) empirical findings that high alignment accuracy (Hits@1, cosine) can coexist with poor retrieval of translated documents, that aggregate retrieval measures conceal these losses, and that the alignment-translation gap extends to KV-cache translation and clustering. The best retriever (hmoe, R=0.945) is only 7th in held-out Hits@1, while the best aligner by Hits@1 (cca) ranks 5th in retrieval retention

Critical strengths

Clear and well-motivated conceptual distinction. Rigorous theory with a proven if-and-only-if condition. Theorem 1(a) gives a tight (sufficient and necessary) condition α_d + α_b < θ_b − θ_d, and Theorem 1(b) constructs two orthogonal maps that fit all anchors, achieve perfect held-out Hits@1, yet produce disjoint vs. identical answers. Comprehensive, well designed benchmark, ETBench covers a substantial matrix (24 directed pairs × 4 corpora = up to 96 cells per method, 10 methods including a sensible identity control and representative paired/unpaired/direct translators). The protocol is carefully specified: a fixed D0/D1/D2 partition, queries always target-encoded, the evaluator holding f_t(D1) for the all-native reference, and anchors excluded from both indexes. The grouping methodology (full / D1 / D2) is the paper's most valuable empirical instrument. It convincingly demonstrates that full-index aggregate scores hide catastrophic translated-document losses (e.g., identity R=0.664 with R1=0.054, R2=1.117; vec2vec R=0.599 with R1=0.004). This directly supports the "common retrieval measures can conceal" claim.

Generalization beyond retrieval.: RQ4 (KV-cache translation on Qwen3-4B→8B with MMLU, and MTEB clustering) and RQ5 (two-source mixed index) show the alignment-vs-downstream gap is not an artifact of one task. The finding that GW has the highest cache Hits@1 (0.982) but near-chance MMLU (0.263/0.317) is a striking and persuasive illustration

Critical weaknesses

No statistical significance or variance reporting in the main results. Table 1 reports means over 24 pairs × 4 corpora but gives no variance, confidence intervals, or significance tests. Given the paper's core claim that rankings by alignment vs. retrieval differ, the robustness of these rank differences (e.g., hmoe 7th in Hits@1 but 1st in retrieval) would be much stronger with dispersion statistics. Per-cell results exist (Appendix I) but no aggregate uncertainty is presented.

Geometry-prediction result (RQ3) is largely negative but under-explored. The best mean Spearman correlation is only ρ=0.29 (mutual k-NN@10) with wide, sign-flipping ranges. This supports "geometry cannot robustly predict translation," but the paper does not probe whether query-dependent quantities (the ranking gaps that Theorem 1 says actually matter) could predict it - which would have closed the loop between theory and practice.

Causal attribution of "aligning with target encodings is key" is correlational. The claim that translation-oriented methods (embconv, la2m, hmoe) win because they exploit D2 native encodings is plausible, but these methods also differ in capacity, locality, and training objective. The paper does not isolate the D2-exploitation factor (e.g., an ablation adding target-space-awareness to an otherwise identical map), so the mechanism remains inferential.

Definition of the "translation" default is slightly confusing. Section 3 states "pairwise translation from D1 to D2 is assumed by default," which is awkward wording - translation maps from source space to target space for D1 documents; D2 are native. This phrasing risks confusing the reader about what is translated.

Questions for authors

In Table 1, can you provide variance/standard deviations or significance tests for the key rank inversions (hmoe 7th in Hits@1 vs. 1st in retrieval; cca 1st in Hits@1 vs. 5th in retrieval)? How stable are these across the 96 cells?

Theorem 1(b) requires anchors that do not span the space. Empirically, with the large anchor budgets in RQ2 (up to ~1M, 19.4% of FEVER), do anchors effectively span the embedding space? If so, how do you reconcile the empirical "arbitrarily small error, drastically different retrieval" claim with the appendix Remark that no such pair exists under spanning anchors?

RQ3 shows encoder geometry poorly predicts translation (best ρ=0.29). Theorem 1 says ranking gaps matter. Did you test whether query-dependent quantities (e.g., the distribution of θ_b − θ_d gaps) predict retrieval retention better than geometry alone?

Can you isolate the claimed causal role of "aligning with target encodings" via an ablation, rather than comparing methods that differ along multiple axes simultaneously?

For the identity control with width mismatch, you "zero-pad the narrower space to the wider one." For pairs like qwen3-8b→qwen3-0.6b (4096→1024), how is identity defined (truncation? the table shows identity R values for this pair)? Please clarify the dimension-reduction case.

What explains the asymmetry in directed pairs (e.g., qwen3-0.6b→qwen3-8b vs. qwen3-8b→qwen3-0.6b)? Do the conclusions hold symmetrically?

4 comments

Dear reviewer 2Bms,

Thanks for your careful review and constructive feedback. We appreciate your recognition of the clear distinction between alignment and translation. We address your concern and questions as below:

W1 & Q1: Missing variance and uncertainty estimates in the main results. How stable are the key alignment–retrieval rank inversions across the evaluation cells?

This concern is also raised in Reviewer RHWm’s W3. We repeated the stochastic methods using five training seeds (42–46), keeping all other settings identical to those in Table 1. The details are reported in our response to Reviewer RHWm’s W3.

Note the key rank inversions persist in the five-seed averages: HMoE ranks seventh in Hits@1 (0.8125 ± 0.0013) but first in full-index retrieval retention (0.9445 ± 0.0016), whereas CCA ranks first in Hits@1 (0.9388) but fifth in retrieval retention (0.8713). These results support the stability of our aggregate finding under randomness. We will incorporate these results into the revised manuscript and welcome the reviewer’s suggestions for further robustness analyses.

W2 & Q3: The connection between encoder geometry, query-dependent ranking gaps, and retrieval remains under-explored.

This concern is closely related to reviewer RHWm’s Q1 and reviewer 74vd’s W1 & Q1. As detailed in those responses, we added an analysis of query-dependent certification rates motivated by Theorem 1(a). Across the ten methods, the certification rate w.r.t. Theorem 1’s condition correlates strongly with nDCG@10 retention (Spearman ρ = 0.903), compared with 0.309 for held-out Hits@1 and 0.212 for held-out cosine. It also correlates positively with observed top-10 preservation (ρ = 0.685). These results support the empirical relevance of considering translation errors relative to query-specific ranking gaps.

W3 & Q 4: The causal role of exploiting target-space encodings is not isolated. Can a controlled ablation isolate the role of target-space awareness?

Using the same MLP backbone, we ablate global and local structure losses. Let TT be the translator, xi,yix_i,y_i the source/target encodings of anchor ii, zi=T(xi)z_i=T(x_i), and Dcos⁡D_{\cos} cosine distance. Define LS=1∣S∣∑(i,j)∈S∣Dcos⁡(zi,zj)−Dcos⁡(yi,yj)∣L_S=\frac{1}{|S|}\sum_{(i,j)\in S}|D_{\cos}(z_i,z_j)-D_{\cos}(y_i,y_j)|. The global loss is Lglobal=LPL_{\mathrm{global}}=L_P, where PP contains all ordered anchor pairs within a batch; the local loss is Llocal=LNL_{\mathrm{local}}=L_N, where N⊆PN\subseteq P connects each anchor to up to 100 nearest neighbors.

Hits@1CosinenDCG retention
Vanilla MLP0.90530.73390.8500
Vanilla MLP + global structure loss0.8829 (↓0.0224)0.7343 (↑0.0004)0.8781 (↑0.0281)
Vanilla MLP + local structure loss0.8531 (↓0.0522)0.7241 (↓0.0098)0.8701 (↑0.0201)
Vanilla MLP + both0.8668 (↓0.0385)0.7247 (↓0.0092)0.8812 (↑0.0312)

As shown in the table, adding target-space structure constraints improves the MLP’s retrieval retention despite reducing Hits@1 and Cosine metrics. These additional results further support our central claim: better embedding alignment does not necessarily lead to better retrieval.

  • W4 & Q5: The default translation setting is ambiguously.

We zero-pad the lower-dimensional space to match the higher-dimensional one, preserving its pairwise distances and cosine similarities. The configurations used in our experiments are reported in Appendix E (“Translation Methods”). We will carefully revise this section to eliminate ambiguity and ensure clarity.

  • Q2: Does the non-spanning-anchor assumption in Theorem 1(b) remain relevant at large anchor budgets?

We agree with that when source anchors span the space, exact fitting uniquely determines a linear map, so Theorem 1(b)’s construction no longer applies. However, its “arbitrarily small error” statement is an existence result under the stated assumptions, not an empirical claim about large-anchor settings or an explanation of all failures. Those settings are addressed by Theorem 1(a), which applies regardless of anchor span: even a uniquely fitted map can have held-out errors, whose effects depend on query-specific ranking gaps. We will clarify this distinction in the revision.

  • Q6: What explains directional asymmetry, and do the conclusions hold in both directions?

Reversing an encoder pair changes the target query representations, the native document space, and the mapping task. Retention is also measured against a different native baseline. These changes provide possible explanations for directional asymmetry. We examined three encoder pairs and their reverse directions. Our main conclusion remains supported in both directions.

Translation directionBest-Hits@1 ≠ Best-retention
bge-large → NV-Embed-v2√
NV-Embed-v2 → bge-large√
GritLM → NV-Embed-v2√
NV-Embed-v2 → GritLM√
Qwen3-0.6B → Qwen3-8B√
Qwen3-8B → Qwen3-0.6B√

Best,

The authors from Submission4958

Reviewer 2Bms: do you think your concerns (particularly concerning statistical rigor) have been addressed?

Dear Reviewer 2Bms,

Thank you again for your time and effort in reviewing our manuscript. As the discussion period is coming to a close, we wanted to kindly check whether our responses have addressed your concerns. If any questions remain or further clarification would be helpful, we would be glad to provide it before the deadline.

In addition, the further experiments in our response to Reviewer 74vd may also help address your W2 & Q3 ("The connection between encoder geometry, query-dependent ranking gaps, and retrieval remains under-explored"). Within each of the ten methods, across all 96 pair–corpus cells, the certificate ratio (Eq. 2), which compares translation errors with query-specific ranking gaps, correlates more strongly with D1 nDCG@10 retention than held-out cosine does (mean ρ = 0.52 vs. 0.17). Across methods, it also correlates significantly more strongly with full-index retention than both cosine and Hits@1 (ρ = 0.90 vs. 0.21 and 0.31; p < 0.02).

Best regards,
The Authors of Submission4958

Dear Reviewer 2Bms,

As the discussion period draws to a close, we just wanted to gently follow up on our responses. In them, we added five-seed results for the main table (W1 & Q1), an analysis linking query-specific ranking gaps to retrieval (W2 & Q3), a controlled ablation of target-space structure (W3 & Q4), and clarifications on the identity baseline, the non-spanning-anchor assumption and directional asymmetry (W4, Q2, Q5, Q6). We would be grateful to hear whether our responses have addressed your concerns, and we would be glad to address any remaining questions.

Thank you very much for your time and thoughtful review.

Best regards,

The Authors of Submission4958

Reviewer_RHWmOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies embedding translation for mixed-index retrieval. It derives a necessary and sufficient condition for robust top-kk preservation and introduces ETBench, which evaluates alignment and retrieval separately for translated and native documents.

Critical strengths
  1. The paper identifies an important failure mode: strong alignment and reasonable aggregate retrieval scores can hide severe losses on translated documents.
  2. The theoretical analysis clearly relates retrieval stability to angular errors and query-dependent ranking margins.
  3. The provenance-based evaluation provides useful diagnostics across diverse translation methods.
Critical weaknesses
  1. Limited differentiation from prior work. Embedding-Converter already evaluates mixed converted/native corpora against native target-model results (Appendix D.3). The main benchmark contribution therefore appears to be grouped diagnostics and systematic comparison, rather than a new retrieval setting.
  2. Weak connection between theory and experiments. ETBench does not evaluate the proposed condition’s certification coverage or relationship with observed top-kk changes, despite having the native embeddings needed for this analysis.
  3. Limited robustness evidence. The main evaluation uses four relatively small corpora and a fixed partition. Uncertainty estimates and sensitivity to splits and mixture proportions are needed.
Questions for authors

How often does the theoretical condition certify top-kk preservation on ETBench, and is it practically informative?

6 comments

Dear reviewer RHWm,

Thank you for the thoughtful review and the helpful suggestions. We are pleased that you found the failure mode we identify important. We here answer your questions as follow:

W1: Limited differentiation from prior work.

Below we compare Embedding-Converter’s mixed-corpus evaluation (App. D.3) with ETBench. We do not claim the mixed-index setting itself as a contribution. Our central claim is that better embedding alignment does not necessarily lead to better mixed-index retrieval, and aggregate retrieval scores can conceal severe losses on translated documents. This identifies and exposes a mismatch that has not been fully exploited in prior studies. ETBench systematically examines this mismatch by evaluating alignment alongside retrieval quality, with retrieval metrics reported separately for translated and native documents. Compared with App. D.3, which evaluates one translator on one same-family encoder pair, ETBench covers 9 translators and 24 directed pairs from 18 encoders.

Embedding-Converter (App. D.3)ETBench (ours)
Embedding models1 directed pair, gecko003→gecko004 (one model family)18 encoders forming 24 directed pairs, across families, sizes and dimensions
Translators1 translator9 translators
Retrieval metricsFull nDCG@10 onlyRetention of nDCG@10, MRR@10 and Recall@100 reported separately for full, translated (D1) and native (D2) documents
Alignment metricsNoneHeld-out Hits@1 and cosine, set against retrieval
Theory on explaining mechanismNoneTheorem 1: a necessary and sufficient condition under which alignment errors preserve the top-k set, and a construction showing that alignment scores alone cannot certify retrieval

W2 & Q1: Weak Connection between theory and experiments. Theoretical condition on ETBench.

In the paper, the theorem characterises when alignment is sufficient to preserve retrieval results. We didn’t directly use it in the evaluation since it would incur even higher computation complexity than exhaust Recall@kk or nDCG@kk. However, we do find that the certificate condition of Theorem 1 is strongly correlated to both the retention metric used in the evaluation and real top-k answer preservation. Specifically, let the certificate ratio be the percentage of evaluated queries satisfying Eq. (2) of Theorem 1(a) with k=10k=10 . As shown in the table below, It correlates positively with top- kk preservation (Spearman ρ=0.685\rho=0.685) and strongly with retrieval retention (ρ=0.903\rho=0.903), whereas conventional alignment metrics show significantly weaker associations.

MethodCertificate Ratio (%)Hits@1 (D0)Hits@1 (D1)Cosine (D0)Cosine (D1)nDCG@10 retention (%)Real Top-10 preservation (%)
Identity0.020.2530.2520.1760.17666.403.52
Relative1.200.9970.5200.9870.97879.601.63
Procrustes0.980.9130.8980.9480.93460.301.03
CCA1.340.9730.9360.9490.91887.101.61
GW0.200.8490.8370.8030.80359.300.64
MLP5.700.9540.9060.8670.83187.405.87
Vec2Vec0.070.0010.0010.5030.50359.902.59
EmbConv8.170.9500.8700.8790.78688.3010.18
LA2M18.720.8930.8940.8310.85294.1020.21
HMoE12.870.8340.8120.8820.85994.5018.57
Spearman (ρ\rho) with retention0.9030.2480.3090.3090.212/0.842
Spearman (ρ\rho) with real Top-10 preservation0.685-0.164-0.067-0.176-0.2240.842/

Here both retention and preservation are measured in the mixed-index setting. We will incorporate these results into the revised manuscript to strengthen the connection between theory and experiments.

W3: Limited robustness evidence.

To address reviewer’s concern, we repeat all stochastic methods with five training seeds (42-46), keeping other settings identical to Table 1. The variation in retention across seeds is small compared with the differences between methods, and they are also support our manuscripts. We will incorporate these results into the revised manuscript.

MethodHits@1 (D1)Cosine (D1)nDCG@10[Full]Δ [Full]
procrustes0.89834 ± 0.001330.93434 ± 0.000220.60270 ± 0.00022−0.00030
cca0.938780.918650.87134+0.00000
GW0.83732 ± 0.003810.80383 ± 0.000220.58776 ± 0.00497−0.00524
Vec2Vec0.00891 ± 0.001890.50329 ± 0.001900.59776 ± 0.00419−0.00124
EmbConv0.87081 ± 0.001010.78618 ± 0.001150.87696 ± 0.00267−0.00604
LA2M0.89418 ± 0.001320.85178 ± 0.001300.94063 ± 0.00011−0.00037
HMoE0.81254 ± 0.001340.85923 ± 0.001290.94454 ± 0.00164−0.00046

Best regards,
The authors of Submission 4958

Reviewer RHWm: do you find that your concerns were addressed, particularly about novelty?

Dear Reviewer RHWm,

Thank you again for your time and effort in reviewing our manuscript. As the discussion period is coming to a close, we wanted to kindly check whether our responses have addressed your concerns. If any questions remain or further clarification would be helpful, we would be glad to provide it before the deadline.

In addition, the further experiments in our response to Reviewer 74vd may also help address your W2 & Q1 ("Weak Connection between theory and experiments."). Within each of the ten methods, across all 96 pair–corpus cells, the certificate ratio (Eq. 2), which compares translation errors with query-specific ranking gaps, correlates more strongly with D1 nDCG@10 retention than held-out cosine does (mean ρ = 0.52 vs. 0.17). Across methods, it also correlates significantly more strongly with full-index retention than both cosine and Hits@1 (ρ = 0.90 vs. 0.21 and 0.31; p < 0.02).

Best regards,
The Authors of Submission4958

Comments on the paper

2 comments
Area_Chair_mucrOpenReview ↗

Please submit author responses.

Dear Authors,

Given the deadline on 18 November, please submit your responses to the reviews to facilitate a productive discussion with the reviewers.

Thanks, AC

Dear AC,

Thank you for the reminder. We have submitted our rebuttal and sincerely appreciate the time and effort you and the reviewers have devoted to evaluating our work. We look forward to the discussion.

Best regards,

The authors of Submission 4958

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.