acceptodds
Under review as a conference paper at ICLR 2027

LEARNING WHAT MATTERS: REWEIGHTING FROZEN EMBEDDINGS FOR DENSE RETRIEVAL

Abstract

Dense retrievers enable efficient semantic search using query and document embeddings. Once deployed, however, improving ranking quality of a dense retriever requires careful balancing of its performance with the cost of changing existing retrieval infrastructure. This motivates adapting how retrieved documents are scored while retaining the encoder, stored embeddings, and index. Yet fixed similarity scoring does not adapt dimension weights to the candidates competing for relevance. We introduce Shared Signed Gate (SSG), a lightweight reranker that learns which embedding dimensions help distinguish relevant documents from retrieved hard negatives. Using explicit dimension-level supervision, a small network predicts candidate-conditioned residual weights from the query and candidate statistics. Across nine encoders and seven BEIR datasets, SSG improves average nDCG@10 for every encoder and achieves the highest average among evaluated baselines for all three encoders used in detailed comparisons. It also improves rankings produced by other retrieval methods. With BGE-S, SSG adds approximately 7.8% to measured end-to-end latency while preserving the encoder, corpus embeddings, and retrieval index. These results show that candidate-conditioned dimensional weighting can improve ranking quality while reusing the representations and retrieval infrastructure already in place.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Dense retrieval, lightweight reranking, frozen embeddings, candidate-conditioned adaptation, dimension-wise reweighting, relevance-supervised gating

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_CP46OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Shared Signed Gate (SSG), a lightweight reranker that improves dense retrieval ranking quality while retaining the encoder, stored embeddings, and retrieval index. SSG uses explicit dimension-level supervision to learn candidate-conditioned residual weights that distinguish relevant documents from hard negatives. Across nine encoders and seven BEIR datasets, SSG improves average nDCG@10 for every encoder and outperforms evaluated baselines in detailed comparisons.

Critical strengths
  1. The paper proposes a simple yet effective approach to improving dense retrieval by learning dimension-wise weights. The relevance-derived supervision from positive documents and hard negatives is intuitive and provides a meaningful training signal.
  2. SSG improves retrieval quality without modifying the pretrained embedding model, recomputing document embeddings, or rebuilding the retrieval index, making it relatively easy to integrate into existing retrieval systems.
  3. The paper evaluates SSG across nine embedding models and seven BEIR datasets, achieving consistent improvements in average nDCG@10 across all nine models. The experiments also include comparisons with several baselines and detailed ablation studies.
Critical weaknesses
  1. Insufficient and unfair reranker comparisons. The paper lacks comparisons with stronger and more recent rerankers, such as BGE-reranker, Jina-reranker, and Qwen-reranker. Moreover, the MiniLM cross-encoder is evaluated in a zero-shot setting, while SSG is trained on each target dataset, making the comparison potentially unfair.
  2. Limited practicality and generalizability. SSG requires separately trained models for each dataset–encoder pair, along with domain-specific relevance annotations. This introduces additional annotation and training costs, limiting its practical applicability. Moreover, cross-domain transfer is not evaluated, leaving its ability to generalize to unseen domains unclear.
  3. Missing query-only fine-tuning baseline. Given the availability of supervised training data, a simpler alternative would be to fine-tune only the query encoder, e.g., using LoRA, while keeping document embeddings and the retrieval index fixed. This could improve retrieval effectiveness without introducing an additional reranking stage and its associated inference overhead. The paper does not compare against this alternative.
  4. Limited improvements relative to inference overhead. SSG introduces substantial additional latency for larger embedding models (e.g., 23.3% for Qwen3-8B), while the improvements on several datasets are marginal. For example, with Qwen3-8B, nDCG@10 increases only from 0.611 to 0.612 on FiQA, 0.915 to 0.916 on TREC-COVID, and 0.671 to 0.673 on NQ. These limited gains raise concerns about its practical cost-effectiveness, particularly for stronger retrievers.
Questions for authors
  1. Could the authors compare SSG with stronger rerankers, such as BGE-reranker, Jina-reranker, and Qwen-reranker, and provide a fairer comparison under similar supervision settings?
  2. How well does SSG generalize across different domains? Can a model trained on one dataset be directly applied to another without additional annotations and retraining?
  3. Why not directly fine-tune the query encoder using LoRA with the same training data, while keeping the document embeddings and index fixed? How would this compare with SSG in terms of retrieval performance and inference efficiency?
  4. Given the substantial inference overhead on larger embedding models and the marginal improvements on several datasets, in what scenarios would SSG offer a favorable accuracy–latency trade-off?
6 comments
AuthorsOpenReview ↗

Response to Reviewer CP46

We sincerely thank the reviewer for the careful reading and constructive suggestions. We appreciate your positive assessment of our methodology, experiments, and ablation studies. Your feedback has helped us improve the clarity of the paper and strengthen its empirical evaluation. Following your suggestions, we have added experiments addressing each concern. Revised passages in the manuscript are marked in blue.

W1 / Q1. Stronger rerankers and comparable supervision

We thank the reviewer for suggesting stronger reranker comparisons. We now evaluate BGE-reranker-v2-m3, Jina-reranker-v2-base-multilingual, and Qwen3-Reranker-0.6B, alongside MiniLM, both without target-domain adaptation and with supervised LoRA fine-tuning. For each encoder, SSG and all text rerankers operate on the same top-100 retrieved candidates. Each reranker's LoRA adapter is fine-tuned separately using the corresponding encoder's candidates and target-domain relevance labels. We search eight LoRA configurations per dataset (two ranks × four learning rates), selecting checkpoints by validation nDCG@10.

Average test nDCG@10 across seven datasets is reported below.

MethodBGE-SQwen3-8B
Base Retrieval0.5190.647
MiniLM + LoRA0.4990.523
BGE-reranker-v2-m3 + LoRA0.5370.589
Jina-reranker-v2 + LoRA0.5330.589
Qwen3-Reranker-0.6B + LoRA0.5620.601
SSG (Ours)0.5590.661

With BGE-S, SSG nearly matches the strongest fine-tuned reranker (0.559 vs. 0.562 nDCG@10) while achieving approximately 75× lower end-to-end latency (44.08 vs. 3305.66 ms/query). With Qwen3-8B, SSG achieves higher average effectiveness (0.661 vs. 0.601) at approximately 8.3× lower latency (445.67 vs. 3714.31 ms/query). These results demonstrate that SSG offers competitive or superior ranking effectiveness at a fraction of the inference latency of larger neural rerankers.

The complete comparisons appear in Section 5.5 and Table 4, with training settings in Table 7 and dataset-level results in Appendix A.6, Table 11.

W2 / Q2. Practicality and cross-domain generalization

We thank the reviewer for raising the question of cross-domain transfer without additional annotations or retraining. We have added experiments that train SSG separately on NFCorpus, FiQA, and SciFact using BGE-S, then directly apply each selected predictor to the other six datasets without target-domain fine-tuning or hyperparameter selection. We additionally evaluate Learning to Select (LTS) and CMC under the same source-to-target transfer setting, retaining each method's original inference procedure.

Unseen datasets improved over Base Retrieval:

Training sourceSSG (Ours)CMCLTS
NFCorpus6 / 64/60/6
FiQA6 / 64/60/6
SciFact2 / 63/60/6

Notably, SSG trained on either NFCorpus or FiQA improves retrieval across all six unseen datasets, demonstrating consistent transfer from these sources without target-domain supervision. For example, NFCorpus-trained SSG improves ArguAna from 0.592 to 0.612 and NQ from 0.492 to 0.513, while even slightly outperforming target-trained SSG on Touché-2020 (0.352 vs. 0.350).

Importantly, these models are trained on individual domain-specific datasets rather than large, heterogeneous collections, and none is explicitly optimized for cross-domain transfer. Despite this setting, SSG demonstrates stronger transfer consistency than the evaluated baselines, supporting its practicality when new target-domain annotations are unavailable.

The complete cross-domain results are provided in Appendix A.7, Table 13.

AuthorsOpenReview ↗

Response to Reviewer CP46

W3 / Q3. Query-only fine-tuning with LoRA

We agree that query-only LoRA is a relevant alternative and have added it for both BGE-S and Qwen3-8B. The baseline adapts the query encoder using target-domain supervision while keeping document embeddings and the retrieval index fixed. We evaluate eight configurations per dataset, with up to 20 training epochs and validation-based hyperparameter and checkpoint selection.

Average test nDCG@10 and end-to-end latency (ms/query) are reported below.

MethodBGE-S: nDCG@10BGE-S: LatencyQwen3-8B: nDCG@10Qwen3-8B: Latency
Base Retrieval0.51942.710.647440.85
Query-encoder LoRA0.51048.180.637464.87
SSG (Ours)0.55944.080.661445.67

Under this protocol, SSG achieves higher average effectiveness and lower measured end-to-end latency than query-only LoRA for both encoders. With BGE-S, SSG achieves 0.559 versus 0.510 nDCG@10; with Qwen3-8B, it achieves 0.661 versus 0.637. These results show that directly fine-tuning the query encoder does not provide a better accuracy-latency trade-off than SSG under the evaluated settings, despite avoiding a separate reranking stage.

Implementation details and results appear in Tables 4, 7, 11 and 12.

W4 / Q4. Favorable accuracy-latency trade-off

We thank the reviewer for raising this concern. To better establish scenarios where SSG offers a favorable accuracy-latency trade-off, we additionally compare it against query-encoder LoRA and larger LoRA-fine-tuned rerankers under a common NVIDIA A100 setup. While our original L4 profiling reports 23.3% overhead for Qwen3-8B, the new evaluation allows direct comparison with alternative methods under matched hardware conditions. As detailed in Appendix A.5, this overhead is dominated by accessing the retrieved document vectors; candidate-statistics computation, gate prediction, and rescoring together generally require approximately 2-3 ms/query. The following table reports seven-dataset macro-average nDCG@10 and end-to-end latency (ms/query), including query encoding, retrieval, and applicable reranking costs.

MethodBGE-S: nDCG@10BGE-S: LatencyQwen3-8B: nDCG@10Qwen3-8B: Latency
Base Retrieval0.51942.710.647440.85
Query-encoder LoRA0.51048.180.637464.87
MiniLM (LoRA)0.499301.700.523693.71
BGE-reranker-v2-m3 (LoRA)0.5372039.670.5892450.97
Jina-reranker-v2 (LoRA)0.533753.550.5891161.00
Qwen3-Reranker-0.6B (LoRA)0.5623305.660.6013714.31
SSG (Ours)0.55944.080.661445.67

In this matched setup, SSG adds only 3.2% and 1.1% latency over Base Retrieval with BGE-S and Qwen3-8B, respectively. With BGE-S, SSG nearly matches the strongest fine-tuned reranker at approximately 75× lower end-to-end latency. More notably, with Qwen3-8B, SSG not only surpasses all evaluated fine-tuned rerankers in macro-average nDCG@10 (0.661 vs. at most 0.601), but also achieves approximately 8.3× lower end-to-end latency than Qwen3-Reranker-0.6B. Although gains over Base Retrieval are marginal on some datasets, these results highlight SSG’s ability to deliver competitive ranking effectiveness at substantially lower end-to-end inference latency than larger neural rerankers, even when operating with a strong embedding encoder. SSG is therefore particularly well suited to latency-sensitive systems that must preserve their existing encoder, stored document embeddings, and index. Detailed results and profiling are in Section 5.5 and Appendix A.6.

AuthorsOpenReview ↗

Thank You for Your Feedback!

We sincerely hope that these clarifications and additional experimental results have addressed your concerns and strengthened your assessment of our work. We would be grateful if you could kindly reconsider your rating in light of the new evidence.

Reviewer_yvVFOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper is about the problem of fetching the relevant documents to a user query (e.g., in RAG systems). Dense retrievers, coupled with post-retrieval reranking methods, are commonly used to address this task. The authors state that once such systems are deployed, improving ranking quality through a new embedding model requires re-encoding the corpus and rebuilding the index. They instead propose Shared Signed Gate (SSG), a learned lightweight reranker that ranks the top-100 retrieved documents by identifying (and reweighting) the embedding dimensions that distinguish relevant documents most from the retrieved hard negatives. Instead of weighting each embedding dimension equally (as in dot-product), SSG learns for each query which dimensions to amplify or attenuate. At train time, a target vector is built for each query by comparing the per-dimension contributions of relevant documents and top-ranked hard negatives. A small MLP is trained to predict this target from the query embedding and the mean and std of the retrieved candidate embeddings. At test time, the predicted vector reweights the embedding dimensions, and the candidates are rescored and reranked. The model is evaluated on seven BEIR datasets where a separate predictor is trained for each dataset-encoder pair. Their results show that SSG improves the results (on average) for every encoder.

Critical strengths

The paper addresses an practical problem in a realisting setting: that of deplying dense retrievers without having to re-encode the corpus, change the index or replace the encode. The proposed fix is simple and lightweight (a query-conditioned MLP reweights the embedding dimensions) and the results indicate that it consistently improves all the tested encoders an a variety of datasets. The results also show that the query-conditioning is important for the weighting (a global gate gains almost nothing), re-enforcing the value of their method. This gives practitioners a cheap add-on that works across encoders of very different sizes.

Critical weaknesses

None

Questions for authors

None

1 comment
AuthorsOpenReview ↗

Thank You for Your Assessment!

We sincerely thank you for your thoughtful review and positive assessment of our work. We especially appreciate your recognition of SSG’s practical value as a lightweight addition to existing retrieval systems and the importance of query conditioning. Your encouraging feedback means a great deal to us. We have further strengthened the revised manuscript with additional baseline comparisons, cross-domain transfer experiments, and analyses of supervision requirements and inference efficiency. Thank you again for your time and careful consideration.

Reviewer_ZZmEOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces Shared Signed Gate (SSG), a lightweight reranker over frozen query and document embeddings. It constructs dimension-level training targets from the difference between relevant-document and hard-negative contributions. A small network predicts a gate from the query and candidate statistics, then uses nonnegative residual weights to rerank the original top-100 documents. Separate predictors are trained for each dataset–encoder pair. Experiments cover nine encoders and seven BEIR datasets, with detailed baseline comparisons for three encoders

Critical strengths
  1. The method addresses a real deployment constraint and shows broad gains. SSG preserves the encoder, stored document vectors, and first-stage index. It improves the reported average for all nine encoders. For BGE-S, average nDCG@10 rises from 0.519 to 0.559, with 16.80 ms added latency in the reported setup. This is a useful operating point when rebuilding an index is costly.

  2. The appendix contains meaningful controls. The paper compares dimensional target regression with pairwise and listwise ranking losses, removes candidate statistics, changes the scoring rule, and reports five training seeds. These experiments provide evidence for the chosen design within the tested setting.

Critical weaknesses
  1. A very close prior method is missing from the comparison. Learning to Select: Query-Aware Adaptive Dimension Selection for Dense Retrieval, released in February 2026, already constructs dimension targets from relevant-versus-hard-negative contributions, learns a predictor from frozen embeddings, and leaves document vectors and the index unchanged. SSG adds candidate statistics and continuous residual weighting, which are meaningful differences. However, comparison with unsupervised DIME/ECLIPSE and global adapters does not establish the value of these additions over that supervised approach. The query-only ablation is helpful but does not reproduce its target distribution, objective, and masking rule. This missing comparison directly affects my assessment of novelty and significance.
Questions for authors
  1. How does SSG compare with Learning to Select using the same training labels, encoder, candidate pool, and validation budget? Can the experiment separate the contributions of candidate conditioning and continuous weighting?
  2. How sensitive are the baseline comparisons to tuning and training budget? Appendix A.2 uses a fixed configuration for IMRNNs and three training epochs for CMC, while SSG uses validation-based selection over a longer schedule. A matched-budget check would make the comparative conclusion more convincing.
  3. How much target-domain supervision is needed for the gains? The current results use a separate predictor for each dataset–encoder pair, including very small query collections. A learning curve or results across query splits would help assess practical usefulness.
8 comments
AuthorsOpenReview ↗

Response to Reviewer ZZmE

We thank the reviewer for the careful assessment and for recognizing SSG’s deployment relevance and the value of its controlled experiments. Your suggestions helped us strengthen the comparison with closely related work and examine the effects of baseline tuning and supervision. We have added the experiments described below. Revised manuscript passages are marked in blue; the additional fixed-candidate diagnostic for Learning to Select is reported in this rebuttal.

W1 / Q1. Comparison with Learning to Select and contributions of candidate conditioning and continuous weighting

We agree that Learning to Select is a closely related supervised baseline and should have been included. We implemented it using the same encoder checkpoints, query partitions, and relevance judgments as SSG, while retaining its own relevance-derived target, KL objective, and binary masking rule. Hyperparameters and checkpoints are selected using validation nDCG@10.

The revised Table 1 reports the following seven-dataset macro-average test nDCG@10:

EncoderLearning to SelectSSG
BGE-S0.4620.559
Arctic-M0.5540.588
Qwen3-0.6B0.5770.603

For the main comparison, Learning to Select performs full-corpus retrieval with its predicted query mask. To directly address your request for a shared candidate pool, we additionally evaluate the saved BGE-S Learning to Select predictor on the exact top-100 candidate lists used by SSG. We retain its original validation-selected checkpoint and retention fraction, changing only the inference candidate pool.

BGE-S evaluationMean nDCG@10
Base Retrieval0.519
Learning to Select - full-corpus retrieval0.462
Learning to Select - fixed SSG candidate pool0.478
SSG - same fixed candidate pool0.559

Restricting Learning to Select to the same top-100 candidates improves its average nDCG@10 from 0.462 to 0.478, while SSG achieves 0.559. Thus, SSG’s advantage persists when both methods rank exactly the same documents. However, this comparison alone cannot attribute the difference specifically to candidate conditioning or continuous weighting, since the methods also use different training targets and objectives.

To isolate these two design choices, we additionally compare four SSG variants that combine query-only or candidate-conditioned prediction with hard selection or continuous weighting. All variants retain SSG’s dimensional targets and training loss and rerank the same candidates (Appendix A.8, Table 14). Results are seven-dataset macro-average test nDCG@10 using BGE-S.

Gate inputHard selectionContinuous weighting
Query only0.4400.532
Query + candidate statistics0.4450.559

Adding candidate statistics improves average nDCG@10 from 0.532 to 0.559 under continuous weighting. Continuous weighting also outperforms hard selection under both input settings. These ablations support the contributions of both design choices within SSG.

The related-work discussion and implementation details appear in Section 2 and Appendix A.2, with comparative results in Section 5.1.

Q2. Sensitivity to training and validation budgets

We agree that the original fixed IMRNNs configuration and three-epoch CMC schedule did not adequately address sensitivity to baseline tuning. We have expanded baseline training and allocated broader hyperparameter searches: SSG evaluates five target-temperature configurations, compared with 12 training configurations for CMC and 15 for IMRNNs. Learning to Select evaluates five target temperatures and additionally searches nine mask-retention fractions. Configurations and checkpoints are selected using validation nDCG@10. Table 1 has been updated with these results.

BaselineTraining scheduleHyperparameter search
CMC20 epochsFour learning rates × three objective-weight settings
IMRNNs30 epochsFive learning rates × three weight-decay settings
Learning to Select100 epochsFive target temperatures × nine retention fractions

After this expanded tuning, the seven-dataset averages are:

MethodBGE-SArctic-MQwen3-0.6B
CMC0.5230.5690.563
IMRNNs0.4960.5350.550
Learning to Select0.4620.5540.577
SSG0.5590.5880.603

SSG retains the highest average for all three encoders. These results show that its advantage persists after replacing the original short or fixed baseline settings with broader validation-based tuning.

The complete settings are documented in Appendix A.2.

AuthorsOpenReview ↗

Response to Reviewer ZZmE

Q3. Required supervision and sensitivity to query splits

We thank the reviewer for suggesting these analyses. We have added learning curves and experiments across query splits using BGE-S. The results show that SSG can improve retrieval with relatively few training queries on some datasets, but the supervision required varies across datasets.

Training-data sensitivity: Figure 5 evaluates 10%, 20%, 40%, 60%, 80%, and 100% of the eligible training pool and reports the actual query counts. At the smallest evaluated fraction, SSG already improves over Base Retrieval on:

DatasetTraining queries at 10%Result relative to Base Retrieval
NFCorpus259Improvement
ArguAna63Improvement
NQ155Improvement

SciFact benefits progressively from additional training data. However, FiQA, TREC-COVID, and Touché-2020 fall below Base Retrieval at the smallest fraction. Gains are also not uniformly monotonic. These results show that the amount of supervision needed depends on the dataset; they do not establish a universal minimum number of labeled queries.

Query-split robustness: We evaluate split seeds 42, 43, and 44 while fixing the training seed at 42 and using the full eligible training pool. SSG improves over Base Retrieval in all 21 dataset-split comparisons, with each baseline evaluated on the corresponding test split.

The learning curves and split analysis are reported in Appendix A.8, Figure 5 and Table 16.

Together, these analyses show that limited target-domain training data can yield gains on several datasets and that improvements persist across the evaluated query partitions. The learning curves also identify settings where very small training sets are insufficient. We have clarified this limitation in the Discussion and identified further analysis of supervision sensitivity as a direction for future work.

AuthorsOpenReview ↗

Thank You for Your Feedback!

We sincerely hope that these clarifications and additional experimental results have addressed your concerns and strengthened your assessment of our work. We would be grateful if you could kindly reconsider your rating in light of the new evidence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.