acceptodds
Under review as a conference paper at ICLR 2027

RADAR: Radial Attribute Distillation for Augmented Representations

Abstract

Dense embeddings primarily encode similarity through the vector direction, while vector magnitude is often normalized away or lacks an explicit semantic role. We introduce RADAR (Radial Attribute Distillation for Augmented Representations), a general framework for augmenting pretrained embeddings with latent attributes without disrupting their underlying semantic geometry. RADAR learns from multiple heterogeneous and noisy proxy signals to identify a latent attribute that is both supported by the proxies and predictable from the original representation. It then encodes the predicted attribute through bounded radial scaling: embedding direction—and hence the original semantic representation—is preserved, while vector norm carries complementary information that can be exploited directly by inner-product retrieval. The framework is domain-agnostic and does not require retraining the underlying embedding model or observing proxy signals at inference time. We evaluate RADAR using scientific literature retrieval as a challenging testbed, where citations and venue-related signals provide imperfect proxies for scholarly impact. Across four retrieval benchmarks, augmenting Qwen3-Embedding-8B with RADAR improves average nDCG@10 from 45.16 to 46.02 and Recall@50 from 50.01 to 51.31, while retaining single-stage vector retrieval. The distilled attribute also exhibits stronger associations with held-out peer-review judgments and prospective citation outcomes than direct proxy prediction and conventional factor-based alternatives on most measures. Our results suggest that radial degrees of freedom provide a simple mechanism for incorporating complementary latent attributes into existing embedding spaces while preserving their semantic structure and computational advantages.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
dense retrieval;representation learning;

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_rjM8OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RADAR, a pipeline for augmenting unit normalized embeddings by changing their norm, thus leading to altered ranking in retrieval settings. Using this method, the norm of the embeddings can therefore be adapted to encode an additional property without retraining the original embedding model, thus allowing for altered ranking behavior while retaining the underlying angular similarity structure. The framework thereby allows for using multiple auxiliary proxies for the desired property that shall be encoded by the vector norm and creates a scheme that predicts a robust value from these individual auxiliary predictors (with "robust" meaning a single feature that is reliably predictable from the embedding while staying faithful to the aspect that the proxies encode).

Experiments are conducted in the scientific domain, so that the norm is in practice adapted to encode the importance of a paper, as indicated by three auxiliary predictors: citations received one, two and three years after publication, and the standing of the venue the paper is published at. The underlying assumption is that a paper with high citation count or a paper that was published at a better venue should rank higher in a retrieval setting since it likely contains a fundamental contribution instead of a minor incremental change.

The individual predictors are used to create a more robust feature vector for each training example, by not encoding values directly, but instead relying on ranks across comparable papers (i.e., published in the same year) and additional normalization. Using these feature vectors created for training samples, the authors then train an additional model that predicts these feature vectors directly from an embedding created by an existing embedding model, which therefore makes this method applicable to any embedding at inference time (without needing actual citation counts or venue information).

Multiple features are then mapped onto a single scalar via a procedure that optimizes for maintaining most of the available variance of the three-dimensional feature vectors while focusing on a subspace that can be reliably predicted the given embedding by the neural network predictor. An extensive formal analysis demonstrates that this specific setup is beneficial, since 1) it bounds the percentage of explained variance of the trained predictor to be close to that of the optimal predictor 2) it reliably selects a direction close to the first pricipal component if the others do not encode sufficient variance, and 3) since the hyperparameter gamma (which controls the strength of the norm change) can be used to bound the maximal change in ranking that can occur.

The proposed method is then evaluated on multiple benchmarks, applied to a Qwen3-Embedding-8B model. Multiple existing models are included as baselines, including other strategies that also just adapt the norm of the predicted embeddings. It is shown that the proposed method outperforms all other methods on the benchmarks that are evaluated. Additionally, ablations are performed for several parts of the pipeline and an unrelated benchmark task from the SciRepEval benchmark is also evaluated.

To already preempt the strengths and weaknesses: This paper seems to generally propose a very sensible method that has good theoretical justification and demonstrates improved performance on the specific benchmarks that were evaluated. However, the evaluation fails to demonstrate the the method has any real benefit for other task types that are not as closely related to predicting highly cited papers, and does therefore not prove that the method is useful for any common real-world task. Ultimately, I therefore feel like this paper is precisely on the edge between a weak accept and a weak reject, with the final decision being the positive one due to the sound derivation and justification of the method.

Critical strengths
  • The proposed method seems like a reasonable strategy to use incorporate these auxiliary signals into alterations of the embedding norm. One specific benefit thereby is especially the robustness of the feature preprocessing (i.e., using ranks instead of values and normalizing at different parts of the pipeline).
  • A major contribution is also the optimization of the vector v that is used as weight on the individual auxiliary features to combine them into a scalar. The proposed scheme optimizes it to focus on the subspace of the feature space (i.e., of the multi-dimensional feature vector) that is both well predictable and encodes much of the variance encoded in the features. this makes an inherent trade-off visible and adresses it via a sensible optimization strategy.
  • The theoretical analyses provide good support for the underlying strategy, proving that the specific optimization that optimizes for retaining a lot of feature variance while also encouraging good predictability has likely positive effects on the quality of the representation (exact effects can not be established though due to requiring knowledge of the optimal predictor of feature vector Y given embedding U). The additional analysis on the preservation of ranking for specific choices of the hyperparameter gamma then underscores the main argument of just reranking while keeping most of the structure intact.
  • The evaluations include a good set of baselines, including comparable methods that also just alter the embedding norm, and evaluating the proposed scheme across multiple embedding backbones. This therefore demonstrates that the proposed method leads to small gains on the four specific benchmark tasks that were evaluated. The ablations then also show very well that several parts of the pipeline are crucial in achieving good performance.
Critical weaknesses
  • The main weakness of this work is the limited scope of the evaluation. The underlying method is presented as a way of injecting useful information into existing embeddings, which is supposed to lead to generally improved embeddings. However, the evaluation does only establish that injecting the specific information does improve performance on exactly that type of task that benefits specifically from the injected information, thus not proving generally improved performance. The reason for this is that the paper evaluates just four main benchmark tasks and that all of them are strictly related to either predicting whether one paper should cite another, or to predict google scholar rankings, which likely ranks highly cited papers higher than others. This is a biased evaluation setup since the proposed method gives higher weight to generally highly cited papers, which of course are also more likely to be a valid target in all of these benchmarks (since highly cited papers are simply more likely to either be real citations or recommended citations, which is what these benchmarks rely on for construction). This becomes even more limiting considering that these specific tasks are usually not tasks that an embedding model is used for in practice. Usually, an embedding model is used to find highly relevant results for search queries (which are not (!) supposed to be just the highly cited foundational works but actually the most relevant results for the specific query), for stuff like RAG (which also needs to most precise matches and not just the most popular ones), or for semantic visualization, which generally does not benefit from few papers that have generally increased similarity to all of their neighbors due to a larger embedding norm. The existing evaluation therefore does not establish that the resulting model is useful in any real downstream application.
  • This becomes even more suspect considering that exactly one task of the SciRepEval benchmark is used for a further small-scale evaluation, which is limited to just those methods that alter the embedding norm of the Qwen model and not the other standard baselines. Why not include the other baselines or the unlatered Qwen model in this evaluation? As it is now, we do not know how any of these approaches performs compared to normal scientific embedding models on this task. More interestingly, the SciRepEval benchmark comprises 24 tasks in total, so why just select one of them and not just include a complete evaluation of all baselines on all tasks of the benchmark? Note that I am aware that this benchmark has several issues. However, including the results from this benchmark can still give valuable insights into the properties of the different methods, and omitting them simply leads to a state in which this paper fails to demonstrate broader usefulness of the proposed method.
  • The ablations show that each part of the pipeline is crucial for achieving the performance that outperforms the baselines. However, most of the baselines that perform a similar strategy of altering the embedding norm of the frozen embedding backbone seem to operate just on one of the proxy signals, and it is not clear if any of them had access to all three that were used for RADAR. Since dropping any of them makes RADAR not outperform the other baselines any more, this questions if the general training scheme is really responsible for the improved performance, or just the combination of multiple proxies for paper importance.
Questions for authors

Do any of the baselines use all three proxies? Why did you not evaluate on more general benchmark tasks?

3 comments
AuthorsOpenReview ↗

Response to Reviewer rjM8 (1/2)

We greatly appreciate the reviewer's detailed, specific, and constructive comments. We address each of the reviewer's comments and questions below.

W1: We agree that citation-based benchmarks may partly favor influential papers. However, we would like to clarify three points. (1) Scholarly importance is part of what we aim to capture. For scientific literature retrieval, users often want not only relevant papers but also important or foundational ones. Our goal is to improve retrieval by combining both types of information. (2) Our benchmarks do not directly use our training proxies as relevance labels. We intentionally avoided this when designing the experiments. SuReIR measures a paper's importance within a specific survey, rather than its global citation count or venue ranking. GSRR uses Google Scholar results, which may reflect prominence, as the reviewer correctly points out. (3) Our improvements are not limited to these two new benchmarks. RADAR also improves on MasterSet and RMC, two independently developed benchmarks (+0.52 and +1.16 nDCG@10, respectively). Interestingly, GSRR shows the smallest improvement (+0.19), suggesting that the gains are not simply driven by Google Scholar's preference for prominent papers.

On SuReIR, all impact-aware baselines, including CitationPred, perform below the backbone (−0.05 to −0.61 nDCG@10), whereas RADAR improves it by 1.58 points. This suggests that simply favoring papers with high predicted citations is not sufficient to reproduce RADAR's gains on SuReIR.

In addition, the influence of the learned attribute is bounded. For documents with positive cosine similarity, RADAR can reverse the order of two documents only when the ratio of their semantic scores is below eγe^{\gamma}, which is about 1.05 at γ=0.05\gamma=0.05 (Proposition 4). Therefore, a highly cited paper is not ranked above one with a clearly higher semantic score.

We also agree that it is important to show benefits in downstream applications. To evaluate the generalization of RADAR beyond scientific literature, we have conducted additional cross-domain experiments on product search (WANDS) and duplicate-question retrieval (CQADupStack) in Appendix E.6. WANDS uses human relevance annotations, and CQADupStack uses duplicate questions marked by Stack Exchange users. RADAR learns a separate attribute in each domain from domain-specific proxies, while the backbone remains frozen. The main results (%) are shown below, and results for all baselines are reported in Table 13 of Appendix E.6.

MethodWANDS nDCG@10WANDS Recall@50CQADupStack nDCG@10CQADupStack Recall@50
Qwen3-Embedding-8B74.8226.4348.6073.60
Qwen3-Reranker-8B75.6927.3550.6375.66
Proxy-Based Methods (Qwen3-Embedding-8B)
+MultiProxyPred75.77$_{\pm0.052}$27.45$_{\pm0.020}$50.69$_{\pm0.059}$75.65$_{\pm0.035}$
+FactorDistill75.6327.4049.8674.70
RADAR (Ours)76.73±0.081_{\pm0.081}27.93±0.017_{\pm0.017}51.18±0.029_{\pm0.029}76.46±0.053_{\pm0.053}

RADAR achieves the best results on both datasets. Compared with the backbone, it improves nDCG@10 and Recall@50 by 1.91 and 1.50 points on WANDS, and by 2.58 and 2.86 points on CQADupStack. All four gains are significant after Holm correction (Table 14 of Appendix E.6).

Regarding visualization, RADAR only rescales each embedding by a factor between 1 and eγe^{\gamma} and does not change its direction. The original embedding can be recovered by normalization, so visualizations based on the original embeddings or on cosine similarity are not affected.

AuthorsOpenReview ↗

Response to Reviewer rjM8 (2/2)

W2: We thank the reviewer for this constructive comment. We would like to clarify that the Peer Review Score task in Section 5.4 is used to validate the learned latent impact score, not to evaluate retrieval. It measures how well the predicted impact score agrees with peer-review scores (Table 3), so only methods that predict such a score can be compared.

We agree that a broader SciRepEval evaluation is valuable. Following the reviewer's suggestion, we have added five subtasks of the SciRepEval benchmark, covering ad-hoc search (TREC-COVID and Search) and proximity (RELISH, CoView, and CoRead), with standard baselines, including Qwen3-Embedding-8B and scientific embedding models (Appendix E.5). We focus on ranking tasks because they directly evaluate retrieval quality, which is how RADAR is used. The main results (nDCG@10, %) are shown below, and results for all baselines are reported in Table 10 of Appendix E.5.

MethodTREC-COVIDSearchRELISHCoViewCoReadAvg.
SPECTER272.0479.8467.4987.6789.1679.24
Qwen3-Embedding-8B86.2984.6374.4489.4691.9085.34
Qwen3-Reranker-8B87.0985.6474.9190.1191.6885.89
Impact-Aware Methods (Qwen3-Embedding-8B)
+MultiProxyPred86.82$_{\pm0.20}$86.13$_{\pm0.24}$76.07$_{\pm0.13}$90.22$_{\pm0.16}$92.18$_{\pm0.10}$86.28
+RADAR (Ours)86.16$_{\pm0.18}$86.40±0.17_{\pm0.17}76.91±0.08_{\pm0.08}90.25±0.15_{\pm0.15}92.53±0.07_{\pm0.07}86.45

RADAR achieves the highest average nDCG@10, and significantly improves over the backbone on four of the five tasks after Holm correction, including Search (+1.77) (Table 11 of Appendix E.5). On TREC-COVID the difference from the backbone is within noise (−0.13, not significant), consistent with the bounded ranking perturbation in Proposition 4.

W3: We thank the reviewer for this important comment. In fact, two baselines use all proxies. CitationPred and VenuePred each use one proxy family, whereas MultiProxyPred and FactorDistill use the same citation and venue proxies as RADAR. MultiProxyPred averages the citation-based and venue-based predictors, and FactorDistill uses a classical linear factor model. In addition, two ablations in Table 2 (− Text Predictability and − Variance Retention) keep all proxies and only change the PALIF objective. These methods therefore use the same information as RADAR. Their macro-averaged results (%) on the four benchmarks are:

Method (all proxies)nDCG@10Recall@50
Impact-Aware Methods (Qwen3-Embedding-8B)
+MultiProxyPred45.42$_{\pm0.15}$50.41$_{\pm0.11}$
+FactorDistill45.2750.38
RADAR − Text Predictability45.1350.35
RADAR − Variance Retention44.9050.12
RADAR46.02±0.08_{\pm0.08}51.31±0.07_{\pm0.07}

With the same proxies, all these methods fall below RADAR. RADAR's gains over the strongest impact-aware baseline for each metric (+0.59 nDCG@10 over CitationPred and +0.90 Recall@50 over MultiProxyPred) are also significant after Holm correction (Table 6 of Appendix E.1). Removing a proxy family (− Citation Proxies or − Venue Proxies in Table 2) reduces the information available to RADAR, so a performance drop is expected. When the information is the same, the results suggest that using multiple proxies alone is not sufficient, and that the way PALIF combines the proxies is important for the gains.

Q1: Yes. MultiProxyPred and FactorDistill use the same citation and venue proxies as RADAR. The two ablations in Table 2 (− Text Predictability and − Variance Retention) also use all proxies.

Q2: Most general retrieval benchmarks provide only relevance labels. They do not indicate which relevant documents are also important, so they cannot directly evaluate the goal of RADAR, which is to rank important documents higher among relevant ones. This is why we constructed SuReIR. For example, in SuReIR, a paper cited in a survey is treated as relevant, and it receives a higher grade if the survey describes it as foundational (e.g., "seminal" or "first proposed") or cites it repeatedly. We believe such labels better reflect papers that are both relevant and important. To further examine general retrieval, we have also evaluated RADAR on five SciRepEval ranking tasks and on two cross-domain datasets (WANDS and CQADupStack), where RADAR achieves the best average results (Appendices E.5 and E.6).

We hope these responses address the reviewer's concerns, and we are happy to discuss any remaining questions.

Thank you for the extensive response.

Regarding W1, if your goal is to uprank papers with high influence, then this can be simply done by scaling the norm by the number of citations (or some transform of it, like the log or something), and does not require such a complex scheme, especially since 1) papers published at major venues can have little influence, and arxiv papers are sometimes some of the most influential ones and 2) citation counts are easily accessible for pretty much all papers, thus replacing the need for predicting them with a neural network.

For this exact reason, I personally feel like the main contribution of your work is not simply the application to scientific retrieval (since otherwise the contribution would not be sufficient), but it instead is the general scheme of using different factors (that might not be present at inference time) and combining them in a robust way. The evaluation on the scientific domain does prove that this method then has one specific benefit, which is rather limited for proving that the general method makes sense. As a result, I feel like weakness 1 still stands.

Regarding point 1.2, I did not mean to imply that you encode the exact signal that is evaluated. But you encode a signal that certainly is highly correlated with what the tasks demand, and this weakens the proof that this method is generally of benefit. You say that other methods like CitationPred do not show the same behavior. While this might be true, analyzing this myself as a reviewer is certainly out of the scope of what is required, and if this point is a major argument that disproves my high-level concern, then it might be a good choice to include a deeper analysis of all this in the paper, so that a reviewer does not need to come up with it themselves.

For the two additional tasks that you evaluate, it is not clear to me how they fit the general framework. For duplicate question search, for example, why would it be an inherent benefit to encode the rating of the question? If two questions are duplicates of each other, then what does the rating of them have to do with that?

Regarding W2, I honestly feel like selectively evaluating on a subset of benchmark tasks is simply a suspicious choice. I explicitly do not want to imply that you do this with bad intent since you likely made that decision in good faith, but as a reader you simply can not be sure if the tasks were simply selected due to your model performing well on them.

If you think that just a specific set of tasks is relevant for your model, then it would be best to nevertheless evaluate it on all tasks, and then discuss the benefits of your model on the important subset to prove your claims.

So overall I support the addition, but it should use all tasks from the benchmark.

Ultimately, the response and the reported additions slightly improve this work, but since I feel like my original assessment of this work was rather generous originally, I will leave my score as it is.

Reviewer_eWwQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper proposes RADAR, a post-hoc method that adds a query-independent document attribute (here, scholarly impact) to frozen dense embeddings. The attribute is distilled from three noisy proxies (two citation windows and venue rank) by PALIF, which picks the proxy direction most predictable from the embedding while retaining most of the leading proxy variance. It is then encoded by rescaling each document vector, which preserves cosine geometry and single-vector MIPS. The authors back both steps with theoretical guarantees. Experiments cover four scientific-literature retrieval benchmarks (two new) and four backbones, plus external validation against peer-review scores and prospective citations. They report +0.86 nDCG@10 and +1.30 Recall@50 on Qwen3-Embedding-8B.

Critical strengths

S1 The overall design is practical and transparent, with attribute inference and scaling running fully offline and the method works without retraining a frozen encoder.

S2 External validity analysis goes beyond what is typically expected of this type of paper.

Critical weaknesses

W1 RADAR builds on existing retrieval backbones, so experiments should show how its gains compare against comparable methods, beyond the reranker comparison (the rerankers score relevance only, without impact information, and perform at or below the backbone on average, suggesting they do not fit these benchmarks well). Baselines that use the same impact score through a different mechanism (e.g., naive re-ordering of the top-k, an additive prior via an extra MIPS coordinate, or reciprocal rank fusion), or that inject the same criterion without a learned prior (e.g., instruction-prompted retrieval or reranking), would make the mechanism defensible, beyond the trivial conclusion that task-specific information improves performance.

W2 The paper presents RADAR as domain-agnostic, though only validating it on a single-domain (computer-science papers) with a single proxy family (citations and venue, M=3). Either the evidence needs to cover at least 2 further domains with different proxies, or the claims should be scoped to scholarly retrieval, in which case the impact seems limited.

The paper is technically sound and the idea is appealing, but the current evidence does not support its claims of a defensible mechanism or of general applicability. Closing this gap would require new baselines and experiments in further domains, which I do not expect to be feasible to run and verify within the rebuttal period. My current rating is therefore 2.

Questions for authors

To avoid redundant responses, only questions unrelated to the weaknesses above are raised.

Q1 What happens with the same radial prior when a random rank score is used? This could bound how much of the gain comes from the distilled attribute versus the reweighting mechanism, which could help guide future work which additions to draw inspiration from.

Q2 Different methods like computer vision (which you mention as future work) have already scaled embeddings to encode external information successfully. Did you draw inspiration from any fields other than NLP, and how does your positioning change when taking these into account?

3 comments
AuthorsOpenReview ↗

Response to Reviewer eWwQ (1/2)

We sincerely thank the reviewer for the thoughtful and constructive review. The review raised two main concerns: comparisons with alternative baselines and evaluation beyond a single domain. The revised paper addresses both. Appendix E.2 reports the requested comparisons with alternative baselines, and Appendix E.6 evaluates RADAR in two additional domains with different proxies (product search and duplicate-question retrieval). We address each of the reviewer's comments and questions below.

W1: We thank the reviewer for this constructive comment. As suggested, we have added these baselines to Appendix E.2 of the revised paper. All of them use the same frozen Qwen3-Embedding-8B backbone. The first group uses the same PALIF scores in different ways: impact-only re-ordering of the top-100 candidates, an additive prior implemented as an extra MIPS coordinate, and reciprocal rank fusion. The second group expresses the same preference through instructions, without a learned prior: instructed embedding and instructed reranking with Qwen3-Reranker-8B. The table below reports the average over the four benchmarks (full results and details in Appendix E.2):

MethodAvg. nDCG@10Avg. Recall@50
Qwen3-Embedding-8B45.1650.01
Impact-only re-ordering (top-100)38.4941.16
Additive prior (extra MIPS coordinate)45.5050.45
Reciprocal rank fusion34.2439.05
Instructed embedding45.4050.60
Instructed reranker45.5350.57
RADAR46.0251.31

The reciprocal rank fusion configuration fuses the semantic ranking with a query-independent impact ranking over the full corpus with equal weights, which allows the impact ranking to dominate. With the same PALIF scores, re-ordering and the additive prior are worse than RADAR on all eight benchmark–metric pairs, and RADAR also outperforms the additive prior on both metrics in our cross-domain experiments (Appendix E.6). This suggests that the gains come not only from the learned scores but also from the radial form. The two instruction-based methods, whose prompts favor influential and highly cited papers, stay below RADAR on average, so these prompts do not reproduce the gains of a learned document-level prior.

W2: We are grateful to the reviewer for raising this important point. To validate the effectiveness of RADAR across domains, we have evaluated it in two further domains with different proxies: product search on WANDS, using rating count and average rating, and duplicate-question retrieval on CQADupStack (12 Stack Exchange forums), using question votes, view counts, and asker reputation. In both datasets, the relevance judgments are separate from the feedback signals used as proxies. We keep the same frozen Qwen3-Embedding-8B backbone, PALIF objective, and radial rule, and learn a separate attribute from each domain's own proxies. Each document's attribute is predicted without using that document's own proxy values (five-fold cross-fitting). Hyperparameters are selected on held-out validation queries in each domain, which gives the same values as in the main experiments. The table below shows the main comparisons (results for all baselines and experimental details are in Appendix E.6).

MethodWANDS nDCG@10WANDS Recall@50CQADupStack nDCG@10CQADupStack Recall@50
NV-Embed-v275.3226.6750.2575.54
Qwen3-Embedding-8B74.8226.4348.6073.60
Qwen3-Reranker-8B75.6927.3550.6375.66
Proxy-Based Methods (Qwen3-Embedding-8B)
+MultiProxyPred75.77$_{\pm0.052}$27.45$_{\pm0.020}$50.69$_{\pm0.059}$75.65$_{\pm0.035}$
Impact-Integration Controls (Qwen3-Embedding-8B)
+Additive Prior76.10$_{\pm0.037}$27.52$_{\pm0.016}$50.82$_{\pm0.051}$75.84$_{\pm0.088}$
+Random Prior74.76$_{\pm0.012}$26.38$_{\pm0.025}$48.45$_{\pm0.038}$73.52$_{\pm0.040}$
RADAR (Ours)76.73±0.081_{\pm0.081}27.93±0.017_{\pm0.017}51.18±0.029_{\pm0.029}76.46±0.053_{\pm0.053}

RADAR performs best on both datasets and both metrics. Its gains over the backbone (+1.91 and +1.50 on WANDS; +2.58 and +2.86 on CQADupStack) are all significant after Holm correction. These results show that RADAR can also be applied beyond scholarly retrieval when suitable domain proxies are available.

AuthorsOpenReview ↗

Response to Reviewer eWwQ (2/2)

Q1: We thank the reviewer for this useful suggestion. We agree that a random score is a helpful negative control.

We have added this control to Appendix E.2 for your reference. This control applies the same radial encoding, but randomly permutes the normalized PALIF scores Î_d across documents. Since Î_d is a corpus-level rank, this is equivalent to assigning each document a random rank score, while keeping the score distribution unchanged. The random prior does not improve over the backbone on any of the eight benchmark–metric pairs:

MethodSuReIR nDCG@10SuReIR Recall@50GSRR nDCG@10GSRR Recall@50RMC nDCG@10RMC Recall@50MasterSet nDCG@10MasterSet Recall@50
Qwen3-Embedding-8B58.4134.4184.9283.2018.3847.0418.9335.37
+Random Prior58.23$_{\pm0.45}$34.25$_{\pm0.24}$84.29$_{\pm0.18}$83.20$_{\pm0.12}$18.25$_{\pm0.28}$46.91$_{\pm0.37}$18.59$_{\pm0.07}$35.23$_{\pm0.17}$
RADAR (Ours)59.99±0.17_{\pm0.17}36.54±0.14_{\pm0.14}85.11±0.05_{\pm0.05}83.37±0.09_{\pm0.09}19.54±0.19_{\pm0.19}48.27±0.15_{\pm0.15}19.45±0.23_{\pm0.23}37.04±0.18_{\pm0.18}

Therefore, radial scaling with randomly assigned impact scores does not reproduce RADAR's gains, suggesting that the learned document-specific impact scores also matter.

Importantly, our existing baselines (CitationPred, VenuePred, MultiProxyPred, and FactorDistill) already use the same radial scaling rule and the same embedding backbone as RADAR. Therefore, their performance differences help isolate the benefit of learning a better attribute, rather than simply changing the ranking scores.

Q2: We thank the reviewer for pointing out this important connection. We agree that MagFace and AdaFace are relevant. MagFace trains an encoder so that the feature norm reflects face image quality, while AdaFace uses the feature norm as a proxy for image quality to adapt its training margin.

However, RADAR addresses a different problem. Our work was mainly motivated by query-independent document priors in information retrieval. RADAR starts from pretrained and frozen embeddings. It learns a latent document attribute from multiple noisy proxies, selecting a factor that retains the main proxy variance while remaining predictable from the frozen representation. The learned attribute is then encoded in the embedding norm without retraining the encoder or changing its semantic directions.

We believe this distinction is important for representation learning: RADAR provides a general way to create richer representations from existing pretrained models.

We agree that these two CV studies are related works. We have included them and clarified that our novelty lies in learning and adding new attributes to frozen representations, not in the use of vector magnitude itself.

We hope these responses address the reviewer's concerns, and we are happy to discuss any remaining questions.

I thank the authors for the substantial effort they invested in the rebuttal and for running the requested experiments, which meaningfully strengthen the paper, and I have raised my soundness score accordingly. I maintain my overall rating for two reasons, (i) some of the new baselines appear to be implemented in weak forms, e.g., equal-weight RRF over the full corpus and impact-only re-ordering both let the impact signal dominate, so they do not seem to test the alternatives fairly; (ii) RADAR’s margin over the strongest alternatives (the additive prior and the instructed reranker) is about 0.5 nDCG@10 on average, and no significance is reported against them, so the specific benefit of the radial mechanism remains modest. Nonetheless, the discussion has increased my confidence in the paper, and I will reflect this verbally in my final summary.

The authors also primarily pointed to the appendices for their new evidence - I would advise the authors to find ways to integrate those into the main text in a future revision.

Reviewer_U8zyOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces RADAR, a framework that augments frozen document embeddings with a predicted latent attribute through radial scaling. Its core component, PALIF, selects a proxy-space direction that balances predictability from embeddings against retention of dominant proxy variance. The predicted attribute determines document-vector magnitude while preserving embedding directions and cosine similarities. Experiments on scientific literature retrieval show consistent improvements across four embedding backbones, with additional validation against peer-review scores and future citation outcomes.

Critical strengths

1.Simple and practical integration. RADAR augments existing embeddings without retraining the encoder or requiring document proxies at inference time. Attribute prediction and scaling occur offline, preserving compatibility with single-vector maximum-inner-product retrieval and low online latency.

2.Explicit and interpretable factor-selection objective. PALIF addresses a meaningful tension between proxy fidelity and predictability from frozen representations. Cross-fitted prediction errors reduce reliance on optimistic in-sample estimates, while the variance-retention constraint provides a transparent way to control the selected attribute.

3.Consistent results with complementary validation. Improvements extend across four retrieval benchmarks and four embedding backbones. Peer-review judgments, prospective citation outcomes, and leave-one-proxy-family-out experiments provide useful evidence beyond retrieval performance alone.

Critical weaknesses

1.The novelty of radial encoding is limited. Equation (5) multiplies cosine similarity by a query-independent document prior; for positive similarities, this is equivalent to adding the attribute to log-similarity, as Section 4.3 acknowledges. Direction preservation and the ranking-perturbation bounds follow directly from bounded scalar rescaling. The substantive novelty therefore appears concentrated in PALIF.

2.The new benchmarks may favor the attribute being introduced. SuReIR assigns higher relevance grades using foundational-status cues and citation frequency within surveys, while GSRR adopts Google Scholar results as positive judgments. These choices may reward scholarly prominence alongside query-specific relevance, potentially favoring a citation- and venue-derived attribute. Questions for Authors

3.The statistical reliability of the retrieval gains is unclear. Average nDCG@10 improves by 0.86 points over the underlying Qwen3 retriever and by 0.59 points over the strongest impact-aware alternative. However, the main retrieval tables report neither confidence intervals nor significance tests. Replacing exponential scaling with “direct impact scaling” produces a dramatic performance drop, but the alternative is insufficiently specified.

Questions for authors

See the weakness.

4 comments
AuthorsOpenReview ↗

Response to Reviewer U8zy (1/2)

We thank the reviewer for the careful reading and constructive comments. We address each of the reviewer's comments below.

W1: We thank the reviewer for this comment. We agree that radial scaling is mathematically simple. However, we see this simplicity as a strength, not a weakness.

Our contribution has two parts. First, PALIF learns an unobservable document attribute (e.g., scholarly impact) from multiple noisy signals. Second, RADAR creates a new, richer representation by combining two types of information in a single embedding: semantic meaning in its direction and the learned attribute in its magnitude.

We believe this is an important contribution to representation learning. Rather than training a new embedding model, RADAR augments existing representations while preserving their original semantic directions and cosine similarities. This simple design is intentional. RADAR can complement many existing embedding models and support fast, single-stage retrieval without costly reranking. Our experiments show that RADAR achieves better retrieval accuracy than Qwen3-Reranker-8B while being 19–290 times faster.

Moreover, the additional experiments in the revised paper show that the choice of integration mechanism also matters (Appendix E.2, Table 7). With the same PALIF scores, an additive prior on the raw cosine score and top-100 re-ordering are worse than RADAR on all eight benchmark–metric pairs, and reciprocal rank fusion and direct scaling are even worse than the backbone.

The novelty is therefore not the scaling equation alone, but a simple and practical way to create richer representations from existing embeddings. We will make this contribution clearer in the revised paper.

W2: We thank the reviewer for this concern. We agree that citation-based benchmarks may partly favor influential papers. However, we would like to clarify three points. (1) Scholarly importance is part of what we aim to capture. For scientific literature retrieval, users often want not only relevant papers but also important or foundational ones. Our goal is to improve retrieval by combining both types of information. (2) Our benchmarks do not directly use our training proxies as relevance labels. This is one thing that we intentionally avoided during the experimental design. SuReIR measures a paper's importance within a specific survey, rather than its global citation count or venue ranking. GSRR uses Google Scholar results, which may reflect prominence, as the reviewer correctly points out. (3) Our improvements are not limited to these two new benchmarks. RADAR also improves on MasterSet and RMC, two independently developed benchmarks (+0.52 and +1.16 nDCG@10, respectively). Interestingly, GSRR shows the smallest improvement (+0.19), suggesting that the gains are not simply driven by Google Scholar's preference for prominent papers.

In addition, following the reviewer's comment, we re-evaluated SuReIR with binary relevance in the revised paper, so that grade-2 papers no longer receive a higher weight (Appendix E.3). RADAR still improves nDCG@10 over the backbone by 1.31 points (1.58 with graded relevance):

SuReIR nDCG@10GradedΔBinarizedΔ
Qwen3-Embedding-8B58.41–69.03–
Impact-Aware Methods (Qwen3-Embedding-8B)
+CitationPred57.98$_{\pm0.31}$−0.4368.53$_{\pm0.12}$−0.50
+VenuePred58.36$_{\pm0.22}$−0.0568.99$_{\pm0.14}$−0.04
+MultiProxyPred58.11$_{\pm0.17}$−0.3068.69$_{\pm0.09}$−0.34
+FactorDistill57.80−0.6168.36−0.67
RADAR (Ours)59.99±0.17_{\pm0.17}+1.5870.34±0.10_{\pm0.10}+1.31

CitationPred and the other impact-aware baselines remain below the backbone under both relevance settings. This suggests that simply favoring papers with high predicted citations is not sufficient to reproduce RADAR's gains on SuReIR.

We further evaluated RADAR on SciRepEval's ad-hoc Search task, where relevance is not defined by citation counts. RADAR improves nDCG@10 over the backbone by 1.77 points (Appendix E.5), providing additional evidence that our gains are not limited to citation-based relevance judgments.

We will clarify that our main contribution is a simple, efficient way to create richer embeddings for tasks where both semantic relevance and document importance matter.

AuthorsOpenReview ↗

Response to Reviewer U8zy (2/2)

W3: We thank the reviewer for this important comment. We have added significance tests in Appendix E.1 of the revised paper.

(1) Statistical reliability: Results of stochastic methods in Table 1 are averaged over five random seeds (standard deviations in Table 5). Using paired bootstrap confidence intervals and Holm-corrected randomization tests (Table 6), we find that RADAR's average gains are significant over both the backbone and the strongest impact-aware baseline:

Macro-average gain [95% CI]nDCG@10Recall@50
RADAR vs. Qwen3-Embedding-8B+0.86 [+0.62, +1.10]+1.30 [+1.03, +1.56]
RADAR vs. strongest impact-aware baseline+0.59 [+0.40, +0.79]+0.90 [+0.67, +1.13]

The strongest baseline is CitationPred for nDCG@10 and MultiProxyPred for Recall@50. CIs are unadjusted; all four gains are significant after Holm correction (p<0.05p<0.05).

(2) Direct impact scaling: In direct impact scaling, each document embedding is multiplied by I^d\widehat{I}_d instead of exp⁡(γI^d)\exp(\gamma \widehat{I}_d). We have added its definition to Appendix E.2. Since I^d\widehat{I}_d can be close to zero, a relevant document with low predicted impact can lose most of its semantic score. This is consistent with the sharp drop in Table 2. In RADAR, the factor stays between 1 and eγe^{\gamma} (about 1.05 at γ=0.05\gamma=0.05), so for documents with positive cosine similarity, two documents can swap order only if the ratio of their semantic scores is below eγe^{\gamma} (Proposition 4).

We hope these responses address the reviewer's concerns, and we are happy to discuss any remaining questions.

Thank you for the detailed response. The expanded evaluations help address my concerns. However, I remain insufficiently convinced by the motivation for framing RADAR as a richer representation-learning framework: radial scaling still amounts to incorporating a query-independent document prior into the retrieval score, with the main methodological novelty concentrated in PALIF. While the new results support the practical utility of this design, they do not fully resolve my concerns about its conceptual novelty or establish a consistent advantage over strong impact-aware alternatives across tasks. Thus, I maintain my original rating.

AuthorsOpenReview ↗

Response to Reviewer U8zy's Follow-up

We sincerely thank the reviewer for taking the time to read our response. We also appreciate the reviewer's acknowledgment that the expanded evaluations help address the concerns and support the practical utility of our design. We would like to further clarify the two points raised in the follow-up: RADAR's conceptual novelty and its advantage over strong impact-aware alternatives across tasks.

Conceptual novelty: We agree that, for positive cosine scores, ranking by RADAR is equivalent to ranking by the log cosine score plus a query-independent document prior (Section 4.3). However, our contribution is not this formula itself, but where we place the prior. Dense retrieval usually normalizes away the vector norm. RADAR instead stores a learned document attribute in the norm of a frozen embedding, while the direction keeps the original semantics. The attribute thus becomes part of the representation, and the augmented embedding can be indexed and searched like any other embedding, without changing the query embedding or adding a reranking stage. This is our motivation for viewing RADAR at the representation level.

PALIF is designed for this setting. New documents have no proxy data, so the attribute must be predicted from the frozen embedding. PALIF learns, from noisy proxies, an attribute that can be predicted in this way. PALIF and radial encoding are therefore designed to work together.

The way the prior is integrated also matters. With radial scaling, the attribute can only re-order documents with similar cosine scores (Proposition 4), so semantic relevance remains the main ranking signal. With the same PALIF scores, an additive prior on the raw cosine score performs worse than RADAR on all eight benchmark–metric pairs in Table 7 and on all four measures in Table 14. We therefore view RADAR as a way to augment existing embeddings, rather than only a document prior added to the retrieval score.

Consistent advantage: We respectfully believe that the revised paper provides evidence of a consistent advantage. We summarize the relevant results below. All listed methods use the frozen Qwen3-Embedding-8B backbone, and each row compares RADAR with the strongest alternative in the corresponding table.

EvaluationMetricStrongest alternativeRADARΔ
Four main benchmarks, average (Tables 1 and 5)nDCG@10+CitationPred 45.43$_{\pm0.18}$46.02±0.08_{\pm0.08}+0.59
Four main benchmarks, average (Tables 1 and 5)Recall@50+MultiProxyPred 50.41$_{\pm0.11}$51.31±0.07_{\pm0.07}+0.90
SciRepEval, five-task average (Table 10)nDCG@10+MultiProxyPred 86.2886.45+0.17
WANDS (Table 13)nDCG@10+Additive Prior 76.10$_{\pm0.037}$76.73±0.081_{\pm0.081}+0.63
WANDS (Table 13)Recall@50+Additive Prior 27.52$_{\pm0.016}$27.93±0.017_{\pm0.017}+0.41
CQADupStack (Table 13)nDCG@10+Additive Prior 50.82$_{\pm0.051}$51.18±0.029_{\pm0.029}+0.36
CQADupStack (Table 13)Recall@50+Additive Prior 75.84$_{\pm0.088}$76.46±0.053_{\pm0.053}+0.62

Across the four main benchmarks, the five SciRepEval tasks, and the two new domains, RADAR ranks first among all compared methods on 15 of the 17 dataset–metric settings. On the four main benchmarks, the macro-average gains over the strongest impact-aware baseline are significant on both metrics (Table 6). As discussed above, RADAR also outperforms the raw-cosine additive prior, with three of the four differences in Table 14 significant after Holm correction. Finally, RADAR improves all three additional backbones on every benchmark and metric (Table 9). Taken together, these results show that RADAR's advantage is consistent across diverse retrieval tasks and is not confined to a particular benchmark or evaluation setting.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank all reviewers for their careful reading and constructive comments. We have uploaded a revised version of the paper. The main additions are:

  1. Statistical significance (Appendix E.1). We report standard deviations over five seeds, paired bootstrap confidence intervals, and significance tests with Holm correction for the main results.
  2. Other ways of using the impact scores (Appendix E.2). We compare RADAR with an additive prior, top-100 re-ordering, reciprocal rank fusion, and direct impact scaling, all using the same PALIF scores. We also compare with instruction-based retrieval and reranking, and with a random-prior control. This appendix also defines direct impact scaling.
  3. Binary relevance on SuReIR (Appendix E.3). RADAR still improves over the backbone when all relevant papers receive the same weight.
  4. Five ranking tasks from SciRepEval (Appendix E.5). These include two ad-hoc search tasks and three proximity tasks, evaluated with the standard baselines.
  5. Two cross-domain evaluations (Appendix E.6). We evaluate product search on WANDS (e-commerce) and duplicate-question retrieval on CQADupStack (community QA), each with its own proxies.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.