acceptodds
Under review as a conference paper at ICLR 2027

Efficient Entity Disambiguation with Adaptive Candidate Pruning

Abstract

Entity disambiguation (ED) systems often use a two-stage pipeline: a candidate generator produces a fixed-size candidate set, and a more expensive candidate reranker selects the final prediction. However, because mention ambiguity varies substantially, reranking the same number of candidates for every mention wastes computation on easy cases. To address this issue, we propose a lightweight adaptive candidate pruning method that requires no additional model training. Inspired by conformal prediction, candidates are retained when their generator-score difference from the top candidate is below a threshold, yielding smaller sets for confident mentions and larger sets for ambiguous ones. We tune the threshold on development data for end-to-end accuracy, prioritizing practical utility over a formal coverage guarantee. Experiments on a standard ED benchmark show that our method reduces the average number of candidates passed to the reranker by 94% and achieves a 4.56 speedup, while improving accuracy. Further analyses demonstrate robustness to hyperparameter selection and show that the method places pruning boundaries close to the candidate depth required for each mention.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Natural Language Processing, Entity Disambiguation, Entity Linking, Efficient Method

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_JzzXOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the computational redundancy introduced by fixed-size candidate sets in conventional entity disambiguation (ED) systems and proposes a lightweight adaptive candidate pruning method that requires no additional model training. Inspired by conformal prediction, the proposed method defines a nonconformity score as the difference between each candidate's generator score and that of the top-ranked candidate. A threshold is selected using the development set to adaptively determine the candidate set size for mentions with different levels of ambiguity, thereby reducing the computational cost of cross-encoder reranking. The authors conduct experiments on benchmarks including ZELDA and GERBIL, evaluating the method with different candidate generators, independently trained ED pipelines, and initial candidate set sizes. In the main ZELDA experiments, the method reduces the average number of candidates passed to the reranker from 30 to 1.79, achieves a 4.56× inference speedup, and improves macro-average accuracy from 82.7% to 83.5%.

Critical strengths

S1. The method performs adaptive pruning using existing candidate generator scores without requiring additional model training, substantially reducing cross-encoder inference costs while maintaining or improving prediction accuracy.

S2. The paper compares against fixed-size and adaptive pruning baselines, conducts additional experiments across different encoders, candidate set sizes, and benchmarks, and uses Rescue–Harm analysis to explain the accuracy improvements resulting from pruning.

Critical weaknesses

W1. Figure 3 on page 7 refers to the average candidate set size, whereas the experimental setup on pages 5–6 specifies that mentions retaining only one candidate are counted as requiring zero reranked candidates. The authors should clarify the distinction between the number of retained candidates and the number of candidates actually passed to the reranker.

W2. Although improving inference efficiency is a primary objective, Equation (11) selects the pruning threshold mainly based on development-set accuracy, considering candidate count only when multiple thresholds achieve the same accuracy. Despite the existing threshold sensitivity analysis, further evaluation under explicit computational budget constraints would help establish whether the strategy achieves a favorable accuracy–efficiency trade-off.

Questions for authors

Q1. Does the "Average Candidate Set Size" reported in Figure 3 actually refer to the number of candidates passed to the cross-encoder? Could the authors clarify the distinction between retained candidate set size and the number of candidates requiring reranking, and ensure consistent terminology across the figures and tables?

Q2. Table 13 in Appendix D.1 shows that 74.3% of mentions retain only one candidate after pruning, allowing them to bypass cross-encoder inference. Could the authors further quantify the respective contributions of bypassing reranking entirely and reducing the number of reranked candidates for mentions with multiple retained candidates to the overall inference speedup?

Q3. The proposed method relies on score differences relative to the top-ranked candidate without explicitly accounting for the reliability of that candidate. Could the authors analyze the behavior of the pruning strategy when the top-ranked candidate is incorrect or the candidate generator exhibits low confidence, particularly regarding the resulting pruning errors?

4 comments
AuthorsOpenReview ↗

Official Comment by Authors (1/2)

Dear Reviewer JzzX,

We would like to thank you for carefully reviewing our manuscript and for your valuable comments. We particularly appreciate your recognition of key contributions of our work, including efficiency and accuracy gains brought by our method, and our comprehensive evaluation and analysis. Below, we address the questions and concerns raised in your review.


W1. Figure 3 on page 7 refers to the average candidate set size, whereas the experimental setup on pages 5–6 specifies that mentions retaining only one candidate are counted as requiring zero reranked candidates. The authors should clarify the distinction between the number of retained candidates and the number of candidates actually passed to the reranker.

Thank you for pointing this out. Figure 3 describes the average number of candidates passed to the cross-encoder. We have revised Figure 3 and its caption to clarify this.


W2. Although improving inference efficiency is a primary objective, Equation (11) selects the pruning threshold mainly based on development-set accuracy, considering candidate count only when multiple thresholds achieve the same accuracy. Despite the existing threshold sensitivity analysis, further evaluation under explicit computational budget constraints would help establish whether the strategy achieves a favorable accuracy–efficiency trade-off.

Thank you for this comment. To further investigate the accuracy–efficiency trade-off beyond configurations selected based on the development-set accuracy, we have conducted a post-hoc analysis comparing our method with the baselines, including additional ones described in Appendix D.6 of the revised manuscript, at similar candidate set sizes passed to the reranker.

Using the same trained pipeline and initial candidate sets, we sweep the hyperparameters of our method and the baselines, including k=1,…,30k=1,…,30 for Tuned-Fixed-kk and buffer size B=0,...,29B=0,...,29 for Adaptive-kk. For each setting, we plot accuracy against the micro-average number of candidates passed to the cross-encoder.

Figure 5 of our revised manuscript shows that our method generally achieves higher accuracy than the baselines at comparable numbers of candidates. Top1-Top2 Gate, which is one of the additional baselines, approaches the accuracy of our method that uses the threshold selected on the development set, only by requiring substantially more candidates. These results indicate that our method achieves a superior accuracy–efficiency trade-off across the evaluated settings, compared to the baselines.

We have added the details of these analyses to Appendix D.7 of our revised manuscript.

AuthorsOpenReview ↗

Official Comment by Authors (2/2)

Q1. Does the "Average Candidate Set Size" reported in Figure 3 actually refer to the number of candidates passed to the cross-encoder? Could the authors clarify the distinction between retained candidate set size and the number of candidates requiring reranking, and ensure consistent terminology across the figures and tables?

Thank you for this question. We clarified this point in our response to your W1 above. We kindly ask you to refer to it.


Q2. Table 13 in Appendix D.1 shows that 74.3% of mentions retain only one candidate after pruning, allowing them to bypass cross-encoder inference. Could the authors further quantify the respective contributions of bypassing reranking entirely and reducing the number of reranked candidates for mentions with multiple retained candidates to the overall inference speedup?

Thank you for this question. We have conducted additional analysis to separately examine the efficiency gains of bypassing reranking entirely and reducing the number of reranked candidates for mentions with multiple retained candidates. The results show that the former accounts for 67.5% of the inference-time reduction our method achieves relative to No-Pruning, while the latter accounts for 32.5%. These results indicate that bypassing reranking is a larger source of inference-time savings, while pruning candidate sets for mentions that still require reranking provides a substantial additional efficiency gain.

We have added the details of this analysis to Appendix D.3 of our revised manuscript.


Q3. The proposed method relies on score differences relative to the top-ranked candidate without explicitly accounting for the reliability of that candidate. Could the authors analyze the behavior of the pruning strategy when the top-ranked candidate is incorrect or the candidate generator exhibits low confidence, particularly regarding the resulting pruning errors?

Thank you for this question. We added an analysis of pruning behavior by top-ranked candidate correctness and candidate generator confidence, using the same dual-encoder pipeline as the main experiment. For correctness, we group mentions according to whether the top-ranked candidate is correct and whether gold is in the initial candidate set. For confidence, we group all test mentions into low-, medium-, and high-confidence categories using tertile boundaries of the top-two score margin. We show the results in the table below. Retained candidates refers to the average number of candidates retained after pruning, counting mentions with only one retained candidate as one.

The correctness analysis shows that our method retains more alternatives when the top-ranked candidate is incorrect but the gold is in the initial set, suggesting that it adjusts the candidate set sizes in response to the top-ranked candidate correctness. Nonetheless, the gold entity is pruned for approximately 45% of these mentions, and harms exceed rescues, revealing a challenging accuracy–efficiency trade-off. On the other hand, the confidence analysis shows that pruning maintains or improves end-to-end accuracy relative to No-Pruning across all evaluated confidence groups.

We have added the details of this analysis to Appendix D.2 of our revised manuscript. We believe this revision further enhances the analytical depth of our work.

GroupMentionsRetained CandidatesGold Pruned (%)No-Pruning Acc.Ours Acc.RescueHarm
Correctness
Top-ranked correct21,6081.340.095.697.74480
Top-ranked incorrect, gold in initial set3,7655.0345.733.227.388309
Gold absent from initial set1,90211.19—0.00.000
Confidence
Low14,9183.8111.366.668.1522300
Medium8,0241.002.795.895.8149
High4,3331.000.699.199.100

We hope that our response has addressed your concerns. By conducting additional analyses and revising the manuscript, we believe the manuscript's soundness and analytical depth have been substantially improved, further strengthening our work. If you have any further questions, we would be happy to address them. Thank you again for your time and consideration.

Best,

Authors

Reviewer_JzzXOpenReview ↗

To authors

Thank you for the detailed response and additional analysis. The new examples help clarify how the reranker improves selection accuracy by distinguishing correct entities from highly ranked but semantically misleading candidates. The supplementary case studies in Appendix D.5 also provide useful evidence supporting this explanation. Although a more systematic quantitative analysis would further strengthen the findings, the response adequately addresses my main concerns. I therefore maintain my original rating (Weak Accept).

AuthorsOpenReview ↗

Response to Reviewer JzzX

Dear Reviewer JzzX,

Thank you very much for your reply.

We are glad to hear that our response has adequately addressed your concerns. We also appreciate that you are leaning towards acceptance of our work, with your rating being Weak Accept.

However, we noticed that your reply appears to refer to the points discussed in our response to Reviewer 2 (N81Y). Could we kindly confirm whether this means that you also found our response to Reviewer N81Y's concerns satisfactory, in addition to our response to your own concerns?

Also, we noticed that your initial rating is still listed as 2 (Weak Reject). Given your indication that your current assessment is Weak Accept, we would greatly appreciate it if you could update your rating to 3 (Weak Accept).

If you have any further questions or concerns, please do not hesitate to let us know. We will make every effort to address them before the rebuttal deadline.

We sincerely appreciate your time and consideration.

Best,

Authors

Reviewer_N81YOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the entity disambiguation task, which is formulated as a two-stage pipeline consisting of candidate generation followed by candidate reranking. Given an input mention, the system first retrieves a set of candidate entities and then applies a reranker to select the final prediction.

The main contribution of the paper is a simple candidate-pruning method that is inserted between these two stages, reducing the number of candidates passed to the reranker. Experimental results show that this approach improves inference efficiency, up to a 4× speedup compared to the no-pruning setting, while also yielding higher accuracy.

Critical strengths
  • Strengths
    • The contribution is simple and easy to understand, and the paper is well written and easy to follow.
    • The authors conduct experiments on ZESHEL, and the proposed method outperforms several baselines, including tuned-fixed-k and adaptive-k (both efficiency and accuracy).
    • The paper also includes useful analyses, such as a sensitivity study on the pruning threshold and a comparison between the selected candidates and the gold entity rank.
Critical weaknesses
  • Limitations
    • My main concern is that the proposed method is evaluated on only one pipeline configuration, consisting of a dual-encoder candidate generator and a cross-encoder reranker.
      • Since both the dual-encoder and cross-encoder are initialized from BERT and trained on the same data, there may be confounding factors that make it difficult to isolate the source of the observed improvements.
      • It would therefore be interesting to examine whether the proposed method remains effective when the two components are developed more independently. For example, the authors could consider using BM25 as the candidate generator or an LLM-based reranker to assess the robustness and generalizability of the approach.
Questions for authors
  • Question
    • I am a bit confused about the claimed accuracy improvement. My understanding is that, for difficult examples, no candidates are pruned, while for easy examples, the correct entity is already obvious and only incorrect candidates are removed. Given this, it is not clear to me how pruning leads to higher accuracy.
      • It would be more interesting to see a more detailed error analysis identifying which types of examples benefit most from pruning in terms of accuracy, and why.
      • For example, for cases where the reranker selects an answer that would have been pruned by the proposed method, it would be useful to understand what causes the reranker to make that mistake.
4 comments
AuthorsOpenReview ↗

Official Comment by Authors (1/2)

Dear Reviewer N81Y,

We sincerely thank you for your careful evaluation of our manuscript and your constructive feedback. We especially appreciate your recognition of the strengths of our work, such as the efficiency and accuracy gains of our method, detailed analysis, and well-written, easy-to-follow presentation. Below, we address the questions and limitations raised in your review.


Limitations

  • My main concern is that the proposed method is evaluated on only one pipeline configuration, consisting of a dual-encoder candidate generator and a cross-encoder reranker.
    • Since both the dual-encoder and cross-encoder are initialized from BERT and trained on the same data, there may be confounding factors that make it difficult to isolate the source of the observed improvements.
    • It would therefore be interesting to examine whether the proposed method remains effective when the two components are developed more independently. For example, the authors could consider using BM25 as the candidate generator or an LLM-based reranker to assess the robustness and generalizability of the approach.

Thank you for this suggestion. To examine the generalizability of our approach, we have added an experiment with a zero-shot LLM-based reranker. We implemented the LLM-based reranker based on ChatEL[1], which generates a contextual explanation of the mention and then selects from the retained candidates through a multiple-choice prompt. We construct a pipeline using this LLM reranker and the same dual-encoder candidate generator as the main experiment, and evaluate it on ZELDA using the initial candidate set size K=30K=30.

The table below shows the results. Our method achieves the highest macro-average accuracy while passing substantially fewer candidates to the reranker and incurring a lower API cost than No-Pruning. Additionally, although our method is slightly more expensive than Adaptive-kk, it achieves higher accuracy. These results show that the benefits of our method extend to zero-shot LLM-based reranking.

We have added the details of this experiment to Appendix D.8 of the revised manuscript.

MethodMacro Acc. (%)API cost (USD)Avg. CandsAIDA-BTWEEKIREDDIT-POSTSREDDIT-COMM.WNED-CWEBWNED-WIKISLINKS-TAILSLINKS-SHADOWSLINKS-TOP
Oracle89.30.222.8492.485.596.693.376.493.699.381.185.2
No-Reranking81.00.000.0083.577.992.387.170.5‾\underline{70.5}90.299.061.966.2
No-Pruning80.31.3630.0084.579.482.077.967.484.695.372.079.6
Tuned-Fixed-k79.80.9210.0084.078.381.177.466.684.996.371.6‾\underline{71.6}78.3‾\underline{78.3}
Adaptive-k81.7‾\underline{81.7}0.160.5885.4‾\underline{85.4}80.1‾\underline{80.1}88.5‾\underline{88.5}87.0‾\underline{87.0}69.888.898.465.971.5
Ours82.50.23‾\underline{0.23}1.79‾\underline{1.79}86.281.087.586.270.789.8‾\underline{89.8}98.8‾\underline{98.8}67.675.0

[1] ChatEL: Entity Linking with Chatbots (Ding et al., LREC-COLING 2024, pp.3086–3097)

AuthorsOpenReview ↗

Official Comment by Authors (2/2)

Question

  • I am a bit confused about the claimed accuracy improvement. My understanding is that, for difficult examples, no candidates are pruned, while for easy examples, the correct entity is already obvious and only incorrect candidates are removed. Given this, it is not clear to me how pruning leads to higher accuracy.
    • It would be more interesting to see a more detailed error analysis identifying which types of examples benefit most from pruning in terms of accuracy, and why.
    • For example, for cases where the reranker selects an answer that would have been pruned by the proposed method, it would be useful to understand what causes the reranker to make that mistake.

Thank you for this question. The candidate reranker can improve accuracy in entity disambiguation pipelines as it is more expressive than the candidate generator. However, the reranker can also select an incorrect distractor, even when the generator ranks the correct entity at or near the top of the candidate list, since an example that appears easy for the generator is not necessarily easy for the reranker.

To identify representative cases where pruning improves the accuracy, we manually inspected the prediction examples of the dual-encoder pipeline on ZELDA. The inspection revealed cases where the reranker selects a topically related distractor, i.e., an entity that matches the surrounding topic but is not the correct entity for the mention. The following examples illustrate this pattern. Parenthesized numbers indicate ranks in the original generator list.

Mention and contextGoldNo-PruningOursRetained Cands
“Over seventy percent of the world’s beer is consumed in China”China (1)Beer in China (2)China1
“…Windows hosting with ASP.Net, ASP, PHP…”Active Server Pages (2)Internet Server Application Programming Interface (22)Active Server Pages2

For China, the reranker selects Beer in China, which matches the sentence’s topic, although the mention refers to the country. Pruning removes this distractor while preserving China, preventing the reranking error. For ASP, the reranker would select another server technology instead of Active Server Pages, but pruning removes this distractor at generator rank 22 while retaining the correct entity at rank 2, allowing the reranker to select Active Server Pages.

We have added these examples and their analysis to Appendix D.5 of the revised manuscript.


Thank you again for your constructive feedback and valuable suggestions. We hope that our response has addressed your concerns. Your feedback has helped strengthen our claims and improve the overall clarity of the paper. If you have any further questions, we would be happy to address them. Thank you again for your time and consideration.

Best,

Authors

Thanks for the response and the additional results. I will keep my original assessment, mainly because I was already the most positive one.

AuthorsOpenReview ↗

Response to Reviewer N81Y

Dear Reviewer N81Y,

Thank you very much for your reply. We are glad that you acknowledge our additional experiments and analyses. We are also very grateful for your positive evaluation of our work.

We will ensure that the revisions we made based on your valuable suggestions will be even more refined and included in the camera-ready version.

Best,

Authors

Reviewer_uwz6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper works with the standard two-stage entity disambiguation pipeline: a BERT-base dual encoder retrieves 30 candidates and a BERT-base cross encoder reranks them. The proposal is to prune the candidate list before reranking. A candidate survives if its dual-encoder score is within a threshold of the top-1 score; the threshold is a conformal quantile of the gold gaps on a held-out split of ZELDA train, and the miscoverage level α is then chosen on a grid by end-to-end dev accuracy. If only one candidate survives, the cross encoder is skipped. On the nine ZELDA test sets this gives slightly higher macro accuracy than no pruning (+0.7) with under two candidates per mention and roughly a 4.5× reduction in total time, and it beats Adaptive-k by 0.6. Additional experiments swap the retriever (E5), the reranker (DeBERTa-v3), the benchmark (GERBIL) and the score definition, and report bootstrap confidence intervals.

Critical strengths
  • The problem is real and the pipeline is the one practitioners actually run. Cutting CE calls by 94% at no accuracy cost is a useful engineering result if it holds up.
  • The paper is unusually candid. Section 3.2 says plainly that exchangeability fails on ZELDA, that the coverage guarantee may not hold, and that α is treated as a hyperparameter; Appendix A names the obvious untested alternative.
  • The evaluation is broader than usual for a method this small (several K, two retrievers, two rerankers, a second benchmark, timings over three runs, per-dataset CIs, coverage and rescue/harm counts). Tables 2 and 13 in particular let the reader see what the method is really doing.
  • The writing is clear and the method is reproducible from the description.
Critical weaknesses

1. The retained set is a well-known heuristic, and the "conformal" layer does not change that. "Keep a candidate if its score is within a margin of the best one" is absolute threshold pruning from Freitag & Al-Onaizan (2017), the ε-band TAGME used for entity disambiguation back in 2010, and, because the dual encoder is trained with a softmax over dot products, exactly min-p sampling. Intrator et al. (2024), which the paper cites as a generic CP-for-IR reference, already take a conformal quantile of a top-1-normalised score, which is the exponentiated version of the same gap. None of this is discussed. What is left as new is parameterising the threshold through α. But α maps monotonically onto a threshold and α is picked by dev accuracy on a grid, so the procedure is a grid search over the threshold with a different axis label. Appendix A says that directly tuning the gap threshold is future work; I think it is the same experiment.

2. The conformal framing is slightly misleading, and the citations used to justify dropping the guarantee do not support that move. Nominal coverage is 95% but observed coverage in Table 2 is several points lower even after accounting for the top-30 recall ceiling, and calibration and α-selection use the same dev labels, so the finite-sample guarantee would not hold even in-distribution. Kasa et al. is cited as support, but its point is that CP breaks under shift and that one should adapt the score with unlabelled test data, which the submission does not do; Pearson et al. does not appear to use CP at all. Meanwhile Li et al. (2022) already give certified error control for candidate pruning in two-stage ranking and are cited in one clause without comparison. If a guarantee is wanted it is cheap: conformal risk control on the label-free loss "pruned prediction differs from unpruned prediction", calibrated on unlabelled target-domain mentions, would give a real bound without any exchangeability assumption on labels. Without something like that, the method should be called a quantile-parameterised score-gap threshold.

3. The accuracy gain is a score-fusion effect that is never controlled for. Pruning can only improve over no pruning because the dual-encoder gap acts as a hard prior on the cross encoder; Table 2 shows more rescues than harms, and the no-reranking row already rescues many mentions on its own. The cheapest control, a convex combination of the two scores with the weight tuned on dev (with or without pruning), is absent, so I cannot tell how much of the gain is pruning and how much is ensembling implemented as a gate. The E5 experiment makes this worse: there, full reranking is below no reranking, i.e. the cross encoder is net harmful with a stronger retriever. That points to an undertrained cross encoder (one epoch) rather than to a virtue of pruning.

4. The baseline set omits the obvious comparators and the main competitor is handicapped. I would expect an absolute probability threshold, cumulative-probability truncation, a top1−top2 margin gate with a dev-tuned threshold, fixed-k at a matched budget (Tuned-Fixed-k lands on k=10, against under two candidates for the proposed method, so Table 1 is not a same-budget comparison), and at least one learned truncation method, which Section 7 dismisses for relying on learned components although their only input is the score sequence. Adaptive-k, with its buffer of three, can never return fewer than four candidates and so structurally cannot skip the cross encoder, which is where most of the speed-up comes from.

5. The claims overreach, and there is no training variance. The conclusion says the method outperforms the baselines in both efficiency and accuracy and is robust to hyperparameter choice. On GERBIL with long candidate lists it is worse than Adaptive-k on accuracy, time and candidates at once. Figure 3 shows a sharply peaked α curve in which only two grid points beat Adaptive-k. The macro gain comes from a few datasets while S-SHADOW, the split built to test shadowed entities, loses, and the method has the lowest coverage of all systems. Transfer across domains is asserted from a Wikipedia-calibrated threshold without per-domain coverage. All accuracies are from a single seed; the extra seeds in Appendix C.4 appear only as a scatter plot. Minor: the no-reranking timing differs between Table 1 and Table 14, and the GERBIL numbers are far below usual ED results on those sets, which suggests an undescribed NIL / KB convention.

Questions for authors
  1. Plot dev accuracy against the gap threshold directly, for your quantile grid and for a uniform grid with the same number of points. Do they select the same threshold?
  2. Please add a dual-encoder / cross-encoder score-fusion baseline with and without pruning. How much of the gain over no pruning survives?
  3. Please add matched-budget comparators (fixed k = 2 or 3, a margin gate, a probability threshold) as accuracy-vs-candidates curves, and let Adaptive-k return a single candidate so that it can also skip the cross encoder.
  4. Why does full reranking reduce accuracy under the E5 retriever? Would a cross encoder trained for more than one epoch change Table 1?
  5. Report mean ± std over the five seeds you already trained, for Ours, Adaptive-k and No-Pruning.
6 comments
AuthorsOpenReview ↗

Official Comment by Authors (1/4)

Dear Reviewer uwz6,

We would like to thank you for carefully evaluating our manuscript and for the constructive feedback. We are particularly grateful for your recognition of the strengths of our work, including the practical relevance and substantial efficiency gains of our method, the comprehensive evaluation and analysis, and clarity of the presentation. Below, we address the questions and concerns raised in your review.


W1 The retained set is a well-known heuristic, and the "conformal" layer does not change that.

Thank you for this comment and for bringing potentially related work. Our work studies adaptive candidate pruning using the score difference from the top candidate in practically relevant modern ED pipelines, comprising candidate generators and computationally expensive rerankers, with the aim of addressing the accuracy–efficiency trade-off. We provide empirical evidence that this lightweight mechanism substantially reduces reranking computation while maintaining or improving end-to-end accuracy. In Section 7.1 of our revised manuscript, we have clarified the difference between our work and the related work you suggested.

Freitag & Al-Onaizan (2017) [1] and min-p sampling [2] apply filtering during autoregressive decoding, with min-p sampling in particular selecting next tokens from a fixed vocabulary. In contrast, our work tackles a substantially different ED task: we apply pruning between candidate generation and reranking to reduce the computational demands of a separate model when predicting an appropriate entity from a large set of candidates. While the vocabulary space in token generation typically contains tens to hundreds of thousands of tokens, the knowledge bases in ED can contain millions to tens of millions of entities. Therefore, it is challenging to directly apply techniques developed for autoregressive decoding to ED, requiring methods tailored to such large and noisy search spaces. In this regard, our method is designed to address these real-world challenges, and we believe that it offers sufficient novelty.

Intrator et al. (2024) [4] apply conformal prediction to information retrieval, aiming to produce compact retrieved sets while obtaining theoretical coverage guarantees. However, in practical applications, information retrieval is often used to send candidates to a downstream model for further processing, so candidate selection must account for the accuracy and efficiency of the entire pipeline. Instead of only focusing on a single component such as retrievers, our work addresses this pipeline-level problem in modern neural ED: we use adaptive candidate pruning to control the workload of a computationally expensive reranker while maintaining or improving end-to-end prediction accuracy.

TAGME [3] represents a classical ED architecture that uses Wikipedia link statistics and score-based candidate filtering. While this architecture can achieve high accuracy in a simplified closed setting where target entities are covered by Wikipedia and sufficient link statistics are available, it may struggle to adapt to a more practical open setting that includes new entities for which such link information is not available. In contrast, modern neural ED pipelines provide greater flexibility that allows entities uncommon or unseen during model training to be identified, while requiring substantially more computation. Our work utilizes adaptive candidate pruning with the aim of leveraging this flexibility without incurring substantial computation, tackling the accuracy–efficiency trade-off in neural ED pipelines.

To avoid confusion regarding these prior works and to clarify the contributions and novelty of our work, we have included the above discussion in Section 7.1 of our revised manuscript. We kindly ask that you acknowledge our contributions on these points.

Appendix A says that directly tuning the gap threshold is future work; I think it is the same experiment.

For the direct tuning of the threshold, please refer to our response to your Q1 shown below.

[1] Beam Search Strategies for Neural Machine Translation (Freitag & Al-Onaizan, NGT 2017, pp.56–60)

[2] Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs (Nguyen et al., ICLR 2025, pp.70333–70366)

[3] TAGME: on-the-fly annotation of short text fragments (by wikipedia entities) (Ferragina & Scaiella, CIKM '10, pp.1625–1628)

[4] Streamlining Conformal Information Retrieval via Score Refinement (Intrator et al., FEVER 2024, pp.186–191)

AuthorsOpenReview ↗

Official Comment by Authors (2/4)

W2 The conformal framing is slightly misleading, and the citations used to justify dropping the guarantee do not support that move.

Thank you for this comment. As you acknowledge in the Strength section of your review, Section 3.2 of our manuscript already explains that exchangeability may fail in ZELDA, that the coverage guarantee may not hold, and that we treat α\alpha as a development-tuned hyperparameter. Thus, our work already explicitly distinguishes the proposed method from standard conformal prediction. To further clarify this point, we have added “quantile-parameterised score-gap threshold” framing to Abstract, and Sections 1 and 3, as you suggested.

At the same time, conformal prediction still provides insights into adaptive candidate pruning mechanisms, even when the theoretical guarantee may not be obtainable. Kasa et al. (2025) [5] adapt conformal scores using unlabeled target-domain data and demonstrate practical improvements when coverage guarantees are unavailable. We cite their work as motivation for investigating the empirical utility of conformal-inspired procedures even when the statistical guarantee may not hold. Pearson et al. (2026) [6] explicitly use density-based conformal prediction, treating its outputs as uncertainty signals to decide which inputs require human review, as described in Section 4 of the work titled “Conformal Prediction for HAIC” (pp.6-7). This work motivates us to use conformal-inspired mechanisms for practical downstream decisions of candidate pruning. We revised Sections 3.2 and 7.2 of our manuscript to clarify these points.

Regarding Li et al. (2022) [7], we have added a mention of certified error control in Section 7.2 of our revised manuscript to elaborate on this work. However, it still requires the exchangeability assumption between calibration and test data, making it impossible to directly apply to the common ED setups. Additionally, controlling a difference between pruned and unpruned predictions is an interesting research topic, but it would be a different guarantee from the trade-off between end-to-end accuracy and efficiency, which our work deals with. If there is anything else that we should discuss, please let us know.

[5] Adapting Prediction Sets to Distribution Shifts Without Labels (Kasa et al., PMLR, 2025)

[6] Uncertainty-Aware Systems for Human-AI Collaboration (Pearson et al., TMLR, 2026)

[7] Certified Error Control of Candidate Set Pruning for Two-Stage Relevance Ranking (Li et al., EMNLP 2022, pp.333-345)


W3 The accuracy gain is a score-fusion effect that is never controlled for.

Thank you for this comment. According to your suggestion, we have added a score-fusion experiment using the same trained pipeline, combining the raw dual-encoder and cross-encoder scores as sfusion(m,e)=λsde(m,e)+(1−λ)sce(m,e).s_{\mathrm{fusion}}(m,e)=\lambda s_{\mathrm{de}}(m,e)+(1-\lambda)s_{\mathrm{ce}}(m,e). We evaluate this score fusion with and without our candidate selection procedure, tuning the fusion weight on development data. This gives No-Pruning explicit access to the generator information that our method incorporates through the selection of the pruning threshold.

While No-Pruning achieves slightly higher macro-average accuracy than the proposed method (84.1% vs. 83.8%) with score fusion, In the main experiment, our method improves accuracy over No-Pruning when reranking uses cross-encoder scores alone. Therefore, the improvement of No-Pruning under fusion suggests that incorporating dual-encoder guidance into the final prediction may partly explain our method’s accuracy gains. Our method incorporates this guidance while also reducing the number of candidates requiring cross-encoder inference, achieving a favorable balance in the accuracy–efficiency trade-off.

We have added this experiment and discussion to Appendix D.4 of the revised manuscript. For the E5 pipeline, we conducted an additional experiment where we trained the cross-encoder for two epochs, and we include the results in our response to your Q4. We kindly ask you to refer to it.

We believe that your question regarding the source of the accuracy improvement has been clarified by our additional experiment.

AuthorsOpenReview ↗

Official Comment by Authors (3/4)

W4 The baseline set omits the obvious comparators and the main competitor is handicapped.

Thank you for this comment. We have expanded the evaluation to include additional score-based baselines, i.e., Absolute Probability, Cumulative Probability, and Top1-Top2 Gate, and a learned truncation method, i.e., Choppy[8].

The table below shows that our method achieves higher macro-average accuracy while passing fewer candidates to the cross-encoder and requiring shorter inference time than each of these added baselines.

MethodAcc.Inference Time (s) ↓Pruning Time (s) ↓Avg. Cands ↓
Absolute Probability83.1419 ± 2.000.7 ± 0.41.91
Cumulative Probability83.3422 ± 1.40.5 ± 0.31.93
Top1–Top2 Gate83.4780 ± 20.30.4 ± 0.46.89
Choppy82.8783 ± 4.01.0 ± 0.39.00
Ours83.5380 ± 3.30.3 ± 0.21.79

Furthermore, we have conducted a post-hoc accuracy–efficiency trade-off analysis for comparisons of our method with baselines at similar candidate budgets. For this analysis, we sweep the hyperparameters of our method and the baselines, evaluating fixed-kk for k=1,…,30k=1,…,30, and vary Adaptive-kk’s buffer from 0 to 29. This explicitly includes fixed k=2k=2 and k=3k=3, as well as Adaptive-kk with B=0B=0, which can return a single candidate and bypass the cross-encoder. For each setting, we plot macro-average test accuracy against the average number of candidates passed to the cross-encoder.

We show the resulting plot in Figure 5 of our revised manuscript. It shows that our method generally achieves higher accuracy than Absolute Probability, Cumulative Probability, Fixed-kk, and Adaptive-kk at comparable numbers of candidates. Top1–Top2 Gate approaches the accuracy of our method that uses the threshold selected on the development set, only by requiring substantially more candidates. This indicates that our method obtains the favorable accuracy–efficiency trade-off compared with the baselines, including when Adaptive-kk can skip reranking.

We have added the details of these analyses to Appendices D.6 and D.7 of our revised manuscript.

[8] Choppy: Cut Transformer for Ranked List Truncation (Bahri et al., SIGIR '20, pp.1513–1516)


W5 The claims overreach, and there is no training variance.

Thank you for your comments under this section. Below, we provide our responses to the points raised here.

W5-(1) The conclusion says the method outperforms the baselines in both efficiency and accuracy and is robust to hyperparameter choice. On GERBIL with long candidate lists it is worse than Adaptive-k on accuracy, time, and candidates at once.

We have revised the phrasing to “outperformed the baselines in both efficiency and accuracy under most of the experimental configurations” in line 513 of Conclusion of our revised manuscript to better reflect this point.

W5-(2) Figure 3 shows a sharply peaked α curve in which only two grid points beat Adaptive-k.

We respectfully note that in Figure 3, our method achieves higher accuracy than Adaptive-kk at four consecutive grid points α∈{0.03,0.04,0.05,0.06}\alpha \in \{0.03, 0.04, 0.05, 0.06\}, instead of two, and it selects α=0.05\alpha = 0.05, which lies near the center of this range.

W5-(3) The macro gain comes from a few datasets while S-SHADOW, the split built to test shadowed entities, loses, and the method has the lowest coverage of all systems.

We would like to clarify that in the main experimental results shown in Table 1, our method achieves the highest accuracy in eight out of nine ZELDA test datasets, instead of just a few, and all of these results contribute to its macro-average accuracy gain.

W5-(4) Transfer across domains is asserted from a Wikipedia-calibrated threshold without per-domain coverage.

Instead of the coverage of the candidate generator, our domain transfer statement concerns the empirical accuracy and efficiency gains in the end-to-end ED, which we believe is practically more important than the generator coverage. To clarify this point, we adjusted the phrasing in Section 5 of the revised manuscript.

W5-(5) All accuracies are from a single seed; the extra seeds in Appendix C.4 appear only as a scatter plot.

We have added the mean and standard deviation of the evaluation metrics across the five training seeds to Appendix C.4 of our revised manuscript. The results show that across the training seeds, our method achieves a higher macro-average accuracy while sending a smaller number of candidates to the cross-encoder than the baselines.

W5-(6) Minor: the no-reranking timing differs between Table 1 and Table 14, and the GERBIL numbers are far below usual ED results on those sets, which suggests an undescribed NIL / KB convention.

Thank you for letting us notice these points. There was a typo in Table 14 (Table 21 in the revised manuscript), and we have corrected it in our revised manuscript.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewers,

We sincerely thank you for your time, careful evaluation, and constructive feedback.

We apologize for the delay in our response, as conducting additional experiments to thoroughly address your comments required more time than anticipated.

We have carefully revised the manuscript based on your valuable comments.

  • The blue-highlighted areas correspond to revisions for Reviewer uwz6.
  • The orange-highlighted areas correspond to revisions for Reviewer N81Y.
  • The green-highlighted areas correspond to revisions for Reviewer JzzX.
  • The purple-highlighted areas correspond to revisions for both Reviewers uwz6 and JzzX.

We have also included a note that shows the list of questions and concerns raised in the reviews and our corresponding revisions at the end of our updated manuscript.

We sincerely appreciate your constructive suggestions. We believe that the revisions have substantially improved the clarity and quality of the manuscript, and we hope that our updates satisfactorily address all of your concerns.

Thank you again for your careful review.

Best,

Authors

AuthorsOpenReview ↗

Final Remarks

Dear Reviewers, ACs, SACs, and PCs,

We sincerely thank you for your careful evaluation and constructive feedback. As the discussion period nears its end, we would like to summarize our responses to the feedback received.


This work contributes by proposing a lightweight, adaptive candidate-pruning method in entity disambiguation (ED) pipelines that addresses the practically important accuracy–efficiency trade-off.

We greatly appreciate the reviewers’ recognition of the following strengths:

  • Practical Advantages: Substantial efficiency gains while maintaining or improving accuracy (uwz6, N81Y, JzzX) in practically relevant settings (JzzX).
  • Strong Empirical Evidence: Comprehensive experiments covering various models, configurations, benchmarks, and baselines (JzzX) that provide strong empirical evidence (uwz6).
  • Useful Analysis: Useful analysis to explain the behavior of the proposed method (uwz6, N81Y, JzzX).
  • Clear Presentation: Clearly-written and easy-to-follow manuscript (uwz6, N81Y).

We have mainly received three types of questions and concerns.

1. Additional Experiments (uwz6 W4, W5, Q1, Q4, Q5; N81Y Limitations)

Experiments using additional models, baselines, and training seeds were requested to verify the robustness and generalizability of our method. To address this, we conducted all the requested experiments: we added an LLM-based zero-shot reranker, cross-encoders trained for longer epochs, three score-based and one learned baseline, and results from five training seeds. The results show that the gains of our method persist across these variations, further strengthening empirical evidence for our work, as explicitly recognized by Reviewer uwz6.

2. Additional Analysis (uwz6 W3, W4, Q1, Q2, Q3; N81Y Question; JzzX W2, Q2, Q3)

Reviewers requested additional analyses to compare the accuracy at matched candidate budgets, to explain how our method improves accuracy and efficiency, and to investigate the challenges our method faces. In response, we added an accuracy vs. candidate counts plot (response to uwz6 W4, Q3; JzzX W2) and direct threshold tuning comparisons (uwz6 Q1), supporting our method’s favorable accuracy–efficiency trade-off; score-fusion controls (uwz6 W3, Q2) and error analyses (N81Y Question; JzzX Q3), clarifying the sources of accuracy gains and remaining failure cases; and decomposition of efficiency gains (JzzX Q2), quantifying the contributions of reranking bypass and candidate reduction. These results reveal detailed behaviors of our method, which is considered “useful” by the Reviewers, and further enhance the analytical depth of our work.

3. Clarify Positioning (uwz6 W1, W2)

Reviewer uwz6 raised a concern about our method’s connection to conformal prediction. In response, we clarified that our method adapts the threshold selection framework of conformal prediction to ED, prioritizing empirical utility over theoretical coverage guarantee. We also provided further explanation of prior work that motivates this adaptation, and added a more precise framing to the description of our method. As a result, we believe that the distinction between our method and standard conformal prediction has become clearer, as Reviewer uwz6 acknowledges.

Reviewer uwz6 also raised a concern regarding the significance of our contribution, which is the only remaining concern. To address this, we revised Section 7.1 to clarify two methodological contributions of our work specific to modern two-stage ED pipelines. First, we introduce and empirically validate an adaptive pruning strategy that uses the generator’s scores to reduce reranking cost. Our score-fusion analysis suggests that the accuracy gains partly depend on how generator and reranker scores jointly influence the final prediction, underscoring the need to evaluate candidate pruning across the full ED pipeline. Second, because the generator’s scores have model-specific scales, we use quantiles to express pruning aggressiveness on an interpretable scale and select the threshold based on the accuracy of the full pipeline. Furthermore, we also emphasized that our systematic analysis of this trade-off is a contribution in its own right. Given this paper’s primary area, “Applications to Language”, where practical effectiveness and empirical insights are important considerations, we ask that its overall significance be assessed in light of both these methodological contributions and the practical value and strengthened empirical evidence that Reviewer uwz6 recognized.


We have provided detailed responses to all reviewer questions and concerns, and incorporated them into our revised manuscript. We show the list of these revisions and corresponding reviewer comments at the end of our manuscript (pp.42–44). Together with the additional experiments, analyses, and clarifications, we believe that our manuscript has been substantially strengthened.

Thank you again for your time and consideration.

Best,

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.