acceptodds
Under review as a conference paper at ICLR 2027

On the Limits of Candidate Generation in Biomedical Entity Linking

Abstract

Biomedical entity linking systems rely on candidate generation to retrieve potential entities prior to disambiguation. This stage imposes a hard upper bound on end-to-end performance: if the correct entity is not retrieved, no downstream model can recover it. Despite this, candidate generation is typically treated as a standalone retrieval problem, with limited attention to its structural limitations. In this work, we argue that candidate generation is inherently incomplete under any single retrieval paradigm and that improving coverage requires exploiting complementary retrieval behaviors. We introduce a heterogeneous ensemble framework that combines diverse candidate generators and analyze how their differences contribute to increased coverage. Empirically, our approach substantially improves the inclusion of gold entities in candidate sets across multiple biomedical benchmarks. When paired with standard disambiguation models, including cross-encoders and LLM-based rerankers, these gains translate into consistent end-to-end improvements, with an average increase of 8.1 absolute points across all datasets with our best disambiguator. We further show that diversity—not just individual model strength—is an important contributor to these gains, and we quantify the diminishing returns of adding additional generators. Finally, we demonstrate that a simple heuristic for aggregating candidates achieves near-optimal performance relative to an oracle, suggesting that most gains arise from coverage expansion rather than complex fusion strategies. Our results highlight candidate generation as a fundamental bottleneck in entity linking and provide a principled approach to mitigating it.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Entity linking, Biomedical, Candidate Generators, Entity Disambiguation, BERT-based retriever models, LLMs, Fusion strategy

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_xwt3OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the limitations of candidate generation in biomedical entity linking, highlighting how incomplete candidate retrieval constrains downstream disambiguation performance. To address this bottleneck, the authors propose a heterogeneous ensemble framework that combines multiple candidate generators with complementary retrieval strategies, including lexical, dense, and autoregressive methods. Using a simple round-robin aggregation strategy, the framework improves candidate coverage across six biomedical benchmarks without increasing the candidate set size. When combined with cross-encoders and LLM-based disambiguators, the approach achieves consistent improvements in end-to-end accuracy, with an average gain of 8.1 percentage points over the best evaluated baselines. The paper further demonstrates that retrieval diversity is crucial for improving coverage, while additional generators yield diminishing returns, and simple aggregation performs comparably to an oracle strategy.

Critical strengths
  1. Clear Writing and Organization. The paper is well-written and clearly structured, making it easy to follow the motivation, proposed methodology, and key findings. The presentation is generally clear, and the figures effectively illustrate the proposed framework.

  2. Comprehensive Experimental Evaluation. The paper provides extensive experiments across six biomedical entity linking benchmarks. The authors conduct thorough ablation studies and analyses of model diversity, candidate overlap, aggregation strategies, and computational efficiency, which strengthen the empirical findings and support the main conclusions.

Critical weaknesses
  1. The proposed framework primarily combines existing candidate generation models using a simple round-robin aggregation strategy. While the empirical improvements are promising, the core methodology is relatively straightforward, and similar ensemble-based retrieval approaches have been explored in prior work [1, 2]. The paper could better clarify its methodological contributions and distinguish its approach from existing methods.

  2. The paper’s central claim is that candidate generation is inherently incomplete under any single retrieval paradigm. However, the experiments only demonstrate that combining several existing retrieval models improves candidate coverage. This is an expected consequence of combining models with partially overlapping predictions and does not establish any fundamental limitation of individual retrieval paradigms. The paper lacks a theoretical analysis or controlled experiments demonstrating that a sufficiently strong single retriever cannot achieve comparable coverage. Therefore, the empirical evidence does not adequately support the paper’s broader claim about the inherent limitations of candidate generation.

  3. Several major conclusions of the paper appear intuitive rather than scientifically surprising. For example, combining diverse retrieval models increases candidate coverage, models with different architectures retrieve complementary candidates, and adding more models eventually produces diminishing returns. These observations are well-established principles in ensemble learning and information retrieval. Although the authors provide extensive empirical evidence, it is unclear what fundamentally new insights the experiments offer beyond confirming these existing intuitions in the biomedical entity linking domain.

  4. Although the paper emphasizes computational efficiency, its full framework requires running five candidate generators, including computationally expensive neural and autoregressive models. This introduces additional inference latency, memory requirements, and deployment complexity compared with a single retriever. The lightweight ensemble experiment provides some evidence of efficiency, but it compares against a computationally expensive baseline rather than systematically evaluating the accuracy–latency trade-off across different ensemble configurations. Consequently, the practical advantages of the proposed framework remain insufficiently established, particularly for latency-sensitive applications.

[1] Sung, Mujeen, et al. "Biomedical entity representations with synonym marginalization." Proceedings of the 58th annual meeting of the association for computational linguistics. 2020.

[2] Borchert, Florian, et al. "xMEN: a modular toolkit for cross-lingual medical entity normalization." JAMIA open 8.1 (2025): ooae147.

Questions for authors

N/A

Reviewer_dzyuOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper combines heterogeneous candidate generators for biomedical entity linking using round-robin interleaving and deduplication. It evaluates candidate recall and downstream linking accuracy on six benchmarks, and analyzes generator complementarity, aggregation strategies, and computational cost.

Critical strengths

The paper evaluates six biomedical benchmarks and holds the final candidate budget fixed when comparing individual generators with the ensemble. This is a useful design choice for testing whether complementary retrieval improves coverage without simply increasing the number of candidates presented downstream.

Critical weaknesses
  1. Substantial numerical inconsistencies make the results difficult to trust.

    Table 21 divides test samples into Seen and OOV groups. Their sample-weighted accuracies should reproduce the overall accuracy for the same predictions. They do not: | Dataset | Model | Reconstructed from Table 21 | Reported overall | |---|---|---:|---:| | NCBI-Disease | ArboEL | 0.7415 | 0.802 | | NCBI-Disease | SapBERT | 0.7307 | 0.800 | | NCBI-Disease | SciSpacy | 0.6926 | 0.748 | | NCBI-Disease | BioBART | 0.5668 | 0.627 | | NCBI-Disease | Ensemble + Qwen3.5-27B | 0.8687 | 0.875 | | NLM-Chem | Ensemble + Qwen3.5-27B | 0.8976 | 0.900 | | NLM-Gene | Ensemble + Qwen3.5-27B | 0.5864 | 0.591 |

    For example, ArboEL’s NCBI-Disease result in Table 21 gives (795 × 0.801 + 165 × 0.455) / 960 = 0.7415, approximately 6.05 percentage points below the reported 0.802. This cannot be explained by rounding. Additional errors reinforce this concern. Section 4.4 reports ArboEL’s GNormPlus accuracy as 80.0%, while Tables 2, 3, and 8 report 58.5%. Table 8 labels 0.687 → 0.734 as +2.4 percentage points, although the difference is +4.7 points. These discrepancies go beyond an isolated typo. The authors must establish which results are correct and whether tables were produced from different runs, configurations, or evaluation procedures.

  2. Figures 1 and 2 have a strongly templated, AI-generated appearance. Appearance alone does not establish their provenance, but their concrete problems are clear: Figure 1 introduces an undefined “dense space” for heterogeneous generators, while Figure 2 depicts six models despite the five-model ensemble and devotes substantial space to decorative icons and a generic “performance boost” graphic. These elements obscure rather than explain interleaving, deduplication, and truncation.

    The writing similarly presents a simple ensemble as a “principled” account of candidate-generation “limits.” The experiments establish improvements for particular model combinations, not inherent limitations of every single retrieval paradigm. Scientific communication requires the authors to check the accuracy and purpose of every sentence and figure, regardless of which tools were used to produce them.

Questions for authors
  1. Please reconcile every discrepancy listed above. Do Tables 2, 3, and 21 use the same checkpoints, test samples, and predictions? If so, why do the Seen/OOV results fail to reconstruct overall accuracy? If not, explicitly identify the differences.
  2. Please provide per-example predictions, evaluation scripts, and experiment identifiers sufficient to regenerate the affected tables. Which reported results and conclusions change after correction?
  3. Please clarify whether AI tools were used to generate the figures or draft the text, and how their factual accuracy was checked. In particular, what does Figure 1’s “dense space” represent, and why does Figure 2 show six models? These issues require correction regardless of their origin.
  4. What evidence supports the universal claim of inherent incompleteness, beyond the observed errors of the evaluated generators? Please narrow the claims if no such evidence is available.
Reviewer_aaYZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies candidate generation as a bottleneck in biomedical entity linking. It combines five existing generators through performance-ordered round-robin interleaving, keeping the final candidate budget fixed. Across six benchmarks, the ensemble improves candidate recall and downstream linking accuracy with cross-encoder and LLM disambiguators. Additional experiments examine generator complementarity, ensemble composition, fusion strategies, computational cost, and performance on entities unseen during training.

Critical strengths
  1. Consistent candidate-recall improvements under a fixed output budget. Table 2 reports improvements over all evaluated individual generators and xMEN across six datasets. Holding the final candidate count fixed makes this meaningful evidence that combining generators can improve candidate quality.

  2. Evidence connecting retrieval improvements to downstream performance. Appendix A.4.4 applies the same Qwen3.5-27B disambiguator to ArboEL and ensemble candidates. The composition ablations and unique-retrieval analysis provide additional evidence that multiple generators contribute useful candidates.

Critical weaknesses
  1. Missing matched strong baselines. Table 17 fixes Qwen3.5-27B but compares ensemble candidates only against ArboEL. On GNormPlus, ArboEL’s Recall@64 is 0.661, versus 0.908 for SapBERT and 0.918 for xMEN. Comparisons against SapBERT/xMEN with the same LLM and prompting settings are needed to establish the ensemble’s incremental value.

  2. Insufficient support for the diversity and near-optimality claims. Table 4 changes both architectural diversity and retrieval strength: SciSpacy substantially outperforms the replaced BioBART on GNormPlus. Moreover, the oracle is described inconsistently as optimal candidate selection and optimal generator ordering. Its constraints and candidate-recall gains need clarification before concluding that further fusion optimization is unnecessary.

  3. The motivation is broader than the experimental evidence. Complementary errors among the evaluated models provide limited support for intrinsic limitations of individual retrieval paradigms. The overlap and ablation analyses also do not establish whether complementarity arises from the synonymy, abbreviation, and compositional phenomena used to motivate the approach.

  4. Generalization evidence remains limited. The experiments primarily evaluate target-specific configurations on biomedical literature. Table 21 measures generalization to entities absent from task training within the target knowledge base; held-out-domain evaluation would better establish transfer. Its results also require reconciliation: the NCBI-Disease strata imply ArboEL accuracy of approximately 0.742, versus 0.802 in Table 2.

  5. The efficiency claim needs a clearer cost breakdown. Table 8 combines training and inference time, so the reported speedup does not establish lower deployment latency. Separating training, indexing, candidate retrieval, and disambiguation costs would clarify the practical trade-off, particularly because the ensemble executes multiple generators.

  6. Robustness checks cover a limited setting. Multi-seed pipeline experiments are restricted to NCBI-Disease, while the largest downstream gain occurs on GNormPlus. Repeated runs on the gene datasets and paired confidence intervals would strengthen confidence in the reported improvements; repeated greedy LLM inference alone captures little of the pipeline’s training variability.

Questions for authors
  1. With Qwen3.5-27B and all other disambiguation settings fixed, how does the ensemble compare against SapBERT and xMEN candidates? These comparisons are particularly important on GNormPlus and NLM-Gene.

  2. What precisely can the oracle optimize: generator ordering, candidate allocation, or unrestricted selection from the pooled candidates? Please specify the algorithm and report numerical heuristic-versus-oracle Recall@k alongside downstream accuracy.

1 comment
AuthorsOpenReview ↗

Rebuttal by Authors (6)

We have now completed the multi-seed experiments to address Weakness 6.

To address this concern, we extended our multi-seed experiments to GNormPlus as requested. We repeated the experiments with 2 additional random seeds, for a total of 3 runs, covering candidate generation, downstream disambiguation and heuristic-versus-oracle comparisons. This allows us to assess the stability of the entire pipeline beyond repeated greedy LLM inference.

Table 18: Mean and standard deviation of Recall@k over three runs on GNormPlus and NCBI-Disease. We report individual generators whose outputs varied across seeds; SapBERT produced identical results across all runs.

DatasetkArboELBioBARTEnsemble
GNormPlus10.593 ± 0.0060.085 ± 0.002—
100.665 ± 0.0080.420 ± 0.0020.885 ± 0.002
640.673 ± 0.0080.571 ± 0.0050.941 ± 0.005
NCBI-Disease10.808 ± 0.0020.631 ± 0.001—
100.906 ± 0.0010.793 ± 0.0010.950 ± 7 × 10⁻⁴
640.926 ± 2 × 10⁻⁴0.885 ± 0.0010.990 ± 2 × 10⁻⁴

Table 19: Mean and standard deviation of end-to-end accuracy over three runs on GNormPlus and NCBI-Disease.

DatasetCross-encoderQwen3.5-9BQwen3.5-27B
GNormPlus0.770 ± 5 × 10⁻⁴0.645 ± 0.0010.832 ± 5 × 10⁻⁴
NCBI-Disease0.853 ± 2 × 10⁻⁴0.867 ± 2 × 10⁻⁵0.876 ± 7 × 10⁻⁵

Table 20: Mean and standard deviation over three runs comparing heuristic and oracle candidate aggregation on GNormPlus and NCBI-Disease.

DatasetDisambiguatorHeuristicOracle
GNormPlusCross-encoder0.770 ± 5 × 10⁻⁴0.776 ± 8 × 10⁻⁴
Qwen3.5-9B0.645 ± 0.0010.652 ± 5 × 10⁻⁴
NCBI-DiseaseCross-encoder0.853 ± 2 × 10⁻⁴0.853 ± 7 × 10⁻⁵
Qwen3.5-9B0.867 ± 2 × 10⁻⁵0.870 ± 3 × 10⁻⁴

The results demonstrate that our heterogeneous ensemble maintains strong and stable performance across random seeds on both NCBI-Disease and GNormPlus. In particular, on GNormPlus, the ensemble achieves Recall@64 of 0.941 ± 0.005, substantially outperforming individual generators. These coverage improvements also translate into consistent downstream performance, with Qwen3.5-27B achieving an accuracy of 0.832 ± 0.0005, confirming that our largest reported downstream gain is robust across seeds. Furthermore, the heuristic aggregation strategy remains close to the gold-aware oracle, with differences of only 0.6 and 0.7 percentage points for the cross-encoder and Qwen3.5-9B, respectively, on GNormPlus. These findings are consistent with those observed on NCBI-Disease. Overall, these additional experiments extend our robustness evaluation beyond NCBI-Disease and demonstrate that the improvements in candidate coverage, downstream accuracy, and the near-oracle performance of our aggregation heuristic remain stable across random seeds, including on the dataset exhibiting our largest downstream improvement.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Revised Manuscript Uploaded With Reviewer Feedback

Dear Reviewers and Area Chair,

We have uploaded a revised version of the manuscript that incorporates all the changes and improvements discussed in our rebuttal responses. The updated PDF addresses the reviewers concerns and includes the corresponding revisions and clarifications.

We sincerely thank the reviewers for their valuable feedback and constructive suggestions, which have helped us improve the quality and clarity of our work.

AuthorsOpenReview ↗

Summary of Contributions and Rebuttal Updates

Dear Area Chair, Program Chair and Reviewers,

We sincerely thank the reviewers for their feedback and would like to briefly highlight the contributions of our revised submission.

Our work establishes new state-of-the-art results on the evaluated biomedical entity linking benchmarks, improving candidate Recall@64 by 6.4 percentage points and downstream linking accuracy by 4.7 points over the strongest evaluated baselines. Beyond these improvements, we provide a systematic investigation of candidate generation, demonstrating the importance of heterogeneous rather than homogeneous ensembles, quantifying generator complementarity and diminishing returns, and showing that a simple round-robin aggregation strategy approaches the performance of a gold-aware oracle. We further demonstrate that a lightweight version of our ensemble beats strong baselines while remaining computationally competitive.

During rebuttal, we conducted additional experiments, including stronger matched baselines, multi-seed robustness evaluations, and efficiency analyses, while addressing the reviewers' concerns and incorporating the corresponding revisions into the manuscript.

We recognize and appreciate the concerns raised during the review process and have made substantial efforts to address them. We sincerely hope that these additional results and clarifications will help inform the final assessment and provide a fuller picture of the paper's contributions.

We believe our findings offer valuable insights and practical benefits to the biomedical entity linking community. Thank you very much for your time and thoughtful consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.