- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates the limitations of candidate generation in biomedical entity linking, highlighting how incomplete candidate retrieval constrains downstream disambiguation performance. To address this bottleneck, the authors propose a heterogeneous ensemble framework that combines multiple candidate generators with complementary retrieval strategies, including lexical, dense, and autoregressive methods. Using a simple round-robin aggregation strategy, the framework improves candidate coverage across six biomedical benchmarks without increasing the candidate set size. When combined with cross-encoders and LLM-based disambiguators, the approach achieves consistent improvements in end-to-end accuracy, with an average gain of 8.1 percentage points over the best evaluated baselines. The paper further demonstrates that retrieval diversity is crucial for improving coverage, while additional generators yield diminishing returns, and simple aggregation performs comparably to an oracle strategy.
Critical strengths
-
Clear Writing and Organization. The paper is well-written and clearly structured, making it easy to follow the motivation, proposed methodology, and key findings. The presentation is generally clear, and the figures effectively illustrate the proposed framework.
-
Comprehensive Experimental Evaluation. The paper provides extensive experiments across six biomedical entity linking benchmarks. The authors conduct thorough ablation studies and analyses of model diversity, candidate overlap, aggregation strategies, and computational efficiency, which strengthen the empirical findings and support the main conclusions.
Critical weaknesses
-
The proposed framework primarily combines existing candidate generation models using a simple round-robin aggregation strategy. While the empirical improvements are promising, the core methodology is relatively straightforward, and similar ensemble-based retrieval approaches have been explored in prior work [1, 2]. The paper could better clarify its methodological contributions and distinguish its approach from existing methods.
-
The paper’s central claim is that candidate generation is inherently incomplete under any single retrieval paradigm. However, the experiments only demonstrate that combining several existing retrieval models improves candidate coverage. This is an expected consequence of combining models with partially overlapping predictions and does not establish any fundamental limitation of individual retrieval paradigms. The paper lacks a theoretical analysis or controlled experiments demonstrating that a sufficiently strong single retriever cannot achieve comparable coverage. Therefore, the empirical evidence does not adequately support the paper’s broader claim about the inherent limitations of candidate generation.
-
Several major conclusions of the paper appear intuitive rather than scientifically surprising. For example, combining diverse retrieval models increases candidate coverage, models with different architectures retrieve complementary candidates, and adding more models eventually produces diminishing returns. These observations are well-established principles in ensemble learning and information retrieval. Although the authors provide extensive empirical evidence, it is unclear what fundamentally new insights the experiments offer beyond confirming these existing intuitions in the biomedical entity linking domain.
-
Although the paper emphasizes computational efficiency, its full framework requires running five candidate generators, including computationally expensive neural and autoregressive models. This introduces additional inference latency, memory requirements, and deployment complexity compared with a single retriever. The lightweight ensemble experiment provides some evidence of efficiency, but it compares against a computationally expensive baseline rather than systematically evaluating the accuracy–latency trade-off across different ensemble configurations. Consequently, the practical advantages of the proposed framework remain insufficiently established, particularly for latency-sensitive applications.
[1] Sung, Mujeen, et al. "Biomedical entity representations with synonym marginalization." Proceedings of the 58th annual meeting of the association for computational linguistics. 2020.
[2] Borchert, Florian, et al. "xMEN: a modular toolkit for cross-lingual medical entity normalization." JAMIA open 8.1 (2025): ooae147.
Questions for authors
N/A
Rebuttal by Authors (6)
We have now completed the multi-seed experiments to address Weakness 6.
To address this concern, we extended our multi-seed experiments to GNormPlus as requested. We repeated the experiments with 2 additional random seeds, for a total of 3 runs, covering candidate generation, downstream disambiguation and heuristic-versus-oracle comparisons. This allows us to assess the stability of the entire pipeline beyond repeated greedy LLM inference.
Table 18: Mean and standard deviation of Recall@k over three runs on GNormPlus and NCBI-Disease. We report individual generators whose outputs varied across seeds; SapBERT produced identical results across all runs.
Table 19: Mean and standard deviation of end-to-end accuracy over three runs on GNormPlus and NCBI-Disease.
Table 20: Mean and standard deviation over three runs comparing heuristic and oracle candidate aggregation on GNormPlus and NCBI-Disease.
The results demonstrate that our heterogeneous ensemble maintains strong and stable performance across random seeds on both NCBI-Disease and GNormPlus. In particular, on GNormPlus, the ensemble achieves Recall@64 of 0.941 ± 0.005, substantially outperforming individual generators. These coverage improvements also translate into consistent downstream performance, with Qwen3.5-27B achieving an accuracy of 0.832 ± 0.0005, confirming that our largest reported downstream gain is robust across seeds. Furthermore, the heuristic aggregation strategy remains close to the gold-aware oracle, with differences of only 0.6 and 0.7 percentage points for the cross-encoder and Qwen3.5-9B, respectively, on GNormPlus. These findings are consistent with those observed on NCBI-Disease. Overall, these additional experiments extend our robustness evaluation beyond NCBI-Disease and demonstrate that the improvements in candidate coverage, downstream accuracy, and the near-oracle performance of our aggregation heuristic remain stable across random seeds, including on the dataset exhibiting our largest downstream improvement.