- Rating
- 3(+2)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies citation-retrieval failures of text embeddings and proposes neighbourhood citation retrieval, which adds and ranks papers cited by a query’s nearest textual neighbours. Across scientific papers and patents, it measures retrieval gains and their sources, raising MasterSet R@100 from 0.392 to 0.569 without training while testing popularity and organizational effects.
Critical strengths
- The training-free neighbour vote raises MasterSet R@100 from 0.392 to 0.569 with a simple, interpretable mechanism.
- Shared-candidate comparisons and degree-preserving rewiring distinguish useful citation structure from popularity and candidate-pool expansion.
- Temporal, co-author, assignee, and matched-removal controls reveal meaningful differences between scientific retrieval and patent citation recovery.
Critical weaknesses
- Citations and citation communities proxy research agenda, but no independent labels validate shared problems or approaches.
- The broad text-saturation claim exceeds the tested encoders; oracle fitted vectors do not establish limits of trained late-interaction retrieval.
- The 4M-paper corpus contains only 11% of query references on average, so its recall results concern a selective bibliography subset.
Questions for authors
- Can experts label shared problems and approaches for cited and uncited candidates, then evaluate whether voting improves agenda relevance?
- Can you test a trained late-interaction retriever under the same temporal and retrieval-budget constraints, and measure voting’s additional gain?
- Can you expand a query subset to near-complete reference coverage and report full-bibliography recall alongside in-corpus recall?
Follow-up for Reviewer 63eQ: near-complete reference coverage (Q3)
We ran the test you asked for. With near-complete coverage the gain holds, with the same encoder and the same queries. A corpus of about 170M papers, built from OpenAlex, Crossref and Semantic Scholar records, holds 91% of each query's references on average (median 94%), against 11% in the paper's 4M corpus. Over each query's OpenAlex reference list, the vote adds +9.5 R@100 points [+9.0, +10.0] to cosine, and over the references in the corpus +10.5, against +8.0 in the 4M corpus (table). On the 532 queries with at least 95% of their references in the corpus it adds +9.6. So the low coverage of the 4M corpus did not inflate the gain.
Setup. The queries, rules and time constraint are those of the paper corpus (§3, App. B), and the follow-up to Reviewer 18DR gives the scale results of the same run. The text model is the public Qwen3-Embedding-0.6B with exact cosine search, and other versions of a query (the same title, or cosine at least .95) are removed like the query itself. The table uses the 1,168 of the paper's 1,171 queries that both corpora can score. Coverage is the number of a query's references in this corpus's citation graph divided by the length of its OpenAlex reference list. Full-list recall divides by all of the query's references in OpenAlex (or by the in-corpus count when larger, 12 queries), so references outside the corpus count as misses.
At 170M papers, the 1,025 of these queries with at least 80% of their references in the corpus gain +9.6 [+9.1, +10.2] over the full list, and the 532 with at least 95% gain +9.6 [+8.8, +10.4]. We will add these results to App. L.