- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes LLM4Impact to predict future citation counts using paper text, citation relationships, research domain, and publication time. Specifically, It combines paper text, citation graphs, and language model representations to predict future citation counts for newly published papers. It uses research domain and publication time to weight these sources and calibrate predictions. It also introduces a dataset of two million papers with yearly and monthly prediction targets.
Critical strengths
The dataset is the clearest contribution. Its scale, publication-time citation graphs, and yearly and monthly labels could be useful for future citation prediction research.
Critical weaknesses
-
The practical value of the task is unclear. The paper mentions identifying emerging research and supporting research evaluation, but does not validate either use case. It remains unclear what decisions would improve with more accurate citation predictions.
-
The methodological novelty is limited. The paper combines established text encoders, graph models, and language models, but does not clearly explain what we learn beyond improved prediction scores.
-
The evidence for the core design choices is incomplete. Temporal encoding and adaptive fusion are emphasized, but direct comparisons against simpler alternatives are missing.
-
The paper states that the Qwen models were evaluated via both prompting and fine-tuning, but It seems Tables 2 and 3 report only zero-shot results. A key control is missing. The same Qwen2.5-7B backbone fine-tuned with LoRA on title and abstract alone.
Questions for authors
How do the scope and experiments of this study support its claimed practical uses? Which results show that more accurate citation prediction helps people make better decisions?