- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents RAG-Scholar, separating paper-level source identification from local evidence selection. It constructs concise answer contexts using hierarchical retrieval, GNN-fused academic entity representations, and text-guided visual cropping. The research problem is well-motivated: redundant topically relevant content in scholarly literature often occupies limited context space and displaces valuable evidence. The authors demonstrate promising quality–context length tradeoffs and supply thorough implementation and cost details.
Critical strengths
-
Separating paper retrieval and evidence selection is a reasonable design. For queries constrained by affiliations, methods, or datasets, fragment-level similarity alone cannot meet source constraints.
-
RAG-Scholar yields a 16.11-point Fact F1 gain over CompAct in Table 1, and feature fusion brings a 9.27-point Fact F1 improvement over the explicit feature branch in Table 3. These findings indicate the full system is worthy of research.
Critical weaknesses
-
The dataset is small: 146 core papers (max 597 in extensions), and fact evaluation uses only 24 text questions and 15 visual questions. The limited question distribution and narrow focus on RAG/graph-related papers support prototype testing but fail to validate generalization to large scholarly corpora or general academic evidence synthesis.
-
The paper’s core motivation states that independent relevance ranking preserves redundant content and displaces complementary evidence. Nevertheless, its text-centric designs, such as hierarchical filtering, source limits, graph-augmented ranking fusion, and maximum per-paper support scoring, only reduce source concentration. They do not directly measure whether a new fragment brings incremental answer-relevant information over previously selected evidence.
Questions for authors
See Weakness