- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents MoRAG, an on-device graph-based RAG framework designed for resource-constrained smartphones. It introduces three main components: (1) MRGraph, a lightweight two-layer graph structure that preserves entities, relations, and source paragraphs; (2) EvoNER, an incremental entity recognition mechanism for adapting to unseen entity types; and (3) ATPlanner, a single-call LLM planner for efficient hybrid retrieval.
Critical strengths
-
The paper addresses an important and practical challenge of deploying graph-based RAG systems on resource-constrained smartphones. The proposed framework provides a meaningful step toward enabling efficient and privacy-preserving on-device knowledge retrieval.
-
MRGraph combines structured entity relations with original paragraph evidence, improving robustness against inaccurate knowledge extraction by compact models. Its lightweight routing index also reduces the overhead associated with traditional hierarchical graph construction.
Critical weaknesses
-
The paper is primarily motivated by the challenges of deploying graph-based RAG on smartphones, including limited computational resources, energy constraints, and continuously evolving personal knowledge. However, the experimental evaluation does not sufficiently align with this motivation. The main results focus on conventional QA benchmarks, while mobile-specific evaluations are relatively limited. In particular, the paper lacks a unified evaluation of QA quality, latency, memory consumption, and energy efficiency under comparable on-device settings, as well as experiments involving realistic personal knowledge retrieval and continuous updates. Consequently, the experimental evidence does not fully substantiate the paper’s central mobile-oriented claims.
-
The core components of MoRAG largely build upon established techniques, including heterogeneous knowledge graphs, clustering-based indexing, continual NER, knowledge distillation, and LLM-guided retrieval planning. The paper does not sufficiently clarify its methodological contributions beyond integrating these techniques for mobile deployment.
-
MoRAG and Naïve RAG are evaluated directly on smartphones, but several other baselines are evaluated in cloud environments under adapted configurations. These differences raise concerns about the fairness and comparability of the reported results, particularly regarding deployment efficiency.
-
The evaluation of EvoNER lacks direct comparisons with existing continual NER methods and comprehensive measurements of catastrophic forgetting. Additionally, the computational and energy costs of incremental training on smartphones are not sufficiently investigated.
-
The paper does not thoroughly evaluate how graph size affects memory consumption, indexing overhead, and retrieval latency as the knowledge base grows. Furthermore, the relatively simple entity canonicalization mechanism may introduce incorrect entity merging, while the cost of maintaining the routing index under frequent updates remains unclear.
Questions for authors
See weaknesses above.