acceptodds
Under review as a conference paper at ICLR 2027

MoRAG: Bringing Graph-Based Retrieval-Augmented Generation to Real Smartphones

Abstract

Graph-based RAG organizes fragmented text into explicit knowledge structures for retrieval through entities, relations, and article summaries. However, directly bringing graph-based RAG to smartphones creates a mismatch between cloud-oriented graph methods and mobile constraints: compact on-device models are more likely to miss entities, predict incorrect relations, or merge distinct entity mentions, while multi-level graph summaries require frequent, large-context LLM invocations. We build MoRAG, to the best of our knowledge, the first graph-based RAG framework end-to-end deployed on commercial smartphones. MoRAG combines MRGraph, a two-layer mobile graph that preserves entities, relations, and source paragraphs; EvoNER, which incrementally learns unseen entity types and locally updates the graph; and ATPlanner, which uses one LLM call to guide hybrid retrieval under a fixed budget. On real smartphones, MoRAG outperforms prior mobile RAG baselines by up to 22.4 points in correctness, constructs knowledge graphs 13.77× faster, and consumes only 1.36% of the energy compared with the latest lightweight methods. **Code & Demo**: https://anonymous.4open.science/r/MoRAG-31B4/.

Primary area:
foundation or frontier models, including LLMs
Keywords:
On-device Retrieval-Augmented Generation, Graph-based RAG, Mobile Large Language Models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_RgbNOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents MoRAG, an on-device graph-based RAG framework designed for resource-constrained smartphones. It introduces three main components: (1) MRGraph, a lightweight two-layer graph structure that preserves entities, relations, and source paragraphs; (2) EvoNER, an incremental entity recognition mechanism for adapting to unseen entity types; and (3) ATPlanner, a single-call LLM planner for efficient hybrid retrieval.

Critical strengths
  1. The paper addresses an important and practical challenge of deploying graph-based RAG systems on resource-constrained smartphones. The proposed framework provides a meaningful step toward enabling efficient and privacy-preserving on-device knowledge retrieval.

  2. MRGraph combines structured entity relations with original paragraph evidence, improving robustness against inaccurate knowledge extraction by compact models. Its lightweight routing index also reduces the overhead associated with traditional hierarchical graph construction.

Critical weaknesses
  1. The paper is primarily motivated by the challenges of deploying graph-based RAG on smartphones, including limited computational resources, energy constraints, and continuously evolving personal knowledge. However, the experimental evaluation does not sufficiently align with this motivation. The main results focus on conventional QA benchmarks, while mobile-specific evaluations are relatively limited. In particular, the paper lacks a unified evaluation of QA quality, latency, memory consumption, and energy efficiency under comparable on-device settings, as well as experiments involving realistic personal knowledge retrieval and continuous updates. Consequently, the experimental evidence does not fully substantiate the paper’s central mobile-oriented claims.

  2. The core components of MoRAG largely build upon established techniques, including heterogeneous knowledge graphs, clustering-based indexing, continual NER, knowledge distillation, and LLM-guided retrieval planning. The paper does not sufficiently clarify its methodological contributions beyond integrating these techniques for mobile deployment.

  3. MoRAG and Naïve RAG are evaluated directly on smartphones, but several other baselines are evaluated in cloud environments under adapted configurations. These differences raise concerns about the fairness and comparability of the reported results, particularly regarding deployment efficiency.

  4. The evaluation of EvoNER lacks direct comparisons with existing continual NER methods and comprehensive measurements of catastrophic forgetting. Additionally, the computational and energy costs of incremental training on smartphones are not sufficiently investigated.

  5. The paper does not thoroughly evaluate how graph size affects memory consumption, indexing overhead, and retrieval latency as the knowledge base grows. Furthermore, the relatively simple entity canonicalization mechanism may introduce incorrect entity merging, while the cost of maintaining the routing index under frequent updates remains unclear.

Questions for authors

See weaknesses above.

Reviewer_uNCbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper presents MoRAG, the first end-to-end graph-based RAG framework deployed on commercial smartphones. To address three key barriers of mobile graph RAG — poor entity/relation extraction by small on-device models, prohibitive cost of LLM-based multi-level graph summarization, and difficulty of incremental graph updates — the framework introduces three components: MRGraph, a lightweight two-layer graph with an entity-paragraph base layer and a routing index layer; EvoNER, an incremental named entity recognizer that learns unseen entity types via amortized LLM supervision and knowledge distillation; and ATPlanner, a single-LLM-call retrieval planner that guides hybrid search under a fixed token budget.

Critical strengths
  1. Pioneering end-to-end on-device graph RAG deployment with convincing real-device performance data, filling an important practical gap in on-device personal assistant development.
  2. Mobile-native design that minimizes LLM invocations, preserves raw text to compensate for extraction errors, and supports incremental updates, distinguishing it from cloud-native graph RAG designs.
  3. Meaningful real-world gains in both accuracy (up to +22.4 points) and system performance (13.77× faster graph construction, 73× lower energy consumption).
Critical weaknesses
  1. Uneven baseline comparison: most competing methods are evaluated in the cloud rather than natively ported to device, which somewhat weakens the persuasiveness of the accuracy gains.
  2. Only two Android phones from one brand are tested, leaving portability across chips, OSes, and memory configurations unclear.
  3. Limited scope of incremental learning evaluation: continuous learning experiments measure only entity surface overlap and downstream F1 at discrete checkpoints, without evaluating long-term catastrophic forgetting or end-to-end system performance under streaming ingestion.
  4. There is little breakdown of which query types MoRAG still fails on, making it hard to tell whether remaining errors stem from entity extraction, retrieval planning, or answer generation.
Questions for authors
  1. Has MoRAG been tested for porting to other mobile platforms (e.g., iOS) or different chip architectures? How much of the performance advantage comes from specific hardware/software optimizations?
  2. Why are most baseline methods evaluated in the cloud rather than implemented natively on-device? Would Naive RAG, as a mobile-native method, serve as a fairer accuracy comparison baseline?
  3. How does MoRAG handle highly nested or domain-specific entity types never seen by EvoNER? Is there an elegant fallback mechanism when entity extraction fails?
  4. The paper states that graph updates account for only 6% of total construction time. Does this ratio remain stable during long-term continuous incremental updates, or does the graph eventually require full rebuild?
Reviewer_ZZxnOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MoRAG, a graph-based RAG framework end-to-end deployed on commercial smartphones. MoRAG combines MRGraph, a two-layer mobile graph that preserves entities, relations, and source paragraphs; EvoNER, which incrementally learns unseen entity types and locally updates the graph; and ATPlanner, which uses one LLM call to guide hybrid retrieval under a fixed budget.

Experiments on four QA benchmarks and two commercial smartphones show up to 22.4 points higher QA correctness than the strongest baseline with the same LLM backbone. Compared with MiniRAG, MoRAG constructs graphs 13.77× faster and consumes only 1.36% of the energy.

The strengths of the paper are its two-layer graph design and its experiments on commercial smartphones. MRGraph supports efficient retrieval while preserving source paragraphs, and the real-device experiments demonstrate practical value. However, the discussion of prior on-device RAG methods is too brief to support the motivation and the first contribution. The experiments also leave insufficient whether EvoNER retains existing-type knowledge while it learns new types. Finally, missing training and retrieval details make the method difficult to reproduce.

Critical strengths
  1. Efficient retrieval with source-grounded evidence. MRGraph combines efficient retrieval with source-grounded evidence. Its two-layer design uses a routing index for fast access and an entity-paragraph graph to preserve entities, relations, and source paragraphs. Runtime grounding restores neighboring paragraphs to reduce context loss.

  2. Practical value of real-smartphone experiments. The authors deploy MoRAG on two commercial smartphones and report results from real-device execution. These experiments provide evidence of on-device feasibility, making the work relevant to practical mobile deployment.

Critical weaknesses
  1. The review of on-device RAG is insufficient. The paper cites several mobile RAG studies but mainly describe mobile RAG as relying on vector retrieval. Their methods and limitations are not discussed in enough detail to explain what remains unresolved. This weakens the motivation for the study. Meanwhile, the limited review raises concerns about overclaiming the first contribution. The proposed assumptions need a more comprehensive discussion of prior work.

  2. Insufficient validation of EvoNER. Table 4 reports NERsurf and QA F1 at 0%, 50%, and 100% of training progress. However, the results do not establish whether EvoNER learns unseen entity types while preserving existing-type knowledge. In addition, Section 3.2 introduces knowledge distillation in Equations (5)–(6) to prevent forgetting, yet no ablation tests whether knowledge distillation prevents forgetting. Meanwhile, Equation (4) defines three stages, but the conditions for moving between training stages and the resource scheduling rules also remain unclear. These gaps limit reproducibility of incremental updates.

  3. Several retrieval designs are insufficiently explained. Section 3.1 introduces ‘symbolic’ in Equation (3) for token efficiency, but the paper does not show how ‘symbolic’ saves tokens if ‘content’ is used. The routing index uses both centroids and summaries. Figure 5(c) removes the whole index, so the effect of summary quality on retrieval remains unclear. The paper also states that summaries for multiple clusters are generated in a single turn, without explaining how the input fits within the context window. In addition, candidate ranking and neighboring-paragraph selection in Section 3.3 also need more detail.

Questions for authors

What kinds of daily questions would benefit most from MoRAG’s compared with other RAG methods, especially when the answer involves information from different apps?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.