- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes CompassRAG, a training-free multi-turn GraphRAG framework that keeps a source-linked triple memory (DEG) across turns, folds old retrieval text into it, and reweights graph edges by the current query before PPR, with gains reported on MuSiQue, 2Wiki and HotpotQA.
Critical strengths
The method beats a fairly broad set of single-turn, RL-based and learning-free multi-turn baselines on all three benchmarks, and the ablations cover most components.
Critical weaknesses
The “w/o DEG” variant already beats the strongest multi-turn baseline, so a good part of the gain seems to come from the authors’ own reasoning loop and prompt rather than from the DEG or the retriever, and this isn’t separated out. Most component gains are around one point or less, there are no variance or significance numbers, and each test set has only 1,000 questions, so it’s hard to tell which gains are real. Questions take only about 1.5 to 2.4 turns on average, so the context growth problem that memory folding is meant to solve barely shows up, and there are no end-to-end token or cost numbers to back the efficiency story. The novelty is moderate: query-conditioned edge weighting is close to QAFD-RAG, structured working memory for agentic RAG isn’t new, and everything is tested with a single 70B reasoner.
Questions for authors
ightRAG gets close to zero EM on all three datasets, which looks like a setup problem. How was it configured, and is the comparison fair?
Dear Reviewer wAcp,
Thank you again for your thoughtful review. As the discussion period ends today, we would greatly appreciate it if you could take a moment to review our detailed rebuttal and revised manuscript.
We have carefully addressed your concerns through additional same-scaffold comparisons across three retrieval backbones, repeated runs demonstrating low performance variance, and end-to-end token-efficiency analysis. We have also clarified our methodological differences from QAFD-RAG, the role of DEG in evidence management, and the source and evaluation settings of the LightRAG results. We hope these new results help resolve your concerns and would be glad to discuss any remaining questions before the deadline.
Thank you very much for your time and consideration.