acceptodds
Under review as a conference paper at ICLR 2027

Guess First, Correct the Rest: Graph Speculative Retrieval for Multi-hop Reasoning

Abstract

Retrieval-augmented generation (RAG) has been widely employed to reduce hallucinations in large language models (LLMs) by integrating external knowledge. However, traditional RAG systems struggle with fragmented knowledge in large-scale corpora, rendering them inadequate for multi-hop reasoning tasks. Although graph-enhanced RAG (GraphRAG) systems leverage graph structures to capture more knowledge associations, they still suffer from low-quality graphs, leading to two key limitations: (1) Redundant retrieval results: Excessive noise in the graph makes relevant evidence hard to reach. (2) Biased retrieval paths: The rigid graph structure fails to dynamically match the reasoning process. This is essentially because existing GraphRAG systems struggle to overcome the quality bottleneck of graph construction, thus requiring a flexible retrieval strategy to reduce blind graph traversal. Motivated by this, we propose G-Speculator, which tentatively speculates clues on the graph to guide multi-hop retrieval. Specifically, it consists of two components: (1) Hypothetical Knowledge Graph Construction: We use a lightweight model to serialize passages into entity-relation sequences and construct a graph (HypoKG), which preserves contextual word order as a linguistic prior for speculative retrieval; (2) Speculation-driven Passage Retrieval: We continuously locate clues and speculate bridge entities on HypoKG to obtain multi-hop evidence, while using chain-of-thought (CoT) to assess evidence sufficiency and adjust retrieval direction. Extensive experiments on four datasets demonstrate that G-Speculator outperforms state-of-the-art baseline methods, validating its practicality for real-world applications. Our code and datasets are available at https://anonymous.4open.science/r/G-Speculator.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Retrieval Augmented Generation, Graph Retrieval Augmented Generation, Large Language Model

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_9dMCOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

G-Speculator targets two GraphRAG failure modes that the authors attribute to low-quality graphs: redundant retrieval results (global traversal over a noisy graph returns many irrelevant passages) and biased retrieval paths (a static graph cannot adapt when relations are missing). It indexes the corpus with spaCy only. Each sentence is split at named entities into an entity–relation sequence, and the non-entity spans become hyperedges that connect all entities of that sentence (HypoKG). At query time, an LLM parses the question into triples with placeholders. Clues (entity, relation span) are scored by entity-linking confidence × IDF plus relation similarity, and the bridge entity is "speculated" as the nearest entity on the opposite side of the relation span in the sentence sequence. A CoT step then judges whether the evidence is sufficient and either answers or issues a new sub-query, with dense retrieval as a fallback when retrieval stalls. The method reports the best average F1 on HotpotQA, 2Wiki, MuSiQue, and ScaleQA, with fewer retrieved passages and zero LLM tokens for indexing.

Replacing exhaustive traversal with cheap local guessing plus LLM correction is a practical idea, and the results are strong. However, the efficiency claim covers only the retriever, while the method makes more LLM calls than the strongest baseline. The speculation rule is a positional heuristic that relies on undocumented components (notably prepending passage-title entities to every sentence) and has no handling of pronouns or ambiguous relations. HypoKG is never shown to fix the reachability problem used as motivation. The delta over KG-Translator is modest, a closely related method (T2RAG) is missing, and the evaluation is limited to Wikipedia-based multi-hop QA.

Critical strengths
  1. The preliminary study is a useful diagnostic. It quantifies evidence reachability on OpenIE-built KGs (second-evidence reachability drops to 55.0–66.8%) and illustrates the two failure cases concretely.
  2. Indexing is cheap and simple: spaCy only, zero LLM tokens, and about 226 s per 5k passages, vs. 4.30M/1.28M input/output tokens for OpenIE-based HippoRAG. Graph retrieval itself is fast (0.56 s retriever time, +0.13 s when the corpus grows 3.5×).
  3. Results are strong and the retrieved context is concise: best average F1 on all four datasets (overall 74.22 vs. 68.45 for KG-Translator), with about 2.9 retrieved passages and a Precision of 82.54 vs. 63.17 on 2Wiki.
  4. The appendix is thorough (hyperparameters, LLM backbones, entity linkers, case and error studies, component-wise latency), and code is released.
Critical weaknesses
  1. Efficiency is overclaimed.
    • Table 3 reports indexing cost and retriever runtime, but online cost is dominated by LLM calls. G-Speculator makes about 3.91 LLM calls per query (R 1.72 + I 1.98 + A 0.21). That is about 60% more than KG-Translator (2.45) and lower only than HippoRAG+IRCoT (5.34).
    • Table 5 shows that each round is dominated by query-graph construction (≈1.46 s) and CoT analysis (≈2.72 s), while clue exploration and speculation take ≈0.11 s. End-to-end per-query latency and online input/output tokens are not reported for any method.
    • According to Algorithm 2, every round parses the query graph and then runs CoT, so R (1.72) and I (1.98) should be equal. Please explain the difference.
    • The generative entity linker (Llama-3.2-1B trained on KG-Translator's data and run on a GPU) is not included in the cost comparison.
  2. The speculation rule is a positional heuristic, and key components are undocumented.
    • The stated assumption is that subject and object are directly adjacent to the predicate, but neither Eq. 4 nor the code enforces adjacency. A clue pairs any entity with any relation span in the same sentence (ϕ(r) = E_s). Two examples from the paper:
      • Case 1: "Aerodynamik" is separated from "from their tenth studio album" by two other elements.
      • Case 2: "1173" lies between "Richardis of Bavaria" and "in".
    • Pronouns are not handled at indexing time. They are not named entities, so they are absorbed into relation spans ("It is endemic to", "He married"). The only coreference step is an instruction in the CoT prompt, applied after retrieval.
    • Instead, the released code (modules/hypokg.py) prepends the passage-title entities to every sentence sequence (sid2seq[sid] = title_ents + sent_sequence) and attaches them to all hyperedges of every sentence. This is why Cases 2 and 4 start with "Otto I" and "Emu" before "He married" and "It is endemic to".
      • In effect, it is an implicit rule that a pronoun refers to the title entity, and it is not described in Sec. 3.1, Algorithm 1, or Appendix F.1.
      • It attaches the wrong subject whenever a pronoun refers to a non-title entity or a sentence has a different explicit subject. The wrong bridge "Jülich" in Case 2 is itself a prepended title entity: p8 is "William IV, Count of Jülich", and "Jülich" does not occur in that sentence.
      • It assumes titled, entity-centric passages.
      • When a title contains several entities, their order comes from a set, so positions are not deterministic across runs.
      • An ablation of this component is needed.
    • Relation spans are untyped, and function words are not filtered.
      • A span such as "in" gets the same similarity score whether it is temporal or locative.
      • Identical spans within a sentence share one relation id, so speculation always uses the first occurrence. For example, in "(15 April 1452 in Vinci – 2 May 1519 in Amboise)", the second "in" can never yield Amboise.
      • Entity types are only used as a binary named vs. value-like filter on non-final hops. The type implied by the query predicate (e.g., "born location is") is not used.
  3. HypoKG is not shown to address the motivating problem.
    • The reachability analysis (Fig. 2a) is computed on OpenIE KGs, never on HypoKG.
    • HypoKG's hyperedges are intra-sentence, so links across sentences still depend on exact entity-string matches (e.g., "Richardis" vs. "Richardis of Bavaria" in Case 2).
    • As it stands, unreachable evidence appears to be recovered by LLM query rewriting and the dense fallback rather than by the graph. Please report reachability on HypoKG under the same protocol.
  4. Novelty is limited relative to prior work, and closely related methods are missing.
    • Much of the pipeline follows KG-Translator: spaCy-based hypergraph construction, the same generative entity linker and its training data, the same test splits, and ScaleQA. The main additions are keeping word order, the positional speculation rule.
    • Missing comparison with T2RAG (Table 1, multi-step methods, p. 7). T2RAG also decomposes the question into triplets with placeholders and resolves them iteratively. It explicitly targets the efficiency of multi-round RAG and reports better results than IRCoT. This is very close to the query-graph construction and sequential placeholder filling used here, so it should be discussed and compared in both accuracy and cost.
    • Missing comparison with HyperGraphRAG. HypoKG is a hypergraph, yet the paper neither discusses nor compares with HyperGraphRAG, which also uses hyperedges over multiple entities (n-ary relational facts) for RAG. The paper should explain how HypoKG differs, since HyperGraphRAG's hyperedges are n-ary facts extracted by an LLM while HypoKG's are spaCy sentence-level spans, and how their retrieval strategies differ. It should also compare the two in accuracy and cost.
  5. The evaluation scope is narrow.
    • All four datasets are Wikipedia-based multi-hop QA (e.g., p. 24), and ScaleQA merges the other three. Their passages are short, titled, and entity-centric, which favors spaCy NER and the title-entity heuristic.
    • HotpotQA, 2Wiki, and MuSiQue alone are no longer sufficient for current RAG research. Please evaluate on GraphRAG-Bench (Xiao et al., 2025), which tests domain-specific reasoning over 20 textbooks across 16 disciplines.
    • The GraphRAG-Bench used in Appendix D.2 is a different benchmark (Xiang et al., 2026). Even there, the paper reports only contextual summarization and creative generation and compares only three baselines. Fact retrieval and complex reasoning, the two tasks closest to the paper's claims, are omitted.
    • Precision favors methods that return fewer passages, while all baselines are fixed at top-5. Recall is reported only for MuSiQue in the multi-step setting (Fig. 7).
    • The conciseness claim does not hold everywhere. On MuSiQue in the multi-step setting, G-Speculator retrieves 5.59 passages on average (more than top-5), and Precision drops from 45.80 to 37.81 as the number of steps grows.

Minor Weaknesses

  • Obs. 16 (p. 23) claims G-Speculator is "insensitive to the choice of sentence embedding model", but there is no reference point. Adding Vanilla RAG with the same three embedding models would show whether the stability comes from speculative retrieval or from embeddings simply mattering little on these datasets.
  • Sec. 4.1 says all methods use the same LLM (GPT-4o-mini), but Search-R1 and R1-Searcher are RL-trained policy models. Please clarify how they were run.
  • Obs. 5 says triple extraction costs "up to 4.37×10⁶" input tokens, but Table 3 lists at most 4.30 for HippoRAG and GFM-RAG.
  • The formal definition in Sec. 3.1 (R = ∪_s R_s with ϕ(r) = E_s) is ambiguous when the same span appears in several sentences. The code uses per-sentence relation ids.
  • The case studies are inconsistent. In Case 2, the clue entity is "Otto I, Count of Guelders", but the sequence contains "Otto I". In Case 3, clue c2 uses "United States of America", which does not appear in s532 (it contains "American").
  • The AI-use statement says generative AI was used to "generate synthetic datasets for the preliminary study", while Appendix E describes LLM-extracted evidence chains from PER-QA. Please clarify what was synthesized and how it was validated.
  • The "guess first, correct the rest" framing invites comparison with speculative decoding, but the analogy is loose. The correction here is an unverified LLM judgment (Case 4 accepts "Australia" as sufficient), and it costs extra sequential LLM calls rather than a cheap parallel verification.

References

  • Gong et al., "Beyond Chunks and Graphs: Retrieval-Augmented Generation through Triplet-Driven Thinking" (T2RAG), Findings of ACL 2026 / arXiv:2508.02435: placeholder-triplet decomposition with iterative resolution, efficiency-oriented, compared against IRCoT.
  • Luo et al., "HyperGraphRAG: Retrieval-Augmented Generation via Hypergraph-Structured Knowledge Representation," NeurIPS 2025: hypergraph-based RAG with n-ary relational facts as hyperedges.
  • Xiao et al., "GraphRAG-Bench: Challenging Domain-Specific Reasoning for Evaluating Graph Retrieval-Augmented Generation," arXiv:2506.02404, 2025: a domain-specific benchmark built from 20 textbooks across 16 disciplines.
  • Otmazgin et al., "F-coref: Fast, Accurate and Easy to Use Coreference Resolution," AACL-IJCNLP 2022 (System Demonstrations): a lightweight coreference resolver that could replace the title-entity heuristic.
Questions for authors

Questions

  1. What are the end-to-end per-query latency and online LLM token counts of G-Speculator, KG-Translator, HippoRAG+IRCoT, T2RAG, and HyperGraphRAG? Why do R and I differ in Table 3?
  2. What is evidence reachability on HypoKG under the protocol of Sec. 2?
  3. How much does prepending title entities contribute (ablation)? How does the method behave on corpora without passage titles, or when a pronoun refers to a non-title entity?
  4. How are ambiguous or repeated relation spans (e.g., two "in"s in one sentence) handled? Why not constrain the speculated entity using the predicate's expected type and spaCy labels?
9 comments

Dear Reviewer 9dMC,

We sincerely appreciate the time and efforts you have devoted to identifying potential issues in our paper. We are also grateful for the opportunity to further clarify the design motivations and experimental details of our framework, which helps us refine the work.


W1 & Q1: Regarding the efficiency

  • (1) End-to-end per-query latency and online input/output tokens

Thanks for your thoughtful suggestions. Following your suggestion, we supplement online LLM token counts and per-query latency to further evaluate the efficiency and cost of each method.

Table 1: Online LLM token counts, per-query latency (s), and F1 on 2Wiki.

MethodInput tokenOutput tokenTotal tokenPer-query latencyF1
HyperGraphRAG25707.18169.2225876.405.3056.07
GFM-RAG+IRCoT3640.64127.183767.828.4669.36
HippoRAG+IRCoT3062.4890.583153.068.5576.96
T2RAG3084.98305.023390.005.7977.61
KG-Translator2026.18104.352130.537.8078.52
G-Speculator (ours)2008.79297.432306.226.7787.54

As shown in Table 1, G-Speculator delivers notable performance improvements with competitive token cost and latency. Compared to KG-Translator, although our method makes more API calls, it uses 8.25% more total tokens due to differences in prompt design, while reducing latency by 13.21% and improving F1 by 9.02%.

  • (2) API calls for query graph and CoT

Thanks for the focused comments. The R-stage API calls are activated only by query graph construction, while the I-stage calls are activated by graph speculative retrieval or evidence expansion. G-Speculator reuses CoT to assess evidence sufficiency after expansion, accounting for the difference between I-stage (1.98) and R-stage (1.72) calls. We have supplemented this step in Algorithm 2 in the revised paper.

  • (3) Training cost of the entity linking model

Thank you for the critical feedback. The entity linking model has modest training overhead, requiring only 197.33 seconds on a single RTX 5090 GPU.


W2 & Q3 & Q4: Regarding further explanation of speculation rule

  • (1) Clarification of the speculation assumption

Thank you for the detailed review. Our linguistic assumption only applies to the speculation phase: given a clue (e′,r′)(e', r'), we take the nearest entity on the side of r′r' opposite to e′e' as the speculated entity e∗e^*, which is adjacent to r′r'. Notably, the explored clue (e′,r′)(e', r') does not require adjacency, it merely serves as an anchor for the retrieval direction, and can be any entity-relation pair in the same sentence.

  • (2) Explanation of pronoun handling and title entity prepending

Thank you for the constructive suggestions. We demonstrate G-Speculator's effectiveness on G-Bench (Novel/Medical) in Appendix D.2, which adopts an untitled setting. In addition, title prepending is a common configuration for sentence-level retrieval (e.g., LinearRAG, KG-Translator). This is because sentences after splitting often start with pronouns, requiring title entities for pronoun disambiguation. Following your suggestion, we provide additional ablation on title entities.

Table 2: Ablation study of title prepending on three multi-hop datasets.

VariantHotpotQA2WikiMuSiQue
Full Model82.1387.5448.92
w/o title prepending79.0584.7546.65

As shown in Table 2, the ablation validates the effectiveness of title prepending. Notably, G-Speculator remains competitive even without title prepending, indicating that the core gains stem from the speculative retrieval mechanism.

  • (3) Clarification on relation spans

Thanks for your expertise. Each relation span in a sentence is assigned a distinct id based on its position during serialization. Furthermore, during graph speculative retrieval, even when relation predicates are ambiguous or repeated, we ensure precise retrieval through the following mechanisms:

  1. Multi-candidate exploration for comprehensive clues. Hyperparameter α\alpha (set to 3) controls clue exploration width, such that even the same relation spans yield multiple candidate paths rather than stopping at the first occurrence.

  2. Clue trajectory scoring for implicit correction. Multiple clues form a trajectory π\pi, where ω(π)\omega(\pi) is jointly determined by every clue's quality. Any weak clue drags down the overall ranking, allowing high-quality trajectories to naturally emerge.

  3. Relation adjacency position implies expected entity types. The subject-predicate-object adjacency assumption reflects predicate semantics, where the entity position implies the expected type (e.g., a person after "married").

W3 & Q2: Regarding the evidence reachability of HypoKG

Thanks for your insightful comments. Following your suggestion, we analyze evidence reachability on HypoKG and compare it against the OpenIE-based KG.

Table 3: Evidence reachability on MuSiQue.

GraphE1 (1-hop)E1 (2-hop)E1 (3-hop)E1 (unreachable)E2 (1-hop)E2 (2-hop)E2 (3-hop)E2 (unreachable)Overall
OpenIE-based KG64.07.64.623.831.219.04.845.055.0
HypoKG69.06.42.022.647.69.85.237.462.6 (+7.6%)

As shown in Table 3, HypoKG effectively improves evidence reachability, which is reflected in two aspects: (1) complete evidence chains are more reachable (+7.6%), giving G-Speculator access to broader evidence; (2) evidence is reached at earlier hops (16.4% higher 1-hop reachability for E2), simplifying the retrieval process.


W4: Regarding the comparison with additional baselines

Thanks for your expert feedback. Following your suggestion, we add T2RAG and HyperGraphRAG as additional baselines.

Table 4: Supplementary baseline experiments.

MethodHotpotQA2WikiMuSiQue
HyperGraphRAG75.0956.0736.92
T2RAG77.1177.6137.43
G-Speculator82.1387.5448.92

As shown in Table 4, G-Speculator maintains the best performance among baselines. Cost analysis is reported in W1 & Q1 (1).

Furthermore, these methods still suffer from redundant retrieval results (Section 2.2, Case 1) and biased retrieval paths (Section 2.2, Case 2), due to:

  • Undifferentiated traversal wastes computation on irrelevant nodes and introduces redundant evidence;
  • Missing relations or noise can divert traversal from correct reasoning paths.

Motivated by this, we propose a novel speculation paradigm. Its core idea draws from human reasoning in complex tasks: rather than exhaustively reviewing all materials, humans first form a tentative judgment and then seek targeted evidence to verify it.

  • We construct HypoKG, which serializes passages into ordered entity-relation sequences preserving contextual word order. Unlike graphs that serve merely as passive knowledge structures, HypoKG actively provides linguistic priors to avoid blind traversal.
  • On top of HypoKG, we propose G-Speculator, which first tentatively speculates evidence chains based on a linguistic assumption, then verifies and corrects them through CoT-guided analysis, precisely identifying and fully comprehending core evidence.

In addition, we also provide detailed explanations in the FAQs section (Appendix A.1 and A.3).

W5: Regarding the comparison across a broader evaluation scope

  • Evaluation on additional benchmarks

Thanks for your helpful suggestions. We discuss the selection of multi-hop datasets in FAQs (Appendix A.4) and present the cross-domain performance in Appendix D.2. Following your suggestion, we further evaluate on additional cross-domain benchmarks and tasks.

Table 5: Accuracy (ACC) on GraphRAG-Bench (CS).

MethodACC
HippoRAG72.6
LinearRAG71.5
KG-Translator71.9
G-Speculator74.0

Table 6: Accuracy (ACC) on GraphRAG-Bench (Medical & Novel)'s Fact Retrieval (FR) and Complex Reasoning (CR) tasks.

MethodMedical (FR)Medical (CR)Novel (FR)Novel (CR)
HippoRAG+IRCoT58.855.248.237.1
KG-Translator42.742.138.930.9
G-Speculator58.958.247.638.1

As shown in Tables 5 and 6, G-Speculator generalizes well across different domains and tasks.

  • Detailed retrieval performance

Thank you for the constructive feedback. Following your suggestion, we report more comprehensive retrieval performance across question types.

Table 7: Retrieval quality report. We report Precision (left) and Recall (right).

Method2Wiki Br.2H^{\text{2H}}2Wiki Cp.2H^{\text{2H}}2Wiki In.2H^{\text{2H}}2Wiki Cp.4H^{\text{4H}}2Wiki Bc.4H^{\text{4H}}MuSiQue Br.2H^{\text{2H}}MuSiQue Br.3H^{\text{3H}}MuSiQue Br.4H^{\text{4H}}
IRCoT29.6 | 74.032.7 | 81.828.5 | 71.323.8 | 59.647.0 | 58.827.1 | 67.731.5 | 52.527.8 | 34.7
Search-R131.1 | 77.733.8 | 84.528.2 | 70.429.4 | 73.651.4 | 64.229.0 | 72.625.4 | 42.320.6 | 25.8
R1-Searcher33.2 | 83.036.8 | 92.031.9 | 79.833.8 | 84.440.8 | 51.029.4 | 73.626.8 | 44.724.9 | 31.2
HippoRAG+IRCoT37.7 | 94.339.2 | 98.133.3 | 83.339.8 | 99.675.4 | 94.327.5 | 68.731.6 | 52.731.3 | 39.1
GFM-RAG+IRCoT35.4 | 88.636.0 | 90.032.1 | 80.336.6 | 91.662.4 | 78.121.6 | 53.923.5 | 39.225.7 | 32.1
KG-Translator58.6 | 95.968.1 | 95.350.5 | 90.572.2 | 96.873.2 | 87.131.6 | 68.729.8 | 47.726.5 | 31.6
G-Speculator81.5 | 92.593.7 | 95.374.9 | 84.990.9 | 96.878.0 | 90.146.3 | 67.134.8 | 54.930.3 | 39.9

As shown in Table 7, G-Speculator effectively retrieves the required evidence while filtering out redundant passages. This reduced noise allows the QA model to focus on the key evidence during path correction, leading to improved QA results.

Reviewer_DkykOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the problem of multi-hop proof exploration during graph search and generation, with a particular focus on issues such as information loss, redundant searches, and search path deviations that arise during graph construction. The proposed G-Speculator system utilizes lightweight named entity recognition and statement serialization techniques to construct HypoKG, a hyper-tool based on hypothetical knowledge. This method preserves the original order between entities and text fragments while constructing the graph. Subsequently, the system decomposes the problem into three query items and derives bridge entities based on entity indices and the relative positions of relationships, thereby enabling local multi-hop proof search. Furthermore, the system employs a correction module based on large language models (LLMs) to evaluate the sufficiency of proofs, directly generating responses or the next query, and resuming the retrieval process that had stalled due to intensive searching. Experimental evaluations covered datasets such as HotpotQA, 2WikiMultihopQA, MuSiQue, and ScaleQA, as well as component absorption experiments, sensitivity analysis, model substitution tests, cross-domain validation, and efficiency analysis. Under the reported experimental conditions, compared to graph-building methods that rely directly on generative LLM models, the proposed method significantly improves answer F1 scores and retrieval accuracy while reducing indexing costs.

Critical strengths
  1. HypoKG preserves the spacing between input items and the original text, as well as their spatial relationships, thereby enabling local language indexing without relying entirely on explicit relation extraction or global propagation across the entire graph. The evidence we seek can still be traced back to the source text without invoking the large language model (LLM) generated to construct the graph. This is particularly important because the method introduces a viable search mechanism with clear and independent biases, while simultaneously reducing, to some extent, the excessive indexing workload associated with GraphRAG methods.

  2. The evaluation includes various types of questions, such as two- to four-step reasoning, dataset expansion, and various reference tasks based on RAG and GraphRAG. Table 1 shows that the overall average F1 score is 74.22, while KG-Translator achieves only 68.45. The appendix also provides MuSiQue’s re-identification rates, cross-domain testing, model replacement, and component-level timestamp results. Compared to evaluations focused solely on a single target task, these results offer a more practical perspective and better reflect the system’s overall performance.

Critical weaknesses
  1. Appendix F.2/J states that evidence accumulated based on multi-stage baselines will be re-evaluated to determine the top five. However, G-Speculator collects this evidence and uses CoT reasoning to have the correction module derive the final answer. Table 6 shows that on the MuSiQue dataset, Level 4 questions have an average of 5.59 contexts per question, while Level 4 questions have an average of 6.80 contexts. Therefore, the shared final prompt cannot guarantee either the similarity of evidence constraints or the consistency of the answer generation process. The significance of this difference becomes evident when conducting simple ablation experiments. On the 2Wiki dataset, the search accuracy of the error-free method and the error-containing method was nearly identical, at 81.49% and 81.44%, respectively, but there was a significant difference in F1 scores, at 70.80% and 75.05% (Tables 14 and 17). This indicates that even when the results from the two methods are similar, reasoning at the answer level still plays a crucial role. Although the additional researcher comparisons in the appendix are valuable for reference, they cannot substitute for a systematic review of these factors in the main evaluation. This issue directly affects our ability to determine whether the reported advantages stem from the proposed research mechanisms.

  2. In the pilot study, entity paths exceeding three steps were considered unreachable; if the first evidence item was inaccessible, the second was also considered inaccessible (Appendix A.2/E). This choice helps diagnose the limitations of specific OpenIE graphs or sequential search protocols, but it does not establish universal performance limits for GraphRAG systems. Systems incorporating semantic links, paragraph retrieval, or hybrid retrieval may not be subject to the same constraints. Conversely, reductions in IDF and reasoning depth provide preliminary support for this approach. However, backward reasoning primarily has a negative effect and offers no additional value for belief rules capable of reliably selecting alternative entity strategies. Furthermore, even when the correction function is removed, iterative retrieval and response reasoning undergo changes. Therefore, although the existing evidence further enhances the system’s overall effectiveness, it remains difficult to explain which factors make specific reasoning mechanisms effective, or which factors contribute more significantly than the designs of existing iterative studies. This largely limits the scope of application for the new reliable knowledge proposed in this paper.

Questions for authors
  1. How were the final top five answers selected based on the evidence accumulated under the multi-stage reference benchmark? What query and evaluation functions were used, and how was duplicate content handled? Which parts of the answers provided by G-Speculator were generated directly by the correction module, and can these answers be applied to shared Q&A prompts? Additionally, please specify whether the additional comparisons among researchers were conducted using native generation modes or through a unified external Q&A model.

  2. Comparison with major competitors such as KG-Translator or HippoRAG+IRCoT. We used the same answer model and reasoning prompts, setting comparable context token budgets and LLM reasoning budgets. While retaining the evidence database, ensure that F1 scores, precision, and recall can be reported for different budget levels. Please verify whether MuSiQue maintains its advantage in the number of top five results or across different response generation workflows. This is particularly important in this comparison.

  3. Do the performance thresholds described in Section 2 apply only to the three-level hierarchical protocol proposed in the appendix? If the conclusions are generalizable, please analyze the feasibility of achieving this goal under different jump constraints and assess whether persistent bottlenecks remain even when significant associations or direct chained searches are allowed. Otherwise, please clearly specify the scope of application of the conclusions and distinguish between cases where the selected protocol is infeasible and cases where the system is unable to obtain evidence.

  4. In Table 3 and Appendix d-3, while the timestamps for search and generation are presented in a practical manner, they do not directly show the relative efficiency of complete queries. Please list, based on consistent metrics, the total latency and input/output token consumption for the main methods in each query (including query construction, any corrections, intermediate inference calls, and response generation). If the current evidence is based solely on indexing costs and the operational advantages of recursive servers, the scope of the term “end-to-end efficiency” should be appropriately limited.

5 comments

Dear Reviewer Dkyk,

Thanks for your positive feedback on our work. We appreciate your recognition of our contributions and the importance of our research. Your support further strengthens our confidence in the proposed approach.


W1-a & Q1: Regarding further explanation of answer generation

  • (1) QA settings of the baseline and G-Speculator

Thanks for your professional suggestions. For baselines, we set top-5 retrieved passages as context to generate an answer, where multi-step baselines rank their accumulated evidence across steps and select top-5. For our method, the number of retrieved passages is adaptively determined by β\beta (the number of returned trajectories). In the code, we maintain a cache of used passages to avoid duplicate passage retrieval across multiple steps.

  • (2) Answer sources of G-Speculator

Thanks for your careful review. The final answer comes from the Correction stage or Zero-shot QA (only triggered when Correction yields no answer). Following your suggestion, we report the answer source for each dataset.

Table 1: Answer sources on four multi-hop datasets.

Answer Generation StageHotpotQA2WikiMuSiQueScaleQA
Correction stage89.00%89.21%67.62%79.40%
Zero-shot QA stage11.00%10.79%32.38%20.60%

As shown in Table 1, most answers are generated in the correction stage, as G-Speculator thoroughly understands evidence during this stage.

  • (3) Applying the shared Q&A prompts

Thanks for your valuable comments. The result from the correction stage can serve as supplementary evidence to guide Q&A. We add experiments on applying such evidence to shared Q&A prompts.

Table 2: Performance (F1, %) of shared Q&A prompts.

Method2Wiki
KG-Translator78.5
G-Speculator84.7

As shown in Table 2, G-Speculator maintains the best performance under this QA setting.

  • (4) Answer generation modes

Thanks for your helpful suggestions. All baselines generate answers through the unified external Q&A model (GPT-4o-mini) in the main experiment. This is because most RAG and GraphRAG methods focus on retrieval strategy design and do not support native answer generation. The exception is the search agents Search-R1 and R1-Searcher, which support native generation. We report their performance under native generation in Appendix D.5, as excerpted below:

Table 3: Comparison (F1) with the native generation modes of search agents.

Methods2Wiki
Search-R164.1
R1-Searcher77.8
G-Speculator87.5

As shown in Table 3, G-Speculator outperforms search agents under their native generation modes.


Q2: Regarding the performance under different budget levels

Thank you for the thoughtful review. It is important to assess the budget levels and performance of different methods. Following your suggestion, we provide precision, recall, and F1 scores under different budget levels (step counts), corresponding to different token and price budgets.

Table 4: Comparison under different budget levels. We report token counts, token price ($, ×10−6\times10^{-6}), retrieval, and QA performance (%) on 2Wiki.

MethodBudget levelInput tokenOutput tokenTotal tokenInput priceOutput priceTotal pricePrecisionRecallF1
HippoRAG+IRCoT2-step3062.4890.583153.06459.3754.35513.7246.0492.9276.96
GFM-RAG+IRCoT2-step3640.64127.183767.82546.1076.31622.4041.2184.6769.36
KG-Translator2-step2026.18104.352130.53303.9362.61366.5463.1792.4678.52
G-Speculator1-step1347.51197.831545.34202.13118.70320.8281.4479.5475.05
G-Speculator2-step (default)2008.79297.432306.22301.32178.46479.7782.5491.0587.54
G-Speculator3-step2403.87352.362756.23360.58211.42572.0082.0392.2288.77
G-Speculator4-step2875.47414.933290.40431.32248.96680.2881.5992.5889.51
G-Speculator5-step3318.84466.503785.34497.83279.90777.7281.5192.5989.56

*Price budgets are calculated based on GPT-4o-mini input and output pricing (Input: $0.15/M, Output: $0.6/M).

As shown in Table 4, G-Speculator maintains lower or comparable token counts and prices compared to major competitors under default settings, while achieving the best performance.

Furthermore, we verify the QA performance of MuSiQue against major competitors under the top-5 retrievals.

Table 5: F1 scores (%) on MuSiQue under the top-5 retrieval setting.

MethodMuSiQue
HippoRAG+IRCoT37.81
KG-Translator43.32
G-Speculator47.14

As shown in Table 5, G-Speculator maintains its advantage on MuSiQue under the top-5 results.

W2-a & Q3: Regarding the applicability of evidence reachability conclusions

Thanks for your expertise. It is important to clarify the scope of the evidence reachability conclusions. Following your suggestion, we provide a further explanation.

Table 6: Evidence reachability under different hop constraints.

Dataset1-hop2-hop3-hop4-hop5-hop6-hop7-hop8-hop9-hop10-hop
HotpotQA30.251.258.063.668.271.473.273.874.274.2
2Wiki47.857.266.872.076.278.679.279.479.479.4
MuSiQue25.446.855.060.866.470.671.473.473.473.4

As shown in Table 6, evidence reachability presents a persistent bottleneck. Increasing the hop limit yields diminishing marginal returns, and reachability no longer improves when extended to 10 hops. Notably, in practical retrieval, additional hops also cause candidate node expansion and noise accumulation, making retrieval more difficult.


Q4: Regarding the efficiency

Thank you for the constructive feedback. Following your suggestion, we provide the token consumption and per-query latency for major methods.

Table 7: Input/output token consumption, per-query latency (s), and F1 on 2Wiki.

MethodInput tokenOutput tokenTotal tokenPer-query latencyF1
GFM-RAG+IRCoT3640.64127.183767.828.4669.36
HippoRAG+IRCoT3062.4890.583153.068.5576.96
KG-Translator2026.18104.352130.537.8078.52
G-Speculator (ours)2008.79297.432306.226.7787.54

As shown in Table 7, G-Speculator achieves significant performance gains with competitive token consumption and latency. Compared to the second-best performer KG-Translator, G-Speculator uses 8.25% more total tokens, while reducing latency by 13.21% and achieving an absolute F1 improvement of 9.02%.

W1-b & W2-b: Regarding further explanation of G-Speculator's components

Thank you for the generous feedback. Existing GraphRAG methods suffer from redundant retrieval results and biased retrieval paths (Section 2.2, Case 1&2), which motivates us to propose a novel speculation paradigm. Our G-Speculator first hypothetically speculates the most likely evidence chains, then verifies and corrects them. This mirrors human reasoning in complex tasks, where humans form a tentative judgment first and then seek targeted evidence to verify it.

To further illustrate the advantages of G-Speculator's mechanism, we present the following experiments:

  • (1) Component analysis of Graph Speculative Retrieval

For the graph speculative retrieval module, its performance is mainly affected by the linguistic assumption and the scoring function. We consider two ablations: opposite speculation strategies violating the linguistic assumption, and the scoring function without the key IDF component.

Table 8: Ablation study of the graph speculative retrieval module (excerpted from Figure 4 of the paper, single-step setting)

VariantHotpotQA2WikiMuSiQue
Full Model (single-step)74.575.028.0
w opposite speculation72.4 (-2.1%)64.7 (-10.3%)25.5 (-2.5%)
w/o idf74.0 (-0.5%)67.9 (-7.1%)26.1 (-1.9%)

As shown in Table 8, G-Speculator's speculation and scoring strategies are effective. Furthermore, to further investigate the rationale behind the speculation strategy, we provide additional speculation strategy variants, including nearest-adjacency, opposite, random, second-adjacency, and embedding-based speculation.

Table 9: Ablation study of different speculative strategies (single-step setting).

VariantHotpotQA2WikiMuSiQue
Full Model (single-step)74.575.028.0
w opposite speculation72.4 (-2.1%)64.7 (-10.3%)25.5 (-2.5%)
w random speculation74.0 (-0.5%)67.2 (-7.8%)26.4 (-1.6%)
w second-adjacency speculation72.1 (-2.4%)67.9 (-7.1%)24.8 (-3.2%)
w embedding-based speculation71.4 (-3.1%)64.1 (-10.9%)23.4 (-4.6%)

As shown in Table 9, the experimental results demonstrate that our linguistic assumption outperforms alternative strategies, as it determines the precision of graph speculative retrieval.

  • (2) Component analysis of Retrieval Path Correction

For the retrieval path correction module, it affects multi-step retrieval and answer generation. Removing this module restricts G-Speculator to zero-shot QA only under the single-step setting.

Table 10: Ablation study of the retrieval path correction module (excerpted from Figure 4 of the paper, multi-step setting)

VariantHotpotQA2WikiMuSiQue
Full Model (multi-step)82.187.548.9
w/o correction72.5 (-9.6%)70.8 (-16.7%)26.7 (-22.2%)

As shown in Table 10, G-Speculator benefits from the CoT-driven correction strategy, which continuously supplements evidence and deepens the understanding of evidence to both accurately guide retrieval and infer the answer. To clarify how retrieval path correction affects answer generation, we present an ablation study under the single-step setting, which reflects the direct impact of correction on answer reasoning, while leaving retrieval unaffected.

Table 11: Ablation study of the retrieval path correction module (excerpted from Figure 4 of the paper, single-step setting)

VariantHotpotQA2WikiMuSiQue
Full Model (single-step)74.575.028.0
w/o correction72.5 (-2.0%)70.8 (-4.2%)26.7 (-1.3%)

As shown in Table 11, G-Speculator's correction improves QA accuracy. This is because, while analyzing evidence sufficiency, the correction module sorts out logical relationships among evidence, such that the naturally derived answer is grounded in a full understanding of the evidence.


Thank you again sincerely for your time, efforts, and thoughtful feedback. We have incorporated your helpful comments in the revised PDF. If you have any additional comments or suggestions, we would greatly value your comments.

Reviewer_SAarOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Existing GraphRAG systems suffer from performance degradation in multi-hop reasoning due to information loss during graph construction and the limitations imposed by rigid graph structures. In particular, relevant evidence may be unreachable within the constructed graph or may require unnecessarily long traversal paths.

To address these limitations, this paper proposes G-Speculator, a framework that constructs a Hypothetical Knowledge Graph (HypoKG) while preserving entity mentions, relation spans, and their original word order. Rather than traversing the entire graph, G-Speculator employs linguistic assumptions about the relative positions of entities and relations to speculate bridge entities and retrieve relevant evidence. When the retrieved evidence is insufficient, an LLM performs chain-of-thought analysis to generate new queries and correct the retrieval direction.

Experiments on four multi-hop QA datasets demonstrate improvements over existing GraphRAG and multi-step RAG methods in both QA F1 and retrieval precision.

Critical strengths
  1. Analysis of structural limitations in existing GraphRAG systems. The authors investigate the underlying causes of retrieval performance degradation in existing GraphRAG systems, focusing on limitations arising from graph construction and connectivity. Their analysis of evidence reachability in multi-hop QA reveals that the second supporting passage is substantially less reachable than the first. Even when relevant evidence is reachable, the required graph traversal path may be considerably longer than the actual reasoning chain, resulting in increased retrieval costs. These observations provide empirical evidence that missing relations and information loss during graph construction can limit multi-hop retrieval performance.

  2. A flexible approach to entity exploration. Unlike conventional knowledge graphs that rely on explicit knowledge triple extraction, HypoKG preserves entity mentions, relation spans, and their relative positions in the original text. This allows the system to exploit potential associations between entities appearing in the same context, even when their relations are not explicitly extracted. The proposed approach leverages the common Subject-Predicate-Object structure of English sentences to speculate bridge entities by selecting the nearest entity on the opposite side of a relation. This provides a simple and computationally efficient retrieval strategy. Rather than exhaustively traversing the graph, the method selects a small number of promising clue trajectories and dynamically adjusts the retrieval direction through LLM-based analysis when additional evidence is needed. Furthermore, avoiding LLM-based triple extraction during graph construction offers practical advantages in terms of indexing cost.

  3. Extensive comparisons across baselines and datasets. The authors evaluate their approach on four multi-hop QA datasets: HotpotQA, 2WikiMultihopQA, MuSiQue, and ScaleQA, covering different question types and reasoning complexities. The experiments include three zero-shot LLMs, vanilla RAG, eight GraphRAG methods, and seven multi-step methods, providing a broad comparison with existing approaches. In addition to QA F1, the authors report retrieval precision and efficiency, allowing the proposed framework to be evaluated from multiple perspectives. Ablation studies examine the effects of speculation direction, IDF correction, retrieval path correction, and evidence expansion, demonstrating the contributions of these components across multiple datasets.

Critical weaknesses
  1. Limited validation of the generalizability of the linguistic assumptions. The proposed method relies on the common Subject-Predicate-Object structure of English sentences, selecting the nearest entity on the opposite side of a relation as the bridge entity. However, this assumption may not directly transfer to languages with different word orders, such as Korean and Japanese. Even in English, the assumption may not consistently hold for passive constructions, complex relative clauses, or sentences containing multiple entities. Since all experiments are conducted on English-language datasets, it remains unclear whether the proposed linguistic assumption generalizes to different languages and syntactic structures. Additional evaluations on multilingual datasets or sentences with more complex syntactic structures would provide stronger evidence for the generalizability of the proposed approach.

  2. Limitations of the retrieval evaluation metrics. The paper evaluates retrieval performance using precision alone. However, multi-hop QA often requires multiple pieces of evidence to be jointly retrieved. Even if all retrieved passages are relevant, missing one or more necessary supporting passages may prevent the model from completing the reasoning process correctly. Therefore, precision alone is insufficient to determine whether the retrieved passages provide complete evidence for multi-hop reasoning. Reporting retrieval recall or complete evidence-chain recall would provide a more comprehensive assessment of whether G-Speculator successfully retrieves all necessary supporting evidence.

  3. Insufficient direct evaluation of bridge entity speculation accuracy. One of the central contributions of the paper is the speculation of bridge entities based on the relative positions of entities and relations within sentences. Although the authors provide an ablation study that modifies the speculation direction, this experiment only measures downstream QA performance. It does not directly establish how accurately the proposed mechanism identifies the correct bridge entities. Furthermore, it remains unclear whether selecting the nearest entity on the opposite side of a relation is more effective than other reasonable entity selection strategies. A direct evaluation of speculation accuracy against gold bridge entities, along with comparisons against embedding-based or dependency-parsing-based entity selection, would provide stronger support for the proposed mechanism.

Questions for authors
  1. Have the authors directly evaluated the accuracy of the proposed bridge entity speculation mechanism? In particular, how reliably does the linguistic assumption hold for passive constructions, complex sentence structures, or sentences containing multiple entities?

  2. Could the authors report retrieval recall or complete evidence-chain recall in addition to precision? Measuring the proportion of questions for which all required supporting passages are successfully retrieved would help clarify the effectiveness of the proposed retrieval strategy.

  3. How does the graph reachability of gold supporting evidence differ between conventional knowledge graphs and HypoKG when constructed from the same corpus? Such a comparison would provide more direct evidence of whether HypoKG effectively alleviates the structural limitations identified in the preliminary study.

6 comments

Dear Reviewer SAar,

We sincerely appreciate your time and effort in providing insightful comments and constructive feedback. To address your concerns, we conduct rigorous supplementary experiments and detailed analyses to further clarify the technical details.


W1 & W3 & Q1: Regarding the generalizability of the linguistic assumptions

  • (1) Evaluation across different syntactic structures

Thank you for your expertise. It is important to assess the generalization of the linguistic assumption across different syntactic structures. Following your suggestion, we evaluate bridge entity speculation accuracy on three types of syntactic structures: active, passive, and clause-based sentences (complex).

Table 1: Bridge entity speculation accuracy under different syntactic structures on three multi-hop datasets.

Syntactic StructureHotpotQA2WikiMuSiQue
Active53.9975.0035.03
Passive59.7972.1323.17
Complex60.8383.7643.30
Overall58.4079.4037.40

As shown in Table 1, our linguistic assumption generalizes across different sentence structures and performs best on complex structures, indicating that G-Speculator can accurately locate the core semantics relevant to the retrieval need through graph speculative retrieval.

Furthermore, we also provide bridge entity speculation accuracy on sentences with different numbers of entities.

Table 2: Bridge entity speculation accuracy on sentences with different numbers of entities.

Entity CountHotpotQA2WikiMuSiQue
<=560.5381.0839.59
>555.1076.9632.91
Overall58.4079.4037.40

As shown in Table 2, G-Speculator maintains stable speculation accuracy across sentences with different entity counts. Although speculation accuracy decreases on entity-rich sentences, G-Speculator can compensate through correction, which revises the retrieval direction when speculation is inaccurate.

In addition, adapting existing RAG methods to multilingual datasets requires jointly considering entity recognition, embedding models, and prompt design, which we will explore in future work.

  • (2) Evaluation across different speculation mechanisms

Thanks for your professional suggestions. Following your suggestion, we provide additional speculation strategies to validate the rationale of the existing strategy. Specifically, we adopt nearest-adjacency, opposite, random, second-adjacency, and embedding-based speculation.

Table 3: Bridge entity speculation accuracy under different speculation strategies on three multi-hop datasets.

RulesHotpotQA2WikiMuSiQue
Nearest-adjacency Speculation (ours)58.479.437.4
Opposite Speculation53.2 (-5.2%)59.6 (-19.8%)30.2 (-7.2%)
Random Speculation57.4 (-1.0%)63.6 (-15.8%)34.8 (-2.6%)
Second-adjacency Speculation46.2 (-12.2%)55.2 (-24.2%)32.6 (-4.8%)
Embedding-based Speculation49.8 (-8.6%)42.6 (-36.8%)27.6 (-9.8%)

As shown in Table 3, G-Speculator's bridge entity speculation strategy based on the linguistic assumption is the most effective and significantly outperforms other reasonable strategies.

W2 & Q2: Regarding the retrieval quality

Thank you for the generous feedback. Retrieval quality has two core dimensions: evidence Precision and Recall. Existing GraphRAG methods have an advantage in evidence recall, but they also introduce noise that affects the QA model's understanding and reasoning. Therefore, GraphRAG systems need to balance Precision and Recall. Following your suggestion, we report detailed retrieval quality on diverse question types.

Table 4: Retrieval quality report. We report Precision (left) and Recall (right).

Method2Wiki Br.2H^{\text{2H}}2Wiki Cp.2H^{\text{2H}}2Wiki In.2H^{\text{2H}}2Wiki Cp.4H^{\text{4H}}2Wiki Bc.4H^{\text{4H}}MuSiQue Br.2H^{\text{2H}}MuSiQue Br.3H^{\text{3H}}MuSiQue Br.4H^{\text{4H}}
IRCoT29.6 | 74.032.7 | 81.828.5 | 71.323.8 | 59.647.0 | 58.827.1 | 67.731.5 | 52.527.8 | 34.7
Search-R131.1 | 77.733.8 | 84.528.2 | 70.429.4 | 73.651.4 | 64.229.0 | 72.625.4 | 42.320.6 | 25.8
R1-Searcher33.2 | 83.036.8 | 92.031.9 | 79.833.8 | 84.440.8 | 51.029.4 | 73.626.8 | 44.724.9 | 31.2
HippoRAG+IRCoT37.7 | 94.339.2 | 98.133.3 | 83.339.8 | 99.675.4 | 94.327.5 | 68.731.6 | 52.731.3 | 39.1
GFM-RAG+IRCoT35.4 | 88.636.0 | 90.032.1 | 80.336.6 | 91.662.4 | 78.121.6 | 53.923.5 | 39.225.7 | 32.1
KG-Translator58.6 | 95.968.1 | 95.350.5 | 90.572.2 | 96.873.2 | 87.131.6 | 68.729.8 | 47.726.5 | 31.6
G-Speculator81.5 | 92.593.7 | 95.374.9 | 84.990.9 | 96.878.0 | 90.146.3 | 67.134.8 | 54.930.3 | 39.9

As shown in Table 4, G-Speculator retrieves necessary evidence while avoiding interference from excessive irrelevant passages, enabling it to better understand key evidence during the retrieval path correction stage, ultimately delivering superior QA performance.


Q3: Regarding the evidence reachability of HypoKG

Thank you for the professional feedback. It is important to analyze evidence reachability on HypoKG. Following your suggestion, we report a comparison between HypoKG and the OpenIE-based KG.

Table 5: Evidence reachability on MuSiQue.

GraphE1 (1-hop)E1 (2-hop)E1 (3-hop)E1 (unreachable)E2 (1-hop)E2 (2-hop)E2 (3-hop)E2 (unreachable)Overall
OpenIE-based KG64.07.64.623.831.219.04.845.055.0
HypoKG69.06.42.022.647.69.85.237.462.6 (+7.6%)

As shown in Table 5, HypoKG effectively alleviates the structural flaws. This is reflected in two aspects: (1) improved reachability of complete evidence chains (e.g., a 7.6% overall improvement), enabling G-Speculator to access more comprehensive evidence; (2) earlier reachability (e.g., a 16.4% increase in 1-hop reachability for E2), which reduces the retrieval complexity of G-Speculator.


We once again express our sincere appreciation for your time, efforts, and thoughtful feedback. We have revised the PDF according to your constructive suggestions. If you have any further comments or suggestions, we would welcome any further feedback.

Thank you for the detailed response and additional experiments. I believe that my previous concerns have been substantially addressed. However, I would like to clarify two remaining points.

  1. Complete evidence-chain recall: In addition to recall at the individual supporting passage level, could the authors report the proportion of questions for which all gold supporting passages are successfully retrieved?

  2. Distinguishing the contributions of retrieval and CoT reasoning: Since a substantial proportion of final answers are generated directly during the retrieval path correction stage, it is difficult to determine whether the improvements in QA performance stem from better retrieval or additional CoT reasoning. Could the authors provide a controlled experiment in which all methods are evaluated using the same QA procedure?

I will consider raising my rating after reviewing the authors' responses to these two points.

Comments on the paper

1 comment

Dear Reviewers and Area Chairs,

Thank you sincerely for your time, effort, and thoughtful feedback during the author-reviewer discussion phase. Your insights have been crucial in helping us refine and improve our work. Below, we briefly summarize our main contributions and the key points during the rebuttal phase.

Overall Assessment

Our submission initially received ratings of (3, 2, 2), and the reviewers have raised their scores to (3, 3, 3), with corresponding confidence ratings of (2, 3, 4). Specifically, Reviewer Dkyk explicitly stated that they would maintain their positive score. Reviewers SAar and 9dMC initially gave a rating of 2 and have raised their ratings to 3.

Reviewer SAar: The new results have sufficiently addressed my remaining concerns and strengthened the empirical evidence supporting the paper. I appreciate the authors' efforts. I have updated my score and now lean toward acceptance.

Reviewer Dkyk: The new results and clarifications have strengthened my confidence in the paper, and I will maintain my original rating.

Reviewer 9dMC: Thank you for the detailed rebuttal and the additional experiments. I have raised my score from 2 to 3.

Over the past few days, we have continued to refine the paper and have incorporated the reviewers' constructive suggestions in the revised version.

Strengths of Our Work

  • Our preliminary study is a useful diagnostic that reveals key limitations of existing GraphRAG systems: redundant retrieval results and biased retrieval paths. (Reviewers SAar and 9dMC)
  • HypoKG efficiently serializes passages into entity-relation sequences while preserving contextual word order as a linguistic prior. (Reviewers SAar, Dkyk, and 9dMC)
  • Rather than exhaustively traversing the graph, G-Speculator flexibly speculates a few promising clue trajectories and dynamically adjusts the retrieval direction through LLM-based analysis. (Reviewers SAar and Dkyk)
  • Extensive experiments offer a more practical perspective and better reflect G-Speculator's strong performance. (Reviewers SAar, Dkyk, and 9dMC)

Addressed Concerns

  • Evidence Reachability of HypoKG: Reviewers SAar and 9dMC suggested that we evaluate evidence reachability on HypoKG compared to OpenIE-based KG, and we have added this reachability analysis. Further, following Reviewer Dkyk's suggestion, we provide a clarification of the conclusions on evidence reachability.
  • End-to-end Efficiency: Following the suggestions of Reviewers Dkyk and 9dMC to provide efficiency analysis, we analyzed per-query end-to-end token consumption and latency.
  • Retrieval Quality: Reviewers SAar, Dkyk, and 9dMC suggested reporting detailed retrieval quality. We provided a comprehensive retrieval quality report, including precision, passage recall, and evidence-chain recall.
  • Evaluation Scope: (1) Reviewers SAar and 9dMC noted the lack of a generalization analysis of our speculation rule, and we added analyses across syntactic patterns and speculation strategy ablations. (2) Reviewer Dkyk suggested clarifying component roles in the speculation paradigm, and we performed comprehensive analyses. (3) Reviewer 9dMC recommended adding more diverse baselines and datasets, and we presented additional comparisons on these extensions.

Thank you again for your time and expertise throughout this review process. We deeply appreciate your insights and recognition of our work, and each suggestion you raised has given us valuable new perspectives.

We hope this summary could be helpful for the next stage of review and discussion.

Best regards,

Authors of Paper 17047

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.