acceptodds
Under review as a conference paper at ICLR 2027

WARP-G: Workload-Aware Regional Provisioning for GraphRAG

Abstract

GraphRAG connects evidence across documents through entity relations and graph propagation, but constructing a full-corpus graph can incur substantial offline cost. Efficient GraphRAG methods typically simplify construction or select structurally important content. These approaches do not directly account for uneven query demand or regional differences in the benefit of graph retrieval over a passage baseline. We introduce **WARP-G** (Workload-Aware Regional Provisioning for GraphRAG), which uses an evidence-annotated design workload to select regional graph indexes. WARP-G partitions the corpus using query co-access and weak semantic links, prioritizes a limited set of graph probes by demand, baseline evidence gaps, and regional size, and measures their retrieval gains. It then selects graphs with positive conditional utility on a shared design sample and reuses their indexes at inference time, alongside full-corpus passage retrieval. We evaluate WARP-G with HippoRAG2 on four QA benchmarks. Retained indexes account for 0–14.0% of Full Graph construction tokens, and first-use graph-token cost falls by 38.7–67.2%, including design and online retrieval. On 2WikiMultihopQA, WARP-G retains one graph costing 8.2% of Full Graph construction tokens, while improving ER@10 by 1.00 percentage point and answer F1 by 1.74 points over Full Graph. WARP-G thus reduces graph-related cost while preserving or improving answer quality relative to the passage baseline on the evaluated workloads.

Primary area:
foundation or frontier models, including LLMs
Keywords:
GraphRAG, workload-aware provisioning, selective graph construction, corpus partitioning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_dsKNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

WARP-G selectively builds regional GraphRAG indexes using workload information. It forms passage regions, probes regional HippoRAG2 indexes with evidence-annotated design queries, and retains combinations with positive conditional retrieval utility. Full-corpus passage retrieval remains available when graph augmentation is unhelpful or unavailable. The experiments use 800 design and 200 held-out test queries per dataset across four QA workloads. WARP-G retains zero to two graphs and reports 38.7–67.2% lower first-use graph-token cost than Full Graph, including discarded probes and design evaluations. Selection and partition controls examine the resulting retrieval quality, retained-index size, and design costs.

Critical strengths
  1. A useful deployment problem with careful graph-cost accounting (PDF pp. 5–8, Sections 3.4–4.3; pp. 15–18, Appendix A.3 and Tables 6–8). The method treats graph construction as an optional investment while preserving passage access over the full corpus. This is a useful formulation because a deployment can decline graph augmentation without giving up its basic retrieval coverage. The reported first-use accounting includes discarded probes and conditional evaluations, counts reused indexes once, and distinguishes logical token counts from physical usage after caching. These savings therefore account for design work even when the resulting probe indexes are not retained. The cost boundary is also explicit: it covers graph design and online graph retrieval for 200 test queries, while shared passage indexing, reranking, and answer generation are excluded. Within that scope, the 38.7–67.2% savings are meaningful evidence that selective regional provisioning can substantially reduce graph expenditure relative to constructing a full-corpus graph. They should be interpreted as graph-token savings rather than an equivalent reduction in all end-to-end serving costs.

  2. Transparent results that expose workload-dependent trade-offs (PDF pp. 7–9 and 19–20, Tables 1–2 and 9–11). On 2Wiki, the retained graph uses 8.2% of Full Graph's construction tokens and obtains answer F1 of 64.21, compared with Hybrid's 63.06 and Full Graph's 62.47. The paper reports zero-graph outcomes on HotpotQA and NQ rather than assuming that graph construction must always help. Frequency-only, gain-only, and partition controls further show that smaller retained indexes, lower first-use cost, and higher retrieval scores are different objectives. Section 4.5 explains the distinction between filling an externally given capacity and spending design effort to select capacity automatically. The retrieval-depth and routing diagnostics also help explain why reaching an evidence region need not translate into better answer quality. This reporting makes the method's intended use and its limitations substantially easier to assess.

Critical weaknesses
  1. The incremental value of choosing graph capacity automatically is not yet established by an independent capacity comparison (PDF pp. 5, 8–9 and 18–20, Sections 3.4/4.5 and Tables 2/9). The frequency-only, gain-only, and random-selection controls inherit WARP-G's retained-region count. These are informative tests of which regions to select at a given capacity, but they do not independently evaluate how many graphs should be retained. This distinction matters because choosing capacity, including choosing zero, is one of the principal benefits claimed for the probing and conditional-selection procedure.

    Frequency-only obtains nearby ER@10 values with lower first-use costs: 2.786 versus 6.713 million tokens on 2Wiki, and 3.826 versus 8.429 million on MuSiQue. However, it retains larger indexes, and its stated cost excludes the work needed to determine the capacity inherited from WARP-G. These results therefore do not establish that frequency-only dominates the proposed method. The authors already explain the two deployment settings; the unresolved issue is quantitative evidence for the additional automatic decision. A practical, independently specified or selected capacity rule would provide that comparison if any selection overhead were counted. The savings against Full Graph support selective provisioning, but do not by themselves show how much value this particular automatic selection procedure adds over a simpler deployment rule.

  2. One completed design sample per workload leaves uncertainty about the usefulness of the selected deployment (PDF p. 5, Section 3.4; pp. 15–16 and 20, Appendices A.3/B). Although the overall design workload has 800 queries, the final conditional-selection stage compares up to 16 graph sets using at most 32 shared queries and accepts the largest strictly positive estimated marginal gain. Each dataset has one completed design/test evaluation with seed 42. The paper explicitly discloses this limitation and provides held-out test-query bootstrap intervals; those intervals address uncertainty conditional on the selected deployment, not the effect of changing the design sample that produced it.

    This matters because the method's output is a deployment decision. Different selection samples might choose different graph sets yet yield similarly useful quality–cost trade-offs, which would be reassuring. Conversely, materially different held-out utility or cost would limit the reliability of automatic provisioning. The current results do not establish either outcome, so I do not interpret them as evidence of observed instability or overfitting. A limited resampling analysis of the final decision stage, reusing existing probes and holding the partition and shortlist fixed, would address part of this uncertainty without rerunning the entire design pipeline. The reported +1.15/+1.10 answer-F1 gains over Hybrid are point estimates; paired uncertainty would also clarify their strength without treating interval overlap as a significance test.

I lean toward weak rejection. The optional-index formulation, transparent workload analysis, and design-inclusive graph-cost accounting are valuable. My reservation is whether the evidence is sufficient for the automatic selection contribution: its capacity decision lacks an independent comparison, and the usefulness of that decision has not been assessed across selection samples. A focused comparison and limited resampling study showing a repeatable, useful quality–cost trade-off would materially improve my assessment. WARP-G need not minimize every cost or choose identical graphs across samples; it should demonstrate why its additional design effort produces a useful deployment decision.

Questions for authors
  1. For W1 — main priority: Can an independently specified or selected capacity rule be compared under the same partition, retrieval backend, and full-corpus passage access? Please explain how the capacity is determined without inheriting WARP-G's output, count any selection overhead, and report retrieval quality, retained-index cost, and first-use cost separately. This would clarify the deployment conditions under which automatic selection justifies its design effort.
  2. For W2: With the existing probed indexes, partition, and shortlist fixed, how do resamples of the final selection queries affect selected capacity, cost, and held-out quality? The useful outcome is the distribution of quality–cost choices, not necessarily agreement on the exact graph set. Please distinguish this targeted check of the final decision stage from variability in the entire design pipeline.
  3. For W2: If per-query scores are available, can paired uncertainty for the answer-F1 changes against Hybrid supplement the existing retrieval intervals and gain-only test? This would clarify the strength of the answer-quality evidence without requiring a new benchmark or a different generation setup.
3 comments
AuthorsOpenReview ↗

Response to Reviewer dsKN

Thank you for the clear and practical suggestions. Your point about choosing the graph count, rather than only choosing regions for a given count, was especially helpful. We ran the capacity and repeat-sampling tests you asked for, and added confidence intervals for answer F1.


Q1 / W1: Choosing the graph count and paying for that choice

In the paper, frequency-only used the same graph count as WARP-G. We now also test frequency ranking with fixed counts k=0,1,2,4,8k=0,1,2,4,8, and a method that chooses kk using 200 labeled validation queries. Its score is 0.5 ER@10+0.5 CE@100.5\,ER@10+0.5\,CE@10. We count all the graph construction and validation graph retrieval used to make that choice.

The WARP-G numbers are from the paper. For frequency at the matching graph count, the cost and retrieval scores are from the paper and F1 comes from saved original-run answers. Other counts are from new runs. Costs are in millions of graph tokens (M).

DatasetMethodF1ER@10Retained (M)First-use (M)
2WikiHybrid, k=0k=063.0679.2500
2WikiFrequency, k=1k=164.4979.131.9462.786
2WikiFrequency, k=2k=262.6478.752.6954.391
2WikiFrequency, k=4k=462.0478.885.6298.505
2WikiFrequency, k=8k=861.8579.008.71313.696
2WikiValidation-selected——018.491
2WikiWARP-G64.2180.000.7986.713
MuSiQueHybrid, k=0k=043.0773.5100
MuSiQueFrequency, k=1k=145.2373.081.3011.796
MuSiQueFrequency, k=2k=243.3273.582.9253.826
MuSiQueFrequency, k=4k=444.7172.926.4708.283
MuSiQueFrequency, k=8k=844.9373.2510.56313.821
MuSiQueValidation-selected——015.811
MuSiQueWARP-G44.1773.622.5988.429

First-use cost includes graph construction and online graph retrieval. WARP-G's design and online costs are 6.068M + 0.646M (2Wiki) and 7.538M + 0.891M (MuSiQue), as reported in the paper. The validation-based capacity search alone uses 18.491M / 15.811M tokens.

A small fixed count can work very well. The hard part is picking that count before seeing test results. WARP-G uses a small set of probes, reuses the graphs it builds, and stops when another graph no longer adds value. This is also why it selects zero graphs on HotpotQA and NQ, while keeping full passage retrieval.


Q2 / W2: Changes across selection samples

We ran the test you suggested: 20 samples of 32 selection queries (seeds 1000–1019). We kept the regions, probed graphs, shortlist, and test queries unchanged.

DatasetDifferent graph setsRuns with k=0/1/2k=0/1/2First-use (M), mean ± SDRetained build range (M)
2Wiki56 / 11 / 36.491 ± 0.6830–2.743
MuSiQue87 / 10 / 37.751 ± 0.3940–2.598

The exact graph IDs can change, but every run keeps at most two graphs. We also reran the full pipeline with different data splits and regions. Compared with Hybrid on each matching split, WARP-G changed answer F1 by 0.00 and +0.19 points on 2Wiki (seeds 43/44), and +1.36 points on MuSiQue (seed 43). This gives us a second view of how the method behaves when the design data changes.


Q3 / W2: Confidence intervals for answer F1

We added 10,000 paired bootstrap samples using the original paper's 200 test queries:

ComparisonF1 difference (points)95% CI
2Wiki: WARP-G – Hybrid+1.15[-1.04, +3.45]
MuSiQue: WARP-G – Hybrid+1.10[-1.29, +3.58]
2Wiki: WARP-G – Full Graph+1.74[-1.52, +5.25]

We agree with your point that the exact region IDs matter less than the final quality and cost. In the paper's 2Wiki result, WARP-G keeps one region costing just 8.2% of Full Graph construction tokens, with 64.21 F1 (Hybrid: 63.06; Full Graph: 62.47). Across four workloads, WARP-G reduces first-use graph tokens by 38.7–67.2%, even after counting the work done to choose the graphs.

Thank you again for suggesting these tests. We would be happy to discuss any other questions.

Thank you for showing that WARP-G uses fewer graph tokens than the validation-based selector even after accounting for selection costs. I maintain my rating of 2 (weak rejection) because the fixed k=1 policy used fewer graph tokens than WARP-G and achieved higher F1 point estimates on both datasets, and answer F1 remains unreported across the 20 selection resamples.

Thank you for the follow-up. Our contribution is deciding where graphs are useful and how many to retain, using limited probing, conditional selection, and index reuse. Fixed k = 1 sets capacity in advance; it cannot decide when to retain no graph or multiple graphs. Its success on two workloads does not establish a generally suitable capacity.

Choosing capacity also has a cost: the evaluated validation search uses 18.491M/15.811M graph tokens, versus WARP-G’s 6.068M/7.538M design tokens on 2Wiki/MuSiQue, including probing and selection. Fixed k = 1 has higher F1 point estimates and lower first-use cost, but WARP-G has higher ER@10: 80.00 vs. 79.13, and 73.62 vs. 73.08. The k = 1 results are strong, but for a new workload we still need a way to decide which graphs to retain and how many, without spending too much on that decision.

We have now completed the missing answer-quality evaluation for the 20 selection resamples:

DatasetF1, mean ± SDFirst-use graph tokens (M), mean ± SD
2Wiki62.41 ± 0.476.491 ± 0.683
MuSiQue44.51 ± 1.067.751 ± 0.394

Only the selection-query sample changes; the partition, probed indexes, shortlist, and test queries remain fixed. Here we check how much answer quality and cost change when we use a different query sample to select the graphs.

Reviewer_hYKuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes WARP-G, a workload-aware framework that reduces GraphRAG construction costs by identifying where regional graph indexes add value over passage retrieval. It partitions the corpus using query co-access and semantic links, probes selected regions to measure retrieval gains, and selects graph combinations based on their conditional benefit. Selected indexes are reused alongside full-corpus passage retrieval. Experiments demonstrate the effectiveness of the proposed method.

Critical strengths
  1. The paper is logically coherent, with well-designed figures and tables.
  2. The authors presents a well-motivated perspective on GraphRAG construction, allocating regional graph indexes according to query demand and measured benefit over passage retrieval.
  3. Experiments demonstrate reduced graph-related token costs, including design overhead, while maintaining competitive answer quality on the evaluated workloads.
Critical weaknesses
  1. The added value of probing and conditional selection over simpler policies remains unclear. Frequency-only uses fewer first-use graph tokens with similar evidence recall , while gain-only achieves higher retrieval scores on MuSiQue. In addition, answer quality is not reported for the frequency-only and gain-only ablations.
  2. The stability of regional graph selection is insufficiently evaluated. The existing results are based on a single design/test split and a small shared selection sample, leaving sensitivity to these choices unassessed.
  3. The evaluation uses only HippoRAG2 for regional indexes. Whether WARP-G’s quality–cost benefits extend to GraphRAG backends with different construction and retrieval mechanisms remains unclear.
Questions for authors
  1. Could you report answer quality for the frequency-only and gain-only ablations to clarify whether WARP-G’s additional design cost yields better answers?
  2. How do the set of retained regional graphs, together with retrieval and answer quality, vary across different design splits and shared selection samples?
  3. Could you evaluate WARP-G with another GraphRAG backend to assess whether its quality–cost benefits extend beyond HippoRAG2?
1 comment
AuthorsOpenReview ↗

Response to Reviewer hYKu

Thank you for the helpful questions. We agree that good retrieval scores should also lead to useful answers. We have added answer F1 for the original ablations, run more tests with different samples, and tried WARP-G with a second graph backend.


Q1 / W1: Answer quality of simpler selection methods

We went back to the saved answer outputs from the original runs. Here are the answer scores and first-use graph-token costs (M = million):

DatasetMethodF1EMFirst-use (M)
2WikiHybrid63.0660.000
2WikiFull Graph62.4759.5010.943
2WikiWARP-G64.2161.006.713
2WikiFrequency-only64.4962.002.786
2WikiGain-only65.0462.505.927
MuSiQueHybrid43.0732.000
MuSiQueFull Graph43.2833.0019.845
MuSiQueWARP-G44.1732.008.429
MuSiQueFrequency-only43.3232.003.826
MuSiQueGain-only44.0033.009.037

Simple ranking can work well when we already know how many graphs to keep. In the paper, both ablations were given WARP-G's final graph count. They test which regions to pick, while WARP-G also decides how many to build. On HotpotQA and NQ, it decides to build no graph at all, while still searching the full passage corpus.

Here is the paper's retrieval comparison with the same graph count:

DatasetMethodER@10CE@10Retained build (M)
2WikiWARP-G80.0053.000.798
2WikiFrequency79.1352.501.946
2WikiGain78.8852.500.849
MuSiQueWARP-G73.6245.502.598
MuSiQueFrequency73.5846.502.925
MuSiQueGain74.5448.502.087

We also made the candidate pool exactly the same for all methods (3 probed regions on 2Wiki and 6 on MuSiQue). Conditional selection chose r0007 on 2Wiki, while frequency chose r0000. On MuSiQue, both chose {r0012, r0022}. This makes sense to us: conditional selection does not need to choose differently every time. It asks whether adding one more graph still helps after accounting for those already chosen.

We tested frequency ranking with its own preset capacities k=0,1,2,4,8k=0,1,2,4,8 and a policy that picks kk with 200 validation queries. That validation search used 18.491M / 15.811M design graph tokens on 2Wiki/MuSiQue. WARP-G's original design cost was 6.068M / 7.538M. This is why we care about finding the graph count, not just ranking regions once that count is given.


Q2 / W2: Results with different samples

We picked a new set of 32 shared selection queries 20 times, keeping the region split, probed graphs, shortlist, and test set fixed:

DatasetDifferent graph setsRuns selecting k=0/1/2k=0/1/2First-use (M), mean ± SD
2Wiki56 / 11 / 36.491 ± 0.683
MuSiQue87 / 10 / 37.751 ± 0.394

We also reran the whole process with new train/test splits, new Leiden regions, and new probing and selection:

Dataset / seedSelected regionHybrid F1WARP-G F1F1 changeFirst-use (M)
2Wiki / 43r001063.5563.550.005.229
2Wiki / 44r000160.3460.52+0.196.087
MuSiQue / 43r001244.6045.96+1.366.030

These are new runs, shown separately from the original paper results (64.21 F1 on 2Wiki; 44.17 on MuSiQue). Different samples can lead to different regions, but the method keeps the final set small and does not give up full-corpus passage retrieval.


Q3 / W3: Pilot with another graph backend

Thank you for the suggestion. We started a pilot with Microsoft GraphRAG v3.3.0 to test WARP-G with a different graph structure. We integrated its entity/relationship extraction and community summaries into regional indexing, and mapped retrieval results to passage IDs for the same reranker and answer generator. The integration works on a small pilot dataset.

The main challenge is the time and cost of a full Microsoft GraphRAG index. Entity extraction and community summaries require many LLM calls, making full-corpus indexing resource-intensive. Based on our progress, we expect that completing a full index and fair end-to-end comparison would be difficult within the rebuttal period. We are continuing this work and hope to share a complete evaluation later.

WARP-G selects regions by measured gains over passage retrieval, not by any one graph representation or algorithm. The pilot is a useful first step toward testing this with another backend.

Thank you again. We would be happy to discuss any further questions

Reviewer_RmdBOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the author(s) present WARP-G, which uses an evidence-annotated design workload to select regional graph indexes for GraphRAG. It’s evaluated with HippoRAG2 on four QA benchmarks and retains indexes account for 0-14.0% of Full Graph construction tokens, and first-use graph-token cost falls by 38.7-67.2%. Overall, the paper has made some novel contributions, but there are still some issues.

Critical strengths

S1. WARP-G addresses a useful deployment problem: allocating graph construction according to query demand and measured benefit over full-corpus passage retrieval. Separating probing from conditional selection accounts for interactions between regional indexes, reuses constructed indexes, and permits passage-only deployment, making the framework relevant to resource-constrained GraphRAG systems.

S2. The experiments demonstrate substantial graph-token savings against Full Graph, including design overhead. On 2Wiki, WARP-G achieves the highest reported answer F1 while retaining a graph requiring 8.2% of full construction tokens. The explicit reporting of design costs, dense fallbacks, and mixed ablation results makes the quality-cost trade-offs easier to assess.

Critical weaknesses

W1. The quality-cost advantage over simpler construction and selection strategies is not sufficiently established. KET-RAG and G2ConS are evaluated at a fixed 80% passage-coverage setting, leaving their performance at lower budgets unexplored. MuSiQue's gain-only ablation achieves higher retrieval point estimates and lower retained construction cost, but higher first-use cost (Tables 2 and 9). Selection variants can also use different candidate pools, so these results do not isolate the benefit of conditional selection or establish that it justifies the additional design effort.

W2. The evidence for stable gains and practical applicability is limited. Each dataset uses one split with 200 test queries; Appendix B.2 reports bootstrap intervals, but paired intervals for the main answer-F1 differences and variation across design samples are missing. Selection uses at most 32 design queries, making its sensitivity to sampling important. The corpora contain 6,119-11,655 passages and use 800 evidence-annotated design queries each, while the cost analysis centers on graph tokens. These settings leave uncertainty about selection reliability and savings under larger workloads or different annotation and query-volume conditions.

Questions for authors

Q1. Which KET-RAG and G2ConS mechanisms are retained in the adaptations, and how was 80% passage coverage chosen? How does answer quality compare across budgets against these methods and a frequency baseline with independently selected capacity, accounting for selection and design costs?

Q2. How are the shared conditional-evaluation queries sampled, and how do they relate to the local probe samples? What variation in selected regions, answer F1, and first-use cost occurs across design samples, and do paired confidence intervals support the main answer-F1 gains?

Q3. On the same probed candidate pool, does conditional selection improve answer quality relative to simple ranking at comparable first-use cost? This comparison would clarify whether conditional evaluation provides an advantage beyond candidate availability.

Q4. What evidence-label availability, query volume, and workload stability does the intended deployment assume? Under these assumptions, how do amortized costs compare with Full Graph and simpler strategies when design overhead and retrieval costs are included?

3 comments
AuthorsOpenReview ↗

Response to Reviewer RmdB

Thank you for the careful review. We ran new experiments to address your main concern: whether WARP-G is worth the effort of choosing how many graphs to build.


Q1 / W1: Baselines, smaller budgets, and how to choose a budget

KET-RAG keeps lexical/semantic neighbors, PageRank-based KG skeleton selection, and keyword-bipartite indexing. G2ConS keeps concept co-occurrence, PageRank scores, and core-concept pruning. Both share the HippoRAG2 engine and reader. The original 80% coverage was a reference, not a tuned setting. At 80%, graph construction still uses 84.4–93.2% of Full Graph tokens (paper Table 7).

We have now tested smaller coverage levels. Each cell shows answer F1 / first-use graph tokens (M):

Coverage2Wiki KET-RAG2Wiki G2ConSMuSiQue KET-RAGMuSiQue G2ConS
5%62.34 / 2.48766.07 / 2.64342.20 / 3.40943.57 / 3.439
10%62.08 / 3.16164.21 / 3.57342.95 / 4.59843.48 / 4.679
20%61.08 / 4.22862.53 / 4.93642.33 / 6.74144.27 / 7.024
40%59.08 / 6.10961.17 / 7.000——

G2ConS at 5% does well on 2Wiki: more graph does not always mean better answers. The best coverage also depends on the workload. WARP-G tries to make this choice using past queries, evidence missed by passage retrieval, and the gain we measure from each graph.

We tested frequency ranking with fixed counts k=0,1,2,4,8k=0,1,2,4,8, and a rule that picks kk using 200 labeled validation queries. We count its construction and validation costs. WARP-G values are from the paper; matching-frequency costs are from the paper, F1 from saved original answers, and other counts from new runs.

DatasetMethodF1Retained (M)First-use (M)
2WikiFrequency, k=1k=164.491.9462.786
2WikiFrequency, k=2k=262.642.6954.391
2WikiFrequency, k=8k=861.858.71313.696
2WikiValidation-selected—018.491
2WikiWARP-G64.210.7986.713
MuSiQueFrequency, k=2k=243.322.9253.826
MuSiQueFrequency, k=8k=844.9310.56313.821
MuSiQueValidation-selected—015.811
MuSiQueWARP-G44.172.5988.429

A small fixed budget can work well. The key question is how to choose that budget before seeing the test answers. This is exactly what WARP-G aims to address. Our validation search costs 18.491M / 15.811M graph tokens on 2Wiki/MuSiQue. WARP-G's design costs are 6.068M / 7.538M (paper Table 6). It tests only promising regions, keeps the indexes it has built, and stops when another graph no longer helps.


Q2 / W2: Stability across selection samples

Each local probe uses up to 64 queries linked to its region. Final selection compares graphs on the same 32 or fewer design queries. We repeated that step 20 times while keeping the regions, built graphs, shortlist, and test set unchanged:

DatasetDifferent graph setsNumber of runs with k=0/1/2k=0/1/2First-use (M), mean ± SD
2Wiki56 / 11 / 36.491 ± 0.683
MuSiQue87 / 10 / 37.751 ± 0.394

The chosen regions sometimes differ, but all 40 runs select at most two graphs. We also reran the full pipeline with new data splits. Compared with the matching Hybrid runs, F1 changed by 0.00 and +0.19 points on 2Wiki and +1.36 points on MuSiQue. For the original paper runs, paired bootstrap intervals for WARP-G minus Hybrid are +1.15 points on 2Wiki (95% CI [-1.04, 3.45]) and +1.10 on MuSiQue ([-1.29, 3.58]).


Q3: Same candidate graphs

We gave all methods the same probed graphs (3 on 2Wiki, 6 on MuSiQue). On 2Wiki, conditional selection chose r0007, while frequency chose r0000. On MuSiQue, both chose {r0012, r0022}. Our method checks whether a new graph still adds value after the other selected graphs are taken into account. In the paper's matched-capacity comparison on 2Wiki, WARP-G reaches 80.00 ER@10 versus frequency's 79.13, with 0.798M versus 1.946M retained construction tokens.


Q4: Practical use and long-term cost

We use 800 historical, evidence-labeled design queries per workload and assume the query mix stays fairly stable. Built indexes can then be reused. At 100k queries, a cost projection gives 329.0M vs. 622.7M graph tokens on 2Wiki and 453.2M vs. 632.3M on MuSiQue (WARP-G vs. Full Graph), counting design and online graph retrieval. In the original experiments, first-use graph tokens fall by 38.7–67.2%.

We hope these experiments show why deciding what to build matters, not just building smaller graphs. If you have any other questions, we would be happy to discuss them.

Reviewer_RmdBOpenReview ↗

Reviewer's response for the authors' rebuttal

Thank you for the additional experiments. I acknowledge I have read the rebuttal posted by the authors to my review. I still prefer to retain my original rating for the following reasons: (1) Some small fixed budget G2ConS settings achieve higher F1 on both 2Wiki and MusiQue datasets at lower first-use cost. (2) The validation-selected baseline does not report F1, despite its high first-use cost. Therefore, it seems that WARP-G’s overall quality-cost benefit remains unclear.

Thank you for the follow-up. We agree that the low-budget G2ConS results are strong. We would like to clarify the problem WARP-G solves and complete the missing F1 results.

1. The validation-selected baseline uses Hybrid after paying for the selection process.

Here, k is the number of regional graphs retained for serving. The validation policy chooses k = 0 on both datasets, so its final system is exactly Hybrid. Its F1 is therefore 63.06 on 2Wiki and 43.07 on MuSiQue. We should have reported these values explicitly.

DatasetMethodAnswer F1First-use graph tokens (M)
2WikiValidation-selected frequency63.0618.491
2WikiWARP-G64.216.713
MuSiQueValidation-selected frequency43.0715.811
MuSiQueWARP-G44.178.429

Although this baseline ultimately keeps no graphs, it has already spent tokens building and evaluating candidates. WARP-G includes the same types of costs in its accounting. Against this evaluated selection policy, it obtains higher F1 point estimates with 63.7% and 46.7% fewer first-use graph tokens.

2. WARP-G provides a way to make the deployment decision before seeing test results.

The favorable G2ConS settings differ: 5% coverage on 2Wiki and 20% on MuSiQue. These results show that a small graph can work well, but they do not tell a deployer which coverage to choose for a new workload. If a suitable budget is already known, a fixed-budget method can indeed be preferable.

WARP-G addresses the case where that choice still needs to be made. It uses historical queries to identify promising regions, builds a limited set of regional graphs to measure their benefit, and checks whether adding a graph still helps alongside those already selected. It then reuses the selected indexes, or keeps passage-only retrieval when no evaluated addition helps.

The methodological contribution is this complete decision process: choosing graph locations and retained capacity while limiting and accounting for the cost of making that choice. We do not claim that it beats every fixed-budget configuration. Our additional comparison shows a concrete benefit over the evaluated validation-based capacity selector, while the G2ConS sweep shows strong alternatives at particular budgets. We hope the assessment can consider both the quality of individual configurations and the method used to choose a deployment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.