- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes TREEFUSE for query-adaptive context construction when a full document already fits within a long-context window. TREEFUSE constructs a source-grounded semantic tree and selects context using semantic node scores, lexical matching, and ancestor closure. The selected contexts retain local source text and are reused across four answer models. Validation covers LongBench tasks, LaRA-32K, several context-construction baselines, component studies, context-length analyses, and an input-side cost model.
Critical strengths
-
The paper addresses an important long-context question with a complete and source-grounded system. TREEFUSE uses reusable semantic trees for routing but returns local source text to the reader. This reduces dependence on generated summaries as final evidence and makes the method relevant to long-document QA.
-
The evaluation is broad and transparent. The same constructed contexts are reused across four answer models. The paper reports average gains with substantial context reduction, provides several ablations, and explicitly states that budget matching is approximate and the cost model is an input-side proxy.
Critical weaknesses
- The complete-system comparison does not isolate semantic-tree design from scorer and preprocessing choices. The evaluated systems use different architectures, training procedures, model generations, and query-time preprocessing protocols. The results show that the configured TREEFUSE system performs well, but they do not establish how much of the advantage comes from the tree representation rather than scorer capability or preprocessing work. Some budget-matched comparisons also match only average characters rather than per-query reader tokens.
- Potentially score-changing statistical analysis. Are the LaRA and Qasper gains stable under question-level paired bootstrap or randomization tests?
- Potentially score-changing control. How sensitive are the results to the constructor and semantic scorer? For compatible hierarchical methods, does TREEFUSE retain its advantage with a shared scorer or matched query-time preprocessing work?
Questions for authors
- Budget clarification: What happens when the BM comparisons use paired per-query token budgets under each answer model's tokenizer rather than average retained characters?
- Novelty clarification: What is the key technical difference from PageIndex, DISRetrieval, and the complete HiChunk pipeline, and why were these systems not compared under a common protocol?
4 comments
Rebuttal by Authors (Part 2/3)
(Continued from Part 1)
W2 & Q1: Paired Per-Query Token Budget Accounting — Frozen BM contexts already match TreeFuse within 0.45%–4.10% mean absolute per-query tokens
We apologize for the misunderstanding; our actual BM implementation achieves near-exact character-level budget matching on a per-question basis. In our ablations (§4.3, Table 4; Appendix E.1), BM targets TreeFuse's per-query character budget (adding random unselected nodes for Semantic-only (BM), or adjusting the lexical node threshold with ancestor closure for Lexical-only (BM)). Because whole nodes are selected without mid-node truncation, small per-query residuals occur. Measuring the frozen reader inputs with the Qwen3.8-Max tokenizer, the mean absolute per-query relative gap, , is very tight:
Semantic-only (BM): Mean absolute gap is 1.20% chars / 1.69% tokens on HotpotQA, and 0.27% chars / 0.45% tokens on 2WikiMQA.Lexical-only (BM): Mean absolute gap is 0.73% chars / 3.08% tokens on HotpotQA, and 1.56% chars / 4.10% tokens on 2WikiMQA.
Thus, on the frozen contexts, per-query reader token budgets already align within 0.45%–4.10% while preserving the exact QA-F1 gains in Table 4 (+2.72 over Semantic-only (BM) and +6.16 over Lexical-only (BM)). We now describe the per-query BM setting in §4.3 and Appendix E.1.
W3: Question-Level Statistical Significance — Paired bootstrap & sign-flip tests confirm significant gains in 19/24 comparisons, including 5/5 over LaRA baselines and 11/12 on multi-hop QA
To test question-level stability beyond 3-run decoding variance, we ran paired bootstrap (50,000 resamples) and two-sided sign-flip randomization tests on the frozen results. For each question, we average the 3 runs per model and equally average paired differences across the four fixed answer models ( denotes ; 95% CIs in percentage points):
| Baseline vs. TreeFuse | Qasper (QA-F1) | HotpotQA (QA-F1) | 2WikiMQA (QA-F1) | LaRA (Judge Acc.) |
|---|---|---|---|---|
| LongLLMLingua | CI [19.18, 27.09], | CI [10.73, 18.87], | CI [12.92, 20.30], | CI [15.26, 20.34], |
| Provence | CI [0.94, 5.34], | CI [2.76, 7.44], | CI [8.00, 14.53], | CI [5.87, 9.35], |
| HiChunk | CI [-0.11, 2.93], | CI [2.62, 6.85], | CI [1.97, 4.91], | CI [4.59, 8.19], |
| BRIEF-Pro | CI [-0.43, 4.25], | CI [-0.29, 6.22], | CI [1.08, 6.63], | CI [5.84, 9.98], |
| LongRefiner | CI [15.18, 24.38], | CI [9.24, 17.73], | CI [22.59, 32.78], | CI [37.17, 43.60], |
| Raw (Full Context) | CI [-0.004, 1.99], | CI [0.49, 2.99], | CI [1.03, 3.81], | CI [-0.67, 1.30], |
- Overall & Multi-hop: Mean differences are positive in all 24 comparisons, and 19/24 are statistically significant (with document/question-package cluster sensitivity analyses confirming the exact same 19 conclusions), including 5/6 on HotpotQA and 6/6 on 2WikiMQA (beating both Raw and compression baselines).
- LaRA & Qasper stability: On LaRA, TreeFuse's gains over all 5 compression baselines (5/5) are highly significant (). On Qasper, gains over LongLLMLingua, Provence, and LongRefiner are significant (3/6). We now present these results in §4.2 and Appendix C.1.
(Continued in Part 3)
Rebuttal by Authors (Part 3/3)
(Continued from Part 2)
Q2: Technical Distinctions & Common-Protocol Evaluation with PageIndex, DISRetrieval, and Complete HiChunk — TreeFuse outperforms all three hierarchical systems by +1.33 to +3.20 Overall QA-F1
Key technical differences in selection and evidence assembly:
- PageIndex performs iterative LLM navigation over a ToC-like tree; DISRetrieval parses trained RST discourse trees for dense hierarchical sentence retrieval; and Complete HiChunk uses dense retrieval (
BGE-M3) followed byAuto-Merge(which expands a parent node to include all its child chunks once a threshold is met). - TreeFuse instead decouples routing metadata from source evidence on a shared source-grounded tree: structure-conditioned semantic scoring and subunit BM25 attribution independently nominate nodes, and ancestor completion adds only each ancestor's local introductory span—preserving structural framing and exact lexical hits without pulling in irrelevant sibling subtrees.
New common-protocol evaluation on LongBench (§4.1–§4.2, Table 1; Appendix B.3): We evaluated PageIndex and DISRetrieval alongside Complete HiChunk, Raw, and TreeFuse under the exact same aligned question sets and three-run evaluation protocol across all four answer models (QA-F1; bold = best). The HiChunk results in the main table of our originally submitted PDF already used the original HiChunk paper's complete pipeline and settings; only the construction ablation adapts HiChunk units to our scorer.
| Dataset | Method | Qwen3.8-Max | DeepSeek-V4-Pro | Kimi-K2.6 | GLM-5.3 | Overall |
|---|---|---|---|---|---|---|
| Qasper | Raw | 48.89 | 51.20 | 50.35 | 46.61 | 49.26 |
| PageIndex | 49.44 | 47.91 | 47.73 | 43.78 | 47.21 | |
| DISRetrieval | 45.49 | 44.21 | 43.70 | 42.65 | 44.01 | |
| HiChunk (Complete) | 49.70 | 49.88 | 49.24 | 46.85 | 48.92 | |
| TreeFuse (Ours) | 50.96 | 51.35 | 50.62 | 48.05 | 50.25 | |
| HotpotQA | Raw | 75.69 | 72.20 | 59.47 | 70.68 | 69.51 |
| PageIndex | 72.93 | 70.80 | 60.15 | 68.26 | 68.03 | |
| DISRetrieval | 65.23 | 53.66 | 45.36 | 57.29 | 55.38 | |
| HiChunk (Complete) | 70.53 | 69.57 | 58.16 | 68.21 | 66.62 | |
| TreeFuse (Ours) | 77.39 | 73.64 | 63.50 | 70.38 | 71.23 | |
| 2WikiMQA | Raw | 84.19 | 79.28 | 47.44 | 71.62 | 70.63 |
| PageIndex | 82.33 | 78.84 | 50.03 | 72.98 | 71.05 | |
| DISRetrieval | 74.82 | 53.67 | 35.89 | 56.33 | 55.18 | |
| HiChunk (Complete) | 80.65 | 79.54 | 47.75 | 70.79 | 69.69 | |
| TreeFuse (Ours) | 84.85 | 79.00 | 54.44 | 74.08 | 73.09 |
TreeFuse achieves the highest four-model Overall QA-F1 on all three datasets, beating the strongest hierarchical baseline on each task by +1.33 on Qasper (50.25 vs. 48.92 for HiChunk), +3.20 on HotpotQA (71.23 vs. 68.03 for PageIndex), and +2.04 on 2WikiMQA (73.09 vs. 71.05 for PageIndex). We have integrated these comparisons into the revised manuscript.
Closing Remark
Thank you again for your careful review and for identifying the controls needed for a fair assessment. We have separated constructor and scorer effects through controlled ablations, clarified per-query budget accounting, added paired statistical tests, and compared complete hierarchical baselines while distinguishing native HiChunk from its scorer-adapted ablation. We hope these results address your concerns. If you find the updated evidence persuasive, we would sincerely appreciate your consideration of a higher rating.
Eager to discuss our rebuttal for Submission53818
Dear Reviewer,
Thank you again for your time and constructive feedback. We are following up on our rebuttal and would greatly appreciate your thoughts on whether the following clarifications and results address your concerns:
- Tree construction versus scorer effects (W1, W4): Shared-scorer controls show that our tree representation adds +2.78 average QA-F1 over the HiChunk adapter. A lightweight 4B scorer also retains higher task-level average scores than Raw and the evaluated compression baselines.
- Budget fairness (W2, Q1): Token accounting on the frozen reader inputs shows mean absolute per-query token-budget gaps of only 0.45%–4.10%, while preserving the reported budget-matched ablation gains.
- Statistical reliability (W3): Paired bootstrap and randomization tests establish significant gains in 19/24 comparisons, including 11/12 on multi-hop QA.
- Hierarchical baseline comparisons (Q2): New evaluations of PageIndex and DISRetrieval, alongside complete HiChunk, show that TreeFuse achieves the highest four-model average QA-F1 on all three datasets, exceeding the strongest hierarchical baseline by 1.33–3.20 points.
Further details are available in our responses above. We would greatly appreciate hearing whether your concerns have been addressed. If any points remain unresolved or need further clarification, please feel free to let us know—we would be happy to discuss them.
Rebuttal by Authors (Part 1/3)
Thank you for your positive assessment and for highlighting the exact controls needed to isolate our tree representation, scorer, and budget accounting. We address your concerns in three parts: Part 1 covers the constructor/scorer controls (W1, W4); Part 2 covers per-query token budgets (W2, Q1) and paired statistical tests (W3); Part 3 covers new comparisons with
PageIndexandDISRetrieval(Q2).W1 & W4 (Part A): Isolating Constructor vs. Scorer under a Shared Protocol — Our scorer/selector adds +2.75 QA-F1 onto HiChunk units, and our tree constructor adds a further +2.78 QA-F1 under the exact same scorer
To separate the tree constructor (which defines node boundaries, parent–child links, and metadata) from the query-time selector (semantic scoring, adaptive cutoffs, lexical union, and ancestor closure), we compare representation and construction variants while holding the
Qwen3.6-35B-A3Bscorer andQwen3.8-Maxreader strictly fixed (§4.3; 3-run mean QA-F1):35B-A3B+ BM2535B-A3B+ BM2535B-A3B+ BM2535B-A3B+ BM2535B-A3B+ BM2535B-A3B+ BM25This controlled progression cleanly separates the two components:
35B-A3Bscorer and selector, full TreeFuse outperformsHiChunk adapterby +2.78,Multi-stageby +2.22,TreeFuse (flat)by +3.62 (confirming hierarchy-aware routing and ancestor completion), and flat chunkers (Fixed 200/Chonkie) by +5.00 / +4.57.W4 (Part B): Sensitivity to Semantic Scorer Scale — Replacing
35B-A3B(2.86B active MoE) with a denseQwen3.5-4Bscorer still beats Raw and all compression baselinesNote that our default
Qwen3.6-35B-A3Bis an MoE model with only 2.86B activated parameters per token (Appendix C.4). To test sensitivity to scorer capacity while holding the constructed trees fixed, we replaced the query-time scorer with a lightweightQwen3.5-4Bmodel across all four readers (§4.3, Table 7; 3-run mean QA-F1):With the 4B scorer, TreeFuse still improves over Raw on all three tasks (+0.94, +0.47, +0.84) and surpasses the strongest baseline on every task (+1.28, +1.80, +1.78), while
35B-A3Badds a further +0.05 to +1.62 points.(Continued in Part 2)