acceptodds
Under review as a conference paper at ICLR 2027

QESChunker: A Single Objective Unifies Overlapping and Non-Overlapping Chunking for RAG

Abstract

Chunking is a fundamental preprocessing step in retrieval-augmented generation (RAG), yet most existing chunkers rely primarily on heuristics. We introduce the Question–Evidence-Supervised Chunker (QESChunker), to our knowledge the first method for constructing static chunk indices from question–evidence supervision. QESChunker constructs document-specific chunk indices by optimizing over human-annotated or LLM-generated question–evidence pairs, without access to held-out test queries. We formulate non-overlapping and overlapping index construction through a unified question–evidence utility-maximization objective, turning chunking into a supervised combinatorial optimization problem. Under the non-overlap constraint, the objective decomposes over chunks and can be optimized exactly in polynomial time by dynamic programming. When overlap is allowed, the objective becomes monotone submodular; exact optimization is NP-hard, but our greedy achieves a -approximation. Across two self-constructed and three public datasets, both variants outperform existing chunking methods in complete-evidence recall in nearly all evaluated settings, under the same overlap regime and a fixed hybrid retriever. These results recast chunking as supervised, budget-aware index construction for evidence retrieval.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
retrieval-augmented generation, document chunking, question-evidence supervision, combinatorial optimization, submodular maximization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_yhrMOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper constructing static chunk indices from question–evidence supervision. QESChunker constructs document-specific chunk indices by optimizing over human-annotated or LLM-generated question–evidence pairs, without access to held-out test queries. The main idea is turning chunking into a supervised combinatorial optimization problem. The method show good emperical results.

Critical strengths
  1. The paper propose an interesting and novel method.
  2. It provide clear theoretical analysis.
  3. The experiments show good improvements over diverse baselines across multiple datasets.
Critical weaknesses
  1. The hit-gated coverage objective assumes that if a chunk contains at retriever's top-kk hit atomic block, the chunk has a chance of being retrieved. But retrieval is eventually performed on chunk-level. Then there is a mismatch. Is there any evidence to support this assumption? I think this is important because this is an important assumption supporting the design.
  2. There are some predefined hyperparameter. It's unclear how sensitive it is to the hyperparameter value change. For example, The non-overlapping DP algorithm requires the number of chunks KK, rather than determining it adaptively based on document structure and evidence distribution. 3.The overlapping chunk selection formulation measures indexing cost by the total number of atomic blocks But I think there can be some mismatch to actual cost which is highly depens on token counts. I think the setting is a bit unrealistic.
  3. The experiments primarily evaluate complete-evidence recall. While this metric does not directly show improvements in answer quality. Especially it's unclear to me if the assumption if a chunk contains at retriever's top-kk hit atomic block, the chunk has a chance of being retrieved is robust or not. Is the retriever doesn't work, complete-evidence recall doesn't mean anything to me.
  4. All experiments use the same embedding–BM25 hybrid retriever. It remains unclear if we change the retreiver, if this method still works
Questions for authors

I think it will be helpful to report end-to-end computational cost. Generating Question–Evidence pairs can be expensive.

2 comments
AuthorsOpenReview ↗

Response to Reviewer yhrM

We thank the reviewer for comments.

Q End-to-end computational cost

In the experiments, we generate N5\dfrac{N}{5} Question–Evidence pairs for each article, where NN denote the number of atom blocks of that article. The medium number of atom blocks of qasper, peerqa and privacyqa is about 400, so we generate about 80 Question–Evidence pairs in medium. Using API of qwen-3.7 flash with 16 concurrency, we need several minutes. Actually, we also test the relation between the number of Question–Evidence pairs and the recall in both nonoverlap and overlap assumption in QASPER. The result is as follow.

# of Q-E pairs0%5%10%20%40%60%80%100%
Recall at 10% (Nonoverlap)0.59720.59440.59660.59930.60200.60760.61060.6197

We find that in 0%(only structural information), the recall is very near paragraph chunker. Between 0% and 40% Question-Evidence pairs, the recall is fluctuating. However, above 40%, recall have a obvious increasing and in 100% it has about 2.25% earning against 0%.

# of Q-E pairs0%5%10%20%40%60%80%100%
Recall at 10% (Overlap)0.57300.56620.55660.55750.56840.57700.58330.5943

We also find that between 0% and 40% Question-Evidence pairs, the recall is fluctuating. And however, above 40%, recall have a obvious increasing and in 100% it has about 2.13% earning against 0%.

Thank you for the response. But I think my main concern(Critical Weaknesses listed above) remains unresolved. I will keep my score.

Reviewer_fwZ7OpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies document chunking for RAG, aiming to construct a static chunk index that improves evidence retrieval under a limited context budget. The authors argue that existing chunking methods largely rely on heuristics rather than explicitly optimized from question–evidence supervision. They propose QESChunker, which uses document-specific question–evidence pairs to optimize chunk boundaries. Its objective combines evidence coverage, gated by the presence of question-relevant atomic blocks within a chunk. Under two setting discussed from the paper, non-overlapping and overlapping, the author proposed two algorithms. Based on the empirical performance on five datasets, authors show that QESChunker demonstrate a relatively high performance gain from most settings covered by the paper.

Critical strengths
  1. The proposed method QESChunker uses question–evidence supervision to provide a task-oriented objective for chunk construction. The optimization explicitly connecting chunking decisions to evidence retrieval under a limited context budget. It can a practically useful alternative to purely heuristic text segmentation method.

  2. The paper provides a theoretical formulation of non-overlapping and overlapping chunk construction. For the non-overlapping setting, the authors derive an exact dynamic programming algorithm. For the overlapping setting, they establish NP-hardness and apply a greedy algorithm with a provable approximation guarantee.

Critical weaknesses
  1. The performance gain is marginal even compared with simple paragraph-based chunking. In the non-overlapping setting, the proposed method, QESChunker, improves over this naive heuristic baseline by less than two percentage points across all reported settings on Congress, QASPER, and PeerQA, despite being optimized with extra supervision. Without further analysis of the additional cost and qualitative results, it is hard to judge the practical contribution of these performance gains.

  2. The evaluation scale is too small. The two self-curated datasets only has one passage, so having hundreds of questions does not provide sufficient document diversity. The paper also lacks evaluation on large-scale cross-document retrieval, which makes it hard to assess whether the proposed method can generalizes to broader domain.

  3. QESChunker is optimized for each document using a predefined set of question–evidence pairs, which may make preprocessing costly when the preparation of these pairs is taken into account. However, the paper does not sufficiently report the total preprocessing cost. Also, the method seems is optimizing for a question bank as a prior. It needs further analysis on the method's robustness and generalization after chunking.

Questions for authors
  1. Could the authors evaluate the proposed chunking method on larger-scale question-answering datasets, such as HotpotQA, in a cross-document retrieval setting?
  2. Could the authors provide a cost and efficiency analysis of the preprocessing stage, including the preparation of question–evidence pairs?
2 comments
AuthorsOpenReview ↗

Response to Reviewer fwZ7

We thank the reviewer for comments.

Q1 HotpotQA dataset

We thank the reviewer for this suggestion and agree that HotpotQA is a valuable benchmark for multi-hop and cross-document question answering. We would like to respectfully clarify, however, that the central claim of our paper concerns the effect of the proposed chunking method in the QA setting considered in this work, rather than state-of-the-art performance in large-scale cross-document retrieval. Our evaluation already covers five datasets, three of which are public benchmarks, and these datasets collectively provide a sufficiently broad basis for testing the main claims of the paper. We therefore believe that an additional evaluation on HotpotQA is not a necessary requirement for supporting our contribution. Moreover, evaluating the method on HotpotQA in a cross-document retrieval pipeline would jointly change the retriever, the distribution of evidence across documents, and the multi-hop reasoning requirements, making it difficult to attribute performance differences specifically to the chunking strategy, which is the variable of interest in our study. To avoid overclaiming, we will clarify the scope of our evaluation in the revised manuscript and discuss extension to large-scale cross-document retrieval benchmarks such as HotpotQA as an important direction for future work. We hope this clarification addresses the reviewer’s concern while keeping the evaluation aligned with the central contribution of the paper.

Q2 cost and efficiency analysis

In the experiments, we generate N5\dfrac{N}{5} Question–Evidence pairs for each article, where NN denote the number of atom blocks of that article. The medium number of atom blocks of qasper, peerqa and privacyqa is about 400, so we generate about 80 Question–Evidence pairs in medium. Using API of qwen-3.7 flash with 16 concurrency, we need several minutes. Actually, we also test the relation between the number of Question–Evidence pairs and the recall in both nonoverlap and overlap assumption in QASPER. The result is as follow.

# of Q-E pairs0%5%10%20%40%60%80%100%
Recall at 10% (Nonoverlap)0.59720.59440.59660.59930.60200.60760.61060.6197

We find that in 0%(only structural information), the recall is very near paragraph chunker. Between 0% and 40% Question-Evidence pairs, the recall is fluctuating. However, above 40%, recall have a obvious increasing and in 100% it has about 2.25% earning against 0%.

# of Q-E pairs0%5%10%20%40%60%80%100%
Recall at 10% (Overlap)0.57300.56620.55660.55750.56840.57700.58330.5943

We also find that between 0% and 40% Question-Evidence pairs, the recall is fluctuating. And however, above 40%, recall have a obvious increasing and in 100% it has about 2.13% earning against 0%.

Thank you for providing additional experimental results to address my concerns. I have raised my score accordingly. However, my main concerns remain unresolved.

Reviewer_YSBtOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces the Question–Evidence-Supervised Chunker, QESChunker, which the authors describe as the first method for constructing static chunk indices from question–evidence supervision. QESChunker constructs document-specific chunk indices by optimizing over human-annotated or LLM-generated question–evidence pairs, without access to held-out test queries. They model a document as a sequence of atomic blocks and define a two-term objective comprising a hit-gated weighted coverage term and a tiered structural term that rewards chunks ending at paragraph, sentence, or clause boundaries. They formulate non-overlapping and overlapping index construction through this unified question–evidence utility-maximization objective, turning chunking into a supervised combinatorial optimization problem. They evaluate both variants on self-constructed Chinese policy corpora and public English corpora and report that both variants outperform existing chunking methods in complete-evidence recall in nearly all evaluated settings, under the same overlap regime and a fixed hybrid retriever.

Critical strengths
  • The formulation is clean, and the theory is easy to follow. One hit-gated coverage objective gives an additive per-chunk utility under a partition, so an exact DP applies, and a weighted coverage plus modular prior under overlap, so standard budgeted submodular tools apply. The NP-hardness reduction for contiguous intervals is the one genuinely new theoretical piece, and it is carefully done in the Appendix.

  • The empirical comparison is broad on the baseline side. The paper covers fixed-length, paragraph, TextTiling, semantic clustering, LLM-based LumberChunker, sentence-window, parent-document, and auto-merging. The budget curves also show a consistent pattern where the gain is largest at tight context budgets, which is the regime practitioners care about.

Critical weaknesses
  • The comparison is between supervised and unsupervised. QESChunker fits boundaries per document using question–evidence pairs from that same document, either cross-validated questions on the self-built corpora or LLM-generated questions with evidence IDs on the public ones. Every baseline sees no supervision at all. Without a baseline that uses the same supervision, it is not possible to tell whether the gain comes from the optimization framework or just from having document-specific supervision.

  • The experimental protocol has leakage and matching issues. The structural weight, length bounds, and whole-paragraph exception are all tuned on the public-health white paper, which is also one of the five evaluation corpora and the one with the largest reported gains. The self-built corpora are a single document each, and the Ethics Statement says their evidence is LLM-annotated, so fitting and test labels come from the same kind of source.

  • For the overlap setting, the index budget is three times the document length, while the fixed-window baselines index roughly one and a quarter times the document, so the claim of the same overlap regime is not quite right.

  • Matched returned context is not the same as matched index size. Several margins are under one point on PeerQA, PrivacyQA, and the Congress report. All results are complete-evidence recall only, with no answer-level evaluation, which the authors acknowledge.

Questions for authors
  • Add at least one baseline that uses the same fitting question. If QESChunker still wins clearly, my assessment would improve.

  • What is the actual index size, in atoms indexed, for each overlap method in Table 4? How does QESChunker greedy perform when its budget is matched to the fixed-window or sentence-window index size?

  • The hit set is built from top-κ atom similarity with the question embedding, while evaluation uses a hybrid BM25 and embedding retriever over chunks with section-title prefixes. How sensitive are the results to this mismatch, and to κ?

  • How many fitting questions are needed before QESChunker beats the paragraph baseline? A curve of recall against the amount of supervision would show where the method is useful and would address the cold-start limitation directly.

  • An end-to-end answer quality evaluation on at least one public corpus would make the practical value much clearer.

2 comments
AuthorsOpenReview ↗

Response to Reviewer YSBt(1/2)

We thank the reviewer for comments.

Q1 supervised baseline

We thank the reviewer for raising the question of supervision-matched comparisons. Introducing question–evidence supervision into static chunk-index construction is itself a central contribution of this work, together with the utility formulation and its combinatorial optimization. The supervision is therefore an integral part of the proposed approach.

In reviewing the relevant literature, we have not identified a directly applicable prior method that uses document-specific question–evidence pairs to optimize contiguous chunk boundaries for a static index. Existing trained chunkers typically learn from structural boundaries or segmentation demonstrations, while query-adaptive methods address a different inference setting. These methods do not directly provide the requested same-supervision comparison. Because of this reason, our experiments evaluate the proposed approach as a whole against established chunking strategies.

We are very sorry that we can not support another supervised baseline.

Q2 Index size

Among those baseline we compare in the overlap assumption, fixed window has 1.25n index size, and parent document and auto merging both have 1.0n index because of the special retrieval strategy.

We claim that we use three times of document length, but by testing again, we find at 1.25n index, we also have essential advantage those baseline, even at 1.0n index, where we can not cover all atom block in document, we beat those baselines which have 1.0n index. The result in QASPER is as below.

MethodIndex sizerecall at 10%
Our method at 1.25n1.25n0.5943
Our method at 1.0n1.0n0.5844
Fixed window L=bl1.25n0.5531
Fixed window L=2bl1.25n0.4862
Sentence-Window k=11.0n0.5581
Sentence-Window k=21.0n0.5328
Parent-Document1.0n0.5572
Auto-Merging1.0n0.5363

Q3 Mismatch between small-to-big retriever and hybrid retriever

From our design perspective, the mismatch is a deliberate tractability trade-off, not an evaluation inconsistency.

Firstly, it makes the objective combinatorial. The atom-level hit set discretizes retrieval feasibility, so the objective becomes weighted coverage plus a modular prior. This preserves additivity for the non-overlap DP and monotone submodularity for the overlap greedy. Embedding the full hybrid top-kk behavior inside the objective would destroy these structures.

Secondly, it is computationally necessary. The number of possible chunks is O(nL)O(nL), worst case O(n2)O(n^2). Computing and storing embeddings for all candidate chunks, then repeatedly evaluating hybrid scores inside the optimizer, is infeasible at our scale. The hit set needs only n atom embeddings per document, and chunk coverage can then be computed incrementally.

Thirdly, the proxy is conservative and compatible. The hit set uses the same embedding model as the evaluation retriever’s embedding component. A chunk with no hit atom is unlikely to be surfaced; a chunk with a hit atom retains real retrieval signal. BM25 adds complementary lexical evidence rather than contradicting the gate.

Last but not least, the hybrid retriever validates it empirically. Under the same hybrid retriever, overlap regime, and returned-context convention, QESChunker leads in 9/10 non-overlap and 10/10 overlap display cells, with the largest gains at tight budgets. This indicates the supervision signal transfers to chunk-level hybrid retrieval.

In conclusion, the mismatch is intentional: it converts chunking into a solvable supervised combinatorial optimization problem while avoiding the impossible cost of embedding and simulating every possible chunk. The hybrid-retriever experiments verify the resulting indices in the actual retrieval setting.

AuthorsOpenReview ↗

Response to Reviewer YSBt(2/2)

We thank the reviewer for comments. We continue with the last two questions.

Q4 The number of supervised questions

We measure the relation between the number of Question-Evidence Pair and recall in both nonoverlap and overlap assumption in QASPER. The result is as below, where the first line denote the percent of all pairs we used in the main experiment.

# of Q-E pairs0%5%10%20%40%60%80%100%
Recall at 10% (Nonoverlap)0.59720.59440.59660.59930.60200.60760.61060.6197

We find that in 0%(only structural information), the recall is very near paragraph chunker. Between 0% and 40% Question-Evidence pairs, the recall is fluctuating. However, above 40%, recall have a obvious increasing and in 100% it has about 2.25% earning against 0%.

# of Q-E pairs0%5%10%20%40%60%80%100%
Recall at 10% (Overlap)0.57300.56620.55660.55750.56840.57700.58330.5943

We also find that between 0% and 40% Question-Evidence pairs, the recall is fluctuating. And however, above 40%, recall have a obvious increasing and in 100% it has about 2.13% earning against 0%.

Q5 End-to-end Evaluation

We measure the end-to-end Answer-f1 in both nonoverlap and overlap assumption in QASPER. The result is as below. In order to compare conveniently, we show the recall and Answer-f1 in the second column and in the third column respectively.

Method(Nonoverlap)recall at 10%Answer-f1
Our method0.619739.164
Equal-length split0.517636.504
Segment-by-paragraph0.586037.746
TextTiling0.515636.970
ClusterSemanticChunker0.486235.234
LumberChunker0.535636.309
Method(Nonoverlap)recall at 10%Answer-f1
Our method at 1.25n0.594340.671
Fixed window L=bl0.553136.349
Fixed window L=2bl0.486235.333
Sentence-Window k=10.558139.681
Sentence-Window k=20.532837.843
Parent-Document0.557237.865
Auto-Merging0.536339.718

Our method in nonoverlap and overlap assumption list first in both recall and Answer-f1 in each assumption.

We would like to clarify two points. First, the original question–evidence pairs used for boundary fitting were lost. To conduct the additional experiments, we regenerated these question–evidence pairs following the same protocol. Consequently, some numerical results may differ slightly from those reported in the submitted version; however, the overall conclusions remain unchanged. Second, due to the rebuttal time limit, we conducted additional experiments only on QASPER for the specific concerns raised by the reviewers. We will complete the corresponding experiments on the remaining datasets in the final version.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.