acceptodds
Under review as a conference paper at ICLR 2027

RAISE: Loss-Aware, Direction-Free Retrieval Over Hierarchical Documents

Abstract

When querying over thousands of multimodal documents, the evidence a query needs often sits in a table, a chart, or even within the page layout. Recent retrievers therefore build each document into a tree whose internal nodes carry short summaries of their children, and an agent descends it like a table of contents, opening only the promising branch. A summary, however, is written before any query is known and under a length budget, so it cannot keep every detail of its children. We call the resulting limitation the abstraction bottleneck: when a summary omits the detail a query asks about, the agent can no longer tell the right branch from its siblings, and must either guess or open them all. We present Retrieval over Abstraction-Induced Summary Erasure (RAISE), which records the omission caused by summarization and routes on it. Offline, RAISE contrasts every summary against the children it was written from and records what the summary omitted as loss witnesses, each attached to the parent-to-child edge it came from, which together form the Loss Ledger. Online, an agent traverses the documents by extending a map of its traversal so far, and the witnesses serve it in two ways: (i) a witness is itself retrievable, so a query can match it and start its traversal from the edge the witness is attached to, and (ii) the witnesses are drawn on the map, so the agent can judge a branch before opening it. The agentic traversal is also direction-free: it can start traversing at any level of a tree without losing the hierarchy, and extend downward or upward at every step, so no direction has to be fixed before retrieving. Extensive experimental results show that RAISE achieves the best average accuracy on eight multimodal document QA benchmarks in the open-corpus setting, outperforming the strongest of seven competitors by 5.8 points of exact match on average.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Retrieval-Augmented Generation, Multimodal Document Retrieval, Hierarchical Retrieval, Agentic Retrieval, Abstraction Loss, LLM Agent

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_PgBtOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

RAISE records facts that document summaries leave out and keeps a link from each fact to its source. An agent can search these facts and use them to decide which parts of a document to open. On eight document-QA benchmarks where the document is not given, RAISE achieves 43.1% average exact match compared with 37.3% for ViDoRAG.

Critical strengths
  1. Recording omitted facts gives the agent a way to find evidence that would otherwise be hidden by a summary, while the source links let it open the original content (Sections 2 and 4).
  2. The gains span several document collections, and removing the ledger lowers average exact match by 4.0 points, showing that it contributes to the result (Tables 1 and 3).
Critical weaknesses
  1. The no-ledger controls keep summaries written to rely on the ledger, so they do not show whether identifying omissions is better than indexing the same number of ordinary source facts (Appendices D.5 and E).
  2. The answer gate requires the agent to read a cited source but does not check that the source supports its answer, so an incorrect ledger fact may still mislead the agent (Section 4.2; Appendix E).
Questions for authors
  1. How does RAISE compare with indexing the same number of source facts without first checking what the summaries omitted?
  2. Have you checked how often an incorrect ledger fact leads to a wrong final answer even after the agent opens its source?
2 comments
AuthorsOpenReview ↗

Response to Reviewer PgBt (1/2)

We thank the reviewer for the clear reading of the method and for two questions that each point at a measurable quantity. For W1/Q1, we compared the ledger with indexing the same number of ordinary source facts and re-indexed the corpora with summaries that do not rely on the ledger. For W2/Q2, an audit counts how often an incorrect ledger fact leads to a wrong final answer after the agent opens its source. The full tables are in the revised manuscript, which we have uploaded (Table 3, §5.4, §5.5, and Appendices F.2 and F.4).



[W1/Q1: Indexing the same number of ordinary source facts]

W1. The no-ledger controls keep summaries written to rely on the ledger, so they do not show whether identifying omissions is better than indexing the same number of ordinary source facts (Appendices D.5 and E).

Q1. How does RAISE compare with indexing the same number of source facts without first checking what the summaries omitted?

To test whether identifying omissions is better than indexing the same number of ordinary source facts, we added two controls on all eight corpora. (a) Indexing the same number of source facts without first checking what the summaries omitted, which keeps the summaries of the paper. (b) A re-indexing with a plain summarization prompt, so that the no-ledger control keeps summaries that were not written to rely on the ledger.

(a) In the control, we replaced the loss witnesses with facts extracted by one call per internal node with the same backbone, which sees the node's children and not the parent summary and receives no instruction about omissions. Two variants are kept. (i) The same number of facts per edge as the ledger holds there. (ii) The same cap per edge as the witness extraction, eight per child. The facts are embedded, placed in the loss channel and searched exactly as the witnesses are.

Facts stored on an unopened branchEMF1ΔEMstepstime (s)index (MB)
omitted facts (loss witnesses, RAISE)43.156.2-6.5723.519,715
(i) ordinary source facts, same number per edge40.953.6−2.26.9524.719,715
(ii) ordinary source facts, cap 8 per child41.455.6−1.76.7824.233,533
no facts (w/o ledger)39.150.9−4.07.3125.73,613

Identifying omissions is better than indexing the same number of ordinary source facts by 2.2 EM, and a 1.7 times larger index without the check recovers 0.5 EM more.
(i) With the same number of ordinary source facts per edge, accuracy falls to 40.9 EM, 2.2 points below the omitted facts and 1.8 above the no-ledger control (39.1). At the same per-edge count, ordinary source facts scored 40.9 EM and omission-aware witnesses scored 43.1. With the alternative summaries, adding the ledger improved EM from 39.4 to 42.7.
(ii) Under the cap of eight per child, the extractor tries to cover nearly everything in each child, so the index grows to 1.7 times the size of the ledger and accuracy recovers to 41.4 EM, still 1.7 points below the omitted facts.

(b) The summaries of the paper are written to rely on the ledger by sentences of the prompt of §F.2 that allow a summary to leave out what the ledger will hold. We removed such sentences and kept the 120-word budget and every other rule. The witnesses were re-extracted against the new summaries, and the no-ledger control and the full system were run again on all eight corpora.

Summary promptno-ledger control: EMno-ledger control: F1full: EMfull: F1ΔEM (full − control)
§F.2 (summaries written to rely on the ledger)39.150.943.156.2+4.0
plain (such sentences removed)39.451.442.755.8+3.3

The summaries written to rely on the ledger account for 0.7 EM of the ledger's 4.0 EM gain, and the ledger still gains 3.3 EM over a control whose summaries were not written for it: with such summaries the no-ledger control reaches 39.4 EM against 39.1 with the summaries of the paper, and the full system 42.7 against 43.1. Of the 0.7 EM, 0.3 is the handicap of the no-ledger control and 0.4 the full system's benefit from such summaries.

The revised manuscript adds the two rows of (a) and the two configurations of (b) to Table 3, discusses both in §5.4, and gives the two prompts in Appendices F.4 and F.2.

AuthorsOpenReview ↗

Response to Reviewer PgBt (2/2)

[W2/Q2: How often an incorrect ledger fact leads to a wrong final answer after the agent opens its source]

W2. The answer gate requires the agent to read a cited source but does not check that the source supports its answer, so an incorrect ledger fact may still mislead the agent (Section 4.2; Appendix E).

Q2. Have you checked how often an incorrect ledger fact leads to a wrong final answer even after the agent opens its source?

To measure how often an incorrect ledger fact leads to a wrong final answer after the agent opens its source, we audited every ledger fact that the agent followed to its source on 500 queries per corpus, or all queries of a corpus that has fewer. The agent follows a fact when it reads the node the fact points to, after the fact reached the agent either as a result of the loss search or as a label on the map edge into that node. Claude Opus 5.5 compared each followed fact with the source text the agent read and marked it as supported by the source, contradicted by it, or absent from it. A fact counts as incorrect when it is contradicted or absent. We chose a judge stronger than the model that wrote the facts (Qwen3.6-35B-A3B), so that the facts are not judged by their own writer. The final answer is judged by the LLM judge of Appendix D.1.

Ledger facts the agent followed to their sourcenfinal answer: correctfinal answer: wrongfinal answer: wrong-answer ratevalue certified from the source
correct (supported by the source)8,4355,5102,92534.7%44.6%
incorrect (contradicted or absent)1,6161,02559136.6%40.7%
all followed facts10,0516,5353,51635.0%44.0%
Queries, counted once eachnshare
exposed to at least one incorrect ledger fact841
answered wrongly27933.2%
the answer repeats the error of the fact91.1%
the answer is wrong for a different reason26831.9%
the records do not allow a judgment20.2%

After the agent opens the source, a wrong final answer follows an incorrect ledger fact in 36.6% of the cases (591 of 1,616) against 34.7% after a correct fact (2,925 of 8,435). Several such facts can share the same final answer, so we also counted by query: 841 distinct queries were exposed to at least one incorrect fact, and 279 of them (33.2%) were answered wrongly. For each of the 279 we compared the fact, the source text the agent read, the value it certified, and the final answer. In 9 the answer repeats the error of the fact (1.1% of the exposed queries), in 268 the answer is wrong for a different reason, and in 2 the records do not allow a judgment. The agent certifies a value from the source in 40.7% of the cases after an incorrect fact and in 44.6% after a correct one.

The revised manuscript includes this audit as §5.5 and Table 4, with the query-level counts.

Reviewer_LrXWOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses a limitation of hierarchical retrieval for multimodal document QA. When documents are organized into trees of summaries, important details may be lost during summarization, making it difficult for a retrieval agent to identify the right branch. The authors refer to this problem as the abstraction bottleneck.

To address it, the paper proposes RAISE, which introduces a Loss Ledger to record information omitted by parent summaries. These records, called loss witnesses, are attached to tree edges and can be retrieved directly or used to guide an agent's navigation. The system also maintains a traversal map that allows the agent to start at different levels of the document hierarchy, along with an evidence verification mechanism to support answer generation.

The authors evaluate RAISE on eight multimodal document QA benchmarks against seven baselines. The method achieves an average EM of 43.1% and F1 of 56.2%, outperforming the strongest baseline by 5.8 EM points. Ablation studies also show that removing the Loss Ledger reduces average EM by 4.0 points.

Critical strengths

S1. The Loss Ledger is a useful and well-motivated design.

The paper offers a practical way to address information loss in hierarchical retrieval. Rather than making summaries longer, RAISE records omitted information separately and makes it available during retrieval. Attaching loss witnesses to tree edges also allows the agent to use them when deciding which branches to explore.

The ablation results support the usefulness of this design. Removing loss search or map observations reduces EM by 1.7–1.8 points, while removing the entire Loss Ledger causes a 4.0-point drop. Although these results do not fully establish the novelty of loss-aware extraction, they show that the proposed components work well together.

S2. The experimental evaluation is extensive.

The paper evaluates RAISE on eight benchmarks covering web pages, long PDFs, tables, and charts. The authors also provide detailed ablations and additional analyses of retrieval accuracy, agent behavior, backbone sensitivity, and computational cost.

I particularly appreciate the breakdown of evidence recall at different stages of retrieval, as it helps explain where the system succeeds and where information is lost. The experiment using Claude Code as an alternative agent harness is also interesting and demonstrates that the retrieval tools can be used outside the authors' original implementation.

Critical weaknesses

W1. The novelty of loss-aware extraction is not sufficiently established.

My main concern is that loss witnesses are quite similar to the atomic propositions used in existing fact-level retrieval methods, such as Dense X Retrieval. RAISE differs by extracting facts that are missing from parent summaries, but it is unclear whether this selection strategy is actually better than directly extracting important facts from the original content.

The current ablations do not fully answer this question. In particular, the summarization prompt in §F.2 is designed with the assumption that omitted information will be recorded by the Loss Ledger. As a result, removing the Ledger may disadvantage the baseline. The comparison with summary previews in §D.5 is useful, but it does not replace a comparison with a generic proposition index.

I would like to see a controlled experiment that extracts the same number of facts without considering summary omissions, while keeping the indexing budget and retrieval pipeline unchanged. This would help clarify whether loss-aware extraction is a necessary part of the method or simply one way to build an additional fact index.

W2. The efficiency gains and the effects of some runtime mechanisms are not fully clear.

RAISE achieves promising accuracy, but its offline indexing cost is relatively high, especially for long documents. The reported online processing time also excludes retrieval overhead, making it difficult to assess the overall efficiency of the system in practical settings.

Another concern is that the runtime automatically adds loss-search results to the map at every step and performs additional reads when the step budget is nearly exhausted (§C.2). These mechanisms may contribute to the reported performance, but their individual effects are not evaluated. The large gap between map-level recall (92.9%) and read-level recall (57.4%) also raises questions about how effectively the agent uses the available evidence.

Finally, the open-corpus conversion removes a substantial number of queries from several benchmarks. While the filtering procedure is explained, its impact on the generality of the results deserves further discussion.

Questions for authors

Q1. How does loss-aware extraction compare with generic fact extraction?

Could the authors add a baseline that directly extracts important atomic propositions from child nodes without considering their parent summaries? Ideally, this comparison should use the same LLM, number of indexed facts, embedding model, retrieval channels, and orchestrator. This is the most important question for my assessment, as it would help establish the specific benefit of loss-aware extraction.

Q2. How much does the summarization strategy affect the results?

The summarization prompt in §F.2 explicitly allows details to be omitted because the Loss Ledger is expected to preserve them. What happens if a standard summarization prompt is used instead? Would the w/o ledger baseline become stronger, and would the full RAISE system still show a similar improvement?

Q3. How much do the automatic runtime mechanisms contribute to performance?

Could the authors separately disable the automatic loss-search injection and the end-of-budget sweep, and report the resulting accuracy and computational costs? This would help distinguish the contribution of the agent's navigation decisions from that of the runtime safeguards.

Q4. How practical is the indexing cost for frequently updated document collections?

Could the authors provide a more complete cost analysis that accounts for both offline indexing and online retrieval? It would be useful to know how many queries per document are needed to justify the additional indexing overhead. I would also appreciate a brief analysis of whether benchmark filtering changes the difficulty or distribution of the evaluated queries.

I would consider raising my rating if the authors can satisfactorily address these concerns, particularly Q1 and Q2.

7 comments
AuthorsOpenReview ↗

Response to Reviewer LrXW (1/5)

We thank the reviewer for the thorough review, which reached the prompts in §F.2 and the runtime details in §C.2, and for stating which experiments would change the assessment. Every concern below is addressed with an additional experiment. Such experiments are (i) a generic fact index under matched budgets (W1/Q1), (ii) a re-indexing with a plain summarization prompt (W1/Q2), (iii) a runtime ablation of the automatic loss search and the end-of-budget sweep (W2b/Q3), (iv) measured end-to-end latency with the number of queries per document needed to justify the additional indexing overhead (W2a/Q4a), (v) a decomposition of the map-to-read recall gap (W2c), and (vi) a comparison of the retained and dropped queries (W2d/Q4b). The full tables are in the revised manuscript, which we have uploaded (Table 3, §5.4, and Appendices B.4, D.2, D.9.4, F.2, and F.4), and the main numbers are quoted inline.

Since each part of W1 and W2 corresponds to one of the questions, we have paired them as we understood them, W1(a)(c) with Q1, W1(b) with Q2, W2(a)(d) with Q4, and W2(b) with Q3, and answer each pair once; W2(c) is answered on its own. We hope the reviewer will excuse this regrouping of the review text.



[W1(b)/Q2: Effect of the summarization prompt]

W1(b). The current ablations do not fully answer this question. In particular, the summarization prompt in §F.2 is designed with the assumption that omitted information will be recorded by the Loss Ledger. As a result, removing the Ledger may disadvantage the baseline. The comparison with summary previews in §D.5 is useful, but it does not replace a comparison with a generic proposition index.

Q2. How much does the summarization strategy affect the results?

The summarization prompt in §F.2 explicitly allows details to be omitted because the Loss Ledger is expected to preserve them. What happens if a standard summarization prompt is used instead? Would the w/o ledger baseline become stronger, and would the full RAISE system still show a similar improvement?

To test whether the prompt of §F.2 weakens the w/o ledger control, we re-indexed the eight corpora with a plain prompt. The prompt of §F.2 lets a summary omit details on the assumption that the ledger records them. The alternative prompt tells the summarizer that omitted material has no separate record and asks it to mention every child. Without the ledger, EM increased from 39.1 to 39.4. With the ledger, EM changed from 43.1 to 42.7. The within-prompt gains were 4.0 and 3.3 points. Since a different summary omits different content, the ledger was re-extracted from the new summaries, and both configurations were run again on the full panel.

Summary promptw/o ledger: EMw/o ledger: F1full: EMfull: F1ΔEM (full − w/o)
ledger-aware (§F.2, paper)39.150.943.156.2+4.0
plain (ledger-assuming sentences removed)39.451.442.755.8+3.3

The ledger's gain survives a standard summarization prompt: it shrinks from 4.0 to 3.3 EM, which remains larger than the effect of any other single removal in Table 3 (1.7 to 2.8 EM). With plain summaries, the w/o ledger configuration moves from 39.1 to 39.4 EM and the full system from 43.1 to 42.7 EM. The gain holds under both prompts because the bottleneck is abstraction under a budget rather than the wording of the prompt: whichever prompt writes it, a 120-word summary cannot carry the row-level bindings and numbers that the queries ask for, and such facts are what the witnesses record.

The revised manuscript adds the two plain-prompt rows to Table 3, reports them next to the main ablation in §5.4, and gives the plain prompt in Appendix F.2.

AuthorsOpenReview ↗

Response to Reviewer LrXW (2/5)

[W1(a)(c)/Q1: Omission-aware extraction versus a generic fact index]

W1. The novelty of loss-aware extraction is not sufficiently established.

W1(a). My main concern is that loss witnesses are quite similar to the atomic propositions used in existing fact-level retrieval methods, such as Dense X Retrieval. RAISE differs by extracting facts that are missing from parent summaries, but it is unclear whether this selection strategy is actually better than directly extracting important facts from the original content.

W1(c). I would like to see a controlled experiment that extracts the same number of facts without considering summary omissions, while keeping the indexing budget and retrieval pipeline unchanged. This would help clarify whether loss-aware extraction is a necessary part of the method or simply one way to build an additional fact index.

Q1. How does loss-aware extraction compare with generic fact extraction?

Could the authors add a baseline that directly extracts important atomic propositions from child nodes without considering their parent summaries? Ideally, this comparison should use the same LLM, number of indexed facts, embedding model, retrieval channels, and orchestrator. This is the most important question for my assessment, as it would help establish the specific benefit of loss-aware extraction.

We claim that the benefit of the Loss Ledger comes from its selection criterion rather than from the form of its entries: a fact is stored because the summary above omitted it. The loss witnesses share their form with the atomic propositions of Dense X Retrieval, a self-contained sentence with a pointer to its source, and the two differ in which facts they keep. Dense X Retrieval decomposes every passage into all of its propositions, so its index restates the corpus at sentence granularity. A witness is extracted only where a summary dropped part of a child's content, so the ledger holds the complement of the summaries, the facts that abstraction lost, and nothing that a summary already conveys.

To separate the selection criterion from the presence of a searchable fact index, we added a generic proposition index under the conditions listed in Q1. In the control, the loss witnesses are replaced with propositions extracted without any loss information. Specifically, one extraction call per internal node, with the same backbone (Qwen3.6-35B-A3B), extracts atomic facts (propositions) from each child node without seeing the parent summary, and the facts are recorded on the edge from the parent to the child, where the witnesses would be; no instruction mentions summaries or omissions. Two variants are kept. (i) Extracting the same number of facts per edge as the ledger holds there. (ii) Extracting facts under the same cap per edge that the witness extraction uses, eight per child. The propositions are embedded with the same model (nemotron-vl-1b), placed in the same loss channel with the same k and the same map slot, and searched by the same orchestrator under the same 14-step budget.

Facts stored on an unopened branchEMF1ΔEMstepstime (s)index (MB)
loss witness (RAISE, full)43.156.2-6.5723.519,715
(i) generic propositions, count matched to the ledger40.953.6−2.26.9524.719,715
(ii) generic propositions, cap 8 per child (same budget rule)41.455.6−1.76.7824.233,533
summary, whole40.552.7−2.67.9426.93,613
nothing (w/o ledger)39.150.9−4.07.3125.73,613

In the matched variant,
(i) the same number of generic propositions per edge brings accuracy down to 40.9 EM, 2.2 points below the witnesses and 1.8 above the configuration without any ledger (39.1 EM). Generic propositions improved mean EM from 39.1 to 40.9. Replacing them with omission-aware witnesses at the same per-edge count raised EM to 43.1. With a cap of eight entries per child, generic propositions reached 41.4.
(ii) In the cap-8 variant, the extractor tries to cover nearly everything in each child, so the index grows to 1.7 times the size of the ledger (33,533 against 19,715 MB), and accuracy recovers only to 41.4 EM, still 1.7 points below the witnesses. Explicitly extracting an omitted fact therefore matters beyond adding searchable facts, and the ledger does not win by having more text in the channel: it holds less text than the cap-8 index and reaches higher accuracy.
Offline, the two extractors cost the same number of calls per document, one per internal node.

The revised manuscript adds the two generic-fact rows to Table 3, discusses them in §5.4, and gives the extraction prompt in Appendix F.4.

AuthorsOpenReview ↗

Response to Reviewer LrXW (3/5)

[W2(a)/Q4: Indexing cost and latency with retrieval]

W2. The efficiency gains and the effects of some runtime mechanisms are not fully clear.

W2(a). RAISE achieves promising accuracy, but its offline indexing cost is relatively high, especially for long documents. The reported online processing time also excludes retrieval overhead, making it difficult to assess the overall efficiency of the system in practical settings.

[…]

Q4. How practical is the indexing cost for frequently updated document collections?

Could the authors provide a more complete cost analysis that accounts for both offline indexing and online retrieval? It would be useful to know how many queries per document are needed to justify the additional indexing overhead. […]

Cost. To account for the two costs that Table 2 leaves out, we measured both. (i) Online retrieval overhead: end-to-end wall clock for every method of Table 2 on one uncontended lane with 100 queries per corpus, split into LLM calls, vector search and other work. (ii) Offline indexing: from the measured per-query times and the indexing cost of Table 24, the number of queries per document at which RAISE's lower per-query time pays back its additional indexing overhead relative to ViDoRAG, the main competitor.

Methodestimated LLM time (s): all queriesestimated LLM time (s): sampled queriesmeasured wall-clock time (s): totalmeasured wall-clock time (s): LLM callsmeasured wall-clock time (s): vector searchmeasured wall-clock time (s): other
PixelRAG2.12.13.02.90.10.0
SimpleDoc13.313.818.016.90.20.8
DocLens50.345.934.030.30.03.7
ViDoRAG29.026.533.631.71.70.2
VisDoMRAG18.818.126.122.31.02.7
FiF†21.525.846.243.82.50.0
HiKEY2.12.12.12.00.20.0
RAISE23.524.825.424.21.00.2

(i) The measured wall clock stays close to the estimate, and the retrieval overhead inside it is trivial. On the sampled queries the estimate of Table 2 gives 24.8 s per query for RAISE and the measured wall clock is 25.4 s. Of the 25.4 s, vector search takes 1.0 s (3.9%) and other work 0.2 s. Against ViDoRAG, the main competitor, RAISE takes 25.4 s per query to 33.6 s, so the ordering of Table 2 holds once retrieval is included. Across the eight methods, RAISE is the fourth fastest measured end to end against the sixth by the estimate.

Corpusindexing (s/doc): ViDoRAGindexing (s/doc): RAISEindexing (s/doc): RAISE − ViDoRAGmeasured latency (s/query): ViDoRAGmeasured latency (s/query): RAISEmeasured latency (s/query): RAISE − ViDoRAGqueries/doc needed to justify the additional indexing overhead
MultimodalQA4.78.03.339.927.0−12.90.3
MMLongBench-Doc12.059.647.642.127.4−14.73.2
MMDocIR21.966.644.729.117.5−11.63.9
LongDocURL30.982.551.633.915.8−18.12.8
WebQA2.85.22.423.826.2+2.4none
DoubleBench12.137.125.040.230.5−9.72.6
VisDoMBench5.922.316.419.320.3+0.9none
MMDocRAG21.490.068.640.738.7−1.935.7

(ii) The required number of queries depends on the corpus. RAISE is faster per query on six of the eight corpora, where 0.3 to 3.9 queries per document justify the additional indexing (35.7 on MMDocRAG, where the online difference is only 1.9 s). On WebQA and VisDoMBench RAISE is also slower per query, so it costs more on both sides. The indexing overhead is indeed largest for long documents: the four corpora with the largest additional indexing (44.7 to 68.6 s per document) are also where RAISE saves 11.6 to 18.1 s per query on three of them, so fewer than four queries per document justify it there. Table 24 measures the cost of adding a document: a new document adds its own tree, summaries and witnesses without touching the rest of the index (Appendix D.9.4). For a changing collection, the relevant workload is the queries served between rebuilds per document reindexed. In-place edit costs remain unmeasured.

§5.3 of the revised manuscript states on how many corpora RAISE's estimated and measured time are lower than ViDoRAG's, the right block of Table 2 reports the measured latency, and Appendix D.9.4 the number of queries per document needed to justify the additional indexing overhead.

Reviewer_xCJbOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

RAISE addresses a limitation of hierarchical document retrieval: a summary can omit the detail needed to choose the correct branch. Offline, the method compares each summary with its children and records omitted content with links to its source. These loss witnesses form a Loss Ledger and are searchable alongside summaries and original document components. Online, an agent maintains a search map containing retrieved nodes and their ancestors, so it can enter the hierarchy at different levels and move upward or downward. The system also requires declared answer fields to have evidence linked to sources marked as read. Experiments on eight corpora in an open-corpus question-answering setting compare RAISE with seven competing systems and examine its components and costs. The results support a useful retrieval system, although the additional value of specifically detecting omissions remains incompletely isolated.

Critical strengths

1.The index directly records details that summaries can hide from retrieval. Section 2 identifies why routing only through summaries can miss relevant branches. The proposed witnesses address that problem by storing omitted details and source locations. Section 4 uses them in two ways: queries can retrieve them directly, and the agent can inspect them before opening a branch. Keeping these navigation records separate from source evidence used for answering is also useful. This is a concrete indexing contribution, without needing to regard the paper's simple indistinguishability argument as a major theoretical result.

2.The gains are substantial, and the ablations distinguish the ledger's two uses. Table 1 reports average EM/F1 of 43.1/56.2, versus 37.3/50.2 for ViDoRAG, the strongest listed competitor. Table 3 lowers EM to 39.1 without the ledger and separately disables ledger search and displayed annotations. Appendix D.5 also tests summary previews. With loss search disabled in both conditions, witness annotations achieve 41.4 EM versus 40.5 for whole-summary previews. Thus, the evidence supports more than simply displaying extra text. The breadth of these results and the explicit limitations support weak acceptance.

Critical weaknesses

1.The experiments do not separate detecting omissions from adding a source-grounded fact index. Appendix D.5 compares witnesses with existing summaries, but not with facts extracted directly from sources under the same offline model-call, storage, and online-context budgets. Appendix F.2 explicitly lets summaries omit details because the ledger will record them. The ledger ablation therefore evaluates this joint summary-and-ledger design; it does not isolate whether selecting facts specifically because they were omitted is better than indexing facts more generally. Appendix E acknowledges this limitation. It does not invalidate the system gains, but it limits how much methodological novelty can be attributed to the omission-detection criterion itself.

2.Query filtering and baseline adaptation limit the scope of the comparative conclusion. Appendix B.4 explains the open-corpus conversion. Table 4 retains 269 of 2,325 LongDocURL queries, 184 of 1,091 MMLongBench-Doc queries, and 451 of 2,055 MMDocRAG queries. The filtering models see only the question, so their agreement assesses whether it contains a retrieval anchor, not whether that anchor identifies the intended source within the corpus. Some competitors also receive documents from a router. The reported router accuracy of 79.4% is informative, but does not isolate the impact of that adaptation on comparative rankings. These choices are disclosed and all methods use the same retained questions. The results therefore support superiority on these constructed subsets, especially over native open-corpus competitors, more strongly than superiority in the original benchmark settings.

3.Presentation issue in Figure 3. The unlabeled blank map (M_1) in the first frame can be mistaken for missing content. Please label it as the initially empty search map and clarify that each frame shows the state before executing the action batch displayed below it. This would make the transition from (t=1) to (t=2) easier to follow.

Questions for authors

1.Under matched offline model calls, index size, and online budgets, how does the ledger compare with a general source-grounded fact index? Can the results distinguish the benefit of detecting omissions from the benefit of adding searchable facts?

2.Is there a human audit or corpus-level check of the retained questions' retrieval anchors? How sensitive are the rankings to filtering and document routing, and is an oracle-routing result available for methods originally designed to receive a known document?

3.Tables 2 and 20 report estimated model-processing time and offline indexing cost. What measured full-query latency, or analysis of how many queries offset the additional indexing cost, supports the practical efficiency claim when retrieval and ledger construction are included? Please delimit the claim if these measurements are unavailable.

6 comments
AuthorsOpenReview ↗

Response to Reviewer xCJb (1/4)

We thank the reviewer for the constructive feedback, in particular for stating precisely which control our ablations lack and where the comparative claim should be delimited. All the concerns are addressed below with additional experiments. (i) A generic source-grounded fact index is added under matched budgets, and the corpora are re-indexed with a plain summarization prompt (W1/Q1). (ii) Oracle document routing for the single-document baselines is paired with a human audit of the retained queries (W2/Q2). (iii) Measured full-query latency is reported together with the number of queries that offset the additional indexing cost (Q3). The full tables are in the revised manuscript, which we have uploaded (Table 3, §5.4, and Appendices B.4, D.9.4, F.2, and F.4). The main numbers are quoted inline.



[W1/Q1 (a): A generic source-grounded fact index under matched budgets]

W1. The experiments do not separate detecting omissions from adding a source-grounded fact index. Appendix D.5 compares witnesses with existing summaries, but not with facts extracted directly from sources under the same offline model-call, storage, and online-context budgets. Appendix F.2 explicitly lets summaries omit details because the ledger will record them. The ledger ablation therefore evaluates this joint summary-and-ledger design; it does not isolate whether selecting facts specifically because they were omitted is better than indexing facts more generally. Appendix E acknowledges this limitation. It does not invalidate the system gains, but it limits how much methodological novelty can be attributed to the omission-detection criterion itself.

Q1. Under matched offline model calls, index size, and online budgets, how does the ledger compare with a general source-grounded fact index? Can the results distinguish the benefit of detecting omissions from the benefit of adding searchable facts?

To isolate the omission-detection criterion from the joint summary-and-ledger design, we added two controls on all eight corpora. (a) A generic source-grounded fact index under the three matched budgets of Q1, which removes the selection criterion and keeps the summaries. (b) A re-indexing with a plain summarization prompt, which removes the dependence of the summaries on the ledger and keeps the ledger.

(a) In the generic fact index, we replaced the loss witnesses with facts extracted directly from the sources, without regard to what the summaries omitted. Offline model calls are matched by construction: one extraction call per internal node with the same backbone (Qwen3.6-35B-A3B), which sees the node's children and not the parent summary and records atomic facts. Index size is matched in variant (i), which stores the same number of facts per edge as the ledger holds there; variant (ii) applies the same cap per edge as the witness extraction, eight per child. Online budgets are identical: the facts are embedded with the same model (nemotron-vl-1b), placed in the same loss channel and the same map slot, and searched by the same orchestrator under the same 14-step budget. At the same per-edge fact count and under the same 14-step limit, loss witnesses scored 43.1 EM and generic facts scored 40.9. Mean input use was 72,186 and 73,834 tokens per query, respectively.

Facts stored on an unopened branchEMF1ΔEMstepstime (s)index (MB)
loss witness (RAISE, full)43.156.2-6.5723.519,715
(i) generic facts, count matched to the ledger40.953.6−2.26.9524.719,715
(ii) generic facts, cap 8 per child (same budget rule)41.455.6−1.76.7824.233,533
summary, whole40.552.7−2.67.9426.93,613
nothing (w/o ledger)39.150.9−4.07.3125.73,613

The results distinguish the two benefits: adding searchable source-grounded facts at the ledger's size is worth 1.8 EM over no index, and selecting them because the summary omitted them is worth the remaining 2.2 EM; a 1.7 times larger index without the criterion recovers 0.5 EM more.
(i) With the same number of generic facts per edge, accuracy falls to 40.9 EM, 2.2 points below the witnesses and 1.8 above the configuration without any ledger (39.1). Generic facts improved EM from 39.1 to 40.9. Replacing them with loss witnesses at the same per-edge count raised EM to 43.1. With a cap of eight per child, generic facts reached 41.4.
(ii) Under the cap of eight per child, the extractor tries to cover nearly everything in each child, so the index grows to 1.7 times the size of the ledger (33,533 against 19,715 MB) and accuracy recovers to 41.4 EM, still 1.7 points below the witnesses.
Neither difference is bought with an offline budget, since both variants make one extraction call per internal node, as the ledger does.

AuthorsOpenReview ↗

Response to Reviewer xCJb (2/4)

[W1/Q1 (b): A re-indexing with a plain summarization prompt]

W1. […] Appendix F.2 explicitly lets summaries omit details because the ledger will record them. The ledger ablation therefore evaluates this joint summary-and-ledger design; it does not isolate whether selecting facts specifically because they were omitted is better than indexing facts more generally. […]

(b) Several sentences of the prompt of §F.2 let a summary omit details because the ledger will record them. The plain prompt removes such sentences and keeps the 120-word budget and every other rule, so the summaries are written without the ledger in view. The witnesses were re-extracted against the new summaries, since a witness records what its summary omitted, and both configurations were run again on all eight corpora.

Summary promptw/o ledger: EMw/o ledger: F1full: EMfull: F1ΔEM (full − w/o)
§F.2 (summaries may omit what the ledger records)39.150.943.156.2+4.0
plain (such sentences removed)39.451.442.755.8+3.3

The joint design produces 0.7 EM of the ledger's 4.0 EM gain: under the plain prompt the configuration without the ledger reaches 39.4 EM and the full system 42.7, so the gain is 3.3 EM against 4.0 under the prompt of §F.2. The prompt of §F.2 costs the configuration without the ledger 0.3 EM and gives the full system 0.4.

The revised manuscript adds the two generic-fact rows and the two plain-prompt rows to Table 3, states the attribution in §5.4, and gives the two prompts in Appendices F.4 and F.2.



[W2/Q2 (i): What the filter guarantees]

W2. Query filtering and baseline adaptation limit the scope of the comparative conclusion. Appendix B.4 explains the open-corpus conversion. Table 4 retains 269 of 2,325 LongDocURL queries, 184 of 1,091 MMLongBench-Doc queries, and 451 of 2,055 MMDocRAG queries. The filtering models see only the question, so their agreement assesses whether it contains a retrieval anchor, not whether that anchor identifies the intended source within the corpus. […]

Q2. Is there a human audit or corpus-level check of the retained questions' retrieval anchors? […]

To check how far the comparative conclusion depends on the filtering and on the document routing, we added the checks the question names. (i) Filtering: a human audit of the retrieval anchors, which checks what the filtering models cannot, namely whether the anchor identifies the intended source within the corpus. One author audited 100 retained queries per filtered corpus, searched the corpus for other documents that the anchor could refer to, and judged whether it identifies a single document. We also compared the retained and dropped queries on the two properties the benchmarks annotate for every query. (ii) Document routing: SimpleDoc and DocLens were re-run with the gold document supplied for every query, which removes the 20.6% of queries on which the shared router picks the wrong document and leaves everything else in their pipelines as it is. The table under (ii) lists the eight methods of Table 1 on the eight-corpus average, sorted by EM, with the two oracle rows inserted.

Corpusaudited queriesanchor identifies a single documentanchor shared by several documentsno anchor
DoubleBench10093%6%1%
LongDocURL10092%5%3%
VisDoMBench10095%5%0%
MMDocRAG10095%5%0%
MMLongBench-Doc10091%6%3%
MMDocIR10088%12%0%
all six60092.3%6.5%1.2%

(i) The filtering admitted few queries whose anchor fails to identify the source within the corpus: the annotator judged 92.3% of the audited queries (554 of 600) to identify a single document. Of the rest, 6.5% name an entity or a title that several documents of the corpus share, and 1.2% carry no anchor at all. MMDocIR has the lowest share of single-document anchors at 88% and VisDoMBench and MMDocRAG the highest at 95%. The other 7.7% stay in the panel for every method alike.

Six filtered corpora, averagedqueriesmulti-page evidence %image, table or chart evidence %
retained2,32144.859.6
dropped8,03647.461.0

In a manual audit of 100 retained queries per filtered corpus, 554 of 600 anchors identified a single document. Our comparative claim applies to the evaluated open-corpus subsets. The retained and dropped sets differ by 2.6 points in the share of multi-page evidence and by 1.4 points in the share of image, table or chart evidence.

AuthorsOpenReview ↗

Response to Reviewer xCJb (3/4)

[W2/Q2 (ii): Oracle routing for the single-document methods]

W2. […] Some competitors also receive documents from a router. The reported router accuracy of 79.4% is informative, but does not isolate the impact of that adaptation on comparative rankings. These choices are disclosed and all methods use the same retained questions. The results therefore support superiority on these constructed subsets, especially over native open-corpus competitors, more strongly than superiority in the original benchmark settings.

Q2. […] How sensitive are the rankings to filtering and document routing, and is an oracle-routing result available for methods originally designed to receive a known document?

MethodSettingEMF1
RAISEopen-corpus43.156.2
ViDoRAGopen-corpus37.350.2
SimpleDocoracle doc given36.848.4
VisDoMRAG8 BM25 candidate docs given34.146.7
SimpleDoctop-1 router doc given32.843.8
HiKEYopen-corpus28.940.5
DocLensoracle doc given27.137.8
FiFopen-corpus25.435.3
DocLenstop-1 router doc given25.035.3
PixelRAGopen-corpus24.734.5

(ii) On the eight-corpus average, the ranking of RAISE and ViDoRAG holds with the router removed, and the gold document reorders only the methods below them. The oracle rows are scored on the kept queries of Table 5, the same queries as Table 1. For a query with several gold documents, the oracle supplies the first listed one. With the gold document, SimpleDoc rises from 32.8 to 36.8 EM and DocLens from 25.0 to 27.1, and RAISE, which has to find the document itself, stays ahead of both by 6.3 and 16.0 EM. The gold document moves SimpleDoc past VisDoMRAG to third, 0.5 EM behind ViDoRAG, and DocLens past FiF to sixth. Per corpus it lifts SimpleDoc past RAISE on VisDoMBench (38.4 against 37.5) and MMDocRAG (13.1 against 12.9) and nowhere else; DocLens does not reach second place on any corpus. The oracle rows are an upper bound for the two methods, which were designed to receive a known document: finding the document is the step that RAISE performs itself over the whole corpus and that the shared router gets wrong on 20.6% of the queries.
The conclusions stay delimited as the reviewer asks: §5.2 and §6 of the revised manuscript state them for the open-corpus subsets we constructed.

The revised manuscript adds the audit and the per-corpus oracle results (Table 8) to Appendix B.4.



[W3: Figure 3, the empty map in the first frame]

W3. Presentation issue in Figure 3. The unlabeled blank map (M_1) in the first frame can be mistaken for missing content. Please label it as the initially empty search map and clarify that each frame shows the state before executing the action batch displayed below it. This would make the transition from (t=1) to (t=2) easier to follow.

Thank you for catching this. The first panel now shows ∅ in the region labelled M1M_{1}. The caption states that each frame precedes the action batch printed below it, so the transition from t = 1 to t = 2 reads as the effect of that batch. The revised manuscript includes the corrected figure.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response to All Reviewers

Global response to all reviewers

We thank the three reviewers. The reviews share one concern: the ablations show that the Loss Ledger helps, but not whether selecting facts because a summary omitted them is better than indexing the same number of ordinary source facts (xCJb W1/Q1, LrXW W1/Q1, PgBt W1/Q1), and the summarization prompt of §F.2 lets the summaries rely on the ledger, which may weaken the w/o ledger control (xCJb W1, LrXW W1(b)/Q2, PgBt W1). To answer both, we ran two controls on all eight corpora with the same model calls, embedder, channel, orchestrator and step budget. (i) A generic fact index replaces the witnesses with facts extracted without any loss information, in a count-matched and a cap-8 variant. (ii) A re-indexing with a plain summarization prompt that does not assume the ledger.

Facts stored on an unopened branchEMF1ΔEMstepstime (s)index (MB)
loss witness (RAISE, full)43.156.2-6.5723.519,715
(i) generic facts, count matched to the ledger40.953.6−2.26.9524.719,715
(ii) generic facts, cap 8 per child41.455.6−1.76.7824.233,533
summary, whole40.552.7−2.67.9426.93,613
nothing (w/o ledger)39.150.9−4.07.3125.73,613
RAISE with a plain summarization prompt (w/o ledger → full)39.4 → 42.751.4 → 55.8+3.37.28 → 6.4125.1 → 24.13,593 → 16,974

(i) Of the ledger's 4.0 EM gain, 1.8 comes from adding searchable source facts at the ledger's size and 2.2 from selecting them because the summary omitted them: the count-matched generic index reaches 40.9 EM, 2.2 points below the ledger and 1.8 above no index. The cap-8 index grows to 1.7 times the ledger's size and reaches 41.4 EM, against 43.1 for the ledger. (ii) The ledger's gain survives summaries that do not rely on it: with the plain prompt, the configuration without the ledger reaches 39.4 EM and the full system 42.7, so the gain is 3.3 EM against 4.0 under the paper's prompt. The full tables are in Table 3 and §5.4 of the revised manuscript, with the two prompts in Appendices F.2 and F.4.

Beyond the two controls, we added the following, listed in the order of the reviews.

  • A human audit of 100 retained queries per filtered corpus, checking whether the retrieval anchor identifies a single document, and oracle document routing for SimpleDoc and DocLens (xCJb W2/Q2).
  • End-to-end latency measured on an uncontended lane, including retrieval, for all eight methods of Table 2, with a per-corpus analysis of the queries per document that offset the additional indexing cost (xCJb Q3, LrXW W2a/Q4).
  • A runtime ablation that disables the automatic loss-search injection and the end-of-budget sweep separately (LrXW W2b/Q3).
  • A decomposition of the gap between map-level recall and read-level recall (LrXW W2c).
  • A comparison of retained and dropped queries on multi-page evidence and visual evidence (xCJb W2/Q2, LrXW W2d/Q4).
  • An audit of every ledger fact the agent followed to its source, judged by Claude Opus 5.5 against the source text, counting how often a wrong final answer followed an incorrect fact and, query by query, how often the answer repeated the error of the fact (PgBt W2/Q2).

The revised manuscript, which we have uploaded, contains the following changes, and every added or changed passage of text is marked in blue.

  • Table 3 and §5.4: the two generic-fact rows (xCJb W1/Q1, LrXW W1/Q1, PgBt W1/Q1).
  • Table 3 and §5.4: the plain-prompt re-indexing (xCJb W1/Q1, LrXW W1/Q2, PgBt W1/Q1).
  • Table 3: the two runtime-ablation rows (LrXW W2b/Q3).
  • Appendix B.4: the audit of the retained queries and the comparison of retained and dropped queries (xCJb W2/Q2, LrXW W2d/Q4).
  • Appendix B.4: the oracle-routing results (Table 8) (xCJb W2/Q2).
  • Table 2 and §5.3: the measured latency; Appendix D.9.4: the table of queries per document that offset the additional indexing cost (xCJb Q3, LrXW W2a/Q4).
  • Appendix D.2: the decomposition of the map-to-read gap (LrXW W2c).
  • §5.5 and Table 4: the audit of the ledger facts the agent followed, with the query-level counts (PgBt W2/Q2).
  • Figure 3: the labelled initial map and the clarified caption (xCJb W3).
AuthorsOpenReview ↗

Discussion Update and Request for Follow-Up

Dear Area Chair and Reviewers,

We thank the three reviewers for the detailed reviews and the discussion. We have addressed every concern of the three reviews with additional experiments and uploaded the revised manuscript, with every change marked in blue and a point-by-point response under each review. Reviewer LrXW wrote that all concerns are addressed and raised the rating, and Reviewer xCJb thanked us for the detailed reply and kept the score.

What the reviews asked and what we added

  • Loss witnesses versus a generic fact index (all three reviewers). The reviewers asked whether storing the facts that the summaries omitted is better than storing any source facts, and whether the no-ledger control is handicapped by summaries written to rely on the ledger. We answer with a generic fact index under matched budgets and a re-indexing with a plain prompt: selecting the omitted facts accounts for 2.2 of the ledger's 4.0 EM gain, and the gain survives the plain prompt (3.3 EM).
  • Query filtering and document routing (xCJb, LrXW). The reviewers noted that the open-corpus conversion drops many queries and that two baselines receive their document from a router. A human audit found a single-document anchor in 92.3% of the retained queries, the retained and dropped queries share nearly the same mix of multi-page and visual evidence, and RAISE still outperforms the two baselines given the gold document.
  • Latency and indexing cost (xCJb, LrXW). We measured end-to-end latency with retrieval included: 25.4 s per query for RAISE against 33.6 s for the main competitor. The additional indexing cost is offset after 0.3 to 3.9 queries per document on five of the eight corpora.
  • Runtime mechanisms and the map-to-read gap (LrXW). The reviewer asked for the contribution of two runtime mechanisms and for an explanation of the gap between map-level and read-level recall. We ablated the two mechanisms separately, recomputed the recall tables under one definition and denominator, and traced the gap query by query.
  • Incorrect ledger facts (PgBt). The reviewer asked how often an incorrect ledger fact leads to a wrong answer. We conducted an audit of every ledger fact the agent acted on, judged against its source: 16.1% were inconsistent, and only 9 of the 279 wrong answers that followed one repeated its error.
  • Figure 3 (xCJb). The initially empty map in the first frame is now labelled, and the caption states that each frame precedes the action batch below it.

We are grateful for reviews that named exactly which controls and measurements were missing, and for the reviewers' engagement in the discussion. We would appreciate it if Reviewer PgBt could let us know whether the response addresses the two concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.