RAISE: Loss-Aware, Direction-Free Retrieval Over Hierarchical Documents

When querying over thousands of multimodal documents, the evidence a query needs often sits in a table, a chart, or even within the page layout. Recent retrievers therefore build each document into a tree whose internal nodes carry short summaries of their children, and an agent descends it like a table of contents, opening only the promising branch. A summary, however, is written before any query is known and under a length budget, so it cannot keep every detail of its children. We call the resulting limitation the abstraction bottleneck: when a summary omits the detail a query asks about, the agent can no longer tell the right branch from its siblings, and must either guess or open them all. We present Retrieval over Abstraction-Induced Summary Erasure (RAISE), which records the omission caused by summarization and routes on it. Offline, RAISE contrasts every summary against the children it was written from and records what the summary omitted as loss witnesses, each attached to the parent-to-child edge it came from, which together form the Loss Ledger. Online, an agent traverses the documents by extending a map of its traversal so far, and the witnesses serve it in two ways: (i) a witness is itself retrievable, so a query can match it and start its traversal from the edge the witness is attached to, and (ii) the witnesses are drawn on the map, so the agent can judge a branch before opening it. The agentic traversal is also direction-free: it can start traversing at any level of a tree without losing the hierarchy, and extend downward or upward at every step, so no direction has to be fixed before retrieving. Extensive experimental results show that RAISE achieves the best average accuracy on eight multimodal document QA benchmarks in the open-corpus setting, outperforming the strongest of seven competitors by 5.8 points of exact match on average.