DocISE: Implicit Structure Encoder for Semi-Structured Document Question Answering

Semi-structured documents are ubiquitous in scientific reports, financial state- ments, and technical manuals. Question answering over such documents requires simultaneous understanding of text, tables, charts, and complex hierarchical lay- outs. Existing methods either rely on repeatedly calling large language models (LLM) for structure parsing and retrieval, leading to high costs and latency, or extracting the content of the document and losing layout and hierarchy informa- tion, sacrificing answer accuracy. To address this, we propose DocISE, a sys- tem that achieves state-of-the-art accuracy with minimal computational expense. Our approach introduces three core modules. (1) An Implicit Structure En- coder trained via contrastive learning and a structure consistency loss. This mod- ule jointly embeds hierarchical relationships, spatial positions, and textual con- tent into a dense vector space to capturing document structure. (2) A Hybrid Retrieval Pipeline that leverages BM25, layout fingerprints, and a lightweight cross-encoder to perform high-precision retrieval drastically reducing costs and latency. (3) A Topology-Aware Evidence Expansion mechanism that adaptively retrieves spatially adjacent and structurally related evidence, overcoming the ev- idence omission common in fixed-path retrieval methods. We evaluate DocISE on four benchmarks, and comprehensive evaluations show that DocISE achieves a new state-of-the-art accuracy on the four benchmarks. Crucially, it does so with a 50× reduction in cost and 4× lower query latency compared to the previous state-of-the-art Method. These results demonstrate that DocISE establishes a new optimal trade-off among accuracy, cost, and speed, offering a practical and scal- able path for accurate semi-structured document analysis. Source code is available at https://anonymous.4open.science/r/DocISE.