acceptodds
Under review as a conference paper at ICLR 2027

Beyond Chunks and Binary Graphs: Multimodal Hypergraph Representation for Long Document Question Answering

Abstract

Long document question answering often requires grounding answers in textual, tabular, and visual evidence distributed across multiple pages, especially for visually rich documents such as scientific papers and technical reports. Existing Retrieval-Augmented Generation (RAG) approaches typically retrieve text chunks, multimodal chunks, full-page images, or binary graph structures, but these retrieval units provide limited support for organizing heterogeneous evidence into coherent retrieval units. We introduce DocHyper, a multimodal document hypergraph representation in which document entities are connected by hyperedges that encode n-ary evidence units within individual elements and among heterogeneous elements in the surrounding document context. Built from parsed document elements, DocHyper supports entity- and hyperedge-level retrieval with page- and layout-aware source grounding. Experiments on LongDocURL and MMLongBench-Doc show that our approach improves answer quality over chunk-, multimodal-chunk-, page-image-, and graph-based retrieval baselines. Ablations further analyze entity-level, hyperedge-level, and binary-graph retrieval, providing evidence for the value of organizing retrieved multimodal evidence as higher-order units with source-page grounding.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Hypergraph, Hypergraph Representation, Multimodal Long Document, Document Question Answering

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_KXExOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes DocHyper, a multimodal hypergraph-based retrieval framework for long-document question answering. The method extracts entities from text, tables, images, and charts, and organizes them into within-element and cross-element hyperedges. It then performs dual-path entity- and hyperedge-level retrieval, mapping the retrieved evidence back to source-page images for VLM-based answer generation. Experiments on LongDocURL and MMLongBench-Doc demonstrate improvements over several chunk-based, graph-based, and multimodal retrieval baselines, with additional ablation studies investigating the contributions of different retrieval structures.

Critical strengths
  1. Relevant problem and practical framework. The paper addresses an important challenge in long-document QA: retrieving and organizing heterogeneous evidence distributed across text, tables, and figures. The proposed framework provides a coherent pipeline for multimodal evidence indexing, structured retrieval, and source-page grounding.

  2. Promising empirical results and useful ablation studies. DocHyper achieves competitive improvements on two established long-document QA benchmarks. The experiments cover multiple retrieval paradigms, and the ablation studies provide useful evidence that combining entity-level and hyperedge-level retrieval is beneficial. These results suggest practical potential for structured multimodal retrieval.

Critical weaknesses

1. Insufficient evidence for the necessity and effectiveness of the proposed hypergraph representation.

The central claim of the paper is that higher-order multimodal hypergraph representations provide advantages over conventional chunk-based and graph-based retrieval. However, the experiments do not sufficiently isolate the contribution of the hypergraph structure itself. DocHyper combines LLM-based multimodal semantic extraction, entity and hyperedge indexing, dual-path retrieval, and source-page grounding. Importantly, the final answer generator primarily receives retrieved page images rather than explicit hypergraph evidence.

Although Table 2 includes graph-structure ablations, these experiments do not rule out the possibility that a simpler retrieval system using the same extracted entities and semantic descriptions, but without hypergraph connections, could achieve comparable performance. Moreover, cross-element hyperedges are constructed primarily through same-page heterogeneous entity grouping, which does not necessarily capture meaningful higher-order semantic relationships.

Consequently, the reported improvements do not convincingly establish that the proposed hypergraph representation is necessary, rather than the gains arising from richer semantic indexing and improved page selection. This substantially weakens the paper's central technical contribution.

2. Inadequate evaluation against recent SOTA retrieval methods, modern VLMs, and agentic approaches.

The empirical evaluation is insufficient for establishing the significance and competitiveness of the proposed method.

First, two highly relevant and formally published methods, HippoRAG 2 (ICML 2025) and HyperGraphRAG (NeurIPS 2025), are relegated to the appendix. In particular, HyperGraphRAG is conceptually close to DocHyper and should receive a more thorough and controlled comparison. The existing comparison also involves different answer-generation contexts, making it difficult to attribute the performance gap specifically to the proposed representation.

Second, all main retrieval experiments rely on Qwen2-VL-7B-Instruct, a relatively outdated VLM released in 2024. More recent models, such as Qwen3-VL, offer substantially stronger multimodal understanding capabilities. It remains unclear whether the proposed retrieval improvements persist when stronger VLMs are used as the answer-generation backbone.

Third, the paper lacks comparisons with recent long-document VLM/agent systems, such as DocAgent (EMNLP 2025) and MM-Doc-R1 (Findings of ACL 2026). Given that modern VLMs and tool-using agents can directly process long multimodal documents, it remains unclear whether constructing a hypergraph index provides any accuracy advantage over such alternatives.

Questions for authors
  1. In the graph-structure ablation in Table 2, are the extracted semantic contents, embeddings, retrieval budgets, and final QA contexts kept identical when replacing hyperedges with pairwise edges? What specific component of the performance improvement can be attributed exclusively to higher-order hypergraph connectivity?

  2. Can the authors compare DocHyper against a simpler dual-index retrieval baseline that uses exactly the same extracted entities, multimodal semantic descriptions, and source-page images, but does not construct or traverse hypergraph connections? Additionally, how much performance is lost when cross-element hyperedges are removed? These results would be important for establishing the necessity of the proposed representation.

  3. Do the reported improvements persist when using a modern VLM, such as Qwen3-VL-8B, as the shared answer generator for DocHyper and representative retrieval baselines? How does DocHyper compare with recent VLM-based document agents under comparable evaluation protocols, particularly in terms of answer accuracy?

Reviewer_vFoDOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduced DOCHYPER, a RAG framework for long MM document question answering. DOCHYPER builds a multimodal hypergraph. It extracts entities as nodes, and uses hyperedges to group them into n-ary units. During retrieval, it matches both entities and hyperedges, and traces back to the original source pages. Finally, author use VLM uses these page images to answer the question. Experiment results on two different datasets show that the system achieves SOTA performance

Critical strengths
  1. Most graph-based RAG methods only use binary edges. But in a real world, a table, its caption, and the main text usually form a joint context together. Using hyperedges to group multiple heterogeneous elements into one intact evidence unit makes a lot of sense.

  2. The author design specific processors for different elements (text, tables, images, charts). Also, the dual-path retrieval strategy is very well designed.

Critical weaknesses
  1. If we look at Table 7, building the hypergraph index requires a massive amount of LLM API calls. For LongDocURL, it costs $0.26 per document. For real-world industry applications with millions of documents, this heavy LLM-based extraction pipeline is too expensive and extremely slow. The paper should add lightweight options.

  2. In Section 3.3, the author claims it builds cross-element hyperedges to group heterogeneous evidence. But the method is just using "round-robin sampling" to group representative entities on the same page. This is basically a simple spatial co-occurrence trick, not a real semantic n-ary relationship. Furthermore, if a figure is on page 5 but the text is on page 6, this same-page grouping strategy will completely fail to connect them.

  3. The whole hypergraph structure strongly relies on the MinerU tool. If the parser makes mistakes, the extracted entities and hyperedges will be totally wrong. The paper lacks robustness analysis on parser errors.

Questions for authors
  1. The index construction cost in Table 7 is very high. Do you think this method can be practically deployed for a large-scale document database?

  2. In Section 3.3, cross-element hyperedges only group entities on the same page. What if the text description and the corresponding figure are on different pages? How does DOCHYPER solve these cross-page relations?

  3. How sensitive is your model to the errors made by the MinerU? Have you tested the framework with a weaker or different parser?

Reviewer_y8KEOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents DocHyper, a multimodal hypergraph method for long-document question answering. It groups document evidence into hyperedges within each page. It retrieves both entities and hyperedges to select relevant pages. The page images are then used to generate answers. Experiments on LongDocURL and MMLongBench-Doc show improvements over the evaluated baselines.

Critical strengths
  1. The method extracts and groups evidence from text, tables, and images. It retains source-page information so retrieved evidence can be traced back to the document.

  2. The paper reports improvements over the evaluated baselines on two benchmarks. It also includes comprehensive ablations of entity retrieval, hyperedge retrieval, and graph structure.

Critical weaknesses
  1. Unclear differences from prior work. Existing multimodal hypergraph RAG methods, including EvoGraph-R1 [1] and Hyper-M2RAG [2], are neither discussed nor compared against in the paper. Moreover, the proposed bipartite indexing and dual-path retrieval already appear in HyperGraphRAG. The authors should clarify what DocHyper adds beyond these related hypergraph methods.

  2. Missing key comparisons. Comparisons with multimodal graph methods, such as RAGAnything [3] or MegaRAG [4], are still needed to substantiate the motivation in L017-019. DocHyper uses page images as input to the answer generator (L294-297). Comparisons with stronger page-image retrievers, such as recent ColQwen variants and GME [5], are therefore important.

[1] EvoGraph-R1: Self-Evolving Multimodal Knowledge Hypergraphs for Agentic Retrieval. CVPR, 2026.

[2] Hypergraph-based Multimodal Retrieval-Augmented Generation with Incremental Refinement. ACM MM, 2026.

[3] RAG-Anything: All-in-One RAG Framework. arXiv:2510.12323, 2025.

[4] MegaRAG: Multimodal Knowledge Graph-Based Retrieval Augmented Generation. ACL, 2026.

[5] Bridging Modalities: Improving Universal Multimodal Retrieval by Multimodal Large Language Models. CVPR, 2025.

Questions for authors
  1. Differences from prior hypergraph work. What does DocHyper add beyond EvoGraph-R1 [1], Hyper-M2RAG [2], and HyperGraphRAG?

  2. Multimodal graph baselines. How does DocHyper compare with RAGAnything [3] or MegaRAG [4]?

  3. Cross-page links. DocHyper can retrieve multiple pages. However, graph expansion stays within each page (L202-205, L253-261, L279-296). Why are cross-page links not included? How does this affect questions that need evidence from multiple pages?

  4. Page-image retrieval baselines. How does DocHyper compare with recent ColQwen variants and GME [5]?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.