acceptodds
Under review as a conference paper at ICLR 2027

Neuro-Symbolic Reasoning Framework for Long-Document Understanding

Abstract

Multimodal long-document understanding remains challenging because relevant evidence is sparsely distributed across lengthy and redundant documents. Existing retrieval-based methods optimize page-level relevance only, yet individually relevant pages may still fail to collectively provide all the evidence required by a question. Moreover, cross-page reasoning is often generated without explicit verification, allowing unsupported steps and logical gaps to remain hidden under incomplete evidence. We therefore formulate long-document understanding as a neuro-symbolic reasoning process that couples neural evidence discovery with symbolic verification. Neural models derive question-specific evidence requirements, localize complementary evidence across pages, and construct a structured reasoning hypothesis. Symbolic constraints evaluate evidence coverage, validate intermediate reasoning, and enforce answer conditions. Failed checks trigger a bounded repair loop to retrieve missing evidence and revise the reasoning hypothesis, while unresolved cases lead to abstention. Experiments on three multimodal long-document benchmarks demonstrate state-of-the-art performance, improved evidence localization and token efficiency, and interpretable evidence navigation with auditable verification traces.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Long-Document Understanding

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_mqaJOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a neuro-symbolic reasoning framework for long-document understanding, leveraging components such as the Evidence Loop to improve structured evidence retrieval, verification, and reasoning. Experiments on MMLongBench, DocBench, and LongDocURL show improvements in accuracy, recall, and token efficiency.

Critical strengths
  1. The motivation is clear. The paper identifies fine-grained evidence retrieval and evidence-grounded reasoning as two bottlenecks in long-document understanding, and designs corresponding mechanisms around evidence sufficiency and reasoning support.

  2. The components of the framework are well defined. The paper provides clear terminology and representations for the different modules in the proposed neuro-symbolic reasoning framework.

Critical weaknesses

The paper's formulation of long-document understanding may be somewhat biased or overly simplified. The paper abstracts the task into a clearly separated "evidence retrieval–reasoning" pipeline, which may be too coarse for genuinely complex long-document understanding. Equation (4) appears to rely on the same assumption, namely that the question itself provides enough information to derive most of the evidence that needs to be located in the document. However, in more challenging long-document QA tasks, such as multi-hop reasoning, evidence requirements may emerge dynamically rather than being fully enumerated before reasoning begins. In other words, evidence retrieval and reasoning can be intrinsically coupled. An analysis or discussion of how the proposed framework would handle such tasks, beyond the current setting considered in the paper, would substantially strengthen the contribution.

I have reservations about the novelty of the proposed framework. The paper appears to use a rather broad definition of symbolic, while symbolic reasoning differs considerably from classical notions such as formal logical deduction, program execution, or theorem proving. A substantial portion of the actual reasoning is still carried out by the LLM, and key components such as Evidence Requirements are not replaced by a formal reasoning system. Even the Evidence Assembler mainly serves to organize evidence rather than functioning as a new reasoning module. Overall, the method appears closer to an agentic tool-use protocol built around an LLM (or in the paper's terminology, a "neural model"). The relationship between the proposed framework and symbolic reasoning therefore needs to be clarified further.

The experiments do not provide sufficiently strong evidence for the paper's core claims. For example:

  1. Figure 4 appears to present a relatively simple case. The target pages are already explicitly specified in the question, and the model only needs to identify the signatures and count them to obtain the answer. The symbolic component appears to play a rather limited role. From both the task difficulty and the execution trajectory, this example is not very convincing evidence for the advantage of neuro-symbolic reasoning. The repair case in the appendix also involves relatively simple evidence completion, and the paper does not show how the baseline methods fail on the same case.

  2. Table 1 lacks results for many existing methods using Qwen3.6-Plus, which leaves relatively few comparisons under the same base model. In addition, have the authors considered evaluating more conventional retrieval models, such as ColPali, together with Qwen3.6-Plus on MMLongBench?

  3. Given the concern above that only a limited part of the reasoning process is actually symbolic, it is unclear whether the benefits of the proposed framework would persist when using frontier LLMs with substantially stronger native reasoning capabilities.

Questions for authors

Regarding the details of the proposed neuro-symbolic reasoning framework, I have several questions:

  1. The linear combination in Equation (5) appears to merely balance two relevance estimators. The evidence captured by the two estimators is not necessarily complementary, and both could still miss the same relevant evidence.

  2. Building on this issue, once RR in Equation (6) fails to fully cover the ground-truth evidence set, the coverage objective will continue operating over an incomplete evidence set. The repair mechanism triggered by the Evidence Loop also does not seem particularly suited to making the model realize that some evidence has been overlooked. Instead, it mainly guides the model to retrieve evidence that has already been identified as necessary but has not yet been found.

Did I misunderstand the mechanism here?

Other questions for the authors are noted above in the Weaknesses section.

Reviewer_dydFOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a two-stage neuro-symbolic framework for question answering on multimodal long documents. First, the framework decomposes the question into specific evidentiary requirements and selects pages based on page-level relevance, coverage of new requirements, and a penalty for redundancy. Subsequently, it constructs answer specifications and structured reasoning hypotheses, independently verifying evidentiary support, reasoning steps, and answer conditions. A limited correction loop is executed if a lack of evidence is detected, whereas the answer is rejected if specifications are not met. Experimental results demonstrate improved end-to-end accuracy on MMLongBench and DocBench, as well as enhancements in evidentiary page recall and estimated token efficiency on MMLongBench and LongDocURL.

Critical strengths
  1. The method employs a rigorous problem-decomposition logic, and its feedback mechanism offers genuine practical value. It clearly distinguishes between "intra-page relevance" and "cross-page evidentiary sufficiency," as well as between "evidence existence" and "reasoning validity." This allows the framework's modules to collaborate effectively under shared constraints, demonstrating significant practical utility.

  2. The method delivers highly competitive results, with particularly notable performance gains in handling "unanswerable" questions. Compared to replication experiments using Qwen3.6-Plus for DocLens, the proposed method achieved performance improvements of 5.0 points on MMLongBench and 2.9 points on DocBench. The gains were especially pronounced for "unanswerable" questions, with scores rising by 14.4 points and 4.8 points, respectively. Furthermore, regarding evidence localization, the method improved recall while simultaneously optimizing estimated token consumption.

Critical weaknesses

Implementation Details: Please provide the implementation details regarding requirement coverage, verbosity penalties, evidence sufficiency rules, and AcceptRepair. Could you also provide a complete, concrete example that includes the actual input, structured state, and execution results?

Contribution Analysis: Under identical input and reasoning conditions, how does the performance of this method compare to a standalone verifier that does not employ explicit answer specifications? Regarding the "removal of neuro-symbolic reasoning" mentioned in Table 3, which specific operations or model calls were removed?

Comparison Fairness and Costs: Did the implementations of this method and DocLens utilize the same parsing results, tools, and visual inputs? Which stages are covered by the costs shown in Figure 3? Do these costs include the overhead associated with visual tokens or document preprocessing?

Questions for authors

please see Critical Weaknesses

Reviewer_FWJzOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper casts multimodal long-document QA as a two-stage neuro-symbolic process. In localisation, pages are parsed once by MinerU2.5-Pro into a memory M[p_i] = (I_i, T_i, L_i, U_i); the question is decomposed into atomic evidence requirements r_j = (n_j, d_j, tau_j, Q_j); pages are scored by C(p) = alpha*R_mem(p) + (1-alpha)*R_nav(p) and selected greedily under p* = argmax [C(p) + lambda|DeltaCov(p;R,P)| - rho(p,P)] until budget K is reached or marginal gain stops being positive, with explicit page references resolved and merged in. In reasoning, an evidence pack E(P) preserves page identity, layout, OCR text, and localised visual regions; an answer specification S(Q) = (kappa, eta, C, O, omega) is derived; a neural hypothesizer emits ordered steps h_k = (e_k, o_k, y_k, nu_k, delta_k); an independent verifier returns V = (V_con, V_step, V_spec, G, s) with s in {verified, partially verified, not verified}; concrete evidence gaps in G trigger a bounded repair loop over a registered operator set for at most T rounds, after which the same verifier is re-run and a state is retained only if it resolves the target gap or improves verification without invalidating supported constraints; the final answer is produced only if V_spec satisfies omega, otherwise the system abstains. Reported results: 73.0% overall on MMLongBench against a self-reproduced DocLens baseline of 68.0% under the same Qwen3.6-Plus backbone, with 77.3% on the unanswerable subset versus 62.9%; 86.3% on DocBench versus 83.4% (UNA 90.1% versus 85.3%); 68.10% on LongDocURL versus 66.37%. Evidence-page retrieval improves together with cost: on LongDocURL, page recall 69.53 -> 79.66 and full recall 58.13 -> 67.88 with estimated tokens 858.4M -> 509.6M; on MMLongBench, 84.93 -> 87.99 and 77.74 -> 83.09 with 198.6M -> 141.0M. Component ablation on MMLongBench gives 73.0 (full), 72.4 (no repair loop), 71.7 (no symbolic localisation), and 68.8 (no symbolic reasoning).

Critical strengths
  1. The unanswerable-subset results are the most convincing evidence, because they cannot be explained by better answer generation alone. On MMLongBench UNA the method reaches 77.3% against 62.9% for the reproduced DocLens under the same backbone (+14.4), and on DocBench UNA 90.1% versus 85.3%. Since these items require recognising that the document does not support an answer, the gain is consistent with the claimed mechanism — question-derived answer specifications plus step-wise verification, rather than a stronger reader — and it addresses the failure mode the paper is explicitly about, namely plausible answers produced under incomplete evidence. The per-step verification records additionally give trace-level auditability that agentic document frameworks usually lack.
  2. Coverage-aware selection improves localisation and cost simultaneously, which is unusual. Against the reproduced DocLens with the same Qwen3.6-Plus backbone, the method improves LongDocURL page recall by 10.1 points and full recall by 9.7 points while reducing estimated token consumption by 40.6% (858.4M to 509.6M), and on MMLongBench by 3.1 and 5.4 points at 29.0% fewer tokens. The selection objective gives one mechanism for both effects — rewarding newly covered requirements while penalising redundancy — so the two improvements are not traded against each other but follow from the same term, and the token counts are reported alongside the recall numbers.
Critical weaknesses
  1. The "symbolic" component is, in execution, a prompted VLM judgement shared with the rest of the pipeline, so the claimed independence and auditability are weaker than stated. Appendix C states that crop interpretation and semantic support checks "remain neural and may be marked uncertain" and that only metadata lookup, OCR retrieval, and counting execute deterministically; the verifier, hypothesizer, answer generator, and final LLM judge are all Qwen3.6-Plus. Section 3.2 calls the verifier independent on the grounds that it receives the same evidence pack, but identical inputs under a shared backbone do not yield independent errors: a systematic perception bias of the backbone is reproduced in both the hypothesis and its verification. The "auditable verification traces" therefore record one model's self-report about another instance of itself, and the guarantee the framework advertises — that unsupported reasoning is rejected by a check outside the neural proposal — is not in place.
  2. There is no abstention accounting and no statistical support, which together make the state-of-the-art claim unverifiable. The framework abstains whenever V_spec fails, but the paper never reports the abstention rate, abstention precision, the false-abstention rate on answerable items, or accuracy restricted to the answerable subset. This matters directly: if the system also abstains on answerable items, overall accuracy is not comparable with baselines that always answer, and the +14.4 UNA gain could partly reflect a more conservative default. Independently, no confidence interval, significance test, or multi-seed result appears anywhere; the only same-backbone comparison is against the authors' own DocLens reproduction (68.0 -> 73.0), while DocDancer (Gemini-2.5-Pro, 65.9) and DocLens (Gemini-2.5-Pro, 67.6) use different backbones. On MMLongBench MP there are 859 answerable questions, so one point is about nine items and the ablation differences of 0.6 and 1.3 points are within noise. Finally, accuracy is scored by a Qwen3.6-Plus judge while the method's outputs are also Qwen3.6-Plus, so self-preference is not excluded.
Questions for authors

Clarifications (answers could change my assessment):

  1. Please report the abstention rate overall and by subset, the proportion of abstentions that are truly unanswerable, the false-abstention rate on answerable items, and accuracy on the answerable subset only, with the same baselines. Without these, the MMLongBench and DocBench accuracies are not comparable across systems with different refusal policies, and the UNA gain cannot be attributed to verification rather than to increased caution.
  2. Are the hypothesizer, verifier, answer generator, and final judge all Qwen3.6-Plus? If so, what does "independent verification" mean operationally, and what happens if the verifier uses a different backbone such as Gemini-2.5-Pro? Please report inter-verifier agreement on step statuses (Cohen's kappa) and the resulting accuracy.
  3. No error bars or significance tests appear in Tables 1-3 or Figure 5. With 859 MP questions on MMLongBench, are the ablation differences (72.4, 71.7, 68.8 versus 73.0) significant? What is the size of the subset behind Figure 5's 44.7 -> 53.2 and 37.5 -> 57.5?
  4. How were K, alpha, lambda, rho, and T chosen, and what is the sensitivity to each? Since K directly governs the token/recall trade-off, please show that the -40.6% token reduction in Figure 3 is not simply a smaller page budget.

Suggestions for future work (not affecting my assessment): Replacing crop-level semantic support checks with deterministic operators, or at least calibrating and reporting verifier confidence; multi-document extensions, since the problem is currently defined over a single document D = {p_i}; longer repair horizons and multi-step planning, compared against simply increasing K.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response and Summary of Rebuttal Updates

Dear Area Chair and Reviewers,

Thank you for your constructive feedback during the rebuttal period. We summarize our main responses and manuscript updates below.

  • Evaluation Robustness (Reviewer FWJz): Additional analyses show that gains do not come from excessive refusal and remain stable across verifier and judge substitutions. Retrieval gains also persist under matched page budgets (Appendices F.3–F.6).
  • Method Clarity (Reviewer dydF): We added a complete execution trace and clarified the implementation and comparison settings. Explicit answer specifications improve UNA accuracy by 4.66 pp over a standalone verifier (Appendices F.1–F.2; reviewer responses).
  • Symbolic Ablation (Reviewer mqaJ): We clarified the role of symbolic control and showed a 1.9 pp accuracy drop when these components are removed. Expanded comparisons show that our advantage over DocLens persists with a stronger backbone (+4.96 pp) (Appendices F.7–F.10).

Detailed responses are available in the corresponding reviewer threads. Thank you for considering our rebuttal and revised manuscript.

Best regards,
Authors of Submission 24461

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.