acceptodds
Under review as a conference paper at ICLR 2027

Layout-Aware Latent Evidence Reasoning for OCR-Free Key Information Extraction

Abstract

Key information extraction (KIE) requires associating textual values with their spatial context and requested fields. OCR-free models typically learn these associations implicitly from answer supervision, while OCR-augmented methods expose explicit evidence at the cost of external OCR dependence and variable-length context at inference. We propose **LAYER**, a **LAY**out-aware latent **E**vidence **R**easoning framework that uses OCR as training supervision to learn a compact internal representation of document evidence. Given a document image and a field query, LAYER predicts a fixed-length sequence of continuous visual latent tokens in parallel. The same language model then conditions on these latents and the original image-query context to autoregressively generate structured answers. Two complementary objectives guide learning: Layout-aware Feature Alignment aligns the predicted latents with visual features of OCR text rendered in its page layout, while Confidence-Gated Contrastive Learning associates answer-value representations with supporting OCR blocks through reliable field-region matches. A two-stage training scheme initializes the latent interface and then jointly optimizes structured generation with both objectives as auxiliary supervision. Inference requires only the document image and query. Experiments on public KIE benchmarks show pooled micro-F1 gains of 1.66%-3.45% over answer-only fine-tuning across three backbone sizes. A controlled timing comparison further shows a end-to-end speedup over OCR as Chain-of-Thought, with answer length fixed.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Key Information Extraction, Optical Character Recognition

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_Ti6yOpenReview ↗
Rating
3(+1)
Confidence
1(−2)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LAYER, a framework for key information extraction that uses OCR text and layout during training to learn compact evidence representations from document images. It combines layout feature alignment with confidence gated contrastive learning to associate answer values with their source regions, then generates structured answers without OCR at inference time. Experiments on public KIE benchmarks demonstrate consistent gains over answer-only fine-tuning across three backbone sizes and a 2.03× end-to-end speedup over OCR-CoT under a fixed answer-length setting.

Critical strengths
  1. combine both page layout and individual fields. Layout feature alignment helps the model learn page structure, while contrastive learning connects answer values to their source regions.

  2. The model produces a fixed sequence of 64 latent vectors in one forward pass, and the paper reports a 2.03 times speedup over OCR CoT when answer length is held constant.

Critical weaknesses
  1. With the development of end-to-end OCR models such as DeepSeek-OCR 2[1] and GLM-OCR[2], it is unclear what additional benefit LAYER’s explicit spatial layout supervision provides. The authors do not discuss this question or compare LAYER with these models on KIE tasks.

  2. The experiments lack comparisons with some methods discussed in the Related Work section, such as LayoutLM and SCOLAR.

  3. The authors do not discuss the training cost of LAYER or compare it with that of other methods.

[1]https://arxiv.org/abs/2601.20552 [2]https://arxiv.org/abs/2603.10910

Questions for authors
  1. Why does LAYER require these relatively complex training mechanisms instead of directly using representations from the encoder of an existing multimodal document model, such as DeepSeek-OCR 2? What is the advantage of the proposed approach? Could the authors also provide KIE comparisons with recent end-to-end OCR models?

  2. At line 208, how was Z* chosen as the teacher target? Do features extracted from the rendered image by the visual encoder have a particular meaning? Why not use features from an OCR encoder or hidden states from the prefill stage?

  3. Could the authors provide additional comparisons with methods mentioned in the Related Work section, such as LayoutLM and SCOLAR?

  4. Could the authors discuss the training cost of LAYER and compare it with that of other methods?

Reviewer_W23rOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

LAYER uses OCR supervision during training to improve OCR-free key information extraction. It predicts 64 visual latents in parallel and trains them with a layout-preserving rendered teacher, alongside a contrastive objective linking answer values to OCR blocks. The method improves over answer-only SFT across three model sizes and reduces latency relative to OCR-CoT in a fixed-answer-length comparison.

Critical strengths
  1. The method is well-motivated with clear figures and a clean derivation of the training objectives. The paper addresses a practical problem: benefiting from OCR evidence without requiring an OCR system at inference. The method is reasonably clear, including the distinction between parallel latent prediction and autoregressive answer generation.

  2. Clear, controlled comparisons to answer-only SFT and a RoT-style latent baseline; ablations disentangle the contributions of LFA and CGCL. Results are reported across multiple datasets (CORD, SROIE, FUNSD, POIE) and three backbone sizes, demonstrating consistent gains and complementarity with model capacity.

Critical weaknesses
  1. My main concern is whether is the robustness to varied layouts, fonts, and noisy documents is not tested. Inference is OCR-free, but training relies on OCR and rendered targets. The paper does not specify the OCR source or test how errors and rendering choices affect performance.

  2. The added value over RoT is unclear. The layout-aware teacher slightly lowers F1 and ANLS, and full LAYER gains only 0.28 F1 points over RoT. SFT+CGCL and RoT+CGCL controls would help show which components account for the gains.

  3. Better extraction does not by itself show spatial grounding. CGCL pairs have nearly identical text, so the gains may mainly come from text matching. The decoder also sees the original image, leaving the role of spatial information in the latents unclear.

Questions for authors
  1. What OCR source was used for each dataset, and which engine/version, if applicable? Were the text and boxes predicted or taken from dataset annotations? Please describe the rendering settings and test sensitivity to OCR errors or rendering choices. This matters for varied real-world documents, even though inference is OCR-free.
  2. How do SFT+CGCL and RoT+CGCL perform under matched training budgets? Does the Table 2 RoT baseline generate latents autoregressively or use LAYER’s parallel approach?
  3. What fraction of fields receives CGCL supervision after filtering and the two-value cap? Does the gain remain without bounding-box features, using the same matched pairs? Testing with shuffled latents while keeping the image unchanged would also help clarify their role.
  4. Some details. Please check the writing and notation more carefully. For example, use consistent notation for LCL\mathcal{L}_{\mathrm{CL}} and LCGCL\mathcal{L}_{\mathrm{CGCL}}, clarify “copy-grounding modules” and “auxiliary copy prediction” in Section 3.4, remove the repeated H20 statement in Section 4.1, and report absolute score changes in percentage points.
Reviewer_AgMBOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LAYER, a framework for OCR-free key information extraction that uses OCR-derived supervision during training. Given a document image and a field query, the model predicts a fixed-length sequence of continuous visual latents in parallel, then generates a structured answer conditioned on these latents and the original image–query context. Layout-aware Feature Alignment supervises the latents using visual features of OCR text rendered at its original page coordinates. Confidence-Gated Contrastive Learning aligns answer-value representations with reliably matched OCR blocks. Experiments on CORD, FUNSD, SROIE, and POIE report pooled micro-F1 improvements of 1.66–3.45 percentage points over answer-only fine-tuning across three backbone sizes. A controlled timing comparison reports a 2.03× end-to-end speedup over OCR-CoT with answer length fixed.

Critical strengths
  1. A practical approach to transferring document evidence into an OCR-free model. Using OCR-derived textual and spatial information during training, while avoiding external OCR at inference, addresses a meaningful deployment concern. The fixed-length latent interface offers a concrete implementation, and the paper clearly describes how predicted latents are used during both training and inference.

  2. Consistent gains over answer-only fine-tuning across model sizes. Table 3 reports improvements in both pooled micro-F1 and ANLS at all three evaluated backbone sizes. Appendix B additionally reports improvements over SFT on FUNSD and POIE. These results support the usefulness of the overall training recipe within the evaluated settings.

Critical weaknesses

The central rendering-and-alignment framework closely follows Render-of-Thought, which already uses visual features of rendered text to supervise continuous latent reasoning. Parallel visual latent construction also has precedents in DeepLatent and SCOLAR. Likewise, grounding for KIE and text–region contrastive alignment have been explored in STNet and Wukong-Reader, respectively. These methods are not identical to LAYER, but they place its novelty primarily in the specific combination of layout-preserving OCR supervision, the shared-backbone parallel latent interface, and confidence-gated field-level alignment. Such a combination could be valuable, but Table 2 does not convincingly establish the contribution of its distinctive components. RoT achieves 89.70 micro-F1, the layout-aware variant without CGCL achieves 89.55, and full LAYER achieves 89.98. Thus, layout-aware supervision alone decreases micro-F1 and ANLS, while the complete method improves micro-F1 over RoT by only 0.28 percentage points.

Questions for authors
  1. What are the results of RoT + CGCL without the layout-aware teacher? Does full LAYER consistently outperform this variant and RoT across multiple random seeds? Please also clarify whether the RoT control uses autoregressive or parallel latent generation and how its training budget is matched.

Comments on the paper

1 comment

Thank you for the further clarification. I appreciate the authors' efforts and the additional explanations. I raise my score but lower the confidence. I still keep neutral about the work because the entire article lacks a sense of closure.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.