acceptodds
Withdrawn from ICLR 2027

Focus on What Matters: Text-Aware, Training-Free Token Pruning for Efficient Document Parsing

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Document parsing methods based on vision-language models (VLMs) typically encode high-resolution document images into dense sequences of visual tokens, resulting in high computational costs. Many of these tokens correspond to blank or background regions, resulting in redundant computation. Token pruning offers an effective way to alleviate this overhead by reducing the number of visual tokens. However, general-purpose token pruning methods suffer from two main limitations: (1) they typically rank visual tokens based on attention scores or feature similarity, potentially discarding tokens from critical text regions in the image; and (2) they struggle to adapt pruning ratios to varying text densities. Both limitations degrade parsing accuracy. To address these limitations, we propose -Doc, a Text-aware, Training-free Token pruning framework for efficient Document parsing. -Doc employs an off-the-shelf text detector to locate text regions and maps them onto the VLM's token grid to construct a retention mask. The framework incorporates two key designs: (1) Flexible Pruning Placement, supporting pruning before or after the visual encoder, with pre-encoder pruning further reducing the encoder's computational cost; and (2) Coverage-Adaptive Thresholding, which uses region dilation and bounding-box merging to regulate mask coverage, thereby adaptively adjusting the token retention ratio. Extensive experiments demonstrate that -Doc prunes approximately 30% of visual tokens on average while largely maintaining parsing accuracy, achieving a better accuracy–efficiency trade-off than existing methods. The code will be publicly available.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Document Parsing, Training-Free Token Pruning, Efficient Vision-Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_49JMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes T³-Doc, a text-aware, training-free token-pruning method for reducing redundant computation in document parsing. It detects text regions, merges and expands them to build an adaptive token-retention mask, and applies the mask either before or after visual encoding. On OmniDocBench, it prunes about 30% of visual tokens while maintaining parsing accuracy, and reduces TTFT by up to 30.9%.

Critical strengths
  1. The paper is clearly written, and the proposed method is simple and practical.

  2. The method uses text regions to preserve important document content, adapts token retention to each page, and supports pruning before or after visual encoding without additional training.

Critical weaknesses
  1. Although T³-Doc is evaluated on several end-to-end document parsers, the paper does not directly assess how well detected text regions cover all information needed for parsing, particularly non-text structural cues such as table boundaries, figures, and spatial relationships. Its discussion of missed detections addresses part of this issue, but the impact of uncovered informative regions remains unclear.

  2. The efficiency results exclude text-detector latency, and the cost of mask construction is not separately reported. Consequently, the paper does not establish the method’s end-to-end latency benefit when its additional processing is included.

  3. The paper compares against attention-based and feature-based pruning methods, but does not isolate the contribution of the selection criterion under identical pruning placement and per-page token budgets. It also lacks a component ablation of box merging and region dilation.

Questions for authors
  1. How often does the retention mask exclude ground-truth content or structural elements, and how does this affect parsing accuracy?

  2. Could you report end-to-end latency, including PP-OCRv6-Medium detection and mask construction, alongside a breakdown of each component?

  3. Could you compare text-region, attention-based, and feature-similarity-based selection while holding pruning placement and per-page retained token counts fixed? Could you also ablate box merging and region dilation separately?

  4. Regarding PreV, wouldn't the irregular shape changes of the visual embeddings before entering the encoder cause the encoder to fail during execution?

Reviewer_uos2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes T3T^3-Doc, a visual token pruning framework for document parsing that requires no additional training. Its core approach uses an off-the-shelf text detector to localize text regions, maps the detected bounding boxes onto the VLM’s visual token grid, and retains tokens that intersect these regions.

Critical strengths
  1. The method addresses the specific requirements of document parsing while maintaining strong performance under token pruning.
  2. The paper evaluates multiple backbones and pruning locations, demonstrating a degree of cross-model applicability on the benchmarks considered.
Critical weaknesses
  1. The paper’s added value largely hinges on the effectiveness of CAT and early pruning. However, the current evaluation does not compare the original detection boxes, dilation alone, merging alone, and the full CAT pipeline. It also lacks a comparison between fixed and page-adaptive token budgets under the same average budget.

  2. The paper does not compare against existing acceleration methods designed for document understanding. Table 1 primarily compares general-purpose visual token pruning methods, while document-specific methods cited in the paper, such as LayoutLite and FastOCR, are not included in the experiments.

  3. Both the main experiments and the evaluations on three additional backbones are conducted on a single dataset, which is insufficient to support broad claims of robustness.

Questions for authors

Please refer to weaknesses.

Reviewer_e56eOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes T³-Doc, a training-free visual token pruning framework for document parsing. It constructs retention masks using an external text detector, preserves surrounding context through region merging and expansion, and supports pruning before or after the visual encoder. On several benchmarks, the method largely preserves parsing performance while targeting approximately 70% mean visual-token retention, with evaluations across multiple document VLMs and general VLMs. The authors also made some convincing ablations studies to explore the effect of retention ratio and text detectors

Critical strengths
  1. The approach is well motivated and practically useful. Text locations provide a direct and interpretable pruning signal for documents. The method requires no updates to VLM weights and offers flexibility through two pruning placements.
  2. The experimental coverage is substantial. The paper compares multiple pruning baselines and evaluates different backbones, retention ratios, detectors, and attention implementations. These experiments provide meaningful support for preserving aggregate parsing quality in the tested settings.
Critical weaknesses
  1. The incremental novelty is insufficiently established. The core design combines an existing text detector with spatially guided token selection and is technically straightforward. Prior work[1] has also used detected text regions to guide visual-feature pruning. The paper should therefore clarify its substantive contribution beyond changes in application setting and component integration. [1] Garcia-Bordils, S., Karatzas, D., Rusiñol, M. (2023). Accelerating Transformer-Based Scene Text Detection and Recognition via Token Pruning. In: Fink, G.A., Jain, R., Kise, K., Zanibbi, R. (eds) Document Analysis and Recognition - ICDAR 2023.
  2. Preservation of essential non-textual information and practical efficiency remain unresolved. Structural cues in complex tables, such as separators and merged-cell boundaries, may lie outside detected text regions. Merging and expansion can mitigate this risk. Additionally, efficiency measurements exclude the required text-detection overhead, leaving the end-to-end benefit unclear.
Questions for authors
  1. Can the authors provide results on complex-table, charts and retention-mask visualizations showing whether essential structural cues are preserved? I wonder if these hard cases fall within the intended scope.
  2. After including text detection, mask construction, and other additional processing, what are the improvements in end-to-end TTFT and full-document generation latency?
  3. The paper preserves original positional information and retains complete patch-merging groups for PreV. Nevertheless, physically removing tokens changes spatial coverage and the distribution of contextual information. Could the authors clarify how the pruned sequence remains consistent with the positional, grid, and attention metadata required by the model? Is there any analysis of sensitivity to this departure from pretraining or post-training inputs, particularly for PreV, where information is removed earlier?
  4. Using the same text detections and spatial retention regions, have the authors compared physical token removal with masking the corresponding input-image regions, for example through background-color filling, blacking out, or blurring?
  5. When the text detector misses or inaccurately localizes content, essential visual information may be removed before subsequent processing. How is this error propagation mitigated, how much can CAT recover, and how strongly do residual errors affect final parsing performance?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.