acceptodds
Under review as a conference paper at ICLR 2027

FastOCR: Dynamic Visual Fixation via KV Cache Pruning for Efficient Document Parsing

Abstract

Vision-Language Models (VLMs) have shown strong promise on Optical Character Recognition (OCR), yet the sheer number of visual tokens needed to encode dense documents makes inference prohibitively expensive. Existing methods like SnapKV evict visual tokens, which often fails on OCR tasks given the high information density of document images. We observe that, although document images appear globally dense and seemingly unprunable, a VLM's top-attended visual-token sets are line-local and drift gradually across decoding steps, much as a human reader fixates on successive words rather than on a whole page at once. Turning it into a method, we propose **FastOCR**, a training-free framework with two complementary modules: *Focal-Guided Pruning* identifies a small set of focal layers and selects the most task-relevant visual tokens at each step, while *Cross-Step Fixation Reuse* warm-starts each step from the previous one. FastOCR changes only which cached entries are attended and evicts none, so every token stays recoverable at every later step. Under matched nominal reference attention budgets, FastOCR outperforms every baseline we evaluate on OmniDocBench and olmOCR-Bench, and generalizes across various VLMs. On Qwen2.5-VL it retains **98.3%** of the unpruned model's accuracy on OmniDocBench and **97.3%** on olmOCR-Bench, with a **5%** visual-token budget at pruned layers. Latency measurements reach up to **3.4** attention speedup on olmOCR.

Primary area:
foundation or frontier models, including LLMs
Keywords:
vision-language models, efficient inference, optical character recognition, visual token pruning, document understanding, training-free acceleration

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.40 over 5 reviews, above 68% of the others.
Reviewer_bqC8OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes FastOCR, a training-free method that speeds up VLM-based document parsing by cutting the number of visual KV entries attended to at each decoding step. It builds on the observation that a model’s top-attended visual tokens are line-local and drift gradually during decoding, much like a human reader’s gaze. Accordingly, it combines Focal-Guided Pruning, which selects a few focal layers to pick the top-attended image tokens and propagates them to the remaining layers, with Cross-Step Fixation Reuse, which warm-starts each step from the previous one. Because the full KV cache is retained, no visual token is ever permanently lost. On OmniDocBench, FastOCR retains up to 98.3% of the unpruned accuracy on Qwen2.5-VL with only 5% of image tokens attended at pruned layers, outperforming all four baselines and reaching up to 3.4× attention speedup.

Critical strengths
  1. The writing is clear and easy to follow.

  2. The problem and insight are well-motivated.

  3. The method is deployment-friendly.

Critical weaknesses
  1. One design of this method is selecting focal layers by image-attention ration. But there is no comparison between this selection method and other simple alternatives, like random selection, last few layers, and etc. I think this ablation can further judge the effectiveness about the selection of the focal layers.

  2. Did you analyze what characteristics the selected focal layers? Could the online warmup selection be replaced with offline selection?

  3. Could you clarify the experimental setup in Table 1? How did you delete or retain tokens in each baselines? At which stage and layers is each baselines' pruning applied under the matched budget? Some gaps are very large and implementation details are needed to support fair comparison.

Questions for authors

See weaknesses.

Reviewer_McEjOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes a training free method for KV cache pruning in document OCR tasks. Specifically, drawing inspiration from the way humans read text (with a high degree of locality and sequentially), the proposed method prunes tokens dynamically, both across the depth of the model and over decoding steps. Specifically, the proposed method first identifies important layers for the LVLM, via image-text attention over the prefilling stage or over a few decoding steps. These layers are used via top-k selection to pick the visual tokens over which the model attends. Finally, the last layer's attention pattern is used to inform the first layer at the start of the next step. The proposed method is designed for document data and is found to be effective in benchmarks, maintaining a lot of the baseline model's performance while improving inference speed.

Critical strengths

The proposed method is intuitive and easy to understand, grounding its motivation on observations from how humans read text and validated via analysis on LVLM's behavior. The results achieved are promising, particularly the fact the performance barely drops even at very high pruning levels.

Critical weaknesses

The paper's main weakness relates to the lack of adequate discussion and appropriate comparison with other works. Specifically, the paper does not mention several prior works that are much closer to its subject, namely training-free visual token/KV cache compression/pruning for LVLMs. Such papers include [1,2,3,4,5,6] (all of which are training free). Of them, only [3] is included in the relared works, but it is not added to the comparisons. This weakness extends further to the benchmarks: only two benchmarks are tested (OmniDocBench and olmOCR-Bench), and popular OCR benchmarks such as OCRBench, DocVQA and TextVQA are ommited, making the method hard to position compared to relevant literature. Finally, even for the methods for which results are presented, it is unclear under which setting they are tested. That is only examined in Appendix sections B and C, but the explanation is extremely convoluted, so that it is unclear what settings were used in Table 1 for each method. Furthermore, it is noted in the Appendix that the alignment relies on a theoretical FLOP calculation as opposed to realized FLOPs or latency. This brings the fairness of the comparisons into question: the choice to not use latency as the alignment metric is not justified, and the proposed method is not contrasted with other methods in terms of latency or VRAM usage, two critical components that should complement the performance comparisons presented in the paper.

  • [1] Shu, Jinsong, et al. "MM-ShiftKV: Decode-Aware Prefill-Stage KV Selection for Multimodal Large Language Models." Findings of the Association for Computational Linguistics: ACL 2026. 2026.
  • [2] Wang, Jiahui, et al. "Sparsemm: Head sparsity emerges from visual concept responses in mllms." 2025 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2025.
  • [3] Wan, Ben, et al. "Rtprune: Reading-twice inspired token pruning for efficient deepseek-ocr inference." arXiv preprint arXiv:2605.00392 (2026).
  • [4] Khaki, Samir, et al. "Sparsevila: Decoupling visual sparsity for efficient vlm inference." 2025 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2025.
  • [5] Yang, Cheng, et al. "Topv: Compatible token pruning with inference time optimization for fast and low-memory multimodal vision language model." 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025.
  • [6] Liu, Xuyang, et al. "Mixing importance with diversity: Joint optimization for kv cache compression in large vision-language models." International Conference on Learning Representations. Vol. 2026. 2026.
Questions for authors
  • What is the rationale behind the choice to align the methods using the theoretical FLOP count methodology presented in the Appendix as opposed to latency or realized FLOPs??
  • How does the proposed method compare with other methods in terms of inference latency?
  • How does the proposed method score on the benchmarks mentioned above?
6 comments

Dear Reviewer McEj,

With the discussion period concluding in a few days, we wanted to politely check in and see if our two-part response and the blue-highlighted revised manuscript have resolved your concerns. Following your feedback, we expanded our related work and empirical comparisons with recent training-free LVLM compression methods (Sec. 2, App. C.3), clarified the Table 1 retention ratios (Sec. 4.1), and added end-to-end wall-clock latency / VRAM analyses (App. D.1, App. K) as well as a full 1,000-question OCRBench evaluation (Sec. 4.2, App. F).

Please let us know if these revisions address your questions or if we can provide any further clarification!

Reviewer_McEjOpenReview ↗

Response to the author's rebuttal (part 1)

I thank the authors for their responses and for the effort they made to address the issues raised by me and other reviewers.

The revision improves the manuscript in several respects, and I want to acknowledge them:

  • Section 4.1 now clearly states the retention ratio for each baseline in Table 1, and makes clear the comparisons are, in terms of retention ratio, fair to the other methods.
  • The related works include other works previously missing.
  • OCRBench evaluations for the proposed method and the baseline LVLM are a good addition to the paper.
  • The end-to-end timings and VRAM measurements against the unpruned model in appendix D.1 fulfil a must-have requirement for the paper. That said, several of my concerns remain only partly addressed. I respond to each point below.

Q1 / W3: Rationale for nominal FLOPs alignment

The added retention ratios make Table 1's settings clear, and I consider that part of W3 resolved. To be clear, I have no objection to matching the attended token count as a way to compare accuracy at equal access. However, I still find the rationale for using nominal FLOPs as the measure of the method's efficiency unconvincing.

  • A1. Given how the FLOPs are computed in this theoretical formulation, they are equivalent to the attended token count at a single reference step and carry no further information.
  • A2. This approach only counts the attention and omits the FLOP costs of other ops in the model (e.g. the MLPs of the transformer blocks), so the reductions it shows don't necessarily translate into actual per-step compute or latency over the full architecture. The authors' measurements show the gap: for Qwen2.5-VL at B8 / 4096 tokens the 2.16x nominal FLOPs reduction (15.1G to 7.0G) translates into only 1.13x speedup per decoder forward (Table 2).
  • A3. This metric also omits costs such as score computation and per-layer gather operations, which grow as the sequence grows. Even for attention alone, in the same setting a 6.16x reduction in attended tokens (665 vs 4096) translates into only 1.66× measured speedup (Table 2).
  • A4. In the response, the nominal alignment is motivated as being "independent of kernel differences", and appendix D states that FastOCR's token selection is compatible with FlashAttention-2 fused attention kernels. I understand that a new method may not yet have fully optimized kernels and may not reach its full potential with existing tools. However, I do not think this is a reason to leave out the latency comparison or actually measured FLOPs. If FastOCR is slower than other methods for these reasons, this should be reported and discussed in the paper, so that it is clear where the method currently stands, and comparisons should be presented with other methods under as equal a setting as possible to account for such differences.

Summary: I consider nominal FLOPs to be an inadequate metric for evaluating the efficiency of this method, more so considering they only cover attention and are calculated rather than measured. The paper is transparent about these limitations (Appendices B and C), but stating them does not make up for the missing measurements.

Reviewer_McEjOpenReview ↗

Response to the author's rebuttal (part 2)

Q2 / W4: End-to-end inference latency and VRAM usage

My question was how FastOCR compares with other methods in terms of latency. The new timings in appendix D.1 are useful, but they only compare FastOCR with the unpruned model.

  • B1. As far as I could tell, no latency or VRAM numbers are reported for any of the baselines in any setting. Every timing in the paper compares FastOCR with the unpruned model only. The ~84% reduction in HBM read traffic is also calculated from the attended token count (665/4096) rather than measured.
  • B2. The authors state openly that peak VRAM is unchanged relative to the unpruned model. This means that, by the paper's own acknowledgement, FastOCR is worse in terms of VRAM usage than the eviction methods it is compared with. Eviction methods also attend over a smaller contiguous cache of the same average length, without any of the per-step processes FastOCR requires to select and gather the tokens it attends to. These factors make it unclear whether FastOCR is faster than them in practice, and make the absence of latency comparisons a serious gap.
  • B3. The paper and the response state that FastOCR targets long documents and that it is not built for short outputs (these finish within the warmup and prefill dominates so compression methods are more suitable). Appendix K also notes that selection and gather costs may outweigh the saved work when attention is a small part of the forward. However, there is no concrete definition of what constitutes a sufficiently long document. From the paper, a user cannot tell at which output length or number of image tokens FastOCR becomes preferable to the competing methods. This should have been explored, on top of the generally missing latency comparisons.

Summary: This point remains unaddressed with respect to competing methods. Since FastOCR is by design worse than eviction-based competitors in terms of VRAM, latency is its main strong point, and it should be evaluated and compared thoroughly, not only through theoretical and calculated metrics. Any form of comparison would do (at equal budget, equal accuracy or equal latency), but none is provided, even though this was also requested by Reviewer bWAY. At a minimum the paper should have measured latency comparisons with competing methods in some setting. Ideally, latency and performance should be reported over a range of output lengths and input sizes (e.g. with appropriate benchmark datasets or subsets of them), so as to fully support the method's effectiveness and to make clear the settings it targets. As it stands, the proposed method's goal is improved latency in long documents (otherwise what is the point of attending over fewer tokens), but no comparisons are made to demonstrate this except with the unpruned model, making it impossible to assess FastOCR relative to competing methods for any given document length.

Q3 / W2: Evaluation on OCRBench and other benchmarks

I thank the authors for the full OCRBench evaluation, which I consider a good addition. However, it only partly addresses my concern, which was that the method is hard to position relative to the relevant literature.

  • C1. The OCRBench results in the paper only compare FastOCR with the unpruned model, not against other methods. Comparisons with other methods on OCRBench given in the response are, without explanation, on a 100-sample subset as opposed to the full benchmark, and don't include the methods of Table 1.
  • C2. DocVQA and TextVQA are still not evaluated. I understand the authors' reasoning which is that short-answer benchmarks are not the proposed method's target, but that does not constitute an argument against including them with the appropriate context. If anything, the argument that prefill dominates latency in these tasks supports stating them as an explicit limitation (see C3).
  • C3. If short-answer tasks are outside the method's scope, this should be stated clearly in the paper as a limitation, together with a quantitative definition of what short-answer means. If short-answer tasks are instead within scope, relevant benchmarks and comparisons are needed to support statements such as "FastOCR generalizes cleanly to diverse recognition, KIE, and VQA tasks" (Appendix F). Importantly, the response's OCRBench results show prefill compression methods (MM-ShiftKV, MixKV) matching FastOCR's accuracy with lower VRAM, so it is not clear that FastOCR outperforms them there. This makes the question of what document lengths the method excels at even more relevant, and it should be answered concretely and comprehensively.

Summary: I consider this point only partly addressed.

Reviewer_xBQPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper is on speeding up OCR when using a VLM during decoding. To do so, the authors are based on the observation, that for transcription,  during decoding, a VLM does not need to use all visual tokens, but instead can parse the most relevant ones at the specific step of decoding. This is referred to as "Dynamic Visual Fixation" in the submitted work. At the next step t, the focus can move along the text in the way we humans read a document. The work is split into two parts, focal-guided pruning (FGP) and cross-step fixation reuse (CSFR). FGP, during warm-up decoding, records the image-attention ratio and, based on this, chooses a specific small set of layers that are called focal. These focal layers plus layer 0 have access to and attend to the full KV cache and are able to select the most important image tokens and pass the indices to the subsequent non-focal layers. Then CSFR reuses the previously selected image-token indices until the first available focal layer is reached again. Once the first focal layer is reached, it attends to the full cache and updates the image-token selection. One important note is that  the full KV cache is retained during the whole process. The only thing changing is the selection of the image-token indices at non-focal layers.

Critical strengths
  • Strong results on OmniDocBench. For Qwen2.5-VL overall falls from 67.31 to 66.20 with selected layers access only 5% of image tokens.
  • The method is train-free and straightforward.
  • Appendix reporting is transparent. Many details of the paer’s setup are reported, i.e how budgets are matched, what the timings exclude, and that memory is not reduced.
Critical weaknesses
  • Novelty. I think that the novelty of this work is somewhat limited. Similarly to FastOCR, TidalDecode [1] proposes a token selection mechanism for LLM decoding. It proposes selecting top tokens at a few designated layers and reusing those positions in the remaining layers. Similarly, OmniKV [2] proposes focal layers whose selected indices are reused by subsequent non-focal layers. Given these similarities, I believe that the authors should i) cite the above and ii) use it as baselines where possible
  • Baseline comparisons are a bit biased towards FastOCR.  SnapKV, DivPrune and CDPruner apply KV-cache eviction, reducing the number of stored tokens. The main objective is memory reduction. On the other hand, FastOCR and Quest keep the full KV cache in memory, so comparing the three aforementioned baselines with FastOCR and Quest is a comparison of methods with different objectives.
  • Given that the three methods (SnapKV, DivPrune and CDPruner) delete parts of the image to save memory, losing accuracy is not so surprising.  Quest, on the other hand, like FastOCR, keeps everything and selects vision tokens at each decoding step. However, I find it a bit surprising that Quest results are so low.  My understanding is that the authors used the official implementation (https://github.com/mit-han-lab/Quest) of Quest. Could the authors confirm this? However, it seems built on text-only LLMs. How did the authors adapt to VLMs? What was changed? What is the page size used? Are image tokens treated as text tokens? Could the authors provide details on the adaptation?
  • In Table 4, the contributions of FGP, focal layer choice,  and CSFR, reuse of the previous step’s indices, are not clearly identified. The improvement from adding CSFR is apparent but during this  comparison, the placement and number of focal  layers changed.
  • Adding FGP changes both full-cache access and computational cost, while adding CSFR also changes the placement of full-attention layers. Therefore, the reported gains cannot be attributed to either component alone. Comparisons at matched computational budgets, with fixed full-attention layers when testing CSFR, would provide stronger evidence. Have authors made an experiment they keep exactly the same focal layers and just replace the reused set with layer 0 choice? Have they also considered the worst case scenario, i.e replace the previous step indices with random ones?
  • Quality and speed are not measured under the same configuration for olmocr. In Tab. 2 speed is measured with ρ = 0.1, τ = 0.05. Quality in Table 3 is measured with ρ = 0.2, τ = 0.1 (chosen by a small grid search). Why did the authors choose this? It seems to me that the quality is measured with in a more expensive setting. That is, the 3.36× attention speedup on olmOCR does not have its accuracy number. Similarly, the 96.5% quality retention has no matching speed number.

[1] TidalDecode: Fast and Accurate LLM Decoding with Position Persistent Sparse Attention Lijie Yang, Zhihao Zhang, Zhuofu Chen, Zikun Li, and Zhihao Jia  ICLR, 2025 [2] OmniKV: Dynamic Context Selection for Efficient Long-Context LLMs, Jitai Hao, Yuke Zhu, Tian Wang, Jun Yu, Xin Xin, Bo Zheng, Zhaochun Ren, Sheng Guo, ICLR 2025

Questions for authors

How is FastOCR different from TidalDecode and OmniKV?  Both works select tokens at a few focal layers and reuse those indices in the other layers. I understand that FastOCR is applying selection only to image tokens but I am unsure how different the works are.  How does FastOCR compare against TidalDecode and OmniKV at a matched budget? What are Quest's settings? Please report page size and selection granularity. Which repository was used, how was it adopted? Do authors have an explanation for the dramatic performance drop of this method?  How does Quest perform with page size 1 or with a per-step top-k? Is the Quest implementation based on the official code? If so, what was changed to work with VLMs? Are there results where the focal layers stay exactly the same and only CSFR is replaced by layer 0's own selection? What happens if the reused set is replaced with random tokens, or with tokens from several steps earlier? What was different about the configuration for omlOCR when measuring quality and speed?

2 comments
AuthorsOpenReview ↗

Follow-up on Our Response to Reviewer xBQP

Dear Reviewer xBQP,

As the discussion period draws to a close in a few days, we wanted to politely check in and see if you have had a chance to review our two-part response and the blue-highlighted revised manuscript. In particular, following your suggestions, we added controlled comparisons with TidalDecode, OmniKV, and Quest (Sec. 2, App. C.3), single-variable ablations isolating FGP and CSFR (Sec. 4.3, App. H), and unified olmOCR-7B end-to-end latency evaluations (Sec. 4.2, App. D.1, App. K).

Please let us know if these revisions and experiments address your concerns, or if there are any remaining questions we can clarify before the deadline!

AuthorsOpenReview ↗

Follow-up on Our Response to Reviewer xBQP

Dear Reviewer xBQP,

As the discussion closes today (11.18), we wanted to briefly follow up on our two-part response. We would appreciate knowing whether it addresses your concerns, or whether any specific points remain unresolved. We remain available to provide clarification before the deadline.

Thank you for your time and consideration.

Reviewer_xU2wOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies training-free KV cache pruning to speed up VLMs on dense OCR tasks. Most existing visual token pruning or KV eviction methods (like SnapKV or FastV) permanently drop tokens early on. That works fine for general VQA where a few patches are enough to spot the main object, but it breaks OCR because visual tokens dropped at step 10 might contain text needed at step 200.

The authors observe that during OCR decoding, VLMs behave a lot like human readers: at certain middle-to-late "focal" layers, attention focuses on a few text lines and moves smoothly across steps (consecutive top-5% attended sets have around 0.51 IoU, vs. 0.14 when step order is shuffled). Based on this Dynamic Visual Fixation pattern, they propose FastOCR. It keeps the full KV cache in memory without evicting anything. After a short 9-step decoding warmup to find the focal layers with the highest image attention, only Layer 0 and those focal layers run full attention to refresh the top-kk visual token indices. All other non-focal layers just gather and attend to those 5% selected visual tokens (plus text tokens). Early layers before the first focal layer reuse the deepest focal layer's token set from the previous decoding step (CSFR). Tested on Qwen2.5-VL, dots.ocr, olmOCR, and Nanonets-OCR across OmniDocBench and olmOCR-Bench, FastOCR keeps 96% to 100% of the unpruned accuracy under a 5% visual budget at pruned layers, easily beating SnapKV, DivPrune, CDPruner, and Quest, while getting up to 3.36x attention speedup (1.73x decoder-forward speedup) on olmOCR-7B.

Critical strengths
  1. The empirical analysis of how VLM attention moves during OCR decoding (Section 3.1 and Appendix H) is very thorough. By comparing consecutive top-5% visual token sets against both random uniform samples and step-shuffled controls, the authors clearly show that attention is line-local and shifts gradually over time, rather than just sticking to a static set of favorite tokens. This gives a very solid reason for doing dynamic index gathering instead of permanent token eviction.

  2. The method is simple, training-free, and works out-of-the-box across four different OCR models while baselines collapse at tight budgets. I also really appreciate how transparent and careful the experimental accounting is in the appendices. Matching nominal GQA FLOPs across baselines (Appendix B and C), checking output lengths and repetition loops (Appendix I), and openly discussing limitations (Appendix J) make the empirical claims very trustworthy.

Critical weaknesses
  1. Zero reduction in peak GPU memory: Unlike physical eviction methods (such as SnapKV) whose primary motivation is freeing up VRAM to support larger batch sizes or long contexts, FastOCR retains the entire KV cache in GPU memory. It only prunes the attention computation (FLOPs/latency). In real-world multi-page document parsing deployments where GPU memory footprint is often the primary bottleneck, FastOCR does not relieve the memory wall.

  2. Wall-clock speedup diluted by non-contiguous tensor gathering: While the paper reports an impressive 3.36x attention speedup, the decoder forward speedup drops to 1.73x. In real-world deployments, dynamic index gathering and non-contiguous memory access introduce non-trivial kernel overhead. When combined with vision encoding and the 9-step full-attention warmup, the end-to-end wall-clock throughput gain is likely much more modest than the FLOPs reduction suggests.

  3. Low information density in Figure 3: Figure 3 occupies substantial main-text space (nearly one-third of Page 5) but conveys very low information density. Panel (a) uses two large pie charts to convey merely two scalar numbers (42% vs. 14%), while panel (b) displays a 50-step heatmap for only a single sample where attention bands remain virtually static across steps. Replacing this with a compact layer-wise attention profile (e.g., mean and variance across all 32 layers over multiple benchmark samples) would substantially improve the data-ink ratio and provide stronger statistical evidence.

Questions for authors
  1. In Appendix A, the focal layer set CC is profiled and shared across the whole batch. Figure 6 shows that on OmniDocBench, almost the exact same focal layers get picked across all samples. Still, if a batch mixes very different inputs (for example, a dense multi-column newspaper page and a simple slide) or different prompts, could batch-level sharing ever pick suboptimal layers for some samples in the batch?

  2. Table 2 gives a clear breakdown of per-step attention and decoder latency over 128 generated tokens. Just to give a fuller picture of real-world deployment, what does the end-to-end wall-clock speedup look like when transcribing a full page of text (say, 1,000+ generated tokens, including vision encoding, prefill, the 9 warmup steps, and the tensor gather overhead)?

Reviewer_bWAYOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FastOCR, a training-free method for reducing attention computation during VLM-based document OCR. The authors observe that highly attended visual-token sets tend to concentrate around text rows and change gradually across decoding steps, which they call Dynamic Visual Fixation (DVF). Based on this observation, FastOCR combines Focal-Guided Pruning (FGP) and Cross-Step Fixation Reuse (CSFR). FGP selects focal layers through decoding-time profiling and propagates their selected visual-token positions to subsequent non-focal layers. CSFR uses the previous step's deepest focal selection to initialize the next step. The full KV cache is preserved throughout decoding.

Experiments on OmniDocBench and olmOCR-Bench show that FastOCR retains much of the original OCR performance and outperforms the evaluated pruning baselines under matched nominal attention budgets. Quality retention is evaluated across four models. The reported maximum speedups are 3.36x for attention and 1.73x for decoder forwards.

Critical strengths
  1. FastOCR addresses an important limitation of visual-token pruning for OCR. By preserving the full KV cache and restricting attention access instead of permanently removing tokens, the method allows previously unattended visual information to remain available. The results are promising. In Table 1, FastOCR achieves an Overall score of 66.20 on Qwen2.5-VL, compared with 67.31 for vanilla inference and 24.82 for DivPrune, the strongest evaluated baseline at the same nominal budget. The results across multiple models and benchmarks support the effectiveness of the complete method.

  2. The paper provides useful evidence of temporal continuity in visual attention during OCR decoding. Section 3.1 and Appendix H measure token-set overlap and spatial movement using random and step-shuffled controls. In unpruned Qwen2.5-VL, the consecutive-step IoU is 0.511, compared with 0.137 after shuffling. This supports the observation that temporal continuity is not solely induced by FastOCR's selection and reuse mechanisms.

Critical weaknesses
  1. The independent contributions of FGP and CSFR are not clearly established. In Table 4, disabling CSFR reduces the Overall score from 66.20 to 44.56, but also changes the full-attention layer placement and nominal computation budget. Thus, the comparison does not isolate the benefit of cross-step reuse. FGP also propagates visual-token selections from focal layers to subsequent non-focal layers, but the paper does not directly examine whether these selections are suitable for layers with potentially different attention distributions. Given the connections to existing query-aware and layer-adaptive sparsity methods, controlled comparisons with fixed focal layers and matched access budgets would help clarify the contribution of these design choices. If such comparisons show clear additional benefits, I would reconsider my current rating.

  2. The generality of the attention observations and the necessity of online focal-layer selection remain unclear. Appendix H measures natural attention only at selected layers of unpruned Qwen2.5-VL, while the dots.ocr measurements are collected during FastOCR decoding. Although the four-model experiments support the applicability of the complete method, they do not establish that DVF is a shared natural property across models. Moreover, Appendix G shows that focal-layer selections are highly stable across samples under the examined configuration (rho = 0.2), raising the question of whether a fixed layer set obtained from offline calibration could achieve similar results without per-batch profiling.

Questions for authors
  1. Table 4 shows a substantial performance drop when CSFR is disabled, but the ablation also changes the placement of full-attention layers. Could the authors provide a comparison with identical full-attention layer positions and token-access budgets? For example, how does using the previous step's focal selection compare with using the current layer-0 selection or a temporally shuffled selection under otherwise identical conditions?

  2. FGP propagates visual-token selections from focal layers to subsequent non-focal layers. Have the authors examined how well these selections align with the attention distributions or independently selected top-k tokens of subsequent layers? Does the alignment change with the distance from the preceding focal layer? It would also be useful to compare the proposed focal-layer selection strategy with fixed or uniformly spaced layers under the same budget.

  3. Appendix G suggests that focal-layer positions are relatively stable across samples. How does the default online profiling strategy compare with a fixed focal-layer set obtained from a small calibration set? Also, does the temporal fixation behavior reported in Appendix H appear in other unpruned models, such as dots.ocr? Please clarify whether the measurements remain similar under the default decode-profiling configuration.

  4. Could the authors report full-document inference latency and peak memory usage against a strong baseline such as Quest or DivPrune? The current timing experiments include profiling and selection overhead, but do not establish comparative end-to-end runtime over complete documents. Such measurements would help clarify the practical quality-latency trade-off.

1 comment

Dear Reviewer bWAY,

As the discussion period nears its end, we wanted to politely follow up and see if our response and the blue-highlighted revised manuscript have addressed your questions. Following your constructive suggestions, we added controlled CSFR and focal-layer placement ablations (Sec. 4.3, App. H), downstream attention alignment analyses, natural DVF continuity measurements on unpruned dots.ocr (Sec. 3.1, App. I), and end-to-end request latency breakdowns (App. D.1, App. K).

We would be grateful to hear if these additions resolve your concerns, or if there is anything else we can clarify before the discussion closes!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.