- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper addresses faster decoding for document OCR VLMs. AR OCR models are slow because each token needs one forward pass. Committing several tokens at once with diffusion can break reading order and markup structure. The paper therefore proposes to separate drafting from commitment. The proposed method, GravityOCR, adds only a mask token embedding to a pretrained AR OCR model and trains it jointly as a block-diffusion drafter and a causal AR verifier. At inference, the diffusion path drafts a whole block in one pass, and the AR path commits only the prefix that matches its next-token predictions. The authors claim that this keeps the output identical to greedy AR while greatly speeding up decoding, and that GRPO recovers accuracy close to the base model.
Critical strengths
-
It achieves the fastest page throughput among the systems measured while keeping top-tier accuracy. It presents a wide range of experimental comparisons and analyses, such as the comparison with direct block diffusion and breakdowns by batch size, output length, and content type. It also does not hide the results that the gain shrinks with larger batches and is small for short outputs.
-
The paper is well written and clean. The figures make the problem setting and the method clear at a glance, and the decoding procedure and measurement settings are fully written out in the appendix, so it was easy to follow.
Critical weaknesses
-
It is not clear what this paper contributes. The design where diffusion drafts and AR verifies on one backbone already exists in TiDAR and other work the authors cite as closest, and the authors themselves write that they applied it to OCR. At inference the whole block is predicted at once and then verified sequentially by AR, so the advantage of diffusion, unmasking high-confidence tokens first regardless of position, is not used, and the method looks in effect like MTP with an adaptive window. Meanwhile the only element aimed at OCR is the post-training reward, so it is hard to judge clearly what contribution the authors established to solve the problems of the OCR task.
-
The results themselves are not very impactful. The OmniDocBench Overall score is actually lower than the base model. On speed, the page-level end-to-end gain is only 1.32×, and it shrinks to 1.13× at larger batch sizes, so the benefit is limited in high-load serving. To judge whether this trade-off is good, the base model should be made to emit more tokens at once and compared on accuracy at a speed similar to GravityOCR. The MTP in the paper was measured on a different serving engine, and its TPF excludes the drafting cost, so it is not a comparison on the same basis. I cannot tell whether the method is better than alternatives that lose no accuracy, so it is hard to assess its value.
(Minor details) There is no limitations section and no failure case analysis. Inputs where the verifier often rejects the draft and conditions where the speed gain shrinks are not summarized, and without this analysis users cannot judge how far the method applies.
Questions for authors
-
What are the speeds in Table 1 if GLM-OCR's MTP is measured on SGLang, the same engine as GravityOCR? The GLM-OCR technical report reports 5.2 tokens per step and about 50% higher throughput with MTP, so why is it 3.7 TPF and slower than AR in this paper? Can it be compared on the same basis with MTP using more draft tokens?
-
In one-shot drafting the whole block is masked and in the end it is resolved sequentially, so what does bidirectional attention actually contribute in the proposed method? If the draft is switched to causal attention under the same training, how does the acceptance length change?
-
TiDAR handles drafting and verification in one forward pass, so why split them into two? Please state clearly how this paper's contribution differs from such similar prior work.
-
Are there any other elements or analyses designed for OCR besides the reward? Before the OCR-targeted post-training, does the same method give the same gain on a general (M)LLM? After RL with OCR rewards, does the model lose its original general ability?
6 comments
Response to Reviewer PGVp (Part 2/2)
3. [W1, Q3] Relation to TiDAR
TiDAR and GravityOCR both use a single shared backbone for diffusion drafting and AR verification. As discussed above, our contribution lies in the OCR-focused analysis and validation, rather than this shared self-speculation principle. TiDAR’s fusion verifies the current draft and prepares possible next drafts in one forward pass. This execution strategy is complementary to our contributions and can also be applied to GravityOCR without retraining.
In our 32-token-draft configuration, applying TiDAR-style fusion processes 1,089 token positions per forward, versus 33 per forward in two-pass execution. At batch 1 in Hugging Face, fusion achieves 1.24× end-to-end throughput over two-pass execution. However, fusion trades fewer sequential forwards for additional speculative computation. With a fixed draft length, this additional work can outweigh the savings from fewer forwards as larger batches become compute-bound. We therefore treat fusion as an optional, workload-dependent optimization rather than an unconditional replacement for two-pass execution.
4. [W1, Q2] Architectural Choices
The within-block attention pattern is an architectural choice within GravityOCR, rather than the central contribution of this work. We adopted bidirectional attention used in prior block-diffusion models, allowing draft positions to exchange contextual representations during parallel prediction. To examine this design choice, we additionally trained a causal-attention variant under the same data and loss recipe, following the reviewer’s suggestion. The comparison at 10k training steps is shown below:
| Draft attention | TPF | Accepted draft tokens/round | Overall |
|---|---|---|---|
| Bidirectional | 6.75 | 11.50 | 95.02 |
| Causal | 7.88 | 13.77 | 94.96 |
At 10k steps, the causal variant yields more accepted draft tokens per round and higher TPF, while the Overall scores differ by 0.06 points. This provides a useful design insight and a promising direction for further improving drafting efficiency within the same OCR self-speculation approach. We will include this ablation in the revised paper and examine whether the trend persists with longer training.
5. [Q4] Performance after OCR-Reward RL
GravityOCR starts from GLM-OCR, which was already specialized for document OCR before our adaptation and GRPO. All three evaluated OCR tasks improve after GRPO: the Overall gain does not come at the expense of text, table, or formula recognition. Changes in non-OCR capabilities were not evaluated.
| Task | Before | After |
|---|---|---|
| Text (1−Edit) | 95.96 | 95.98 |
| Table (TEDS) | 92.23 | 92.80 |
| Formula (CDM) | 96.56 | 96.71 |
6. [Minor] Limitations and Failure Cases
Short outputs offer limited speedups because decoding accounts for little total runtime. We will document these applicability limits and draft-rejection cases at word boundaries, punctuation, and LaTeX commands in the revised paper's limitations and failure analysis.
Thank you again for your detailed feedback. We have uploaded a revised manuscript, with revisions marked in blue. The updated paper clarifies the OCR-focused contribution and incorporates the same-engine MTP and DFlash comparisons, the TiDAR-style fusion experiment, and the causal-versus-bidirectional draft-attention ablation. It also includes the OCR-versus-image-description analysis, draft-rejection patterns, and a dedicated limitations section.
Following an additional draft-length sweep for the fine-tuned MTP head, we selected draft length 15, which achieved the highest throughput among the tested settings. We updated Table 2 accordingly: decode-only throughput increases from 1,849 to 1,917 tokens/s, and page throughput from 1.153 to 1.191 pages/s. GravityOCR achieves 1.455 pages/s, corresponding to 1.22× the page throughput of this updated MTP baseline.
Please let us know if you have any remaining concerns or would find further clarification helpful. If our responses and revisions have addressed your concerns, we would be grateful if you could consider revising your score upward.
Response to Reviewer PGVp (Part 1/2)
Thank you for the detailed review and for recognizing our page throughput, recognition accuracy, and breadth of experimental analysis. We also appreciate your positive assessment of the presentation, figures, and decoding and measurement procedures.
1. [W1, Q3, Q4] Contribution to OCR and Other Domains
GravityOCR builds on existing joint AR–diffusion self-speculation; we do not propose a new framework. We analyze OCR's parallel-generation potential and limitations, demonstrate practical acceleration with near-AR quality, and show that AR-path OCR-reward post-training improves recognition while preserving drafting efficiency.
Our additional Diffusion Gemma experiment found 3.6× lower mean draft entropy and 2.17× higher TPF on OCR than image description (23.57 vs. 10.88). This supports OCR as a favorable setting for parallel diffusion generation. However, parallel predictability alone does not ensure accurate transcription: GravityOCR's direct-diffusion outputs retain errors (Fig. 2), and AR verification improves quality at the same checkpoint (Sec. 4.3).
Resource constraints prevented applying our training recipe to a general (M)LLM. We instead compared OCR and image description using the general-purpose self-speculative VLMs Fast-dVLM and Nemotron. Neither achieved OCR-level TPF on image description: 1.62 vs. 2.65 for Fast-dVLM and 1.75 vs. 5.54 for Nemotron. Thus, even with the same model and decoding method, the evaluated non-OCR task did not reach OCR's drafting efficiency. This task dependence reinforces the value of analyzing and validating self-speculation for OCR. We will add these analyses.
2. [W2, Q1] Recognition Quality and Speed
The practical value is faster document processing while retaining recognition quality close to GLM-OCR. GravityOCR achieves 1.73× the page throughput of GLM-OCR AR while maintaining an Overall score of 95.16 versus 95.48 for GLM-OCR. Post-submission serving optimizations increased page throughput for both AR and GravityOCR; this comparison uses the improved AR baseline. At batch sizes 64 and 256, GravityOCR achieves 6,157.6 and 6,655.1 tok/s, respectively, versus 3,580.2 and 4,302.2 tok/s for AR decoding of the same checkpoint. The resulting 1.72× and 1.55× speedups show that the throughput advantage persists at the evaluated larger batch sizes.
In response, we evaluated GLM-OCR MTP and a 165M DFlash drafter trained for original GLM-OCR on our data, both in SGLang. To match the data used for DFlash training, we also fine-tuned GLM-OCR’s existing MTP head on GravityOCR’s training data and labels. We initialized it from the released MTP weights and trained for 22,000 steps with the rest of GLM-OCR frozen.
Tokens/round counts all committed tokens; crop throughput is decode-only.
Timings use the paper's SGLang protocols and GLM-OCR AR baseline. Default MTP was slower than AR (0.67× crop, 0.84× pages/s). For this comparison, we optimized MTP serving by enabling draft-extend CUDA graphs and reducing CPU-side and synchronization overhead. These optimizations increased crop decoding and page throughput from 0.67× and 0.84× to 1.27× and 1.15× relative to AR. The fine-tuned-head configuration achieved 2.39× and 1.37×, respectively.
To address your question about the reported token efficiency and longer drafts, we additionally varied the MTP draft length. We obtained 5.33 tokens per step at length 7, close to the reported 5.2, and 6.65 at length 20. The table's MTP row uses length 7 on 8,922 crops.
GravityOCR improves quality over direct diffusion (Sec. 4.3) and MinerU-Diffusion (Table 1). It achieves 1.455 pages/s versus 1.153 for MTP with a fine-tuned head and 1.143 for DFlash (1.26–1.27× higher), with 2.55× crop throughput over DFlash. We will add these results.