acceptodds
Under review as a conference paper at ICLR 2027

One Glyph, Many Tokens: Tokenization Bottlenecks in Generative OCR

Abstract

In generative optical character recognition (OCR), a visually atomic character may correspond to multiple output tokens. This representation mismatch forces the model to transcribe a single glyph through a sequence of autoregressive predictions, increasing the risk of error propagation. Our analysis reveals a pronounced degradation in OCR performance as token fragmentation increases, suggesting that output representation itself can remain a bottleneck. To address this, we introduce Glyph-Aligned Output-Space Adaptation (GAOSA), a representation level adaptation approach that aligns the output granularity of fragmented characters with visually atomic glyphs. GAOSA replaces fragmented multi-token decoding paths with atomic character-level output units. It initializes these units from their constituent-token representations and jointly adapts the token embeddings and language-modeling head under transcription supervision. We primarily study GAOSA in historical Chinese OCR, where standard SFT improves target character F1 from 7.6% to 59.7%, while GAOSA further raises it to 88.6%, demonstrating substantial gains beyond those achieved through supervision alone. To support full-page transcription, we further synthesize target characters into historical layouts and train with page-level supervision, enabling the model to jointly learn layout understanding and character recognition. The resulting model achieves a normalized edit distance (NED) of 0.11 on AncientDoc, compared with 0.19 for PaddleOCR-VL-1.6 and 0.18 for Qwen3.6-27B. In addition, GAOSA yields consistent gains across distinct scripts and symbol systems. On HHD-Ethiopic, GAOSA improves F1 from 42.6% (SFT only) to 76.2%; on OCRIPA, it improves F1 from 73.9% to 90.8%. These results establish glyph-aligned output representation as an effective intervention beyond conventional SFT across diverse scripts and symbol systems. The model and benchmarks will be publicly available.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Vision-Language Model, Optical Character Recognition, Glyph-Aligned Vocabulary, Tokenization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_66SVOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the token fragmentation problem in generative OCR, with a particular focus on rare Chinese characters found in ancient texts. The authors argue that splitting a complete character shape into multiple output tokens may create a bottleneck at the output representation level. To address this, the paper proposes a character-shape-aligned output space adaptation method GAOSA: Assigns dedicated tokens to selected fragmented characters, initializes them using the mean of the representations of their original constituent tokens, and adjusts language-side parameters through page-level supervision.

The most compelling comparison starts from the same first-stage checkpoint and uses the same second-stage training samples. Experiments with a second backbone model provided evidence of compatibility, with the complete system achieving the lowest normalized edit distance (NED) among the compared systems on four Chinese classical text benchmarks. Since this method simultaneously modifies both the tokenization scheme and the optimization of the output layer, these results more directly support the effectiveness of the entire adaptation process, rather than merely demonstrating the effect of changes at the token granularity.

Critical strengths

Compared to simply adding supervised training, this approach achieved significant additional gains. The methods in Table 3 used the same initialization and the same second-stage training data; therefore, the 28.87 F1 percentage point improvement over conventional SFT is more convincing than a comparison with an unadapted external model. Modifying only the tokenizer without providing targeted supervision also failed to yield any improvement. These results support the practical value of this method in challenging OCR scenarios and demonstrate that simply constructing a new vocabulary is insufficient to generate benefits.

Critical weaknesses
  1. The configuration of the existing SFT baseline needs to be explained more clearly. Sections 3.2–3.4 acknowledge that GAOSA simultaneously modifies both the word segmentation method and language-side optimization, but Table 3 does not list the trainable modules and input-output weight sharing schemes corresponding to the conventional SFT with the same level of clarity, which hinders an accurate understanding of the comparison results. It would be helpful to include a brief configuration table based on the completed experiments. The abstract and conclusions should also align with the qualifications in Section 3.4, presenting the benefits in terms of the effectiveness of the complete output space mapping scheme.

  2. The annotation guidelines and the division of ancient text data require clarification. Section 4 states that the re-annotated reference transcriptions include page elements such as headers and footers, whereas Appendix A.2 and Figure 6 describe the removal of headers and footers as non-transcription noise. This may indicate that different guidelines were used for training and evaluation, but the paper has not yet clarified this point. Furthermore, the section on ancient text data needs to explain how the self-built training set and evaluation set were divided. These issues constitute insufficient reporting and do not imply that data contamination has been found or that the results are invalid. Clarifying the existing annotation guidelines and division process will help in assessing the meaning of the reported scores.

Questions for authors
  1. Existing baseline settings (request for clarification): Which modules were trained separately for the standard SFT and GAOSA in Table 3? Do the input embedding layer and the output head share weights? Please list the settings already used in a small table.
  2. Annotation Guidelines (Request for Clarification and Manuscript Revision): Do the reference transcriptions for AncientDoc and MTHv2 include headers and footers? Please explain the discrepancy between the descriptions in Section 4 and Appendix A.2/Figure 6, indicate whether different standards were intentionally used for training and evaluation, and briefly describe how the model outputs were processed prior to scoring.
  3. Existing Data Partitioning (Request for Clarification): How are FELDoc-1K and HCLDoc-582 separated from the training data? Please briefly describe the units used to partition real pages (e.g., pages or bibliographic entries/editions), and explain how synthetic evaluation pages are generated or reserved.
3 comments
AuthorsOpenReview ↗

Response to Weakness 1&Question 1

We sincerely thank the reviewer for the thorough and positive evaluation of our work. We greatly appreciate your recognition of our approach to challenging OCR scenarios and your suggestions for our work. We have carefully considered all questions raised and provide our clarifications and responses below.

Review Question:

Existing baseline settings (request for clarification): Which modules were trained separately for the standard SFT and GAOSA in Table 3? Do the input embedding layer and the output head share weights? Please list the settings already used in a small table.

Response:

Thank you for raising this important question! Following your suggestion, we have explicitly clarified whether the input embedding and LM head share weights (Section 5.3, lines #417–419), as well as which modules are trained, in the revised manuscript (Section 5.3, lines #416–417).

Specifically, the original Qwen3.5-4B uses tied input embedding and output LM-head weights. Standard SFT retains this weight tying under the original tokenizer (τ₀), whereas GAOSA unties the embedding and LM head after vocabulary expansion under the glyph-aligned tokenizer (τ₁), allowing them to be optimized independently. In both configurations, the visual encoder and multimodal projector are frozen, LoRA is applied to the language backbone, and the embedding and LM-head parameters are updated during Stage-2 training.

A similar concern was also raised by another reviewer. To further address these concerns, we conducted additional controlled ablation experiments, which have been incorporated into the revised manuscript (Section 5.3, Table 3). The complete configurations and results are summarized below.

AblationTokenizerEmbeddingLMHP ↑R ↑F1 ↑
Baseτ₀××10.256.117.64
1 (Standard SFT)τ₀✓✓62.8056.9159.68
2τ₁××10.206.097.61
3τ₁✓×33.0213.3318.83
4 (Full GAOSA)τ₁✓✓91.0086.2888.55

Here, ✓ and × indicate whether the corresponding parameters are updated during Stage-2 training, while Base denotes the Stage-1 checkpoint without Stage-2 adaptation.

Standard SFT and GAOSA correspond to Ablations 1 and 4, respectively. Both use the same Stage-1 checkpoint, training data, and supervision, with embedding and LM-head parameters updated in both configurations. GAOSA additionally introduces glyph-aligned atomic tokenization and its associated output-space adaptation, improving F1 from 59.68% to 88.55% (+28.87 percentage points). The remaining ablations further clarify the roles of embedding and LM-head adaptation.

AuthorsOpenReview ↗

Response to Weakness 2&Questions 2&3

Review Question:

Annotation Guidelines (Request for Clarification and Manuscript Revision): Do the reference transcriptions for AncientDoc and MTHv2 include headers and footers? Please explain the discrepancy between the descriptions in Section 4 and Appendix A.2/Figure 6, indicate whether different standards were intentionally used for training and evaluation, and briefly describe how the model outputs were processed prior to scoring.

Response:

We sincerely thank you for carefully identifying this inconsistency and apologize for the writing error in Section 4. To clarify, the reference transcriptions for both AncientDoc and MTHv2 exclude non-transcriptional elements, including headers, footers, and the central book-fold area. We follow a single, unified annotation protocol throughout our study, as described in Appendix A.2 and illustrated in Figure 6. No different annotation standards were intentionally applied to training and evaluation. The discrepancy in Section 4 was an editorial error, which we have corrected in the revised manuscript (4. Evaluation Datasets, lines #278–282).

The reason for reannotating AncientDoc and MTHv2 is that their original annotations cannot be reliably converted into the required page-to-sequence transcription format through rule-based methods. In particular, complex double-column annotations, historical reading orders, and text segmentation make it difficult to automatically construct accurate and consistently serialized reference transcriptions. Our manual re-annotation addresses these challenges and ensures consistency with the unified transcription protocol. These data have already been made available anonymously on Hugging Face (AnonymousGAOSA/AncientDoc_Plus and AnonymousGAOSA/MTHv2_Plus).

Regarding evaluation, no additional post-processing is applied to our model's generated transcriptions before scoring (4. Evaluation Datasets, lines #299–301). The decoded outputs are directly compared with the corresponding ground-truth references, without rule-based content filtering, reading-order correction, or layout reconstruction. This is also an advantage of our end-to-end page-to-sequence model: through supervision from both synthetic and manually annotated data, the model learns to identify relevant textual content, handle complex document layouts, follow the correct reading order, and exclude non-transcriptional elements directly during generation, rather than relying on external post-processing procedures.

Review Question:

Existing Data Partitioning (Request for Clarification): How are FELDoc-1K and HCLDoc-582 separated from the training data? Please briefly describe the units used to partition real pages (e.g., pages or bibliographic entries/editions), and explain how synthetic evaluation pages are generated or reserved.

Response:

We thank the reviewer for highlighting the need to clarify our data partitioning protocol. Following your suggestion, we have revised Section 4 (4. Evaluation Datasets, lines #285–290) to explicitly describe the partitioning units, synthetic evaluation data preparation, and training–evaluation separation.

Specifically, we adopt page-level partitioning based on unique page IDs, rather than partitioning by bibliographic entries or editions. The 291 real evaluation pages in HCLDoc-582 are strictly excluded from the 18,432 training pages.

For synthetic evaluation data, HCLDoc-582 contains 291 reserved, degradation-augmented synthetic pages, while FELDoc-1K consists of 1,000 synthetic pages generated separately using the same synthesis engine. All evaluation page IDs are excluded from both Stage-1 and Stage-2 training.

These clarifications have been incorporated into the revised manuscript to make the training–evaluation separation and benchmark construction protocols explicit.

Thank you to the authors for the detailed response. I appreciate the clarifications and the effort put into addressing my comments.

Reviewer_syZFOpenReview ↗
Rating
2(−1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes GAOSA that introduces atomic character tokens, to address token fragmentation in generative OCR. Compared with data-matched SFT, it improves target-character F1 from 59.7% to 88.6% on FELDoc-1K, with gains also observed on Ethiopic and IPA.

Critical strengths

The paper investigates token fragmentation in generative OCR, and proposes GAOSA. The proposed GAOSA introduces atomic character tokens and adapts the embedding and output layers, with validation on historical Chinese documents, Ethiopic, and IPA.

Critical weaknesses
  1. The scalability of atomic character tokenization remains unclear. Would broader character coverage consistently improve performance? An analysis of recognition accuracy and computational cost across vocabulary sizes would help establish the practical trade-offs, particularly when extending GAOSA to a unified multilingual OCR model.
  2. Figure references alternate between “Fig.” and “Figure”, so this should use a consistent format throughout the manuscript.
  3. The core idea of assigning dedicated tokens to fragmented characters has already been explored in historical OCR, notably in TongGuOCR. Clarify GAOSA’s technical advances over such approaches and isolate the benefits of its initialization and adaptation strategy through controlled comparisons.
Questions for authors

Could GAOSA also benefit mathematical expression recognition, where a single visual symbol (e.g., α represented as \alpha) may require multiple output tokens?

3 comments
AuthorsOpenReview ↗

Response to Weakness 1&2&3

Thank you very much for your thoughtful and positive review, as well as the positive scores. We sincerely appreciate your encouraging assessment of our GAOSA. We have carefully considered all concerns raised and provide our clarifications and responses below.

Review Weakness 1:

PP-OCRv6 does not appear to support Korean, Arabic, or Hindi. The authors should verify its language coverage and clarify which recognition models were used to evaluate these languages.

Response:

Thank you for your suggestion. In the original manuscript, the evaluated recognition model is PaddleOCR-VL-1.6 [1], not PP-OCRv6. These are two different model families: PP-OCRv6 is a dedicated text recognition system, whereas PaddleOCR-VL-1.6 employs a document parsing pipeline that integrates layout analysis with a vision-language model (VLM). Our evaluation across the four benchmarks goes beyond multilingual character recognition, encompassing document layout understanding and reading-order recognition. We therefore adopt PaddleOCR-VL-1.6 as the baseline to evaluate these capabilities within a unified document parsing framework.

Specifically, the complete PaddleOCR-VL-1.6 pipeline uses PP-DocLayoutV3 for layout detection, and PaddleOCR-VL-1.6-0.9B for region-level content recognition. Therefore, the recognition is performed by this multilingual VLM rather than PP-OCRv6. According to the official PaddleOCR-VL documentation, the model supports over 100 languages, explicitly including Korean, Arabic, and Hindi [1]. Thus, the language coverage limitations of PP-OCRv6 do not apply to the baseline evaluated in our experiments.

To eliminate this ambiguity, we will clarify the baseline model name, its recognition component, and language support in the experimental setup. (Appendix C, lines #864-917)

References

[1] https://www.paddleocr.ai/main/en/version3.x/pipeline_usage/PaddleOCRVL.html?utm_source=chatgpt.com

Review Weakness 2:

The evaluation is conducted on text-line crops, without explicit results for curved or non-horizontal text, leaving the method’s applicability to complex real-world layouts unclear. Please clarify how such cases are handled and provide representative evaluations.

Response:

We sincerely thank the reviewer for this valuable suggestion. To evaluate the robustness of our method in challenging document scenarios, we conducted experiments on the HCLDoc-582 benchmark, which includes complex document layouts featuring curved and non-horizontal text. The experimental results are presented in Table 5, and representative examples from HCLDoc-582 are provided in Appendix A (Figure 6, line #832).

Furthermore, to enhance the applicability of our method to real-world scenarios, our synthetic data generation pipeline explicitly simulates various geometric distortions and visual degradations, including text bending, rotation, and blurring. These augmentations expose the model to diverse and challenging document appearances during training, improving its ability to handle complex layouts and text distortions. Further details of the synthesis process are illustrated in Figures 4 and 5 of Appendix A.1 (line #705).

Review Weakness 3:

Figure references alternate between “Fig.” and “Figure”, this should use a consistent format throughout the manuscript.

Response:

Thank you for this helpful suggestion. We have revised the manuscript to consistently use “Figure” throughout, ensuring uniform formatting for all figure references.

The comments you addressed were intended for another submission, and I mistakenly posted them here. I have now replaced that review with the correct review of your paper. Thank you for your response, and I sincerely apologize for the mix-up.

Could you please respond to the updated review? I am sorry for the confusion and the additional work my mistake has caused. Thank you for your patience and understanding.

AuthorsOpenReview ↗

Added Response to Question 1

Review Question:

Could GAOSA also benefit mathematical expression recognition, where a single visual symbol (e.g., α represented as \alpha) may require multiple output tokens?

Response:

We thank the reviewer for this suggestion. To examine the applicability of GAOSA to mathematical expression recognition, we conducted an additional experiment on visually atomic mathematical symbols with different fragmentation degrees under the original tokenizer. The experimental setup and results have been added to Appendix E (GAOSA on Mathematical Expression Recognition) of the revised manuscript.

Specifically, we evaluated 340 mathematical and related symbolic characters, categorized by their fragmentation degree kk under the original tokenizer, where kk denotes the number of output tokens required to represent a single character. The evaluation includes 95 characters with k=1k=1, 152 with k=2k=2, and 93 with k=3k=3. We compared three configurations: (1) the original Base model, (2) conventional SFT using the original tokenizer, and (3) SFT with GAOSA, which introduces glyph-aligned atomic tokens and jointly adapts the embedding and LM head.

Experimental Results. The character-level recognition accuracy across different fragmentation degrees is summarized below.

Table 1. Character-level recognition accuracy (%) across different fragmentation degrees.

Fragmentation Degree (kk)SamplesBaseSFTGAOSA
k=1k=19548.4257.8978.95
k=2k=21529.2123.6899.34
k=3k=3934.3017.2098.92
Overall34018.8231.4793.53

Results Analysis. GAOSA achieves 93.53% overall accuracy, outperforming the Base model (18.82%) and conventional SFT (31.47%) by 74.71 and 62.06 percentage points, respectively. Compared with conventional SFT, the improvements are 21.06, 75.66, and 81.72 percentage points for k=1k=1, k=2k=2, and k=3k=3, respectively. The gains are particularly pronounced at higher fragmentation degrees (k>1k>1).

These results are consistent with GAOSA's mechanism. By representing a visually atomic symbol with a single output token, GAOSA reduces the number of autoregressive predictions and avoids intermediate within-symbol decoding steps. For k=1k=1, where the decoding length remains unchanged, the improvement may instead reflect output-representation adaptation.

Overall, the results show that GAOSA's benefits extend beyond historical Chinese characters to mathematical symbols, supporting its applicability to symbol recognition in mathematical expressions.

Evaluation Character Inventory. For completeness, Table 2 lists the 340 mathematical and related symbolic characters used in this experiment, grouped by their fragmentation degrees under the original tokenizer.

Table 2. Mathematical and related symbolic characters grouped by fragmentation degree (k=1,2,3k=1,2,3).

k=1k=1 (95 characters)k=2k=2 (152 characters)k=3k=3 (93 characters)
α β γ δ ε ζ η θϵ ϑ ϖ ϱ ϕ Ψ Ϝ ϝ Ϟ ϟ Ϛ Ϡ ∓⋆ ⋅ ⊎ ⊓ ⊔ ⋄ ⊕ ⊖
ι κ λ μ ν ξ ο π∗ ∘ ∩ ∪ ∨ ∧ ∖ ≀ ◁ ‡ ∔ ≠ ≐⊗ ⊘ ⊙ ⊲ ⊳ ⊴ ⊵ ⨿
ρ σ ς τ υ φ χ ψ≡ ≅ ≃ ∝ ≪ ≺ ≻ ∣ ∥ ≍ ∴ ∵ ∤⊞ ⊟ ⊠ ⊡ ⊺ ⊼ ⊻ ⋏
ω Γ Δ Θ Λ Ξ Π Σ∦ ≊ ≎ ≏ ≑ ≔ ≕ ≗ ≙ ≜ ≬ ≶ ≷⋎ ⋒ ⋓ ⋇ ⊝ ⊛ ⊚ ⋉
Υ Φ Ω ± × ÷ ∙ △≲ ≳ ≰ ≱ ≁ ≉ ≇ ≄ ↔︎ ⇐ ⇔ ↦ ↩︎⋊ ⋈ ⋔ ⊸ ⪯ ⪰ ⊂ ⊃
▽ ▷ † ≤ ≥ ≈ ∼ ≫↪︎ ↼ ⇀ ↽ ⇁ ⇌ ⟵ ⟶ ⟷ ⟸ ⟹ ⟺ ⟼⊆ ⊇ ⊑ ⊒ ⊢ ⊣ ⊨ ⊥
∽ ≮ ≯ ← → ⇒ ↑ ↓↕︎ ⇑ ⇓ ⇕ ↗︎ ↘︎ ↙︎ ↖︎ ⇝ ↭ ⇄ ⇆ ⇉⌣ ⌢ ⊏ ⊐ ⊊ ⊋ ⊀ ⊁
↳ ∀ ¬ ∈ ′ √ ○ …⇇ ⇈ ⇊ ↢ ↣ ↲ ⇋ ↶ ↷ ↺ ↻ ⇠ ⇢⊄ ⊅ ⊈ ⊉ ⊬ ⊭ ⋢ ⋣
♡ ✓ □ ■ ▲ ▼ ▶︎ ◆↚ ↛ ↮ ⇎ ⇏ ⇍ ∑ ∏ ∐ ∫ ∬ ∭ ∮⋪ ⋫ ⋬ ⋭ ⋂ ⋃ ⋁ ⋀
★ ®︎ ⟩ § ¶ ©︎ £ °∞ ∂ ∇ ∃ ∅ ∋ ℵ ℏ ℓ ℘ ℜ ℑ ∠⨁ ⨂ ⨀ ⨄ ⨆ ⨌ ⨎ ⨏
℃ ‰ ™︎ € ¥ æ œ ø♣︎ ♢ ♠︎ ♭ ♮ ♯ ∄ ℶ ℷ ℸ ◀︎ ◊ ∡⊤ ⋮ ⋯ ⋱ Ⓢ ⌈ ⌉ ⌊
ß ı ł Æ Ø Ł µ∢ ▵ ✠ ⟨ ‖ ȷ Œ ↬ Ω⌋ ⌜ ⌝ ⌞ ⌟
Reviewer_cm92OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Glyph-Aligned Output-Space Adaptation (GAOSA) to address token fragmentation in generative OCR. The method introduces character-level output tokens and adapts the corresponding embeddings and language modeling head. Experiments on historical Chinese OCR and other scripts demonstrate improvements in character recognition and full-page transcription.

Critical strengths
  1. The paper investigates an interesting problem concerning token fragmentation in generative OCR, particularly for rare characters and historical documents.

  2. The method achieves substantial improvements over conventional SFT and demonstrates effectiveness across different scripts and VLM backbones.

Critical weaknesses
  1. The methodological novelty and practical motivation are insufficiently established. GAOSA mainly relies on vocabulary expansion and embedding adaptation, which are closely related to existing output-space adaptation techniques. Its technical novelty over these approaches is not clearly demonstrated. Furthermore, conventional OCR methods already achieve strong performance on character-level and text-line recognition without the token fragmentation issue. However, the paper lacks comparisons with traditional OCR methods, making the practical necessity and advantages of the proposed VLM-based approach difficult to assess.

  2. The experimental validation does not sufficiently establish the effectiveness of the proposed mechanism. Although substantial quantitative improvements are reported, the experiments mainly focus on final recognition performance, with limited mechanism-oriented analysis and visualization. The ablation studies are insufficient to isolate the effects of tokenization, embedding adaptation, and output-head optimization. More targeted qualitative analysis and controlled experiments are needed to verify how GAOSA addresses token fragmentation and clarify the contributions of its individual components.

Questions for authors
  1. Could the authors clarify the key technical novelty of GAOSA compared with existing vocabulary expansion and output-space adaptation methods, such as FOCUS and EEVE?

  2. Could the authors include comparisons with strong conventional OCR methods, particularly for character-level and text-line recognition, to better justify the necessity and practical advantages of the proposed VLM-based approach?

  3. Could the authors provide more controlled ablation studies to distinguish the contributions of atomic tokenization, embedding initialization, and language-modeling head adaptation? In particular, how much of the performance improvement can be attributed specifically to glyph-aligned tokenization?

  4. Could the authors provide more targeted qualitative visualizations and error analysis to directly demonstrate how GAOSA addresses token fragmentation, rather than only reporting improvements in final recognition results?

1 comment
AuthorsOpenReview ↗

Response to Question 1

Thank you for your thoughtful review and the time you dedicated to our work! We appreciate your positive assessment of the problem we address and the progress we have made. Below, we provide clarifications and additional experiments to address your concerns.

Reviewer Question 1:

Could the authors clarify the key technical novelty of GAOSA compared with existing vocabulary expansion and output-space adaptation methods, such as FOCUS and EEVE?

Response:

Thank you for the suggestion. We compare GAOSA with FOCUS and EEVE in the following three respects and incorporate a concise version into the revised manuscript (1. Introduction, lines #78–84; 2. Related Work, lines #147–154):

1. Different adaptation target: language-side adaptation vs. visual-to-language alignment. FOCUS and EEVE address vocabulary and representation adaptation for a target language, whereas GAOSA aims to address the mismatch between visual characters and output-token granularity in OCR transcription. For example, EEVE studies vocabulary expansion for adapting English-centric language models to Korean [1,2]. In contrast, GAOSA addresses an OCR-specific mismatch in which a visually atomic character may be fragmented into multiple tokens and therefore require multiple autoregressive predictions. As shown in Figure 1(a), recognition performance drops sharply as token fragmentation increases, motivating us to align the output granularity with the visual transcription unit.

2. Different technical novelty: vocabulary/representation construction vs. OCR output-space construction. FOCUS initializes new-token representations using semantic similarity and Sparsemax over shared source–target vocabulary tokens, while EEVE combines corpus-driven vocabulary expansion, constituent-subword initialization, and staged parameter updates [1,2]. The key novelty of GAOSA lies not in simply adding new tokens, but in aligning the output granularity with visually atomic character units in OCR, such that each selected fragmented character is represented by a single atomic token, with this alignment learned through vision-conditioned supervision. This replaces the original multi-token decoding path with a single atomic prediction while preserving autoregressive full-page transcription. Initialization and optimization of the embedding and LM head are used to learn this redesigned output space, rather than constituting the core novelty of GAOSA.

3. Different supervision paradigm: text-only continued pretraining vs. vision-conditioned SFT. FOCUS is an initialization method whose adapted representations are subsequently learned through text-only language-adaptive continued pretraining, while EEVE directly adopts staged continued pretraining after vocabulary expansion. In contrast, GAOSA learns the redesigned atomic output units through vision-conditioned supervised fine-tuning on paired page images and transcription sequences. With target-character-enriched layout synthesis, this supervision further enables the model to jointly learn character recognition, reading order, and full-page transcription, while the same output-space intervention generalizes across different scripts, symbol systems, and VLM backbones.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Response to Question 3

Review Question:

Could the authors provide more controlled ablation studies to distinguish the contributions of atomic tokenization, embedding initialization, and language-modeling head adaptation? In particular, how much of the performance improvement can be attributed specifically to glyph-aligned tokenization?

Response:

We sincerely thank the reviewer for this constructive suggestion. To address this concern, we conducted additional ablations to systematically investigate the contributions of glyph-aligned tokenization, embedding, and language-modeling head (LMH) adaptation. These results have been incorporated into the revised manuscript (Section 5.3 Ablation Study).

Experimental Setup. We use the Stage-1 checkpoint of Qwen3.5-4B as the common starting point, denoting the original pretrained tokenizer as τ₀ and the proposed glyph-aligned tokenizer as τ₁. Introducing τ₁ requires expanding the embedding and LM head to accommodate newly added atomic tokens, whose representations are initialized from their constituent tokens. However, initialization alone does not establish effective visual-to-token alignment; subsequent adaptation under vision-conditioned supervision is necessary to learn these new representations.

During Stage-2 LoRA training, we systematically vary the tokenizer and whether the embedding and language-modeling head (LMH) parameters are updated, indicated by ✓ and ×, respectively. The frozen configurations serve to evaluate the effects of omitting these adaptations. The Base row represents the Stage-1 checkpoint without Stage-2 training, while all Stage-2 configurations use identical training data and supervision. We evaluate performance on fragmented target characters (k > 1) on FELDoc-1K, following the original evaluation protocol.

AblationTokenizerEmbeddingLMHP ↑R ↑F1 ↑
Baseτ₀××10.256.117.64
1τ₀✓✓62.8056.9159.68
2τ₁××10.206.097.61
3τ₁✓×33.0213.3318.83
4 (Full GAOSA)τ₁✓✓91.0086.2888.55

Analysis. The additional experiments provide three important findings to address your concern.

(1) Comparison between Ablations 1 and 4. Under matched SFT conditions, including the same Stage-1 checkpoint, training data, and embedding/LM-head adaptation strategy, glyph-aligned tokenization (τ₁) improves F1 from 59.68% (τ₀) to 88.55%, yielding a 28.87-point gain (48.4% relative improvement). Since atomic tokenization requires corresponding embedding and output-layer adaptation, these updates are integral to implementing τ₁. This comparison demonstrates the substantial benefit of aligning output granularity with visually atomic characters.

(2) Comparison between Ablations 2 and 3. Atomic token construction and constituent-based initialization alone achieve only 7.61% F1, comparable to the Stage-1 baseline (7.64%). Enabling embedding adaptation increases F1 to 18.83% (+11.22 points), indicating that embedding adaptation is necessary to learn effective representations for newly introduced atomic tokens.

(3) Comparison between Ablations 3 and 4. Further enabling LM-head adaptation increases F1 from 18.83% to 88.55% (+69.72 points), demonstrating its critical role in learning to predict the newly introduced atomic tokens.

Overall, these ablations demonstrate that glyph-aligned tokenization, together with its necessary output-layer adaptation, provides a 28.87-point F1 improvement over conventional SFT under matched supervision. Embedding and LM-head adaptation further enable effective learning and prediction of atomic tokens, supporting output-granularity alignment as a key factor in mitigating token fragmentation.

AuthorsOpenReview ↗

Response to Question 4

Review Question:

Could the authors provide more targeted qualitative visualizations and error analysis to directly demonstrate how GAOSA addresses token fragmentation, rather than only reporting improvements in final recognition results?

Response:

We sincerely thank the reviewer for this valuable suggestion. To address this concern, we have added targeted qualitative visualizations and error analysis in the revised manuscript (Section 5.6, Figure 3) to directly illustrate how GAOSA mitigates token fragmentation.

Specifically, Figure 3 visualizes characters with different fragmentation degrees (k = 4, 3, and 2), comparing three decoding paths: (1) the ground-truth token sequence under the original tokenizer τ₀, (2) the actual predictions of Qwen3.5-4B under τ₀, and (3) the GAOSA predictions under the glyph-aligned tokenizer τ₁. These comparisons reveal a clear error propagation effect: incorrect predictions at intermediate decoding steps (e.g., the second or third token) can cause subsequent tokens to deviate from the ground-truth sequence, as later predictions depend on the erroneous prefix.

In contrast, GAOSA represents each visually atomic character as a single glyph-aligned token, eliminating the need for multiple autoregressive predictions within a character. This shorter decoding path reduces opportunities for intermediate errors and their propagation, enabling more accurate recognition of fragmented characters. These visualizations provide direct qualitative evidence of how GAOSA mitigates token fragmentation, complementing the quantitative improvements reported in our experiments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.