acceptodds
Under review as a conference paper at ICLR 2027

FineRich: Towards Holistic Fine-Grained Text-Rich Image Understanding via Versatile HTML-Based Synthesis

Abstract

Fine-grained Text-rich Image Understanding (Fine-grained TIU) is challenging for multimodal large language models (MLLMs) to accurately perceive and understand textual details from their long visual contexts. However, existing benchmarks remain limited in scope and cannot comprehensively evaluate these capabilities. In this paper, we introduce FRBench, a holistic benchmark that systematically covers all 90 combinations of six image types, three granularity levels, and five task domains, enabling a thorough evaluation for fine-grained TIU. To construct high-quality data, we develop a versatile HTML-based synthesis pipeline that renders diverse text-rich images and derives source-traceable TIU annotations. Specifically, we convert source images into HTML and edit evidence region related to the query to increase local information density. Using this pipeline, we further automatically build a 34K training set and propose FineRich, a novel two-stage training method for fine-grained TIU, which includes: 1) a region-to-image OPSD stage that first strengthens basic fine-grained text recognition; and 2) a multi-task GRPO training stage further specializes the model across diverse TIU tasks. Experiments show that FineRich achieves leading performance on FRBench while improving performance on other fine-grained TIU and general OCR benchmarks, demonstrating strong fine-grained capabilities and remarkable generalization.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Text-Rich Image Understanding, Fine-Grained Image Understanding

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_EDN8OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FRBench and FineRich to address the limitations of existing Multimodal Large Language Models (MLLMs) in recognizing and understanding small text regions within complex text-rich images. To construct the dataset, the authors introduce an HTML-based synthesis pipeline that converts existing images into HTML and increases local information density by inserting additional DOM elements around question-relevant evidence. The paper also proposes FineRich, a two-stage training framework using the synthesized data. In the first stage, Prefix-Guided Region-to-Image OPSD distills fine-grained text recognition capabilities from a teacher that receives cropped target regions into a student that processes the full image, using teacher-guided prefix correction and sparse token-level distillation. In the second stage, Multi-task GRPO jointly optimizes performance across understanding, grounding, recognition, referring, and parsing tasks. Experiments across various MLLMs show that FRBench reveals the limitations of existing models in fine-grained understanding, while FineRich achieves substantial improvements over the original backbones on FRBench.

Critical strengths
  • One of the main strengths of this paper is the introduction of a relatively comprehensive benchmark for evaluating fine-grained text-rich image understanding. Unlike existing benchmarks that focus on specific image types or tasks, FRBench systematically combines image types, granularity levels, and task domains. By evaluating not only understanding but also grounding, recognition, referring, and parsing, it enables a broader assessment of spatial reasoning and structural understanding beyond simple OCR accuracy. Furthermore, evaluating performance across different granularity levels provides valuable insights into the limitations of existing MLLMs from multiple perspectives.
  • Another strength is the proposed training strategy for transferring accurate text recognition capabilities from a crop-level teacher to a full-image student. Prefix-Guided Region-to-Image OPSD introduces teacher-guided prefix correction to address recognition errors caused by incorrect early token predictions. Instead of applying distillation uniformly to all output tokens, it selectively distills tokens based on uncertainty and teacher–student disagreement. This design is well motivated by the challenges of accurate text recognition in dense visual contexts.
Critical weaknesses
  • A critical concern is whether the competing methods in Tables 2 and 5 were also fine-tuned. Since FineRich involves additional training, comparing it against models without fine-tuning would raise concerns about fairness. If the baselines were not fine-tuned, they should also be trained on the same data, even with standard next-token prediction, to ensure a fair comparison.
  • The first stage of FineRich is closely related to Vision-OPD, as both distill knowledge from a teacher using cropped regions into a student processing the full image. Although the paper introduces prefix correction and sparse refinement as key differences, it does not directly compare against Vision-OPD under the same synthesized data and training settings. Comparisons with standard supervised fine-tuning are also needed to establish the benefits of the proposed training strategy.
  • Prefix-Guided Region-to-Image OPSD combines teacher-guided prefix correction, sparse refinement. However, Table 6 only compares OPSD, Prefix-OPSD, GRPO, and their combinations, without isolating the effects of sparse refinement and selection ratio. These ablations are essential to demonstrate whether the proposed mechanisms provide meaningful benefits over simpler distillation approaches.
Questions for authors
  • Were the other models in Tables 2 and 5 fine-tuned under the same conditions as FineRich? If the competing models were evaluated without additional training, it is difficult to establish the effectiveness of the proposed method from these comparisons alone. Could the authors provide a baseline fine-tuned with standard next-token prediction using the same synthesized data and training budget to clarify whether the improvements come from the proposed training strategy or simply from additional data and training?
  • How much improvement does proposed method provide over the Vision-OPD? Since both methods transfer knowledge from cropped image regions to a full-image student, a direct comparison under the same backbone, synthesized training data, and training budget is necessary.
  • Could the authors provide additional ablations to independently evaluate the contributions of sparse refinement and selection ratio? These experiments would help determine whether the proposed method offer meaningful improvements over standard distillation.
7 comments
AuthorsOpenReview ↗

Response to Reviewer EDN8 - Part 1

W1&Q1. Fine-tuning fairness and the contribution of the training strategy

**1) **Thank you for raising this concern. The comparison models in Tables 2 and 5 were evaluated using their original released checkpoints or APIs, without additional fine-tuning on our synthesized data. To distinguish the contribution of our training strategy from that of additional training data, we conducted comparisons using the same backbone and synthesized training data.

Training recipeQA ANLSGrounding IoURecognition ANLSReferring ANLSParsing AVGOverall
Qwen3-VL-8B-Instruct78.5518.0073.9637.7468.5560.31
Standard SFT77.7219.0784.1854.6768.0963.50
Vision-OPD (Recognition + Referring)72.8923.5785.1981.4662.3464.06
Vision-OPD (QA only)78.3114.6470.3839.6169.0259.88
OPD + Sparse Token Selection77.2717.4181.4470.7068.2965.00
OPD + Prefix Rewriting76.4120.5080.7976.2767.7565.62
Prefix-OPD (Full)77.9026.1583.4278.2968.5267.48
SFT → GRPO79.7827.1985.3272.2770.2368.19
Vision-OPD (Recognition + Referring) → GRPO80.4631.1784.6676.0370.6169.34
FineRich-8B (Prefix-OPD → GRPO)80.0538.5485.9380.6572.1271.71
  • Compared with standard SFT, our Prefix-OPD achieves 67.48 Overall, versus 63.50 for SFT, an improvement of 3.98 points. This shows that our distillation strategy provides additional benefits beyond standard next-token training on the synthesized data.

  • Compared with plain region-to-image OPD, our Prefix-OPD improves Overall from 64.06 to 67.48, a gain of 3.42 points, supporting the benefit of the proposed prefix correction and sparse refinement.

  • After subsequent GRPO training, FineRich achieves 71.71 Overall, outperforming SFT → GRPO (68.19) by 3.52 points and plain OPD → GRPO (69.34) by 2.37 points. FineRich also outperforms SFT → GRPO across all five task metrics.

These comparisons show that standard training methods already benefit from our synthesized data, while the proposed training strategy produces further improvements using the same data. Thus, both the synthesized data and the training method contribute to the final performance.

Regarding the original comparisons, the models in Tables 2 and 5 were evaluated using their released checkpoints or APIs, without additional fine-tuning on our synthesized data. Table 2 provides a benchmark leaderboard of existing models, while Table 5 evaluates whether FineRich’s improvements over its original backbones transfer to external benchmarks. We agree that these comparisons alone do not isolate the contribution of the training strategy, and we will add the above training baselines to clarify this distinction.

2) The comparison models in Tables 2-5 were evaluated using their released checkpoints or APIs, without additional fine-tuning on our synthesized data. But we have to emphasize that the experiments in this paper are fair. On one hand, these tables also serve as the FRBench leaderboard for analyzing the performance of current MLLMs on fine-grained TIU. On the other hand, our goal is to improve the fine-grained TIU capabilities of the base models. Both the novel training method and the training data generated by our synthesis pipeline are contributions of this work. FineRich-4B and FineRich-8B outperform their respective Qwen3-VL-Instruct base models by a large margin and even surpass most large-scale open-source MLLMs (Tables 2–5). To further demonstrate the effectiveness and generalizability of FineRich, we also trained FineRich-27B based on Qwen3.8-27B, achieving an improvement of +12.31 and surpassing Gemini, the strongest closed-source MLLM.

ModelUnderstanding (ANLS)Grounding (IoU ↑)Recognition (ANLS)Referring (ANLS)Parsing (AVG ↑)Overall ↑
Gemini3.681.0535.3392.1087.8271.4272.75
Qwen3.8-27B81.0625.1284.8455.9469.7765.76
FineRich-27B88.3360.2596.3188.3972.8278.07
AuthorsOpenReview ↗

Response to Reviewer EDN8 - Part 2

W2&Q2. SFT vs. Vision-OPD vs.Prefix-OPD (FineRich):

The “OPSD” baseline in table 6 corresponds to Vision-OPD (Recognition + Referring), our task-specific adaptation of Vision-OPD to text-rich image understanding. Instead of applying distillation directly to QA, this variant uses region-level Recognition and Referring tasks, where explicit OCR targets provide more direct supervision for reading the relevant text. To clarify this distinction and evaluate our design, we conducted additional comparisons among our Prefix-OPD, standard SFT, QA-based Vision-OPD, and Recognition + Referring-based Vision-OPD. Results are shown in the following table and we obseve that Prefix-OPD achieves a better balance across tasks than standard SFT or Vision-OPD. Prefix-OPD achieves an Overall score of 67.48, compared with 63.50 for SFT and 64.06 for Vision-OPD trained on Recognition and Referring. The latter achieves strong region-level reading performance, but lower QA and Parsing scores. Prefix-OPD improves QA from 72.89 to 77.90 and Parsing from 62.34 to 68.52, while also improving Grounding from 23.57 to 26.15. This comes with some reduction in Recognition and Referring, consistent with the task trade-offs discussed in the paper.

Training configurationQA ANLS ↑Grounding IoU ↑Recognition ANLS ↑Referring ANLS ↑Parsing AVG ↑Overall ↑
Qwen3-VL-8B78.5518.0073.9637.7468.5560.31
Standard SFT77.7219.0784.1854.6768.0963.50
Vision-OPD (QA only)78.3114.6470.3839.6169.0259.88
Vision-OPD (Recognition + Referring),72.8923.5785.1981.4662.3464.06
Prefix-OPD77.9026.1583.4278.2968.5267.48

W3&Q3. Ablation of components within Prefix-OPSD:

Thank you for highlighting the need to evaluate the individual prefix correction and sparse refinement of Prefix-OPSD. We conducted the following ablations of on the 8B backbone, showing that:

  1. Adding sparse refinement increases Overall from 64.06 to 65.00, while prefix correction alone increases it to 65.62. Combining both modules achieves 67.48, a 3.42-point improvement over plain OPD and a 1.86-point improvement over prefix correction alone. This directly measures the additional contribution of sparse refinement beyond prefix correction;

  2. The task-level results also reveal the effects of these components. Plain OPD achieves strong Recognition and Referring performance, but lower QA and Parsing scores. The full method improves QA by 5.01 points, Parsing by 6.18 points, and Grounding by 2.58 points, with some reduction in Recognition and Referring. These results align with our motivation to mitigate over-specialization during crop-to-image distillation. We will incorporate these ablations and explicit component definitions into the revised manuscript.

Training configurationQA ANLS ↑Grounding IoU ↑Recognition ANLS ↑Referring ANLS ↑Parsing AVG ↑Overall ↑
Plain OPD (Recognition + Referring)72.8923.5785.1981.4662.3464.06
OPD + Sparse Refinement77.2717.4181.4470.7068.2965.00
OPD + Prefix Rewriting76.4120.5080.7976.2767.7565.62
Prefix-OPD (Full)77.9026.1583.4278.2968.5267.48

Q3.2. Effect of the token selection ratio

Thank you for suggesting this ablation. We evaluated selection ratios of 25%, 50%, 75%, and 100% on the 8B backbone. The ratio specifies the fraction of eligible continuation positions selected for distillation, excluding the supplied corrective prefix.

Selection ratioQA ANLS ↑Grounding IoU ↑Recognition ANLS ↑Referring ANLS ↑Parsing AVG ↑Overall ↑
25%77.9026.1583.4278.2968.5267.48
50%77.7022.0883.0376.3467.2366.01
75%76.9719.1782.6275.9868.2265.95
100%76.4120.5080.7976.2767.7565.62

The 25% setting achieves the highest scores across all five task domains, outperforming the 50%, 75%, and 100% settings by 1.47, 1.53, and 1.86 Overall points, respectively. These results support concentrating distillation on positions prioritized by student uncertainty, teacher–student disagreement, and reliability. Increasing the selection ratio includes lower-ranked positions and does not improve performance in these experiments. We therefore use 25% as the default and will include this ablation in the revised manuscript.

AuthorsOpenReview ↗

Seeking Further Feedback

Dear Reviewer EDN8,

Thank you again for your constructive review and helpful suggestions. In our rebuttal, we clarified the evaluation settings and the purpose of the original benchmark comparisons, and added SFT and Vision-OPD baselines, including their subsequent GRPO refinement, using the same backbone and synthesized training data. We also clarified the relationship between OPSD and Vision-OPD and provided separate ablations of prefix correction, sparse refinement, and the token selection ratio. Furthermore, we included additional FineRich-27B results to demonstrate the effectiveness of our approach on a larger backbone. We hope these experiments and clarifications provide a more complete picture of the contributions of our synthesized data and training strategy, the benefits of individual components, and the applicability of FineRich across model scales.

As the discussion period draws to a close, we would greatly appreciate your feedback on these additional results and clarifications. Please let us know if any concerns remain or if further clarification would be helpful. We welcome the opportunity to discuss these points with you.

Thank you once again for your time and consideration.

Best regards, The Authors

Reviewer_ydQrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents FineRich, a framework for fine-grained text-rich image understanding. It introduces FRBench, a benchmark containing 2,190 samples across six image types, three granularity levels, and five tasks. An HTML-based synthesis pipeline increases local information density while providing text and region annotations. The authors also generate 34k training samples and train Qwen3-VL models through prefix-guided region-to-image distillation followed by multi-task GRPO. FineRich improves FRBench overall scores from 58.41 to 70.39 for the 4B model and from 60.31 to 69.96 for the 8B model, with additional gains on external benchmarks.

Critical strengths
  1. The benchmark and synthesis pipeline are the most useful contributions. FRBench evaluates local text understanding across different image types, tasks, and region sizes, rather than relying only on full-image question answering. And the HTML representation also provides a practical way to modify local content and is easy to obtain the corresponding text and position annotations.

  2. The reported gains are substantial and extend beyond FRBench, and the stage-wise and prefix-guidance ablations also provide evidence that the training pipeline is useful.

Critical weaknesses
  1. I think the contribution of the proposed training components is not sufficiently isolated. As table 6 supports the two-stage pipeline and prefix guidance, but does not separately evaluate sparse token selection or reliability weighting, and how much these components contribute beyond the two-stage pipeline and prefix guidance.

  2. The relationship between the “OPSD” baseline and Vision-OPD is unclear. Moreover, the full pipeline uses additional training data and computation compared with direct GRPO.

  3. Important training details are missing from the manuscript, the candidate-set size used for prefix checking is also not clearly specified. These omissions make it difficult to reproduce the results and the training budget needed.

Questions for authors
  1. Under matched data and training budgets, how much do sparse token selection and reliability weighting improve over plain crop-to-image distillation?

  2. Could you provide the complete training configuration and clarify which components are included in each OPSD variant in Table 6?

7 comments
AuthorsOpenReview ↗

Response to Reviewer ydQr - Part 1

W1 / Q1. Contributions of the training components

Thank you for highlighting the need to clarify and evaluate the individual training components. Reliability weighting is integrated into our sparse refinement module, where it contributes to both token selection and KL-loss weighting. The row previously named “Sparse Token Selection” evaluates this complete module, including reliability weighting. We will rename it “Sparse Refinement (Token Selection + Reliability Weighting)” to make its scope explicit.

We conducted the following additional ablations on the 8B backbone.

Training configurationQA ANLS ↑Grounding IoU ↑Recognition ANLS ↑Referring ANLS ↑Parsing AVG ↑Overall ↑
Plain OPD (Recognition + Referring)72.8923.5785.1981.4662.3464.06
OPD + Sparse Refinement77.2717.4181.4470.7068.2965.00
OPD + Prefix Rewriting76.4120.5080.7976.2767.7565.62
Prefix-OPD (Full)77.9026.1583.4278.2968.5267.48

Adding sparse refinement increases Overall from 64.06 to 65.00, while prefix rewriting alone increases it to 65.62. Combining both modules achieves 67.48, a 3.42-point improvement over plain OPD and a 1.86-point improvement over prefix rewriting alone. This directly measures the additional contribution of sparse refinement, including reliability weighting, beyond prefix guidance.

The task-level results also reveal the effects of these components. Plain OPD achieves strong Recognition and Referring performance, but lower QA and Parsing scores. The full method improves QA by 5.01 points, Parsing by 6.18 points, and Grounding by 2.58 points, with some reduction in Recognition and Referring. These results align with our motivation to mitigate over-specialization during crop-to-image distillation. We will incorporate these ablations and explicit component definitions into the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer ydQr - Part 2

W2&Q2. Relationship to Vision-OPD and comparison with training baselines

Thank you for pointing out the ambiguity. The “OPSD” baseline in Table 6 corresponds to our Recognition + Referring adaptation of Vision-OPD. It uses a crop-view teacher to supervise a full-image student. For text-rich images, these two tasks provide explicit OCR supervision for reading the target region. We additionally evaluated QA-based Vision-OPD to distinguish the effects of the distillation task.

Our Prefix-OPD builds on this baseline by adding prefix correction and sparse refinement, with reliability weighting integrated into sparse refinement. The complete FineRich pipeline subsequently applies multi-task GRPO. The following table clarifies these configurations and their results on the 8B backbone.

Training configurationQA ANLS ↑Grounding IoU ↑Recognition ANLS ↑Referring ANLS ↑Parsing AVG ↑Overall ↑
Qwen3-VL-8B78.5518.0073.9637.7468.5560.31
Standard SFT77.7219.0784.1854.6768.0963.50
Vision-OPD (QA only)78.3114.6470.3839.6169.0259.88
Vision-OPD (Recognition + Referring), termed “OPSD” in Table 672.8923.5785.1981.4662.3464.06
OPD + Sparse Refinement77.2717.4181.4470.7068.2965.00
OPD + Prefix Correction76.4120.5080.7976.2767.7565.62
Prefix-OPD (Full first stage)77.9026.1583.4278.2968.5267.48
Direct GRPO79.6925.3784.1573.5968.5267.11
SFT → GRPO79.7827.1985.3272.2770.2368.19
Vision-OPD (Recognition + Referring) → GRPO80.4631.1784.6676.0370.6169.34
FineRich-8B(Prefix-OPD → GRPO)80.0538.5485.9380.6572.1271.71

The full pipeline uses the same training data as the direct-GRPO baseline. Its distinction lies in the training procedure, which first develops region-level reading through distillation and then jointly optimizes the task domains through GRPO. In the submitted results, this procedure improves Overall from 67.11 for direct GRPO to 69.96 for FineRich.

The distillation stage incurs additional computation, including teacher forward passes. We will explicitly report the stage-specific training steps and computational costs, and identify the extended two-epoch result separately, so that the training budgets are clear.

AuthorsOpenReview ↗

Response to Reviewer ydQr - Part 3

W3: Training details and prefix candidate-set size

Thank you for pointing out the missing implementation details. We clarify the training configurations for both stages below and will include them in the revised manuscript.

1. Prefix-guided region-to-image OPD

We implement OPD in veRL, initializing the student and frozen teacher from the same backbone. The dataset contains approximately 20,000 task examples, comprising 10,000 Box OCR examples and 10,000 Referring examples. The student receives a full-page image with a red target box for Box OCR, or an unmarked page with normalized bounding-box coordinates for Referring. The teacher receives the corresponding evidence crop and scores tokens under the same response history. We train for one epoch at a learning rate of 1×10−61\times10^{-6}, using 32 prompts per batch, two responses per prompt, and a maximum prompt length of 8,192 tokens.

Distillation uses the teacher’s top-100 tokens plus a residual-probability bucket, gathering student probabilities for the same token IDs. Sparse refinement combines student entropy, JS-divergence, and history-based reliability weighting to select up to 25% of eligible positions. Prefix checking compares the first four sampled tokens with the teacher’s top-1 predictions under the same histories, giving a prefix length of four and candidate-set size of one. Any mismatch triggers replacement with the first four teacher-generated prefix tokens and generation of a new continuation. The objective is LOPD=Lsparse,KL+0.1Lprefix,CE.\mathcal{L}_{\mathrm{OPD}} = \mathcal{L}_{\mathrm{sparse\\,KL}} + 0.1\mathcal{L}_{\mathrm{prefix\\,CE}}., with reliability-weighted forward KL on selected continuation positions and prefix cross-entropy only on corrected responses. The KL coefficient is 1.0, the teacher remains frozen, and training uses no task reward or policy-gradient term.

2. Multi-task GRPO

GRPO initializes from the OCR and Referring distillation checkpoint, with a frozen copy serving as the reference. Before prompt-length filtering, the dataset contains approximately 14,000 examples across QA, Grounding, Referring, Box OCR, and Parsing. Both model sizes train for one epoch at an actor learning rate of 1×10−61\times10^{-6}. Each rollout batch contains 64 prompts × eight responses, divided into two actor minibatches of 256 responses. Maximum prompt and response lengths are 8,192 and 4,096 tokens, respectively. The policy-ratio clipping range is [0.8, 1.2], and the reference KL coefficient is β=0.01\beta=0.01.

QA, Referring, and Box OCR use ANLS-style similarity, with 5% relative tolerance for numeric ChartQA answers. Grounding uses the maximum IoU between valid predicted and reference boxes. Document parsing averages 1−TextED1-\mathrm{TextED} and TEDS over reference components, while table-only examples use TEDS.

Only QA and Grounding apply the format multiplier 0.9+0.1f0.9+0.1f. For QA, f=1f=1 requires exactly one complete <answer>...</answer> span, followed only by whitespace. For Grounding, f=1f=1 requires at least one <box>...</box> entry, with every entry containing four valid coordinates satisfying 0≤x1<x2≤10000\le x_1 < x_2\le1000 and 0≤y1<y2≤10000\le y_1 < y_2\le1000. Otherwise, f=0f=0. Valid outputs receive the full task score, and invalid outputs receive 90%. Box OCR, Referring, Document Parsing, and Table Parsing use their task scores directly.

Reviewer_PPn2OpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the difficulty that multimodal large language models (MLLMs) face in perceiving and understanding local fine-grained information in text-rich images. It makes contributions along three lines: an evaluation benchmark, a data synthesis pipeline, and a training method.

Benchmark. The authors construct FRBench, which spans 6 image types (Document, Infographics, Chart, Table, Slide, and WebPage), 5 task domains (Understanding/QA, Grounding, Recognition, Referring, and Parsing), and 3 granularity levels (super fine, fine, and coarse) defined by the ratio of the evidence-region area to the full image area. It covers all 90 combinations and contains 2,190 manually verified samples. The average evidence-region area ratio is 1.15%, lower than that of existing comparable benchmarks. For full-page parsing, the authors further propose two new metrics, RTM and RROD, to measure whether a model preserves a specified local detail when parsing the full image.

Data synthesis pipeline. Source images from existing datasets are converted into HTML and filtered by image similarity. Structurally similar but content-different distractor DOMs are then inserted into the container of the evidence DOM and scaled down to fit within the original container, thereby increasing local information density. The edited HTML is finally rendered into fine-grained images, with text content and coordinate annotations obtained directly from the DOM.

Training method. Using the same pipeline, the authors synthesize 34K training samples and propose FineRich, a two-stage training method. The first stage is prefix-guided region-to-image on-policy self-distillation (Prefix-guided OPSD), which transfers the model's recognition ability on cropped regions to full-image inputs. The second stage is multi-task GRPO.

Results. The authors evaluate 21 open-source and 2 closed-source models on FRBench. FineRich-4B and FineRich-8B, both built on Qwen3-VL, achieve Overall scores of 70.39 and 69.96, respectively, second only to Gemini3.6 (72.75) and surpassing GPT5.5 as well as open-source models with hundreds of billions of parameters. They also yield moderate improvements on external fine-grained benchmarks and general OCR benchmarks.

Critical strengths

1. A well-motivated problem and a systematically designed benchmark

Fine-grained text-rich image understanding is an important weakness of current MLLMs, yet existing benchmarks typically cover only a few image types or tasks. FRBench is systematically designed along three dimensions (image type, task type, and granularity), covers all 90 combinations, and provides a clear, quantifiable definition of "fine-grained" based on the evidence-region area ratio. This allows evaluation results to be decomposed and analyzed along each dimension; for example, Tables 3 and 4 reveal that most models degrade as granularity becomes finer and that charts are the most challenging image type. The newly proposed RTM and RROD metrics also address a shortcoming of existing full-page parsing metrics, which tend to be dominated by large bodies of content and fail to reflect whether local details are preserved. In addition, the paper provides a fairly comprehensive evaluation of 23 mainstream models, which serves as a valuable reference for the community.

2. A novel and scalable HTML-based synthesis approach

Using HTML as an intermediate representation for data synthesis simultaneously addresses two difficulties in constructing fine-grained data. First, by inserting structurally similar distractor DOMs and scaling them down, local information density can be increased in a controllable manner while preserving the natural appearance of the page, yielding more challenging samples. Second, text content and coordinates can be obtained directly from the DOM, eliminating the high cost of box-level manual annotation for the grounding, recognition, and referring tasks. The pipeline is applicable to multiple image types, and its later stages can be carried out by lighter-weight models, giving it good scalability.

3. An effective training method that yields competitive results at small model scales

Prefix-guided OPSD targets a specific failure mode, namely the student model reading the wrong region when given the full image, by introducing a prefix-correction mechanism; the design motivation is clear. The ablation study (Table 6) also candidly presents the trade-offs between tasks across training stages. The resulting 4B and 8B models outperform GPT5.5 and open-source models with hundreds of billions of parameters on FRBench, without performance degradation on external fine-grained benchmarks or general OCR benchmarks, indicating a degree of practical value.

Critical weaknesses

1. The evaluation protocol for fine-grained region tasks is not validated, which undermines the benchmark's core conclusions

The central premise of FRBench is fine-grained evaluation: evidence regions occupy only 1.15% of the image area on average, and no more than 0.2% at the super fine level. However, the paper provides no analysis of the reliability of the evaluation protocol itself at such small scales. This manifests in two respects.

  • Grounding is measured by a single metric, with no thresholded metrics or distributional analysis. Only continuous mean IoU is reported for grounding; neither thresholded metrics such as [email protected]/0.5/0.75 nor IoU distributions broken down by granularity are provided. For very small target boxes, an offset of only a few coordinate units can cause a sharp drop in IoU, so mean IoU cannot distinguish between "approximately correct localization with imprecise boundaries" and "completely wrong localization." The paper concludes from this metric that grounding is the most prominent weakness, with most models scoring below 30 IoU, but it is currently impossible to tell whether this reflects a genuine lack of model capability or the instability of IoU itself at small scales.
  • The ground-truth boxes may be systematically biased. The GT boxes are taken directly from the bounding boxes of DOM elements, which may include padding and line-height spacing and are therefore not equivalent to tight boxes around the rendered text; this discrepancy accounts for a larger proportion of the box for small elements. The paper does not discuss the consistency between DOM bounding boxes and the visual text regions.

Impact: The main conclusions in Tables 2 and 3 regarding grounding and granularity difficulty, as well as FineRich's large improvement in grounding (IoU from 15.95 to 42.22 for the 4B model), all rest on this evaluation protocol. Since the validity of the protocol has not been verified, the credibility of these conclusions is materially affected.

2. Insufficient assurance of the representativeness and QA validity of the synthesized data, with model gains coming mainly from in-distribution data

  • The HTML synthesis pipeline introduces selection bias and struggles to cover genuinely complex real-world documents. Source images are first converted into HTML by an MLLM and then filtered at a threshold of Ts = 0.6 on the average of CLIP, SSIM, and OCR-BLEU similarity, but the paper does not report the proportion of images filtered out or the distribution of the filtered samples. Real-world documents with complex layouts, photographs, or unstructured visual elements are precisely those hardest to convert faithfully into HTML, and are therefore the most likely to be removed at this stage; the authors themselves show conversion failures such as mismatched chart values and unreproducible photographs in Figure 6(a)(b). The similar performance on source images and HTML-rendered images in Figure 4 only indicates that the retained samples were converted with high fidelity; it does not show that the pipeline can cover difficult real-world documents. Moreover, all final rendered images are clean, born-digital images that exclude realistic conditions such as scanning, photographing, handwriting, skew, degradation, and multi-page documents. Consequently, the difficulty of FRBench arises mainly from the increased local information density produced by scaling and distractor insertion, rather than from intrinsic layout complexity.
  • QA validity is checked only at the HTML text level. After distractor DOMs are inserted, whether the answer to the original question remains unchanged is judged by an LLM based on the HTML text of the evidence DOM and the new DOMs (Figure 15); no check whatsoever is performed on the rendered image. Whether the scaled-down text remains legible, and whether the inserted DOMs introduce visual ambiguity (e.g., multiple price labels for a single product in Figure 6(c), or the color–category mismatch in Figure 6(d)), cannot be detected by text-level checks. The authors could have introduced an MLLM to re-verify answerability and answer uniqueness on the rendered images, but did not; QA quality therefore ultimately relies on manual verification as the last line of defense. For the training set, only 200 samples were spot-checked, with an 83.5% acceptance rate, implying that roughly 16.5% of the training samples may be noisy.
  • Training and test data share the same sources, giving FineRich an in-distribution advantage. Training images are drawn from the training sets of DocVQA, InfoVQA, and ChartQA, while test images are drawn from the test sets of the same datasets, and both are generated by the same synthesis pipeline, resulting in highly consistent visual styles. On external benchmarks (Table 5), FineRich-4B improves the fine-grained TIU average by only 4.35 points and the general OCR average by only 1.05 points, a marked gap from its roughly 12-point Overall gain on FRBench (58.41 → 70.39).

Impact: Both core contributions of the paper, namely building a benchmark from synthesized data and training models on synthesized data, depend on the quality and representativeness of the synthesized samples. Given the selection bias, the insufficient quality assurance, and the in-distribution advantage, it remains unclear whether FRBench reflects the difficulty of fine-grained understanding in real-world scenarios, and how much of FineRich's lead over models such as GPT5.5 stems from genuine capability rather than adaptation to the style of the synthesized data.

3. No release of data, code, or models, limiting reproducibility and value to the community

The paper provides no plan to release code, data, or model weights, and does not include a reproducibility statement. A main contribution of this paper is the HTML synthesis pipeline together with the resulting FRBench and 34K training samples; for a work whose core contribution is a benchmark and data construction, these resources constitute the substance of the contribution itself. Although the appendix provides all prompts used in the synthesis pipeline (Figures 12–16), the prompts alone are insufficient to reproduce the full pipeline, including the implementation of the similarity filtering, the rules for DOM insertion and scaling, and the criteria for manual verification.

Impact: Without public release of the data and pipeline, FRBench cannot be adopted by the community as an evaluation standard, and the evaluation and data-quality issues raised in Weaknesses 1 and 2 cannot be independently examined by third parties, which substantially diminishes the practical contribution of this work.

4. Insufficient disclosure of training details and unexplained experimental results

This point is relatively minor, but it still affects the assessment of the method's effectiveness.

  • Incomplete training details. The reward design for multi-task GRPO is described in a single sentence in the main text; how f in the format coefficient (0.9 + 0.1f) is determined is not defined, and no ablation is provided. Key hyperparameters, including the learning rate, batch size, number of epochs, GRPO group size G, number of training steps, and computational resources, are not reported, nor are the decoding settings and input resolution used during evaluation.
  • Some results in Table 2 are counterintuitive and are not discussed. For example, FineRich-4B achieves a slightly higher Overall score than FineRich-8B (70.39 vs. 69.96), with a more pronounced gap on grounding (42.22 vs. 36.31); Kimi-K3 scores lower in Overall than its predecessor Kimi-K2.6 (58.82 vs. 60.20), and its Referring ANLS is only 20.68 while its Recognition ANLS reaches 89.09; and Qwen3.5-397B also scores lower in Overall than Qwen3.5-27B (57.3 vs. 62.1). In addition, in Table 6, prefix guidance affects grounding in opposite directions at the two model scales (from 26.32 to 19.40 for 4B, and from 23.57 to 26.15 for 8B). Since the paper does not report variance across multiple runs, it is difficult to judge whether these differences exceed random fluctuation.

Impact: Although these issues do not directly invalidate the main conclusions of the paper, they weaken the interpretability and reproducibility of the experimental results. The authors are encouraged to provide the complete training configuration and to analyze the anomalies noted above.

Questions for authors
  1. Reliability of small-target grounding evaluation. Could the authors report the distribution of grounding IoU (e.g., histograms or quantiles) separately for the super fine, fine, and coarse levels, and supplement thresholded metrics such as [email protected]/0.5/0.75? At the super fine level, what proportion of predicted boxes overlap with the GT boxes but receive low scores solely because of low IoU? Furthermore, are the GT boxes taken directly from the bounding boxes of DOM elements (e.g., via getBoundingClientRect), and do they include padding and line-height spacing? If tight boxes around the rendered text were used instead, would the IoU scores and the ranking of models change?

  2. Selection bias of the synthesis pipeline and QA validity. At the image-to-HTML stage, how many source images were filtered out by the similarity threshold Ts = 0.6? How are the filtered samples distributed in terms of image type and layout complexity? In addition, have the authors verified, on the rendered images (using an MLLM or human annotators), that the QA pairs of the synthesized samples remain answerable with a unique answer? If so, please report the corresponding acceptance rate.

  3. Details of manual verification. Please disclose the number and professional background of the annotators, the specific criteria used for verification, and the inter-annotator agreement (e.g., Cohen's κ). Among the 12.4% of samples rejected during manual verification, what proportion is attributable to each failure type (e.g., invalidated QA, rendering errors, annotation errors)? The parsing ground truth is generated by an LLM; was it separately subjected to manual proofreading?

  4. Release plan. Do the authors plan to publicly release FRBench (including the images, HTML source files, and all annotations), the synthesis pipeline code, the 34K training samples, and the FineRich model weights? If so, please indicate the expected timeline and license.

  5. Training details and explanation of experimental results. Please provide the complete configuration of multi-task GRPO, including the learning rate, batch size, number of epochs, group size G, number of training steps, and computational resources, as well as how f in the format coefficient (0.9 + 0.1f) of the reward is determined; please also specify the decoding settings and input resolution used during evaluation. Regarding some of the results in Table 2, such as FineRich-4B outperforming FineRich-8B in both Overall and grounding, and Kimi-K3's Referring ANLS (20.68) being far lower than its Recognition ANLS (89.09), do the authors have an explanation?

11 comments
AuthorsOpenReview ↗

Response to Reviewer PPn2 - Part 1

Thank you for your careful review and comprehensive feedback. Your comments offered valuable suggestions and prompted us to think more deeply about several aspects of our work. We appreciate the opportunity to discuss these points with you. We address your comments below and will incorporate the relevant clarifications and changes to update the manuscript.

Q1.1&Q1.2&W1.1: Grounding Distribution Analysis:

We present the IoU distribution and thresholded accuracy by granularity for GPT5.5 and Kimi-K3, whose IoU on super-fine targets is substantially lower than on fine or coarse targets. The IoU distributions are shown below, leading to the following observations:

1) The low IoU on super-fine targets comes mainly from completely wrong localization. Predictions with IoU = 0 are much more common on super-fine targets than at the other granularity levels. To assess how far these predictions fall from the target, we measure the shortest edge-to-edge distance between the predicted and ground-truth boxes. On super-fine targets, this distance exceeds one ground-truth box diagonal for 80.46% of zero-overlap predictions from Kimi-K3, which are completely wrong localization rather than slight offsets.

2) The proportion of low but positive IoU scores (0 < IoU ≤ 0.3) is also lower on super-fine targets than on fine or coarse targets. This indicate that, compared with fine or coarse targets, super-fine targets have smaller share of grounding predictions in the low-IoU range, proving that the effect of minor IoU differences remains limited as regions get smaller.

ModelGranularityIoU = 00 < IoU ≤ 0.3IoU > 0.3 ([email protected])IoU > 0.5 ([email protected])IoU > 0.75 ([email protected])Mean IoU
GPT5.5Super-fine51.7530.7017.548.771.7513.26
GPT5.5Fine30.3827.8541.7724.057.1727.27
GPT5.5Coarse3.4550.5745.9834.4812.6431.70
Kimi-K3Super-fine76.3213.1610.535.262.636.84
Kimi-K3Fine58.6518.9922.3613.923.8015.26
Kimi-K3Coarse44.8333.3321.8414.948.0518.28

Q1.3&W1.2: Consistency between DOM bounding boxes and the visual text regions:

To obtain bounding boxes for text-bearing evidence DOMs, we call Range.getClientRects() on text ranges after trimming leading and trailing whitespace. We take the smallest rectangle enclosing the returned text rectangles, yielding a tight text-layout bounding box that excludes the surrounding element’s padding. Thus, the reported IoU scores and model rankings in our paper already use these tight text-layout boxes.

AuthorsOpenReview ↗

Response to Reviewer PPn2 - Part 2

Q2.1&Q2.2&W2.1: Selection bias of the synthesis pipeline:

The table below shows the distribution of samples filtered out at the image-to-HTML step. We use the established multiscale edge-grid coverage (%) metric to measure image complexity and classify images with scores above 75% as high-complexity images. We have the following observations:

  1. The overall image filtered-out rate is 35.41%. Infographics and web images have higher filter-out rates, primarily because infographics have complex layouts and web images contain real-world photographs. Nevertheless, although complex images are relatively difficult to convert, many can still be converted successfully with capable MLLMs such as Kimi-K2.6. As shown in the table, the difference in mean complexity between retained and filtered-out images is limited, and the retained set even contains more high-complex images.

2)In the final FRBench, 52.05% of the source images are high-complex. Figure 4 shows that MLLMs still struggle on the source FRBench images, even for large-scale MLLMs like Qwen2.5-VL-72B. These results indicate that our synthesis pipeline can cover difficult ducument.

  1. We acknowledge that our pipeline can only synthesize born-digital images and its coverage excludes real-world photographs. But we have to declare that born-digital images are also common in real-world applications, where FineRich can offer practical benefits. On one hand, we have covered various text-rich image types. On the other hand, central contribution of this paper is to propose a reliable data synthesis method for large-scale fine-grained TIU samples. Training on such synthesized data can improve fine-grained capabilities and even transfer to real-world images. Table 5 illustrates gains on DOGR and CC-OCR, which includes realistic photo understanding.
Image TypeFiltered-out RatioMean Complexity (Filtered-out)Mean Complexity (Retained)High-Complexity Share (Filtered-out)High-Complexity Share (Retained)
Doc27.16%80.76%68.45%18.35%29.55%
Info56.57%94.30%90.94%55.30%40.68%
Chart34.56%55.88%53.53%4.76%4.23%
Table28.68%96.36%89.23%27.66%57.48%
Slide29.03%75.14%65.69%15.64%22.38%
Web60.92%56.55%59.52%10.60%9.81%
Total35.41%70.55%65.97%16.01%21.76%

Q2.3&W2.2: QA validity check:

This question has several facets, which we address in detail below:

  1. QA pairs are from text-centric datasets, which focus on textual information in text-rich images. Thus, we can obtain evidence DOM from the converted HTML texts and use it as a reference to generate new DOMs. All systhesis steps are from text to text. So checking whether the new DOMs change the original answer can primarily rely on the text.

  2. The construction failures in our paper are caused by low quality not invalid QA. The synthesized layouts violate real-world conventions, but the underlying QA pairs still remain valid. This is because QA task only evaluates whether the model can identify evidence and use it to answer the question. As long as the information needed to answer the question is clearly presented and supports a unique answer, these samples can still be used to evaluate QA. For example, Fig. 7(c) shows a single product with multiple prices, which is unrealistic. However, the answer to “Which discount offers the biggest savings?” remains unchanged because each new DOM shows a smaller discount than the evidence DOM.

  3. To verify this, we manually checked all 500 synthesized QA samples. In only six cases did the inserted DOMs introduce visual ambiguity and invalidate the original QA pair. Thus, 98.8% (494/500) of the samples remained answerable with a unique answer. Other rejected samples were filtered out because their rendered images violated real-world conventions, degrading image quality.

We also evaluated Kimi-K2.6 on all filtered-out samples, obtaining an ANLS of 71.4%, which means that images violated real-world conventions are still can be answered. And we manually examined the cases that Kimi-K2.6 answered incorrectly. Apart from the visually ambiguous cases, the errors arose from difficulties in recognizing or understanding fine-grained texts.

AuthorsOpenReview ↗

Response to Reviewer PPn2 - Part 3

W2.3: In-distribution advantage on FineRich:

  1. We agree that training data leverages training splits of DocVQA, InfoVQA, and ChartQA, which are also test split sources of FRBench. However, table, slide and webpage data are absent from our training set and account for 48.2% of FRBench. On samples of these three image types, FineRich-4B improves over Qwen3-VL-4B by +27.71%,+21.29%,+17.93%, respectively (Table 4), demonstrating out-of-domain generalization for unseen image types.

  2. On external fine-grained TIU benchmarks, FineRich-4B improves the average from 56.32 to 60.67, which is a substantial relative improvement of 7.7%. The limited gain on general OCR is due to FineRich focus on fine-grained tasks, such as grounding and recognition for local regions. But general OCR benchmarks assess a broader range of skills, where fine-grained tasks only occupies a small part. For example, only 11.33% of CC-OCR samples are related to fine-grained recognition. On this fine-grained subset, FineRich-4B achieves +5.99 (10.83%) gain. But across CC-OCR as a whole, it just gets +1.52 (2.01%) gain.

Q3.1. Details of manual verification:

Three graduate students in computer science served as annotators. Before verification, they received training on the criteria for each task, with examples of acceptable and unacceptable samples. For Understanding, the rendered image had to be visually realistic and support the unique answer to the question. For Recognition and Referring, the reference answer had to exactly match the text in the target region. For Grounding, the reference bounding box had to enclose the evidence needed to answer the question. For Parsing, verification was guided by the OmniDocBench annotation guidelines.

In the first round, two annotators independently verified the samples and achieved 96.3% raw agreement before adjudication. The third annotator resolved disagreements. In the second round, a professional annotator with experience in OCR annotation audited a random 10% of the verified samples and all audited samples passed.

Q3.2. Failure Type Distribution:

We reviewed all rejected samples and first divide the failures into two categories: cases where the newly inserted DOM elements invalidate the original QA, and cases where the resulting synthetic images are unrealistic. The first category accounts for 8.3% of all failures. The second can be further divided into three types: duplicated DOM elements, misaligned visual information (e.g., bars that do not match the chart axes in Figure 7), and missing real photographs. These account for 38.9%, 29.2%, and 22.2% of all failures, respectively.

Q3.3. parsing annotation check:

All annotations are checked via manual proofreading, including parsing ground truth.

Q4&W3. Release plan:

Of course we will release all until this paper is accepted, including FRBench (our sythesized images, source images, all annotations and HTML files), training set, synthesis pipeline code, FineRich method code and FineRich model weights. We are so welcome for more evaluations on FRBench.

Reviewer_BUUcOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper focuses on fine-grained text-rich image understanding. It introduces an HTML-based synthesis pipeline and contributes FRBench, an comprehensive benchmark. It then proposes a two-stage training recipe (OPSD + GRPO) on a similarly synthsized dataset. The proposed method achieves competitive performance on FRBench. Gains are also seen on external fine-grained benchmarks.

Critical strengths
  1. A comprehensive benchmark with exact region annotations from rendering is valuable.

  2. The data synthesis pipeline is helpful to scale up the training data. However, this is contingent on whether this pipeline will be open-sourced (Also see Q1)

Critical weaknesses
  1. Overclaim. The authors "leading performance on FRBench". However, Gemini 3.6 is the best (72.75% vs 69.96%)

  2. Missing baselines: Generally, supervised fine-tuning and Vision-OPD are straightforward baseline methods to run. For OSPD particularly, simple zoom-in or crop-based inference baselines are absent. In fact, measuring across multiple families of MLLMs is not scientific given (1) different training data, (2) different formatting convention, and (3) varied preprocessing defaults.

  3. Mixed results for the model size. FineRich-4B sometimes outperforms FineRich-8B (Tables 2-6). This suggests the margins between the two may be within noise. The authors did not explain this.

Questions for authors
  1. Any plan for benchmark release? Does this include the entire synthetic data pipeline or only the data?
9 comments
AuthorsOpenReview ↗

Response to Reviewer BUUc - Part 1

Thank you for the constructive review and for highlighting the value of FRBench’s region annotations and the synthesis pipeline. We address each point below.

Q1. Scope of the FRBench claim and additional 27B results

ModelUnderstanding (ANLS)Grounding (IoU ↑)Recognition (ANLS)Referring (ANLS)Parsing (AVG ↑)Overall ↑
Gemini3.681.0535.3392.1087.8271.4272.75
Qwen3.8-27B81.0625.1284.8455.9469.7765.76
FineRich-4B77.4342.2286.2686.8767.5970.39
FineRich-8B*80.0538.5485.9380.6572.1271.71
FineRich-27B88.3360.2596.3188.3972.8278.07

We apologize for the imprecise wording. In the original Table 2, FineRich achieved the highest Overall scores among the evaluated open-source models, while Gemini 3.6 achieved the highest score overall. We will revise the corresponding statements to clarify this distinction.

To test whether FineRich’s gains extend to a larger backbone, we trained FineRich-27B from Qwen3.8-27B. It achieved 78.07 Overall, establishing a new state of the art on FRBench among all evaluated open-source and closed-source models. This represents a 12.31-point improvement over its base model and exceeds Gemini 3.6, the strongest baseline in Table 2, by 5.32 points. The largest gains over the 27B base model were in Grounding (+35.13 IoU points) and Referring (+32.45 ANLS points), the tasks most directly related to locating and reading specific regions. FineRich-8B* denotes the model after extending training to two epochs.

These results, together with the gains at 4B and 8B, demonstrate that our synthesized training data and training method together enable further improvements in fine-grained text-rich image understanding across model scales, including a stronger 27B backbone.

AuthorsOpenReview ↗

Response to Reviewer BUUc - Part 2

Q2: Missing SFT and Vision-OPD training baselines.

Thank you for highlighting the need for stronger training baselines. We agree that comparisons with existing models alone do not isolate the benefit of our training strategy. We also apologize for the unclear naming: the “OPSD” baseline in our paper corresponds to Vision-OPD (Recognition + Referring), our task-specific adaptation of Vision-OPD to text-rich image understanding. Instead of applying distillation directly to QA, this variant uses region-level Recognition and Referring tasks, where explicit OCR targets provide more direct supervision for reading the relevant text. To clarify this distinction and evaluate our design, we conducted additional comparisons with standard SFT, QA-based Vision-OPD, and Recognition + Referring-based Vision-OPD, together with component ablations and comparisons of their subsequent GRPO refinement, as shown below:

Training recipeQA ANLSGrounding IoURecognition ANLSReferring ANLSParsing AVGOverall
Qwen3-VL-8B-Instruct78.5518.0073.9637.7468.5560.31
Standard SFT77.7219.0784.1854.6768.0963.50
Vision-OPD (Recognition + Referring)72.8923.5785.1981.4662.3464.06
Vision-OPD (QA only)78.3114.6470.3839.6169.0259.88
OPD + Sparse Token Selection77.2717.4181.4470.7068.2965.00
OPD + Prefix Rewriting76.4120.5080.7976.2767.7565.62
Prefix-OPD (Full)77.9026.1583.4278.2968.5267.48
SFT → GRPO79.7827.1985.3272.2770.2368.19
Vision-OPD (Recognition + Referring) → GRPO80.4631.1784.6676.0370.6169.34
FineRich-8B (Prefix-OPD → GRPO)80.0538.5485.9380.6572.1271.71

The additional results clarify two aspects of our approach:

First, the complete first-stage method achieves a better balance across tasks than standard SFT or crop-to-image distillation alone. Prefix-OPD achieves an Overall score of 67.48, compared with 63.50 for SFT and 64.06 for Vision-OPD trained on Recognition and Referring. The latter achieves strong region-level reading performance, but lower QA and Parsing scores. Prefix-OPD improves QA from 72.89 to 77.90 and Parsing from 62.34 to 68.52, while also improving Grounding from 23.57 to 26.15. This comes with some reduction in Recognition and Referring, consistent with the task trade-offs discussed in the paper.

The component ablations further support the combined design: adding sparse token selection or prefix rewriting individually yields Overall scores of 65.00 and 65.62, respectively, whereas the full first-stage method reaches 67.48. These results support their combined effectiveness, without suggesting that each component uniformly improves every task.

Second, the benefit remains after GRPO refinement. FineRich-8B achieves 71.71 Overall, outperforming SFT → GRPO by 3.52 points and Vision-OPD → GRPO by 2.37 points. Relative to Vision-OPD → GRPO, the largest improvement is in Grounding (+7.37 IoU points), accompanied by gains in Recognition, Referring, and Parsing. These comparisons provide stronger evidence that the proposed first-stage training contributes to the final result beyond the gains obtained by applying GRPO after SFT or standard crop-to-image distillation.

AuthorsOpenReview ↗

Response to Reviewer BUUc - Part 3

Q3: Comparison with higher-resolution and crop-based inference baselines.

We added comparisons with higher-resolution full-image inference and ground-truth evidence-crop inference, focusing on Recognition and Referring, the two tasks used in our region-to-image distillation stage. Both require transcribing text from a specified region: Recognition identifies the region through a visual box, whereas Referring provides its coordinates in text. These tasks therefore directly assess whether increasing resolution or isolating the target region can address the difficulties targeted by our method.

Inference setupRecognition ANLS ↑Referring ANLS ↑Avg. input tokensAvg. output tokens
Original full image74.2335.133,88036.7
Higher-resolution full image75.4854.207,92951.1
Ground-truth evidence crop (oracle)96.1195.6415169.0
FineRich-4B, original full image87.3887.893,88095.0

Higher-resolution inference improves Recognition and Referring by 1.25 and 19.07 points, respectively, but approximately doubles the input token count. At the original input length, FineRich-4B still outperforms this baseline by 11.90 and 33.69 points, suggesting that resolution alone does not resolve the difficulty of reading specific regions within a full image.

Evidence-crop inference achieves 96.11 Recognition ANLS and 95.64 Referring ANLS, supporting our motivation to transfer strong crop-level reading capabilities to full-image inputs. FineRich substantially narrows this gap without increasing input tokens or requiring a crop-based teacher at inference, although its longer outputs incur additional decoding cost.

Q4. Mixed results across model sizes

We extended training to two epochs for both the 4B and 8B models and evaluated a larger 27B model to better understand the observed differences.

ModelQA ANLSGrounding IoURecognition ANLSReferring ANLSParsing AVGOverall
FineRich-4B77.4342.2286.2686.8767.5970.39
FineRich-4B (2 epochs)76.5047.5084.0881.1167.8070.05
FineRich-8B80.2936.3183.9580.9669.5569.96
FineRich-8B (2 epochs)80.0538.5485.9380.6572.1271.71
FineRich-27B88.3360.2596.3188.3972.8278.07

1) The 8B model benefits more from additional training, while individual task rankings remain dependent on the backbone.

Extending training increases FineRich-8B’s Overall score from 69.96 to 71.71 (+1.75 points), whereas FineRich-4B changes from 70.39 to 70.05 (−0.34 points). Under the same two-epoch schedule, 8B outperforms 4B by 1.66 points overall. This suggests that the 8B model requires more training to realize its potential, while further training of the 4B model primarily shifts the balance between tasks. Moreover, in our evaluations on other benchmarks, we also observed task-dependent strengths between Qwen3-VL-4B and Qwen3-VL-8B, with the 8B model not consistently outperforming the 4B model on every task.

2) The 27B results show a clear improvement at a larger model scale.

FineRich-27B achieves 78.07 Overall and outperforms both smaller models across all five task domains, including their two-epoch results. It also improves over its own backbone by 12.31 points, demonstrating that our data and training approach continue to benefit a stronger model. We will incorporate these additional results and the discussion of training duration into the revised paper.

Comments on the paper

2 comments

We would first like to express our gratitude to the Program Chairs, Senior Area Chairs, and Area Chairs for their efforts, as well as to the dedicated reviewers for their insightful comments on our paper.

Additionally, we appreciate the reviewers for highlighting the strengths of our work:

  1. A comprehensive and systematically designed benchmark. FRBench covers six image types, five task domains, and three granularity levels, with precise region annotations and region-aware parsing metrics that enable detailed analysis of fine-grained text-rich image understanding.

  2. A novel and scalable HTML-based synthesis pipeline. The pipeline generates challenging text-rich images with controllable local information density and distractors, while obtaining text and positional annotations directly from the DOM to reduce manual annotation costs.

  3. A well-motivated region-to-image training framework. Prefix correction and sparse refinement help transfer local reading capabilities from a crop-level teacher to a full-image student, while subsequent multi-task GRPO directly optimizes task-specific objectives.

  4. Strong improvements at small model scales and transfer beyond FRBench. FineRich-4B and FineRich-8B substantially improve over their backbones, remain competitive with much larger models, and achieve gains on external fine-grained benchmarks.

Based on the reviewers’ constructive feedback, we conducted additional experiments and provided clarifications during the rebuttal. We will incorporate the following additions into the revised manuscript:

  1. Stronger training baselines. Using the same backbone and synthesized data, Prefix-OPD achieves 67.48 Overall, compared with 63.50 for SFT and 64.06 for Vision-OPD. We also added SFT → GRPO (68.19) and Vision-OPD → GRPO (69.34) comparisons. These experiments provide evidence for the contribution of our training strategy beyond the benefits of the synthesized data.

  2. Component and selection-ratio ablations. Sparse refinement alone, prefix correction alone, and their combination achieve 65.00, 65.62, and 67.48 Overall, respectively. Among selection ratios of 25%, 50%, 75%, and 100%, 25% performs best across all five task domains, improving Overall by 1.86 points over 100% selection.

  3. Additional training-duration and model-scale experiments. Two-epoch training improves FineRich-8B from 69.96 to 71.71 Overall. FineRich-27B achieves 78.07 Overall, exceeding its backbone by 12.31 points and Gemini 3.6 by 5.32 points, with the highest Overall score among all evaluated open-source and closed-source models. We will distinguish the original and extended-training results and clarify the scope of our performance claims.

  4. Inference baselines. We added higher-resolution and evidence-crop comparisons. At the original full-image input length, FineRich-4B exceeds the higher-resolution baseline by 11.90 Recognition ANLS points and 33.69 Referring ANLS points. The strong evidence-crop results further support region-to-image distillation. We report both input and output lengths to clarify inference costs.

  5. Benchmark validation and generalization analysis. We expanded analyses of grounding errors, bounding-box accuracy, synthesis filtering, and manual verification. Of 500 synthesized QA samples, 494 (98.8%) retain a unique answer. We also clarified external transfer: FineRich-4B improves the average score on external fine-grained benchmarks from 56.32 to 60.67, a 7.7% relative gain.

  6. Training and evaluation details. We clarified the relationship between OPSD and Vision-OPD, the integration of reliability weighting into sparse refinement, and the task-dependent behavior of intermediate checkpoints. We also provided data composition, prefix-checking settings, loss functions, learning rates, training budgets, rollout configurations, and task-specific rewards to improve reproducibility.

  7. Public release. We will release FRBench, synthesized and source images, annotations, HTML files, the training dataset, synthesis pipeline code, FineRich training code, and model weights soon to support reproduction and further research.

Once again, we sincerely thank you for your invaluable input and the time you have dedicated to reviewing our paper. Your constructive feedback and insightful suggestions have been instrumental in enhancing the quality of our research.

We wanted to reach out to confirm that all your concerns have been adequately addressed. Please let us know if any questions remain or if further clarification would be helpful. We welcome your feedback and further discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.