- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes FRBench and FineRich to address the limitations of existing Multimodal Large Language Models (MLLMs) in recognizing and understanding small text regions within complex text-rich images. To construct the dataset, the authors introduce an HTML-based synthesis pipeline that converts existing images into HTML and increases local information density by inserting additional DOM elements around question-relevant evidence. The paper also proposes FineRich, a two-stage training framework using the synthesized data. In the first stage, Prefix-Guided Region-to-Image OPSD distills fine-grained text recognition capabilities from a teacher that receives cropped target regions into a student that processes the full image, using teacher-guided prefix correction and sparse token-level distillation. In the second stage, Multi-task GRPO jointly optimizes performance across understanding, grounding, recognition, referring, and parsing tasks. Experiments across various MLLMs show that FRBench reveals the limitations of existing models in fine-grained understanding, while FineRich achieves substantial improvements over the original backbones on FRBench.
Critical strengths
- One of the main strengths of this paper is the introduction of a relatively comprehensive benchmark for evaluating fine-grained text-rich image understanding. Unlike existing benchmarks that focus on specific image types or tasks, FRBench systematically combines image types, granularity levels, and task domains. By evaluating not only understanding but also grounding, recognition, referring, and parsing, it enables a broader assessment of spatial reasoning and structural understanding beyond simple OCR accuracy. Furthermore, evaluating performance across different granularity levels provides valuable insights into the limitations of existing MLLMs from multiple perspectives.
- Another strength is the proposed training strategy for transferring accurate text recognition capabilities from a crop-level teacher to a full-image student. Prefix-Guided Region-to-Image OPSD introduces teacher-guided prefix correction to address recognition errors caused by incorrect early token predictions. Instead of applying distillation uniformly to all output tokens, it selectively distills tokens based on uncertainty and teacher–student disagreement. This design is well motivated by the challenges of accurate text recognition in dense visual contexts.
Critical weaknesses
- A critical concern is whether the competing methods in Tables 2 and 5 were also fine-tuned. Since FineRich involves additional training, comparing it against models without fine-tuning would raise concerns about fairness. If the baselines were not fine-tuned, they should also be trained on the same data, even with standard next-token prediction, to ensure a fair comparison.
- The first stage of FineRich is closely related to Vision-OPD, as both distill knowledge from a teacher using cropped regions into a student processing the full image. Although the paper introduces prefix correction and sparse refinement as key differences, it does not directly compare against Vision-OPD under the same synthesized data and training settings. Comparisons with standard supervised fine-tuning are also needed to establish the benefits of the proposed training strategy.
- Prefix-Guided Region-to-Image OPSD combines teacher-guided prefix correction, sparse refinement. However, Table 6 only compares OPSD, Prefix-OPSD, GRPO, and their combinations, without isolating the effects of sparse refinement and selection ratio. These ablations are essential to demonstrate whether the proposed mechanisms provide meaningful benefits over simpler distillation approaches.
Questions for authors
- Were the other models in Tables 2 and 5 fine-tuned under the same conditions as FineRich? If the competing models were evaluated without additional training, it is difficult to establish the effectiveness of the proposed method from these comparisons alone. Could the authors provide a baseline fine-tuned with standard next-token prediction using the same synthesized data and training budget to clarify whether the improvements come from the proposed training strategy or simply from additional data and training?
- How much improvement does proposed method provide over the Vision-OPD? Since both methods transfer knowledge from cropped image regions to a full-image student, a direct comparison under the same backbone, synthesized training data, and training budget is necessary.
- Could the authors provide additional ablations to independently evaluate the contributions of sparse refinement and selection ratio? These experiments would help determine whether the proposed method offer meaningful improvements over standard distillation.
7 comments
Response to Reviewer EDN8 - Part 2
W2&Q2. SFT vs. Vision-OPD vs.Prefix-OPD (FineRich):
The “OPSD” baseline in table 6 corresponds to Vision-OPD (Recognition + Referring), our task-specific adaptation of Vision-OPD to text-rich image understanding. Instead of applying distillation directly to QA, this variant uses region-level Recognition and Referring tasks, where explicit OCR targets provide more direct supervision for reading the relevant text. To clarify this distinction and evaluate our design, we conducted additional comparisons among our Prefix-OPD, standard SFT, QA-based Vision-OPD, and Recognition + Referring-based Vision-OPD. Results are shown in the following table and we obseve that Prefix-OPD achieves a better balance across tasks than standard SFT or Vision-OPD. Prefix-OPD achieves an Overall score of 67.48, compared with 63.50 for SFT and 64.06 for Vision-OPD trained on Recognition and Referring. The latter achieves strong region-level reading performance, but lower QA and Parsing scores. Prefix-OPD improves QA from 72.89 to 77.90 and Parsing from 62.34 to 68.52, while also improving Grounding from 23.57 to 26.15. This comes with some reduction in Recognition and Referring, consistent with the task trade-offs discussed in the paper.
| Training configuration | QA ANLS ↑ | Grounding IoU ↑ | Recognition ANLS ↑ | Referring ANLS ↑ | Parsing AVG ↑ | Overall ↑ |
|---|---|---|---|---|---|---|
| Qwen3-VL-8B | 78.55 | 18.00 | 73.96 | 37.74 | 68.55 | 60.31 |
| Standard SFT | 77.72 | 19.07 | 84.18 | 54.67 | 68.09 | 63.50 |
| Vision-OPD (QA only) | 78.31 | 14.64 | 70.38 | 39.61 | 69.02 | 59.88 |
| Vision-OPD (Recognition + Referring), | 72.89 | 23.57 | 85.19 | 81.46 | 62.34 | 64.06 |
| Prefix-OPD | 77.90 | 26.15 | 83.42 | 78.29 | 68.52 | 67.48 |
W3&Q3. Ablation of components within Prefix-OPSD:
Thank you for highlighting the need to evaluate the individual prefix correction and sparse refinement of Prefix-OPSD. We conducted the following ablations of on the 8B backbone, showing that:
-
Adding sparse refinement increases Overall from 64.06 to 65.00, while prefix correction alone increases it to 65.62. Combining both modules achieves 67.48, a 3.42-point improvement over plain OPD and a 1.86-point improvement over prefix correction alone. This directly measures the additional contribution of sparse refinement beyond prefix correction;
-
The task-level results also reveal the effects of these components. Plain OPD achieves strong Recognition and Referring performance, but lower QA and Parsing scores. The full method improves QA by 5.01 points, Parsing by 6.18 points, and Grounding by 2.58 points, with some reduction in Recognition and Referring. These results align with our motivation to mitigate over-specialization during crop-to-image distillation. We will incorporate these ablations and explicit component definitions into the revised manuscript.
| Training configuration | QA ANLS ↑ | Grounding IoU ↑ | Recognition ANLS ↑ | Referring ANLS ↑ | Parsing AVG ↑ | Overall ↑ |
|---|---|---|---|---|---|---|
| Plain OPD (Recognition + Referring) | 72.89 | 23.57 | 85.19 | 81.46 | 62.34 | 64.06 |
| OPD + Sparse Refinement | 77.27 | 17.41 | 81.44 | 70.70 | 68.29 | 65.00 |
| OPD + Prefix Rewriting | 76.41 | 20.50 | 80.79 | 76.27 | 67.75 | 65.62 |
| Prefix-OPD (Full) | 77.90 | 26.15 | 83.42 | 78.29 | 68.52 | 67.48 |
Q3.2. Effect of the token selection ratio
Thank you for suggesting this ablation. We evaluated selection ratios of 25%, 50%, 75%, and 100% on the 8B backbone. The ratio specifies the fraction of eligible continuation positions selected for distillation, excluding the supplied corrective prefix.
| Selection ratio | QA ANLS ↑ | Grounding IoU ↑ | Recognition ANLS ↑ | Referring ANLS ↑ | Parsing AVG ↑ | Overall ↑ |
|---|---|---|---|---|---|---|
| 25% | 77.90 | 26.15 | 83.42 | 78.29 | 68.52 | 67.48 |
| 50% | 77.70 | 22.08 | 83.03 | 76.34 | 67.23 | 66.01 |
| 75% | 76.97 | 19.17 | 82.62 | 75.98 | 68.22 | 65.95 |
| 100% | 76.41 | 20.50 | 80.79 | 76.27 | 67.75 | 65.62 |
The 25% setting achieves the highest scores across all five task domains, outperforming the 50%, 75%, and 100% settings by 1.47, 1.53, and 1.86 Overall points, respectively. These results support concentrating distillation on positions prioritized by student uncertainty, teacher–student disagreement, and reliability. Increasing the selection ratio includes lower-ranked positions and does not improve performance in these experiments. We therefore use 25% as the default and will include this ablation in the revised manuscript.
Seeking Further Feedback
Dear Reviewer EDN8,
Thank you again for your constructive review and helpful suggestions. In our rebuttal, we clarified the evaluation settings and the purpose of the original benchmark comparisons, and added SFT and Vision-OPD baselines, including their subsequent GRPO refinement, using the same backbone and synthesized training data. We also clarified the relationship between OPSD and Vision-OPD and provided separate ablations of prefix correction, sparse refinement, and the token selection ratio. Furthermore, we included additional FineRich-27B results to demonstrate the effectiveness of our approach on a larger backbone. We hope these experiments and clarifications provide a more complete picture of the contributions of our synthesized data and training strategy, the benefits of individual components, and the applicability of FineRich across model scales.
As the discussion period draws to a close, we would greatly appreciate your feedback on these additional results and clarifications. Please let us know if any concerns remain or if further clarification would be helpful. We welcome the opportunity to discuss these points with you.
Thank you once again for your time and consideration.
Best regards, The Authors
Response to Reviewer EDN8 - Part 1
W1&Q1. Fine-tuning fairness and the contribution of the training strategy
**1) **Thank you for raising this concern. The comparison models in Tables 2 and 5 were evaluated using their original released checkpoints or APIs, without additional fine-tuning on our synthesized data. To distinguish the contribution of our training strategy from that of additional training data, we conducted comparisons using the same backbone and synthesized training data.
Compared with standard SFT, our Prefix-OPD achieves 67.48 Overall, versus 63.50 for SFT, an improvement of 3.98 points. This shows that our distillation strategy provides additional benefits beyond standard next-token training on the synthesized data.
Compared with plain region-to-image OPD, our Prefix-OPD improves Overall from 64.06 to 67.48, a gain of 3.42 points, supporting the benefit of the proposed prefix correction and sparse refinement.
After subsequent GRPO training, FineRich achieves 71.71 Overall, outperforming SFT → GRPO (68.19) by 3.52 points and plain OPD → GRPO (69.34) by 2.37 points. FineRich also outperforms SFT → GRPO across all five task metrics.
These comparisons show that standard training methods already benefit from our synthesized data, while the proposed training strategy produces further improvements using the same data. Thus, both the synthesized data and the training method contribute to the final performance.
Regarding the original comparisons, the models in Tables 2 and 5 were evaluated using their released checkpoints or APIs, without additional fine-tuning on our synthesized data. Table 2 provides a benchmark leaderboard of existing models, while Table 5 evaluates whether FineRich’s improvements over its original backbones transfer to external benchmarks. We agree that these comparisons alone do not isolate the contribution of the training strategy, and we will add the above training baselines to clarify this distinction.
2) The comparison models in Tables 2-5 were evaluated using their released checkpoints or APIs, without additional fine-tuning on our synthesized data. But we have to emphasize that the experiments in this paper are fair. On one hand, these tables also serve as the FRBench leaderboard for analyzing the performance of current MLLMs on fine-grained TIU. On the other hand, our goal is to improve the fine-grained TIU capabilities of the base models. Both the novel training method and the training data generated by our synthesis pipeline are contributions of this work. FineRich-4B and FineRich-8B outperform their respective Qwen3-VL-Instruct base models by a large margin and even surpass most large-scale open-source MLLMs (Tables 2–5). To further demonstrate the effectiveness and generalizability of FineRich, we also trained FineRich-27B based on Qwen3.8-27B, achieving an improvement of +12.31 and surpassing Gemini, the strongest closed-source MLLM.