- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Generation-aware Layout Analysis (GLA), a task that recovers text, per-element allocated regions (the space available for replacement content rather than visible text bounds), semantic categories, and alignment from raster document images, as spatial constraints for element-level editing and translation. The authors build a 37K-image HTML-rendered training set across nine domains and curate GLA-Bench, a 1,000-image human-verified benchmark with four diagnostic tasks. They train 4B VLM parsers with multi-task SFT, propose EAA-GRPO (assigning element-level advantages by comparing the same element across rollouts), and accelerate decoding with MTP and task-adapted DFlash. The 4B parser reaches 46.59 [email protected] on full parsing versus 6.26 for the best zero-shot baseline; layout-only RL transfers to full parsing on both backbones; task-adapted drafting reaches 2.18× autoregressive throughput.
Critical strengths
- Well-motivated and clearly formulated task (Sec. 1, Fig. 1). The visible-bounds vs. allocated-region distinction is concrete and practically relevant; the representation is independently measurable, decoupled from full document derendering (Sec. 2).
- Solid dataset engineering: a hierarchical, cue-based annotation policy (Sec. 3.1), human correction with cross-checking and adjudication for the benchmark (App. B.4), and a complete category/alignment vocabulary (App. B.3).
- EAA is well-motivated: image-level GRPO aggregation can obscure element-level differences in dense outputs, and the across-rollout comparison is cleanly differentiated from MCR-GRPO's within-response normalization (Sec. 2, Sec. 4.2, Fig. 5).
- Broad evaluation: four complementary tasks with strict joint metrics (Sec. 3.3), plus a blinded 20-reviewer human arena with bootstrap CIs whose rankings agree with benchmark rankings (Table 8).
- Good engineering transparency: hardware, hyperparameters, and measurement boundaries are documented (App. C.1), including the differing timing protocols for MTP and DFlash.
Critical weaknesses
- The benchmark's annotation quality is not yet established. Sec. 3.2 states "We plan an independent agreement assessment" — i.e., no IAA exists at submission time. Since allocated regions are admittedly ambiguous and resolved by a conservative heuristic policy, IAA is essential to interpret what any [email protected] score means. Relatedly, App. C.3 defines Fleiss' κ, P_o, U, Spearman's ρ and Kendall's τ but never reports their values. Both are fixable and I ask the authors to provide them in the rebuttal.
- All external baselines are zero-shot general-purpose VLMs (Table 2). The headline "46.59 vs 6.26" mostly reflects task-specific adaptation (the 4B base models score 0.00 Joint with CER far above 100%, indicating format failure). Missing: (a) an equally sized VLM fine-tuned on the same data, to separate data vs. method contributions; (b) a pipeline baseline (OCR + layout detector + heuristics). dots.ocr is cited but never evaluated.
- No end-to-end validation of the motivating applications. The paper's premise is that allocated regions help editing/translation, but no experiment feeds GLA outputs into an actual editing or translation pipeline to compare against visible-bounds constraints. Current evidence shows the representation is learnable and measurable, not that it helps generation.
- Evidence for EAA-GRPO is thinner than its narrative weight. The benchmark ablation covers a single backbone (Table 4), the gain over GRPO is +1.05 [email protected] without any variance across seeds, Fig. 3 uses only 64 validation images, and the GRPO row for Qwen3-VL-4B is missing. Key design choices (ε_box=0.05, the 0.95 trusted threshold, η=0.5) are not ablated.
- Table 3's interpretation is confounded: after SFT on allocated-region targets, the same model is compared against two different reference types, so the crossover mixes training-target alignment with task difficulty. It demonstrates that explicit supervision works, but only indirectly that spatial allocation is intrinsically learnable.
- The 867/934 element-level subset sampling is unspecified; no per-domain breakdown; MTP (Table 5) uses [email protected] and a different timing protocol than DFlash (Table 6); related work on layout-aware RL for document parsing (e.g., Infinity Parser, arXiv:2506.03197) and speculative decoding for document parsing VLMs (e.g., HSD, arXiv:2602.12957) is not discussed — please verify and comment.
Questions for authors
- Can you provide preliminary IAA for allocated regions (e.g., IoU statistics on a doubly-annotated subset) and category/alignment agreement, plus the human-agreement ceiling on [email protected]?
- Please report the Fleiss' , , U, Spearman's ρ and Kendall's defined in App. C.3.
- How were the 867 bbox2text and 934 text2bbox instances sampled from the 19,858 elements?
- Can you add a same-data, same-scale SFT baseline on a different backbone (or a pipeline baseline) to isolate the contribution of the training data vs. the framework?
- Can you report GRPO (without EAA) on Qwen3-VL-4B, and mean±std over at least 2–3 seeds for Table 4?
- Why is RL restricted to img2layout? Does adding img2full RL (with a text reward) help or hurt?
- Can you provide a small end-to-end comparison (allocated region vs. visible bounds as constraints in a text-replacement or translation pipeline)?
- Does the 2.18× DFlash speedup hold under the MTP timing protocol (including initialization), and under higher concurrency?
7 comments
Response to Reviewer RWTM (Part 2/7)
W2 & Q4 — Separating data and method contributions (1/2)
(1) The reason why the base models have zero Joint scores with normal CER.
As explained in Appendix C.4, CER measures transcription given a target box, whereas full-image [email protected] requires the same element to satisfy IoU ≥ .75, correct category and alignment, and exact normalized text matching simultaneously. The zero Joint scores arise from incorrect category predictions rather than the format failure: the unadapted models have 0.00 category F1, preventing elements from contributing to Joint. In Fig. 15, for example, the prediction is valid JSON, has IoU 0.898, and predicts the correct alignment, but labels a Text element as affiliation. Thus, incorrect field values can yield zero Joint even when the output is parseable.
(2) Contribution of task-specific data.
The base-versus-SFT comparisons on the two 4B backbones show the gains from supervised task adaptation, with the same four-task metrics as the main table:
| Model | bbox2text CER ↓ | bbox2text EM ↑ | text2bbox mIoU ↑ | text2bbox mAcc ↑ | img2layout mF1 ↑ | img2layout [email protected] ↑ | img2full mF1 ↑ | img2full Cat.F1 ↑ | img2full Align. ↑ | img2full [email protected] ↑ |
|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3-VL-4B | 68.71 | 61.94 | 47.15 | 24.24 | 11.25 | 0.00 | 11.16 | 0.00 | 57.69 | 0.00 |
| Qwen3-VL-4B + SFT | 10.00 | 87.77 | 73.13 | 59.76 | 51.82 | 43.70 | 50.94 | 79.97 | 91.27 | 43.96 |
| Qwen3.5-4B | 127.64 | 48.10 | 33.21 | 14.05 | 7.84 | 0.00 | 0.17 | 0.00 | 48.86 | 0.00 |
| Qwen3.5-4B + SFT | 13.71 | 86.39 | 66.05 | 47.47 | 40.41 | 33.88 | 38.95 | 78.64 | 93.02 | 32.93 |
SFT increases img2full mF1 by 39.78 points on Qwen3-VL-4B and 38.78 points on Qwen3.5-4B, and raises [email protected] from 0.00 to 43.96 and 32.93, respectively. On text2bbox, allocated-region mIoU increases by 25.98 and 32.84 points. On bbox2text, EM increases by 25.83 and 38.29 points. These comparisons establish the contribution of task-specific supervision on both backbones.
(3) Contribution of EAA.
Comparing EAA-GRPO with standard GRPO under the same reward function, SFT initialization, and training data isolates the contribution of element-level advantage assignment. We report the complete four-task comparison as means ± standard deviations across three seeds:
| Backbone | Method after SFT | bbox2text CER ↓ | bbox2text EM ↑ | text2bbox mIoU ↑ | text2bbox mAcc ↑ | img2layout mF1 ↑ | img2layout [email protected] ↑ | img2full mF1 ↑ | img2full Cat.F1 ↑ | img2full Align. ↑ | img2full [email protected] ↑ |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3.5-4B | GRPO | 12.74 ± 1.63 | 85.97 ± 1.26 | 66.98 ± 0.29 | 48.84 ± 0.37 | 43.10 ± 0.28 | 37.08 ± 0.51 | 41.84 ± 0.24 | 80.79 ± 0.41 | 93.39 ± 0.06 | 36.03 ± 0.44 |
| Qwen3.5-4B | EAA-GRPO | 11.95 ± 1.35 | 86.74 ± 1.17 | 67.25 ± 0.22 | 49.39 ± 0.36 | 44.69 ± 0.21 | 38.16 ± 0.37 | 43.20 ± 0.17 | 82.43 ± 0.38 | 93.52 ± 0.08 | 37.21 ± 0.39 |
| Qwen3-VL-4B | GRPO | 9.93 ± 0.95 | 88.67 ± 0.82 | 73.51 ± 0.33 | 60.31 ± 0.39 | 52.86 ± 0.51 | 45.12 ± 0.71 | 51.65 ± 0.46 | 80.34 ± 0.34 | 91.40 ± 0.15 | 45.20 ± 0.46 |
| Qwen3-VL-4B | EAA-GRPO | 9.89 ± 1.07 | 88.92 ± 0.99 | 73.75 ± 0.30 | 60.59 ± 0.43 | 55.53 ± 0.32 | 47.07 ± 0.68 | 54.13 ± 0.23 | 80.66 ± 0.32 | 92.05 ± 0.13 | 46.88 ± 0.52 |
On Qwen3.5-4B, EAA improves img2layout mF1 and [email protected] by 1.59 and 1.08 points, and img2full mF1 and [email protected] by 1.36 and 1.18 points. On Qwen3-VL-4B, the corresponding gains are 2.67 and 1.95 points for img2layout and 2.48 and 1.68 points for img2full. The regional task means also improve on both backbones. These controlled comparisons distinguish EAA's additional gains from those due to supervised task adaptation. We have added the complete comparison to Appendix C.10 (Table 16).
Response to Reviewer RWTM (Part 3/7)
W2 & Q4 — Separating data and method contributions (2/2)
(4) Same-data document-parsing baseline.
Thank you for the suggestion. We include SFT controls for two related models: dots.ocr, which targets document layout and content parsing, and LocateAnything, which provides regional localization and partial parsing capabilities. We fine-tune both on the same GLA training data, four-task mixture, and output schema as our Qwen-based parsers, and evaluate them before and after SFT using the same benchmark protocol. The metric order follows Table 2 of the paper.
| Model | bbox2text CER ↓ | bbox2text EM ↑ | text2bbox mIoU ↑ | text2bbox mAcc ↑ | img2layout mF1 ↑ | img2layout [email protected] ↑ | img2full mF1 ↑ | img2full Cat.F1 ↑ | img2full Align. ↑ | img2full [email protected] ↑ |
|---|---|---|---|---|---|---|---|---|---|---|
| dots.ocr | 34.55 | 60.21 | 0.00 | 0.00 | 12.84 | 0.00 | 13.20 | 14.66 | 0.00 | 0.00 |
| dots.ocr + SFT | 38.62 | 59.86 | 40.44 | 20.49 | 14.23 | 9.43 | 14.19 | 24.48 | 92.82 | 10.17 |
| LocateAnything | 138.80 | 44.52 | 54.98 | 30.12 | 10.35 | 0.00 | 9.71 | 0.10 | 0.00 | 0.00 |
| LocateAnything + SFT | 27.41 | 66.78 | 66.80 | 48.04 | 26.73 | 14.89 | 25.33 | 51.24 | 90.27 | 10.31 |
SFT increases img2full mF1 from 13.20% to 14.19% for dots.ocr and from 9.71% to 25.33% for LocateAnything, gains of 0.99 and 15.62 percentage points, respectively. Their [email protected] scores increase from 0.00% to 10.17% and 10.31%. On text2bbox, mIoU increases from 0.00% to 40.44% for dots.ocr and from 54.98% to 66.80% for LocateAnything.
Native dots.ocr targets full-page document parsing, whereas text2bbox asks it to return the region corresponding to a specified text query. Under the current query protocol, its responses do not provide valid boxes accepted by the evaluation interface, giving 0.00 for both mIoU and mAcc. Its native output also lacks the alignment field required by GLA, so Align. and [email protected] are 0.00. SFT adapts dots.ocr to the GLA query and output schema, yielding text2bbox mIoU and mAcc of 40.44% and 20.49% and img2full alignment accuracy of 92.82%. The gains from initially zero scores reflect improved instruction following and output-schema compliance, enabling the model to produce valid GLA-structured predictions, rather than improvements in localization alone. These results demonstrate the value of task-interface adaptation. We have added the settings, four-task results, and explanation of the zero scores to Appendix C.7, “SFT Adaptation of Related Models” (Table 13).
[1] LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding
[2]dots.ocr: Multilingual Document Layout Parsing in a Single Vision-Language Model
Response to Reviewer RWTM (Part 1/7)
We thank you for your positive assessment of our task, dataset, EAA design, and evaluation. Please kindly find our response to your comments below. Additionally, all modifications to the manuscript have been highlighted in blue for easy reference. Please feel free to let us know if you have any additional concerns or questions.
W1 & Q1 — Annotation agreement and the human Joint reference
We randomly sample 100 images from the nine domains of GLA-Bench, containing 1,621 elements, for independent double annotation. We report boundary IoU, category agreement, and alignment agreement, grouped by explicit containers versus regions inferred from structure, whitespace, or neighboring elements. We also compute inter-annotator layout [email protected] under the same evaluation protocol as a human-agreement reference.
The overall human layout [email protected] is 79.73%, with 92.79% for explicitly constrained regions and 73.30% for inferred regions. Boundary agreement is higher when explicit constraints are available, while category and alignment agreement remain high in both groups.
We have added the protocol and agreement results to Appendix B.4, “Human Annotation and Quality Control” (Table 8), and summarized the main findings in Section 3.2.
W1 & Q2 — Human-study agreement and ranking correlations
Thank you for pointing out the missing statistics in Appendix C.3. Using the three initial judgments for each comparison, before discussion or adjudication, we compute Fleiss’ κ, pairwise observed agreement Po, and unanimity rate U for both arenas. We also compare the human preference rankings with benchmark rankings based on [email protected] on the shared set of 100 images.
The human and benchmark rankings agree completely in both arenas, with Spearman’s ρs and Kendall’s τb both equal to 1.000. We have added these statistics in Table 11, alongside the human rankings in Table 10, in Appendix C.3.