- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper addresses bilingual document-page generation, where long passages must remain legible while satisfying layout and appearance requirements. It introduces DocPage-1M, comprising 969,808 pages from 361,517 PDFs and averaging 1,372 visible characters per page. dots.mocr supplies regional text and boxes, while VLMs describe pictures and global design; refilling these references produces dense captions with content, coordinates, and typographic attributes. SenseNova-U1.5-8B-MoT-Preview is fully fine-tuned using flow matching on 70% document and 30% general data, then optimized with Flow-GRPO and MixGRPO-style sampling. A Gemini VLM assigns three-level regional legibility rewards, and group-relative regional advantages are mapped to spatially corresponding image tokens for local policy updates. On the authors’ 400-page bilingual DocPageBench, SFT+RL with their prompt enhancement reports Text/Layout/Aesthetics scores of 0.8260/0.8686/0.8425 (Avg. 0.8457), versus 0.8585 Avg. for GPT Image 2. BizGenEval Avg.(4) increases from 0.6785 for Base to 0.7068. Further experiments examine SFT initialization, page-level updates, OCR rewards, and best-of-N test-time selection, including RL@1 scoring 0.8480 versus 0.8507 for SFT@64.
Critical strengths
- The regional post-training recipe targets an important failure mode and is supported by multi-faceted analysis.
- The work combines a substantial supervision resource with practical analysis of inference cost.
Critical weaknesses
- The headline comparison with existing generators is confounded by prompt adaptation. In Table 2, SFT+RL uses Our PE, for which Gemini 3.1 Pro rewrites the request, completes missing content, and inserts coordinates when absent (Appendix C.3), whereas every external baseline uses Original requests. This choice is material: under Original inputs, SFT+RL obtains Avg. 0.6483 and Layout 0.7325, rather than 0.8457 and 0.8686 with Our PE. Its Original Layout is below GPT Image 2 (0.7542), Seedream-5.0-Pro (0.7396), and Qwen-Image-3-Pro (0.7384). Base also improves from 0.5845 with Original inputs to 0.6609 with Our PE, independently demonstrating the adapter’s effect. Consequently, the claims of ranking second only to GPT Image 2 and achieving the highest Layout do not establish generator superiority under a common input pipeline. This directly affects the paper’s principal empirical positioning.
- The experiments do not isolate the contribution of spatial credit assignment, and variability is unmeasured. Figure 6 compares the full method with a page-level variant, but Appendix D.3 explicitly notes that the variants also differ in loss reduction, including a window-length factor, and that these effects were not separately ablated. Moreover, Appendices D.1 and E.1 report only one training run per configuration. Thus, the observed 0.5559-versus-0.4570 result supports the complete regional update recipe over one page-level implementation, but does not establish how much benefit comes specifically from mapping advantages to local image tokens—the central algorithmic distinction from prior scalar-reward approaches. Without a mask-only control and an estimate of run variability, the paper’s mechanistic interpretation remains under-supported.
Questions for authors
- Can you evaluate all external baselines with the same saved Our PE captions, or otherwise provide a common prompt-adaptation budget, and report the resulting rankings? If such adaptation is unsuitable for some models, what is the justification for treating Original-versus-Our PE as the main comparison rather than the available Original-versus-Original results?
- Can you provide a control that removes only the spatial masks while holding regional advantages, loss normalization/reduction, window-length factors, and optimization budget fixed? Do the Figure 6 and Figure 7 conclusions persist across multiple training seeds, and what is the observed variability?
- What human or independent-judge validation supports the Gemini L1/L2/L3 legibility labels and DocPageBench’s VLM-based Layout and Aesthetics scores? In particular, does independent evaluation preserve the reported ordering of SFT+RL, SFT, OCR-reward RL, and page-level RL? Appendix E.3 currently acknowledges that the shared-judge analysis does not establish human preferences.
- What are the source collections and licenses for the 361,517 PDF files, what screening was performed for personal or sensitive content, and do the authors have redistribution rights for the page images and derived captions used in DocPage-1M and DocPageBench?
3 comments
Response Part 2
Q3: Do human or independent judgments validate the reported quality gains and regional legibility grades?
A: To validate aesthetics and legibility beyond the training judge, we added blinded evaluations of the aesthetic improvement from SFT to RL, text-rendering quality in two key ablations, and agreement with the L1/L2/L3 regional legibility grades. Each task contains 50 samples, each independently assessed by three annotators. The two ablation comparisons use models trained for 200 updates.
| Evaluation | Samples | Result |
|---|---|---|
| SFT+RL vs SFT: page aesthetics | 50 pairs | Preference for SFT+RL: 73% [61%, 84%] |
| Spatial feedback vs uniform page feedback: text quality | 50 pairs | Preference for spatial feedback: 73% [61%, 84%] |
| Regional VLM reward vs regional OCR reward: text quality | 50 pairs | Preference for VLM reward: 86% [76%, 94%] |
| Regional grades vs annotator consensus | 50 regions | Exact agreement: 72%; weighted κ = 0.669 |
Preference scores use consensus judgments, counting ties as one half; brackets indicate 95% confidence intervals. Regional consensus is the median of the three grades.
Our findings:
- Independent judgments preserve the direction of the key comparisons. They favor RL over SFT in page aesthetics, spatially corresponding feedback over uniform page feedback, and VLM rewards over OCR rewards in text-rendering quality, supporting the main quality gains and the two ablation conclusions.
- Independent annotations support the grading used in our legibility reward. Agreement with the original regional grades reaches 72%, with a weighted κ of 0.669. Only 1 of 50 regions (2%) exhibits a two-level disagreement between L1 and L3, providing additional support for the regional quality assessments underlying our reward.
We will include these evaluations and the annotation protocol in the revised manuscript.
Q4: What are the data sources and licenses, and how are screening and redistribution handled?
A: Our PDFs were collected from publicly accessible datasets and web sources for research and have been filtered through a sophisticated automated pipeline. We plan to release the model weights. Before publicly releasing the page images and derived captions from DocPage-1M and DocPageBench, we will conduct another round of rigorous screening and manual review and verify redistribution permissions to ensure that the released materials contain no personal or sensitive information and comply with the applicable licensing terms. We will document the main source collections, applicable licenses, and specific release scope in the revised manuscript's data statement.
Following Up on Our Response
Dear Reviewer Wu5L,
Thank you again for your careful assessment and helpful suggestions. We hope our response addresses the points you raised and would be grateful to hear whether any substantial issues remain.
Best regards,
The Authors
Response Part 1
We thank the reviewer for the careful assessment and for recognizing the value of the regional post-training approach and its supporting analyses.
Q1: How do all external baselines rank when evaluated with the same PE descriptions?
A: We have now evaluated all eight external baselines under the same PE inputs. Their outputs were regenerated using exactly the same 400 saved PE descriptions as our model, giving all models the same text and layout information.
Each metric is averaged over the valid samples for each model.
Our findings:
We will add this comparison and explicitly state that the ranking holds under the shared PE condition.
Q2: Does a matched control isolate spatial feedback, and do the ablation conclusions hold across training seeds?
A: To isolate the contribution of spatial feedback, the new control makes only one change to the optimization: regional advantages are pooled into a single value per page and applied to the same text regions. The SFT initialization, regional reward computation, text-region weighting, loss normalization and reduction, window-length factor, and 200-update training budget are all kept identical. Both variants are evaluated on the same 256 validation pages.
Both settings cover the same 1,273 target regions, with three complete ratings per region. L1 and L3 denote the lowest and highest legibility tiers, respectively.
To assess training variability, we also repeated three configurations—regional VLM feedback, uniform VLM feedback within each page, and regional OCR feedback—with three training seeds (42, 123, and 2026) per configuration. For this comparison, we evaluate all RL runs at 100 updates on the same 256-page validation set.
RL results are mean ± sample standard deviation across three training seeds; SFT is a single shared reference. Each run covers the same 1,273 target regions, with no missing VLM ratings.
Our findings:
We will add the controlled comparison, the means and standard deviations, and the per-seed results to the revised manuscript.