acceptodds
Withdrawn from ICLR 2027

Rendering as Supervision: Exact-Geometry Synthetic Documents for Grounded Document Understanding

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Production document AI can read but cannot _point_: models answer questions yet emit no dependable evidence location, blocking auditable extraction and human review. On our probe, open vision–language bases at every scale (0.8B–9B) produce no parseable box at all, and the strongest frontier model is coordinate-frame-unreliable (0.157 [email protected] as instructed; 0.892 only under an oracle frame) — while the same models read text from _given_ regions at up to 0.86 hit rate. The missing box supervision is unaffordable to annotate and unreliable to OCR precisely where degradation strikes. Our position: _when you synthesize the degradation, you own its geometry_. Our engine persists the exact, invertible correspondence field of every degraded view (882 B/page, bit-reproducible), turning 126K rendered pages into 233K zero-error grounding labels. A dosed two-stage recipe confers coordinate emission on a 4B model (0.821 mIoU, generalizing across document classes, model families, and languages), improves real-document extraction by +14.8 exact-EM, and regresses no tracked public benchmark. Controlled studies then map _how_ label quality matters: i.i.d. label noise at OCR grade costs 3–11 mIoU (monotone in both domains; 3 seeds synthetic), whereas an equal-dose systematic bias is learned and fully correctable (0.526 raw, 0.841 after un-shifting — the exact-label anchor): exactness pays against the error structure OCR produces. A token-level equivariance loss on the same field fails for a measured reason: the warp is sub-resolution for the vision tokenizer.

Primary area:
datasets and benchmarks
Keywords:
document understanding, synthetic data, visual grounding, vision-language models, label noise, data-centric AI

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Q4dAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a synthetic document generation framework that preserves exact geometric correspondences during document rendering and degradation. By tracking geometric transformations, the proposed pipeline generates accurate grounding annotations without relying on OCR-based pseudo-labeling. The authors construct a large-scale synthetic dataset and introduce a two-stage training strategy to improve document grounding and extraction. Experiments investigate grounding accuracy, label noise, catastrophic forgetting, and cross-domain generalization.

Critical strengths
  1. The paper addresses a practically relevant problem in document understanding, particularly the generation of reliable grounding annotations without expensive manual labeling.

  2. The synthetic data generation pipeline is carefully engineered, and the analysis of different label noise patterns provides useful empirical observations.

  3. The experiments cover several model scales, training configurations, and evaluation benchmarks, providing a relatively comprehensive empirical investigation.

Critical weaknesses
  1. Limited methodological novelty. The main idea of transporting annotations through known geometric transformations is closely related to established synthetic data generation and geometric supervision techniques. More importantly, the homography-only baseline achieves comparable or slightly better grounding performance than the proposed full correspondence field (0.840 vs. 0.838 mIoU), raising questions about the necessity of the central technical contribution.

  2. The improvements appear strongly dependent on output-format adaptation. The 500-sample experiment already achieves 0.767 mIoU, suggesting that much of the improvement comes from teaching the model to produce valid coordinates rather than learning substantially stronger visual grounding representations. More controlled comparisons are needed to isolate these effects.

  3. The practical benefits for document understanding remain insufficiently demonstrated. Although the method improves exact-match accuracy on real documents, the semantic correctness analysis suggests that much of this gain results from output formatting rather than improved understanding. Furthermore, the pure task-QA baseline achieves higher exact-match accuracy than the final proposed model (0.738 vs. 0.697), indicating a trade-off between grounding and extraction performance.

  4. The generalization evidence is not sufficiently convincing. The main real-document evaluation relies on relatively small, non-public subsets, limiting independent verification. Meanwhile, improvements on several public benchmarks involve replaying their training data, making it difficult to attribute the gains to the proposed synthetic supervision. Additional evaluation on independently held-out real-world benchmarks would strengthen the conclusions.

  5. The experimental comparisons require further investigation. The comparison between exact labels and OCR-generated pseudo-labels is affected by differences in retained training samples, which may confound the effects of label accuracy and data quantity. In addition, the relatively low accuracy at strict localization thresholds ([email protected] = 0.255) indicates that precise evidence localization remains challenging.

Questions for authors
  1. Given that homography-only supervision performs comparably to the full correspondence field, what is the practical advantage of explicitly maintaining dense non-rigid geometric correspondences?

  2. Can the authors provide controlled experiments that disentangle improvements from coordinate-format learning, annotation quality, and training-data quantity?

2 comments
AuthorsOpenReview ↗

Response to Reviewer Q4dA

We thank the reviewer for identifying the confounds in the submitted experiments. The new controls changed several of our conclusions.

1. The dense field is unnecessary at our production dose

The homography-only result is a genuine negative result for the central representation. At the production crumple amplitude, homography-only supervision reaches 0.840 mIoU and the full field reaches 0.838. We no longer claim that the dense component always improves training.

We re-rendered matched page sets at larger amplitudes to measure when it does help.

Crumple amplitudeMedian shiftFull fieldHomography only
0.4–1.2% of short side8.7 px0.838 ± 0.0040.840 ± 0.001
1.5–2.0%16.2 px0.6960.675
3.0–5.0%37.3 px0.683 ± 0.0040.614 ± 0.006

Each pair uses the same source pages and training-item count within its row. Only the crumple correction applied to the training boxes differs. The production and largest doses use three seeds; the middle dose uses two. Different rows use different corpora and test slices, so only within-row arm differences are comparable. At the middle dose, the full field improves [email protected] by 0.055, but 81 test items cannot resolve the stricter-threshold differences. At the largest dose, three seeds show consistent gains at all three thresholds.

The practical advantage is therefore conditional. The dividing region is on the order of a 10 px median shift, not a universal requirement for dense geometry. Persisting the field also supplies the instrument needed to measure this boundary for a new rendering setting.

2. Format learning, annotation quality, and sample count are separated

We added two controls for the requested disentanglement.

The format-versus-precision control compares training-set sizes at several IoU thresholds. With 500 exact labels, the model reaches 0.928 [email protected] but only 0.187 [email protected]. With 20K labels, these values become 0.968 and 0.313. Most of the loose-threshold improvement is indeed obtained early. The larger corpus mainly buys stricter geometric precision.

The annotation-quality control uses exactly the 24,287 image–question pairs retained by the OCR pipeline in both arms.

Training labelsItemsmIoU[email protected]
Exact24,2870.840 ± 0.0020.881
OCR pseudo-labels24,2870.751 ± 0.0020.633
Exact, full set30,0000.838 ± 0.0040.871

Dropping 19% of the exact-label samples has no measurable effect. The 0.089 mIoU difference between the matched arms is therefore attributable to label quality rather than sample retention.

3. Independent real-domain evaluation

We added public, third-party human-annotated XFUND-zh and FUNSD tasks, with 150 items per task and no evaluation-image overlap with training.

Domain and taskBaseOurs
XFUND-zh ref2box, [email protected]0.0270.853 ± 0.000 (3 seeds)
XFUND-zh answer-with-box, [email protected]0.0130.813 (seed 42)
XFUND-zh box-to-text, hit0.3270.933 (seed 42)
FUNSD ref2box, [email protected]0.0400.507 ± 0.009 (3 seeds)
FUNSD answer-with-box, [email protected]0.0070.400 (seed 42)
FUNSD box-to-text, hit0.2800.713 (seed 42)

Ref2box uses three training seeds; the other rows are seed-42 probes. The ± values are seed standard deviations, not sampling confidence intervals.

No FUNSD evaluation image occurs in training, and this evaluation uses no public replay from either target dataset. The final recipe does contain unrelated English DocVQA and InfographicVQA QA replay, but no English box annotations. We therefore claim cross-language grounding transfer, not an English-free training set.

By contrast, synthetic-only training does not improve the public QA benchmarks. We now state this directly. Final-recipe gains on DocVQA, InfographicVQA, and DocVQA-degraded use their training data as replay and are labeled replay repair rather than synthetic transfer.

4. Strict precision and the QA trade-off remain limitations

We agree that 0.255 [email protected] is not sufficient for high-precision automated redaction. The real-domain results are lower still: 0.180 on XFUND-zh and 0.016 on FUNSD. These numbers now appear in the main limitations.

We also agree that the pure-QA arm exposes a trade-off. It reaches 0.738 exact-EM, compared with 0.697 for the final model, but emits no coordinates on any of the 278 ref2box queries. The unified model gives up 4.1 exact-EM points to provide grounding. We no longer describe it as an unqualified reading improvement.

Finally, prompt alignment accounts for 87% of the original +14.8 exact-EM difference. The prompt-aligned base reaches 0.678, leaving a 1.9-point difference to the final model. The revised abstract, results, and conclusion withdraw the unqualified headline.

I appreciate the author’s time and response, which have addressed my concerns regarding label quality, data volume, and the experiments conducted on real-world datasets. However, I still believe this paper lacks innovation, and the actual improvement in document understanding capabilities remains limited. Furthermore, the quality of the paper does not meet the standards required for ICLR. Therefore, I am maintaining my score.

Reviewer_JtDUOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a synthetic document generation and rendering engine that pairs degraded document images (e.g., scan, photo, print) with exact geometric correspondence fields (Φ\Phi) to automatically generate noise-free grounding labels. Based on this pipeline, the authors construct a large-scale synthetic dataset containing QA and grounding pairs, and propose a two-stage fine-tuning strategy to improve visual document understanding and grounding capabilities in VLMs. Extensive empirical studies are conducted to analyze label noise, replay strategies, and model scaling.

Critical strengths
  • Practical Synthetic Framework and Systematic Evaluation: The paper proposes a practical pipeline that leverages rendering geometry to extract exact grounding labels, supported by controlled experiments that examine how different label noise types (e.g., Gaussian noise vs. systematic offset) affect grounding performance.
Critical weaknesses
  • Lack of Novel Insights and Predictable Findings: Using rendering geometry for exact supervision is an established practice in synthetic document generation, and this work offers limited fresh technical insight. While the paper includes numerous controlled experiments, many of the takeaways (e.g., clean labels outperform noisy ones, or models handle systematic offsets better than random Gaussian noise) are expected and largely intuitive.

  • Weak Real-World Transfer with Unproven Benefits for QA and Grounding: The experiments fall short of proving the actual value of the synthetic data on real-world downstream tasks. On QA benchmarks, fine-tuning purely on synthetic data causes performance drops relative to the base model (gains only show up after adding public real-data replays). On grounding tasks, the paper lacks systematic evaluation on standard human-annotated datasets (e.g., FUNSD/CORD), relying mostly on in-distribution synthetic test splits. It remains unclear whether the synthetic dataset brings tangible benefits to real-world QA or grounding capabilities.

  • Inflexible Baselines and In-Distribution Bias: The claim that zero-shot base models fail completely at grounding rests on a single rigid prompt format, ignoring native grounding prompt syntaxes supported by modern VLMs (e.g., Qwen-VL series). This makes the baseline comparison less convincing.

Questions for authors
  • Efficacy of Synthetic Data on Real Tasks: Can you present the performance of models trained exclusively on synthetic data across standard real-world QA benchmarks (without public replay) and human-annotated grounding benchmarks (e.g., FUNSD/CORD) to demonstrate direct positive transfer?

  • Prompt Alignment for Baselines: Can you evaluate zero-shot base models using their native, standard grounding prompt formats to ensure a fair comparison?

2 comments
AuthorsOpenReview ↗

Response to Reviewer JtDU (1/2)

We thank the reviewer for separating two questions that our submitted version conflated: whether synthetic supervision improves real-document QA, and whether it improves grounding on real documents. The new experiments give different answers to these questions.

1. Synthetic-only training does not improve the public QA benchmarks

The reviewer is correct about the submitted results. We therefore do not claim direct positive QA transfer. The replay-free comparison is:

Public benchmarkBaseSynthetic-only
DocVQA, lenient EM0.9180.908
InfographicVQA, lenient EM0.7580.720
XFUND-zh KIE, value F10.6400.621
OCRBenchV2, lenient EM0.3790.370
DocVQA-degraded, lenient EM0.8900.883

The final recipe adds DocVQA and InfographicVQA training data as replay. Its gains on those benchmarks, and on the derived DocVQA-degraded set, cannot be attributed to synthetic supervision. We now label them as replay repair in the table caption and text. In particular, we withdraw any suggestion that synthetic-only training improves robustness on the public degraded-QA set.

2. Grounding does transfer to independently annotated real forms

We added the standard human-annotated evaluation requested by the reviewer. These are not new annotations produced by us. The boxes and text come from the original XFUND and FUNSD annotations.

Domain and taskBaseOurs
XFUND-zh ref2box, [email protected]0.0270.853 ± 0.000 (3 seeds)
XFUND-zh answer-with-box, [email protected]0.0130.813 (seed 42)
XFUND-zh box-to-text, hit0.3270.933 (seed 42)
FUNSD ref2box, [email protected]0.0400.507 ± 0.009 (3 seeds)
FUNSD answer-with-box, [email protected]0.0070.400 (seed 42)
FUNSD box-to-text, hit0.2800.713 (seed 42)

Each task contains 150 items. Ref2box uses three training seeds; the other rows are explicitly marked as seed-42 probes. None of the 50 XFUND evaluation images occurs in the checked pool of 33,866 training images, and no FUNSD evaluation image occurs in training. The final recipe does include unrelated English DocVQA and InfographicVQA QA replay, but no English box annotations; we therefore claim cross-language grounding transfer, not an English-free training set. The reported real-form comparisons have zero truncated generations.

This evidence supports grounding transfer, not a general QA improvement. The remaining limitation is substantial: [email protected] is only 0.180 on XFUND-zh and 0.016 on FUNSD. We state this in the main text rather than treating [email protected] as solved precision.

3. Native prompts change the zero-shot conclusion

We agree that the original baseline was unfair. Every open model in the revised Table 3 now uses its native grounding prompt and a 2,048-token budget. The union parser accepts seven box formats and two point formats. It evaluates both pixel and norm-1000 coordinates and retains the better result for each answer.

Under this protocol, Qwen3.5-9B reaches 0.763 [email protected] on the synthetic split. Qwen3-VL-4B emits a valid bbox_2d object for all 278 queries. We withdraw the claim that zero-shot bases cannot emit coordinates.

The defensible result is about precision. The 9B base reaches only 0.065 [email protected]. The Qwen3.5-4B base reaches 0.027 and 0.040 [email protected] on XFUND-zh and FUNSD, respectively, even under the permissive parser. Our model reaches 0.853 and 0.507 on those tasks.

4. What is new beyond "clean labels are better"

We agree that this statement alone is not a useful finding. The revised paper instead makes three specific measurements.

First, output-format learning and geometric precision have different dose curves. With 500 exact labels, [email protected] is already 0.928 while [email protected] is 0.187. With 20K labels, they are 0.968 and 0.313. A small dataset teaches the interface; additional exact labels mainly improve the geometry.

Second, the need for a dense non-rigid field depends on deformation dose. Homography-only supervision ties the full field at our production setting (0.840 against 0.838 mIoU). At the largest tested crumple dose, the full field improves mIoU by 0.069 over three seeds. The middle dose improves [email protected], but its strict-threshold comparison is underpowered. This is a boundary on when the central representation matters, not a claim that it always helps.

Third, we pre-registered predictions before training two interpolation or extrapolation arms and a heavy-tailed noise arm. The sigma=15 predictions hit both decision bands. The sigma=7 predictions miss both bands. The heavy-tailed arm has median error 2.9 px and 95th-percentile error 177 px; it reaches 0.835 mIoU, close to 0.838 with exact labels. This rejects the registered p95-based prediction of 0.593. The result is 0.005 above the median-based decision band, so it is a near-miss rather than a formal hit.

AuthorsOpenReview ↗

Response to Reviewer JtDU (2/2)

The practical conclusion is limited to content-independent random noise. Real OCR errors are structured by layout and content, and real OCR labels perform worse than random noise with a similar median and tail. We now use the random-noise curve as a lower bound, not as a point prediction for OCR.

These changes narrow the paper's contribution. We no longer present a general reading improvement or a universal need for dense geometry. The evidence supports precise evidence grounding and a way to measure the quality of geometric supervision.

Reviewer_h5nwOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper presents a document-synthesis engine that renders pages and degrades them through five production-calibrated channels, persisting an exact correspondence field for each degraded view. Using the synthesized dataset, the authors fine-tuned existing models (Qwen3.5-4B and two other families). They reported results on different training stages, unreleased internal datasets and public datasets. This is a work with a sufficient amount of work. But the overall writing is difficult to follow, and the results do not support, or even contradict, the claimed contributions.

Important Note: The reviewer noticed an excessive trace of AI-assisted writing in this paper (the authors disclose LLM use for drafting and revising the text): too many “A: B” label-colon constructions (“Variance: …”, “Bias: …”, “Mechanism.”); too many over-compressed sentences that are difficult for a human to understand (e.g. “The σ-study shows the floor itself presupposes label exactness”, Sec. 5.2); and too many undefined coined terms (“format-unlocking floor”, “spending the field”, “label debt”). I might be wrong about the cause, but the effect on readability is real, and I consider this a serious problem of writing attitude. I would ask the AC to carefully consider this, even though I rated this paper with weak rejection.

Critical strengths

The data synthesis pipeline is clearly described and the engineering is thorough (exact correspondence field persisted per view, bit-reproducible generation, engine and corpus promised for release).

Critical weaknesses

The authors claimed that “when you synthesize the degradation, you own its geometry.” It is an interesting claim. But the later results hardly support it.

  1. In Section 5.2, the authors conclude “No zero-shot base at any tested scale produces parseable boxes under our protocol”. After checking Table 3, the results are quite strange. According to Appendix A.2, the format the authors use in the prompt is “(x1,y1),(x2,y2)”, and the parser accepts only three fixed formats. However, Qwen models are trained to output grounding as JSON (“bbox_2d”). Using a self-defined output format is very likely to produce unparseable output; Table 3 itself reports that Qwen2.5-VL-7B emitted points (102/278). So the zero-shot result is not trustworthy. The improvement looks like it comes from learning a new output format: the same Qwen3-VL-4B that outputs no coordinates zero-shot reaches 0.841 after SFT on 20K items (Table 3), and 500 samples already reach 0.767 (Table 5).
  2. Section 5.3 shows the same thing. The authors put the containment result in the paragraph instead of a table. Checking exact-EM, it improves (0.549 → 0.697). But checking containment, it decreases (0.824 → 0.736), and the LLM-judge semantic correctness does not improve (88/93 → 82/90). The pure-QA arm without grounding (0.738, Table 5) is even higher than the final recipe. It looks like, after fine-tuning on the synthetic dataset, the format-alignment capability is improved but the actual capability is not. The authors themselves write that the recipe “buys an extraction interface … not a wholesale reading upgrade”, yet the abstract reports “+14.8 exact-EM” without this qualification.
  3. Section 5.4 shows the same result: training on the degraded dataset does not provide robustness.
Questions for authors

See Weakness. I think those are critical.

2 comments
AuthorsOpenReview ↗

Response to Reviewer h5nw

We thank the reviewer for checking the evaluation protocol rather than accepting the reported zeros. The reviewer was right about the central problem. Re-measurement also showed that the prompt-alignment issue is larger than the review estimated.

1. The zero-shot baselines were not trustworthy

Our original parser accepted too few formats. We have now evaluated every open base with its native grounding prompt, a 2,048-token generation budget, and a union parser. The parser accepts seven box formats and two point formats. It also reads both pixel and norm-1000 coordinates and keeps the better result for each answer. This oracle-frame rule deliberately gives the base models the benefit of the doubt.

The submitted claim that the bases produce no parseable boxes is withdrawn. A 9B base reaches 0.763 [email protected] on our synthetic split. Qwen3-VL-4B emits a valid bbox_2d object for all 278 queries with no parse failures.

The corrected results still show a substantial precision problem. The 9B base reaches only 0.065 [email protected], while our final model reaches 0.255 ± 0.009. Qwen3-VL-4B reaches zero at all three reported IoU thresholds despite its perfectly parseable output. Thus the revised claim concerns box precision, not whether a model can emit coordinates.

We also tested whether 500 samples merely teach the output format. They already produce 0.928 [email protected] but only 0.187 [email protected]. Increasing the exact training set to 20K leaves [email protected] nearly saturated at 0.968 while raising [email protected] to 0.313. Format learning and strict-threshold precision therefore behave differently.

2. We withdraw the unqualified +14.8 exact-EM claim

The reviewer correctly identified output alignment as the explanation for most of this number. We measured it directly on the same 740 items.

Base conditionexact-EMcontainment
Original prompt0.5490.853
Request the extracted span only0.6610.881
Three short-answer examples0.6780.884
Our final model0.697 ± 0.0110.736

Prompt alignment accounts for 12.9 of the 14.8 points, or 87%. The remaining difference between the prompt-aligned base and our model is 1.9 points. Containment is lower for our model under every base prompt. We therefore no longer claim a large general reading improvement. The revised abstract and Section 5.3 state this decomposition explicitly.

The pure-QA arm also remains higher than the final model on exact-EM (0.738 against 0.697). It produces no coordinates in any of its 278 ref2box answers, with zero truncation. We now describe the result as a 4.1-point exact-EM trade for grounding capability, not as an unqualified QA win.

3. Synthetic supervision does not improve the public degraded-QA result

We agree with the review's reading of Section 5.4. On DocVQA-degraded, the base scores 0.890 and synth-only training scores 0.883. The final score of 0.912 uses DocVQA training data as replay, so it is replay repair rather than evidence of robustness acquired from synthetic documents. The same caution applies to the other public QA rows with in-domain replay.

The revised manuscript makes this attribution explicit. We retain the five-channel benchmark as a measurement of acquisition difficulty, where photo is the largest gap, but we do not claim that synth-only training improves public degraded QA.

4. Human-annotated real-domain evidence

We added XFUND-zh and FUNSD evaluations with third-party human box annotations and no image overlap with training. Each reported task has 150 items.

Domain and taskBaseOurs
XFUND-zh ref2box, [email protected]0.0270.853 ± 0.000 (3 seeds)
XFUND-zh answer-with-box, [email protected]0.0130.813 (seed 42)
XFUND-zh box-to-text, hit0.3270.933 (seed 42)
FUNSD ref2box, [email protected]0.0400.507 ± 0.009 (3 seeds)
FUNSD answer-with-box, [email protected]0.0070.400 (seed 42)
FUNSD box-to-text, hit0.2800.713 (seed 42)

Ref2box uses three training seeds; the other rows are seed-42 probes. The ± values are seed standard deviations, not sampling confidence intervals.

These results do not rescue the QA claim, and we do not use them for that purpose. They support the narrower grounding claim. In particular, the base remains near zero on real forms even with the permissive parser and oracle coordinate frame.

5. Writing

We accept the review's assessment of readability. The effect is real, and it is our responsibility. The revised manuscript removes the undefined terms "format-unlocking floor", "spending the field", "label debt", and "the empty quadrant". It removes label-colon constructions from running prose and rewrites the compressed passages identified in the review. A new paragraph before the experiments states both what we claim and what we do not claim. Section A lists the corresponding textual changes.

I appreciate the authors' response, which removed the problematic results and promised improvements to the writing. However, I still believe this paper lacks a consolidated set of experiments to convince me of the necessity of this work. Therefore, I believe this paper falls below the standard for ICLR acceptance. I will maintain my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.