acceptodds
Under review as a conference paper at ICLR 2027

w1: Visually Grounded Code Improves Screenshot-to-HTML Generation

Abstract

This paper introduces w1, a very competitive vision-language model (VLM) for converting webpage screenshots into HTML code. We find this task bottlenecked by the quality of supervision targets: code of real webpages contains many tokens, such as invisible elements, unused styles, redundant nesting, and media source strings, which are not visible from the webpage screenshot. It is not possible for a VLM to predict such code from the screenshot. To address this problem, we propose a data pipeline that removes the invisible code while preserving the rendering results. Specifically, our pipeline parses the HTML, matches each DOM element with its computed CSS, removes code without visible effect, canonicalizes render-equivalent structure, and replaces media sources with placeholders. The result is the WebGround-3M dataset. It has 3M screenshot-HTML pairs, where the code grounds in the screenshot. In this dataset, supervision targets are 30% shorter on average while rendering very similar to the original screenshots. On WebGround-3M, fine-tuning a VLM demonstrates an effective improving trend. Moreover, from WebGround-3M, we curate 200k synthetic preference pairs targeting at syntactic repetition and semantic perturbation; direct preference optimization (DPO) using these preference pairs adds further performance gains. Across six public benchmarks, w1 achieves highly competitive code generation performance. In particular, on benchmarks evaluated using GPT scores and the Design2Code metric, w1-9B outperforms zero-shot GPT\-5 and Gemini-3.5-flash, and models trained on WebSight-v1/v2, VinciCoder, MCD, WebCode2M, and Web2Code.

Primary area:
datasets and benchmarks
Keywords:
Vison2Code, Code Generation, Vision Language Model

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_G1YzOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
3(+1)
Presentation
3
Soundness
3
Read review
Paper summary

This work presents w1, a VLM for screenshot-to-HTML transformation. The core idea is to ensure that the output code in the training data is visually grounded by removing invisible elements and unused styles, etc.

The resulting dataset, WebGround-3M, is shown to be more effective and easier to train compared to baselines such as WebSight-v2 and WebCode2M.

The authors also curate 200k preference pairs for DPO, which intentionally target syntactic repetition and semantic perturbation.

The complete pipeline, SFT + DPO, allows Qwen3.5-9B to outperform strong models such as zero-shot GPT-5 and Gemini-3.5-flash.

Critical strengths
  • The idea of ensuring the code is visually grounded for converting webpage screenshots into HTML code is clean and elegant.
  • The derived dataset WebGround-3M shows a clear advantage compared to the WebCode2M baseline at different data scales (100k to 3M). The advantage also holds across four other backbone families.
  • The experiments are reasonably comprehensive: six benchmarks, three model sizes, five backbone families, and an ablation on the preference-pair construction. The synthetic DPO negatives are also a cheap and practical way to handle repetition, which seems to be the dominant failure mode.
Critical weaknesses
  • It seems that there is no plan for releasing the datasets and models. The reproducibility statement only says that the backbones, benchmarks, and source corpus (WebCode2M) are public. Since the main contribution of this paper is the data, not releasing WebGround-3M, the cleaning pipeline, the 200k preference pairs, or the w1 checkpoints significantly limits the value to the community.
  • The claim that cleaning "preserves the rendering results" is only supported by CLIP similarity (98.81% after Step 1), which is a coarse measure. Appendix A also acknowledges that removing frames, canvas, and overlays can change visible content. It would be more convincing to report finer-grained metrics (e.g., the Design2Code block/text/position scores) between the original and cleaned renders, and the fraction of pages where visible content is lost.
  • The cleaned code is visually grounded, but it may be less useful as real-world code: semantic tags (header, nav, main) are normalized to div, hover styles are removed, and selectors are rewritten to compact class names. These do not change the screenshot, but they matter for accessibility, interactivity, and maintainability, which is part of why people want screenshot-to-HTML in the first place. The paper should at least discuss this trade-off.
Questions for authors
  • In Table 2, it seems that the MCD baseline is very competitive while using much fewer pages and tokens (200k pages x 582 tokens, roughly 80x fewer target tokens than WebGround-3M). In fact, at a matched scale of 200k pairs, MCD (5.72 GPT / 84.5 D2C) is better than w1-SFT (5.05 / 80.0, Table 7) with the same Qwen3-VL-4B backbone. Can the authors elaborate more on the pros and cons of using AI-generated HTML code? Would it be a better approach (or more computationally efficient) to use both MCD and WebGround-3M, e.g., by mixing the two or training on MCD first and then on WebGround-3M?
  • Is the DPO gain specific to WebGround-3M? Would the same synthetic preference pairs also improve models fine-tuned on WebCode2M or MCD?
2 comments

We thank the reviewer for the encouraging assessment and the constructive suggestions. All changes are highlighted in blue in the revised manuscript.

W1. No release plan.

We apologize for the omission. The Reproducibility Statement (p. 11) now commits to releasing WebGround-3M, the 200k preference pairs with their corruption scripts, the w1 checkpoints, and the evaluation scripts.

W2. Rendering preservation is supported only by CLIP.

We agree that CLIP is too coarse. App. E.1 (Table 11, p. 26) now reports the Design2Code sub-scores between the original rendering and the rendering after Step 1. The metric is costly to compute, so we use 10,000 uniformly sampled pages:

BlockTextPositionColorCLIPAvg
99.099.598.999.199.599.2

Pages losing visible content. We do not yet have a reliable per-page count and prefer not to report one we cannot stand behind. The cases are those in App. A (frames, canvas, overlays); their aggregate effect is bounded by the scores above. We also (i) no longer claim rendering equivalence: the abstract and Sec. 1 now say "largely / closely preserving", and the Limitation paragraph (Sec. 3, p. 6) states that visible content can be deleted on some pages; (ii) note in Sec. 2.2 that the input screenshot is re-rendered from the cleaned code, so every training pair stays consistent.

W3. Cleaned code may be less useful as real-world code.

This is a fair point: tag normalization, removal of :hover rules, and class renaming affect how directly the code can be used. We explain the design in a new paragraph of Sec. 3 ("What does screenshot-to-code supervise?", p. 6). A static screenshot specifies appearance only. It does not carry the requirements for accessibility, interactivity, or maintainability, so these cannot be well supervised from a screenshot alone. Interactivity is a separate sub-task that needs other inputs, such as several state captures or a text specification. Semantic tags are partly inferable from layout; the new ablation (Table 12) shows that tag normalization is the least important component of Step 1 (-0.09 GPT when removed), so they could be retained at a small cost. Bringing these properties into the task requires suitably designed inputs and benchmarks, since current benchmarks evaluate visual similarity only; we list this as future work.

Q1. Pros and cons of AI-generated HTML; mixing with MCD.

The reviewer's reading of Tables 2 and 7 is correct, and Sec. 3 ("Real-world webpages versus synthetic webpages", p. 6) now explains such.

Pros. Synthetic pages are grounded by construction and short, hence token-efficient: at a matched 200k pairs, MCD reaches 5.72 GPT against 5.05 for WebGround-3M.

Cons. (i) Their quality comes from the teacher: training on them transfers the capability of a frontier model and is bounded by its cost and capability. Our pipeline needs no teacher, only public crawled pages and deterministic rules, so the solution is open and reproducible. (ii) The pages are much simpler than real ones (21.5 vs. 132.3 DOM elements); on CC-Hard, 200k WebGround-3M pairs already beat MCD (5.84 vs. 5.45 GPT). (iii) Crawled pages are practically unlimited, and performance keeps improving up to 3M pairs (Fig. 4), where WebGround-3M overtakes MCD (6.19 vs. 5.72).

Mixing or MCD first. We agree this is promising, since the two sources look complementary. We could not complete these experiments within the rebuttal period, so we make no claim and list them as future work in the same paragraph.

Q2. Is the DPO gain specific to WebGround-3M?

No, although it is largest there by a small margin. For models fine-tuned on WebCode2M and MCD, we built 200k preference pairs with the same construction strategy (that training set's targets as preferred samples, corrupted copies as negatives), so that the preferred samples match what each SFT model was trained to produce. See App. E.3 (Table 13, p. 27).

SFT dataSFT GPT / D2C+ DPO GPT / D2CCode repetition (no </html>, of 1,460)
WebCode2M5.34 / 82.26.03 / 86.3508 → 220
MCD5.72 / 84.55.94 / 85.869 → 34
WebGround-3M6.19 / 86.46.90 / 90.6356 → 108

DPO helps all three, mainly by reducing code repetition. The gain is smallest for MCD, whose SFT model produces short pages and rarely repeats. To the best of our knowledge, we are the first to construct a preference dataset and to apply DPO for this task.

We thank the reviewer again and are happy to answer further questions.

Thanks the authors for the response.

Most of my concerns are resolved. Looking forward to the public release of the relevant resource.

I want to highlight that this paper present a simple, clean, and effective idea of improving screensho-to-html generation, with convincing empirical evidence.

I will increase my score to 4.

Reviewer_EUkiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a vision-language model, w1, that converts webpage screenshots into HTML code. Using the proposed data pipeline, it constructs WebGround-3M, a dataset containing approximately 3M screenshot–HTML pairs. In addition, it curates 200k synthetic preference pairs from WebGround-3M for DPO training to further improve w1’s performance. The dataset is evaluated across multiple mainstream model backbones and achieves higher average scores than datasets such as WebCode2M and MCD in the reported comparisons.

Critical strengths
  1. To remove noisy code, the paper proposes a data pipeline for constructing clean screenshot–HTML pairs, resulting in the WebGround-3M dataset.
  2. To improve w1’s performance, the paper constructs 200k preference pairs for DPO training targeting code repetition and visual inconsistency. The proposed dataset and training approach also demonstrate good overall performance on models such as InternVL3.5 and SmolVLM2.
Critical weaknesses

Limited methodological novelty. Key components of the data pipeline, such as HTML simplification and media normalization, have precedents in existing work, including Pix2Struct, WebCode2M, and Design2Code. The paper does not adequately explain the advantages of its specific pipeline over these existing approaches, and the benefits of its individual components have not been sufficiently demonstrated.

Questions for authors

Do code repetition and visual inconsistency also occur after SFT on other datasets, such as WebCode2M? How do these issues compare with those observed after SFT on WebGround-3M?

2 comments

We thank the reviewer for the positive comments and the precise question about failure modes. All changes are highlighted in blue in the revised manuscript.

W1. Limited novelty relative to Pix2Struct, WebCode2M, and Design2Code; advantages and component benefits not sufficiently demonstrated.

We agree that the submission did not explain this clearly. We revised Sec. 3 (p. 6) and Sec. 5 (p. 10) and added an ablation in App. E.2.

Positioning. Existing work already explores data sources along different dimensions: AI-generated (WebSight, MCD), crawled (WebCode2M), and hybrid (Web2Code, VinciCode). Our contribution is not another data source but an optimization strategy for dataset preparation. It targets crawled pages, whose supply is practically unlimited and whose value is limited mainly by noise in the code.

Relation to the three works. The shared strategies serve different goals there.

  • Pix2Struct uses screenshot parsing to pretrain a model for visual language understanding (question answering, captioning). Its target drops CSS, so a browser cannot draw the page from it, whereas ours must render like the screenshot.
  • Design2Code removes external dependencies and uses placeholder media, but only to curate 484 test pages for evaluation.
  • WebCode2M purifies page source and is our starting point, yet our pipeline removes a further 45.5% of the characters from its targets (Table 1).

Advantages. (1) It keeps every page: targets are repaired, not filtered out. (2) Targets contain only what the screenshot determines yet still render; with pages, backbone, and recipe fixed, this improves SFT at every scale (Fig. 4) and on InternVL3.5, SmolVLM2, Phi-4, and Gemma-3 (Fig. 6). (3) Each cleaned target is an exact positive, which makes preference pairs and DPO possible.

Component benefits (new Table 12, p. 26; Qwen3-VL-4B, 200k pairs, six-benchmark means; Step 1 = structural code cleaning, Step 2 = media code unification):

Training targetsGPTD2C
WebCode2M3.8876.1
Step 2 only4.0276.6
Step 1 only4.8679.5
Step 1 + Step 2 (= WebGround-3M)5.0580.0
Step 1 w/o script and metadata removal4.7579.1
Step 1 w/o invisible-element pruning4.5078.2
Step 1 w/o unused CSS and identifier removal4.6178.6
Step 1 w/o tag normalization and wrapper collapsing4.7779.2

Both steps help and are complementary. Within Step 1, invisible-element pruning and unused-CSS removal matter most; these rely on browser measurement and go beyond source-level simplification.

Q1. Do code repetition and visual inconsistency also occur after SFT on other datasets such as WebCode2M?

Yes, both occur, and code repetition is more frequent after SFT on WebCode2M. See App. E.3 (Table 13, p. 27) and Sec. 4.4 (p. 9). We split the 1,460 benchmark generations (Qwen3-VL-4B) into the two failure modes:

ModelCode repetition (no </html>)Visual inconsistency (ends with </html>)GPTD2C
WebCode2M SFT508 (34.8%)952 (65.2%)5.3482.2
+ DPO220 (15.1%)1,240 (84.9%)6.0386.3
MCD SFT69 (4.7%)1,391 (95.3%)5.7284.5
+ DPO34 (2.3%)1,426 (97.7%)5.9485.8
WebGround-3M SFT356 (24.4%)1,104 (75.6%)6.1986.4
+ DPO (w1)108 (7.4%)1,352 (92.6%)6.9090.6

A generation that has not emitted </html> within the 8,192-token budget is cut off, which in practice happens when the model keeps repeating code. A generation that ends with </html> is a complete page whose remaining errors are visual; GPT and D2C reflect its quality.

  1. Code repetition is worse after SFT on WebCode2M (34.8% against 24.4% for WebGround-3M): longer targets with ungrounded tokens make repetition more likely. MCD-trained models produce short pages and rarely repeat.
  2. Cleaning reduces repetition but does not remove it, which motivates DPO. To the best of our knowledge, we are the first to construct a preference dataset and to apply DPO for this task.
  3. The same pair-construction strategy, applied to each training set's own targets, also helps the other SFT models: it moves generations from repetition to complete pages and raises GPT and D2C.

We hope these additions address the reviewer's concerns and are happy to provide further details.

Thank you for the detailed response, which have addressed my main concerns. I have read the rebuttal and the other reviews.I will keep my rating.

Reviewer_2YxGOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents WebGround-3M, a cleaned screenshot-to-HTML dataset derived from WebCode2M. It removes visually irrelevant code and further applies DPO with synthetic preference pairs to improve visual fidelity. Experiments across multiple benchmarks and model scales show improvements in screenshot-to-HTML generation.

Critical strengths
  1. The paper addresses a practical issue in screenshot-to-HTML training data: visually irrelevant and redundant code.
  2. The experiments cover multiple benchmarks and model scales, showing consistent performance improvements.
Critical weaknesses
  1. The main limitation is the modest novelty. The data-cleaning pipeline largely builds on existing screenshot-to-code data construction practices, such as WebCode2M, without a substantial methodological advance.
  2. The paper provides limited analysis of which components of the cleaning pipeline contribute most to the observed improvements.
  3. The DPO stage relies on synthetic preference pairs constructed from predefined error patterns, making this component largely incremental.
Questions for authors

What are the key methodological differences between WebGround-3M and existing data construction pipelines such as WebCode2M, and which of these differences are essential to the observed improvements?

1 comment

Response to Reviewer 2YxG

We thank the reviewer for the careful assessment. All changes are highlighted in blue in the revised manuscript.

W1 / Q1. Modest novelty; key differences from pipelines such as WebCode2M, and which are essential.

We agree that individual operations have precedents and that the submission did not position the work clearly. We revised Sec. 3 (p. 6) and Sec. 5 (p. 10).

Positioning. Existing work already explores data sources along different dimensions: AI-generated (WebSight, MCD), crawled (WebCode2M), and hybrid (Web2Code, VinciCode). Our contribution is not another data source but an optimization strategy for dataset preparation. It targets crawled pages, whose supply is practically unlimited and whose value is limited mainly by noise in the code. Performance has not saturated at 3M pairs (Fig. 4), which suggests potential for pre-training and not only post-training.

Relation to earlier work. The shared strategies serve different goals there. Pix2Struct uses screenshot parsing to pretrain a model for visual language understanding; its target drops CSS, so a browser cannot draw the page from it, whereas ours must render like the screenshot. Design2Code removes external dependencies and uses placeholder media, but only to curate 484 test pages. WebCode2M purifies page source and is our starting point, yet our pipeline removes a further 45.5% of the characters from its targets (Table 1).

Advantages. (1) It keeps every page: targets are repaired, not filtered out. (2) Targets contain only what the screenshot determines yet still render; with pages, backbone, and recipe fixed, this improves SFT at every scale (Fig. 4; +0.85 GPT / +4.2 D2C at 3M) and on four further backbones (Fig. 6). (3) Each cleaned target is an exact positive, which makes preference pairs and DPO possible.

Essential differences. See the ablation below: the browser-based components of structural cleaning contribute most.

W2. Which components of the cleaning pipeline contribute most.

We added an ablation in App. E.2 (Table 12, p. 26), summarized in Sec. 4.4 (p. 9). Qwen3-VL-4B, 200k pairs, six-benchmark means; Step 1 is structural code cleaning and Step 2 is media code unification.

Training targetsGPTD2C
WebCode2M3.8876.1
Step 2 only4.0276.6
Step 1 only4.8679.5
Step 1 + Step 2 (= WebGround-3M)5.0580.0
Step 1 w/o script and metadata removal4.7579.1
Step 1 w/o invisible-element pruning4.5078.2
Step 1 w/o unused CSS and identifier removal4.6178.6
Step 1 w/o tag normalization and wrapper collapsing4.7779.2

Both steps help and are complementary; Step 1 contributes more (+0.98 GPT alone, against +0.14 for Step 2). Within Step 1, invisible-element pruning (-0.36 GPT when removed) and unused-CSS removal (-0.25) matter most. These rely on browser-computed visibility and on matching CSS against the pruned DOM. Script and metadata removal, the part closest to existing source-level purification, contributes least.

W3. The DPO stage with synthetic pairs is largely incremental.

The algorithm is not new: we use vanilla DPO. The contribution is in the data and its application.

  • First for this task. To the best of our knowledge, we are the first to construct a preference dataset for screenshot-to-HTML generation and to apply DPO to it (Sec. 3, pp. 5-6; Sec. 5).
  • Synthetic negatives are a deliberate choice. On-policy rollouts require generating thousands of tokens and rendering each page, rarely yield a perfect positive, and depend on unreliable visual rewards (Sec. 3). Our pairs need no rollout, rendering, or judge.
  • The error patterns are measured. New App. E.3 (Table 13, p. 27): over the 1,460 benchmark pages, 34.8% of generations fail by code repetition after SFT on WebCode2M and 24.4% after SFT on WebGround-3M; DPO reduces the latter to 7.4%.
  • The strategy transfers. The same pair-construction strategy also improves models fine-tuned on WebCode2M (+0.69 GPT, +4.1 D2C) and MCD (+0.22, +1.3).
  • The gain from DPO is comparable to SFT and consistent. DPO adds 0.70-1.19 GPT at every SFT scale (Fig. 4) and helps at three model sizes (Fig. 5) and on four further backbones (Fig. 6).

We hope these clarifications address the reviewer's concerns and are glad to answer further questions.

Comments on the paper

1 comment

General Response to All Reviewers

We thank all reviewers for their constructive feedback. We have uploaded a revised manuscript in which all revisions are highlighted in blue. Reviewer IDs: R1 = 2YxG, R2 = EUki, R3 = G1Yz (W = weakness, Q = question).

Main improvements

  1. Positioning. Our contribution is not another data source but an optimization strategy for dataset preparation. The dataset would byproduct from proposed dataset preparation strategy. We explain how Pix2Struct, Design2Code, and WebCode2M relate to it.
  2. Component analysis. A new ablation of the two pipeline steps and of the four components of Step 1.
  3. DPO beyond WebGround-3M. We measure the failure modes after SFT on other training sets and apply the same preference-pair construction strategy to them. To our knowledge, this is the first work to construct a preference dataset and apply DPO for this task.
  4. Finer-grained fidelity. Design2Code sub-scores between original and cleaned renderings; the "render-preserving" wording is softened.
  5. Synthetic vs. real data. Pros and cons of AI-generated HTML; combining the two is listed as future work.
  6. Release and scope. A release plan, and a discussion of what a screenshot can supervise.

List of changes

LocationChangeAddresses
Abstract; Sec. 1 (p. 3)"Preserving rendering equivalence" softened to "largely / closely preserving".R3-W2
Sec. 2.2 (p. 4)Design2Code score between original and cleaned renderings; training pairs stay consistent by construction.R3-W2
Sec. 2.3 (p. 5)The two failure modes also occur after SFT on other training sets.R2-Q1, R3-Q2
Sec. 3, "Why corrupted negatives..." (pp. 5-6)First preference dataset and first use of DPO for this task; pointer to results on other training sets.R1-W3, R2-Q1, R3-Q2
Sec. 3, "Real-world versus synthetic webpages" (p. 6)Pros and cons of AI-generated HTML; potential of crawled data for pre-training; combining both as future work.R1-W1, R2-W1, R3-Q1
Sec. 3, "A data optimization strategy, not a new data source" (p. 6)New: positions the contribution and states its advantages.R1-W1, R1-Q1, R2-W1
Sec. 3, "What does screenshot-to-code supervise?" (p. 6)New: a screenshot does not specify accessibility, interactivity, or maintainability; task and benchmark design as future work.R3-W3
Sec. 3, "Limitation" (p. 6)The rules are heuristic and can delete visible content on some pages.R3-W2
Sec. 4.4, "Which cleaning steps matter?" (p. 9)New: summary of the cleaning ablation.R1-W2, R1-Q1, R2-W1
Sec. 4.4, "Failure modes and DPO on other training sets" (p. 9)New: summary of failure modes and DPO on WebCode2M / MCD.R1-W3, R2-Q1, R3-Q2
Sec. 5 (p. 10)Relation to Pix2Struct, Design2Code, and WebCode2M; data preparation beyond SFT.R1-W1, R1-Q1, R1-W3, R2-W1
Reproducibility Statement (p. 11)Release plan: dataset, pipeline code, preference pairs, checkpoints, evaluation scripts.R3-W1
App. E.1, Table 11 (p. 26)New: fine-grained rendering fidelity of the cleaning pipeline.R3-W2
App. E.2, Table 12 (p. 26)New: ablation of Step 1, Step 2, and the Step 1 components.R1-W2, R1-Q1, R2-W1
App. E.3, Table 13 (p. 27)New: code repetition and visual inconsistency after SFT on different training sets, and DPO with the same construction strategy on each.R1-W3, R2-Q1, R3-Q2

We respond to each reviewer individually below and are happy to provide further clarification.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.