acceptodds
Under review as a conference paper at ICLR 2027

IterUI: Learning DOM-Grounded Local Visual Code Editing for UI-to-Code

Abstract

Vision-language models (VLMs) can generate executable webpages from UI screenshots, yet correcting their remaining visual discrepancies remains challenging. Existing full-page refinement treats each correction as another page-generation step, leaving the correspondence between a local visual error and its responsible code implicit and potentially disturbing already-correct regions. We instead formulate UI refinement as local visual code editing, where each action pairs a grounded DOM component with an executable code update whose effect is evaluated after full-page rendering. Based on this formulation, we propose IterUI, a closed-loop framework that detects discrepancies between a reference screenshot and the current rendering, grounds them to DOM components, and generates targeted patches from localized visual evidence, component code, and ancestor layout context. IterUI masks non-target pixels while preserving the original image canvas, retaining the position and scale information needed for visual correction. Each patch is reintegrated into the complete page and accepted only after render-based verification. Because the framework requires only editable source code, browser renders, and the rendered DOM, it can also refine outputs from existing VLMs without updating their parameters. We further train IterUI-Coder-8B on refinement trajectories and optimize its code-generating actions using Group-Scored Relative Policy Optimization (GSRPO), which jointly evaluates groups of rendered candidates to construct comparative visual rewards. Across four UI-to-code benchmarks, using a test-time judge distinct from the training reward model, IterUI-Coder-8B achieves the highest VLM scores among the evaluated open-source models. On UIHTML, IterUI further raises its VLM score from 66.80 to 69.12.

Primary area:
foundation or frontier models, including LLMs
Keywords:
UI-to-Code Generation, Visual Code Editing, Iterative Refinement, Vision-Language Models, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_thUUOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses visual discrepancy refinement in UI-to-code generation. The authors argue that conventional iterative refinement, which prompts VLMs to regenerate entire web pages, leaves the visual-to-code mapping implicit and frequently causes regressions in already correct regions. To address this, the authors introduce IterUI, a closed-loop framework that formulates UI refinement as DOM-grounded local visual code editing.

IterUI comprises four stages: (1) discrepancy detection between the reference screenshot and current render, (2) grounding bounding boxes to DOM tree elements via IoU matching, (3) localized code editing using coordinate-preserving canvas masking and ancestor layout context, and (4) full-page render-based visual verification. The authors also curate 2,000 refinement trajectories to train IterUI-Coder-8B via SFT followed by GSRPO, which jointly scores groups of rendered candidates in a single multimodal call to provide comparative visual rewards. Evaluated on Design2Code, Flame-React, Web2Code, and UIHTML, IterUI-Coder-8B achieves state-of-the-art results among evaluated open-source models and consistently outperforms global iterative revision baselines.

Critical strengths
  1. A concrete and useful visual-to-code editing interface.

IterUI makes the repair target explicit and evaluates executable patches in their full-page context. This is a meaningful alternative to asking a model to regenerate an entire document from visual feedback. The preliminary study supports the importance of explicit code grounding, and the framework appropriately acknowledges that a local source edit can have non-local visual consequences.

  1. Substantial empirical support for the trained model and refinement pipeline.

The paper evaluates both standalone generation and subsequent refinement across four benchmarks, and compares refinement methods from identical starting pages. IterUI-Coder-8B substantially outperforms its base model, while its refinement loop improves both reported metrics on every benchmark. Using a separate test judge and checking verifier agreement against human preferences strengthen the evidence beyond optimization against a single reward model.

Critical weaknesses
  1. The experiments do not cleanly establish that local editing is responsible for the advantage over full-page refinement.

Table 3 compares systems with substantially different call counts, and the supplied text does not establish that full-page refinement receives the same discrepancy feedback and render-based accept/reject mechanism. A baseline that unconditionally retains rewritten pages is disadvantaged relative to a method that rejects unsuccessful candidates. Consequently, the reported improvement may partly reflect verification and candidate selection rather than DOM-grounded local editing itself. A controlled global-editing baseline with the same critic, verifier, and inference budget is important to substantiate the central comparison.

Questions for authors
  1. Does the full-page refinement baseline use the same verifier and acceptance rule as IterUI?

Please clarify whether global candidates are rendered and rejected when they fail to improve the page, and whether both methods receive equivalent discrepancy feedback. If not, can you report a comparison that holds the critic and verifier fixed and matches either total token budget or inference cost?

3 comments
AuthorsOpenReview ↗

Reply to Reviewer thUU (1/1)

Response to W1 & Q1: Controlled comparison with the same critic and verifier

We thank the reviewer for identifying this important confound. Answer to Q1: No. The original full-page baseline did not receive discrepancy feedback and unconditionally retained every rewritten page. To isolate the contribution of DOM-grounded local editing, we conducted additional controlled experiments that equip the full-page baseline with the same critic and verifier as IterUI.

Specifically, both methods use the same initial pages, editor backbone, and three refinement rounds. The critic uses the same model and localization prompt as IterUI, identifying up to five discrepancy regions per round. For the full-page baseline, these regions are included in the rewriting prompt. We also apply the same full-page verification and accept/reject rule to the rewritten candidates, ensuring that global rewriting benefits from the same feedback and verification mechanisms.

Table R1. Change in VLM score relative to direct generation (ΔDirect) on the same 400 pages after three refinement rounds.

EditorFull-page+ Critic+ Verifier+ BothIterUI (All Roles)
Qwen3-VL-8B-4.07-1.67-1.42-0.89-0.07
GLM-5.3-Flash-0.28+0.15+1.3+1.16+4.89

Verification explains part of the original performance gap, but not the advantage of local editing. Adding the verifier alone improves full-page refinement by 2.65 points for Qwen3-VL-8B and 1.58 points for GLM-5.3-Flash. However, even with both the critic and verifier, IterUI (All Roles) still outperforms full-page refinement by 0.82 and 3.73 points, respectively. Moreover, once verification is enabled, adding the same discrepancy feedback to full-page rewriting yields only +0.53 and -0.14 points. This suggests that discrepancy feedback alone is insufficient to close the gap: explicitly grounding discrepancies to editable DOM components is important for translating visual feedback into effective code modifications, consistent with our grounding ablation in Table 1.

The smaller margin for Qwen3-VL-8B reflects a challenging setting in which an untrained 8B model performs all roles, limiting both diagnosis and editing quality. When IterUI-Coder-8B handles the other roles, the same Qwen3-VL-8B editor achieves +1.23; when the trained IterUI-Coder-8B performs all roles, the improvement reaches +3.79 (Table 3).

Budget comparison. We match the initial pages, editor backbone, refinement rounds, critic, and verifier, rather than enforcing identical token consumption. Full-page rewriting inherently requires regenerating substantially more code than localized editing. Even the original full-page baseline, before adding the critic, already consumes more generation tokens than IterUI (All Roles): 5.6K vs. 3.7K per page for Qwen3-VL-8B and 35.8K vs. 25.9K for GLM-5.3-Flash. Adding the critic further increases the full-page baseline's token consumption. Enforcing an identical token budget would therefore require restricting the full-page baseline's generation budget, potentially preventing it from completing the same three refinement rounds. We instead preserve the same number of outer-loop refinement rounds and the same critic and verifier, while reporting the resulting token consumption. This does not imply identical model-call counts. Although this is not a strictly iso-cost comparison, IterUI achieves better refinement quality despite consuming fewer generation tokens, supporting the benefit of DOM-grounded local editing beyond the shared critic and verification mechanisms.

Thank you for your response. After considering the response, I will maintain my score.

We sincerely thank the reviewer for taking the time to consider our response and additional experiments. We appreciate your careful assessment and constructive feedback, which will help us further improve the manuscript.

Reviewer_3BB8OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces IterUI, a framework for refining screenshot-to-code generation through DOM-grounded local visual code editing. Instead of regenerating an entire webpage, IterUI detects visual discrepancies, maps them to DOM components, generates targeted code patches using localized visual evidence and ancestor layout context, and verifies each patch after full-page rendering. The authors further train IterUI-Coder-8B on automatically constructed refinement trajectories and propose Group-Scored Relative Policy Optimization (GSRPO) to optimize code-generating actions using jointly evaluated rendered candidates. Experiments on four UI-to-code benchmarks demonstrate improvements over several one-shot and iterative baselines. However, the methodological novelty, robustness of the automated evaluation, and practical efficiency of the complete refinement pipeline require further investigation.

Critical strengths
  1. The paper formulates UI refinement as explicit DOM-grounded local code editing, providing a practical alternative to repeated full-page regeneration.
  2. The framework integrates discrepancy detection, DOM grounding, ancestor-layout context, and full-page render-based verification into a coherent refinement pipeline.
  3. Experiments on four UI-to-code benchmarks demonstrate that IterUI-Coder-8B achieves competitive visual fidelity, with further improvements from iterative refinement.
  4. The preliminary study and ablation experiments provide useful evidence that explicit code-target grounding and coordinate-preserving visual masking can improve localized repair.
  5. The authors conduct human evaluations and analyze reward-scoring protocols, iteration budgets, localization accuracy, and DOM-grounding robustness, providing a relatively comprehensive empirical investigation.
Critical weaknesses
  1. The methodological novelty is not sufficiently established, as the framework combines existing ideas from visual discrepancy detection, DOM grounding, localized code editing, render-based verification, and relative policy optimization without clearly isolating a fundamentally new algorithmic contribution.
  2. The benefits of iterative refinement are inconsistent across configurations, as IterUI with Qwen3-VL-8B obtains a final VLM score of 59.85 compared with 59.92 for the initial output, despite additional model calls and refinement steps.
  3. The proposed GSRPO primarily adapts group-relative policy optimization to jointly scored rendered candidates, while its group-scoring protocol exhibits lower repeat consistency and weaker agreement with pairwise rankings than independent scoring.
  4. The human-validated visual judge achieves approximately 76% pairwise agreement, leaving substantial uncertainty in the automated reward and acceptance decisions that directly influence both training and inference-time refinement.
  5. The localization and grounding analyses rely partly on 171 single-edit examples and controlled structural stress tests, which may not adequately represent the complexity of naturally occurring webpages with nested components, coupled layout dependencies, and multiple simultaneous discrepancies.
  6. The framework introduces additional inference-time computation, and the evaluation does not sufficiently characterize end-to-end latency and monetary cost under realistic deployment workloads, despite reporting reduced token usage in several comparisons.
  7. The training pipeline depends on proprietary models for trajectory construction and reinforcement-learning rewards, limiting full reproducibility and making the resulting performance potentially sensitive to the behavior of these external models.
Questions for authors
  1. Can the authors more clearly isolate the contribution of DOM-grounded local editing from existing discrepancy-driven refinement and component-based code generation methods under matched evaluation settings?
  2. Why does IterUI fail to improve the initial VLM score in some configurations, and can the authors identify the conditions under which local patches cause neutral or negative full-page changes?
  3. Given the lower repeat consistency of group-scored evaluation, how sensitive is GSRPO training to reward-ranking noise, and can the authors compare its final performance with independently scored rewards under matched training budgets?
  4. Can the authors provide additional human evaluations on diverse webpage styles and complex layouts to establish the reliability of the automated judge beyond the current UIHTML comparison set?
  5. Can the authors evaluate IterUI on naturally occurring webpages with multiple interacting discrepancies and report the end-to-end success rate of the complete detection-to-editing pipeline?
  6. Can the authors report end-to-end latency, total API cost, and wall-clock time against full-page iterative refinement under comparable model configurations and stopping criteria?
  7. To what extent do the training trajectories and reward signals depend on Gemini-3-Flash and GPT-5.2, and can the authors demonstrate that comparable performance can be achieved using open-source trajectory generators and reward models?
8 comments
AuthorsOpenReview ↗

Reply to Reviewer 3BB8 (1/6)

Response to W1: Clarifying the Methodological Contribution

We thank the reviewer for raising the question of how IterUI's methodological contribution can be distinguished from its constituent techniques. Our main contribution is a reformulation of the refinement problem. The central challenge in UI-to-code refinement is translating visual discrepancies into effective code edits: identifying which DOM component is responsible for an error and how to modify it without disrupting the rest of the page. Existing approaches do not explicitly establish this correspondence between visual discrepancies, code-level targets, and verifiable edits.

We formulate refinement as agentic local visual code editing, where the agent identifies a DOM component, executes a localized code patch, and verifies the result through full-page rendering. This forms a closed-loop interaction paradigm in which each patch is a self-contained tool call that can be independently accepted, rejected, or rolled back, while multiple regions can be edited in parallel.

The same loop also serves as a data engine. By retaining verifier-accepted candidates, it produces higher-quality refinement trajectories than direct generation alone. Training on these trajectories enables an 8B model to substantially outperform its base model across all four benchmarks (Table 2).

Response to W2: Understanding Inconsistent Refinement Gains

The Qwen3-VL-8B configuration, in which the untrained model performs every role, represents the lower bound of IterUI. Its limited diagnosis and editing ability caps the gain. It proposes many candidate patches, which explains the extra calls, but few of them pass verification.

The gain grows with role capability. With the same model as editor only, refinement gains +1.23; with the same base model trained for all roles (IterUI-Coder-8B), it gains +3.79 (Table 3). This is why we train IterUI-Coder.

Even in this lower-bound setting, verification mitigates large average regressions: IterUI shows an average change of -0.07 (within the ±0.62 bootstrap margin), whereas full-page refinement with the same model loses 4.07 points at higher generation-token consumption (5.6K vs. 3.7K per page).

Response to W3: Group Scoring and Reward-Ranking Noise

We thank the reviewer for raising concerns about the consistency of group scoring and its implications for policy optimization. Our choice of group scoring is driven by the needs of policy optimization, rather than by a claim that it is a more reliable judge. We acknowledge its lower repeat consistency, as reported in Appendix D.2. GRPO derives advantages from rewards normalized within each candidate group, rather than requiring globally calibrated absolute reward scores. Reward magnitudes within a group still affect these advantages. Group scoring provides a higher-contrast relative training signal, while reducing evaluator calls from eight to one per group compared with independent VLM scoring. Although its lower repeat consistency may introduce reward noise, we empirically evaluate its downstream effects through comparisons with matched optimization steps and candidate-group counts.

The table below summarizes the scoring-protocol analysis and downstream policy performance reported in Tables 13 and 12, respectively.

Reward protocolTop-1 agreement with pairwiseRepeat Kendall τAdvantage rangeCalls per groupTokens per groupFinal VLMFinal DINOv3
Pairwise1––28~214K––
Independent scoring0.320.3851.088~42K64.8987.41
Group scoring0.30.2271.341~23K66.0387.99

Pairwise ranking is too costly for RL training. Although independent scoring is more consistent across repeated evaluations, group scoring achieves better downstream policy performance with fewer evaluator calls and tokens per group under matched optimization steps and candidate-group counts. These results use a test-time judge different from the training reward model, providing evidence that the gains extend beyond scores from the training reward judge.

AuthorsOpenReview ↗

Reply to Reviewer 3BB8 (2/6)

Response to W4: Reliability of the Automated Visual Judge

We agree that a 76% pairwise agreement leaves room for improvement and introduces uncertainty in individual judgments. However, much of this uncertainty stems from the task itself, rather than the judge alone. Two renderings of the same reference often differ in multiple aspects; for example, one may have better spacing while the other has better colors, making even human judgments inherently subjective.

To quantify this uncertainty, we re-annotated the audit pairs with three human annotators and computed their average pairwise agreement, obtaining 80.7% inter-annotator agreement. Our verifier achieves 76.0% agreement with the human majority, comparable to Claude Sonnet 5 (76.4%) and GPT-5.2 (74.3%). While the human-human and model-human agreement measures use different aggregation protocols, these results provide context for the inherent ambiguity of visual judgments.

RaterAgreement (%)
Human vs. human80.70%
Claude Sonnet 576.40%
IterUI-Coder verifier76.00%
GPT-5.274.30%
DINOv3 rule67.30%
Qwen3.5-35B66.20%

Importantly, our goal is not to achieve perfect agreement on every individual pair, but to provide reliable signals for edit verification and model-level comparison. Although individual judgments can be noisy, these errors do not necessarily translate into unreliable aggregate evaluations. Across the seven evaluated models, Elo scores computed from our automated judge's pairwise preferences correlate strongly with human-derived Elo scores, achieving Pearson r = 0.98 and Spearman ρ = 0.96. These correlations are measured at the model level rather than the individual comparison level, supporting the reliability of aggregate model rankings.

We further validate our conclusions using independent evaluation metrics: VLM scores from Claude Sonnet 5, which differs from the GPT-5.2 training reward model; DINOv3 similarity, which requires no VLM; and a human study conducted by professional front-end developers. All three consistently show improvements, with the human study ranking IterUI-Coder first (Tables 2 and 5).

Response to W5: Representativeness of Localization and Grounding Evaluation

We would like to clarify that the controlled sets are diagnostic by design. Evaluating localization and grounding requires knowing the causal DOM node of each discrepancy, which naturally occurring webpages do not provide. We therefore isolate these stages using examples with exact ground truth, while evaluating the complete pipeline on four benchmarks of real webpages.

Our end-to-end evaluation on four real-world benchmarks complements these controlled diagnostic experiments. Generated webpages naturally exhibit multiple visual discrepancies and diverse structural patterns. IterUI processes up to five predicted regions per round over three rounds, with ancestor context and full-page verification to account for potential interactions. While these benchmarks do not provide fine-grained annotations for separately measuring success on coupled discrepancies, the consistent page-level improvements provide evidence of the complete pipeline's effectiveness under naturally occurring webpage complexity. IterUI achieves positive average gains across all four benchmarks, with the largest improvement of +2.32 on UIHTML.

AuthorsOpenReview ↗

Reply to Reviewer 3BB8 (3/6)

Response to Q1: Isolating DOM-Grounded Local Editing

We thank the reviewer for this question. We clarify the contribution of DOM-grounded local editing through controlled ablations, matched refinement comparisons, and comparisons with existing component-based methods.

1. Isolating DOM grounding and localized patching. Table 1 progressively isolates these two components under the same single-edit evaluation settings:

  • DOM grounding (B→C). Configuration B provides the editor with the visual discrepancy region, while Configuration C additionally provides the corresponding DOM-grounded code fragment. This improves repair rates for both editors (97.1%→98.8% and 94.7%→97.7%) while reducing off-target damage (18.7%→17.5% and 34.5%→20.5%). This demonstrates that explicitly linking a visual discrepancy to its editable code target provides benefits beyond discrepancy feedback alone.

  • Localized patching (C→D). Restricting the output to a component-level patch further improves repair quality slightly without increasing off-target damage, while reducing output tokens by 93% (2,250→154) for Qwen3.8-27B and 84% (2,079→343) for GLM-5.3-Flash. Thus, the primary additional benefit of localized patching is editing efficiency and a bounded, independently verifiable edit operation.

2. Controlled comparison with discrepancy-driven full-page refinement. To further isolate local editing from the effects of discrepancy feedback and verification, we conducted additional experiments. We equip full-page refinement with the same critic and verifier as IterUI, while holding the initial pages, editor backbone, and three refinement rounds fixed. Both methods therefore receive equivalent discrepancy feedback and use the same accept/reject mechanism.

Table R1. Change in VLM score relative to direct generation (ΔDirect) on the same 400 pages after three refinement rounds.

EditorFull-page+ Critic+ Verifier+ BothIterUI (All Roles)
Qwen3-VL-8B-4.07-1.67-1.42-0.89-0.07
GLM-5.3-Flash-0.28+0.15+1.3+1.16+4.89

Even under these controls, IterUI outperforms full-page refinement by 0.82 points on Qwen3-VL-8B and 3.73 points on GLM-5.3-Flash. Moreover, the original full-page baseline already consumes more generation tokens than IterUI (5.6K vs. 3.7K and 35.8K vs. 25.9K per page, respectively). Although this is not a strictly iso-cost comparison, the results show that IterUI retains a quality advantage beyond the shared critic and verifier mechanisms, rather than merely benefiting from additional feedback or candidate selection.

3. Distinction from component-based generation. ComUICoder[1] decomposes webpages into semantic blocks for reusable component generation and element-wise refinement. In contrast, IterUI grounds visual discrepancies to specific nodes in the existing DOM and uses these boundaries to constrain executable code modifications, followed by full-page verification. Thus, our DOM edit boundary serves as an explicit visual-to-code correspondence for targeted repair, rather than a decomposition for component reuse.

We additionally compare ComUICoder with the original Qwen3-VL-8B-Instruct backbone and our trained IterUI-Coder-8B using DINOv3 visual similarity scores across the same four benchmarks:

Table R2. DINOv3 visual similarity (higher is better).

DatasetQwen3-VL-8B (Direct)ComUICoder (Initial)ComUICoder (Final)Ours (Direct)Ours (IterUI)
Design2Code90.2283.2984.5593.9394.13
Flame-React85.4881.0781.4792.3992.54
Web2Code89.7179.4480.2693.6194.02
UIHTML78.569.7371.4589.0789.68
Average85.9878.3879.4392.2592.59

Our trained IterUI-Coder-8B already outperforms ComUICoder's final results under direct generation across all four benchmarks (92.25 vs. 79.43 on average). Applying IterUI further improves the average score to 92.59. This comparison demonstrates the effectiveness of our complete system, while the controlled experiments above isolate the contributions of DOM grounding and local editing from model training and verification.

Together, these results demonstrate that DOM-grounded local editing contributes both more effective visual-to-code targeting and more efficient localized modification, beyond the benefits of discrepancy feedback, verification, and component-based generation alone.

[1] Xiao, Jingyu, et al. "Comuicoder: Component-based reusable ui code generation for complex websites via semantic segmentation and element-wise feedback." arXiv preprint arXiv:2602.19276 (2026).

Reviewer_xqEhOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Instead of rewriting whole documents for an edit, the paper formulates the task as local visual code editing. Actions pair grounded DOM components with executable code updates, and their effects are evaluated after full-page rendering. The authors propose IterUI, a closed-loop framework that detects discrepancies between a reference screenshot and the current rendering, grounds them to DOM components, and generates targeted patches from localized visual evidence, component code, and ancestor layout context. Non-target pixels are masked. Each patch is then reintegrated into the page and verified by comparing the rendered results.

IterUI takes editable source code, browser renders, and the rendered DOM, so it can refine outputs from existing VLMs. The authors also train a model dedicated to this task. Gemini-3-Flash is used to generate demonstrations, with 2,000 successful refinement trajectories retained. They use Group-Scored Relative Policy Optimization (GSRPO), jointly scoring candidates to reduce evaluator calls and obtain comparative rewards.

The preliminary study examines whether connecting visual issues with code scope improves repair, using 171 single-edit cases. Adding the oracle region gives mixed results, while adding the corresponding code fragment improves both models. Each refinement step uses three scopes: the image region containing the discrepancy, the targeted DOM component, and the full-page rendering.

The authors report the highest VLM scores among their designated open-source baselines. The IterUI-Coder verifier reaches 76.0% agreement with human decisions on full-page comparison pairs, close to Claude Sonnet 5 at 76.4%.

Critical strengths
  • Human evaluation by three frontend developers. A useful check on the automated judge, although the comparison pairs do not directly validate actual repair decisions.
  • Separate reference/current bounding boxes that accommodate displacement and resizing.
  • Evaluation on over a thousand examples across four datasets, with several component and training analyses.
  • Full-page verification accounts for the fact that local code changes can affect other parts of the page.
  • Limitations are reasonable and discussed.
Critical weaknesses
  • Some repairs might not be easily linked to a single DOM subtree. Highest-IoU geometric targeting may select the affected element even when a parent further up the chain is responsible, as in the button example discussed in the paper. Ancestor context is provided, but it is unclear how the method expands the edit scope when needed, or how large a problem these cases are.
  • In the controlled pilot, changing grounded full-page output (C) to a component patch (D) gives only small additional improvements. The importance of this restriction would need more support (Table 1).
  • GSRPO’s improvements are mixed across datasets in Figure 3, with decreases on Design2Code and Flame-React. This needs to be reconciled with the discussion of further gains and the main results.
  • GSRPO seems to be a minor adaptation of existing methods, with limited algorithmic novelty. Its contribution would need stronger support through the empirical benefits of joint scoring.
  • Kimi K2.5 is listed under closed-source models but has publicly available weights. Was this a reference to a specific version? Otherwise, it should be grouped with open-weight models accessed through an API. This also affects the claim of the highest scores among evaluated open models.
  • ComUICoder (February 2026) and RubSE (August 2026) are relevant works that should be discussed in greater detail.

[1] ComUICoder: Component-based Reusable UI Code Generation for Complex Websites via Semantic Segmentation and Element-wise Feedback. arXiv:2602.19276, 2026. [2] Rubrics as Visual-Repair Context for Self-Evolving UI-to-Code Generation. arXiv:2608.24138, 2026.

Questions for authors
  • How does the approach differ from ComUICoder (February 2026) and RubSE (August 2026), and what does the explicit DOM edit boundary add?
  • How representative are the edits used to common front end developer workflows?
6 comments
AuthorsOpenReview ↗

Reply to Reviewer xqEh (1/3)

Response to W1: Limited Adaptivity of DOM Edit Scope

We thank the reviewer for raising this important concern about cases where the responsible DOM node may be an ancestor of the highest-IoU selected node.

The highest-IoU rule is deterministic, but the box it consumes is predicted by the model, so the choice of edited node is learned. Training improves this choice: Exact@1 node grounding rises from 59.65% (SFT) to 69.59% (GSRPO), although GSRPO optimizes only code-generating actions and box coverage (IoGT) rises by just 2.29 points (Fig. 3, App. C.2). This suggests the model learns to place boxes on the editable node rather than merely covering the visible error.

To measure the remaining mismatch, we ran a paired experiment on the 171 single-edit examples (Sec. 3.1) with predicted boxes from the final model. Highest-IoU selected a descendant of the modified node in 17 cases (9.9%); editing that descendant never helped, while editing its parent fixed 16/17. Unconditional parent expansion is not a remedy: it lowers improvement on the other 154 cases and raises off-target damage from 1.2% to 22.8%, because rewriting a larger subtree tends to drop surrounding content. Given the low frequency and this trade-off, we keep the grounded scope fixed rather than add a further localization stage, which would also add rendering and verification calls per region. We will add this analysis and list selective scope expansion as a limitation in Appendix E.

MetricGrounded node (ours)Always parent
Local improvement (all 171)145 (84.8%)148 (86.5%)
├ 17 descendant cases016
└ Other 154 cases145 (94.2%)132 (85.7%)
Off-target damage2 (1.2%)39 (22.8%)
Improved without damage144 (84.2%)128 (74.9%)

Response to W2: Limited Gains from Component-Level Patching

We agree that the benefit of component-level patching should be clarified beyond repair quality.

Component patching is not primarily intended to improve repair quality. From C to D, repair rate and page score rise for both editors and off-target damage does not increase, while average output length drops by 93% for Qwen3.8-27B (2,250→154 tokens) and 84% for GLM-5.3-Flash (2,079→343 tokens).

The main purposes are lower token cost and an agentic design: each patch is a self-contained tool call on one grounded component, so multiple regions can be edited in parallel, and each patch can be verified or rolled back independently, which a full-page output does not allow. We will add these token statistics to Table 1 and clarify this motivation in Sec. 3.1.

AuthorsOpenReview ↗

Reply to Reviewer xqEh (2/3)

Response to W3: Mixed RL Gains Across Benchmarks

To better understand this discrepancy, we conducted an additional RL experiment by adding 500 GPT-synthesized samples with webpage characteristics similar to Flame-React to the original RL training data, raising its VLM score from 60.25 to 66.86. Since no training set was available for Flame-React, we synthesized these samples and retrained GSRPO. The results are reported below.

DatasetMetricBaseSFTGSRPO (Original)GSRPO (+500 Synthetic)
Design2CodeVLM Score53.0562.7763.2162.97
Accept Rate13.60%27.80%28.20%27.90%
Flame-ReactVLM Score49.2764.560.2566.86
Accept Rate11.10%22.70%21.40%23.20%
Web2CodeVLM Score52.663.0465.165.81
Accept Rate5.30%31.60%33.80%34.30%
UIHTMLVLM Score37.5256.6966.865.92
Accept Rate10.40%30.90%38.50%37.90%

This result is consistent with a distribution-mismatch explanation. Qualitatively, Flame-React pages tend to resemble older, relatively plain HTML websites, whereas the RL training pages and other benchmarks generally feature more modern layouts and richer CSS styling and effects. The difference is also reflected in their structural and visual statistics (medians over sampled pages):

MetricRL DatasetUIHTMLWeb2CodeDesign2CodeFlame-React
HTML elements—278—123.510
divs—92—275
Tag types—20—174
DOM max depth—14—103
Characters / MP421.8261.1242.2521.3106.2
Edge ratio1.58%1.17%1.05%2.62%0.99%
Grouped regions / MP21.4215.8810.5922.468.78

A typical Flame-React page contains only 10 HTML elements and has a DOM depth of 3. Its character density and grouped-region density are only 25% and 41% of those in the RL training data, respectively. SFT teaches a general editing workflow that transfers across benchmarks, while GSRPO further optimizes rendered rewards and is consequently more sensitive to the training distribution. The substantial recovery after adding style-matched synthetic data is consistent with this explanation.

Response to W4: Empirical Value of Joint Scoring

We thank the reviewer for questioning the algorithmic novelty of GSRPO and encouraging us to better substantiate the empirical value of joint scoring.

We do not claim algorithmic novelty in the policy objective, which follows GRPO with DAPO-style clipping. GSRPO is an empirically selected reward design for rendered UI candidates. Candidates from the same state often differ only in subtle visual details, so we compared reward designs and scoring protocols under the same training setup (Appendix D). Joint scoring gave the best page-level results among the four reward designs while requiring one evaluator call per group instead of eight, which makes VLM-based RL on rendered pages affordable.

We will revise the contribution statement to present GSRPO as a practical, empirically validated training component rather than a new RL algorithm.

RewardPage VLMPage DINOv3Evaluator calls / groupTokens / group
DINO only65.5587.87––
DINO + spatial penalty62.7287.47––
Absolute VLM (independent)64.8987.418~42K
Group-scored VLM (ours)66.0387.991~23K

Response to W5: Classification of Kimi-K2.5

We thank the reviewer for pointing out the potential misclassification of Kimi-K2.5 and the resulting implications for our comparison with open models.

Kimi-K2.5 refers to the public release (Kimi Team, 2026), which has publicly available open weights. We will move it to an open-weight group and revise the claim to "competitive with larger open-weight models on the evaluated benchmarks." Kimi-K2.5 has 1T parameters (32B active), compared with 8B for IterUI-Coder. IterUI-Coder-8B outperforms it on UIHTML (Table 2).

AuthorsOpenReview ↗

Reply to Reviewer xqEh (3/3)

Response to W6 & Q1: Comparison with ComUICoder and RubSE

We sincerely thank the reviewer for bringing ComUICoder and RubSE to our attention and for suggesting a more detailed comparison with these relevant works.

ComUICoder decomposes webpages into semantically coherent blocks, merges similar blocks into reusable components, and assembles them into complete webpages with element-wise feedback for refinement. RubSE generates structured visual-repair rubrics, prioritizes the most important issue in each iteration, and maintains repair history to guide subsequent revisions. In contrast, IterUI explicitly grounds visual discrepancies to DOM nodes, generates localized code patches, and verifies each modification through full-page rendering. We further optimize the editing policy with GSRPO.

Quantitative comparison. We compare ComUICoder against both the original Qwen3-VL-8B-Instruct backbone and our trained IterUI-Coder-8B. For ComUICoder, Qwen3-VL-8B-Instruct is used for both initial generation and feedback-based refinement. All results are evaluated using DINOv3 visual similarity scores on the same four benchmarks.

Qwen3-VL-8B (Direct)ComUICoder (Initial)ComUICoder (Final)Ours (Direct)Ours (IterUI)
Design2Code90.2283.2984.5593.9394.13
Flame-React85.4881.0781.4792.3992.54
Web2Code89.7179.4480.2693.6194.02
UIHTML78.569.7371.4589.0789.68
Average85.9878.3879.4392.2592.59

RubSE does not provide a public implementation, so a controlled quantitative comparison under the same backbone and evaluation protocol is currently unavailable.

Response to Q2: Representativeness of Frontend Workflows

To investigate this, we interviewed two senior front-end engineers and designers, and two observations shaped IterUI.

  1. Frontend development typically involves an iterative verify-and-edit loop: developers compare rendered pages with designs, identify the responsible elements, patch their code, and recheck the results. IterUI follows the same workflow through discrepancy detection, DOM grounding, component-level editing, and full-page render verification. The targeted discrepancies, including spacing, alignment, typography, colors, and missing or misplaced elements, are also among the most common issues identified by the experts.

  2. Professional frontend development operates within a broader PRD-driven workflow that specifies interactions, states, and data beyond visual appearance. IterUI focuses on the visual-fidelity stage evaluated by UI-to-code benchmarks. Its bounded component edits also reduce the risk of overwriting unrelated functional code compared with full-page regeneration, which may overwrite existing logic. Extending verification to PRD-level requirements, such as interaction states, is a natural direction for future work.

We will summarize these interviews in the appendix and clarify the scope of IterUI in Appendix E.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.