- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper addresses visual discrepancy refinement in UI-to-code generation. The authors argue that conventional iterative refinement, which prompts VLMs to regenerate entire web pages, leaves the visual-to-code mapping implicit and frequently causes regressions in already correct regions. To address this, the authors introduce IterUI, a closed-loop framework that formulates UI refinement as DOM-grounded local visual code editing.
IterUI comprises four stages: (1) discrepancy detection between the reference screenshot and current render, (2) grounding bounding boxes to DOM tree elements via IoU matching, (3) localized code editing using coordinate-preserving canvas masking and ancestor layout context, and (4) full-page render-based visual verification. The authors also curate 2,000 refinement trajectories to train IterUI-Coder-8B via SFT followed by GSRPO, which jointly scores groups of rendered candidates in a single multimodal call to provide comparative visual rewards. Evaluated on Design2Code, Flame-React, Web2Code, and UIHTML, IterUI-Coder-8B achieves state-of-the-art results among evaluated open-source models and consistently outperforms global iterative revision baselines.
Critical strengths
- A concrete and useful visual-to-code editing interface.
IterUI makes the repair target explicit and evaluates executable patches in their full-page context. This is a meaningful alternative to asking a model to regenerate an entire document from visual feedback. The preliminary study supports the importance of explicit code grounding, and the framework appropriately acknowledges that a local source edit can have non-local visual consequences.
- Substantial empirical support for the trained model and refinement pipeline.
The paper evaluates both standalone generation and subsequent refinement across four benchmarks, and compares refinement methods from identical starting pages. IterUI-Coder-8B substantially outperforms its base model, while its refinement loop improves both reported metrics on every benchmark. Using a separate test judge and checking verifier agreement against human preferences strengthen the evidence beyond optimization against a single reward model.
Critical weaknesses
- The experiments do not cleanly establish that local editing is responsible for the advantage over full-page refinement.
Table 3 compares systems with substantially different call counts, and the supplied text does not establish that full-page refinement receives the same discrepancy feedback and render-based accept/reject mechanism. A baseline that unconditionally retains rewritten pages is disadvantaged relative to a method that rejects unsuccessful candidates. Consequently, the reported improvement may partly reflect verification and candidate selection rather than DOM-grounded local editing itself. A controlled global-editing baseline with the same critic, verifier, and inference budget is important to substantiate the central comparison.
Questions for authors
- Does the full-page refinement baseline use the same verifier and acceptance rule as IterUI?
Please clarify whether global candidates are rendered and rejected when they fail to improve the page, and whether both methods receive equivalent discrepancy feedback. If not, can you report a comparison that holds the critic and verifier fixed and matches either total token budget or inference cost?
3 comments
Thank you for your response. After considering the response, I will maintain my score.
We sincerely thank the reviewer for taking the time to consider our response and additional experiments. We appreciate your careful assessment and constructive feedback, which will help us further improve the manuscript.
Reply to Reviewer thUU (1/1)
Response to W1 & Q1: Controlled comparison with the same critic and verifier
We thank the reviewer for identifying this important confound. Answer to Q1: No. The original full-page baseline did not receive discrepancy feedback and unconditionally retained every rewritten page. To isolate the contribution of DOM-grounded local editing, we conducted additional controlled experiments that equip the full-page baseline with the same critic and verifier as IterUI.
Specifically, both methods use the same initial pages, editor backbone, and three refinement rounds. The critic uses the same model and localization prompt as IterUI, identifying up to five discrepancy regions per round. For the full-page baseline, these regions are included in the rewriting prompt. We also apply the same full-page verification and accept/reject rule to the rewritten candidates, ensuring that global rewriting benefits from the same feedback and verification mechanisms.
Table R1. Change in VLM score relative to direct generation (ΔDirect) on the same 400 pages after three refinement rounds.
Verification explains part of the original performance gap, but not the advantage of local editing. Adding the verifier alone improves full-page refinement by 2.65 points for Qwen3-VL-8B and 1.58 points for GLM-5.3-Flash. However, even with both the critic and verifier, IterUI (All Roles) still outperforms full-page refinement by 0.82 and 3.73 points, respectively. Moreover, once verification is enabled, adding the same discrepancy feedback to full-page rewriting yields only +0.53 and -0.14 points. This suggests that discrepancy feedback alone is insufficient to close the gap: explicitly grounding discrepancies to editable DOM components is important for translating visual feedback into effective code modifications, consistent with our grounding ablation in Table 1.
The smaller margin for Qwen3-VL-8B reflects a challenging setting in which an untrained 8B model performs all roles, limiting both diagnosis and editing quality. When IterUI-Coder-8B handles the other roles, the same Qwen3-VL-8B editor achieves +1.23; when the trained IterUI-Coder-8B performs all roles, the improvement reaches +3.79 (Table 3).
Budget comparison. We match the initial pages, editor backbone, refinement rounds, critic, and verifier, rather than enforcing identical token consumption. Full-page rewriting inherently requires regenerating substantially more code than localized editing. Even the original full-page baseline, before adding the critic, already consumes more generation tokens than IterUI (All Roles): 5.6K vs. 3.7K per page for Qwen3-VL-8B and 35.8K vs. 25.9K for GLM-5.3-Flash. Adding the critic further increases the full-page baseline's token consumption. Enforcing an identical token budget would therefore require restricting the full-page baseline's generation budget, potentially preventing it from completing the same three refinement rounds. We instead preserve the same number of outer-loop refinement rounds and the same critic and verifier, while reporting the resulting token consumption. This does not imply identical model-call counts. Although this is not a strictly iso-cost comparison, IterUI achieves better refinement quality despite consuming fewer generation tokens, supporting the benefit of DOM-grounded local editing beyond the shared critic and verification mechanisms.