acceptodds
Withdrawn from ICLR 2027

ABPQ: Automated Construction of Challenging UI-to-Code Benchmarks via Round-Trip Consistency

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Existing UI-to-Code benchmarks face a tradeoff between evaluation comprehensiveness and construction scalability. Benchmarks containing paired UI-HTML data support fine grained objective evaluation but are costly to construct and are becoming increasingly saturated, whereas benchmarks containing only UI screenshots are easier to scale but typically rely on LLM-as-Judge. To address this issue, we propose PQ Score, which identifies samples that are challenging for models without requiring reference code by measuring information loss during the “screenshot → code → rendered screenshot → code” round trip process. Based on PQ Score, we further propose the ABPQ benchmark construction pipeline with minimal reliance on human effort and use it to construct ABPQ-FULL, containing 500 samples, and its challenging subset ABPQ-HARD, containing 200 samples, from 4,771,476 real UI screenshots. The final samples retain their corresponding HTML, supporting multiple automatic metrics, LLM-as-Judge evaluation, and human evaluation. Experiments on ten representative proprietary and open source models show that ABPQ is more challenging than representative existing benchmarks on nearly all metrics and that current advanced models remain far from saturation. In addition, PQ Score exhibits strong positive correlations with other automatic metrics, validating its effectiveness for difficult sample filtering. ABPQ provides a new approach to constructing scalable, continually updated UI-to-Code benchmarks with minimal reliance on human effort.

Primary area:
datasets and benchmarks
Keywords:
UI-to-Code, Benchmark, Multimodal Large Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_LF8pOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes PQ Score, a reference-free metric for UI-to-Code difficulty based on round-trip consistency. PQ Score is the normalized tree edit distance between the DOM tag trees of P and Q. Using this metric, the authors build the ABPQ pipeline, which filters 4.77M crawled UI screenshots through hierarchical PQ filtering, VLM-based quality control, and manual review. The result is ABPQ-FULL (500 samples) and ABPQ-HARD (200 samples), with reference HTML kept for evaluation. Experiments on proprietary and open-source models show lower scores on ABPQ than on Design2Code, Design2Code-HARD, and Vision2Web for most metrics. The authors also analyze correlations between PQ Score and existing automatic metrics and human judgments.

Critical strengths
  • Timely problem. Existing UI–HTML benchmarks constructed from open webpages may have data leakage issues.
  • New dataset. The paper proposes a new UI2Code dataset for future research.
Critical weaknesses
  • Unclear motivation. The paper’s main argument for scalability is that the pipeline needs only screenshots, but the authors crawl about 7M UI–HTML pairs and keep the HTML as the reference. If the underlying code is already available, why not directly analyze the source HTML? The screenshot-only setting is described, but it would be better to construct it this way from scratch.

  • The proposed PQ score metric lacks rigor. is unclear whether PQ Score is meant to be a filtering tool or an evaluation metric? If it is proposed as a better metric, it should show either (a) better discriminability or (b) better alignment with human judgment. The current evidence supports neither: a) Table 4 shows a strong correlation between PQ score and existing metrics b) Table 5 shows only moderate alignment with human rankings, even worse than some existing metrics

  • The problem scope in static pages may limit realistic application. The benchmark covers only static screenshots. Modern frontend development depends heavily on interactivity and dynamic state. Many of the user tasks listed in Table 9 (e.g., Interactive Use, Shopping, Booking, Gameplay) are interactive by nature, yet only their static appearance is evaluated.

Questions for authors
  1. Why does the benchmark not involve dynamic webpages?
  2. The construction process includes many filtering steps from the original 4M to 500. This limited filtering process may introduce bias in the benchmark. How do you ensure the filtering is representative and doesn't lose information from the current web landscape?
  3. In terms of the benchmark statistics, I'm not sure whether domain coverage, page type, and user task are representative enough for indicating website complexity. The usual metrics are DOM depth, node count, and CSS complexity, etc. (such as in design2code)
  4. Can you explain why the PQ score is a better metric than existing ones?
Reviewer_1orHOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PQ Score, a metric for evaluating UI-to-Code generation based on round-trip consistency, and introduces the ABPQ benchmark construction pipeline. Using this pipeline, the authors construct ABPQ-FULL, a benchmark containing 500 challenging samples, and evaluate ten models using automatic metrics and human assessments.

Critical strengths
  1. The paper addresses an important problem in UI-to-Code evaluation. Its motivation is clear and compelling, particularly given the limitations of relying solely on VLM-based judges.

  2. The paper proposes an automated benchmark construction pipeline and introduces a challenging benchmark, providing useful tools and resources for future research.

  3. The evaluation is comprehensive, covering ten models, multiple automatic metrics, three visual judges, and three human evaluators.

Critical weaknesses
  1. Round-trip consistency does not guarantee faithful reconstruction: incorrect outputs may receive high PQ Scores, while visually accurate outputs with different DOM structures may score poorly. Despite supporting human evaluations and correlation analyses, the frequency of these errors and their impact on sample selection remain unclear.

  2. It is unclear whether low PQ Scores identify broadly challenging UI-to-Code tasks or primarily reflect model-specific generation instability. For example, Figure 3(d) receives a PQ Score of 7.62 despite showing a relatively conventional page layout. While the reconstruction contains visible errors, the figure does not show the P–Q DOM differences responsible for the low score. Analyzing these differences and testing whether the same samples remain challenging for independent models would help clarify what kind of difficulty PQ captures.

  3. My main concern is the practical utility of PQ Score for real-world data filtering. Since the pipeline also uses complexity-based and manual filtering, the final benchmark’s difficulty does not isolate PQ’s contribution. Controlled comparisons with random and complexity-only selection, reporting difficult-sample yield and computational cost, would better establish its practical value.

Questions for authors

See Weaknesses.

Reviewer_P2qmOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents ABPQ, a benchmark for evaluating screenshot-to-HTML generation on challenging real-world webpages. It introduces PQ Score, a round-trip consistency metric based on the tree edit distance between generated DOM structures, to identify difficult samples. A hierarchical filtering pipeline is used to construct ABPQ-FULL and ABPQ-HARD, containing 500 and 200 samples, respectively. Experiments across multiple models suggest that the resulting benchmarks provide greater difficulty and discrimination than existing datasets.

Critical strengths
  1. The paper addresses a relevant problem: existing screenshot-to-HTML benchmarks may not adequately distinguish the capabilities of advanced models.
  2. The automated construction pipeline reduces manual annotation effort and provides a practical approach to collecting challenging evaluation samples.
  3. The evaluation covers multiple models and metrics, offering useful empirical evidence of the benchmark's difficulty.
Critical weaknesses
  1. Questionable validity of PQ Score. The proposed metric relies on DOM tree edit distance as a proxy for reconstruction difficulty. However, visually equivalent webpages can have substantially different DOM structures, making structural consistency an unreliable indicator of visual reconstruction quality.

  2. Insufficient justification of the difficulty criterion. Round-trip consistency does not necessarily imply reconstruction correctness. A model may consistently reproduce an inaccurate layout, while structurally different implementations may render nearly identical pages. This weakens the connection between PQ Score and the intended notion of task difficulty.

  3. Limited methodological novelty. The benchmark construction largely combines established techniques, including round-trip evaluation, tree edit distance, and hierarchical filtering. The main contribution is their application to screenshot-to-HTML benchmark construction rather than a fundamentally new methodology.

Questions for authors

How do the authors justify using DOM-based round-trip consistency as a measure of reconstruction difficulty, given that visually equivalent webpages may have very different DOM structures? In particular, how does PQ Score compare with direct visual similarity metrics in identifying genuinely challenging samples?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.