acceptodds
Under review as a conference paper at ICLR 2027

UIReq-Bench: Benchmarking MLLMs for Industrial UI Code Generation Beyond Visual Fidelity

Abstract

A key challenge for multimodal large language models (MLLMs) is moving beyond visual recognition to constraint-aware cross-modal reasoning. This involves combining visual cues with information from other modalities to understand elements’ relationships under domain-specific rules. This challenge is acutely evident in industrial design-to-code (D2C), which converts user interface (UI) designs into code and requires MLLMs to connect design images with disorganized layer metadata, infer component and layout implementation requirements, and realize them in code under target-library constraints. However, these capabilities remain insufficiently evaluated in realistic industrial settings. To fill this gap, we present UIReq-Bench, a benchmark for evaluating MLLMs’ ability to generate UI code that satisfies implementation requirements in industrial applications. The dataset consists of 2,861 production designs from 17 commercial platforms with 97,652 expert annotations across four categories: Component, Group, Alignment, and Position. These annotations distinguish required constraints from permitted implementation choices. UIReq-Bench defines three tasks: end-to-end UI code generation, requirement inference, and requirement realization. Evaluating eight MLLMs reveals substantial gaps in generating UI code that satisfies implementation requirements, alongside distinct performance profiles in inference and realization. We further show that MLLMs’ visual reconstruction ability does not necessarily imply an ability to generate code that meets these requirements. We release UIReq-Bench to support research on industrial UI code generation.

Primary area:
datasets and benchmarks
Keywords:
Multimodal Large Language Models, Design-to-Code, UI Code Generation, Implementation Requirements, Benchmark

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_rCN4OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

UIReq Bench evaluates whether generated UI code meets component and layout requirements. It contains 2861 annotated industrial designs and evaluates eight models on 421 designs. Separate tasks examine requirement inference and code generation with or without supplied requirements.

Critical strengths

The dataset captures implementation requirements that visual similarity scores can miss. The distinction between required and optional annotations also allows some flexibility in implementation. Supplying requirements improves code scores substantially while changing visual scores much less. This is a useful finding for evaluating UI generation.

Critical weaknesses
  1. The paper does not quantify agreement between the automatic evaluator and independent human judgments. It is unclear how often valid alternative implementations receive incorrect penalties.
  2. The evaluation comes from one commercial ecosystem and reports no repeated-generation results or confidence intervals. The stability of close model rankings is therefore unclear.
Questions for authors

How closely do automatic scores agree with expert judgments, including cases with valid alternative implementations? Are the main findings and model rankings stable across design collections and repeated generations?

Reviewer_WuxxOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

UIReq-Bench evaluates whether multimodal models generate UI code that satisfies implementation requirements, as distinct from visual similarity. The resource pairs production designs from multiple commercial platforms with layer-level expert annotations in four categories: Component, Group, Alignment, and Position,each marked required or optional. Three tasks share that reference: end-to-end generation, requirement inference without code, and generation when the expert requirements are supplied. On a frozen evaluation subset, current models are substantially stronger on component type and grouping than on alignment and positioning. Supplying the expert requirements improves every code metric, while visual fidelity barely moves. The central empirical claim is that a close render can still use the wrong component, grouping, alignment, or positioning.

Critical strengths
  1. The required/optional annotation design addresses a real flaw in prior evaluation. Required entries define what the code must satisfy, and optional entries record permitted alternatives that can match without being demanded. This avoids penalizing valid implementations that differ from a single reference, such as space-between versus an auto margin. Evaluation is no longer limited to visual similarity or aggregate style heuristics. The industrial scale (thousands of production designs across many platforms, with layer-level expert annotations) is a resource that comparable design-to-code benchmarks lack.

  2. The three-task decomposition gives diagnostic findings. Separating end-to-end generation, requirement inference, and realization with supplied requirements shows where models fail. Alignment is weak in both inference and realization, while a model can rank low at inferring a requirement and high at realizing it. Giving the expert requirements improves every code metric, while the visual scores barely move. This supports the paper's main message that visual fidelity is not a proxy for requirement satisfaction. The evaluation subset was frozen at collection level before model inference.

Critical weaknesses
  1. Scorer validity and annotation reliability are not demonstrated. All scores come from an automatic matcher that links annotations to predictions or to rendered DOM nodes using type aliases and an IoU and area-ratio threshold. I do not see a human audit of the matcher on model outputs, so part of the reported gap could be matcher error. The reference comes from two engineers with one reviewer and no inter-annotator agreement. For a benchmark whose main claim is "visually correct but requirement-violating", the reliability of both the labels and the scorer is the main evidence.

  2. The scope and category balance limit the conclusions. The designs come from one e-commerce ecosystem, one design tool, and its target component libraries, so it is unclear whether the findings are about industrial design-to-code or about this ecosystem. Component entries are almost all required, while optional entries are concentrated in position and alignment. The "multiple valid implementations" argument therefore rests on the smaller categories. Alignment and position also have far fewer required entries than group. Only general-purpose MLLMs are evaluated, and no specialized design-to-code systems are run, so the benchmark does not say whether the gap persists for methods built for this task.

Questions for authors

On a sample of T1/T3 outputs, what is the agreement between the automatic scorer and a frontend engineer for type, props, group, alignment, and position? A high agreement would remove my main soundness concern.

Reviewer_X1Z9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

UIReq-Bench evaluates whether multimodal models can generate UI code that satisfies implementation requirements concerning components, grouping, alignment, and positioning. The dataset contains 2,861 production designs and 97,652 expert annotations, distinguishing required choices from permitted alternatives. It combines design images, layer metadata, and target component-library documentation.

The benchmark separates end-to-end generation, requirement inference, and realization with supplied expert requirements. Eight models are evaluated on 421 designs. The results indicate that supplying requirements substantially improves implementation scores while changing visual similarity relatively little.

Critical strengths

The novelty is primarily in benchmark design and annotation methodology.

In benchmark design, evaluating implementation requirements effectively addresses a key limitation of screenshot-based benchmarks, while separating inference and realization enables far more targeted analysis than aggregate end-to-end scores alone.

In dataset construction, I acknowledge the authors‘ effort in spanning layer-level specifications, component-library context, and third-party expert reviews, elevates this dataset well beyond a simple collection of UI screenshots.

Critical weaknesses
  1. Section 3.1 identifies MasterGo files from product teams within one e-commerce ecosystem. Figure 2 shows logos including Taobao, Tmall, Youku, and Alipay, but the paper does not provide a complete platform breakdown or language and regional statistics. Multiple platforms within one ecosystem may share design conventions and component libraries. The results therefore do not establish broad industrial generalization.

  2. A screenshot and layer tree may admit several reasonable container structures or layout strategies. The paper needs stronger evidence that its required annotations represent genuine constraints rather than one expert-approved implementation style.

  3. Geometric thresholds, component aliases, and structural matching rules require comparison against independent human judgments. Valid implementations could be penalized through imperfect correspondence or incomplete alternative annotations.

  4. The benchmark measures selected structural requirements at a rendered configuration. It does not directly test whether higher scores imply better resizing behavior, interaction correctness, or maintainability. These limits should constrain the claims.

Questions for authors
  1. Provide per-platform, language, region, and component-library statistics, including how the five evaluation collections map to the 17 platforms.

  2. What agreement do independent frontend engineers achieve before adjudication, particularly on required versus optional grouping and positioning choices?

  3. Have you manually audited the scorer on generated implementations, including valid alternatives? What are its false-positive and false-negative rates, and how sensitive are conclusions to the geometric thresholds?

  4. Do the main findings hold on a held-out platform, component library, or ecosystem with substantially different language and design conventions?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.