- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The proposed VENUS improves generated websites by teaching models what good design looks like. It defines a ten-dimension design standard, starts with designer-approved pages, and deliberately introduces visual defects to create comparison pairs. These pairs train a small visual judge, which filters web-code training data for a generation model. The resulting VENUS-CODER improves its base model’s ArtifactsBench score from 53.50 to 64.52; designers prefer its pages in 77.6% of majority-decided comparisons.
Critical strengths
This work has low labeling effort. Designers review only a small set of seed pages. Making each one deliberately worse in a single design dimension produces "good vs. worse" pairs automatically. The pair also records which dimension is worse and how severe the defect is.
Across all three tables in experimental section, the results are consistent: the 7B judge matches frontier judges with the highest order consistency (Table 1), the screened corpus lifts the same base by +11.02 on ArtifactsBench to rank first among frontier systems (Table 2), and the size-matched ablation shows the gain comes from screening rather than data volume (Table 3).
Critical weaknesses
This paper used senior designers in multiple places, i.e., judge study, generator study, design standard, and seed page review. however, it doesn't specify whether they are the same designers. This matters for independence. If the evaluators also helped write the standard or select the seed pages, their agreement with the judge could be inflated.
On page 18, Fig A.2, the very different pages get the same score. Task 3 has a full sidebar and header but an empty chart. Task 9 has a laid-out page but a blank board. Task 19 is only a spinner. All three get 2/5, so the scorer doesn't separate "mostly built" from "nothing rendered." That weakens trust in the appearance metric. Is there any other examples for 0,1,3,4 scores? Please also include randomly sampled or median-gap examples in which both pages render fully.
The design standard VENUS-CANON is used at every step, i.e., to make the training data, to train the judge, and to build the pairs that designers check. So the results may only show that the system follows its own rules, not that it captures good design.
VENUS-CANON rewards tidy, conventional, mostly Western-style design, so it may push outputs toward safe, look-alike pages. It may also penalize bold or culturally different styles, such as the dense layouts common on East Asian sites. The paper doesn't measure output diversity or report the designers' backgrounds.
Questions for authors
Please clarify whether the senior designers in the two blind studies are the same individuals, and whether any of them participated in writing VENUS-CANON or screening seeds; overlap would weaken the independence of the human validation.
For Table 3, is it possible to compare with a VLM filtered subset? a stronger alternative filter - not necessarily frontier VLM as the cost is high.
It's supposed to only affect appearance, why table 2 shows it gains a lot in func. as compared to the base mode? it's mentioned in limitation as a by-product but needs more explanation.
4 comments
Reply to Reviewer 3QEx(2/2)
(W4) Our presentation did not make the distinction between style choices and design criteria sufficiently clear. Tidy, conventional Western-style layouts and information-dense East Asian layouts represent different visual approaches. VENUS-CANON evaluates hierarchy, alignment, spacing, colour roles, component coherence, and readability within the chosen approach (Appendix A).
A dense page can receive a high score when its information is legible and well organized; a sparse minimalist page can still have unclear hierarchy or inconsistent components. Figure A9 in Appendix H shows an information-dense dashboard with navigation, summary cards, and tables, receiving 4 / 4 for the base and VENUS-CODER pages. This illustrates that density alone does not imply a low score.
The additional examples in Figures A6–A9 of Appendix H cover editorial layouts, portfolios, utility interfaces, and information dashboards. Figures A12–A14 of Appendix J further illustrate photographic layouts, expressive typography, geometric imagery, and product interfaces. These cases show how the criteria can be applied across different styles and information densities.
(Q2) We conducted the suggested comparison with a general-purpose VLM filter. It examines whether a VLM without VENUS-CANON training can score and rank candidates, select a corpus of the same size, and yield comparable generation quality.
We use Gemini 3.5 Flash to score each page in the same candidate pool, rank the pages, and retain the top subset at the same ratio as the VENUS-JUDGE corpus. We then fine-tune the same base model with the same training settings and evaluate the generated pages under the same protocol:
| Selection method | ArtifactsBench / 100 | WebGen functional accuracy, original protocol | WebGen appearance / 5 |
|---|---|---|---|
| Gemini 3.5 Flash score-and-rank selection | 59.48 | 41.4% | 3.35 |
Under the same candidate-source and corpus-size conditions, this general-purpose VLM filter does not match VENUS-JUDGE's selection results. The comparison supports the value of the selection procedure built from VENUS-CANON. This is an additional experiment reported in the rebuttal.
(Q3) SFT learns complete HTML/CSS/JavaScript implementations, so the selected pages carry supervision beyond isolated visual styling. A page with complete content, usable controls, and working interactions is also a more complete code example. Conversely, empty charts, blank dashboards, or persistent loading spinners often reflect incomplete implementations.
Although VENUS-CANON primarily addresses visual quality, its interaction and motion dimensions require appropriate responses to user actions, and its component dimension requires key controls such as buttons and navigation to render correctly. Candidates must also pass the interaction and motion checks before entering the corpus (Section 3.5). These requirements help explain why the selected training examples can improve functional performance as well as appearance.
Thank you for the clarifications and the additional exp:
W1 (Resolved)
W2 (Partially resolved): The corrected screenshot and additional examples help, but I would still like to see randomly sampled, fully rendered pages with the same score.
W3 (Largely resolved): The external WebDevJudge results show that the judge learns more than its own rubric. The matched Gemini baseline also strengthens the case for screening, though the contribution of the interaction checks to the functional gains remains unclear.
W4 (Still open): The examples show that VENUS can produce different styles, but not whether training makes less conventional styles less common.
I will keep my original score given the open W4
Reply to Reviewer 3QEx
Thank you for carefully reviewing the clarifications and additional experiments, and for recognizing W1 as resolved and W3 as largely resolved. We further clarify the design differences among equally scored pages, the role of design judgement, and the evidence for stylistic diversity.
(W2) Randomly sampled pages with the same score. The new Appendix N presents four randomly sampled, previously unshown WebGen-Bench tasks: a student portal, a children's recycling game, online polls, and stock reports. Both the base and VENUS-CODER receive an appearance score of 5/5. The screenshots allow comparison of composition and information organization within the same score category.
(W3) Design judgement and the dynamic check. The dynamic check evaluates interaction and motion, while VENUS-JUDGE assesses design quality across eight static dimensions, including layout, typography, colour, and components. A page responding to user actions does not ensure well-aligned navigation, clear hierarchy, or coherent components. The designer study in Appendix E also supports this design judgement capability: across pairs spanning the eight static dimensions, the judge agrees with independent designers on 77.2% of their decisive judgements on average.
The generation results provide complementary evidence. WebGen-Bench reports appearance separately, and Tables 2 and 3 show improved visual presentation. ArtifactsBench's per-task checklists provide additional evidence about implementation and presentation quality, while the Coder study in Appendix E confirms that professional designers see benefits in the generated pages. The existing screening comparisons evaluate the combined pipeline and do not separately quantify each check's contribution to generation gains. Our response therefore rests on the independent evaluation of design judgement and the visual improvement of the resulting pages.
(W4) Style and design constraints. We clarify two aspects.
(1) Style and design constraints serve different purposes. Style defines a page's visual language; VENUS-JUDGE focuses on hierarchy, alignment, spacing, readability, and component coherence within that style. Candidates are compared with their respective seeds, without requiring pages in different styles to share a palette or composition.
(2) Style diversity was an important consideration in seed construction. Seed-page production included minimalist, hand-drawn illustrative, soft-gradient, expressive, technology-oriented, and nature-inspired styles. The new Appendix M (Figure A22) shows style examples from a subset of seed pages. For copyright and privacy reasons, it presents reduced views of a subset of seed pages. This shows that stylistic coverage was an explicit choice at the seed-construction stage, with candidates subsequently screened for design quality against their respective seeds. The photography, expressive typography, and geometric compositions in Appendix J further illustrate the model's use of different visual languages.
Reply to Reviewer 3QEx(1/2)
Thank you for your careful review and for recognizing the low annotation cost and the value of the corpus-size-matched ablation. We clarify the evaluators' independence, correct the screenshot presentation, and address the concerns about design standards and functional gains below.
(W1/Q1) Standard development, seed screening, and final human evaluation were conducted by distinct teams. One group of designers developed VENUS-CANON. A separate team of 1 professional designer and 6 professional evaluators familiar with VENUS-CANON screened the seeds. The two human studies—agreement with VENUS-JUDGE and comparison of VENUS-CODER's generated pages—were conducted by the same independent group of designers. None of that group participated in developing the standard or screening the seeds. We clarify these responsibilities and relationships here.
(W2) We identified an error in the displayed screenshot for Task 19 and apologize for the confusion. It has been corrected in Figure A2 of Appendix C (Page Comparisons). We also provide examples across score levels and visual styles to make the evaluator's distinctions easier to inspect.
WebGen-Bench appearance ratings use a 1–5 scale. Figure A15 in Appendix K (Additional Page Comparisons) includes:
These examples span scores from 2 to 5 and include a case where VENUS-CODER scores below the base model. Additional comparisons with equal scores and different styles appear in Figures A6–A9 of Appendix H.
(W3) We understand the concern that using the same standard to construct supervision, train the judge, and build internal evaluation pairs might demonstrate conformity to that standard rather than broader design quality. Three distinctions are relevant.
First, a shared design standard does not imply a single visual style. Style determines the visual language, such as editorial, minimalist, or neon treatments. Design criteria assess the execution within that style: hierarchy, spacing, alignment, readability, and component coherence. VENUS-CANON focuses on these criteria without requiring one fixed style. The seeds cover multiple styles so that the criteria can be exercised across different visual languages.
Second, VENUS-CANON is grounded in professional design practice. Its ten dimensions and six grade anchors were developed through repeated reviews of generated pages with professional interface designers. We organized recurring judgements into properties that can be checked from a rendered page and interpreted consistently by another reviewer (Section 3.1).
Third, external benchmarks and designer assessments examine both judge performance and generated-page quality. VENUS-JUDGE is evaluated on WebDevJudge, an independent benchmark whose preferences come from human comparisons of independently sourced pages. Those pages were not generated under VENUS-CANON, and the pairs were not constructed using our degradation operators. VENUS-JUDGE achieves Strict and Consistency scores of 55.6 and 93.2, compared with 40.8 and 71.2 for the same untrained backbone (Table 1). This supports improved agreement with external human preferences and greater stability under presentation-order swaps.
For VENUS-CODER, we evaluate generated pages on ArtifactsBench and WebGen-Bench and ask designers to compare the outputs directly. The human evaluation setup and results are provided in Appendix E (Alignment with Human Judgement). Together, these evaluations provide evidence at both the preference-judgement and page-generation stages that the design supervision can improve page quality.
We also recognize the scope of this evidence. Clear hierarchy and coherent components are widely useful design properties, but they are not the only principles under which a design can succeed. VENUS-CANON is an executable standard for screening web-design quality, rather than a universal definition of good design.
The origin of the standard is described in Section 3.1, and the distinction between WebDevJudge and the internally constructed evaluation is described in Sections 4.1–4.2. We further clarify the above limitation in this response.