acceptodds
Under review as a conference paper at ICLR 2027

WebViSE: Scaling Visual Design Intelligence from the Open Web for LLM Website Generation

Abstract

Large language models have made remarkable strides in website development, enabling complete and interactive web experiences to be built from natural-language instructions with far less effort. Yet, non-expert users often struggle to articulate sophisticated visual and interaction designs in natural language. Meanwhile, the open web already contains a vast reservoir of such design knowledge, embodied in mature website implementations. To this end, we introduce **Web**site-derived **Vi**sual **S**kill **E**xtraction (**WebViSE**), a framework that automatically abstracts these implementations into reusable Visual Skills and builds a structured Skill library. To scale the library, WebViSE organizes its Skills in a multi-parent DAG and jointly evolves Skill descriptions and DAG structure using Skill-conditioned Potential Queries, enabling efficient matching between diverse user instructions and relevant Skills. The auto-evolved structure outperforms non-hierarchical LLM retrieval baselines at only 39.09% of the estimated retrieval cost with Qwen3.8-Max. For any models, WebViSE can be directly injected as plug-and-play guidance, enabling diverse models to produce more visually compelling and aesthetically refined web pages. For open-source models, WebViSE can be used to build a data-synthesis pipeline that internalizes visual design intelligence into model weights. Experiments across multiple web development benchmarks and human evaluation consistently show that WebViSE improves the visual quality and interaction fidelity of artifacts. Overall, our work establishes an automated path for extracting, organizing, and transferring design intelligence from the open web to website-building models. We release all extracted Visual Skills, the associated Skill DAG, and our code at https://github.com/Web-Skill-DAG/Web_SKill_DAG.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Web Generation, Skill Library, Structured Retrieval

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_cezxOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents WebViSE, a framework that learns reusable visual design skills from existing websites. These skills are organized into an evolving multi-parent DAG and used to improve website generation models through retrieval-augmented prompting or supervised fine-tuning. The paper evaluates the framework across seven models and three benchmarks.

Critical strengths

The paper addresses an interesting and practically relevant problem. Learning reusable design and interaction patterns from real websites seems like a promising way to improve LLM-based website generation.

The system is fairly comprehensive, and the evaluation across several models and benchmarks is a strength. In particular, the comparisons with generic and randomly selected skills help assess whether the extracted skills provide meaningful benefits.

The DAG evolution ablations also offer some evidence that refining the structure improves retrieval. The human evaluation and skill-augmented finetuning experiments further broaden the evaluation.

Critical weaknesses

My main concern is the novelty. Prior work, including SkillRouter and SkillDAG, has already explored skill extraction, structured retrieval, evolving skill libraries, and skill-based model improvement. Applying these ideas to visual website design is interesting, and the query induced DAG refinement is a useful extension. However, much of the framework appears to combine existing ideas rather than introduce a clearly distinct methodological contribution.

I find the empirical gains less convincing than the presentation suggests. Many improvements are in the range of 0–2 points, and some models perform worse with the proposed approach. For example, Kimi K3 drops by 1.60 points in aesthetics, while GLM-5.3 drops by 2.79 points in WebGen-Bench accuracy. Similarly, skill-augmented fine-tuning improves aesthetics and dynamic interaction but comes at the cost of some functionality metrics. These trade-offs deserve more attention in the discussion.

The human evaluation is useful, but each of the 350 comparisons is judged by only one evaluator making it impossible to assess inter-rater reliability. The retrieval-cost result also needs to be interpreted carefully. The reported 39.09% is relative to an expensive flat-LLM baseline and does not include the offline costs of extracting skills or evolving the DAG. Finally, while the released repository is valuable, it excludes the website-generation code and the generated training and evaluation corpora which limits full reproducibility.

Questions for authors
  1. Can the authors provide repeated-run results or statistical significance tests to show whether the relatively small benchmark gains are reliable?

  2. How much of the improvement comes from the evolving DAG itself? How does it compare with a simpler embedding based retriever using the same skill library and token budget?

  3. Can the authors clarify the discrepancy between the DAG statistics reported in Table 12 and those in the publicly released DAG?

  4. What is the total cost of building and maintaining the skill library, including skill extraction, query generation, and DAG evolution?

3 comments

Dear Reviewer cezx,

As the discussion period is drawing to a close, we would like to kindly follow up on our rebuttal and ask whether our additional experiments and clarifications have addressed your concerns. We sincerely appreciate your time and thoughtful feedback. We would welcome the opportunity to clarify any points that remain unclear and provide any further information that would be helpful for your final assessment.

Thank you again for your consideration.

Sincerely,

The Authors

The rebuttal addresses most my concerns, therefore, I would lean towards acceptance. Weak Accept

We are very encouraged by your response and sincerely appreciate your support. Thank you for taking the time to return to our paper and consider the rebuttal so carefully. We especially appreciate how clearly you explained your concerns in the initial review. This gave us a concrete basis for improving the paper, and it is rewarding to know that our responses have addressed most of those concerns. We have learned a great deal from this exchange. Thank you once more for the care and expertise you have shared with us throughout the review.

Reviewer_FXF4OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces WebViSE, a framework for extracting reusable visual and interaction mechanisms from existing websites and transferring them to LLM-based website generators. The framework constructs Visual Skills from source website implementations, organizes them in a multi-parent directed acyclic graph (DAG), and jointly evolves Skill descriptions and graph structure using Skill-conditioned Potential Queries and retrieval-failure diagnosis. The resulting Skills can be used as plug-and-play guidance for website generation or as a source of training data for supervised fine-tuning. Experiments cover seven website generators, three website-generation benchmarks, two Skill-retrieval benchmarks, and a blinded human preference study. The results suggest that WebViSE can improve visual and interactive website quality while reducing LLM-based retrieval costs. However, the novelty of the individual components, the reliability of the evaluation protocols, and the consistency of the reported improvements require further examination.

Critical strengths
  1. The paper addresses a practical and important problem by extracting transferable visual and interaction mechanisms from existing websites rather than relying solely on generic design instructions.
  2. The multi-parent Skill DAG provides a reasonable representation for Skills that serve multiple visual-design roles, while query-induced evolution offers a systematic approach to refining both retrieval structure and Skill descriptions.
  3. The evaluation spans seven website generators and three complementary benchmarks, covering functional correctness, visual quality, and dynamic interaction rather than relying exclusively on static screenshots.
  4. The ablation study compares WebViSE with general and randomly selected Skills, providing evidence that the applicability of retrieved guidance is important for successful transfer.
  5. The paper explores both inference-time Skill augmentation and Skill-guided supervised fine-tuning, demonstrating the potential for transferring visual design knowledge through complementary deployment strategies.
Critical weaknesses
  1. The methodological novelty is not fully established, as the framework primarily integrates existing ideas from agent Skills, hierarchical retrieval, LLM-based evaluation, graph editing, and supervised fine-tuning, while the unique contribution of the proposed joint evolution mechanism remains insufficiently isolated.
  2. The improvements in website-generation quality are inconsistent across models and metrics, with some configurations exhibiting regressions, including a 2.79-point decrease in WebGen-Bench accuracy for GLM-5.3 and a 1.60-point decrease in Cookie-Bench aesthetics for Kimi K3.
  3. The Skill-retrieval evaluation relies on 850 source-derived queries and an external benchmark of only 119 queries, leaving uncertainty about generalization to a broader distribution of unseen website requests and design requirements.
  4. The external retrieval relevance labels are primarily established through agreement among LLM judges, with human review for unresolved cases, but the paper provides insufficient evidence of annotation reliability and agreement with independent human judgments.
  5. The human preference evaluation uses four evaluators, with each website pair receiving only one preference judgment, making it difficult to assess inter-rater reliability and the statistical significance of the reported preference margins.
  6. The Skill-internalization experiment is limited to one open-weight model and yields mixed results: WebViSE-SFT improves some visual and interaction metrics over NoSkill-SFT but reduces WebGen-Bench accuracy and Cookie-Bench functionality.
  7. The reported retrieval-cost reduction is based on estimated LLM token costs and does not fully characterize the end-to-end cost of constructing, evolving, maintaining, and deploying the Skill library as its scale increases.
Questions for authors
  1. Can the authors isolate the contribution of joint Skill–DAG evolution from that of ordinary DAG-based retrieval, query expansion, and LLM-based reranking under matched token budgets?
  2. How well does WebViSE generalize to entirely unseen website domains, design styles, and user requests that are not closely related to the source websites used to construct the Skill library?
  3. Can the authors provide confidence intervals or statistical significance tests for the benchmark improvements and blinded human preference results, together with inter-rater agreement measurements?
  4. How were the external Skill-utility labels validated against independent human judgments, and what level of agreement was observed between human annotators and the LLM judges?
  5. Why does Skill-guided fine-tuning improve visual and dynamic quality while degrading functional correctness on some benchmarks, and can additional experiments establish whether this trade-off is consistent across models and training configurations?
  6. Can the authors report the total offline cost of Skill extraction and DAG evolution, along with amortized costs under different numbers of downstream website-generation requests, to better substantiate the scalability claims?
Reviewer_bFztOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents WebViSE, a framework for extracting and transferring reusable visual design skills from existing websites. It constructs source-grounded skills that capture visual and interaction mechanisms, organizes them into a dynamically evolving multi-parent Skill DAG, and retrieves relevant skills to guide website generation. The paper further explores skill internalization through supervised fine-tuning. Experiments across multiple models and benchmarks demonstrate improvements in website generation quality.

Critical strengths
  1. Clear motivation. The paper identifies a meaningful limitation of existing website generation methods: their difficulty in reproducing sophisticated visual and interaction mechanisms. Extracting reusable design knowledge from real websites is a reasonable and practical direction.

  2. Well-developed framework. The proposed pipeline covers skill extraction, organization, retrieval, and transfer. The overall design is coherent, and the paper provides detailed descriptions of the individual components.

  3. Comprehensive evaluation. The experiments cover multiple models and benchmarks, including retrieval evaluation, ablation studies, and skill-augmented fine-tuning. The results provide encouraging evidence of the framework's effectiveness.

Critical weaknesses
  1. Limited methodological novelty. Several components, including skill extraction, graph-based retrieval, and skill evolution, build on established approaches. Related work such as Graph-of-Skills has explored similar skill organization and retrieval challenges. The main novelty lies in their adaptation and integration for visual website generation.

  2. Insufficient justification of the Skill DAG. While the proposed multi-parent DAG supports flexible skill organization, its advantages over simpler retrieval approaches or existing graph-based skill retrieval methods are not fully established, particularly when accounting for retrieval cost and system complexity.

  3. Limited evidence of cross-website generalization. Although the experiments demonstrate improvements in generation quality, it remains unclear to what extent the extracted visual mechanisms can generalize to substantially different websites and previously unseen combinations of interactions.

Questions for authors

What are the main advantages of the proposed evolving Skill DAG over existing graph-based skill retrieval methods, such as Graph-of-Skills, when evaluated under comparable retrieval budgets and using the same skill library?

1 comment

Dear Reviewer bFzt,

As the discussion period is nearing its end, we would like to follow up on our response to your review. We sincerely appreciate your careful evaluation and hope that the additional comparisons and clarifications have addressed your concerns. We would be grateful for any further feedback and would welcome the opportunity to clarify any points that would be helpful for your final assessment.

Thank you again for your time and consideration.

Sincerely,

The Authors

Comments on the paper

2 comments
Area_Chair_2A5hOpenReview ↗

AC Note: Rebuttal Received, Please Engage in Discussion

Dear authors and reviewers,

Thank you to the authors for their detailed rebuttal and for the time and care they put into addressing the reviewers' comments. I appreciate the additional clarifications and results.

Reviewers: please read the authors' responses carefully and reply to them before the end of the discussion period. Even a short acknowledgment is valuable. It is especially helpful to say whether your concerns have been addressed and, if not, what remains unresolved. If the rebuttal changes your assessment, please update your score and review accordingly, whether up or down. If your score stays the same, a brief explanation of why helps me and the senior area chairs.

Authors: I will follow the discussion closely and take your rebuttal fully into account when forming my recommendation.

Thank you all for your time and effort.

Best regards, Your Area Chair

AuthorsOpenReview ↗

General response to the AC and all reviewers

Dear Area Chair and Reviewers,

We sincerely thank the Area Chair and all three reviewers for engaging so thoughtfully with our work. Your thoughtful questions have shaped this revision and prompted us to examine our work more carefully. We summarize the main additions and clarifications below.

  • Methodological contribution and the role of evolution (Reviewer cezx, Reviewer FXF4, Reviewer bFzt). We clarified how WebViSE extracts transferable visual and interaction mechanisms and organizes them by overlapping design roles. Its DAG edges represent design-family membership, while query-induced evolution jointly refines Skill descriptions and retrieval routes. The editing ablations attribute most retrieval and efficiency gains to DAG refinement, with Skill refinement further improving retrieval quality.

  • Controlled downstream comparisons (Reviewer cezx, Reviewer FXF4, Reviewer bFzt). We added website-generation experiments with Base, Dense Retrieval, the Initial DAG, and the Evolved DAG for Claude Opus 5 and Qwen3.8-27B. Under the same Skill-context token budget for generators, the full WebViSE retrieval pipeline outperforms Dense Retrieval across all reported metrics for both models. Evolution also improves most downstream metrics over the Initial DAG, linking retrieval gains to generation quality. We report all seven benchmark metrics for these four settings in Appendix E.4 and Table 15 of the revised manuscript.

  • Statistical reliability of benchmark improvements (Reviewer cezx, Reviewer FXF4). We added comparison-specific 95% confidence intervals and two-sided tests for all model–metric comparisons. Most positive differences reach the nominal p < 0.05 threshold, including all seven metrics for GPT-5.6-Sol and Qwen3.8-27B. Gains also occur where baseline scores are already high, supporting WebViSE’s value for strong generators. We report mean differences, 95% confidence intervals, and two-sided p-values for all model–metric comparisons in Appendix D.2 and Table 11 of the revised manuscript.

  • Expanded blinded human evaluation (Reviewer cezx, Reviewer FXF4). We expanded the evaluation to three independent evaluators per website pair, covering 350 pairs with 1,050 judgments. Preference margins favor WebViSE across all seven models, ranging from +20.00 to +33.33 percentage points, with each reaching the nominal p < 0.05 threshold. Fleiss’ κ ranges from 0.7100 to 0.8220. Confidence intervals and preference tests treat each website pair and its three judgments as one sampling unit. We report human preference margins, confidence intervals, p-values, Fleiss’ κ, and unanimous agreement rates in Appendix C.5.1 and Table 9 of the revised manuscript.

  • Independent validation of Skill-utility labels (Reviewer FXF4). Three human annotators assessed 50 randomly sampled External Query–Skill pairs without seeing the original LLM labels. The human median agrees with the LLM consensus on 86.0% of pairs, with linearly weighted Cohen’s κ of 0.825, supporting the reliability of these labels. We report exact agreement rates, linearly weighted Cohen’s κ, Fleiss’ κ, and their confidence intervals in Appendix E.1.1 and Table 12 of the revised manuscript.

  • Cost accounting and reuse (Reviewer cezx, Reviewer FXF4, Reviewer bFzt). We separated offline construction and evolution costs from online retrieval and reported amortized costs across request volumes. The reported 39.09% is the Evolved DAG’s online retrieval cost relative to Flat LLM under Qwen3.8-Max. Relative to the Initial DAG, the online savings offset the Qwen3.8-Max evolution cost after approximately 3,700 requests. We report input/output token usage and estimated offline costs for library construction and DAG evolution in Appendix E.6 and Table 17 of the revised manuscript.

  • Generalization and performance trade-offs (Reviewer cezx, Reviewer FXF4, Reviewer bFzt). We clarified the independent origins of the External requests and downstream benchmarks, distinguishing cross-source transfer from source–Skill recovery. We also discussed qualitative observations concerning visual density, scrolling rhythm, and mobile adaptation to better explain the observed trade-offs, including those in Skill-guided fine-tuning.

We hope WebViSE offers the community a useful foundation for studying how website generators can learn reusable design knowledge from the open web. Your feedback has helped us move toward that goal. We also deeply appreciate the Area Chair’s time and care in guiding the review process. The additional results are in the revised paper, and we would be glad to discuss any point further. We hope they will be helpful for your final assessment. Thank you again for your careful reading, your expertise, and your generous attention to our work.

With sincere gratitude,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.