- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces MockWeb-Bench, a benchmark where agents try to reconstruct a working website. The benchmark consists of 50 offline reference websites and 4,101 evaluation pages, with desktop and mobile views. It evaluates the results on functionality, visual similarity, code quality, and structure. The authors test ten model-agent configurations and analyze their behavior.
Critical strengths
-
Agents can browse working websites while rebuilding them, checking how the pages look and how different features work. This tests both browsing and coding abilities, rather than just copying screenshots.
-
The evaluation is designed carefully. It checks whether reconstructed websites actually work, alongside their visual similarity. The authors also validate the functional tests on the original websites and evaluate ten model-agent configurations.
Critical weaknesses
-
The paper emphasizes interactive reference exploration as its main contribution, but provides limited evidence of what additional insights this setting offers compared to existing benchmarks. It also does not directly measure its added value, for example by comparing reconstruction performance with and without interactive exploration.
-
The authors check that the generated tests work on the original websites, but they do not show how much of each website's functionality the tests cover. Important interactions could still be missed.
Questions for authors
-
What additional insights does interactive reference exploration provide over existing benchmarks?
-
How much of the website's important functionality is covered by the final tests?
-
Since each system was tested only once per website, how reliable are the rankings, especially when the top systems have very similar scores?
3 comments
Response to Q3: Reliability of close rankings
We agree that close scores warrant careful examination of run-to-run variability. We selected GLM-5.3 and Claude Opus 5, whose reported mean scores are close, for an additional repeated-run comparison. Each configuration was independently evaluated three times on each of the same 50 websites, including the original evaluation. Each run started from a fresh workspace, with the same task settings, tools, and time budget.
| Configuration | Original Final (%) | Three-run mean Final (%) | Mean within-website SD (pp) |
|---|---|---|---|
| GLM-5.3 | 53.27 | 54.19 | 3.15 |
| Claude Opus 5 | 52.75 | 52.20 | 2.62 |
The paired mean difference between the per-website three-run averages is 1.99 percentage points, with a 95% confidence interval of [-5.32, 6.90]. The repeated-run means remain close to the original results, while the interval includes zero. We therefore characterize these configurations as having comparable performance and report their observed mean scores without claiming a statistically resolved ordering. The appendix now includes the repeated-run protocol, comparison table, and aggregate-score stability analysis.
Kind Reminder: Your Additional Thoughts on Submission 11898 Are Welcome
Dear Reviewer 3RvU,
Thank you very much for your valuable comments and for taking the time to consider our rebuttal. We understand the discussion period can be demanding on your schedule, and we would welcome the chance to clarify any remaining points before it closes on Nov 18.
We hope our responses have addressed your questions and concerns so far. If anything remains unclear, or if you have further concerns, we would be very glad to follow up and will do our best to clarify promptly.
Thank you again for your feedback; it has genuinely helped us improve the paper.
Warm regards, The Authors of Submission 11898
Overall Response
We appreciate the reviewer’s constructive feedback on the task setting and evaluation. Concretely, we have (i) clarified why recreation provides a verifiable task for studying sustained GUI–CLI coordination and highlighted the existing trajectory findings, (ii) added a reference-first component audit of 60 target pages from 30 websites, documenting behavioral checks for 44 of 48 reference-confirmed core groups, and (iii) included the repeated-run protocol and close-configuration comparison for GLM-5.3 and Claude Opus 5. These explanations, audit criteria, and comparison tables are now documented in the manuscript and appendix.
We appreciate the opportunity to clarify the hybrid capability this task is designed to study. Our objective is to analyze sustained GUI–CLI coordination. Many real-world agent tasks, including office workflows, data processing, and application use, combine GUI observation and interaction with CLI file operations, program execution, and automation. We study how agents gather information, choose GUI or CLI actions, and use observations and execution feedback to guide subsequent work.
We use website recreation as a verifiable task for this analysis: the running reference supplies explicit ground truth for appearance and supported frontend behavior. The reference anchors evaluation of the implementation, while development trajectories show how GUI exploration and CLI work are coordinated. Compared with the fixed target materials in Vision2Web and WebCompass, our setting lets agents choose which pages, visual details, responsive states, and interactions to inspect throughout development.
Our analyses reveal three findings:
This evaluation purpose is explicit in the Introduction and task definition, and the analysis section presents these findings and their implications for hybrid coordination.
Thank you for highlighting coverage of important interactive components. We conducted a reference-first component audit of 60 target pages from 30 websites spanning all ten categories. Core components were identified independently of the tests by their role in primary navigation, content access, or principal frontend operations. Shared widgets and sibling tabs or FAQ entries are grouped, while their instances and behavior variants remain explicit. Coverage requires a test to operate the component and check its observable response.
The existing fixed suite checks the defining operations of most reference-confirmed core component groups in this audit:
The four localized omissions concern command copying, installation-method navigation, a product-information tab group, and a FAQ accordion. The existing evaluation jointly assesses content and navigation, tested interaction outcomes, visual fidelity, quality, and structure, providing a broad assessment of overall reconstruction alongside this substantial core-component coverage. The appendix now reports the audit design, coverage results, and their interpretation.