acceptodds
Under review as a conference paper at ICLR 2027

ViRE-Bench: Benchmarking Repository-Level Frontend Code Editing from Visual Feedback

Abstract

Frontend editing requests often concern visible content, appearance, and spatial relationships. Annotated screenshots ground these requests directly in the rendered interface, allowing users to communicate changes to coding agents without identifying source files or components. Agents must then infer the intended edit, locate the relevant repository code, and implement the change while preserving unrelated content and appearance. We introduce ViRE-Bench, a benchmark of 1,000 repository-level frontend code-editing tasks drawn from 202 real-world HTML, React, Vue, and Angular repositories. The benchmark spans eight edit operations, covers both instance-level and shared-component edits that propagate across pages, and evaluates submission validity and intent satisfaction alongside rendered measures of target fidelity and preservation. Among six models evaluated under a common single-turn protocol, the best-performing model achieves an overall benchmark score of 67.29 and an intent satisfaction rate of 70.2%. Component movement and resizing remain challenging for all six models. Controlled diagnostic experiments show that explicit editing intent yields larger gains than source-location information alone across all three tested models, with two models reaching approximately 95% intent satisfaction under explicit intent. These findings indicate that deriving an actionable editing specification from visual feedback is a primary source of difficulty, while the benefits of source-location assistance vary across models.

Primary area:
datasets and benchmarks
Keywords:
visual feedback, repository-level code editing, frontend development, multimodal coding agents, annotated screenshots, benchmark evaluation

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_xruqOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces ViRE-Bench, a benchmark of 1,000 frontend code-editing tasks built from 202 real repositories across HTML, React, Vue and Angular. Each task gives a coding agent an existing repository and an annotated screenshot. The agent must infer the requested edit, locate the relevant code and implement the change, sometimes across several pages that share a component. The tasks cover eight edit operations. They are constructed from structured edit specifications, from which both the reference implementation and the visual feedback are generated. Evaluation combines validity and operation-specific intent checks with visual measures of target fidelity and preservation of unrelated content. Among six models, the best reaches an intent satisfaction rate of 70.2%.

Critical strengths

The paper targets a practical and underexplored setting, editing real multi-file repositories from annotated screenshots, which complements existing benchmarks on design-to-code generation and text-instructed editing. The benchmark is carefully constructed. The reference implementation and the visual feedback are generated from the same specification, external resources are frozen to reduce rendering variability, and unstable regions are identified through repeated captures. Scoring regions are also derived independently of agent outputs. The evaluation is thoughtfully designed: it separates validity, intent satisfaction, target fidelity and preservation, and its intent checks accept alternative implementations. A human study supports both the tasks and the metrics, with high agreement on intent judgments and a strong correlation with human quality ratings. The experiments are solid, covering six models on all tasks with detailed breakdowns by operation, technology and cross-page consistency.

Critical weaknesses

W1. Tasks and feedback are generated automatically and may differ from real user feedback. Edit targets and parameters are selected by Gemini-3.7-Flash (Sec. 3.2), and the annotations are drawn programmatically from the VES in a uniform style. The Limitations section acknowledges that performance on independently collected human feedback remains untested. Real users often annotate more loosely and ambiguously, and may request several changes in one image.

In addition, for text changes, style changes and component additions, the feedback image includes a short textual note, such as “Give this a deep blue background” (App. E, Table 5). These three operations account for 360 tasks, or 36% of the benchmark, and receive relatively high scores. In contrast, movement, resizing and alignment, which rely on geometric marks only, receive the lowest scores. The differences across operations may therefore partly reflect whether a textual note is present, rather than the difficulty of the operation itself.

W2. Gemini both generates the edit intents and achieves the best results. Gemini-3.7-Flash chooses the targets and parameters for all tasks. It also leads the evaluation by about 10 points over the second model and about 23 points over GPT-5.6-Sol, with especially large gaps on alignment and reordering. With explicit intent, Gemini and GPT both reach about 95% intent satisfaction (Fig. 5), so the gap between them arises mainly when inferring intent from the screenshot. This is exactly the step where a model may benefit from familiarity with edits it proposed itself.

W3. The ambiguity of the visual feedback is not quantified, and there is no human baseline. Intent satisfaction is about 70% under standard input and about 95% with explicit intent. It is unclear how much of this gap reflects model limitations and how much reflects feedback that does not determine a unique edit, especially for numeric parameters such as movement distance or target size. The human study judges request clarity as a binary majority vote, with 95 of 96 tasks approved (App. I). Such a judgment says little about whether a person could recover the exact target, operation and parameters from the image.

W4. The motivation for evaluating agents without execution feedback is unclear. Agents cannot run commands, build the project or preview the rendered page (Sec. 4.1). The paper explains that this standardizes tool access across models. However, practical frontend development relies heavily on checking the rendered result, especially for visual edits such as movement and resizing. Without execution, agents cannot detect build failures or verify whether an edit looks as intended.

W5. The scoring is lenient on incomplete cross-page edits and on damage to unrelated content. For shared-component tasks, intent is checked only on the primary page (Sec. 4.3, App. H), and the primary page receives half the weight in target fidelity. In the example of Fig. 12, the model edits only the primary page and leaves three affected pages unchanged. By my estimate, the overall score would still exceed 80 if preservation stays near its typical value. Yet consistent cross-page editing is a central challenge the benchmark aims to capture.

In addition, conditional preservation lies between 99.4 and 99.8, and target fidelity between 93.7 and 95.9. Both are computed on a different subset for each model, so they add little to the ranking, and the overall score largely tracks intent satisfaction, as the paper notes. Averaging preservation over whole pages may also dilute damage to individual components.

Questions for authors

Q1. On automatically generated feedback. Could the authors report the fraction of tasks with textual notes for each operation? It would help to report validity and intent satisfaction rates per operation, split by whether a note is present. Finally, a small set of real user annotations would show whether the main findings carry over to natural feedback, for example by asking several people to annotate the same edits in their own way.

Q2. On possible bias from intent generation. Could the authors regenerate the intents for a subset of tasks with another model or with human annotators, and check whether model rankings remain stable?

Q3. On ambiguity and a human baseline. I would suggest a human intent-inference study. Annotators would see only the annotated screenshot and write down the target, operation and parameters, and the authors would report agreement with the VES for each operation. This would give an upper bound on how well the intent can be recovered. It would also help interpret the model results, particularly for movement and resizing.

Q4. On the no-execution setting. Could the authors explain the reasons for this choice in more detail, for example cost, safety or reproducibility? It would also be informative to evaluate a subset with build access and a tool that returns a screenshot of the rendered page, and to report whether intent satisfaction and model rankings change.

Q5. On cross-page consistency and preservation. Could the authors apply intent checks on all affected pages for shared-component tasks, or report the fraction of submissions that pass on every page? It would also help to compare target fidelity and preservation on a common subset of tasks that all models pass.

Reviewer_J68oOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents ViRE-Bench, a benchmark for evaluating the ability of coding agents to perform repository-level frontend code edits based on visual feedback. ViRE-Bench consists of 1,000 editing tasks collected from 202 real-world repositories spanning HTML, React, Vue, and Angular, and covers eight categories of editing operations. Unlike prior benchmarks that rely on explicit textual instructions, ViRE-Bench requires agents to interpret annotated screenshots, locate the corresponding source code, and implement the requested modifications while preserving unrelated content and visual appearance. The authors develop an automated pipeline for benchmark construction, together with an evaluation framework that measures performance along four dimensions: submission validity, intent satisfaction, target fidelity, and preservation. Experiments across six models show that spatial editing operations remain particularly challenging, especially moving and resizing components. Further diagnostic analyses reveal that providing explicit editing intent leads to substantially greater performance improvements than providing source-location information alone.

Critical strengths

The paper addresses an important challenge in repository-level code editing: translating visual feedback into precise source-code modifications. The benchmark spans multiple frontend frameworks, diverse editing operations, and changes to components shared across multiple pages, providing a valuable evaluation setting that extends beyond screenshot reconstruction or standalone code generation.

Critical weaknesses
  1. Although the benchmark is constructed from real-world repositories, its editing targets and parameters are generated automatically, while the visual annotations are derived from structured editing specifications rather than collected from real users. Moreover, each task is restricted to a single editing operation. Consequently, the benchmark may not fully capture the ambiguity, diversity, and compositional complexity of real-world visual feedback, where users often request multiple interdependent changes simultaneously. Evaluating on independently collected human editing requests would further strengthen the benchmark’s practical relevance and ecological validity.

  2. Several of the operation-specific intent checks rely on fairly lenient thresholds. Resizing, for instance, only requires the relevant dimensions to shift by at least 10 pixels in the right direction, rather than actually matching the requested size. Likewise, component movement is judged mainly by the direction of displacement and a minimum distance moved. While target fidelity accounts for some of these gaps, edits that are substantially off can still pass the binary intent check. Moreover, intent satisfaction is assessed only on the primary page, even for shared-component tasks that require consistent changes across multiple pages. A stricter or graded intent evaluation that also verifies cross-page correctness would make the benchmark more reliable.

  3. The diagnostic experiments show considerable gains when explicit editing intent is supplied. However, these intent descriptions are derived from the ground-truth Visual Edit Specifications and contain exact target locations and editing parameters, information that is typically unavailable in practice. The experiments therefore do not show whether comparable gains could be obtained through automatic intent extraction. An additional baseline that first converts annotated screenshots into textual editing instructions and then carries out the repository edits would clarify whether the identified bottleneck can be overcome in practical settings. In addition, because the evaluation is single-turn and provides no execution or rendering feedback, it offers limited insight into the behavior of more realistic, iterative coding agents.

Questions for authors

Given that the editing requests and annotations are generated automatically from structured specifications, how closely do they resemble real user feedback? Have the authors considered evaluating on independently collected human annotations to examine differences in ambiguity and task complexity?

Reviewer_Wy45OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces ViRE-Bench for frontend code editing guided by annotated screenshots, comprising 1,000 tasks from 202 real repositories across eight edit operations. Experiments with six models highlight challenges in spatial editing and maintaining consistency across pages. Diagnostic experiments further show that explicitly specifying the intended edits yields larger gains than providing source-location hints alone, suggesting that accurately inferring editing requirements from visual feedback is a key challenge in this setting.

Critical strengths
  1. The paper identifies frontend code editing guided by annotated screenshots, which is a practically relevant problem. The task is clearly motivated.
  2. Accurately extracting editing requirements from visual feedback is a key challenge in this setting. This finding provides useful guidance for practical system design.
Critical weaknesses
  1. The repositories are real, but the annotations are synthetic. The paper does not clarify whether fonts, colors, or other annotation properties were randomized. It also does not test robustness to different annotation styles and ways of expressing requests.
  2. For a benchmark paper, the experiments are relatively small in scale. Comparing different reasoning settings for the same model and different variants within a model family (e.g. Flash vs. Pro), would strengthen the analysis.
Questions for authors
  1. Compared with providing screenshots and textual requests (e.g. DesignBench), what additional capabilities or challenges can be revealed by providing screenshots and annotations, and why is it relevant?
  2. How are the eight edit operations selected, and are any common editing intents currently missing?
Reviewer_rK2sOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes ViRE-Bench, a benchmark for evaluating coding agents’ ability to modify real front-end code repositories based on annotated screenshots. The dataset comprises 1,000 tasks drawn from 202 HTML, React, Vue and Angular repositories, covering eight categories of editing operations, including instance-level modifications and modifications to components shared across pages. The authors have designed a build process based on the Visual Edit Specification (VES), utilising the same specification to generate both reference code modifications and visual feedback, and have proposed an automated evaluation system that comprehensively considers Validity, Intent Satisfaction, Target Fidelity and Preservation.

Critical strengths
  1. This paper extends visual feedback-driven front-end modifications to real-world multi-file repositories, requiring the agent to autonomously understand editing intent, locate source code, and ensure consistency in unrelated content and cross-page shared components. Compared to existing Screenshot-to-Code and Text-Guided Editing Benchmarks, this task setting offers a degree of novelty and practical value.

  2. The authors evaluate Validity, Intent Satisfaction, Target Fidelity and Preservation separately, rather than relying solely on Screenshot Similarity. Automatic Intent Checks achieved a 96.0 per cent agreement rate with the majority of human judgements, whilst the Overall Score showed a Spearman correlation coefficient of 0.892 with manual quality ratings.

Critical weaknesses
  1. Gemini-3.7-Flash was involved in project quality screening, the selection of editing objectives and parameters, and the generation of some visual feedback text; it was also the best-performing model in the benchmark. The fact that the same model was involved in both task construction and evaluation may have aligned the choice of tasks and the phrasing of feedback more closely with the model’s preferences. Currently, there is a lack of a control group featuring tasks generated by independent generators or constructed manually, making it impossible to determine the extent of this influence.

  2. Reference implementations and annotated screenshots for all tasks in this paper were generated by Structured VES. Although this process ensures consistency and verifiability of the tasks, the visual feedback may exhibit a strong degree of templatisation. Feedback from real users typically includes imprecise arrows, ambiguous spatial relationships, non-standard annotations, multiple interrelated modifications, and intentions requiring further clarification. Consequently, it remains unclear whether the model differences observed on the Benchmark, particularly the bottleneck in Visual Intent Understanding can be generalised to visual feedback provided by real users.

Questions for authors
  1. Would it be possible to collect independent, human-annotated screenshots for a small subset of tasks, or at least conduct some analysis to compare the differences and generalisability between automatically generated data and genuine human data?

  2. The paper adopts a Single-Turn, No Execution Feedback evaluation protocol. The Agent can read and modify files, but cannot run the build, test, or browser, nor view the rendered results of the modifications. However, actual front-end development relies heavily on the ‘Render–Inspect–Refine’ cycle. For tasks such as Component Movement, Resizing and Alignment, the Agent typically needs to observe the modified page to correct the layout. Would it be possible, on a representative subset of tasks, to allow the Agent to run web pages, inspect rendered screenshots and make a limited number of modifications?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.