acceptodds
Under review as a conference paper at ICLR 2027

Interaction-as-a-Judge

Abstract

Coding agents are tasked with producing sophisticated interactive applications such as websites, desktop applications, and games. While humans assess these applications interactively, current automated LLM evaluations primarily focus on code analysis and unit tests. This mismatch can cause LLM judges to miss failures that are immediately apparent to human users. We study Interaction-as-a-Judge, an evaluation paradigm where computer-use agents (CUAs) directly interact with an application. We demonstrate that Interaction-as-a-Judge can improve both evaluation and generation of interactive applications. To this end, we collect and release two complementary datasets: 6, 858 pairs of applications with human preferences and corresponding agent traces, and 222 applications with human-verified requirement labels. Relative to standard non-interactive approaches, we find that Interaction-as-a-Judge improves evaluation in terms of predicting human preferences (65.6% → 69.1%) and requirement verification (77.0% → 81.6%). We also observe scaling benefits—across both preference alignment and requirement verification, higher-quality interactions improve performance. Finally, through both inference- and training-time experiments, we demonstrate that Interaction-as-a-Judge can also improve the quality of generated artifacts. At inference time, Interaction-as-a-Judge repairs 75.5% of broken requirements within five attempts, compared to 55.4% for code-only self-feedback. At training time, using Interaction-as-a-Judge as a reward signal improves win-rate over the base model by 6.7pp while reducing both code length bias by 35.7% and build failure rates by 7.2pp compared to a code-only reward signal.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agents, Evaluation, LLM

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.40 over 5 reviews, above 99% of the others.
Reviewer_HUKiOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studied interaction-as-a-judge: let a computer-use agent (CUA) to interact with the applications (websites, games, etc) before sending the interaction trajectory to LLM-as-a-judge. It checks (1) whether interaction-as-a-judge better aligns with human preference, compared to static LLM-as-a-judge, (2) whether interaction-as-a-judge more accurately verifies the CUA’s generated code, and (3) whether interaction-as-a-judge helps the CUA better improve itself, in both inference-only iterative improvements and RL training.

Critical strengths

The research problem (interaction-as-a-judge) is of reasonable value.

Both inference-time agent iteration and RL-based training received convincing improvements from the proposed interaction-as-a-judge feedback. Especially, when using interaction-as-a-judge in RL, model performance outscores its counterpart using code-only judging by 20.1 points under GPT-sol interactive eval.

Two useful datasets are collected with human verification. The first one has user preferences on agent-generated applications. The second one verifies whether the agent-written code completed a user request.

Critical weaknesses
  1. The authors wrote the abstract and most of the intro as if they were the first to use interaction trajectories to evaluate agent-generated applications. This is pretty misleading. Then, in line 46, the authors further said:

While prior works have explored this paradigm (interaction-as-a-judge), …… we are the first to systematically study it in a controlled fashion. Specifically, we investigate how dimensions such as interaction strategy and agent capability affect evaluation accuracy for both pairwise human preference prediction and correctness verification.

This claim is also ambiguous and vague. In this paper’s experiments, which have been done in a similar way (or on a different task) by previous work? Which are original? There is not a comprehensive explanation.

  1. Furthermore, the authors did not exhaustively analyze which “interaction strategy” is the best. In figure 3, all three interactive strategies have similar performance (differences within 2 points). Also, the authors did not analyze: taking token cost into consideration, which strategy is the most cost-effective?

  2. The “agent capability” experiment is also ambiguous. Both CUA and the judge are changed in the experiments (luna, terra, sol). So I don’t understand what model capability the experiment analyzes, the CUA or the judge?

Questions for authors

Can you elaborate how you do the human annotation for AnonCorrectness? The details of this part are missing from the paper.

2 comments
AuthorsOpenReview ↗

Response Part 1

We first want to thank the reviewer for their time, their questions, and their positive sentiment towards our work. We especially want to thank the reviewer for pointing out the general usefulness of interaction-as-a-judge, our convincing inference-time and reinforcement learning experiments, and how useful the new data resources we are releasing will be for the research community. If our answers to your questions are helpful, we would greatly appreciate an update to our score.

“In this paper’s experiments, which have been done in a similar way (or on a different task) by previous work? Which are original?”

We acknowledge that there are numerous past works that explore interaction for evaluation and generation refinement. The differentiation of our work is to more thoroughly understand this emerging paradigm through (1) demonstrating its alignment with human preferences, (2) providing evidence that the paradigm scales as computer use improves, and (3) studying it at an unprecedented data scale.

  1. Human Preference Reward Modeling. Human preferences are a standard practice used for training reward models and post-training including supervised fine-tuning and reinforcement learning (e.g., RLHF, DPO). To our knowledge, no prior work explores the use of computer-use agent interaction traces for human preference reward modeling, simply using an LLM judge in the loop with rubrics. Both human preferences and rubrics-based approaches are valid and widely used in practice, but we are the first to show that interaction-based methods can work with human preferences and reward modeling.

  2. Capability Scaling Effects. One of the key insights we provide is that there seems to be a scaling effect linking better computer-use capabilities and methods to better downstream evaluation and generation. Our work provides the first evidence that changing the computer-use agent’s base model (e.g., GPT 5.6 Luna to GPT 5.6 Sol) or its execution method (heuristic, programmatic, and iterative computer-use) improves performance. This suggests that Interaction-as-a-Judge will scale as researchers develop better models and methods, which was not clear prior to our work.

  3. Dataset Size, Scale, and Realism. We have collected significant amounts of data and are contributing and evaluating over two datasets one of which contains 6.8k tasks (13.6k applications) across 7 diverse categories and the other containing 412 requirements/tasks across 222 games. For some comparison with relevant prior work, AppEvalPilot releases 194 tasks and evaluates on at most 54 tasks in the paper, WebGen-Agent evaluates over 647 requirements/tasks and 101 websites (and the data comes from an entirely different publication, WebGen-Bench), and Verification Horizon evaluates over a private set of 671 web development tasks. Including both of our datasets, we are releasing and evaluating on an order of magnitude more data than these prior works. Additionally, many of these benchmarks use synthetically generated tasks; all our tasks are either direct user requests or derived from real user complaints. As a final point, we are fully releasing our agentic traces, not just the final applications which we believe will be of significant help to future research in this area.

We want to emphasize that we are building upon prior work and believe that we are providing significant value to the community through our data and experiments. We have revised our manuscript to more effectively clarify the scoping and differentiation in the abstract/introduction.

AuthorsOpenReview ↗

Response Part 2

“taking token cost into consideration, which strategy is the most cost-effective?”

We chose a fixed interaction budget to keep the same visual token budget for our reward models and LLM judges, otherwise those costs would vary. Also, we chose a 30 step interaction budget as we found that when given an unlimited turn budget computer-use agents (for both methods) stopped by themselves around 30 steps; increasing step budget for programmatic would likely provide limited additional benefit in this setup. Programmatic computer-use also has to predict from only the initial state which means that error accumulates over time. Lastly, we do not argue that iterative computer-use is always better than programmatic or even heuristic computer-use and cost should certainly be a factor when choosing between each method. Our goal was to show that performance scales between these three computer-use methods including with cost which was not shown in prior work.

However, we do agree this would be an interesting experiment to run. We run two additional experiments: training a reward model with programmatic with GPT 5.6 Luna on xhigh and iterative with GPT 5.6 Luna for 14 iterations rather than 30. The costs of these two experiments are matched. Iterative still improves over programmatic in the full code and App.tsx settings. When there is no code, the two perform similarly. We believe this is because when there is sufficient code, quality matters more since the reward model can extract the majority of basic information from the code. When there is no code, simply having more visual information (30 steps vs 14 steps) can outweigh the benefits of higher quality, but more expensive interactions. We have included this in a revision to our paper in Appendix E.5.

SettingProgrammatic (xhigh), 30 stepsIterative (medium), 14 stepsGap
Full code70.270.9+0.7
App.tsx only70.170.4+0.3
Interaction only68.268.1-0.1

“Both CUA and the judge are changed in the experiments (luna, terra, sol).”

We agree with the reviewer and ran a full cross matrix experiment where we varied the computer-use agent and the judging agent separately. Performance is measured as number of broken requirements correctly identified.

Judge \ CUALunaTerraSol
Luna62.4%64.3%61.5%
Terra66.1%67.4%68.6%
Sol66.1%68.3%69.9%

We find that we observe scaling across both axes; increasing computer-use but keeping the judge fixed improves performance and vice-versa. The main exception is when Luna as a Judge uses Sol computer-use traces; we hypothesize that this is because Luna is a significantly weaker model than Sol and isn’t able to utilize that information properly. Similarly, when Luna is the computer-use agent, Terra and Sol perform the same as the judge (despite Sol being higher capability than Terra), again hinting at issues with a Luna and Sol capability gap. We have included this in a revision of our paper in Appendix F.1. We believe this experiment is important so once all the revisions are in, we will likely incorporate it into the main paper. Thank you for your suggestion.

“Can you elaborate how you do the human annotation for AnonCorrectness?”

As mentioned on page 3, the human annotator manually reviews the game and annotation to verify its correctness. The annotator does this by actually playing the game to verify that the annotation is broken. Working requirements are done in a similar manner, except with the working games and requirements. The requirements are straightforward and non-ambiguous to a human (e.g., “the player must not fall through the floor”). If the question is in regards to how we collected human complaints and human queries in the first place, we can only share limited details during review to avoid de-anonymizing our work. In short, human annotations were collected at scale from everyday users of a widely deployed online platform. We will further document this process in our camera-ready.

Reviewer_QRCPOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Some evaluations for coding agent tasks rely either on test cases for functional correctness or LLM-as-a-judge based on the input code or final output artifact, but such signals are poor for evaluating the quality of an interactive application. The authors thus advocate for a class of methods focused on "interaction-as-a-judge", where computer use agents interact with a website, and a separate LLM judge evaluates its quality based on the interaction. While prior work has studied this (extensively, I might add), the authors argue that their analysis is more comprehensive and spans both evaluation and generation use cases. The author's experiments show that interaction-as-a-judge improves alignment with human preferences, and improve the generation of interfaces via steering and post-training. The authors also release all collected datasets used to design these experiments

Critical strengths
  • The authors extensively explore the benefits of interaction-as-a-judge across evaluation (grounded in human judgments) and generation (post-training)
  • The collected dataset seems useful for researchers who are interested in optimizing computer use agents as judges. Based on the works discussed in the paper, it seems to be a growing field and the community would find this useful!
Critical weaknesses

I think this paper is solid and I would recommend acceptance, but my main concern is that the paradigm of interaction-as-a-judge that the authors present has already been studied extensively in prior work. The authors rightfully point out some of these works in the paper, but there are more (this was just from a preliminary skim):

  1. UXAgent: An LLM-Agent-Based Usability Testing Framework for Web Design
  2. WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratch
  3. Computer-Use Agents as Judges for Generative User Interface

The authors never explicitly state that this is a new paradigm, but the title of this paper makes it come off as such. Also, the only reference to prior work in the introduction is the claim that "we are the first to systematically study it in a controlled fashion" (L48), which is pretty vague

TL;DR – I think the claims in the abstract/introduction of the paper should be a bit more tempered and more concretely connected to prior work. It seems more faithful to say that work has started exploring computer use agents as reward signals (i.e., interaction-as-a-judge) for different reasons, but you extend this to game development and explore its value across varied stages of model development

Questions for authors
  • How does this work differ from "Computer-Use Agents as Judges for Generative User Interface" in particular? They similarly discuss this paradigm of Computer-Use Agents as judges and investigate its value to both human developers, as an evaluation metric, and for improving interfaces
  • The phrase "interaction-as-a-judge" doesn't really make sense with respect to the author's framework. In LLM-as-a-judge, an LLM is doing the judging. In the author's framework as I understand it, the "interaction" is not doing the judging. The interaction just provides extra context for an LLM judge to use. So the two things you are contrasting are judging based on code/artifact versus judging based on interactions, but you wouldn't call the former "Code-as-a-Judge". This is a pretty pedantic point that doesn't really matter (I think interaction-as-a-judge could be memorable to author audiences), but just leaving this thought in case it spurs some more precise terminology
  • I unfortunately find that the term "pp" (i.e., in the introduction) is super common with AI-generated text. I think this paper is overwhelmingly human-written, so I'd recommend reviewing it to be safe (again, not crucial)
2 comments

We first want to thank the reviewer for their time, their questions, and their positive sentiment towards our work. We especially want to thank the reviewer for pointing out how extensive our experiments were (we tried hard to cover as much as we could) and how useful the new data resources we are releasing will be for the research community.

“the claims in the abstract/introduction … should be a bit more tempered and more concretely connected to prior work… UXAgent: An LLM-Agent-Based Usability Testing Framework for Web Design, WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratch… How does this work differ from ‘Computer-Use Agents as Judges for Generative User Interface’ in particular?”

We acknowledge that there are past works that explore interaction for evaluation and generation refinement. The differentiation of our work is to more thoroughly understand this emerging paradigm through (1) demonstrating its alignment with human preferences, (2) providing evidence that the paradigm scales as computer use improves, and (3) studying it at an unprecedented data scale.

  1. Human Preference Reward Modeling. Human preferences are a standard practice used for training reward models and post-training including supervised fine-tuning and reinforcement learning (e.g., RLHF, DPO). None of the three works (and to our knowledge no prior work in general) explores the use of computer-use agent interaction traces for human preference reward modeling. We also train a reward model rather than using an LLM judge in the loop (again, no prior work does this). Both human preferences and rubrics-based approaches are valid and widely used in practice, but we are the first to show that interaction-based methods can work with human preferences and reward modeling. Additionally, none of the three works the reviewer mentioned does any form of RL training in their work.

  2. Capability Scaling Effects. One of the key insights we provide is that there seems to be a scaling effect linking better computer-use capabilities and methods to better downstream evaluation and generation. Our work provides the first evidence that changing the computer-use agent’s base model (e.g., GPT 5.6 Luna to GPT 5.6 Sol) or its execution method (heuristic, programmatic, and iterative computer-use) improves performance. This suggests that Interaction-as-a-Judge will scale as researchers develop better models and methods, which was not clear prior to our work.

  3. Dataset Size and Realism. We have collected significant amounts of data and are contributing and evaluating over two datasets one of which contains 6.8k tasks (13.6k applications) across 7 diverse categories and the other containing 412 requirements/tasks across 222 games. In comparison, Computer-Use Agents as Judges for Generative User Interface evaluates on 52 applications and 1560 tasks and WebGen-Bench has 647 tasks/requirements across 101 websites. It is important to note that both of their tasks are synthetically generated and then verified by a human annotator. On the other hand, our tasks are derived from real user complaints and real user requests and are not synthetic. UXAgent is a user study focused work and not comparable here. As a final point, we are fully releasing our agentic traces, not just the final applications which we believe will be of significant help to future research in this area.

We want to emphasize that we are building upon prior work and believe that we are providing significant value to the community through our data and experiments. We have revised our manuscript to more effectively clarify the scoping and differentiation in the abstract/introduction.

“The interaction just provides extra context for an LLM judge to use.”

We agree with the reviewer that the name is awkward from a semantic perspective. This concern is fully consistent with our internal discussions over the name. Despite the concerns, ultimately we decided on "interaction-as-a-judge" to anchor the paradigm on the widely-known "LLM-as-a-judge", to which it most closely relates.

“the term ‘pp’ … I’d recommend reviewing it to be safe (again, not crucial)”

None of the text in the main paper was AI generated, and we may remove pp if it gives that impression… Thank you for the heads up.

Reviewer_QRCPOpenReview ↗

Response

Thanks for the response. I'll maintain my already positive assessment

Reviewer_K41MOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Interaction-as-a-Judge, a new evaluation paradigm in which computer-use agents interact directly with an application rather than relying on code analysis and unit tests. To support this, the authors release two datasets (AnonPreferences and AnonCorrectness): pairs of applications with human preference labels, and applications with human-verified requirement labels. They show improvements in both the evaluation and the generation of interactive applications compared to standard non-interactive approaches.

Critical strengths
  • The new datasets would substantially help the community build stronger interactive applications with agents. In particular, the pairing of real human preferences with full agent interaction traces is is a valuable resource for agentic web development
  • I really like the idea of using interaction as feedback. It is intuitive, but it has been hard to incorporate into reward models. In particular, the finding that a reward model trained on interaction traces alone, without code, produces strong results.

--> I think the authors' approach is a step forward and would definitely help researchers in this area.

Critical weaknesses
  • The baselines could be stronger. The proposed approach currently uses much more compute than the baselines. A fairer comparison would give the baselines the same compute budget (e.g., more iterations with screenshots).
  • Additionally, the gains over the screenshot baseline seem rather small. Given the compute imbalance, it is hard to tell whether the improvements are significant. I would invite the authors to argue why their method should be the preferable approach.
Questions for authors
  • I would like to see an equal budget baseline against the author's approach.
  • I would also emphasize more what the advantages are of your approach vs. other static approaches -- e.g., what are errors that static approaches miss?
3 comments

We first want to thank the reviewer for their time, their questions, and their positive sentiment towards our work. We especially want to thank the reviewer for pointing out how intuitive the idea is and how useful the new data resources we are releasing will be for the research community. If our answers to your questions are helpful, we would greatly appreciate an update to our score.

“A fairer comparison would give the baselines the same compute budget (e.g., more iterations with screenshots).”

We chose a fixed interaction budget to keep the same visual token budget for our reward models and LLM judges, otherwise those costs would vary. Also, we chose a 30 step interaction budget as we found that when given an unlimited turn budget computer-use agents (for both methods) stopped by themselves around 30 steps; increasing step budget for programmatic would likely provide limited additional benefit in this setup. Programmatic computer-use also has to predict from only the initial state which means that error accumulates over time. Lastly, we do not argue that iterative computer-use is always better than programmatic or even heuristic computer-use and cost should certainly be a factor when choosing between each method. Our goal was to show that performance scales between these three computer-use methods and with cost which was not shown in prior work.

However, we do agree this would be an interesting experiment to run. We run two additional experiments: training a reward model with programmatic with Luna on xhigh and iterative with Luna for 14 iterations rather than 30. The costs of these two experiments are matched. Iterative still improves over programmatic in the full code and App.tsx settings. When there is no code, the two perform similarly. We believe this is because when there is sufficient code, quality matters more since the reward model can extract the majority of basic information from the code. When there is no code, simply having more visual information (30 steps vs 14 steps) can outweigh the benefits of higher quality, but more expensive interactions. We have included this in a revision to our paper in Appendix E.5.

SettingProgrammatic (xhigh), 30 stepsIterative (medium), 14 stepsGap
Full code70.270.9+0.7
App.tsx only70.170.4+0.3
Interaction only68.268.1-0.1

“the gains over the screenshot baseline seem rather small. … it is hard to tell whether the improvements are significant.”

We respectfully disagree that the gains over the screenshot baselines are small. For preferences we show in our paper that the improvements are statistically significant. At scale, even 1% improvements are very impactful (and most real world settings will evaluate significantly more data points than our dataset). Secondly, heuristic (random) computer-use has no inference costs and still provides benefits over code+screenshot, thus there is little reason not to use Interaction-as-a-Judge. Additionally, we show large gains in other experiments. Specifically, we see large gains for broken requirement verification (GPT 5.6 Sol improves from 63% to 71.1%), inference-time scaling (55.4% to 75.5%), and also interesting results with reinforcement learning. Of course, as the reviewer mentioned, we also demonstrate that we can train a model via reinforcement learning with an interaction-only reward.

“I would like to see an equal budget baseline against the author's approach.”

Thank you for the question and we ran a matched-budget experiment above. We have included these in the revision; if the reviewer has any other questions we would be more than happy to answer.

“what are errors that static approaches miss?”

From our personal observation, it falls into two categories. First are situations where only by navigation can you visually observe the full application. Any game that has a start menu or any website with multiple tabs can only be visually observed upon navigation. Second are situations where interaction just makes small, but critical errors obvious. For example, many games flip the left and right keys. Another example, for some Tetris recreations pressing the right arrow triggers too many movements at once. These are obvious upon interaction, but difficult to spot within the code. We have compiled these examples into Appendix J.

Thank you for the detailed response and the additional experiment.

There might have been a misunderstanding. The new matched-budget experiment compares two variants of your own method (programmatic vs. iterative) rather than interaction vs. the screenshot baseline, which was my original concern. If I understand correctly, in your second paragraph, you say that the heuristic (random) interaction has no inference cost yet still outperforms code+screenshot, so the gains cannot be explained by compute alone. I find this argument helpful. Still, the judge processes many more screenshots in the heuristic setting than in the single-screenshot baseline.

However, my concerns are largely resolved, and I increased my score.

Thank you for the positive response and we're happy to hear that we addressed your questions.

In regards to interaction vs the screenshot baseline, for our reward modeling experiments we intentionally keep the token budget the same at a 32k token limit. Thus, while the heuristic (random) trained reward model does process more image tokens, it loses out on an equivalent amount of code tokens. We hope this helps clarify things further.

Reviewer_kyboOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents Interaction-as-a-Judge, a framework that uses interaction traces collected by computer-use agents to evaluate human preferences and functional correctness. It further applies these evaluation signals to iterative repair and reinforcement learning. Experiments show improvements in evaluation accuracy across most settings, requirement repair rates, and build reliability.

Critical strengths
  1. Comprehensive experiments. The evaluation covers preference prediction, requirement verification, inference-time repair, and reinforcement learning, with analyses of interaction strategies, model capabilities, source-code availability, and trace components.
  2. Useful data resources. The two datasets provide real-user pairwise preferences and human-verified functional requirements, supporting future research.
Critical weaknesses
  1. Limited differentiation from prior work. AppEvalPilot already uses GUI interactions to assess functional correctness; WebGen-Agent incorporates interaction feedback into generation refinement and reinforcement learning; and The Verification Horizon studies interaction-based rewards and their mitigation of code-length exploitation. Several central findings therefore have precedents. The paper should clarify which empirical insights extend beyond these prior observations.
  2. Insufficiently controlled comparisons. Methods differ in model calls and observation volume, without matched-budget comparisons. It is therefore unclear whether the gains stem from better interaction strategies or additional resource use.
  3. Limited dataset coverage and annotation details. The correctness dataset covers only 67 game-development trajectories, with requirements drawn primarily from user complaints, limiting its representativeness. Human verification procedures and inter-annotator agreement are also insufficiently documented.
Questions for authors
  1. Which empirical findings extend beyond those reported in AppEvalPilot, WebGen-Agent, and The Verification Horizon?
  2. Do the improvements persist under matched computational and observation budgets?
  3. How well do the findings generalize beyond complaint-derived requirements in games?
  4. How were requirement labels verified, and was inter-annotator agreement assessed?
4 comments
AuthorsOpenReview ↗

Response Part 1

We first want to thank the reviewer for their time and their questions. We especially want to thank the reviewer for pointing out our comprehensive experiments (we tried very hard to cover as much as we could) and the useful new data resources we are releasing. If our answers to your questions are helpful, we would greatly appreciate an update to our score.

“The paper should clarify which empirical insights extend beyond these prior observations.”

We acknowledge that there are numerous past works that explore interaction for evaluation and generation refinement. The differentiation of our work is to more thoroughly understand this emerging paradigm through (1) demonstrating its alignment with human preferences, (2) providing evidence that the paradigm scales as computer use improves, and (3) studying it at an unprecedented data scale.

  1. Human Preference Reward Modeling. Human preferences are a standard practice used for training reward models and post-training including supervised fine-tuning and reinforcement learning (e.g., RLHF, DPO). None of the three works (and to our knowledge no prior work in general) explores the use of computer-use agent interaction traces for human preference reward modeling. Both WebGen-Agent and Verification Horizon use a rubrics / checklist based approach for training which is substantially different and more analogous to our correctness setting (thus for correctness we explored inference-time scaling instead). Additionally, none of them train a reward model, simply using an LLM judge in the loop. Both human preferences and rubrics-based approaches are valid and widely used in practice, but we are the first to show that interaction-based methods can work with human preferences and reward modeling.

  2. Capability Scaling Effects. One of the key insights we provide is that there seems to be a scaling effect linking better computer-use capabilities and methods to better downstream evaluation and generation. AppEvalPilot only uses a single Claude model for computer-use for its main backbone. Verification Horizon only uses its own model for interaction and does not use iterative computer-use, only programmatic. WebGen-Agent varies the VLM judge but shows no evidence of varying the computer-use agent itself. Our work provides the first evidence that changing the computer-use agent’s base model (e.g., GPT 5.6 Luna to GPT 5.6 Sol) or its execution method (heuristic, programmatic, and iterative computer-use) improves performance. This suggests that Interaction-as-a-Judge will scale as researchers develop better models and methods, which was not clear prior to our work.

  3. Dataset Size, Scale, and Realism. We have collected significant amounts of data and are contributing and evaluating over two datasets one of which contains 6.8k tasks (13.6k applications) across 7 diverse categories and the other containing 412 requirements/tasks across 222 games. In comparison, AppEvalPilot releases 194 tasks and evaluates on at most 54 tasks in the paper, WebGen-Agent evaluates over 647 requirements/tasks and 101 websites (and the data comes from an entirely different publication, WebGen-Bench), and Verification Horizon evaluates over a private set of 671 web development tasks. Therefore, including both of our datasets, we are releasing and evaluating on an order of magnitude more data than these prior works. Additionally, many of these benchmarks use synthetically generated tasks; all our tasks are either direct user requests or derived from real user complaints. As a final point, we are fully releasing our agentic traces, not just the final applications which we believe will be of significant help to future research in this area.

There are of course other differences in experimentation. For example, we intentionally opted for getting Interaction-as-a-Judge to work with standard inference-time and RL recipes (e.g., DAPO). WebGen-Agent, on the other hand, implements a new learning algorithm (Step-GRPO).

We want to emphasize that we are building upon prior work (and have already cited all 3 of the above works) and believe that we are providing significant value to the community through our data and experiments. We have revised our manuscript to more effectively clarify the scoping and differentiation in the abstract/introduction.

AuthorsOpenReview ↗

Response Part 2

“Methods differ in model calls and observation volume, without matched-budget comparisons.”

We control for a fixed interaction-step budget. For our experiments, this makes more sense than controlling for a fixed dollar cost budget for several reasons. First, this enables us to keep the same visual token budget for our reward models and LLM judges, otherwise those costs would vary. Second, we chose a 30 step interaction budget as we found that when given an unlimited turn budget computer-use agents (for both methods) stopped by themselves around 30 steps; increasing step budget for programmatic would likely provide limited additional benefit in this setup. Programmatic computer-use also has to predict from only the initial state which means that error accumulates over time. Lastly, our goal was explicitly to experiment with methods of different cost, but with the same model. By doing so we are able to show that scaling the computer-use agent leads to increased performance which was not evident in works prior to ours.

However, we do agree this would be an interesting experiment to run. We run two additional experiments: training a reward model with programmatic with Luna on xhigh and iterative with Luna for 14 iterations rather than 30. The costs of these two experiments are matched. Iterative still improves over programmatic in the full code and App.tsx settings. When there is no code, the two perform similarly. We believe this is because when there is sufficient code, quality matters more since the reward model can extract the majority of basic information from the code. When there is no code, simply having more visual information (30 steps vs 14 steps) can outweigh the benefits of higher quality, but more expensive interactions. We have included this in a revision to our paper in Appendix E.5.

SettingProgrammatic (xhigh), 30 stepsIterative (medium), 14 stepsGap
Full code70.270.9+0.7
App.tsx only70.170.4+0.3
Interaction only68.268.1-0.1

“Limited dataset coverage and annotation details.”

We intentionally chose our 222 different games from 67 development traces rather than 222 different traces. While we could have sampled from entirely different game trajectories, we believe our approach allows for additionally useful temporal connections between various games. Note that the differences between each game in the trajectory can be quite large as the agent turns are long, comprising an average of 7 steps and 6.5 tool calls per turn. Additionally, each of our traces is long, composed of 828 user-agent turns in the full dataset with each turn being itself a detailed agent trace. Additionally, note that our correctness dataset size is comparable to AppEvalPilot (194 tasks) and WebGen-Bench (101 human-verified websites).

As mentioned on page 3, the human annotator manually reviews the game and annotation to verify its correctness. The annotator does this by actually playing the game to verify that the annotation is broken. Working requirements are done in a similar manner, except with the working games and requirements. The requirements are straightforward and non-ambiguous to a human (e.g., “the player must not fall through the floor”) which is also why we do not report inter-annotator agreement. If the question is in regards to how we collected human complaints and human queries/preferences in the first place, we can only share limited details during review to avoid de-anonymizing our work. In short, human annotations were collected at scale from everyday users of a widely deployed online platform. We will further document this process in our camera-ready.

AuthorsOpenReview ↗

Response Part 3

“Which empirical findings extend beyond those reported in AppEvalPilot, WebGen-Agent, and The Verification Horizon?”

The differentiation of our work is to more thoroughly understand this emerging paradigm through (1) demonstrating its alignment with human preferences, (2) providing evidence that the paradigm scales as computer use improves, and (3) studying it at an unprecedented data scale.

“Do the improvements persist under matched computational and observation budgets?”

We do matched observation budget in our original submission (30 steps). For matched computational budget, we do a comparison above and find that quality matters more when code is included. When interaction traces are the only training signal, more, lower quality interactions (programmatic at 30 steps vs iterative at 14 steps) can match performance with a similar budget.

“How well do the findings generalize beyond complaint-derived requirements in games?”

We did not test complaint-driven requirements outside of games in this work. However, we have results for multiple categories for human preferences. This is an area of future work that we may endeavor to do.

“How were requirement labels verified, and was inter-annotator agreement assessed?”

Requirement labels were verified by a human annotator. We did not measure inter-annotator agreement as requirement labels are clear and obvious to a human (e.g., the player must not fall through the platform).

Reviewer_kyboOpenReview ↗

Concern Resolved

Thank you for the detailed clarification; it satisfactorily addresses my concern, and I will raise my score accordingly.

Reviewer_vChfOpenReview ↗
Rating
3
Confidence
1
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies Interaction-as-a-Judge, where a computer-use agent interacts with generated web apps or games to produce traces for evaluation. The authors compare heuristic, one-shot, and iterative interaction policies across preference prediction, requirement verification, inference-time bug repair, and RL post-training.

Critical strengths

Useful datasets on a practical problem: Evaluating interactive software beyond static code is a real need (esp. in my domain of coding agents) and releasing real-world web battles and verified game requirement labels is a helpful resource for the community. They plan to release datasets with the paper.

Broad experimental pipeline: The paper covers multiple settings including LLM judges, reward model training, inference-time self-repair, and DAPO RL training.

Critical weaknesses

Marginal gains over a single static screenshot and a flawed Full Code truncation setup: In Section 4.1 (Figure 3), while the paper highlights a jump from 65.6% to 69.1% on Gemini 3 Flash, simply adding a single landing-page screenshot already reaches 67.6%. Running 30 steps of iterative computer-use only adds +1.5pp on Gemini 3 Flash, +0.8pp on the Full Code reward model (70.0% to 70.8%), and degrades GPT 5.6 Luna (68.3% code-only vs. 67.9% iterative), with gains heavily concentrated in Games and Simulations (Appendix C, Tables 2–5; and in Appendix E.4, Table 11, the first 4 frames / 0.6s already capture most of the 26-frame gain). Furthermore, Appendix D (Table 7) reveals that the Full Code pipeline orders files smallest-to-largest with the main application file placed last, causing ~60% of Full Code + trace inputs to hit the 32k token limit and truncate the main code file. This confounds the comparison between Full Code (70.8%) and App.tsx (70.2%).

Generation results in Section 5 have no human verification: Section 5.1 claims a 75.5% vs. 55.4% repair rate (and 69.7% vs. 57.9% under code verification in Figure 5) without any human verification, relying entirely on GPT 5.6 Sol as the judge even though Sol only achieves 63.0%–71.1% accuracy on broken requirements (Figure 4). Similarly, in Section 5.2 (RL, Figure 6), the games category is excluded, the win-rate gain of the Interaction RM (+6.72pp) barely differs from the Code RM (+6.14pp) as judged solely by an internal auto-rubric judge on a single training seed, and the Interaction RM's build success rate in Figure 6 stays roughly flat from step 0 to step 35 while the Code RM drops sharply as code length balloons (with the base model's test-set build rate not reported).

Questions for authors

None

1 comment

We first want to thank the reviewer for their time, their questions, and their positive sentiment towards our work. We especially want to thank the reviewer for pointing out how comprehensive and broad our experimental pipeline is and how useful the new data resources we are releasing will be for the research community. If our answers to your questions are helpful, we would greatly appreciate an update to our score.

“the Full Code reward model…gains concentrated in Games and Simulations…first 4 frames capturing much of the 26-frame gain.”

We respectfully disagree that the gains over the screenshot baselines are small. For preferences we show in our paper that the improvements are statistically significant. At scale, even 1% improvements are very impactful (and most real world settings will evaluate significantly more data points than our dataset). As noted in the frame-count experiment (Table 11), we continue to see improvements past frame 4, although the first 4 frames are the most substantial. This is a positive result as it shows that a) it is possible to get large improvements from short interactions and b) performance continues to scale with interaction length.

Our gains are particularly strong in games and simulations, but we also see improvements in other categories as well, although less consistently. Regardless, this helps inform the community about which types of tasks benefit most from interaction-as-a-judge which is the goal of our study.

Additionally, we show large gains in other experiments. Specifically, we see large gains for broken requirement verification (GPT 5.6 Sol improves from 63% to 71.1%), inference-time scaling (55.4% to 75.5%), and also interesting results with reinforcement learning.

“Files ordered smallest-to-largest which causes the main application file to be placed last … [causing] truncat[ion of] the main code file.”

Thank you for pointing this out. Our ordering is meant to preserve the main application file as we truncate from the start, not the end. We have clarified this in the updated manuscript.

“Generation results in Section 5 have no human verification relying entirely on GPT 5.6 Sol as the judge”

It is common in machine learning literature to use a more powerful verifier to validate results. For example, SimPO (Meng et al., 2024) uses GPT-4 Turbo to evaluate their trained Mistral-7B and Llama3-7B model results. Additionally, we show during our evaluations that regardless of the evaluation methodology, both agree that Interaction-as-a-Judge outperforms Code Agent-as-a-Judge indicating that there is no methodology bias during the evaluation.

“In Section 5.2, the games category is excluded, win-rate gain of the Interaction RM barely differs from the Code RM”

Our goal is to show that it is feasible to train a coding model with reinforcement learning with an interaction-based reward rather than source code. We are the first to show that you can use an interaction trained human preference reward model and see improvements over the base model. Our similar / slightly-higher win-rate gain over the code only reward model indicates that it is feasible to train a model with an interaction-based reward without loss in performance. This is important as it opens up many doors to new data acquisition opportunities that were previously unavailable (e.g., applications that do not share their code).

“the Interaction RM’s build success rate stays roughly flat from step 0 to step 35 while the code RM drops sharply as code length balloons (with the base model’s test-set build rate not reported”

The first part is not true. In Figure 6, the interaction reward model’s build success rate increases from approximately 60% to 88.5% between step 0 to step 35. There is indeed a negative correlation between code length / bloat and build success rate, but this strengthens the benefit of a model trained using interaction since it reduces code length/bloat. We can see the base-model’s build success rate in Figure 6 where it starts at about 60% in our training set. On our test set, we observed similar performance, which is sensible since prior to training both train and test sets are unseen data to the base model.

Comments on the paper

1 comment

We thank all reviewers for their positive and constructive feedback. We appreciate that all reviewers agree on acceptance and highlighted in particular our "comprehensive experiments" (vChf) and that "both inference-time agent iteration and RL-based training received convincing improvements" (HUKi). We are are also happy to see that reviewers agree the "datasets would substantially help the community build stronger interactive applications with agents" (K41M) and "useful for researchers who are interested in optimizing computer use agents as judges" (QRCP).

In our rebuttal, we believe we have strengthened the final version and addressed remaining concerns.

  • We clarified how our work differentiates itself from prior work on interactive evaluation and generation by (1) demonstrating its alignment with human preferences, (2) providing evidence that the paradigm scales as computer use improves, and (3) studying it at an unprecedented data scale (kybo, QRCP, HUKi).
  • We conducted additional follow-up experiments and analysis for cost-matched methods (kybo, K41M, HUKi).
  • We ran an additional cross matrix experiment between computer-use agents and llm judges with different levels of capability (HUKi).
  • We clarified our reinforcement learning results (vChf).
  • We added examples for error analysis (K41M).

All of the additional details and results have already been added to the revised PDF. We note that reviewer replies indicate that our replies satisfied their questions and concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.