acceptodds
Under review as a conference paper at ICLR 2027

Closing the Specialization Gap between Computer Use Benchmark Scores and the Real World Performance

Abstract

Strong general computer-use performance does not guarantee reliability in a narrow production workflow. We study this specialization gap in browser-based booking, where an agent operates from screenshots, asks for missing information, and must report the outcome truthfully. Although Qwen3.5-35B-A3B reports 54.5% on OSWorld-Verified, it performs poorly under our all-or-nothing booking criterion. We adapt it in two stages: supervised fine-tuning (SFT) teaches the production protocol, tool use, and user interaction, and reinforcement learning (RL) aligns complete trajectories with product logic. Training uses a browser harness that reproduces the production model-facing protocol and an Iron User simulator that answers clarification questions while updating the task ground truth. Evaluation combines local LLM judgments over screenshots, actions, and dialogue with deterministic resolution. On the available evaluation subset, SFT raises task success from 16.7% to 74.1%, and the SFT-plus-RL model outperforms all unadapted systems in the reported RL comparison.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
computer-use agents, GUI automation, supervised fine-tuning, reinforcement learning, agent evaluation, user simulation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_CUwuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper try to study the adaption of general domain computer use models to specialized browser-based booking workflows. Firstly, the authors use SFT training on Qwen3.5-35B-A3B models, in order to teach tool protocols, browser interactions, and user communication. Then, they use trajectory-level reinforcement learning (RL) to improve task execution. Besides, the system includes a browser harness that matches the production model-facing protocol, an Iron User simulator that updates task ground truth during clarification dialogues, and an evaluation pipeline combining LLM judgments with deterministic rules. The paper reports that SFT model improves strict task success from 16.7% to 74.1%. In a separate RL evaluation, SFT+RL increases the composite reward from 0.553 to 0.725 compared with SFT alone. The appendix provides human evaluations of the user simulator and Judge, as well as training stability experiments.

Critical strengths

1. The task setting is practically relevant and goes beyond GUI interaction.

The paper considers more than filling out forms correctly. It also covers asking for missing information, discussing alternatives, explaining why a request cannot be completed, and reporting outcomes honestly. Updating the ground truth to reflect what the user agrees to is a reasonable way to handle changing goals during an interaction.

2. The system is fairly complete, and task-specific training brings substantial gains.

The authors bring together a browser harness, user simulator, evaluator, and post-training pipeline. In Table 1, SFT improves the same base model’s success rate by 57.4 percentage points. This shows that task-specific training is effective for the evaluated booking workflow.

3. The authors include human checks of key components and acknowledge limitations in the results.

The appendix includes human evaluations of Iron User and the Judge. The authors also note that a success rate close to Claude’s does not establish statistical equivalence, and that repeated validation during training cannot replace independent training runs. These are useful qualifications that keep the claims in perspective.

Critical weaknesses

1. The contribution beyond standard domain adaptation is still unclear.

It is not particularly surprising that a general-purpose computer-use model performs poorly in a specialized domain. The paper needs to explain more clearly what makes this specialization gap difficult and why the proposed designs are needed to address it.

Using SFT to learn task-specific knowledge and interaction protocols, followed by RL to improve task performance, is a common approach. There is nothing wrong with using existing algorithms, but the paper then needs to show what additional value its system design provides. The most promising components are Iron User with dynamic ground-truth updates and the staged Judge, yet neither is directly compared with a simpler alternative.

As a result, it is hard to tell whether the gains mainly come from standard task-specific training or from these particular designs. It is also unclear how well the designs would carry over to other tasks. Could the authors compare dynamic ground truth with fixed targets or a simpler simulator, and the staged Judge with a simpler evaluation setup? Which failure cases do these components improve, and by how much?

2. The relationship between training and evaluation data is unclear, making generalization difficult to assess.

Section 5 states that training–evaluation overlap was not measured, and Section 6.3 does not clearly explain how the evaluation subset in Table 1 relates to the training data. The relationship among the 399 tasks in Section 5, the booking subset in Table 1, and the RL evaluation in Table 2 also needs clarification.

Having many different hostnames does not necessarily mean that website layouts, generation templates, or task instances are independent. It is therefore difficult to tell whether the model performs better on unseen tasks or mainly adapts to distributions already covered during training. This does not imply data leakage, but it limits what can be concluded about generalization.

Please clarify the size, source, and scenario composition of each evaluation set, as well as how the sets relate to one another. Could the authors also report overlap with the SFT/RL training data at the task, website, and generation-template levels, and provide results on a clearly separated held-out subset?

3. RL improves the composite reward, but its effect on strict task success remains unclear.

In Table 2, SFT+RL increases the composite reward from 0.553 to 0.725 compared with SFT alone. However, this reward combines several task and behavioral signals. A higher reward could reflect more completed tasks, fewer formatting errors, or better compliance with other rewarded behaviors. It does not directly tell us how much strict task success has improved.

The authors acknowledge that the metrics in the two tables are not directly comparable, but a comparison of SFT and SFT+RL on the same held-out tasks using strict success is still missing. In addition, the Judge’s overall accuracy is 82%, but the paper does not show whether its errors differ across the evaluated models. It is therefore unclear how these errors might affect the comparisons.

Could the authors report strict binary success rates on the same held-out tasks, with confidence intervals that account for the paired evaluation? If human checks are available, do they support the same model ranking and direction of improvement? Breaking down the results into correct bookings, appropriate abandonment, Unknown outcomes, and technical failures would also help show where the RL gains come from.

Questions for authors

Refer to weakness.

Reviewer_xqp6OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper is a case study in turning a general computer-use model into a specialist for one product task: booking tables and appointments on websites from screenshots. The starting point is that Qwen3.5-35B-A3B scores well on OSWorld but does poorly on this task under a strict all-or-nothing criterion. The authors build a training harness that mirrors their production agent, a simulated user (Iron User) that answers clarification questions and updates the task's ground truth as the conversation goes, and an LLM-based judge whose local checks are combined into a final verdict by code. They then fine-tune the model with SFT and GRPO. SFT raises task success from 16.7% to 74.1% (Claude Opus 4.8: 76.0%), and SFT+RL gets the highest reward among the Qwen models they compare. A large part of the paper documents how they kept long-horizon RL from collapsing.

Critical strengths

1. Iron User is a nice idea for training agents that talk to users.

  • When the agent asks a question, the simulated user answers and updates the task's ground truth in the same step, so the reward always matches what the user actually said.
  • Prior simulated users (e.g., in tau-bench, UserRL) mostly work with preferences fixed in advance.
  • It is backed by a human evaluation in the appendix.

2. Careful, honest engineering report.

  • The judge is validated against expert labels, the reward and RL settings are documented in detail, and the RL stability experiments will be useful to people training long-horizon agents.
  • The limitations are stated openly throughout.
Critical weaknesses

1. The main claim, that the specialization gap gets closed, is not measured on a consistent scale.

  • The "general ability" side is the model card's OSWorld score, which the paper itself says is not comparable to the booking results.
  • The two main results use different metrics on different models: Table 1 reports task success for base, SFT and Claude only; Table 2 reports training reward for Qwen models only. SFT+RL never gets a task-success number, and Claude never gets a reward.
  • The reward is the training objective itself (half of it from the same judge used for evaluation) and penalizes not following the custom protocol (e.g., not ending with booked, hitting the turn limit). Low scores for unadapted models may say more about not knowing the protocol than about missing ability.
  • Evaluation is on synthetic websites from the authors' own environment pool, with no real websites or users and no measured overlap with training data, so "real-world performance" is not really tested.
  • Whether the specialized model keeps its general ability (e.g., OSWorld after training) is not reported, so we can't tell whether the gap is closed or just moved.

2. It reads as a single-product engineering report rather than a research contribution.

  • Everything is tied to one task, one private protocol and in-house websites; the paper says the data and infrastructure can't be released, so others can't reproduce or build on it.
  • The broadly useful parts (Iron User, the hybrid judge, the RL stabilization recipe) are not tested on other tasks or compared against alternatives, so it's hard to know what transfers.
Questions for authors

1. Can all models be compared on the same scale?

  • What is the task success rate of SFT+RL on the booking evaluation?
  • What reward does Claude Opus 4.8 get under the RL evaluation protocol?
  • What do the base, SFT and SFT+RL models score on OSWorld (Verified) after training, i.e., does specialization cost general ability?

2. Will the booking task be released?

  • Will the booking environments, the evaluation set and the judge be open-sourced, so that others can reproduce the results and compare against them?

3. Did you observe reward hacking?

  • Some reward terms look easy to exploit: +0.15 for asking the user at least once, regardless of whether a question was needed, and more reward for more steps in the no-form scenario. Notably, RL from the base model scores best exactly on the no-form slice (0.706).
  • Did you see the policy learn to ask unnecessary questions, stretch episodes, or otherwise game the reward? A few example trajectories or simple statistics (e.g., number of clarification turns and episode length before vs. after RL) would help.
Reviewer_5LJcOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents an SFT-and-RL pipeline for adapting Qwen3.5-35B-A3B to browser-based booking tasks, supported by a task-specific interaction harness, training data, a user simulator, and an evaluation framework.

Critical strengths

None.

Critical weaknesses

The manuscript reads more like a project report than a research paper. The introduction does not clearly articulate the research motivation, and the manuscript does not adequately explain how the work relates to or differs from prior research in the field. In its current form, it lacks the methodological rigor and empirical evidence expected of an ICLR submission.

Questions for authors

See weaknesses above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.