acceptodds
Under review as a conference paper at ICLR 2027

Structured Distillation of Web Agent Capabilities Enables Generalization

Abstract

Web agents powered by frontier models are increasingly able to navigate complex websites, but they rely heavily on proprietary APIs, which increases costs and hinders for local deployments. To solve this, smaller models can be hosted locally, but are often less capable. We take a step towards bridging this gap by introducing AGENT-AS-ANNOTATORS, an automatic framework for distilling the capabilities of frontier models to navigate websites into smaller models, which substantially improves their abilities to solve tasks on websites. The framework is designed to replicate how humans would collect trajectories for training web agents; but the entire pipeline is automated using modular LLM components based on frontier models. Using Gemini 3 Pro as the teacher model, we generate 3000 trajectories across six web environments from WebArena, which we filter and use to train a student (Qwen-3.5-9B) with only supervised finetuning (SFT). We find that the resulting agent can transfer beyond the environments used in its training. Notably, on WorkArena L1, success rises from 33.3% to 51.5%, and performance also improves across three additional benchmarks, each testing a different type of generalization. On WebArena, it achieves a success rate of 41.5% despite never seeing any of the benchmark tasks, which nearly doubles the best same-size agent (Go-Browse, 21.7%). Through ablations, we find that the model benefits from each of its components (personas, judge, hints) and reasoning traces. Our results show that structured trajectory synthesis from a single teacher can produce competitive yet locally deployable web agents. We release the code, data and family of models (ranging from 2 to 9B parameters) publicly.

Primary area:
applications to robotics, autonomy, planning
Keywords:
web agents, capability distillation, synthetic trajectories, supervised fine-tuning, web navigation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_535MOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Agent-as-Annotators (A3), an automated framework for synthesizing web agent training data and distilling frontier model capabilities into smaller models. The framework uses persona-conditioned exploration to generate tasks and evaluation hints, a teacher agent to collect trajectories, and an LLM judge to filter successful executions. The resulting dataset, A3-Synth, is used to fine-tune Qwen3.5 models through SFT. Experiments across WebArena and several unseen environments demonstrate substantial improvements over the base models using SFT alone, without reinforcement learning, suggesting promising cross-environment generalization. Ablation studies further examine the contributions of different components, including personas, evaluation hints, judge filtering, and reasoning traces.

Critical strengths
  1. The paper presents an effective pipeline for synthesizing web agent trajectories, demonstrating that a relatively small amount of synthetic data can substantially improve smaller agents through SFT alone. This provides a practical approach to improving web agent capabilities without reinforcement learning.
Critical weaknesses
  1. It remains unclear whether the reported cross-environment generalization gains stem from A3’s specific design or simply from training on high-quality trajectories generated by a strong teacher model. The comparisons with existing synthesis methods are not sufficiently convincing to establish the advantages of the proposed framework.
  2. The framework relies heavily on LLM-based verification to construct high-quality training data, yet the reliability of the judge is not independently evaluated.
Questions for authors
  1. How do the authors demonstrate that the proposed components contribute to cross-environment generalization?
  2. Has the reliability of the proposed LLM judge been validated? In particular, how well do its predictions align with ground-truth verification or human judgments?
  3. The comparisons with existing trajectory synthesis methods are difficult to interpret, as both NNetNav-style and Go-Browse perform substantially worse than the untuned base model. Could the authors explain this degradation and clarify whether the training configurations and data formats were properly aligned?
  4. How does persona conditioning improve the diversity of interaction workflows, rather than merely the topical diversity of generated tasks?
3 comments

Dear Authors,

Thank you for the detailed responses and additional analyses. Several of your new results help clarify my questions.

I would appreciate a more detailed discussion of the OOD generalization results you mentioned.

I also notice that a substantial performance gap remains between the distilled student and the teacher model, despite training on a diverse set of synthetic trajectories from WebArena. While the paper demonstrates promising improvements using SFT alone, the training relies exclusively on successful trajectories. What is the authors' perspective on leveraging failed trajectories (e.g., via error-recovery synthesis, contrastive objectives, or reinforcement learning) to help close this remaining gap on complex OOD tasks?

Overall, your responses have addressed some of my key concerns within the scope of this paper, and I am considering raising my score.

Thank you for the follow-up and for considering raising your score.

I would appreciate a more detailed discussion of the OOD generalization results you mentioned.

Yes, we are happy to expand on them. A3-Qwen3.5-9B improves on all four benchmarks whose environments were never seen during the training (Table 1): WorkArena L1 from 33.3% to 51.5%, WorkArena++ L2 from 2.2% to 9.7%, MiniWoB from 63.2% to 69.0%, and Online-Mind2Web (300 tasks on 136 live websites) from 46.3% to 53.7% (Appendix A.3). The persona module also transfers to these enviroments (368-trajectory models from Table 3):

Training dataWorkArena L1MiniWoBWorkArena++ L2
With personas49.464.78.1
Without personas35.057.41.6

I also notice that a substantial performance gap remains between the distilled student and the teacher model, despite training on a diverse set of synthetic trajectories from WebArena.

Given its size, the student is closer to the teacher than this suggests: although it is a 9B model trained on only 16,353 examples, it recovers about half of the teacher's advantage over the base model on WebArena (31.0% to 41.5%, teacher 51.2%) and matches Qwen3.5-27B, a model three times larger. A3 therefore transfers a large part of a frontier model's capability, for about $3.45K of teacher calls, to a model that runs locally on a single 48 GB GPU. Ultimately, our work shows how to narrow the performance gap enough to make local models useful for browser-use, which is an important step forward for both researchers working on efficient agents and end users wanting to use local web agents.

While the paper demonstrates promising improvements using SFT alone, the training relies exclusively on successful trajectories. What is the authors' perspective on leveraging failed trajectories (e.g., via error-recovery synthesis, contrastive objectives, or reinforcement learning) to help close this remaining gap on complex OOD tasks?

We believe A3 is well positioned to learn from failed trajectories, since the judge component already cagtegorize themby rejecting 677 of the 2,999 episodes and explains why in each case. Directly training on them hurt performance (training on all 2,999 without judge filtering lowers WebArena success from 41.5% to 37.0%, see Table 3). Though the judgments are used for rejection finetuning in our work, it can also be used for reinforcement learning where the judgments can be converted to positive and negative reward; our framework already fully supports this. That said, we believe that this will require substantial changes to the infrastructure needed for efficient training on websites that need to be rapidly reset and rebuilt and error-tolerent; in the current research ecosystem, neither are well supported for browser-use environments. That said, we believe this is an exciting direction for follow-up works, for which we lay the foundations.

AuthorsOpenReview ↗

Thank you for raising score

Dear reviewer,

Thank you for raising your score following our response. We are looking forward incorporating your feedback and suggestions into the updated manuscript.

Reviewer_U7hPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Agent-as-Annotators generates web-agent training data through persona-conditioned exploration, task and hint generation, separate task execution, and judge filtering. Gemini 3 Pro produces 3,000 tasks in six WebArena environments; the retained 2,322 trajectories provide 16,353 observation–action examples. SFT improves Qwen3.5 students. For the 9B model, SR rises from 31.0% to 41.5% on the WebArena test split and from 33.3% to 51.5% on WorkArena L1. The paper reports other benchmark and online-website results, data scaling, and component ablations.

Critical strengths
  1. The WorkArena L1 gain of 18.2 percentage points is measured on a different platform from the training environments. Gains on other benchmarks and Online-Mind2Web support transfer beyond the six training websites.
  2. The paper tests persona, hint, judge, reasoning, and data-size variants and trains the same student on alternative datasets. Full-benchmark paired statistics support the reported improvement over the base model.
Critical weaknesses
  1. The methodological advantage is less well isolated than the gain over the base student. Table 17 does include the same student trained on alternative data, but it does not hold generation models, formats, and generation cost fixed. The persona/hint ablations support improvements on WebArena, but transfer results for these ablations are not reported. Section 6 presents the persona explanation as a hypothesis, which is appropriate. My concern is the strength of the comparative method contribution, not whether the observed transfer exists.
  2. Data-quality calibration, cost accounting, and result reconciliation would strengthen the paper. No independent calibration of the training-data judge is reported. Benchmark performance is evaluated separately, so this concern is about training-data quality. Retained sample counts also do not establish monetary or compute efficiency without generation costs. Finally, Tables 5 and 14–16 report inconsistent per-site values for the full A3-Qwen3.5-9B configuration despite referring to the same 381-task WebArena evaluation. These discrepancies require reconciliation, although they do not by themselves establish that the overall SR or paired improvements are invalid.
Questions for authors
  1. Do the existing persona or hint ablations retain their gains on one held-out benchmark? This would help assess whether these design choices contribute to transfer, given the teacher and data-format differences in Table 17.
  2. Can an independent sample audit estimate the training-data judge's false accepts and false rejects, especially for decisions changed by hints? This would clarify the quality benefit beyond the downstream ablation.
  3. Which runs or checkpoints produced the full-model per-site values in Tables 5 and 14–16? Please reconcile these results, clarify Table 6's split markers, and report the total generation budget, including exploration, execution, and judging.
1 comment
AuthorsOpenReview ↗

10 days since our response was posted; we are looking forward for your reply

Dear reviewer U7hP,

We haven't heard back since we posted our response 10 days ago. We are looking forward your reply, as it is the last day of discussions.

Reviewer_RQjKOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper focuses on the difficulty of obtaining effective training demonstrations for compact browser agents without extensive human annotation. It proposes a method for constructing and selecting training data. A large model generates user profiles and explores websites under those profiles. The resulting observations are used to formulate tasks together with descriptions of what successful completion should look like. A separate execution pass attempts each task using only its instruction. An evaluator then examines the execution, with access to additional task information, and selects demonstrations for training.

Critical strengths
  1. Table 1 and Section 5.1 show an 18.2 improvement on WorkArena L1. This is stronger evidence of transferable browser behavior than improvement on new instructions within the training websites alone.

  2. Both the 9B and 4B students improve on all five benchmarks. The 2B student improves on four, with WorkArena++ remaining at zero. The 9B student also matches the reported 27B checkpoint on WebArena. These results make the dataset useful beyond one isolated model–benchmark combination.

  3. The paper distinguishes information available to the task executor from information supplied to the evaluator. Keeping evaluation hints out of the execution context helps avoid demonstrations that depend on assistance unavailable during deployment. Grounding task creation in observed website content also addresses a concrete source of invalid synthetic instructions.

  4. Table 3 shows that filtering improves performance despite retaining fewer demonstrations, and that removing action rationales reduces success. For the smaller persona and hint experiments, the results explicitly warns against comparing blocks with different dataset sizes, which is appropriate.

Critical weaknesses
  1. All component ablations evaluate WebArena, while the strongest transfer result concerns WorkArena. Therefore, it is unclear whether personas or evaluator hints cause better generalization.

  2. Personas, exploration, trajectory filtering, and supervised training already appear in prior work such as NNetNav. Tables 1–2 do not isolate the benefit of the proposed combination under matched teacher, student, and data budgets.

  3. The same model generates tasks, executes them, and judges success, creating a risk of correlated errors. Improved downstream performance after filtering does not establish accurate labels or explain how hints help.

  4. Tables 1 and 3 omit uncertainty estimates and repeated-run variation. The hint improvement corresponds to approximately nine additional successes, while subset selection and training-token budgets remain unclear.

  5. VisualWebArena partly shares training environments, and screenshots are already training inputs; WorkArena and WorkArena++ share ServiceNow.

  6. Retained dataset size excludes exploration, unsuccessful attempts, and judging costs. Smaller models suggest easier deployment, but Figure 2’s single example cannot establish average execution efficiency.

Questions for authors
  1. What are the numerical results of the NNetNav-style and InSTA-style comparisons mentioned in Section 5.2? Which teacher, student, data budget, training settings, and evaluation settings were held fixed?

  2. Are any component-ablation results already available on WorkArena or another environment excluded from training?

  3. How were the 600- and 368-trajectory datasets selected? Were website composition, training steps, and token counts comparable? For the rationale ablation, what changed during training and inference?

  4. How many runs or seeds underlie Tables 1 and 3?

  5. Do existing logs provide total collection cost? like exploration and rejected attempts, and the hardware and latency of student inference.

1 comment
AuthorsOpenReview ↗

10 days since our response was posted; we are looking forward for your reply

Dear reviewer RQjK,

We haven't heard back since we posted our response 10 days ago. We are looking forward your reply, as it is the last day of discussions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.