- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Agent-as-Annotators (A3), an automated framework for synthesizing web agent training data and distilling frontier model capabilities into smaller models. The framework uses persona-conditioned exploration to generate tasks and evaluation hints, a teacher agent to collect trajectories, and an LLM judge to filter successful executions. The resulting dataset, A3-Synth, is used to fine-tune Qwen3.5 models through SFT. Experiments across WebArena and several unseen environments demonstrate substantial improvements over the base models using SFT alone, without reinforcement learning, suggesting promising cross-environment generalization. Ablation studies further examine the contributions of different components, including personas, evaluation hints, judge filtering, and reasoning traces.
Critical strengths
- The paper presents an effective pipeline for synthesizing web agent trajectories, demonstrating that a relatively small amount of synthetic data can substantially improve smaller agents through SFT alone. This provides a practical approach to improving web agent capabilities without reinforcement learning.
Critical weaknesses
- It remains unclear whether the reported cross-environment generalization gains stem from A3’s specific design or simply from training on high-quality trajectories generated by a strong teacher model. The comparisons with existing synthesis methods are not sufficiently convincing to establish the advantages of the proposed framework.
- The framework relies heavily on LLM-based verification to construct high-quality training data, yet the reliability of the judge is not independently evaluated.
Questions for authors
- How do the authors demonstrate that the proposed components contribute to cross-environment generalization?
- Has the reliability of the proposed LLM judge been validated? In particular, how well do its predictions align with ground-truth verification or human judgments?
- The comparisons with existing trajectory synthesis methods are difficult to interpret, as both NNetNav-style and Go-Browse perform substantially worse than the untuned base model. Could the authors explain this degradation and clarify whether the training configurations and data formats were properly aligned?
- How does persona conditioning improve the diversity of interaction workflows, rather than merely the topical diversity of generated tasks?
3 comments
Thank you for the follow-up and for considering raising your score.
I would appreciate a more detailed discussion of the OOD generalization results you mentioned.
Yes, we are happy to expand on them. A3-Qwen3.5-9B improves on all four benchmarks whose environments were never seen during the training (Table 1): WorkArena L1 from 33.3% to 51.5%, WorkArena++ L2 from 2.2% to 9.7%, MiniWoB from 63.2% to 69.0%, and Online-Mind2Web (300 tasks on 136 live websites) from 46.3% to 53.7% (Appendix A.3). The persona module also transfers to these enviroments (368-trajectory models from Table 3):
| Training data | WorkArena L1 | MiniWoB | WorkArena++ L2 |
|---|---|---|---|
| With personas | 49.4 | 64.7 | 8.1 |
| Without personas | 35.0 | 57.4 | 1.6 |
I also notice that a substantial performance gap remains between the distilled student and the teacher model, despite training on a diverse set of synthetic trajectories from WebArena.
Given its size, the student is closer to the teacher than this suggests: although it is a 9B model trained on only 16,353 examples, it recovers about half of the teacher's advantage over the base model on WebArena (31.0% to 41.5%, teacher 51.2%) and matches Qwen3.5-27B, a model three times larger. A3 therefore transfers a large part of a frontier model's capability, for about $3.45K of teacher calls, to a model that runs locally on a single 48 GB GPU. Ultimately, our work shows how to narrow the performance gap enough to make local models useful for browser-use, which is an important step forward for both researchers working on efficient agents and end users wanting to use local web agents.
While the paper demonstrates promising improvements using SFT alone, the training relies exclusively on successful trajectories. What is the authors' perspective on leveraging failed trajectories (e.g., via error-recovery synthesis, contrastive objectives, or reinforcement learning) to help close this remaining gap on complex OOD tasks?
We believe A3 is well positioned to learn from failed trajectories, since the judge component already cagtegorize themby rejecting 677 of the 2,999 episodes and explains why in each case. Directly training on them hurt performance (training on all 2,999 without judge filtering lowers WebArena success from 41.5% to 37.0%, see Table 3). Though the judgments are used for rejection finetuning in our work, it can also be used for reinforcement learning where the judgments can be converted to positive and negative reward; our framework already fully supports this. That said, we believe that this will require substantial changes to the infrastructure needed for efficient training on websites that need to be rapidly reset and rebuilt and error-tolerent; in the current research ecosystem, neither are well supported for browser-use environments. That said, we believe this is an exciting direction for follow-up works, for which we lay the foundations.
Thank you for raising score
Dear reviewer,
Thank you for raising your score following our response. We are looking forward incorporating your feedback and suggestions into the updated manuscript.
Dear Authors,
Thank you for the detailed responses and additional analyses. Several of your new results help clarify my questions.
I would appreciate a more detailed discussion of the OOD generalization results you mentioned.
I also notice that a substantial performance gap remains between the distilled student and the teacher model, despite training on a diverse set of synthetic trajectories from WebArena. While the paper demonstrates promising improvements using SFT alone, the training relies exclusively on successful trajectories. What is the authors' perspective on leveraging failed trajectories (e.g., via error-recovery synthesis, contrastive objectives, or reinforcement learning) to help close this remaining gap on complex OOD tasks?
Overall, your responses have addressed some of my key concerns within the scope of this paper, and I am considering raising my score.