acceptodds
Under review as a conference paper at ICLR 2027

Learn from Weaknesses: Student-aware Domain Specialization for Small Computer-Use Agents

Abstract

Small computer-use agents (CUAs) are attractive where cost, latency, or privacy matter, but they lag far behind large models in domain-specific software. We find that scaling up generic domain data yields limited gains, as small students differ in their weaknesses even within the same software. Existing failure-driven methods do adapt to the student, but they draw the reference for correct behavior from the student itself, leaving the failures it cannot yet solve uncorrected. We introduce LearnWeak, an annotation-free framework that makes specialization student-aware by contrasting the student with a stronger teacher, which can be a black-box API model. At the task level, it learns from correctable failures, tasks that the teacher solves but the student fails, and synthesizes practice that targets the missing skills. At the state level, it replays the student along the teacher's successful trajectories and corrects only the part of its action that deviates, depending on whether the failure stems from planning or execution. Training on only a few dozen trajectories per domain, LearnWeak improves EvoCUA-8B and OpenCUA-7B by 11.6 and 11.1 points on average across eight OSWorld domains, lets the 8B student surpass its 32B teacher on two domains where it initially trailed by more than 10 points, and outperforms existing data construction methods, including failure-driven training, under the same data budget. Our analyses further show that specialization data is student-dependent, as data built from another student's failures can even underperform the unadapted model. Together, these results suggest that specializing small CUAs hinges less on scaling data than on diagnosing and correcting the failures of the particular student.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Computer Use Agents, Domain Specialization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_A6QuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces LEARNWEAK, a framework for specializing small computer-use agents by targeting their specific weaknesses. It identifies tasks where a stronger teacher succeeds but the student fails, generates practice tasks addressing those failures, and uses selective DPO to correct planning and execution errors. Across eight OSWorld domains, it improves EvoCUA-8B by 11.6 percentage points and OpenCUA-7B by 11.1 points using only a few dozen trajectories per domain. The key finding is that student-specific failure correction is more effective than scaling generic training data.

Critical strengths
  1. Good results on one dataset.
Critical weaknesses
  1. Only evaluated on OSWorld? Not sure how it generalize
  2. Not sure how this is new from previous papers on distillation. Maybe you should talk more on what previous papers did and how yours is different.
Questions for authors
  1. Can you run on more datasets?
  2. Please specify your diff from previous papers.
Reviewer_HUKhOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces LEARNWEAK, a framework for specializing small computer-use agents through student-aware data generation and selective preference optimization. At the task level, the framework identifies correctable failures: tasks that a stronger teacher completes successfully but the student fails. It summarizes these failures into weakness reports and synthesizes targeted practice tasks. At the state level, the student is queried along successful teacher trajectories, and teacher–student action disagreements are converted into preference pairs. A response-span mask determines which tokens receive a DPO loss based on whether the disagreement concerns the tool function or its parameters.

Critical strengths
  1. Substantial improvements across domains and students. Table 1 reports improvements in all eight evaluated domains for both main students. EvoCUA-8B improves from 50.69% to 62.24%, approaching the teacher’s 63.80% macro-average. Improvements across two different starting models make the practical result more compelling.

  2. Informative controls for the training objective. Table 5 holds training examples and optimizer updates fixed while varying objectives and masks. LEARNWEAK-DPO reaches 55.20%, compared with 45.58% for standard DPO and 48.88% for similarly masked SFT. This is strong evidence that the proposed supervision recipe matters beyond simply collecting successful demonstrations.

Critical weaknesses

W1. Action disagreement does not establish a valid preference ordering

In Section 3.2 and Equation 6, the rejected response is newly sampled from the student under a teacher context. It is therefore different from the response observed in the original failed student trajectory. Failure of the original rollout does not establish that this new response is inferior to the teacher’s action.

W2. Student-specific data effects are partly confounded by task-pool differences

Table 3 provides useful evidence that data transfer depends on the source student: both targets perform best on average with their own source-conditioned data, and foreign-source data sometimes reduces performance. However, changing the source student changes both task generation and selection. Matching optimizer updates leaves differences in task difficulty, coverage, retained steps, and unique supervision.

Questions for authors
  1. What aggregation procedure explains the Table A6 percentages, and is the same procedure used in the main tables? Can per-task and per-run outcomes be provided?
  2. What fraction of retained preference pairs contains a semantically valid alternative student action? How often do the syntactic error categories agree with semantic judgments?
  3. Does the own-student data advantage persist when selection operates on a common generated pool with matched retained-step or token budgets?
Reviewer_A5mGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper presents LearnWeak, a distillation method for CUA agents that involves collecting trajectories that teachers succeed and students struggle on, and using a preference optimization loss (DPO) to learn the teacher's behavior when the student's predicted action deviates from what the teacher would have done. The data curation process does not involve any human annotations, but only the teacher / student rollouts and a third-party judge and data synthesis model. Experiments on OSWorld shows improvements on various sub-domains when this procedure is applied.

Critical strengths
  • Empirical results directionally support gains from the proposed approach.
  • The paper presents a good set of ablative studies. Among other things, it shows directionally that the DPO objective is one of the key enablers of the gains observed from the proposed method.
Critical weaknesses
  • The paper could be more explicit with contrasting against existing student-roll-in-teacher-roll-out techniques like DAgger to highlight its contributions and contextualize them in the literature. For instance, the idea of getting teacher demonstration / feedback on student failures can be seen in [1], among other works. A similar DPO-based objective for step-level contrast can also be found in [2].
  • The methodology description and ablations downplayed the role of a critical component: the verifier / query synthesizer / weakness report summarizer. The quality of this model is presumably important to how well the proposed method works, yet it is not clearly explained that this is a third-party strong model (aside from both the teacher model and the student model), which introduces important sources of supervision including data distribution selection and generation that go beyond the teacher's demonstrations. The quality of this model is also not ablated.
  • Empirical results require more statistical rigor to establish gains. The evaluation set of OSWorld are very small: Calc=47 examples, Impress=47, VLC=17, VSCode=23, GIMP=26, Writer=23, Thunderbird=15, OS=24. Most of the gains claimed in the paper amount to the flip of 1-6 examples in each domain, which calls for repeated runs and paired testing to establish where true gains lie.
  • Some technical details are not clearly explained
    • L162 "To generate new queries, ... diverse and semantic ally informative" How specifically?
    • L176 "Iterative generation": Is training involved between the iterations, or just query generation and trajectory collection?

[1] Hailin Chen, Amrita Saha, Steven Hoi, Shafiq Joty. Personalised Distillation: Empowering Open-Sourced LLMs with Adaptive Learning for Code Generation. EMNLP 2023. [2] Weimin Xiong, Yifan Song, Xiutian Zhao, Wenhao Wu, Xun Wang, Ke Wang, Cheng Li, Wei Peng, Sujian Li. Watch Every Step! LLM Agent Learning via Iterative Step-Level Process Refinement. EMNLP 2024.

Questions for authors
  • How did the authors avoid eval data leakage in gathering training examples?
  • How specifically was the ablation on weakness report generation conducted, was the student only trained on the exploration queries? Was the training data budget matched, or did this result in fewer data samples?
  • If the teacher's context is better, why not just use those during training?
  • Why is the reasoning portion never used in DPO, is it really because the reasoning portion does not contribute to errors (the student always has correct reasoning), or that supervision over these long sequences wash out the signal / lead to overfitting of sorts?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.