- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces Work2Skill, a framework for converting human GUI demonstrations into reusable skills for computer-use agents. It includes WorkRecorder, which synchronizes screen recordings, mouse/keyboard events, and available accessibility-tree information, and WorkDemo-350, a curated dataset of 350 successful demonstrations across MyPCBench and OSWorld-Verified. The proposed agentic skill-induction method selectively inspects multimodal evidence and iteratively generates reusable procedural guidance in the Agent Skills format. Experiments across four execution models and two benchmarks demonstrate improvements in downstream task performance over several baselines.
Critical strengths
- The paper investigates the challenge of transferring procedural knowledge from human GUI demonstrations to computer-use agents, with a clear focus on reusable skills and downstream task execution.
- The idea of active exploring skill based on current task state is intuitive.
- The experiments is comprehensive, evaluating performance across two benchmarks and four execution models, with additional experiments on task-instance generalization, cross-model transfer, and evidence ablations.
Critical weaknesses
- This work involves two closely related works: ShowUI-Aloha and Demo2Tutorial. A more detailed comparison with them preferably in a table, would help clarify Work2Skill's methodological novelty and key differences.
- In the experiment part, ShowUI-Aloha retains only action descriptions, while Demo2Tutorial is reduced to text-only guidance. Although a unified interface is reasonable, removing essential components of the original methods may underestimate their capabilities.
- The contributions of WorkRecorder and adaptive evidence exploration are not fully isolated. From my understanding, active & adaptive exploration is one key contribution of the work, which should be investigated seriously. Although Figure 5(b) compares induction prompts, it does not directly evaluate adaptive versus static evidence exploration under identical inputs and computational budgets.
Questions for authors
See weakness.
After reviewing the authors’ response and additional experiments, I find that the task-paired evaluation design has a reasonable basis. The Jitter and cross-task evaluations with frozen skills also partially address my concern about leakage by demonstrating reuse under changed task requirements. I have therefore raised my score to 3.