acceptodds
Under review as a conference paper at ICLR 2027

KnowAct-GUIClaw: Know Deeply, Act Perfectly, Personal GUI Assistant with Self-Evolving Memory and Skill

Abstract

OpenClaw has emerged as a leading agent framework for complex task automation, yet its variants face two core bottlenecks: insufficient cross-platform GUI interaction support and no built-in self-evolution mechanism. We propose the "**Know Deeply, Act Perfectly**" paradigm, in which accumulated human–machine interaction and task-running experience improve execution accuracy and efficiency. Based on this paradigm, we introduce **KnowAct-GUIClaw**, a Know–Route–Act–Reflect framework that pairs a host agent for knowledge-grounded, long-horizon task decomposition with a pluggable GUI subagent equipped with experience-attributable memory and a self-evolving skill library. User profiles and feedback guide task decomposition and tool calls, while validated skills enable fast-path execution. Experiments across Android, iOS, HarmonyOS, and Windows examine manipulation efficiency, accuracy, and cross-platform adaptability. On the 117-task MobileWorld GUI-Only benchmark, the Kimi-K2.6 configuration achieves 64.1% pass@1, exceeding the compared systems, including Seed-2.0-Pro and GPT-5.5. Memory and skills support gains of 8.5 percentage points with Kimi-K2.6 and 16.2 points with Qwen3.5-35B-A3B, the latter using Kimi-derived experience.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.