KnowAct-Memory: A Self-Evolving Skill Memory for Continuous Learning in Mobile Agents

Mobile GUI agents powered by multimodal large language models have been used to automate complex tasks. However, existing agents typically suffer from memoryless planning and regenerate action sequences from scratch for every task. This inefficiency increases inference latency and token cost while limiting the ability to learn from experience. We propose KnowAct-Memory, a self-evolving framework that shifts mobile GUI agents from heavy online planning to efficient state-grounded execution. Instead of replaying raw trajectories, KnowAct-Memory abstracts interaction traces into structured parametric skills, merges and refines them through semantic consolidation, and maintains a dual library distilled from both successful executions and repaired failures. Experiments on AndroidWorld, MobileWorld, and real-device HarmonyOS evaluations show that KnowAct-Memory improves measured success rates while reducing inference cost. With Qwen3-VL-8B on AndroidWorld, it improves success from 45.68% to 52.58% and reduces per-step inference time by 25.4%.