acceptodds
Under review as a conference paper at ICLR 2027

ComAct: COM as a Unified Action Space for Professional Software Agents

Abstract

Current computer-use agents remain fundamentally limited on professional software. GUI-based agents struggle with fragile visual grounding and error accumulation over long action sequences, while MCP- and API-based agents require programmable interfaces that closed-source commercial software rarely provides. We observe that the Component Object Model (COM), a Windows-native interface widely supported by commercial professional applications, provides programmatic access to their full functionality through a uniform invocation mechanism. We therefore present ComAct, which uses COM as a unified action space across professional applications, replacing long, fragile GUI actions with compact executable COM programs. We stress-test ComAct on industrial CAD and build ComCADBench, comprising 1,000 tasks across three mainstream CAD applications, spanning standalone design tasks and multi-stage engineering workflows. Frontier models fail almost completely under GUI interaction but achieve non-trivial success when switched to ComAct, and the same switch also yields consistent zero-shot improvements on Office, Adobe, and OS-shell tasks. Yet off-the-shelf models still exhibit a large gap between executable code and task-correct outputs. To close this gap, we build ComForge} a scalable platform of parallel real computer environments for large-scale training, and train a self-correcting agent through a three-stage progressive framework with continuous geometric rewards. Experiments show our agent achieves state-of-the-art performance on ComCADBench, particularly on long-horizon workflows, and generalizes well to external CAD benchmarks. The environment, code and data will all be publicly released.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.