acceptodds
Under review as a conference paper at ICLR 2027

Built for Humans, Not yet for Agents: Turning Apps into Agent-Native CLIs at scale

Abstract

Applications (Apps) expose rich functionalities, such as document editing and image processing, through graphical user interfaces (GUIs). However, reliably interacting with these GUIs remains challenging for large language model (LLM) agents. Command-line interfaces (CLIs) offer an alternative, but generating com- mands for GUI functionalities requires preserving their implicit context and be- havioral constraints. We present APP2CLI, a framework that generates agent- native commands for existing GUI functionalities in apps. APP2CLI traces GUI “trigger-handler-effect” paths to extract behavioral semantics and represent them in a structured semantic specification. Skill guidance directs the construction of CLI templates through public programmatic interfaces. Cooperating agents im- plement templates as standalone, parameterized commands. We evaluate APP2CLI on 1,467 target GUI functionalities across 11 application units from 9 apps. APP2CLI achieves a 60.05% agent-native commands gen- eration success rate , producing semantically correct commands for 881 target functionalities, . The CLI executions also achieve 98.01% line-coverage overlap with the corresponding GUI executions. Removing the semantic specification or skill-guided template construction reduces the generation success rate to 1.77% and 0.14%, respectively. Code is available at https://zenodo.org/records/22970644.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.