CLI-Gym: Scalable CLI Task Generation via Agentic Environment Inversion

Command-line-interface (CLI) agents must diagnose and repair dependencies, permissions, system files, runtime configurations, and source code, yet executable training data for these interactions remain limited. We introduce CLI-Gym, an execution-feedback-guided agentic environment inversion pipeline. Starting from healthy, unit-test-verified repository environments, an inversion agent constructs counterfactual degraded states, summarizes the changes as reproducible Dockerfiles, and pairs observed failures with natural-language issues and executable verifiers. From 4,066 inversion prompts, CLI-Gym produces 1,655 verified CLI task instances; 81.0% contain an environment modification. A repair model solves 417 tasks, from which shortcut and length filtering retains 291 trajectories for supervised fine-tuning. Under the same OpenHands scaffold, these trajectories improve Qwen3-235B-A22B-Instruct by 21.1 points to 46.1% pass@1 on Terminal-Bench 1.0 and by 12.9 points to 31.0% on Terminal-Bench 2.0, with consistent gains at the 32B scale. A three-expert stratified audit separately characterizes the realism, solvability, issue–test consistency, and shortcut resistance of the full task collection. These results establish the training value of the filtered teacher-solvable subset while making the quality boundary of the complete collection explicit.