LEASH: Uncertainty-Aware Weak-to-Strong Escalation and Oversight for Computer Use

Computer-use agents increasingly tackle long-horizon tasks, where running a frontier model at every step is expensive and a small open-weight model alone often fails. Weak-to-strong escalation offers a middle ground: a weak agent acts by default and hands control to a strong one only when needed. The core challenge is deciding when to escalate, which requires an uncertainty signal cheap enough to check at every step. After handoff the risk remains: the strong agent can still go astray, and a black-box API exposes no internals to monitor. To this end, we propose LEASH, a training-free framework that unleashes the strong agent only when the weak agent's latent uncertainty signals it is needed, and keeps it on a leash afterward. LEASH has three layers: initial task uncertainty for early routing, step-level uncertainty for runtime escalation, and, after handoff, a shadow observer for continued oversight of the strong black-box agent. At the core of all three layers is latent uncertainty: we compare the weak model's hidden states with those from successful and failed trajectories, and a state that looks more like the failures signals higher uncertainty. During weak-agent execution, runtime uncertainty reuses activations already produced during action generation, so it needs no extra forward pass or dedicated monitor training. Experiments on OSWorld across four contemporary weak-agent backbones show that LEASH consistently improves over standalone weak agents and outperforms random escalation by 6.55–11.94 points at matched escalation rates. With Qwen3.5-35B-A3B, LEASH achieves 73.66% task success at $0.4117 per task, outperforming always-on Claude Opus 5 (72.90%) while reducing inference cost by 19.2%.