PhotoArena: Benchmarking Pure-GUI Computer-Use Agents in Adobe Photoshop via Dual-Track Verification

Professional creative software offers multiple routes to task completion, including visual GUI interaction and application scripting. When programmatic shortcuts are available, successful task completion alone does not establish an agent's ability to perform the corresponding visual interactions. We therefore focus on a specific question: can computer-use agents complete professional editing workflows using only screenshots, mouse actions, and keyboard inputs? Studying this capability raises two evaluation challenges. First, the interaction protocol needs to distinguish pure GUI performance from task completion through scripts or application APIs. Second, assessing these workflows calls for evidence of both execution and delivery: a final rendered image can conceal missing task-required editable structures, while final artifacts alone do not reveal whether required intermediate operations were performed. To address these challenges, we introduce **PhotoArena**, a benchmark for professional creative tasks in Adobe Photoshop under a strict **GUI-only interaction protocol**, paired with a **dual-track evaluation framework**. The Outcome Delivery Track checks prompt-required PSD structure and visual completeness before assessing visual consistency, instruction following, and visual quality. The Process Adherence Track aligns execution trajectories with semantic operations using an LLM-based forward anchor and evaluates step-level execution without requiring agent frames to visually match the expert during alignment. We report the two tracks separately to distinguish procedural adherence from final deliverable quality. Evaluated frontier agents execute at least 60% of task-relevant operations, yet end-to-end task success remains at or below 10% under the GUI-only protocol. These findings highlight the difficulty of translating local interaction successes into complete, editable deliverables under the GUI-only protocol.