acceptodds
Under review as a conference paper at ICLR 2027

WorkCraft-Arena: How Can Models Turn Heterogeneous Workspaces to Well-Designed Work Artifacts?

Abstract

Multimodal models are increasingly serving as work assistants, turning heterogeneous workspace materials into slides, HTML pages, posters, and documents. Evaluating these artifacts requires more than checking task completion, because long-horizon creation can lose evidence between understanding and delivery. Additionally, existing aesthetic assessments lack an exploration of the professional delivery requirements and diverse design tastes of work artifacts. We introduce WorkCraft-Arena, a benchmark of 100 tasks across 10 professional work roles and six artifact types. We propose a unified design agent harness that records all creation processes from material understanding to self-revision for complete evaluation. Our arena has established the first comprehensive aesthetic evaluation system for work assistant artifacts, which separates faithfulness from aesthetics and further distinguishes AestheticPass, an absolute professional-readiness judgment, from AestheticElo, a pairwise measure of relative design preference. We conduct experiments across six frontier models and observe that faithfulness and aesthetics are almost orthogonal on the Pareto front. AestheticPass and AestheticElo will present different performance in the system. The results also show that factual grounding, design taste, and delivery reliability diagnose distinct capabilities for models. WorkCraft-Arena therefore provides both a credible measurement system for artifact aesthetics and diagnostic evidence for developing multimodal models and agent harnesses that produce reliable, improvable professional work.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.