CREATIVE DESIGN OF AGENTIC SYSTEMS

Frontier models are increasingly contributing to mathematical problem solving and scientific discovery, making creativity a central capability to evaluate beyond established task performance. Existing agent benchmarks keep the agent architecture fixed and measure execution ability, while creativity studies assess generated content rather than the executable systems agents build. Therefore, we present Agents Creating Agents (ACA), the first benchmark for evaluating the creativity of agents from an architectural perspective: an architect transforms real development tasks into an executable architecture satisfying utility, novelty, and cost awareness, which together form the Creativity score. We further propose Evidence-Grounded Mechanism Search (EGMS), guides architecture search using the causal structure of failures rather than a scalar development score. EGMS improves mean utility by 6.4 points over the strongest of five baselines and raises the Creativity score to 0.272 $\uparrow$. Further analysis shows that the harness influences architectural creativity as much as the backbone model, and a small local model with a compiled domain architecture outperforms on the Codex harness with runtime skill injection, suggesting that architecture can substitute for model scale at execution time. Our code is open at https://anonymous.4open.science/r/ACA-BENCHMARK-2775/.