LoopsBench: From Harness Engineering to Loop Engineering in Coding Agent Evaluation

Long-horizon coding requires agents to implement interdependent requirements, track unfinished work, and preserve completed functionality. Existing benchmarks largely emphasize final outcomes, providing limited insight into this process. We introduce LOOPSBENCH, a benchmark for evaluating loop engineering: the mechanisms that organize agent work across successive steps. It comprises 112 tasks across 8 programming languages and 9 domains, with more than 5,300 separately testable development units organized into dependency DAGs recovered from source materials. The evaluator activates each unit’s hidden tests once its prerequisite units pass their tests and reruns activated tests on subsequent code snapshots to track progress and regressions. We also analyze recorded plans, code changes, test activity, and evaluation outcomes. Across the evaluated models and harnesses, external continuation generally improves resolution. The strongest configuration, Opus-4.7 with Claude Code and external continuation, resolves 25.00% of tasks. Further analysis shows incomplete agreement between recorded plans and recovered dependencies, longer patches than the gold reference on resolved units, and regressions across the four configurations examined in the execution analysis.