acceptodds
Under review as a conference paper at ICLR 2027

Repository-Local Decision Context for Reliable Coding Agents

Abstract

Large language model agents can generate plausible software quickly, but production engineering requires preserving product intent, architectural decisions, edge cases, and escalation boundaries. We investigate whether repository-local decision context improves coding-agent reliability and efficiency. This context includes structured product specifications, architecture decision records, task-local guidance, review policies, and human gates. We began with an industrial case. A multi-agent prototype ran approximately 250 agent sessions over 15 hours. It produced a running application and more than 20,000 lines of reviewable code. Hardening later exposed cyclic dependencies, unsuitable architecture, integration assumptions, and widespread test breakage. Narrower work units improved reviewability, yet technically valid changes remained functionally wrong when product decisions were implicit. A production workflow assigned humans responsibility for domain judgment and consequential decisions while agents performed bounded planning, implementation, testing, and review. An observational matched-product comparison reported roughly five times more output per engineer, but cannot isolate the bundled interventions. We propose a controlled, multi-repository benchmark comparing search-only access, global documentation, task-local decision context, and a complete context-review-escalation workflow. Hidden functional tests, machine-checkable architecture constraints, blinded review, and factorial ablations measure correctness, architectural violations, security, correction effort, token use, and abstention. The study tests which human decisions must be externalized for reliable agentic software development.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.