acceptodds
Under review as a conference paper at ICLR 2027

Built on Broken Assumptions (BOBA): Why Coding Agents Cannot Be Left Alone

32 views

Abstract

Coding agents are autonomous developers. As such, they can help not only when each change is reviewed before the next is requested, but also when a whole sequence of requests is left to them, each built on the last. Although agents rarely notice their own mistakes and report wrong work as complete, evaluation has focused on a single request from human-written code, verified once. In this work, we build BOBA-BENCH, 280 chains of 20 consecutive requests from 140 real repositories, and compare agents on single requests and across chains. All five frontier agents we test complete far fewer chains than requests: 48% on average against 94–98%. Two matched comparisons show that frontier agents’ verified code is as good a base as human code, and that the degradation comes from building on unverified code. Agents submit work as complete without confirming it and build on the broken state unaware, whereas one external “that’s wrong” and a repair round lift chain completion by 8 to 16 points. In simpler terms, what limits coding agents left alone is not the code they write but the mistakes they cannot see: without verification, every request that follows is built on broken assumptions.

Primary area:
foundation or frontier models, including LLMs
Keywords:
models, llms, agents, coding, verification, tests, bugs, benchmarks

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_4rZ1OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates coding agents that tackle a sequence of dependent requests. This paper proposes BOBA-Bench, which has 280 chains of 20 consecutive requests. Data are constructed from 140 Python repositories, and each request is graded by hidden developer tests. This paper evaluates five frontier agents in settings that separate who writes the underlying code from whether that code is verified. Agents complete 48% of chains on average while passing 94-98% of individual requests, fail the next request far more often after an unverified mistake, and gain 8-16 points of chain completion when simply told that verification failed.

Critical strengths
  1. This is a carefully engineered benchmark that measures something single-request evaluation cannot. Coding agents can tackle a sequence of dependent requests. This paper helps measure beyond single requests, constructing data from history commits of 140 repositories.

  2. This paper performs informative experiments. Because runs can resume from the exact failed state, the paper can compare matched arms. The results are quantitative and informative for actionable analysis.

Critical weaknesses
  1. The paper does not establish that a silent miss is the agent's mistake, and its own evidence suggests that a substantial share are not.

In Table 21, it shows 25.2% of the 575 first-attempt misses (about 145) as "Benchmark-side, suspected: test stricter than the specification, broken or ambiguous". The share is 31% for both Fable 5 and GPT-5.6. Yet the main text says a miss "can be attributed to the agent" in section 3.1, and its root-cause list in section 4.2 gives four agent-side causes without mentioning this category.

Also, section D.5 states that "no human-labeled adequacy sample exists yet". Claude Opus models, with GLM-5.2 as an outage fallback, write the specifications. GLM-5.2, Fable 5, and GPT-5.6 audit them with the hidden tests visible, and GPT-5.6 and Fable 5 rewrite the rejected ones. So three of the five evaluated agents audit the specifications they are tested on, and two of them also rewrite some.

There's also other evidence on this, such as:

  • 886 benchmark specifications (20.1%) are never solved blind by either model in 16 attempts (Table 14). They touch 204 chains, and 9 chains consist entirely of them.
  • When two models fail the same chain, they fail at the same request 53.5% of the time, against 5.2% if failure positions are independent (F.7).
  • The post hoc R15 round removes 14 chains whose hidden tests presuppose the human implementation of an earlier request. That defect class is found only where a model happened to expose it.

So all the main findings assume that misses are agent errors: the completion rates, the "mistakes they cannot see", and above all the near-zero reviewer discrimination (Youden J of 0.01-0.15, Table 11). Near-zero discrimination is exactly what to expect if many "wrong" implementations are wrong only relative to tests the specification does not determine. So this can decrease Fable 5's per-request miss rate, yet greatly raise the constant-rate prediction of its completion.

  1. Main claims go beyond what the designs can support.

The chain ends at the first miss, so a constant per-request miss rate already predicts the "48% of chains versus 94-98% of requests" gap. F.7 shows that (1−h)^20 gives 67.9/53.3/48.2/45.9/30.7% against the observed 65.7/52.1/48.2/43.9/27.9%, and per-quarter miss rates rise only slightly over the chain (Table 24). The abstract says "the degradation comes from building on unverified code". But in the autonomous setting, no request is graded on an unverified state.

Also, Appendix J itself warns against treating a non-significant result as equivalence, and several conditions here favor finding no gap, such as:

  • 72-91% of boundaries pass 8/8 from both prefixes (Table 9)
  • 40-64 of the 140 repositories never produce a certified agent prefix and drop out
  • Agent prefixes are retried until they are certified
  • R15 removes 14 chains selected because the agent prefix failes where the human prefix passes, which is the contrast being measured

Section 4.2 states in bold that the odds of failing roughly double "with every additional unverified mistake", while F.4 calls the progression "descriptive, not a controlled dose". The claim that "a repaired state is no longer a broken assumption" (in section 5.1) rests on repaired subsets selected on outcome (F.3, K.1).

So the abstract's two conclusions "degradation comes from building on unverified code" and "verified agent code is as good a base as human code" are the paper's conceptual contribution, and neither is yet supported as stated.

Questions for authors
  1. How can you validate that a silent miss is the agent's mistake, as the evidence suggests that a substantial share are not?

  2. Would you be able to validate and explain the contradictory evidence in weakness 1?

  3. What is the per-request miss rate by specification class? Such as hard but fair, repaired pending validation, rewritten by GPT-5.6 or Fable 5, and original text? How do you reconcile 20% of specifications going 0/16 blind with per-request pass rates of 94-98%?

  4. In arm D, how many own-work failures at k+1 occur in tests that call symbols introduced or changed by request k? How is "request overlaps buggy code" defined in Figure 4?

  5. The noisy supervisor raises false alarms but doesn't miss a true failure? What happens to completion when the supervisor also misses true failures, for example, at recall 0.5-0.9, or when the supervisor is one of the section 4.3 model reviewers? The findings and conclusions in section 5 depend on this.

Reviewer_pGi3OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
  • This work tackles whether agents can handle successive changes to an evolving codebase.
  • The authors introduce BOBA-Bench with 280 chains of 20 requests across 140 repositories.
  • This work evaluated five coding agents and find that a high per-request pass rate does not translate to chained requests.
Critical strengths
  • Compared to prior benchmarks that also tackle repository evolution, BOBA-Bench has more task and repo coverage.
Critical weaknesses
  • While Table 1 compares BOBA-Bench with SWE-Chain and SWE-Milestone, the results are less clear about how these differences affect the findings.
  • The results show that small harness changes substantially improve performance, suggesting the initial claims might be overstated. Further, Fable 5’s completion rate after intervention shows that most chains are solvable with reliable feedback.
  • The organization of results makes it hard to understand nuanced findings beyond what is illustrated in the Figure 1 right. The finding is perhaps unsurprising given prior work; could the authors elaborate on this?
Questions for authors

Please address above weaknesses.

Reviewer_uoPGOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces BOBA-Bench, a benchmark for evaluating coding agents on sustained software development across 280 twenty-request chains drawn from 140 Python repositories. It investigates whether downstream failures arise from limitations of agent-written code as a foundation for further development or from undetected errors in earlier implementations. Experiments with five models compare development on verified human- and agent-written code, continuation from faulty states, cross-model code review, and externally triggered repair. The results suggest that verified agent-written prefixes often support subsequent development at rates close to human-written prefixes, whereas unresolved errors substantially increase downstream failure. External failure feedback combined with repair opportunities improves full-chain completion. The main contribution is a systematic empirical investigation of verification and error recovery in long-horizon coding, supported by a substantial executable benchmark.

Critical strengths

S1. The experimental design addresses an important distinction in long-horizon coding reliability. The paper goes beyond documenting declining success over longer task sequences by separating code authorship from the correctness of the preceding implementation. The verified-prefix comparisons and faulty-state continuation experiments provide complementary evidence about why subsequent development fails. In particular, evaluating the next request on its own tests avoids mechanically counting inherited test failures as new failures. This makes the analysis more informative than aggregate chain-completion results alone.

S2. Broad experimental coverage and substantial validation strengthen the empirical contribution. The study combines five models, 140 repositories, cross-model review, repair interventions, and supervisor false-alarm experiments. The appendices also report repository-clustered uncertainty, test-visibility ablations, harness sensitivity, and detailed benchmark construction and filtering procedures. These analyses provide a useful foundation for studying persistent development and assessing the practical limitations of automated verification.

Critical weaknesses

W1. Outcome-dependent benchmark filtering creates an unresolved selection risk for the central authorship comparison. Appendix D.10 and Table 17 disclose the post hoc removal of 14 chains from six repositories after evaluation revealed failures on agent-written prefixes but success on human-written prefixes. The authors attribute these cases to hidden tests presupposing the human implementation. Removing invalid tests is appropriate, but this performance asymmetry alone does not establish test invalidity: it could also reflect specification violations or latent defects in agent implementations that earlier tests failed to detect. Because the exclusion criterion is directly related to the phenomenon under investigation, the paper needs case-level justification and sensitivity results before and after exclusion. Without this evidence, the robustness of the conclusion that verified agent code is comparably suitable for further development remains uncertain.

W2. The interpretation of comparable performance is stronger than the reported evidence supports. Section 4.2 states that the four stronger models perform equally well on verified human and agent code at any depth. However, Table 9 reports a Fable 5 success-rate difference of −2.3 percentage points with a 95% confidence interval of [−5.2, +0.3], which does not exclude a meaningful disadvantage. Its pass@8 difference is −3.6 points, with an interval of [−7.1, −0.5]. Moreover, these comparisons concern selected, successfully certified prefixes and measure next-request success rather than maintainability more broadly. The evidence supports broadly similar average performance under the tested conditions, but does not establish general equivalence. This distinction matters because the limited role attributed to authorship is central to the paper’s interpretation.

W3. The verification intervention does not fully isolate the value of feedback from the benefit of additional repair opportunities. Section 5 and Appendix K show that external failure feedback followed by repair improves chain completion. However, the intervention jointly supplies information and additional opportunities to revise the implementation. The fresh-conversation control addresses restarting the conversation, but does not provide an equivalent repair opportunity without informative failure feedback. Consequently, the results establish the effectiveness of the combined intervention rather than the independent contribution of the verification signal. Furthermore, the noisy-supervisor experiments vary false-positive rates while detecting every genuine failure, limiting conclusions about realistic supervisors that also miss errors.

Questions for authors
  1. Post hoc filtering: Could you provide case-level evidence that the 14 excluded chains contain tests requiring behavior not specified in the requests, together with the verified-prefix comparison before and after these exclusions? This would help assess whether filtering materially affects the central authorship conclusion.

  2. Scope of the equivalence claim: How should the statement that the four stronger models perform equally well at any depth be reconciled with Fable 5’s negative pass@8 difference in Table 9? What performance difference would you consider practically meaningful when assessing equivalence?

  3. Attribution of repair gains: Do you have a comparison that grants the same repair opportunity and budget without revealing a genuine verification failure? If not, would you scope the conclusion to the combined benefit of external feedback and additional repair?

  4. Consistency of statistical claims: Appendix J withdraws parts of the benchmark-separation argument, while Appendix H and Tables 26–27 retain the earlier analysis. Which comparative claims remain supported in the final manuscript, and does their revision change the claimed advantage over existing benchmarks?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.