acceptodds
Under review as a conference paper at ICLR 2027

Beyond Fail-to-Pass: Iterative Hardening of Co-Generated Bug Reproduction Tests and Fixes

Abstract

Bug reproduction tests (BRTs) turn bug reports into executable signals for automated program repair, but the usual fail-to-pass (\ftp) criterion does not establish whether a test supports correct repair. A test may fail on buggy code and pass on the golden fix while also accepting plausible-but-incorrect patches. We introduce Mutation Patch Evaluation () to distinguish rigorous tests from such lax tests, and show empirically that downstream repair gains concentrate in the rigorous tier. We also identify error coupling in co-generation, where a generated test accepts an incorrect generated patch. Based on these findings, we propose , a co-generation framework for iterative test-and-fix refinement. An initial test-only round targets the observable bug symptom before committing to a fix. Subsequent hardening rounds use mutated variants of the current fix to strengthen the test and refine the patch. On SWE-bench Verified, reaches Resolved, improving on the strongest fix-only and co-generation baselines by and percentage points, respectively. Further experiments show gains across LLM backbones and benchmarks. Code is available at https://anonymous.4open.science/r/COHARDEN-9753.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.