acceptodds
Under review as a conference paper at ICLR 2027

Edit-Induced Dependency Mismatch: Propagation Omissions, Recovery, and Evaluation Blind Spots in Coding Agents

Abstract

Dependency mismatches induced by code edits can cause task evaluation failures or leave unintended defects along existing consumer paths even when the evaluation passes. Focusing on consumer obligations introduced by edits and existing consumers left unsynchronized, we manually audit 2,879 archived deepseek-v4-flash runs across 64 DeepSWE tasks. Three human reviewers each independently reviewed the entire corpus under blinded conditions, establishing a human-confirmed terminal diagnosis set of 473 runs (16.4%), explicitly including disputed-scope cases, across 37 tasks and 85 distinct within-task consumer relations. Of these runs, 177 have execution-failure evidence and 296 have static reachability evidence, with 0 unresolved evidence-source cases; 290 have undisputed scope for all terminal events, and 158 of these have execution support for at least one terminal event. Recovery events also occur in 370 runs. Five local DeepSWE replays show that synchronizing the diagnosed consumers restores the target behavior while the listed controls continue to pass. In the Kysely case, the original task evaluation passes 276/276 tests, yet a new field is still lost during plugin transformation; adding a single line to copy it restores the semantics. From 500 archived SWE-bench Verified runs, three targeted passing patches expose propagation gaps involving glossary keys, range constraints, and coordinate attributes through execution. Three terminal diagnoses and four recovery runs on FeatureBench extend the evidence to Qwen3.8-Flash. These cases support measuring task scores, terminal propagation states, and recovery separately.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.