SWE-Craft: Graph-Guided Task Synthesis for Weak Spots in Coding Agents
Abstract
Executable software engineering (SWE) tasks provide controlled environments for studying and improving coding agents. Existing perturbation-based synthesis scales task generation but offers limited control over the engineering practice represented in the resulting data. We study target-conditioned SWE task synthesis: a supplied capability target specifies desired model behavior or reasoning and guides task design before construction. In our experiments, model-assisted analysis and human synthesis of base-model trajectories motivate these targets. We introduce SWE- Craft, which uses a source-linked code property graph to explore functional modules across functions and files. A task design agent matches repository behavior to a capability target and specifies required and preserved behaviors. The task specification then guides separate construction and validation of executable bug- fixing, feature-addition, and feature-enhancement tasks. Quality-screened teacher trajectories provide supervision for fine-tuning. In a source-blinded LLM audit, SWE-Craft scores 93.16 out of 100 for alignment with a shared task profile, compared with 53.36 for SWE-smith and 42.67 for BugPilot. Fine-tuning Ling-3.0- flash on 1,500 trajectories achieves 46.88% and 68.00% Pass@1 on SWE-bench Pro and SWE-bench Multilingual, outperforming SWE-smith by 2.74 and 3.55 percentage points, and BugPilot by 2.05 and 3.00 points, respectively
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.