MetaCurate: Evolving Curation Filters for High-Quality Agentic Coding Data

Supervised fine-tuning (SFT) on agent trajectories is an effective way to improve coding agents, but selecting useful training examples requires more than identifying successful runs. A resolved patch may follow inefficient reasoning or unnecessary tool use, while a failed run may still contain useful intermediate steps. Training pipelines therefore rely on rules and qualitative assessments to filter trajectories, yet designing these filters typically requires substantial human expertise, and validating improvements through repeated training and evaluation is expensive. We introduce MetaCurate, a training-free framework that evolves executable trajectory curation filters using pairwise LLM judgments. Disagreements between the filter's rankings and the judge's preferences provide targeted feedback for jointly revising its rules, rubric, and aggregation policy. The resulting filter scores trajectories independently, so pairwise comparisons are required only during search. Starting from a blank filter, MetaCurate autonomously discovers curation strategies that outperform a strong hand-authored filter refined through human expertise. We validate downstream utility through controlled SFT experiments on our uncurated trajectory pool and the Nemotron SWE dataset, with random selection and hand-authored filtering as baselines. In our experiments, MetaCurate yields the best performance among the compared methods on SWEVerified, comparing to hand-authored filter and the filter crafted with 3x expensive filter training-loop.