Refining Problem Specifications via Trace-Level Diagnosis for Reasoning Data Synthesis

Synthetic data has become central to LLM post-training, but existing synthesis methods differ in what they tell the generator to synthesize. We call this conditioning signal the specification. Prior methods often use task descriptions or capability labels derived from problems the student answered incorrectly. These signals identify broad weak regions, but they do not explain where the student's reasoning failed. We propose Diagnosis-Driven Synthesis (DDS), a framework that uses the student's incorrect solution trace as a more specific synthesis signal. DDS first performs trace-centric diagnosis: a teacher inspects each failed solution and produces an actionable directive describing where the reasoning broke and what a new problem should require. DDS then introduces diagnostic crossover, which pairs two such diagnoses and asks a generator to create a single coherent problem that requires both weaknesses to be repaired jointly. A verifier filters generated candidates for well-posedness and solution correctness before fine-tuning. On math reasoning, DDS recovers $+12.6$ and $+8.0$ Pass@1 points on the student's failed problems over the base student on Qwen2.5-3B-Instruct and Llama-3.2-1B-Instruct, and improves the held-out math average by $+7.54$ and $+8.32$ points. At the same patch budget, DDS exceeds the strongest baseline by $+2.82$ and $+3.18$ points on the held-out math average. Iterative re-diagnosis further lifts the math average to $48.1$. Beyond math, DDS reaches $59.93$ on a seven-benchmark multimodal average, $+2.26$ over the strongest baseline.