EvoScaffold: Distilling Executable Strategies for Small Language Models

Reasoning distillation transfers the reasoning capabilities of large teacher models to smaller students for more affordable deployment, typically through supervised fine-tuning. Prompt-level distillation avoids student parameter updates by encoding teacher guidance in system instructions, but complex multi-step problems can still exceed what the student can reliably solve in a single call. We introduce EvoScaffold, a non-parametric distillation framework that jointly evolves problem decomposition and subproblem instructions through genetic search for a frozen student. The resulting executable strategies encode reusable reasoning programs that specify student calls and any dependencies between them. Student predictions and execution traces guide teacher-generated revisions, while student task performance governs strategy selection. Once selected, a strategy is reused on unseen instances of the same task family without teacher access or student parameter updates. Experiments across four reasoning tasks and four student sizes demonstrate the effectiveness of EvoScaffold in improving the reasoning performance of frozen students.