ReAD: Reinforcement-Guided Capability Distillation for Large Language Models

Capability distillation specializes a smaller language model toward a specified capability using supervision from a larger teacher. Under a fixed distillation-token budget, an important question is how to allocate supervision across capabilities to improve the target most effectively. Our empirical study shows that single-capability distillation produces budget-dependent cross-capability changes and diminishing target gains, motivating allocation strategies that adapt to the student's evolving state. We propose ReAD, a Reinforcement-guided cApability Distillation framework that combines a task-conditioned allocation prior, on-the-fly teacher supervision, and an uncertainty-aware contextual bandit. ReAD updates its allocation using measured capability changes, rewarding task-aligned gains and penalizing regressions on prioritized capabilities. Across eight target capabilities, two teacher–student pairs, and two token budgets, ReAD achieves higher mean on-target scores than the evaluated target-only baselines. Matched-generator comparisons show additional gains over static and greedy allocation schedules, while transfer diagnostics show smaller off-target declines. Our code is available at: https://anonymous.4open.science/r/ReAD-62B7.