No Task Vector Is an Island: A Comprehensive Study on the Composability of Task Vectors from On-Policy Distillation

Task vectors provide a simple mechanism for composing learned capabilities through model merging. However, the composability of task vectors produced by on-policy distillation (OPD) remains largely unexplored. OPD trains a stu- dent using teacher feedback on student-generated trajectories, yielding parameter updates that differ from those produced by the teacher model, usually by rein- forcement learning (RL). We therefore ask whether OPD task vectors can comple- ment their RL teacher updates and compose effectively across tasks. Across five domains and two model architectures, we find evidence for both forms of compos- ability. Within a task, merging OPD and RL task vectors can outperform both constituent models, even when the OPD student is weaker than its RL teacher. Across tasks, OPD task-vector compositions achieve higher average scores than corresponding RL compositions in seven of eight backbone–merging-rule com- parisons. Parameter-space analyses reveal substantial non-collinearity between OPD and RL updates. Experiment in CODE domain on SMOLLM3-3B shows that the combined direction outperforms either constituent direction at the tested global update norm, supporting directional complementarity in this configuration. Across tasks, OPD updates also show lower overlap among the top-10% feed- forward channels ranked by update energy. Together, these results show that weaker standalone performance does not imply weaker task-vector composabil- ity. OPD task vectors can complement stronger RL teacher updates and combine effectively across tasks, highlighting composability as a distinct property for un- derstanding and evaluating post-training updates.