- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper systematically investigates the composability of task vectors derived from On-Policy Distillation (OPD). Experiments show that merging OPD task vectors with RL teacher vectors can improve single-task performance, and that cross-task merging with OPD vectors generally outperforms merging RL task vectors. The authors validate these findings across two model architectures, five task domains, and multiple merging methods, and further explore potential explanations through parameter-space analyses.
Critical strengths
-
Novel research perspective. The paper investigates the composability of OPD task vectors, highlighting the distinction between standalone model performance and task-vector composability. This provides a new perspective on understanding OPD.
-
Reasonably convincing experimental results. Across two model architectures and multiple task domains, cross-task merging with OPD outperforms RL merging in 7 out of 8 comparisons. Moreover, some same-task merging configurations even surpass the RL teacher models.
-
In-depth parameter-space analysis. The authors analyze the differences between OPD and RL in terms of update norms, sparsity, directional similarity, and channel overlap. They also introduce norm-matching experiments to investigate the directional complementarity of task vectors.
Critical weaknesses
- Confounding factors in the conclusions about cross-task composability. The norms of OPD task vectors are substantially smaller than those of RL task vectors. For example, on SmolLM3, the norms of the Math and Code OPD vectors are only 59.7% and 68.7% of their RL counterparts, respectively. However, Table 2 lacks a norm-matched RL merging baseline. Therefore, the observed advantage of OPD may arise from smaller parameter perturbations rather than inherently better task composability.
- Insufficient evidence for same-task complementarity. The most significant improvement in Table 1 comes from Qwen Code, yet the norm-matching experiment in Section 4.2 is conducted only on SmolLM3 Code. Furthermore, the experiments do not compare against an optimally scaled RL-only direction. Consequently, the current evidence is insufficient to establish that the main performance gains indeed originate from the complementary directions provided by OPD.
- Questionable hyperparameter settings for the merging baselines. Raw Sum is essentially Task Arithmetic (TA) with λ = 1. However, Table 2 shows that TA performs substantially worse than Raw Sum across all configurations. Given the authors' claim that the best merging hyperparameters were selected, why was λ = 1, which would yield a better result, not included in the TA configurations? This raises concerns about the hyperparameter selection and fairness of the baseline comparisons.
Questions for authors
See Weaknesses.
2 comments
A Gentle Follow-up Before the Discussion Period Ends
Dear Reviewer,
Thank you again for your time and thoughtful feedback.
As the discussion period is drawing to a close, we would like to respectfully follow up on our previous message. We would be very grateful if you could let us know whether our responses have adequately addressed your concerns, or if there are any remaining questions that we could further clarify.
We sincerely appreciate your time and consideration.
Best regards,
The Authors
A Gentle Follow-up on Further Feedback
Dear Reviewer,
Thank you very much for your time and valuable feedback.
As the discussion period is approaching its end, with fewer than three days remaining, we would kindly like to ask whether you have any further questions or concerns regarding our responses. We would be more than happy to provide additional clarification if needed.
We sincerely appreciate your time, consideration, and constructive feedback.
Best regards,
The Authors