- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes GMOPD, a gated meta-learning approach to multi-teacher on-policy distillation. It freezes the teacher models and the student’s base weights, represents multi-domain knowledge through shared LoRA matrices, and introduces learnable rank gates. During inner-loop adaptation to a teacher, only the gates are updated to restrict the degrees of freedom. The outer loop optimizes the shared gate initialization based on post-adaptation distillation performance, while training the shared LoRA basis using the multi-teacher distillation loss evaluated before adaptation. This design aims to make the student readily adaptable to individual teachers while remaining directly deployable as a unified model, thereby mitigating cross-teacher interference and forgetting.
Critical strengths
1 The related work is reviewed reasonably thoroughly.
2 The paper identifies the phenomenon that stronger teachers can lead to a worse merged student, which is a practically meaningful research problem.
3 The method is simple and clearly motivated.
Critical weaknesses
1 What does “Preliminary projected” mean in the captions of Tables 4–6 in the appendix? In this context, “projected” suggests estimated rather than measured results, raising serious concerns about the reliability of the reported experimental findings.
2 The baselines, such as ExOPD, are implemented using LoRA updates. Could this constrain their performance, given that these methods predominantly use full-parameter updates in their original papers? At a minimum, the authors should report the performance difference between full-parameter and LoRA-based training for ExOPD.
3 Where is Figure 3 referenced in the main text? Furthermore, its model-size labels—2B, 4B, and 9B—are inconsistent with the experimental settings described in the text: 4B, 8B, and 30B-A3B. Please explain this discrepancy.
4 What are the specific configurations of the multiple teachers within each domain, and how do they differ?
5 In Figure 2, the baseline methods show almost no change in performance as the number of teachers increases. Why is this the case? Is applying the paper’s teacher configuration directly to the other methods a fair comparison?
6 Why were no experiments conducted in thinking mode, given that it is commonly used with many current models?
Questions for authors
see Weaknesses
2 comments
Dear Reviewer,
As the author–reviewer discussion period is approaching its end, we wanted to kindly follow up to ask whether our rebuttal have addressed your concerns, or if there are any remaining issues that we could further clarify.
We would be more than happy to provide additional explanations or evidence before the discussion closes. If you feel that your concerns have been sufficiently addressed, we would sincerely appreciate it if you could acknowledge this in the discussion.
Thank you again for your time and consideration!
Best regards, The Authors
W1: What does “Preliminary projected” mean in the captions of Tables 4–6 in the appendix? In this context, “projected” suggests estimated rather than measured results, raising serious concerns about the reliability of the reported experimental findings.
To clarify, “preliminary projected” refers to our preliminary assessment of variability based on a limited number of measured experimental runs. The revised Tables 4–6 report measured results from three independent seeds, and we have updated the captions to state this explicitly. We also provide the individual results for all five seeds in Table.7-9.
One source of variability is teacher construction: domain-specific RL can produce teachers with different capabilities across seeds, which in turn changes the supervision received by the student during MOPD. This upstream variability may contribute to the variation in the final student performance and should not be conflated with the stability of distillation under fixed teachers.
W2: The baselines, such as ExOPD, are implemented using LoRA updates. Could this constrain their performance, given that these methods predominantly use full-parameter updates in their original papers? At a minimum, the authors should report the performance difference between full-parameter and LoRA-based training for ExOPD.
To clarify, all the baseline methods are implemented by their publicly available code with full-parameter updates, which is the same as their original settings.
In addition, we verify the performance of these baseline methods with LoRA-based training.
The results confirm that restricting ExOPD to LoRA updates can affect its performance. Our method nevertheless outperforms full-parameter ExOPD across all five domains, indicating that its gains do not arise from restricting the baseline to LoRA-based training.
W3: Where is Figure 3 referenced in the main text? Furthermore, its model-size labels—2B, 4B, and 9B—are inconsistent with the experimental settings described in the text: 4B, 8B, and 30B-A3B. Please explain this discrepancy.
Thank you for pointing this out. We have corrected the model-size labels in Figure.3 to 4B, 8B, and 30B-A3B, and carefully check the manuscript.
W4: What are the specific configurations of the multiple teachers within each domain, and how do they differ?
We organize the multiple teachers within each domain according to the specific task categories defined by established benchmarks. Taking the coding domain as an example, we configure four specialized teachers following the four scenarios of LiveCodeBench: (1) Code Generation, which generates solutions from natural language problem descriptions; (2) Self-Repair, which corrects erroneous code based on execution errors or failing test cases; (3) Code Execution, which predicts program outputs given code snippets and input values; and (4) Test Output Prediction, which predicts expected outputs from problem descriptions and test inputs without access to the implementation.
These teachers differ in their task formulations, input contexts, and required reasoning capabilities, allowing them to provide complementary supervision for different aspects of coding proficiency. We add these teacher configurations and their respective roles in Appendix C.1.