- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes MT-SDPO, which combine teacher feedback with correct student rollouts to guide OPD by a EMA self-teacher. From my aspects, the main contribution of this paper is to develop a data curation and synthesis pipeline for LLM MOPD. The pipeline select teachers via answer verification and constructs privileged supervision for SDPO.
Critical strengths
- The research gap is well identified, and the framework follows it logical.
Critical weaknesses
- This method is limited in practical use. 1) inference cost is a major issue in LLM RL and MOPD. Teacher selection and privileged context construction require teacher inference, which could become expensive with multiple large teachers. The paper provides no analysis of the total computational cost and cost-effectiveness. 2) the method emphasizes rollout accuracy through answer verification. The limits of verification are mentioned, but the paper does not discuss how correct final answers with incorrect reasoning chains affect teacher eligibility and online student supervision. 3) the baseline selection is limited. Since the main contribution is data curation and synthesis, SDPO and Domain Routing do not establish how this pipeline compares with alternative data curation and synthesis methods. This comparison is unfair.
- The writing is hard to follow in the method and experiment sections. While using AI to drafting is acceptable, the generated texts need more careful academic inspecting and editing. In this paper, 1) some content is misplaced and irrelevant. such as the last two paragraphs of section 4.3. 2) some non-academic expressions are used. 3) equation references are inconsistent, such as "Equation (4)" vs. "Eq. (7)".
Questions for authors
- What is the total computational cost relative to SDPO and Domain Routing, including offline eligibility checks, online feedback generation, and EMA self-teacher evaluation?