- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies the transferring reasoning abilities from long- to short-context teacher to student models. The focus on on-policy-ness while staying tokenizer agnostic. They show this functioning on several open models specifically intern-s2 seems to provide compelling improvements over closed source variants.
Critical strengths
- The method is simple, and easy to understand. Its more testing a novel paradigm and introducing technical novelties
- The paper is veryt practitioner-friendly: it documents a failure modes of OPD well, e.g. unintended growth of length (or the opposite; truncation) and repetition loops.
- the paper covers quite a lot of models and benches and provides useful ablations (rollout lengths, super relevant for reasoning)
Critical weaknesses
- the main critcism is that tokenizer-agnostic is a bit of a stretch since most of the models have basically the same tokenizer (Lexical overlap ρ is 0.90–0.98), stemming form chinese-english-heavy settings. and maybe unsurprisingly the less similar tokenizer Gemma4 doesnt perform well here.
- generally this reads a bit more like a tech/practitioner report than a fundamental research paper. depending on definitions of novelty, its limited here.
- there are quite some critical confounds: checkpoints are selected based on the exact same test set (AIME), Gemma uses different rollout lengths than the other models, Deepseekv4 is both teacher and judge for proofbench, etc.
Questions for authors
Mostly:
- What is Gemma result with 32k rollouts?
- Can you mitigate the mentioned confounds, e.g. re-select checkpoints on a held-out set? Use different teacher and judge, etc.
Rebuttal
W1 / Q1:"Tokenizer-agnostic" is a stretch; Gemma at 32K rollouts
Gemma at 32K
32K is slightly better than 6K on both benchmarks. The paper used 6K because we held hyperparameters fixed across students for comparability, and at 32K the ref-KL coefficient of 1.0 used elsewhere is unstable on Gemma; 32K requires raising it to 1.5. The 6K setting let us keep a single coefficient across all students. The result above shows the conclusion is not an artifact of the shorter rollout length.
On "tokenizer-agnostic". We will soften the claim to state that the method is validated on high-overlap, Chinese–English-heavy pairs, with Gemma as the lowest-overlap case.
W2:Reads more like a practitioner report; limited novelty
We accept this is a framing disagreement. Our intended contribution is the empirical characterization (supervision coverage, cost, and length dynamics in long-context cross-tokenizer OPD), rather than the alignment rule itself, and we will reframe the paper accordingly.
W3 / Q2:Confounds: checkpoint selection, judge == teacher
Checkpoint selection. We re-selected the checkpoint for Qwen3.5-35B-A3B on Omni-MATH, which is disjoint from every reported benchmark. The held-out criterion selects the same training step, and scores move by less than evaluation noise (~1 pp across seeds), so the contamination did not materialize. We will adopt the held-out criterion in the paper.
Judge == teacher. We re-scored the same saved Section 5.4 proofs under an independent judge (Gemini-3.1-Pro) alongside a same-family one (DeepSeek-V4.1-Flash):
The two judges agree on the base-to-OPD gain to within 0.59 pp, within evaluation noise, so we do not detect self-preference.