- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies on-policy distillation when the teacher and student use different tokenizers. Shared byte-end offsets partition the two tokenizations into 1:1, 1:n, m:1, and m:n spans; unmatched student positions lack teacher logits, and unmatched teacher decisions sit inside one student token. CASE first maps both next-token distributions onto longest shared byte-prefix events, then recovers teacher targets at unmatched student positions by mixing constrained teacher segmentations of the known span bytes, and compares unmatched teacher decisions using reverse prefix masses from the student’s span-start distribution. All three position types are trained with reverse KL, with the student trajectory length as the denominator. The teacher is frozen; inference uses only the student. Experiments distill three heterogeneous pairings among Qwen3-8B, Phi-4-mini-instruct, and Llama-3.2-3B-Instruct on math and code, and evaluate eight benchmarks.
Critical strengths
1.Cross-tokenizer OPD is split into event alignment and unmatched-position recovery, with two operators that can run under different tokenizations. The paper separates which predictive events can still be compared (shared-prefix projection) from where teacher targets must be obtained at unmatched positions (forward expansion / reverse decomposition). Longest-prefix classes are mutually exclusive and conserve probability mass. Forward expansion mixes constrained teacher segmentations of a known byte string; reverse decomposition stays on the student’s span-start distribution and does not invent student states the student never emitted. The toy grouping in §3.5 makes the objects checkable. This is a concrete alternative to ALM-style span Bernoulli likelihoods and SimCT-style summed log-likelihoods. Proposition 1 only says that a finer shared support cannot decrease KL; that is formal consistency, not evidence that CASE is more effective downstream.
2.CASE is best on average in all three pairings, and expansion has a visible effect on the strongest pairing. Table 1 reports 30.31 / 25.64 / 22.10 versus the strongest baseline in each block (SimCT 29.92, SimCT 24.81, ALM 21.56), i.e. +0.39 / +0.83 / +0.54. CASE is best or tied in 18 of 24 pair–benchmark cells and in the top two in 21. Versus the undistilled students the mean gains are +2.84 / +5.21 / +1.67. On Qwen3→Llama, removing expansion drops the eight-benchmark mean from 25.64 to 23.77 and lowers all eight sets. Table 4 shows that non-shared teacher mass is below 1% while 12–51% of teacher mass sits in merged prefix classes, which is the right statistic for why longest-prefix aggregation is doing work rather than only handling unmatched mass. The teacher stays frozen and inference uses only the student.
Critical weaknesses
1.The results support a positive average on all three pairings, but they are not yet enough to judge stability or the contribution of the two operators across pairings. The mean gaps over the strongest baseline are +0.39 / +0.83 / +0.54. That CASE is best on average in all three pairings is a positive signal and should not be set aside only because two of the gaps are small. Qwen3→Phi is mixed on math: ALM wins OpenR1 (17.75 vs 16.54) and AIME25 (7.92 vs 6.25); MinED wins MATH (52.95 vs 47.48). Phi→Llama trails ALM on AIME24, LCB, and MBPP. Expansion is ablated and the three-seed check is run only on Qwen3→Llama, whose +0.83 is the largest mean gap. Seeds 32 and 52 still beat SimCT on that pairing’s eight-benchmark mean (25.24 / 25.23 vs 24.81); the AIME24 seed standard deviation is 1.44. That comparison is CASE multi-seed against a single SimCT run, not a matched multi-seed comparison; the +0.39 gap in particular still lacks uncertainty on both sides. The present evidence therefore does not yet show that the two operators are driving the result on all three pairings.
2.The comparison set and the ablations do not isolate the claimed increment. The method splits a shared event space from unmatched-position targets, but alignment-only, forward expansion alone, reverse decomposition alone, and full CASE are not reported on all three pairings; the expansion ablation is only on Qwen3→Llama. Section 2 groups GOLD with SimCT as methods that bring cross-tokenizer alignment into on-policy distillation, with GOLD building on ULD. Table 1 already includes ULD and SimCT, but not GOLD. If GOLD cannot be compared under the same model pairings, data, and on-policy budget, stating the difference is enough; the missing row is not by itself a severe soundness defect. The present evidence therefore speaks more directly to CASE having the best average on these three pairings than to the two recovery operators being necessary on each pairing.
Questions for authors
1.What are the separate contributions of forward expansion and reverse decomposition on Qwen3→Phi and Phi→Llama? Please report alignment-only, each operator alone, and full CASE under the same initialization, data, and budget, rather than only the expansion-off result on Qwen3→Llama.
- Section 2 groups GOLD with SimCT as OPD-facing neighbors, with GOLD building on ULD. Table 1 already includes ULD and SimCT, but not GOLD. If the setting is compatible, does CASE still lead under the same student initialization, data, and on-policy budget? If it cannot be compared, what differs in the training protocol or the operating range?
3.Are there multi-seed results on the two pairings with smaller mean gaps (+0.39 / +0.54)? If so, please report uncertainty for the matched baselines under the same seeds, rather than comparing CASE multi-seed runs to a single baseline run. What numeric value of (D_{\max}) was used in Eq. (3), and what are the epoch / step / batch-size values behind the shared on-policy budget in §4.1?
3 comments
Response to Reviewer V9MC (Part 2)
Questions:
Q1: Separate operator contributions across pairings under matched settings.
Forward expansion aligns teacher distributions with unmatched student-token positions, while reverse decomposition uses the student's available distribution to align with teacher decisions at unmatched teacher-token positions. These complementary roles correspond to the four tokenization cases in Figure 2, with direct alignment for 1:1, forward expansion for 1:n, reverse decomposition for m:1, and both at their respective unmatched positions for m:n. Together, they aim to provide more even supervision coverage across token positions within each training example.
Our original ablation disabled both operators to assess supervision expansion as a whole. We thank you for highlighting the importance of separate ablations to distinguish their contributions.
The added Qwen3 → Llama ablations using seed 42 are summarized in W2. Both operators improve Avg. over shared-prefix alignment alone, and their combination performs best. We report these results in Figure 3 and Section 4.3, with complete benchmark-level results in Appendix A.3 (Table 7).
Q2: GOLD comparison under matched settings and protocol compatibility.
For the GOLD baseline, we used the Hugging Face implementation as a reference and evaluated it alongside SimCT and CASE under the same student initialization, training data, on-policy budget, and random seeds. As reported in W2 and revised Table 1, CASE exceeds GOLD's mean Avg. by 3.90, 1.56, and 1.76 percentage points for Qwen3 → Phi-4, Qwen3 → Llama, and Phi-4 → Llama, respectively.
Q3: Matched multi-seed uncertainty, maximum search depth, and on-policy budget details.
Matched three-seed comparisons for CASE and all six baselines across all three pairings are reported in W1 and revised Table 1, with results for each seed in Appendix A.4 (Tables 9–11). We use , as stated in Section 3.3, with search-coverage analysis in Appendix A.5.
We have added the training budget to Table 6. All methods use one epoch, a training batch size of 64, and a micro-batch size of 1, with 117 rollout iterations for math and 130 for code.
Comments on the matched seeds, GOLD, and operator split
Thank you for the matched three-seed table, the GOLD row, and the Qwen3→Llama operator split. I am keeping Rating 3. The new numbers answer Q2 and Q3 and make the average lead easier to trust; they do not yet show that both recovery operators are doing the work on all three pairings.
Matched seeds (W1 / Q3). CASE is 30.25 ± 0.05, 25.37 ± 0.24, and 22.03 ± 0.09 against SimCT 29.77 ± 0.13, SimCT 24.72 ± 0.14, and ALM 21.48 ± 0.10 (+0.48 / +0.65 / +0.55). If CASE is first in average in all nine pair–seed cells, that is the right check for the two smaller original gaps. The margins remain small; they are no longer a single-seed comparison.
GOLD (W2 / Q2). Under the same budget CASE is 3.90 / 1.56 / 1.76 above GOLD. That locates CASE relative to the OPD neighbor named in §2. GOLD is not the strongest baseline in any pairing.
Operators (W1 / Q1). The Qwen3→Llama seed-42 split is useful: shared-prefix 23.77, forward-only 25.05, reverse-only 25.25, full CASE 25.64. Both operators help on that pairing. Q1 asked for the same four-way grid on Qwen3→Phi and Phi→Llama. Those two pairings still have only the average, not the mechanism.
Budget (Q3). One epoch, batch 64, micro-batch 1, and 117 math / 130 code rollout iterations should go in Table 6. The numeric from Eq. (3) did not appear in the posted comment, and the submitted §3.3 only defined the symbol. Please state the value in the revision.
Please put the nine pair–seed averages, the GOLD row, the four-way Qwen3→Llama ablation, and the budget in the revision. Alignment-only and each operator alone on the other two pairings would still be the cleanest test of the two-operator claim.
Response to Reviewer V9MC (Part 1)
Thank you for your careful review and constructive feedback.
Weaknesses:
To address the uncertainty in the smaller margins, we have extended matched evaluation to CASE and all six baselines across all three pairings using seeds 32, 42, and 52. Revised Table 1 reports mean ± sample standard deviation. The table below summarizes Avg. (%), the unweighted mean across eight benchmarks.
CASE ranks first in Avg. in all nine pair–seed combinations, supporting consistent average gains across the evaluated seeds and pairings. Results for each seed are provided in Appendix A.4 (Tables 9–11).
For Qwen3 → Phi-4, the MinED-trained student exhibits repetitive generation. MATH's extraction of the final boxed answer is less affected by this behavior, helping explain its stronger score relative to the other three math benchmarks (Section 4.2 and Appendix A.3).
ALM's advantage varies by task and pairing. We hypothesize that its binary objective over an observed aligned span and its complement can benefit math tasks where a few token decisions strongly affect correctness. Aggregating alternative outcomes leaves their relative probabilities unmatched, which may limit the transfer of richer soft-label information and help explain its less consistent advantage across tasks.
We have added separate ablations of forward expansion and reverse decomposition for Qwen3 → Llama using seed 42, with the same student initialization, training data, and on-policy budget. All four variants retain shared-prefix alignment.
Both operators improve Avg. in this pairing, and their combination performs best. We have updated Figure 3 and Section 4.3, with complete results in Appendix A.3 (Table 7).
We have also added GOLD to revised Table 1 under the same student initialization, training data, and on-policy budget as SimCT and CASE. The table below reports Avg. (%) as mean ± sample standard deviation across seeds 32, 42, and 52.
CASE exceeds GOLD's mean Avg. by 3.90, 1.56, and 1.76 percentage points for Qwen3 → Phi-4, Qwen3 → Llama, and Phi-4 → Llama, respectively.