acceptodds
Under review as a conference paper at ICLR 2027

Align and Expand: Recovering Dense Supervision for Cross-Tokenizer On-Policy Distillation

Abstract

On-policy distillation (OPD) aligns teacher knowledge along student-generated trajectories, reducing the discrepancy between training and inference. This advantage has made OPD an increasingly prominent post-training paradigm. However, when the teacher and student use different tokenizers, their vocabularies differ and some student prediction positions are unmatched by the teacher tokenization, rendering conventional distillation objectives inapplicable. Existing methods circumvent this token mismatch through span-level alignment, but typically compress each aligned span into a binary distribution. Such binary representation discards token-level supervision within each span, as well as the dark knowledge contained in the teacher’s probability distribution. To address these limitations, we propose **Cross-Tokenizer Alignment and Supervision Expansion (CASE)**, an align-and-expand framework that establishes span-level correspondences across heterogeneous tokenizers and recovers missing intra-span supervision through teacher-context expansion, ultimately enabling dense supervision at the span level. Across mathematical reasoning and code generation, CASE achieves the best average performance in all three teacher–student pairings, demonstrating robust gains under heterogeneous tokenizers.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Cross-Tokenizer Distillation, On-Policy Distillation, Large Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_V9MCOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies on-policy distillation when the teacher and student use different tokenizers. Shared byte-end offsets partition the two tokenizations into 1:1, 1:n, m:1, and m:n spans; unmatched student positions lack teacher logits, and unmatched teacher decisions sit inside one student token. CASE first maps both next-token distributions onto longest shared byte-prefix events, then recovers teacher targets at unmatched student positions by mixing constrained teacher segmentations of the known span bytes, and compares unmatched teacher decisions using reverse prefix masses from the student’s span-start distribution. All three position types are trained with reverse KL, with the student trajectory length as the denominator. The teacher is frozen; inference uses only the student. Experiments distill three heterogeneous pairings among Qwen3-8B, Phi-4-mini-instruct, and Llama-3.2-3B-Instruct on math and code, and evaluate eight benchmarks.

Critical strengths

1.Cross-tokenizer OPD is split into event alignment and unmatched-position recovery, with two operators that can run under different tokenizations. The paper separates which predictive events can still be compared (shared-prefix projection) from where teacher targets must be obtained at unmatched positions (forward expansion / reverse decomposition). Longest-prefix classes are mutually exclusive and conserve probability mass. Forward expansion mixes constrained teacher segmentations of a known byte string; reverse decomposition stays on the student’s span-start distribution and does not invent student states the student never emitted. The toy grouping in §3.5 makes the objects checkable. This is a concrete alternative to ALM-style span Bernoulli likelihoods and SimCT-style summed log-likelihoods. Proposition 1 only says that a finer shared support cannot decrease KL; that is formal consistency, not evidence that CASE is more effective downstream.

2.CASE is best on average in all three pairings, and expansion has a visible effect on the strongest pairing. Table 1 reports 30.31 / 25.64 / 22.10 versus the strongest baseline in each block (SimCT 29.92, SimCT 24.81, ALM 21.56), i.e. +0.39 / +0.83 / +0.54. CASE is best or tied in 18 of 24 pair–benchmark cells and in the top two in 21. Versus the undistilled students the mean gains are +2.84 / +5.21 / +1.67. On Qwen3→Llama, removing expansion drops the eight-benchmark mean from 25.64 to 23.77 and lowers all eight sets. Table 4 shows that non-shared teacher mass is below 1% while 12–51% of teacher mass sits in merged prefix classes, which is the right statistic for why longest-prefix aggregation is doing work rather than only handling unmatched mass. The teacher stays frozen and inference uses only the student.

Critical weaknesses

1.The results support a positive average on all three pairings, but they are not yet enough to judge stability or the contribution of the two operators across pairings. The mean gaps over the strongest baseline are +0.39 / +0.83 / +0.54. That CASE is best on average in all three pairings is a positive signal and should not be set aside only because two of the gaps are small. Qwen3→Phi is mixed on math: ALM wins OpenR1 (17.75 vs 16.54) and AIME25 (7.92 vs 6.25); MinED wins MATH (52.95 vs 47.48). Phi→Llama trails ALM on AIME24, LCB, and MBPP. Expansion is ablated and the three-seed check is run only on Qwen3→Llama, whose +0.83 is the largest mean gap. Seeds 32 and 52 still beat SimCT on that pairing’s eight-benchmark mean (25.24 / 25.23 vs 24.81); the AIME24 seed standard deviation is 1.44. That comparison is CASE multi-seed against a single SimCT run, not a matched multi-seed comparison; the +0.39 gap in particular still lacks uncertainty on both sides. The present evidence therefore does not yet show that the two operators are driving the result on all three pairings.

2.The comparison set and the ablations do not isolate the claimed increment. The method splits a shared event space from unmatched-position targets, but alignment-only, forward expansion alone, reverse decomposition alone, and full CASE are not reported on all three pairings; the expansion ablation is only on Qwen3→Llama. Section 2 groups GOLD with SimCT as methods that bring cross-tokenizer alignment into on-policy distillation, with GOLD building on ULD. Table 1 already includes ULD and SimCT, but not GOLD. If GOLD cannot be compared under the same model pairings, data, and on-policy budget, stating the difference is enough; the missing row is not by itself a severe soundness defect. The present evidence therefore speaks more directly to CASE having the best average on these three pairings than to the two recovery operators being necessary on each pairing.

Questions for authors

1.What are the separate contributions of forward expansion and reverse decomposition on Qwen3→Phi and Phi→Llama? Please report alignment-only, each operator alone, and full CASE under the same initialization, data, and budget, rather than only the expansion-off result on Qwen3→Llama.

  1. Section 2 groups GOLD with SimCT as OPD-facing neighbors, with GOLD building on ULD. Table 1 already includes ULD and SimCT, but not GOLD. If the setting is compatible, does CASE still lead under the same student initialization, data, and on-policy budget? If it cannot be compared, what differs in the training protocol or the operating range?

3.Are there multi-seed results on the two pairings with smaller mean gaps (+0.39 / +0.54)? If so, please report uncertainty for the matched baselines under the same seeds, rather than comparing CASE multi-seed runs to a single baseline run. What numeric value of (D_{\max}) was used in Eq. (3), and what are the epoch / step / batch-size values behind the shared on-policy budget in §4.1?

3 comments
AuthorsOpenReview ↗

Response to Reviewer V9MC (Part 1)

Thank you for your careful review and constructive feedback.

Weaknesses:

W1: Stability and operator contributions across model pairings.

To address the uncertainty in the smaller margins, we have extended matched evaluation to CASE and all six baselines across all three pairings using seeds 32, 42, and 52. Revised Table 1 reports mean ± sample standard deviation. The table below summarizes Avg. (%), the unweighted mean across eight benchmarks.

Teacher → StudentStrongest baselineCASEGain (percentage points)
Qwen3 → Phi-4SimCT: 29.77 ± 0.1330.25 ± 0.05+0.48
Qwen3 → LlamaSimCT: 24.72 ± 0.1425.37 ± 0.24+0.65
Phi-4 → LlamaALM: 21.48 ± 0.1022.03 ± 0.09+0.55

CASE ranks first in Avg. in all nine pair–seed combinations, supporting consistent average gains across the evaluated seeds and pairings. Results for each seed are provided in Appendix A.4 (Tables 9–11).

For Qwen3 → Phi-4, the MinED-trained student exhibits repetitive generation. MATH's extraction of the final boxed answer is less affected by this behavior, helping explain its stronger score relative to the other three math benchmarks (Section 4.2 and Appendix A.3).

ALM's advantage varies by task and pairing. We hypothesize that its binary objective over an observed aligned span and its complement can benefit math tasks where a few token decisions strongly affect correctness. Aggregating alternative outcomes leaves their relative probabilities unmatched, which may limit the transfer of richer soft-label information and help explain its less consistent advantage across tasks.

W2: Component ablations and the comparability of GOLD.

We have added separate ablations of forward expansion and reverse decomposition for Qwen3 → Llama using seed 42, with the same student initialization, training data, and on-policy budget. All four variants retain shared-prefix alignment.

VariantAvg. (%)
Shared-prefix only23.77
Forward expansion only25.05
Reverse decomposition only25.25
Full CASE25.64

Both operators improve Avg. in this pairing, and their combination performs best. We have updated Figure 3 and Section 4.3, with complete results in Appendix A.3 (Table 7).

We have also added GOLD to revised Table 1 under the same student initialization, training data, and on-policy budget as SimCT and CASE. The table below reports Avg. (%) as mean ± sample standard deviation across seeds 32, 42, and 52.

Teacher → StudentGOLDSimCTCASE
Qwen3 → Phi-426.35 ± 0.0629.77 ± 0.1330.25 ± 0.05
Qwen3 → Llama23.81 ± 0.3124.72 ± 0.1425.37 ± 0.24
Phi-4 → Llama20.27 ± 0.0721.28 ± 0.0422.03 ± 0.09

CASE exceeds GOLD's mean Avg. by 3.90, 1.56, and 1.76 percentage points for Qwen3 → Phi-4, Qwen3 → Llama, and Phi-4 → Llama, respectively.

AuthorsOpenReview ↗

Response to Reviewer V9MC (Part 2)

Questions:

Q1: Separate operator contributions across pairings under matched settings.

Forward expansion aligns teacher distributions with unmatched student-token positions, while reverse decomposition uses the student's available distribution to align with teacher decisions at unmatched teacher-token positions. These complementary roles correspond to the four tokenization cases in Figure 2, with direct alignment for 1:1, forward expansion for 1:n, reverse decomposition for m:1, and both at their respective unmatched positions for m:n. Together, they aim to provide more even supervision coverage across token positions within each training example.

Our original ablation disabled both operators to assess supervision expansion as a whole. We thank you for highlighting the importance of separate ablations to distinguish their contributions.

The added Qwen3 → Llama ablations using seed 42 are summarized in W2. Both operators improve Avg. over shared-prefix alignment alone, and their combination performs best. We report these results in Figure 3 and Section 4.3, with complete benchmark-level results in Appendix A.3 (Table 7).

Q2: GOLD comparison under matched settings and protocol compatibility.

For the GOLD baseline, we used the Hugging Face implementation as a reference and evaluated it alongside SimCT and CASE under the same student initialization, training data, on-policy budget, and random seeds. As reported in W2 and revised Table 1, CASE exceeds GOLD's mean Avg. by 3.90, 1.56, and 1.76 percentage points for Qwen3 → Phi-4, Qwen3 → Llama, and Phi-4 → Llama, respectively.

Q3: Matched multi-seed uncertainty, maximum search depth, and on-policy budget details.

Matched three-seed comparisons for CASE and all six baselines across all three pairings are reported in W1 and revised Table 1, with results for each seed in Appendix A.4 (Tables 9–11). We use Dmax⁡=3D_{\max}=3, as stated in Section 3.3, with search-coverage analysis in Appendix A.5.

We have added the training budget to Table 6. All methods use one epoch, a training batch size of 64, and a micro-batch size of 1, with 117 rollout iterations for math and 130 for code.

Reviewer_V9MCOpenReview ↗

Comments on the matched seeds, GOLD, and operator split

Thank you for the matched three-seed table, the GOLD row, and the Qwen3→Llama operator split. I am keeping Rating 3. The new numbers answer Q2 and Q3 and make the average lead easier to trust; they do not yet show that both recovery operators are doing the work on all three pairings.

Matched seeds (W1 / Q3). CASE is 30.25 ± 0.05, 25.37 ± 0.24, and 22.03 ± 0.09 against SimCT 29.77 ± 0.13, SimCT 24.72 ± 0.14, and ALM 21.48 ± 0.10 (+0.48 / +0.65 / +0.55). If CASE is first in average in all nine pair–seed cells, that is the right check for the two smaller original gaps. The margins remain small; they are no longer a single-seed comparison.

GOLD (W2 / Q2). Under the same budget CASE is 3.90 / 1.56 / 1.76 above GOLD. That locates CASE relative to the OPD neighbor named in §2. GOLD is not the strongest baseline in any pairing.

Operators (W1 / Q1). The Qwen3→Llama seed-42 split is useful: shared-prefix 23.77, forward-only 25.05, reverse-only 25.25, full CASE 25.64. Both operators help on that pairing. Q1 asked for the same four-way grid on Qwen3→Phi and Phi→Llama. Those two pairings still have only the average, not the mechanism.

Budget (Q3). One epoch, batch 64, micro-batch 1, and 117 math / 130 code rollout iterations should go in Table 6. The numeric Dmax⁡D_{\max} from Eq. (3) did not appear in the posted comment, and the submitted §3.3 only defined the symbol. Please state the value in the revision.

Please put the nine pair–seed averages, the GOLD row, the four-way Qwen3→Llama ablation, and the budget Dmax⁡D_{\max} in the revision. Alignment-only and each operator alone on the other two pairings would still be the cleanest test of the two-operator claim.

Reviewer_PPUhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CASE addresses cross-tokenizer on-policy distillation through shared byte-prefix alignment and supervision expansion within aligned spans. It constructs additional targets using alternative teacher tokenizations and reverse prefix decomposition. Experiments show the best average performance across several model pairings, with improvements over the strong baselines.

Critical strengths
  • The distinction between vocabulary alignment and missing prediction-position supervision is clear and well-motivated.

  • CASE provides a reasonable solution of the alignment scenarios and addresses intra-span supervision.

Critical weaknesses
  • Experiment is not sufficient for bar of ICLR. The teacher is up to 8B, and student is over 3B. Need to examine over larger teacher and smaller students.

  • The augmented objectives need stronger probabilistic justification, particularly for overlapping reverse-prefix events.

  • Discussions of hyper-parameters and additional computational costs are absent, making reproducibility and efficiency difficult to assess.

Questions for authors

See the weaknesses

3 comments
AuthorsOpenReview ↗

Response to Reviewer PPUh (Part 1)

Thank you for your feedback on the experimental scope and the justification of our method.

Weaknesses:

W1: Larger teachers and smaller students.

Thank you for suggesting experiments with larger teachers and smaller students. We have extended the original Qwen3-8B→Llama-3.2-3B-Instruct setting by adding Qwen3-32B as a larger teacher and Llama-3.2-1B-Instruct as a smaller student. The left panels of Figure 5 compare SimCT and CASE across all four resulting pairings.

The table below reproduces the complete benchmark-level results (%) in Appendix A.4 (Table 9). The first four rows are undistilled teacher and student baselines. Avg. is the unweighted mean over all eight benchmarks, computed before rounding. Bold indicates the better result between SimCT and CASE within each pairing, including ties.

ModelMethodOpenR1AIME24AIME25MATHCodeforcesLCBHumanEvalMBPPAvg.
Qwen3-8BBase38.3425.8323.7581.9512.4035.1484.6066.9346.12
Qwen3-32BBase41.9930.0022.5086.1314.1932.8089.7177.2549.32
Llama-3.2-3B-InstructBase8.084.170.0039.722.623.2955.5650.0220.43
Llama-3.2-1B-InstructBase3.850.420.4222.380.963.6534.2234.9012.60
Qwen3-8B → Llama-3.2-3B-InstructSimCT11.085.422.5052.623.8812.7060.2949.9524.81
CASE11.159.172.9253.634.1812.9960.5950.5025.64
Qwen3-32B → Llama-3.2-3B-InstructSimCT9.985.002.0850.753.6111.9059.2249.8824.05
CASE10.285.423.3351.533.5412.2359.3850.1824.49
Qwen3-8B → Llama-3.2-1B-InstructSimCT5.210.420.0027.931.084.0834.6032.3313.21
CASE5.290.830.4227.531.274.1735.2132.7013.43
Qwen3-32B → Llama-3.2-1B-InstructSimCT3.930.000.0025.680.833.8934.5333.0512.74
CASE3.990.000.0025.550.993.6534.4534.1012.84

In the 32B→1B setting, overall performance remains close to the undistilled student baseline for both methods, possibly due to the large gap in parameter count between the teacher and student. Both methods score lower overall with the 32B teacher than with the 8B teacher, and 1B code averages remain below the undistilled baseline. Nevertheless, CASE achieves a higher eight-benchmark average than SimCT in all four pairings.

W2: Probabilistic justification of augmented objectives and overlapping reverse-prefix events.

Thank you for raising this concern. In Eq. (5), each candidate is a teacher-token byte string. Its student-side score is the probability that the next student token begins with the consumed teacher prefix followed by that candidate. These compatible token sets can overlap, so a token contributes once to each matching candidate. We retain these contributions and normalize the scores across candidates. Each normalized value represents the candidate's share of the summed scores, defining an auxiliary distribution for matching teacher and student support over the same labels.

We have made the reverse normalization and treatment of overlap explicit in Section 3.3.

AuthorsOpenReview ↗

Response to Reviewer PPUh (Part 2)

W3: Hyperparameters, computational costs, reproducibility, and efficiency.

Thank you for highlighting the need to clarify CASE's hyperparameter sensitivity, computational costs, and reproducibility.

We assess sensitivity to the path budget, which limits the number of complete paths searched per span during forward expansion. We compare budgets of 0, 2, 4, and 8 using Qwen3-8B→Llama-3.2-3B-Instruct with seed 42, fixing Dmax⁡=3D_{\max}=3 and all other settings. Zero disables forward expansion while retaining shared-prefix alignment and reverse decomposition, and 8 is the default. The first two rows report undistilled teacher and student baselines, for which the path budget is not applicable. Scores are percentages, and Avg. is the unweighted mean over all eight benchmarks.

ModelPath budgetOpenR1AIME24AIME25MATHCodeforcesLCBHumanEvalMBPPAvg.
Qwen3-8B—38.3425.8323.7581.9512.4035.1484.6066.9346.12
Llama-3.2-3B-Instruct—8.084.170.0039.722.623.2955.5650.0220.43
Qwen3-8B → Llama-3.2-3B-Instruct011.368.332.0852.584.3413.1359.8350.3825.25
211.194.583.3352.554.1012.5661.2850.4025.00
411.517.083.3352.504.0713.3660.3750.6325.36
811.159.172.9253.634.1812.9960.5950.5025.64

Two paths do not necessarily improve performance and yield a lower math average than zero paths, while four paths approach the performance of eight. Appendix A.7 (Table 14) shows lower boundary recall at two paths, whereas eight retain nearly all reachable boundaries under the search limits. The performance results appear in Section 4.3 and Figure 5 (right), with complete results in Appendix A.5 (Table 10).

For Qwen3-8B→Phi-4-mini-instruct, the CPU-side distribution tries for the two tokenizers have a serialized file size of 24.83 MB. On each GPU, CASE uses a lookup table to map the unconsumed bytes of a teacher token to the corresponding shared node. The table and CASE's internal mapping arrays occupy 156.52 MB and 21.58 MB, respectively, adding 178.10 MB of resident GPU data relative to SimCT.

For math training with this model pair, SimCT takes 2 hours 34 minutes and CASE takes 2 hours 56 minutes, an increase of 22 minutes (approximately 14.3%). Compared with SimCT and other methods that do not require these auxiliary structures, CASE incurs additional storage and training-time costs, which we consider acceptable in the measured setting given the performance gains.

To support reproducibility, we have included the core implementation in the supplementary materials. We will also release the complete training and evaluation code, together with model checkpoints.

AuthorsOpenReview ↗

Reviewer Discussion Follow-up

Dear Reviewer,

I hope this message finds you well.

As the discussion period is drawing to a close, we would like to ensure that we have adequately addressed all your concerns. If you have any further questions, comments, or suggestions, please feel free to let us know. We would be more than happy to provide additional clarification or address any remaining concerns.

Thank you again for your valuable feedback and for the time and effort you have devoted to reviewing our work.

Reviewer_cXMWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies on-policy distillation (OPD) between models with different tokenizers. Aligning the two tokenizations on shared byte offsets produces spans with 1:1, 1:n, m:1 and m:n token correspondences, so some prediction positions have no counterpart in the other model. CASE handles this in two stages:

  • Align: both models’ token distributions are projected onto a common set of events, namely the longest byte prefix that is a token string in both vocabularies (computed with a trie).
  • Expand: at student positions with no teacher counterpart, CASE marginalizes over alternative teacher tokenizations of the known span bytes (forward expansion). At teacher positions with no student counterpart, it compares against the student’s prefix mass from the single student position (reverse decomposition).
Critical strengths
  1. The problem is characterized clearly. The four span scenarios and the two coupled problems they create (a common event space, and supervision at unmatched positions) are laid out cleanly in Figs. 1–2.

  2. The method is principled. The trie projection keeps all probability mass rather than dropping non-shared tokens, and Prop. 1 is a clean refinement result. Forward expansion performs constrained segmentation of known bytes rather than generation. Reverse decomposition avoids conditioning the student on tokens it never emitted.

  3. The paper is honest about failure modes. The emoji drift and repetition analysis (Table 8, App. A.3) and the projection statistics (Table 4) are informative. The seed study (App. A.4) is welcome.

Critical weaknesses
  1. Table 1 reports CASE’s best of three seeds (25.64; mean 25.37 ± 0.24), while all baselines are single runs. On the seed mean, the margin over SimCT drops from 0.83 to 0.56, and the AIME24 gain drops from +3.75 to about +2.1. The other margins (0.39, 0.54) are similarly small, and on 30-problem AIME sets many differences amount to one or two samples.

  2. CASE falls below the undistilled student on Qwen3→Phi-4 OpenR1-Math (16.54 vs. 17.80) and Phi-4→Llama MBPP (48.90 vs. 50.02). Two of three pairs gain under 3 points. There is no same-tokenizer upper bound or SFT/GRPO reference to show what the tokenizer gap costs.

  3. All analyses use only Qwen3→Llama, the most favorable pair. Without expansion, CASE (23.77) trails SimCT (24.81). Forward expansion and reverse decomposition are never ablated separately. The top-k averages in Table 7 are non-monotone, contrary to the conclusion.

  4. The projection loses information too. About half the teacher’s probability mass falls into merged classes for both Phi-4 pairs (Table 4), which partly reintroduces the information loss the paper criticizes in span-level methods.

  5. There is no comparison with the byte-level methods the paper itself cites (Phan et al., 2025; BLD), even though forward expansion is closely related to them. Expansion overhead, D_max, and the rule for which teacher-tokenization paths are kept are not reported.

  6. Related work. It should cite “Knowledge Distillation for Large Language Models through Residual Learning” (ICLR 2026, openreview.net/forum?id=Dh6KxUxG20), which uses cross-model attention over low-dimensional representations for cross-tokenizer distillation and reports beating DSKD and ALM, both baselines here.

  7. Does the Avg. column mix separately trained math and code students? The 1.03 repetition penalty makes the rollouts slightly off-policy.

Questions for authors
  1. Can you report at least 3 seeds for all methods, comparing means rather than seed 42?

2, How large is the same-tokenizer upper bound, and how do SFT and GRPO perform?

  1. Can you ablate forward expansion and reverse decomposition separately, across all three pairs?

  2. What is the compute overhead of expansion relative to SimCT?

4 comments
AuthorsOpenReview ↗

Response to Reviewer cXMW (Part 1)

Thank you for the detailed comments and concrete suggestions.

Weaknesses:

W1: Best-seed reporting, small gains, and statistical reliability.

We agree that randomness can affect the interpretation of small performance differences. We have therefore substantially expanded our experiments to evaluate CASE and all six baselines across all three pairings using matched seeds 32, 42, and 52. Revised Table 1 reports mean ± sample standard deviation.

Given the difficulty of these reasoning and coding tasks, we assess improvements across eight benchmarks and report their unweighted average to provide a broader view of performance beyond individual benchmark scores.

The table below summarizes Avg. (%) for ALM, SimCT, and CASE as mean ± sample standard deviation across the three seeds.

Teacher → StudentALMSimCTCASEGain over strongest baseline (pp)
Qwen3 → Phi-427.95 ± 0.2429.77 ± 0.1330.25 ± 0.05+0.48
Qwen3 → Llama23.30 ± 0.4124.72 ± 0.1425.37 ± 0.24+0.65
Phi-4 → Llama21.48 ± 0.1021.28 ± 0.0422.03 ± 0.09+0.55

CASE achieves the highest mean Avg. in all three pairings and ranks first in Avg. in all nine pair–seed combinations, supporting consistent average gains across the evaluated seeds and pairings. We have updated the discussion in Section 4.2 and provide results for each seed in Appendix A.6 (Tables 11–13).

W2: Performance regressions and missing same-tokenizer, SFT, and GRPO references.

Thank you for raising this concern. Performance declines on individual benchmarks also occur for other baselines. We therefore evaluate both math and code on one in-domain and three out-of-domain benchmarks to assess generalization.

Cross-tokenizer distillation addresses teacher–student pairs with different tokenizers. Even if a similarly capable teacher sharing the student's tokenizer were available, replacing the teacher would change the comparison conditions. This would not isolate the effect of tokenizer mismatch or establish a controlled upper bound for the original pair.

SFT uses reference answers rather than student-generated trajectories, so it cannot preserve exactly the same training sequences as OPD. We nevertheless tested SFT using the original dataset answers and observed substantial performance declines and limited generalization.

We also added GRPO comparisons using four samples per input under two budgets. The first uses 130 rollout iterations, comparable to our OPD iteration count. The second uses 500 iterations to match OPD's total sampling volume. The new results are reported below as percentages, with Avg. computed over all eight benchmarks.

ModelSettingOpenR1AIME24AIME25MATHCodeforcesLCBHumanEvalMBPPAvg.
Phi-4-mini-instructBase17.8010.833.7539.103.0718.6572.8753.6727.47
Phi-4-mini-instructGRPO (130 iterations)18.587.085.4238.653.8719.2975.3853.4827.72
Phi-4-mini-instructGRPO (500 iterations)19.547.505.8337.804.9819.9174.6253.3527.94
Llama-3.2-3B-InstructBase8.084.170.0039.722.623.2955.5650.0220.43
Llama-3.2-3B-InstructGRPO (130 iterations)9.658.330.0040.652.5010.2857.3250.4022.39
Llama-3.2-3B-InstructGRPO (500 iterations)8.312.500.0020.102.500.6255.6450.0317.46

GRPO yields only small average gains for Phi-4-mini-instruct. For Llama-3.2-3B-Instruct, 130 iterations improve Avg., whereas 500 iterations reduce it. By comparison, CASE with Qwen3-8B achieves three-seed mean Avg. scores of 30.25% and 25.37%, respectively, exceeding both GRPO settings. Together with the SFT observations, these results support the benefit of teacher-guided training on student-generated trajectories under the evaluated conditions.

W3: Analysis across model pairs, separate component ablations, and non-monotone top-k results.

We selected Qwen3 → Llama because it has the largest teacher–student performance gap, making differences among distillation methods easier to examine. Our main experiments cover all three pairings with three matched seeds, and CASE ranks first in Avg. in all nine pair–seed combinations.

Following your suggestion, we have added two new ablation experiments that enable forward expansion and reverse decomposition separately in this setting using seed 42. All variants retain shared-prefix alignment.

VariantAvg. (%)
Shared-prefix only23.77
Forward expansion only25.05
Reverse decomposition only25.25
Full CASE25.64
AuthorsOpenReview ↗

Response to Reviewer cXMW (Part 2)

Both operators improve Avg. over shared-prefix alignment, and their combination performs best. We have updated Figure 3 and Section 4.3 accordingly.

We agree that the top-k averages are non-monotonic and have corrected the conclusion. This ablation examines sensitivity to truncating teacher supervision. We thank the reviewer for these suggestions, which helped us strengthen the experimental analysis and clarify our conclusions.

W4: Information loss from merging teacher probability mass in the projection.

We agree that merging loses distinctions among tokens within the same class. CASE uses a distribution trie to aggregate token probabilities by their longest shared prefix. This preserves total probability mass while keeping both tokenizers and model output vocabularies unchanged, providing an explicit correspondence based on token content. The merged mass in Table 4 includes all probability assigned to classes receiving multiple tokens, including shared tokens themselves, and does not quantify the amount of information lost.

Other alignment strategies also involve trade-offs. ALM reduces an aligned span to a binary distribution over the observed outcome and its complement, leaving the relative probabilities of alternative outcomes unmatched. ULD, and GOLD's unmatched-token component, align probabilities by rank, which can pair tokens with different meanings. SimCT aggregates token likelihoods within minimal aligned units, removing distinctions among their individual contributions. We view the residual loss in CASE as a practical trade-off in efficiently aligning fixed heterogeneous output spaces. We will explore ways to further reduce these losses in future work.

W5: Missing byte-level comparisons and expansion overhead, depth, and path-selection details.

Thank you for highlighting these points. We cite Phan et al. (2025) and BLD to describe the broader technical landscape. Phan et al. report offline teacher scoring, while BLD combines byte-level distillation with SFT on fixed training data. Their published setups would require adaptation for a matched comparison on student-generated on-policy trajectories.

CASE aligns token distributions through shared prefixes, using byte strings to locate span boundaries rather than constructing next-byte distributions. Figure 2 distinguishes four cases, 1:11{:}1, 1:n1{:}n, m:1m{:}1, and m:nm{:}n. For m:nm{:}n, forward expansion and reverse decomposition address the respective unmatched student and teacher positions. Our experiments compare six relevant baselines across three model pairings with matched seeds 32, 42, and 52. CASE achieves the highest mean Avg. in all three pairings and the highest Avg. in all nine pair–seed combinations, supporting its effectiveness under the evaluated conditions.

Section 3.3 now specifies Dmax⁡=3D_{\max}=3 teacher tokens and a default budget of eight complete paths per span. Search uses depth-first enumeration, trying longer byte-matching teacher tokens first and stopping after eight complete paths that reproduce the observed span. Paths covering no internal student boundary are then discarded. Retained contexts reaching the same boundary are mixed using normalized teacher path probabilities. Figure 5 (right) and Appendix A.5 (Table 10) show that four paths approach eight in performance, while two do not consistently improve performance. Separately, Appendix A.7 (Table 14) shows that eight paths retain nearly all internal student boundaries reachable under the fixed search limits.

Section 4.3 reports the overall overhead for Qwen3 → Phi-4 math training at the default budget. Training takes 2 h 56 min for CASE versus 2 h 34 min for SimCT, an increase of 22 min (14.3%). CASE adds 178.10 MB of resident auxiliary data per GPU, and the two CPU-side tries have a combined serialized file size of 24.83 MB.

W6: Missing related work on residual-learning cross-tokenizer distillation.

Thank you for highlighting this work. We have added a discussion of On et al. (2026) in Section 2. The published method combines SFT with off-policy distillation and requires additional projector training, so it cannot be applied unchanged to our on-policy setting.

We checked the paper, its official publication pages, and GitHub for released code, but found no public implementation to support a comparison under matched conditions. We thank the reviewer again for this helpful suggestion, which has improved our discussion of related work.

W7: Averaging separately trained students and off-policy effects of the repetition penalty.

We train the math and code students separately. Avg. is the unweighted mean of their scores on the four math and four code benchmarks, respectively.

The 1.03 repetition penalty is a rollout decoding adjustment used to mitigate severe repetition in some settings. Appendix A.3 provides the experimental details and the repetition analysis.

AuthorsOpenReview ↗

Response to Reviewer cXMW (Part 3)

Questions:

Q1: At least three seeds for all methods and comparisons of seed means.

We now report results from three seeds (32, 42, and 52) for CASE and all six baselines in each pairing. The table below gives the mean ± sample standard deviation of Avg. (%) across seeds, with each Avg. computed over eight benchmarks.

Teacher → StudentALMSimCTCASEGain over strongest baseline (pp)
Qwen3 → Phi-427.95 ± 0.2429.77 ± 0.1330.25 ± 0.05+0.48
Qwen3 → Llama23.30 ± 0.4124.72 ± 0.1425.37 ± 0.24+0.65
Phi-4 → Llama21.48 ± 0.1021.28 ± 0.0422.03 ± 0.09+0.55

The mean gains over the strongest baseline range from 0.48 to 0.65 percentage points across the three pairings. Table 1 and Section 4.2 now use these seed means, and Appendix A.6 (Tables 11–13) provides the individual-run results.

Q2: Same-tokenizer upper bound and SFT/GRPO performance.

A controlled same-tokenizer upper bound is unavailable for these fixed model pairings, since replacing the teacher would change the comparison conditions. SFT using the original reference answers reduced performance. The best GRPO Avg. scores are 27.94% for Phi-4 and 22.39% for Llama, compared with CASE's three-seed means of 30.25% and 25.37%, respectively. W2 provides the experimental settings and results.

Q3: Separate forward-expansion and reverse-decomposition ablations across all three pairs.

We added separate component ablations for Qwen3 → Llama. Forward expansion alone and reverse decomposition alone achieve Avg. scores of 25.05% and 25.25%, compared with 23.77% for shared-prefix alignment alone and 25.64% for full CASE. Both contribute, and their combination performs best. W3 discusses these results, now included in Figure 3 and Appendix A.3 (Table 7).

Q4: Expansion compute overhead relative to SimCT.

With the default eight-path budget, total Qwen3 → Phi-4 math-training time is 2 h 56 min for CASE versus 2 h 34 min for SimCT, an overall increase of 22 min (14.3%). Section 4.3 and W5 provide the timing and storage details for the complete CASE pipeline.

AuthorsOpenReview ↗

Reviewer Discussion Follow-up

Dear Reviewer,

I hope this message finds you well.

As the discussion period is drawing to a close, we would like to ensure that we have adequately addressed all your concerns. If you have any further questions, comments, or suggestions, please feel free to let us know. We would be more than happy to provide additional clarification or address any remaining concerns.

Thank you again for your valuable feedback and for the time and effort you have devoted to reviewing our work.

Reviewer_2CorOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CASE (Cross-Tokenizer Alignment and Supervision Expansion), a framework for on-policy distillation between language models with different tokenizers. CASE first constructs a shared predictive space using byte-prefix-based vocabulary alignment, then recovers missing intra-span supervision through teacher-context expansion and reverse decomposition. This enables finer-grained supervision than conventional span-level approaches.

The authors evaluate CASE on three teacher–student pairings across eight mathematical reasoning and code-generation benchmarks. CASE achieves the highest average performance in all three settings, supported by ablation studies on supervision expansion and sensitivity analyses of top-k truncation.

Critical strengths
  1. Well-motivated and technically interesting method.

The paper addresses a practical challenge in cross-tokenizer on-policy distillation. The combination of shared-prefix alignment and supervision expansion provides a structured approach to recovering fine-grained supervision that conventional span-level methods may discard.

  1. Consistent empirical improvements and useful ablations.

CASE achieves the highest average performance across all three teacher–student pairings and performs competitively across diverse benchmarks. The ablation study shows that supervision expansion improves performance across all eight evaluated tasks in the Qwen3-to-Llama setting, providing evidence for the effectiveness of the proposed mechanism.

Critical weaknesses
  1. Limited performance gains and statistical validation.

Although CASE achieves the best average results, its improvements over the strongest baselines are relatively small (0.39–0.83 percentage points). Multi-seed evaluation is provided only for CASE in one teacher–student setting, making the statistical reliability of these improvements difficult to assess.

  1. Insufficient analysis of supervision validity and computational overhead.

The reverse decomposition constructs normalized distributions from student prefix probabilities, but the treatment of potentially overlapping prefix events is unclear. Additionally, teacher-context expansion requires extra computation, yet its search coverage and runtime overhead are not quantitatively evaluated.

Questions for authors

Q1. Could the authors provide multi-seed comparisons against the strongest baselines, particularly SimCT and ALM, to establish whether the reported performance improvements are statistically significant?

Q2. In the reverse decomposition (Eq. 5), how are overlapping prefix events handled when multiple teacher-token strings are prefixes of the same student token? Could the authors clarify the probabilistic interpretation of the resulting normalized distributions?

Q3. Could the authors report the computational overhead and search coverage of teacher-context expansion, including the effect of the search-depth limit? A separate ablation of forward expansion and reverse decomposition would also help clarify their individual contributions.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 2Cor (Part 1)

Thank you for your careful review and constructive questions.

Weaknesses:

W1: Limited performance gains and statistical validation.

We agree that the relatively small margins require broader multi-seed validation. We have expanded the evaluation to CASE and all six baselines across all three teacher–student pairings, using three random seeds (32, 42, and 52). Revised Table 1 reports the mean ± sample standard deviation for every benchmark and Avg.

The table below summarizes the Avg. column of revised Table 1 (%) for ALM, SimCT, and CASE. For each seed, Avg. is the unweighted mean across eight benchmarks. The reported mean and sample standard deviation are then calculated across the three seeds.

Teacher → StudentALMSimCTCASEGain over strongest baseline (pp)
Qwen3 → Phi-427.95 ± 0.2429.77 ± 0.1330.25 ± 0.05+0.48
Qwen3 → Llama23.30 ± 0.4124.72 ± 0.1425.37 ± 0.24+0.65
Phi-4 → Llama21.48 ± 0.1021.28 ± 0.0422.03 ± 0.09+0.55

CASE achieves the highest mean Avg. in all three pairings. It also ranks first in Avg. among all methods for each seed in every pairing (all nine pair–seed combinations). The absolute margins over the strongest baselines remain moderate. The expanded evaluation shows that the average-score advantage is consistent across the evaluated seeds and model pairings. We have updated the main-result discussion accordingly and provide all individual-run results in Appendix A.4, Tables 9–11.

W2: Insufficient analysis of supervision validity and computational overhead.

Thank you for highlighting the need to clarify the supervision construction and its computational costs. We have clarified reverse decomposition in Section 3.3 and added computational-overhead measurements in Section 4.3.

Eq. (5) retains overlapping contributions, with each student token's probability contributing once to every compatible candidate. Normalizing the aggregated scores gives an auxiliary distribution over candidate labels, with probabilities proportional to prefix-compatibility mass.

We analyzed 1,024 saved Qwen3 → Phi-4 math-training responses per seed (32, 42, 52), using spans with verified byte consistency across both tokenizers, the planner, and the saved text. With the maximum search depth fixed at Dmax⁡=3D_{\max}=3 teacher tokens, we vary the maximum number of complete paths searched per span. The table below reports boundary recall, the percentage of internal student-token boundaries retained relative to full search under the same limits.

Path budgetSeed 32Seed 42Seed 52
110.997%10.026%12.202%
278.585%83.505%78.327%
492.957%93.347%91.906%
899.944%99.993%99.949%
≥ 9100.000%100.000%100.000%

Our experiments use a budget of eight complete paths per span, which retains nearly all boundaries found by full search under these limits. We have added this analysis to Appendix A.5 (Table 12), with a reference in Section 3.3.

In addition to search coverage, we report CASE's overall computational overhead. The timings were measured with a relatively generous path budget, for which boundary recall is already close to saturation under the current search limits. For Qwen3 → Phi-4 math training, total training time is 2 hours 34 minutes for SimCT and 2 hours 56 minutes for CASE, an increase of 22 minutes (14.3%). CASE adds 178.10 MB of resident auxiliary data per GPU, and the two CPU-side tries have a combined serialized file size of 24.83 MB.

AuthorsOpenReview ↗

Response to Reviewer 2Cor (Part 2)

Questions:

Q1: Multi-seed comparisons and statistical significance.

We have added three-seed comparisons with SimCT and ALM for all three pairings, as summarized in our response to W1. These results suggest that CASE's gains in Avg. are consistent across the evaluated seeds. Table 1 now reports mean ± sample standard deviation, with updated discussion in Sections 4.2–4.3 and individual-run results in Appendix A.4 (Tables 9–11).

Q2: Overlapping prefix events and probabilistic interpretation.

Each candidate is a teacher-token byte string supported by student tokens whose bytes begin with the consumed teacher prefix followed by that candidate. Its score is the probability assigned to this compatible set by the student's existing next-token distribution. A token contributes once to every compatible candidate, as explained in W2. Normalization turns these scores into an auxiliary distribution over candidate labels, with each probability representing the candidate's share of the summed scores.

Q3: Computational overhead, search coverage, and component ablations.

Our search uses a maximum depth of Dmax⁡=3D_{\max}=3 teacher tokens and a budget of eight complete paths per span. The analysis of responses from Qwen3 → Phi-4 math training in W2 shows that this budget retains nearly all boundaries reachable by full search on the verified byte-consistent spans. This comparison varies the path budget while keeping search depth fixed.

With this budget, CASE's total Qwen3 → Phi-4 math-training time is 22 minutes (14.3%) longer than SimCT's. Detailed timing and storage measurements are given in W2.

Following your suggestion, we have added separate ablations of forward expansion and reverse decomposition in the Qwen3 → Llama setting using seed 42.

VariantAvg. (%)Drop from full CASE (pp)
Shared-prefix only23.771.87
Forward expansion only25.050.59
Reverse decomposition only25.250.39
Full CASE25.64—

All variants retain shared-prefix alignment. Both components improve Avg. over shared-prefix alignment alone, with similar scores for the two single-component variants. Reverse decomposition alone performs slightly better overall and on the in-domain benchmarks, while forward expansion alone performs better on the out-of-domain code benchmarks. Their combination achieves the highest Avg. We have updated Figure 3 and the discussion in Section 4.3, with complete benchmark-level results in Appendix A.3 (Table 7).

AuthorsOpenReview ↗

Reviewer Discussion Follow-up

Dear Reviewer,

I hope this message finds you well.

As the discussion period is drawing to a close, we would like to ensure that we have adequately addressed all your concerns. If you have any further questions, comments, or suggestions, please feel free to let us know. We would be more than happy to provide additional clarification or address any remaining concerns.

Thank you again for your valuable feedback and for the time and effort you have devoted to reviewing our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.