acceptodds
Under review as a conference paper at ICLR 2027

SEAM: Shared-Event Alignment for Cross-Tokenizer On-Policy Distillation

Abstract

On-policy distillation across tokenizers requires predictions in different vocabularies to express a common observation. Shared token identities provide incomplete and asymmetric correspondence, while broader alignment interfaces must still determine how predictions are conditioned at token boundaries. We introduce Shared-Event Alignment via Marginalization (SEAM), which matches local next-byte and control-event distributions through the models’ native outputs. At each student boundary, compatible tokens either extend beyond the query or end there and contribute a native continuation. Marginalizing these routes on both sides preserves earlier native histories and requires no shared token inventory or auxiliary prediction head. We derive the probability decomposition, native-logit gradients, and a bound for thresholded teacher lookahead. Across three teacher–student pairs and five mathematics/programming benchmarks, SEAM achieves the highest average score among the evaluated methods, with gains of 0.47–1.09 percentage points over the strongest baselines. Distribution diagnostics reveal changes in supervision even at common boundaries. A complementary phi-4 (14B) comparison reports a higher mean for the full construction than for teacher-only covering. The results support shared events with explicit local conditioning as a useful interface for cross-tokenizer supervision.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Knowledge Distillation, On-Policy Distillation, Cross-Tokenizer Distillation, Tokenization, Large Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_TgzsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SEAM (Shared-Event Alignment via Marginalization) for cross-tokenizer model distillation and probability alignment. The authors argue that direct comparison of token-level distributions is inappropriate because the teacher and student may differ in their native token boundaries, vocabularies, and the semantics of special tokens. To address this issue, SEAM maps both models into a lower-level shared event space, such as byte, control, and stop events. At student token boundaries, the method defines local observations of the next byte or control event and marginalizes over compatible tokens that cover the corresponding boundary. This produces comparable local conditional distributions that can subsequently be used as distillation objectives.

Critical strengths

The paper introduces a clearer alignment objective for cross-tokenizer distillation by defining mutually exclusive shared next-byte and control events at student token boundaries and marginalizing over compatible tokens that span these boundaries. By separating output representations from assumptions about token boundaries, the method directly addresses the issue that existing interfaces, such as BPM and SimCT, may compare different prediction objects due to differences in token boundaries, vocabulary coverage, and length normalization. This provides a more interpretable and fairer probabilistic objective for cross-tokenizer distillation.

Critical weaknesses

The empirical evidence remains insufficient. In Table 6, the benchmark comparisons primarily use training seed 101, while Full SEAM requires substantially more GPU hours (19.33) than the symmetric/common control (12.50) and the equal-weight control (14.14). Without more comprehensive evaluations across multiple training seeds, compute-matched comparisons, and statistical significance tests, it is difficult to attribute the reported improvements of 31.70 versus 31.04/30.69 and 39.00 versus 38.50/38.38 entirely to shared-event marginalization. These gains may also result from differences in the query set, loss weighting, sampling volume, or training costs.

Questions for authors

The main question concerns whether the reported benefits of Full SEAM have been adequately controlled for differences in computational cost and sampling. Specifically, in Table 6, Full SEAM requires substantially more GPU hours (19.33) than the symmetric/common control (12.50) and the equal-weight control (14.14). Furthermore, Figure 5 indicates that the endpoints of different methods may be affected by differences in accepted-sample counts. Could the authors provide additional results under matched accepted-sample budgets or matched computational budgets, along with results from 2 to 3 independent training seeds and corresponding confidence intervals?

5 comments

Response to Reviewer Tgzs - Part 1 of 5

Thank you for identifying the need to control accepted-response counts, computation, and training randomness. We extended the Table 6 comparison to three training seeds and added direct evaluations under matched accepted-response budgets and common compute caps for seeds 202 and 303. At the matched 78-update endpoints, Full SEAM improves Avg5 over symmetric/common and equal-weight by 0.785 and 0.812 percentage points on average; both paired 95% confidence intervals are positive. It also has the highest measured Avg5 at all six tested seed/compute-budget combinations, with near-tied results at the smallest budget for seed 202. Below, we present these controls first, followed by matched-budget comparisons against the main baselines.

All benchmark scores below are reported as pass@1 × 100 (percent). Avg5 is the equally weighted mean of the scores on GSM8K, MATH-500, MBPP, LiveCodeBench-v6, and TACO. Each job in the additional experiments reported here was run on two NVIDIA H200 GPUs. GPU-hours are calculated as the recorded run time in hours multiplied by the number of GPUs; for these two-GPU jobs, GPU-hours equal twice the run time. Except for the training seed and changes required by the specific comparison, such as the divergence, mixture weight, lookahead threshold, or control variant in the ablation experiments, all other training and evaluation parameters were kept identical to those of the corresponding seed-101 configuration. Score differences are expressed in percentage points.

1. Table 6: matched budgets and three training seeds

We have now completed the Full SEAM, symmetric/common, and equal-weight comparisons on Qwen3-32B → Gemma-2-2B-it for training seeds 202 and 303. These directly extend the comparison in Table 6, which reported seed 101. As defined in Appendix F.2, symmetric/common applies covering to both models only at common boundaries; Full SEAM uses all eligible student boundaries. The equal-weight control keeps the all-student query rule but scales each response's loss sum by its common-to-all boundary-count ratio. The new configurations differ only in the variant, training seed, device assignment, and workspace; the remaining configuration is shared.

Matched updates and accepted-response budgets. All three variants finish at step 78. Using 256 accepted training responses per complete update, steps 20/40/60/78 correspond to 5,120/10,240/15,360/19,968 accepted responses. These budgets match training-response counts, not byte lengths, query counts, teacher calls, or identical sampled trajectories. The following are direct checkpoint evaluations:

Training seedUpdateAccepted responsesFull SEAMSymmetric/commonEqual-weight
202205,12030.44130.04430.087
2024010,24031.32129.94130.699
2026015,36030.80630.33630.441
2027819,96830.92430.28730.325
303205,12030.57529.83729.833
3034010,24030.98530.50130.704
3036015,36031.49530.48130.640
3037819,96831.20330.14630.377

Full SEAM has the highest Avg5 in all eight matched-update comparisons. At the endpoint, its gains over symmetric/common are +0.637/+1.057 points for seeds 202/303, and its gains over equal-weight are +0.599/+0.826 points. These improvements are measured by Avg5, the equally weighted five-benchmark average.

Endpoint compute and uncertainty.

Training seedFull SEAM GPU-hoursSymmetric/common GPU-hoursEqual-weight GPU-hours
20214.408.3814.14
30314.228.6714.11

Full SEAM costs 1.72×/1.64× as much as symmetric/common, but only 1.02×/1.01× as much as equal-weight at these endpoints. Thus, the gain over equal-weight is observed under nearly equal total compute as well as equal update counts. The lower cost of symmetric/common motivates the fixed-compute comparison below.

ControlSeed-101 gainSeed-202 gainSeed-303 gainMean paired gain95% CITwo-sided p
Symmetric/common+0.660+0.637+1.057+0.785[0.198, 1.371]0.0289
Equal-weight+1.010+0.599+0.826+0.812[0.300, 1.323]0.0208

These endpoint comparisons pair training seeds 101, 202, and 303. The seed-101 Avg5 values are 31.70/31.04/30.69 for Full SEAM/symmetric-common/equal-weight, as reported in the original Table 6; all three original variants also completed 78 updates. The mean gains are +0.785 and +0.812 points, respectively. Both marginal 95% confidence intervals are positive, with two-sided paired t-test p-values of 0.0289 and 0.0208, respectively. The p-values are unadjusted, and the confidence intervals are marginal. With only three training seeds, normality cannot be reliably assessed, and the statistics are approximate to the precision of the reported scores.

Response to Reviewer Tgzs - Part 2 of 5

The original seed-101 SEAM run shared GPU resources with other processes, so its recorded execution time was affected by contention. The additional seed-202/303 experiments had exclusive use of their allocated two H200 GPUs. We combine benchmark endpoint scores across seeds for the training-run statistics, while the new budget comparisons use the seed-202/303 cost measurements. The original Table 6 costs were 19.33/12.50/14.14 GPU-hours for Full SEAM/symmetric-common/equal-weight. Comparisons between the original and new cost figures therefore also reflect their execution conditions. The new experiments do not re-evaluate the original separate 800-example validation panel.

Matched computational budgets. We additionally report direct downstream measurements from the budget-matched experiments for training seeds 202 and 303, at 4.30 and 6.50 GPU-hours and at each seed's symmetric/common endpoint budget. The scores below are direct measurements from the budget-matched evaluations. Each variant uses its largest completed update count within the budget, shown in parentheses. Bold marks the highest Avg5 in each row.

Training seedGPU-hour budgetFull SEAM Avg5 (update)Symmetric/common Avg5 (update)Equal-weight Avg5 (update)Gain over commonGain over equal-weight
2024.3030.441 (20)30.435 (38)30.407 (23)+0.006+0.034
2026.5031.030 (32)30.336 (60)30.782 (33)+0.694+0.248
2028.3831.298 (44)30.287 (78)30.948 (46)+1.011+0.350
3034.3030.575 (20)30.161 (38)30.114 (21)+0.414+0.461
3036.5031.812 (33)30.556 (58)30.795 (35)+1.256+1.017
3038.6731.426 (44)30.146 (78)31.017 (45)+1.280+0.409

Full SEAM is above both controls at all six displayed seed/budget combinations. The gains over symmetric/common range from +0.006 to +1.280 points, and those over equal-weight range from +0.034 to +1.017 points. At seed 202 and 4.30 GPU-hours, the margins are only +0.006 and +0.034 points, respectively; this budget therefore shows near-tied performance. Budget points from the same training run are correlated, so they are not counted as additional independent training seeds.

Together, the matched-response and matched-compute measurements strengthen the evidence for the full configuration relative to the two controls. The equal-weight comparison directly probes the weighting rule while retaining the same all-student query rule. The symmetric/common comparison tests the expanded query coverage as a package, with query counts and teacher computation intentionally different. These results reduce the concern that gains arise solely from more accepted responses or a larger overall compute budget. The endpoint comparison spans three training seeds; the new budget controls cover two seeds on the Gemma pair. The results support the full configuration without uniquely isolating every component of shared-event marginalization.

2. Additional accepted-sample controls against the main baselines

We checked the training records to match the accepted responses actually used for optimization. For the five main methods, each complete update uses 256 accepted training responses. The evaluated intermediate checkpoints therefore provide matched accepted-sample budgets:

CheckpointAccepted responses used for training
Step 205,120
Step 4010,240
Step 6015,360

This holds for all five methods in the seed-202 curves across the three model pairs. Comparing the same steps therefore matches both the number of completed updates and the accepted-response count. The 29 completed main runs also consume the same 20,000 prompt-occurrence budget, while accepted-response counts can differ at their endpoints. Equal accepted counts do not imply identical trajectories, since each method samples from its own student policy.

The measured Avg5 scores at these matched accepted-sample budgets are shown below. All results use training seed 202; scores are pass@1 × 100, averaged equally over the five benchmarks. These are direct checkpoint evaluations, without interpolation. Bold indicates the highest score in each row.

Response to Reviewer Tgzs - Part 3 of 5

Model pairAccepted training responsesSEAMBPMSimCTGOLDDSKD
Qwen3-32B → Gemma-2-2B-it5,12030.44130.76029.90330.28727.036
Qwen3-32B → Gemma-2-2B-it10,24031.32130.30328.96019.41628.728
Qwen3-32B → Gemma-2-2B-it15,36030.80630.09528.48621.92228.706
phi-4 → Qwen3.5-2B5,12044.55344.55240.89744.07639.933
phi-4 → Qwen3.5-2B10,24045.07344.51044.74644.29343.574
phi-4 → Qwen3.5-2B15,36045.20444.42845.26344.42644.441
Qwen3-32B → Qwen3.5-2B5,12045.66245.26942.38545.21042.296
Qwen3-32B → Qwen3.5-2B10,24046.60046.00644.71646.02245.024
Qwen3-32B → Qwen3.5-2B15,36046.25745.83345.48045.95144.958

Under matched accepted-sample budgets, SEAM has the highest displayed Avg5 in seven of the nine evaluated settings and at every evaluated budget on the Qwen pair. On Gemma and Phi, its relative performance varies with the sample budget; in particular, the lead over BPM at the smallest Phi budget is only 0.001 points. These rankings describe the measured scores and are not tests of statistical significance.

3. Additional compute-budget controls against the main baselines

For training seed 202, we evaluated actual checkpoints under a common GPU-hour budget for each model pair: 8.32 GPU-hours for Gemma, 6.40 GPU-hours for Phi, and 9.87 GPU-hours for Qwen. Each budget is the smallest endpoint cost among the five methods. Each method uses the largest completed update count within that budget. The scores below are direct downstream measurements from those checkpoints. The common budget is a compute cap; discrete updates need not use exactly identical elapsed GPU-hours.

Model pairGPU-hour budgetSEAMBPMSimCTGOLDDSKD
Qwen3-32B → Gemma-2-2B-it8.3231.23930.36229.04420.21228.859
phi-4 → Qwen3.5-2B6.4045.10244.53245.03344.57943.971
Qwen3-32B → Qwen3.5-2B9.8746.64645.93745.28346.20845.197

For Gemma, the evaluated updates are 42/61/48/43/77 for SEAM/BPM/SimCT/GOLD/DSKD, respectively; for Phi they are 33/72/45/48/78, and for Qwen they are 49/74/52/51/77. At these budgets, SEAM has the highest Avg5 in all three model pairs. Its gains over the strongest baseline at each budget are +0.877 over BPM on Gemma, +0.069 over SimCT on Phi, and +0.438 over GOLD on Qwen. The Phi margin over SimCT is small (0.069 points). These direct measurements show gains under the same compute cap in the tested settings, while the preferred quality–cost trade-off can still depend on the model pair and budget. These main-baseline budget comparisons use training seed 202; the Table 6 controls in Section 1 cover seeds 202 and 303.

4. Training-seed uncertainty against the main baselines

We added independent training seeds 202 and 303 for SEAM and the main baselines across the three model pairs. Complete downstream results are available for 29 of the 30 planned additional runs. We did not pursue further Gemma–GOLD training with seed 303 after observing substantial performance degradation with seed 202. Together with the original training seed 101, these results provide three independent OPD training runs for all other configurations.

The table reports Avg5 mean ± sample standard deviation across training seeds 101, 202, and 303. These standard deviations describe training-run variability, whereas the task-level standard deviations in the experiment tables describe decoding seeds 42–46. We use the original paper's reported seed-101 endpoint averages and the updated seed-202/303 endpoint averages; derived statistics are approximate to the precision of the reported inputs.

Model pairBPMSimCTGOLDDSKDSEAMSEAM − strongest baseline
Qwen3-32B → Gemma-2-2B-it30.375 ± 0.22729.112 ± 1.038—28.798 ± 0.18031.276 ± 0.393+0.900
phi-4 → Qwen3.5-2B44.528 ± 0.35344.999 ± 0.15744.576 ± 0.34843.744 ± 0.20645.677 ± 0.223+0.677
Qwen3-32B → Qwen3.5-2B46.265 ± 0.55445.657 ± 0.42146.345 ± 0.21745.229 ± 0.07147.151 ± 0.083+0.806

The strongest baselines by three-seed mean Avg5 are BPM for Gemma, SimCT for Phi, and GOLD for Qwen. Gemma–GOLD is excluded from the three-seed aggregate because seed 303 is unavailable; its available scores are 30.20 and 22.021 for seeds 101 and 202. SEAM has the highest observed Avg5 among available methods in all nine model-pair/training-seed settings.

Reviewer_ecUaOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses the challenge of On-Policy Distillation (OPD) when the teacher and student models use different tokenizers. To avoid lossy approximations such as vocabulary truncation or naive sequence alignment, the authors propose SEAM (Shared-Event Alignment). SEAM functions by mapping output sequences to a shared byte-level event representation, dynamically calculating exact target probability distributions across mismatched vocabularies using a dual-route (extending and ending) lookup and lookahead approximation.

Contributions Shared-Event Probability Mapping: The formulation of a mathematically precise, byte-level event alignment framework that correctly accounts for vocabulary boundaries during distillation.

Validation The authors evaluate SEAM across three reasonable model pairs (e.g., Qwen3-32B → Qwen3.5-2B, phi-4 → Qwen3.5-2B, Qwen3-32B → Gemma-2-2B-it) on 5 standard downstream tasks.

Critical strengths

Direct and Mathematically Rigorous Alignment: The idea of using a shared byte-level event space to precisely map predictive distributions across different vocabularies is mathematically sound and avoids the arbitrary loss of vocabulary information.

Highly Commendable Transparency and Honesty: Unlike many papers that obscure computational overhead, the authors are highly transparent. They explicitly present and discuss their computational costs in Section 4.5 and Appendix G, showing a rare and commendable level of academic honesty regarding the trade-offs of their method.

Critical weaknesses
  1. Poor Cost-Effectiveness (Diminishing Returns of Exact Matching) As transparently reported by the authors in Section 4.5 and Table 9, SEAM requires 1.8x to 3.28x the training cost (GPU-hours) of the simpler Byte-Prefix Marginalization (BPM) baseline. However, this substantial computational overhead yields a marginal downstream improvement of only 0.47% to 1.09%. While the exact probability calculation is elegant, spending triple the compute to align with the teacher's distribution—which itself is inherently a noisy, imperfect approximation—exhibits severe diminishing returns for practical deployments.

  2. Weak Coupling Between the Algorithm and the "OPD" Title Although "On-Policy Distillation" is prominently featured in the paper’s title and framing, the core contribution—the byte-level shared-event mapping—is actually a general cross-tokenizer alignment algorithm. It is not fundamentally restricted to, or designed specifically for, the on-policy setting and could be equally applied to offline distillation. Framing the entire paper around OPD adds unnecessary conceptual and engineering complexity without a clear methodological justification as to why this alignment must be coupled with online generation.

Questions for authors

How the accuracy of estimator affect the performance after distillation? How to strike the balance between coarse estimator like BPM and your sophisticated probability decomposition?

Not weakness. My personal point of view: From a methodological standpoint, the teacher's logit distribution is inherently imperfect and contains its own biases, noise, and approximation errors—especially when projected across different tokenizer spaces. SEAM introduces highly complex dual-route (extending and ending) calculations and lookahead approximations to "perfectly" reconstruct these probability targets. Spending significant compute to precisely align with an inherently noisy and imperfect target constitutes a form of over-engineering. The empirical results suggest that striving for mathematical "exactness" in cross-tokenizer probability matching yields marginal benefits, as the student simply overfits to the teacher's idiosyncratic distribution.

6 comments

Response to Reviewer ecUa - Part 1 of 4

Thank you for emphasizing cost-effectiveness and the distinction between a general alignment interface and its use in OPD. Our new matched-compute experiments show downstream gains over BPM in all three tested model pairs. We also examine the pruning trade-off and teacher-error agreement, and clarify the OPD framing below.

Scores are pass@1 × 100; Avg5 equally averages GSM8K, MATH-500, MBPP, LiveCodeBench-v6, and TACO. Score differences are percentage points. New runs used two dedicated H200 GPUs; GPU-hours equal twice the recorded runtime in hours. Except for the training seed and the stated experimental variant, training and evaluation settings match the corresponding seed-101 configuration.

1. Does the additional computation yield practical benefit?

The original seed-101 SEAM runs shared GPUs with other processes, so their recorded GPU-hours included resource-contention overhead. The additional seed-202/303 runs used dedicated GPUs. Under the new conditions, seed-202 endpoint costs are:

Model pairSEAM GPU-hoursBPM GPU-hoursSEAM / BPM
Qwen3-32B → Gemma-2-2B-it14.4010.341.39×
phi-4 → Qwen3.5-2B13.176.851.92×
Qwen3-32B → Qwen3.5-2B14.0510.151.38×

The 1.38–1.92× overhead remains material. Its reduction from the original 1.8–3.28× reflects different execution conditions, not an algorithmic speedup. We therefore directly evaluated quality under common compute caps.

For each model pair, we used the smallest endpoint cost among the five methods as the cap and evaluated each method's largest completed update within it. These are direct checkpoint evaluations, without interpolation; discrete updates need not consume identical elapsed GPU-hours. All use training seed 202.

Model pairGPU-hour budgetSEAMBPMSimCTGOLDDSKD
Qwen3-32B → Gemma-2-2B-it8.3231.23930.36229.04420.21228.859
phi-4 → Qwen3.5-2B6.4045.10244.53245.03344.57943.971
Qwen3-32B → Qwen3.5-2B9.8746.64645.93745.28346.20845.197

In SEAM/BPM/SimCT/GOLD/DSKD order, the evaluated updates are 42/61/48/43/77 for Gemma, 33/72/45/48/78 for Phi, and 49/74/52/51/77 for Qwen. SEAM exceeds BPM by +0.877/+0.570/+0.709 points, respectively. It has the highest measured Avg5 at all three caps, although its lead over SimCT on Phi is only +0.069. Thus, the observed gains do not depend solely on more expensive endpoints; these single-seed measurements do not establish superiority at every budget.

Endpoint gains also persist across training seeds 101/202/303. Entries below are mean ± sample standard deviation across training runs; each baseline is the strongest by three-seed mean among methods with complete results.

Model pairSEAM Avg5BaselineBaseline Avg5Mean gain
Qwen3-32B → Gemma-2-2B-it31.276 ± 0.393BPM30.375 ± 0.227+0.900
phi-4 → Qwen3.5-2B45.677 ± 0.223SimCT44.999 ± 0.157+0.677
Qwen3-32B → Qwen3.5-2B47.151 ± 0.083GOLD46.345 ± 0.217+0.806

The paired t-based 95% intervals for these gains are [0.175, 1.626], [0.124, 1.230], and [0.415, 1.197], respectively. They are marginal, unadjusted intervals based on only three training seeds. Statistics use reported endpoint scores, so rounding can affect the last digit. These are three-seed quality estimates; the cost ratios above are seed-202 measurements.

2. Estimator accuracy and the quality–cost trade-off

SEAM makes the prediction object comparable across tokenizers; accurately matching that object is not itself proof of useful transfer. Our practical criterion is downstream quality at a given cost.

On Gemma with seed 202, the default threshold τ=0.001 gives Avg5 30.924, versus 30.129 without threshold pruning (τ=0), a +0.795 difference. Both complete 78 updates with JSD, β=0.5, and smoothing ε=0. These results favor the pruned default in this setting, using downstream quality and measured runtime to guide the trade-off. They do not establish a general relationship between estimator error and downstream performance.

On 255 fixed responses containing 105,688 student boundaries, pruning reduces lookahead expansions from 11,482 to 2,580 (77.5%). Median diagnostic time falls from 41.99s to 40.68s (3.1%), using rounds 2–5 and excluding first-round allocation overhead. These are diagnostic savings, not measured end-to-end training speedups.

Coverage and weighting controls. On Gemma, symmetric/common supervises common boundaries with covering applied to both models; Full SEAM uses all eligible student boundaries. Equal-weight retains Full SEAM's query rule but scales each response's loss sum by its common-to-all boundary-count ratio. The following direct evaluations use each variant's largest completed update within the cap, shown in parentheses.

Response to Reviewer ecUa - Part 2 of 4

Training seedGPU-hour budgetFull SEAM Avg5 (update)Symmetric/common Avg5 (update)Equal-weight Avg5 (update)Gain over commonGain over equal-weight
2024.3030.441 (20)30.435 (38)30.407 (23)+0.006+0.034
2026.5031.030 (32)30.336 (60)30.782 (33)+0.694+0.248
2028.3831.298 (44)30.287 (78)30.948 (46)+1.011+0.350
3034.3030.575 (20)30.161 (38)30.114 (21)+0.414+0.461
3036.5031.812 (33)30.556 (58)30.795 (35)+1.256+1.017
3038.6731.426 (44)30.146 (78)31.017 (45)+1.280+0.409

Full SEAM leads in all six settings, with near-ties at seed 202 and 4.30 GPU-hours. Budget points within a run are correlated. At the shared 78-update endpoint:

MethodSeed 202 Avg5Seed 303 Avg5Two-seed mean Avg5GPU-hours (202 / 303)
Full SEAM30.92431.20331.063514.40 / 14.22
Symmetric/common30.28730.14630.21658.38 / 8.67
Equal-weight30.32530.37730.351014.14 / 14.11

Full SEAM's two-seed mean gains are +0.847 over symmetric/common and +0.7125 over equal-weight. Its cost is nearly equal to equal-weight. These controls support the coverage and weighting design; they do not isolate estimator accuracy.

Matched accepted-response budgets. With seed 202, every complete update uses 256 accepted training responses. Updates 20/40/60 therefore match 5,120/10,240/15,360 responses across all five methods. Each method still generates its own trajectories.

Model pairAccepted training responsesSEAMBPMSimCTGOLDDSKD
Qwen3-32B → Gemma-2-2B-it5,12030.44130.76029.90330.28727.036
Qwen3-32B → Gemma-2-2B-it10,24031.32130.30328.96019.41628.728
Qwen3-32B → Gemma-2-2B-it15,36030.80630.09528.48621.92228.706
phi-4 → Qwen3.5-2B5,12044.55344.55240.89744.07639.933
phi-4 → Qwen3.5-2B10,24045.07344.51044.74644.29343.574
phi-4 → Qwen3.5-2B15,36045.20444.42845.26344.42644.441
Qwen3-32B → Qwen3.5-2B5,12045.66245.26942.38545.21042.296
Qwen3-32B → Qwen3.5-2B10,24046.60046.00644.71646.02245.024
Qwen3-32B → Qwen3.5-2B15,36046.25745.83345.48045.95144.958

SEAM has the highest measured Avg5 in seven of nine settings; its smallest Phi lead is only 0.001 points. Together, the compute and sample controls support a useful trade-off in the tested settings, without identifying a universally optimal threshold.

3. Why the paper focuses on OPD

We agree that the alignment interface is general and can also be used offline. We study OPD because teacher targets are constructed at the current student's evolving, self-generated prefixes, where tokenizer mismatch must be handled during training. OPD identifies our evaluated training setting, not a requirement of the interface. We will revise the framing accordingly; the experiments do not establish superiority over offline distillation.

4. Teacher errors: a limitation shared with other distillation methods

We agree that distillation can transmit teacher errors, and SEAM does not eliminate this limitation. In an additional GSM8K/MATH-500 diagnostic, we measured agreement with a teacher's incorrect scalar answer, conditional on that teacher giving an incorrect scalar answer. Teacher and student predictions were paired by problem and decoding seed (42–46). SEAM's GSM8K agreement ranges from 20.81% to 37.67% across the three model pairs and training seeds 202/303; the evaluated baselines also exhibit wrong-answer agreement. Agreement is partial within this diagnostic, but that does not rule out other forms of teacher-error inheritance.

We also compared the actual SFT starting checkpoints with OPD endpoints. Define D as agreement with one's own teacher minus agreement with the alternative teacher (Qwen3-32B or phi-4). The table reports changes in D, in percentage points, from freshly evaluated SFT predictions, averaged over two training seeds; each agreement rate averages five decoding seeds. Positive SEAM−BPM values mean a larger increase under SEAM. Values derive from rounded diagnostic inputs.

BenchmarkModel pairSEAM change from SFTBPM change from SFTSEAM − BPM
GSM8KQwen3-32B → Gemma-2-2B-it+2.755+3.050−0.295
GSM8Kphi-4 → Qwen3.5-2B+0.745−0.365+1.110
GSM8KQwen3-32B → Qwen3.5-2B+4.435+4.370+0.065
MATH-500Qwen3-32B → Gemma-2-2B-it0.000−2.395+2.395
MATH-500phi-4 → Qwen3.5-2B−1.245−0.310−0.935
MATH-500Qwen3-32B → Qwen3.5-2B−0.685−2.865+2.180

Response to Reviewer ecUa - Part 3 of 4

The SEAM–BPM differences vary in sign across settings; we do not claim equivalence or consistently lower error inheritance under SEAM. Because the teachers fail on different problems, D and its change are descriptive rather than causal measures. The diagnostic covers 65–68% of teacher errors on GSM8K and 42–54% on MATH-500, with approximately 59–75 and 35–50 eligible problems per decoding seed, respectively; it does not assess reasoning-path imitation or other teacher-specific behavior.

The Gemma seed-202 threshold diagnostic gives:

Lookahead thresholdAgreement with own teacher's wrong scalar answers (%)Agreement contrast D (points)Avg5
Default, τ=0.00121.96+3.1630.924
No threshold pruning, τ=023.20+6.5630.129

Removing pruning increases measured error agreement while lowering Avg5 in this run. This does not identify a causal mechanism, but reinforces that more exhaustive computation need not improve transfer. Despite the acknowledged risk of inheriting errors, SEAM achieves downstream gains under the matched compute caps above; we evaluate its practical value through that quality–cost balance.

Supplementary results: per-benchmark scores for training seeds 202 and 303

For completeness, we report the endpoint results for each of the five main methods separately for the two additional training seeds. Each benchmark entry is pass@1 × 100, reported as the mean ± sample standard deviation over five decoding seeds (42–46) for that trained checkpoint. These standard deviations describe decoding variability, not variability across training runs. Avg5 is the equally weighted mean of the five benchmark means; no separate standard deviation is assigned to Avg5 in these tables. The update column identifies each run’s reported endpoint. Endpoints can differ in update count, so these tables describe endpoint performance; the matched-budget analyses are reported separately in this response where applicable. Following the substantial degradation observed for Gemma–GOLD with seed 202, we did not pursue further training with seed 303; its scores are therefore marked unavailable. Bold marks the highest reported mean in each benchmark column and in Avg5 within the same model pair and training seed; all ties are highlighted.

Training seed 202: Qwen3-32B → Qwen3.5-2B

MethodUpdateGSM8KMATH-500MBPPLiveCodeBench-v6TACOAvg5
GOLD7781.971 ± 0.39570.960 ± 0.86553.852 ± 2.81416.284 ± 0.3247.800 ± 1.39646.174
DSKD7779.939 ± 0.48768.320 ± 0.96555.331 ± 1.04416.095 ± 0.3246.300 ± 0.44745.197
SimCT7580.698 ± 0.97268.560 ± 1.52655.175 ± 1.89416.114 ± 1.2105.700 ± 1.25545.249
BPM7781.759 ± 1.07771.040 ± 1.35253.930 ± 1.99515.735 ± 0.7255.900 ± 1.29445.673
SEAM7782.229 ± 0.33772.760 ± 1.16155.409 ± 2.08816.796 ± 0.5348.500 ± 0.79147.139

Training seed 202: phi-4 → Qwen3.5-2B

MethodUpdateGSM8KMATH-500MBPPLiveCodeBench-v6TACOAvg5
GOLD7880.955 ± 0.84867.760 ± 0.99452.607 ± 1.68314.787 ± 1.0196.800 ± 0.90844.582
DSKD7879.545 ± 0.73267.000 ± 1.76150.195 ± 1.32015.716 ± 0.4577.400 ± 1.51743.971
SimCT7879.788 ± 0.69065.880 ± 1.02655.409 ± 2.53116.891 ± 1.2466.600 ± 0.96244.914
BPM7880.470 ± 0.82866.600 ± 1.28153.385 ± 1.74914.901 ± 0.4616.000 ± 0.70744.271
SEAM7881.471 ± 0.60868.480 ± 0.48254.008 ± 0.92916.569 ± 0.3998.700 ± 0.90845.846

Training seed 202: Qwen3-32B → Gemma-2-2B-it

MethodUpdateGSM8KMATH-500MBPPLiveCodeBench-v6TACOAvg5
GOLD6538.165 ± 1.47615.120 ± 1.24649.650 ± 0.8074.171 ± 0.2603.000 ± 0.50022.021
DSKD7756.042 ± 1.04127.960 ± 1.27647.393 ± 1.5177.602 ± 0.4525.300 ± 0.57028.859
SimCT7263.457 ± 1.04126.360 ± 1.51346.693 ± 1.3203.488 ± 0.2452.700 ± 0.44728.540
BPM7563.760 ± 0.79928.160 ± 1.55248.794 ± 1.3087.185 ± 0.4763.900 ± 0.54830.360
SEAM7862.790 ± 0.45629.000 ± 1.90849.961 ± 1.3377.867 ± 0.3215.000 ± 0.61230.924
Reviewer_HprjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents SEAM, which defines a shared next-byte or control event at each student token boundary and computes each model's event distribution through its own vocabulary. The teacher uses a thresholded lookahead at boundaries inside its tokens, and the two distributions are matched with JSD. Across three teacher–student pairs and five math and code benchmarks, SEAM has the best average, by 0.47–1.09 points over the strongest baselines.

Critical strengths
  1. This paper solves a timely problem as most OPD works are constrained by the requirement of having the same tokenizer

  2. The proposed algorithm is well-motivated and formulated

  3. The computational cost analysis is quite helpful

Critical weaknesses
  1. The proposed method only predicts the next byte and it may not distinguish the choice of the next word of the teacher. For example, the student writes “_cat” where the teacher prefers “_dog”, both fall into the same space event at the preceding query. It would be great to add more explanation on this and also provide the frequency of such tokens (e.g., beginning with space) in the OPD training.

  2. Additionally, teacher and student can be matched on different conditionals under SEAM. For example, “running” tokenized as [run][ning] by the student and [runn][ing] by the teacher. Appendix A acknowledges this approximation, but it would be great to measure how large this mismatch is or how often it arises during training.

  3. All results come from a single training seed and the margins are relatively small. SEAM’s margins over the strongest baselines (0.47–1.09 points) are within possible run-to-run variation, and under phi-4 it leads only on TACO (8.10 against 6.80). It would be great to add multi-seed results to demonstrate the advantage of SEAM.

  4. The ablations seems not enough. For example, the ablations leave key design choices unvaried: the divergence (JSD with a fixed mixture weight, against FKL, RKL or other weights). How do the results change with the divergence and the JSD mixture weight?

  5. Key baselines and reference points are missing. For example, ALM and ULD are discussed but not run. Also, without a same-tokenizer upper bound, it is unclear how much of the gain over SFT comes from the teacher.

Questions for authors

Please refer to weakness

5 comments

Response to Reviewer Hprj - Part 1 of 5

Thank you for identifying the event-resolution and conditioning limitations and the need for stronger empirical validation. We added two training seeds across the three model pairs: SEAM's three-seed mean Avg5 exceeds the strongest baseline with complete three-seed results by 0.68–0.90 percentage points. We also quantified space-prefixed tokens and nonshared boundaries, tested alternative divergences and JSD mixture weights, and trained and evaluated ALM and ULD. We address each concern in detail below.

All benchmark scores below are reported as pass@1 × 100 (percent). Avg5 is the equally weighted mean of the scores on GSM8K, MATH-500, MBPP, LiveCodeBench-v6, and TACO. Each job in the additional experiments reported here was run on two NVIDIA H200 GPUs. GPU-hours are calculated as the recorded run time in hours multiplied by the number of GPUs; for these two-GPU jobs, GPU-hours equal twice the run time. Except for the training seed and changes required by the specific comparison, such as the divergence, mixture weight, lookahead threshold, or control variant in the ablation experiments, all other training and evaluation parameters were kept identical to those of the corresponding seed-101 configuration. Score differences are expressed in percentage points.

1. Resolution of next-byte events and space-prefixed tokens

We agree with the reviewer's example: tokens such as cat and dog map to the same space event at that boundary. SEAM aligns the distribution of the next shared event; it does not recover the complete token- or word-level preference within each event. We will clarify this distinction in the paper.

We measured the frequency directly on training rollouts, using 1,024 responses per model pair and verifying that the native teacher and student token sequences reconstruct identical response bytes:

Model pairStudent boundaries whose next token starts with a spaceBoundaries examined
Qwen3-32B → Gemma-2-2B-it45.75%434,818
phi-4 → Qwen3.5-2B55.44%416,456
Qwen3-32B → Qwen3.5-2B49.59%453,648

Space-prefixed tokens account for approximately 46–55% of the examined student boundaries, so the limitation in the reviewer’s example is relevant in practice. We will make this limitation explicit in the manuscript.

A one-byte target still supervises the relative probabilities of different byte and control events, although it cannot distinguish alternatives within the same event at that boundary. The student also retains its pretrained and SFT capabilities and receives supervision at other boundaries. The observed downstream gains are consistent with useful transfer despite this local information loss. They do not establish that one-byte events are optimal, or determine how much performance could be gained from finer event resolution.

2. Frequency and extent of conditional mismatch

The reviewer’s example highlights a distinction between boundary mismatch and the resulting error in the conditional distribution. To quantify how often the boundary issue arises, we measured nonshared student boundaries on the same training samples, together with their distance to the nearest teacher boundary:

Model pairNonshared student boundariesDistance 1 byteDistance 2 bytesDistance ≥3 bytes
Qwen3-32B → Gemma-2-2B-it11.21%90.50%9.13%0.37%
phi-4 → Qwen3.5-2B8.60%96.45%3.49%0.06%
Qwen3-32B → Qwen3.5-2B6.05%83.07%16.88%0.05%

The distance percentages are conditional on a boundary being nonshared. Nonshared boundaries account for approximately 6–11% of student boundaries, with most lying one byte from the nearest teacher boundary. Training logs additionally record an average discarded probability mass of approximately 2.1 × 10⁻⁶ to 4.3 × 10⁻⁶. These measurements quantify boundary disagreement and threshold pruning, respectively; they do not directly measure conditional-distribution error.

3. Additional training seeds and benchmark-level results

We added independent training seeds 202 and 303 for SEAM and the main baselines across the three model pairs. Complete downstream results are available for 29 of the 30 planned additional runs. We did not pursue further Gemma–GOLD training with seed 303 after observing substantial performance degradation with seed 202. Together with the original training seed 101, these results provide three independent OPD training runs for all other configurations.

Response to Reviewer Hprj - Part 2 of 5

The table reports Avg5 mean ± sample standard deviation across training seeds 101, 202, and 303. These standard deviations describe training-run variability, whereas the task-level standard deviations in the experiment tables describe decoding seeds 42–46. We use the original paper's reported seed-101 endpoint averages and the updated seed-202/303 endpoint averages; derived statistics are approximate to the precision of the reported inputs.

Model pairBPMSimCTGOLDDSKDSEAMSEAM − strongest baseline
Qwen3-32B → Gemma-2-2B-it30.375 ± 0.22729.112 ± 1.038—28.798 ± 0.18031.276 ± 0.393+0.900
phi-4 → Qwen3.5-2B44.528 ± 0.35344.999 ± 0.15744.576 ± 0.34843.744 ± 0.20645.677 ± 0.223+0.677
Qwen3-32B → Qwen3.5-2B46.265 ± 0.55445.657 ± 0.42146.345 ± 0.21745.229 ± 0.07147.151 ± 0.083+0.806

The strongest baselines by three-seed mean Avg5 are BPM for Gemma, SimCT for Phi, and GOLD for Qwen. Gemma–GOLD is excluded from the three-seed aggregate because seed 303 is unavailable; its available scores are 30.20 and 22.021 for seeds 101 and 202. SEAM has the highest observed Avg5 among available methods in all nine model-pair/training-seed settings.

We compute paired differences by training seed and report unadjusted two-sided paired t-test p-values and marginal t-based 95% confidence intervals. Selecting the strongest baseline by the observed three-seed mean is descriptive; we additionally retain Qwen–BPM, the strongest baseline in the original main table.

Model pairBaselineMean Avg5 difference95% CITwo-sided p
Qwen3-32B → Gemma-2-2B-itBPM+0.900[0.175, 1.626]0.0333
phi-4 → Qwen3.5-2BSimCT+0.677[0.124, 1.230]0.0342
Qwen3-32B → Qwen3.5-2BGOLD+0.806[0.415, 1.197]0.0125
Qwen3-32B → Qwen3.5-2BBPM+0.887[-0.399, 2.172]0.0973

Gemma–BPM, Phi–SimCT, and Qwen–GOLD have positive marginal 95% intervals and unadjusted p<0.05; Qwen–BPM does not. These results support consistent observed gains across training seeds, without establishing significant superiority over every baseline. With three training seeds, the approximate normality assumption for paired differences cannot be reliably checked. Decoding seeds, tasks, and checkpoints are not treated as independent training replicates.

The reviewer is correct that, in the original Phi table, SEAM leads the compared methods only on TACO. In the additional runs, the improvements over SimCT extend to GSM8K (+1.683/+1.668) and MATH-500 (+2.600/+2.160) for training seeds 202/303. MBPP and LiveCodeBench remain below SimCT. Thus, the new results support gains beyond TACO while also showing that the advantage is task-dependent.

4. Divergence and JSD mixture-weight ablations

Following the reviewer’s suggestion, we compared JSD with FKL and RKL and varied the JSD mixture weight on the Gemma pair with training seed 202. We describe the weighting and smoothing conventions below so that the comparisons can be interpreted consistently.

Let pTp_T and pSp_S denote teacher and student distributions over the same shared events. We define mβ=βpT+(1−β)pSm_{\beta}=\beta p_T+(1-\beta)p_S and minimize

JSDβ(pT,pS)=β,KL(pT∥mβ)+(1−β),KL(pS∥mβ).\mathrm{JSD}_{\beta}(p_T,p_S)=\beta\\,\mathrm{KL}(p_T\Vert m_{\beta})+(1-\beta)\\,\mathrm{KL}(p_S\Vert m_{\beta}).

Here β\beta weights the teacher in both the mixture and KL sum. FKL is KL(pT∥pS)\mathrm{KL}(p_T\Vert p_S) and RKL is KL(pS∥pT)\mathrm{KL}(p_S\Vert p_T); neither uses β\beta (marked “—”). Smoothing replaces each distribution pp with (1−ε)p+εu(1-\varepsilon)p+\varepsilon u, where uu is uniform over shared events. Both distributions receive the same smoothing before computing the divergence; the teacher target is fixed during optimization.

DivergenceβEvent smoothing εLookahead threshold τAvg5
JSD, original anchor0.500.00130.924
JSD0.110⁻⁸0.00130.191
JSD0.510⁻⁸0.00130.735
JSD0.910⁻⁸0.00130.261
FKL—10⁻⁸0.00128.790
RKL—10⁻⁸0.00127.256
JSD, no threshold pruning0.50030.129

At the same smoothing level, JSD with β=0.5 exceeds FKL by 1.945 points and RKL by 3.479 points. Within the JSD family with ε=10⁻⁸, β=0.5 performs best, exceeding β=0.1 and β=0.9 by 0.544 and 0.474 points, respectively. Comparing β variants to their shared smoothed reference separates the mixture-weight change from smoothing; the original unsmoothed β=0.5 anchor remains higher than all three smoothed variants. RKL completed 72 updates, whereas the other configurations completed 78, so its comparison also includes a difference in accepted training volume. These experiments cover one model pair and one training seed.

Response to Reviewer Hprj - Part 3 of 5

5. ALM/ULD evaluations, gains over SFT, and a same-tokenizer reference

We agree that ALM and ULD are relevant comparisons. We trained and evaluated both methods in our cross-tokenizer OPD setting. Neither yielded successful distillation in the tested configurations: the evaluated ULD checkpoint, after approximately 60 training steps, scored substantially below its SFT initialization on all five benchmarks. The evaluated ALM checkpoints also underperformed their corresponding SFT starting models, with particularly severe degradation on the Gemma pair. Specifically, ALM achieved Avg5 43.98 on Qwen3-32B → Qwen3.5-2B at training seed 303, step 60, compared with SFT 44.39, and 18.18 on Qwen3-32B → Gemma-2-2B-it at training seed 202, step 77, compared with SFT 29.67. We report these ALM scores as concrete examples of the observed outcomes; they are not ULD results. These observations are specific to the tested configurations and do not establish that either method necessarily fails under other settings.

GOLD also exhibited substantial degradation on the Gemma pair. Its seed-202 Avg5 fell from 30.287 at step 20 to 22.021 at the endpoint, below the SFT reference of 29.67. In light of this degradation, we did not pursue further GOLD training on the Gemma pair with seed 303. We retain the available seed-202 score as a diagnostic of this degradation, but do not report a complete three-seed Gemma-GOLD aggregate in the comparison table.

Regarding the gain over SFT, we report both the original main-table comparison and the three-training-seed mean:

Model pairFixed SFT Avg5Original SEAM Avg5Original gainThree-seed SEAM mean Avg5Three-seed mean gain
Qwen3-32B → Gemma-2-2B-it29.6731.70+2.0331.276+1.606
phi-4 → Qwen3.5-2B42.5245.76+3.2445.677+3.157
Qwen3-32B → Qwen3.5-2B44.3947.24+2.8547.151+2.761

The SFT entries are fixed starting-model references, not independently retrained SFT averages. SEAM exceeds the corresponding SFT reference in all nine model-pair/training-seed settings. These differences quantify the observed improvement from teacher-supervised SEAM training as a whole; they do not isolate teacher supervision from additional training or the alignment mechanism.

We have not completed a same-tokenizer reference experiment. With the current model set, constructing such a pair would also change the teacher or student, so it would provide contextual evidence rather than a strict upper bound for the original pairings. The SFT comparisons above quantify the observed gains in the tested cross-tokenizer settings.

6. Matched-budget comparisons against the main baselines

Matched accepted-response budgets.

We checked the training records to match the accepted responses actually used for optimization. For the five main methods, each complete update uses 256 accepted training responses. The evaluated intermediate checkpoints therefore provide matched accepted-sample budgets:

CheckpointAccepted responses used for training
Step 205,120
Step 4010,240
Step 6015,360

This holds for all five methods in the seed-202 curves across the three model pairs. Comparing the same steps therefore matches both the number of completed updates and the accepted-response count. The 29 completed main runs also consume the same 20,000 prompt-occurrence budget, while accepted-response counts can differ at their endpoints. Equal accepted counts do not imply identical trajectories, since each method samples from its own student policy.

The measured Avg5 scores at these matched accepted-sample budgets are shown below. All results use training seed 202; scores are pass@1 × 100, averaged equally over the five benchmarks. These are direct checkpoint evaluations, without interpolation. Bold indicates the highest score in each row.

Model pairAccepted training responsesSEAMBPMSimCTGOLDDSKD
Qwen3-32B → Gemma-2-2B-it5,12030.44130.76029.90330.28727.036
Qwen3-32B → Gemma-2-2B-it10,24031.32130.30328.96019.41628.728
Qwen3-32B → Gemma-2-2B-it15,36030.80630.09528.48621.92228.706
phi-4 → Qwen3.5-2B5,12044.55344.55240.89744.07639.933
phi-4 → Qwen3.5-2B10,24045.07344.51044.74644.29343.574
phi-4 → Qwen3.5-2B15,36045.20444.42845.26344.42644.441
Qwen3-32B → Qwen3.5-2B5,12045.66245.26942.38545.21042.296
Qwen3-32B → Qwen3.5-2B10,24046.60046.00644.71646.02245.024
Qwen3-32B → Qwen3.5-2B15,36046.25745.83345.48045.95144.958

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary

We thank the reviewers for their constructive feedback. The main additions and clarifications are:

  • Training-seed robustness. We added experiments with two new training seeds across three teacher–student pairs. Across three training seeds, SEAM achieves mean Avg5 gains of 0.68–0.90 percentage points over the strongest baseline with complete three-seed results in each pair.

  • Benefit under matched budgets. We added comparisons under matched accepted-response budgets and common compute caps. At the tested compute caps with training seed 202, SEAM exceeds BPM by 0.877 / 0.570 / 0.709 points on the Gemma / Phi / Qwen pairs, respectively. The observed gains therefore do not depend solely on more expensive training endpoints.

  • Additional design and baseline evidence. We evaluated alternative divergences, JSD mixture weights, pruning, and coverage/weighting controls, and tested ALM and ULD. These experiments clarify the design choices and quality–cost trade-offs.

  • Scope and limitations. We quantified space-prefixed tokens and boundary disagreement and examined teacher–student wrong-answer agreement. We acknowledge the limitations of one-byte supervision and the risk of inheriting teacher errors. We will revise the manuscript to incorporate the new evidence and distinguish the general alignment interface from its evaluation in OPD, keeping our conclusions specific to the tested settings.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.