acceptodds
Under review as a conference paper at ICLR 2027

Can Open-Weight LLMs be Protected from Distillation?

Abstract

LLM providers are concerned about their model’s ability being learned by competitors via distillation, which improves a model by training on outputs from another stronger model. To stop distillation, prior work post-processes LLMs’ raw reasoning traces before returning them to users to protect commercial APIs. These techniques, however, cannot protect open-weight LLMs, whose reasoning traces, decoding process, and model weights are completely controlled by the distillation attacker. As open LLMs rapidly approach the frontier, protecting them from being distilled emerges as an important problem that is under-explored. Motivated by it, we propose an antidistillation recipe that fine-tunes LLMs to reason in an encoded vocabulary while answering in their original vocabulary. The key idea is to keep reasoning traces useful to the model itself, but almost impossible for other models to learn from. Our encoded Qwen3-8B and QwQ-32B produce non-readable reasoning, making it detrimental for distillation. Thus, the attacker can only distill on final readable answers of our models, which is shown to be ineffective. We also evaluate 7 adaptive attackers that try to recover original reasoning from our encoded reasoning to distill from. The tested recovery attempts either recover at most 11% of the encoded–readable token mapping, or directly drop non-trivial utility of our model, making it a poor distillation teacher. While mitigating distillation, our encoded models still maintain 88%–99% of the original model’s utility scores on GSM8K, MATH-500, MMLU-Pro math, and AIME 2025. As a first step towards antidistillation for open LLMs, this work aims to open up possibilities for providers to release strong models’ weights while limiting distillation by competitors.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
large language models, open-weight models, distillation defense, adversarial training, model privacy, private reasoning traces, tokenization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_PBY4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether open-weight reasoning models can be protected from distillation by encoding their reasoning traces. The authors propose an antidistillation method that fine-tunes the model to reason with an “encoded” vocabulary and generate answers with the regular vocabulary, so that an attacker can distill only from the final answers.

Critical strengths
  1. Interesting idea on forcing to produce encoded reasoning.
  2. Comprehensive results including utility measurements, undefended distillation controls, and recovery attacks.
Critical weaknesses
  1. An attacker may only need similar task performance, without learning the teacher’s reasoning traces. Although answer-only distillation is evaluated, the limited s1K setting does not establish whether this route remains ineffective with more data or different training setups.
  2. Students are trained on only 1,000 traces while also learning a new encoded vocabulary. It is unclear whether their poor performance reflects effective protection or insufficient data to learn the encoding.
Questions for authors

see weakness

Reviewer_kCHaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Reasoning models are highly susceptible to distillation through their reasoning traces. Existing anti-distillation methods, such as summarizing or rewriting reasoning traces, can protect only the traces of closed-source models and do not apply to open-weight models. The authors propose fine-tuning an open-weight model to reason in an encoded vocabulary, together with a loss designed to prevent attackers from recovering the mapping between the encoded and original vocabularies through vocabulary embeddings. After training, a student model fails completely when trained on the encoded reasoning traces, and attackers have difficulty recovering the mapping using simple attacks such as distance-based methods.

Critical strengths
  1. To the best of my knowledge, this is the first work to systematically study anti-distillation for open-weight reasoning models, making the problem setting highly novel. Existing methods mainly rely on the model provider to hide, summarize, or rewrite the reasoning trace at inference time, so they apply only to closed-source APIs where the provider controls the output process. In the open-weight setting, an attacker has direct access to the model weights, can control decoding, and can inspect the raw reasoning trace; consequently, these prior methods no longer apply. This paper addresses the more difficult setting by training the defense directly into the model weights so that the model reasons using an encoded vocabulary.
  2. Encoding the reasoning trace yields strong anti-distillation performance. Experiments with two teacher-student configurations, Qwen3-8B and QwQ-32B, show that distilling on the encoded reasoning traces and answers reduces the students' accuracy to 0 on GSM8K, MATH-500, and MMLU-Pro Math.
Critical weaknesses
  1. The paper uses a fixed, randomly generated one-to-one token mapping to transform the original reasoning sequence into an unreadable encoded sequence. However, this method is essentially similar to a monoalphabetic substitution cipher, and its security warrants further scrutiny. A fixed substitution does not remove the statistical structure of the original sequence, including token frequencies, co-occurrence patterns, and repetition patterns. An attacker may therefore be able to partially or fully recover the mapping by analyzing a large corpus of encoded reasoning traces together with public language models and statistical priors over natural language. This concern is especially important in the paper's open-weight setting. The attacker not only has access to all model parameters, but can also generate unlimited encoded reasoning traces and construct specific inputs to obtain additional statistical constraints. This makes cryptanalysis based on language statistics an adaptive attack that deserves consideration.
  2. Although encoding the reasoning trace as an unreadable sequence effectively prevents a student model from performing distillation, it also significantly harms the user experience. The model provider can recover the reasoning trace, but users cannot use it to inspect the model's reasoning process. In addition, the performance degradation caused by fine-tuning the model cannot be ignored.
Questions for authors
  1. It would be helpful if the authors could provide token-frequency and bigram-frequency statistics for the encoded reasoning traces. A comparison with the corresponding statistics from the base model's reasoning traces could help clarify whether frequency information presents a potential risk to the security of the encoding.
Reviewer_vacAOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a defense against distillation for open-weight reasoning models. The authors add new encoded tokens to the vocabulary. Each one is a randomly assigned, one-to-one stand-in for a readable token that commonly appears in reasoning traces. They then self-distill the model so that it reasons only in encoded tokens and still answers in readable ones.

The training loss contains, purity losses which keep readable tokens out of the thinking span and encoded tokens out of the answer, utility losses which match the base model through KL on the base distribution, and a leakage losses which are hinge penalties that push each encoded token’s activations away from its readable twin and pull its output embedding toward a random decoy.

The method is applied to Qwen3-8B and QwQ-32B, with 88–99% retention on math benchmarks and 83–86% on non-math MMLU-Pro. Students fine-tuned on encoded traces collapse to 0%, and students trained on answers only gain little. Seven attacks are evaluated: embedding and activation nearest-neighbour matching, prompting, deleting the added embeddings, and masking encoded logits. The best one recovers 11% of the mapping, and decoding-level attacks keep at most 32% of the base model’s accuracy.

Critical strengths

Important, under-explored problem. Defending open weights against distillation is well motivated, and the paper correctly explains why API-side defenses (summarizing, rewriting, antidistillation sampling) don’t carry over.

Simple, clever idea. Moving reasoning into a private alphabet keeps the trace legible to the provider, who holds σ. That is a real advantage over latent reasoning for CoT monitoring, and the paper argues it well.

Strong utility results, especially for QwQ-32B (0.95–0.99, with 0.96 on AIME 2025). The model learns to reason in tokens that did not exist before training, which is interesting in its own right.

Good methodological point. The paper insists on teacher–student pairs where undefended distillation actually helps (Fig. 1). Prior work often skips this, and it should become standard.

Fair-minded baseline. DOGe is re-run on the same base models, and the authors are open about not being able to reproduce its claims.

Critical weaknesses

My main concern is that the security evaluation is much weaker than the claims. The defense is in effect a monoalphabetic substitution cipher over tokens. The attacker has unlimited ciphertext and white-box access to a model that both writes and reads the cipher. The attacks tested are mostly ones the training loss was built to defeat (nearest neighbour on embeddings and activations, and prompting the model to think in English). The natural adaptive attacks are missing:

Sequence alignment with a public reference model. The encoded model is trained by KL to copy the base model’s distribution token by token under σ. With greedy decoding on the same prompt, its trace should be close to σ applied to the base model’s trace. In these experiments the base models (Qwen3-8B, QwQ-32B) are public, and the activation attack itself already assumes access to base traces. So aligning encoded and base generations on a few hundred prompts and taking the majority vote per encoded token looks like it could recover close to all of the mapping. If the threat model says the base is never released, the attacker still has close relatives: an earlier checkpoint, a sibling model, or any model trained on similar data. A distribution-level alignment against such a model is a much stronger test than cosine nearest neighbour, and the paper’s threat model is not consistent on whether the attacker has a reference model. Classical decipherment. A 1:1 substitution with K≈8k symbols and unlimited ciphertext is the textbook case for frequency analysis plus n-gram/LM-guided decipherment (EM- or Bayesian-style work by Knight and others). Token frequencies alone pin down many high-frequency function tokens. Nothing in the objective changes the unigram or n-gram statistics of the encoded text; they are kept by design, because the KL loss preserves the base distribution. This attack needs to be evaluated.

Using the model as a translation oracle. The causality loss forces the model to really read its encoded trace, and the answer span must be readable. So an attacker can put an encoded trace in the prompt or a previous turn and ask the model in the answer region to “repeat”, “translate” or “summarize the reasoning above”. Copying single tokens (“repeat after me: r4821”) is the extreme case. The prompting attacks only test whether the model thinks in readable tokens, not whether it will translate encoded content into its readable answer. Learning the encoding instead of breaking it. The defended model learned the encoded vocabulary from about 8k traces. A student could plausibly do the same with more data, logit-level (KL) distillation from the teacher’s full distribution over encoded tokens, or a two-stage scheme (learn the encoded “language”, then reason in it). Distillation is only tested as SFT on 1k s1K samples with randomly initialized new embeddings. The 0% collapse could simply reflect too little data to learn ~8k new tokens. It doesn’t show the traces are useless as a training signal. A curve over the number of distillation samples is needed.

Cheap repair of the weights. Masking encoded logits gives 30–32% retention without further training. An attacker with the weights would next do a short SFT or RL run on the masked model (e.g., on its own correct, now-readable traces) or train a small translation head. How much compute it takes to recover most of the utility this way is the key measure of the defense’s cost to an attacker, and it isn’t reported. The conclusion itself says the defense can only make distillation “more expensive”, so that cost should be measured.

Questions for authors

Please refer to the Weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.