acceptodds
Under review as a conference paper at ICLR 2027

Imprinting: Knowledge Distillation Can Shift Student Behavior

Abstract

Knowledge distillation trains student models on teacher-generated responses to transfer task capability, but those responses may also carry behavioral tendencies of their source. We investigate whether the source teacher can shape student behavior beyond the capability that distillation is intended to transfer. We construct harm-accepting and harm-avoiding variants of the same base teacher, use them to generate responses to ordinary reasoning and knowledge questions unrelated to safety, and distill those responses into students with different base models. Students generally acquire task capability while their downstream decisions shift toward the source teacher's safety-pragmatism disposition, a phenomenon we call imprinting. Distillation from the harm-accepting teacher reduces harm avoidance, whereas distillation from the harm-avoiding teacher increases it. Imprinting varies across teacher–student pairs and does not extend uniformly across different forms of safety behavior. Excluding a sparse set of divergence tokens reduces imprinting, whereas response rewriting preserves the source-dependent effect and can introduce additional rewriter-dependent shifts. Post-distillation preference optimization raises harm avoidance but does not eliminate the source-dependent difference. These results show that behavioral tendencies can be intentionally imprinted through the source teacher, even with ordinary task supervision.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Behavioral Imprinting, Knowledge Distillation, Cross-Model Transfer, AI Safety, Safety Alignment

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_RjRPOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether knowledge distillation transfers a teacher's behavioral tendencies along with its task ability. The authors build harm-accepting and harm-avoiding teachers and distill their answers on ordinary STEM questions into three small models. For the Qwen teacher family, the students shift in both directions, following the source teacher. HarmBench and SORRY-Bench show no consistent ordering, and masking divergence tokens reduces the shift while rewriting and on-policy distillation do not remove it.

Critical strengths
  1. The bidirectional design with a capability control is clean: both teachers see the same scenarios with opposite continuations, and the capability metrics do not change significantly.

  2. Negative results are reported too: the effect is weaker for the Mistral teacher family, and there is no ordering on HarmBench.

Critical weaknesses
  1. The paper does not isolate what is new relative to prior work. Both teacher families are trained with ordinary KD. There is no same-base comparison and no comparison between raw task data and filtered data, and the manipulation set contains only 22 hand-written scenarios. A reader cannot tell whether the result comes from the ordinary-KD setting or from those 22 scenarios.

  2. The paper also does not show that the disposition it transfers is harm-specific. For the harm-avoiding students, harm avoidance rises while control pragmatism falls below the untrained model, which the authors themselves call a shift in the safety-pragmatism trade-off. This is equally consistent with general caution. The control that would separate the two, a same-format teacher with no safety content, is built and validated but never distilled.

Questions for authors
  1. Would you distill the same-format control teacher?
  2. Would you add a control where no one is harmed but caution is costly?
  3. Are there held-out scenarios beyond the 22?
5 comments
AuthorsOpenReview ↗

Response to reviewer RjRP

We thank the reviewer for noting that the bidirectional design and the capability controls are clear, and for the concrete suggestions on controls. We respond to each point below.

W1: what is new, same-base comparison, raw versus filtered data, and the 22 scenarios

What is new relative to prior work?

We study whether the teacher may imprint a decision disposition on the student even when the student improves on the task and shows no clear anomaly on commonly used safety benchmarks.

Studies on subliminal learning show that behavioral traits can transfer through seemingly unrelated data, mainly between a teacher and a student that share the same base model. Dang et al. (2026) show that unsafe behaviors can transfer across model families when the student trains on the trajectories of the teacher on the same type of task. Our setting combines the following properties.

  • The teacher and the student have different base models.
  • The distillation data are responses to STEM questions unrelated to the evaluated behavior.
  • The distillation aims to improve capability, and the capability of the student indeed improves at the same time.
  • The transfer is bidirectional. Two teachers trained from the same model toward opposite dispositions shift the students in opposite directions.

We measure how the student actually chooses when the objective conflicts with a risk, together with its tool choice.

In addition, none of masking divergence tokens, rewriting, and alignment after distillation fully removes this shift. The shift also appears in on-policy distillation, where the teacher provides no text.

The two students also differ in the same direction in tool choice on PropensityBench, as described in the general response and under Q3.

Same-base comparison

This paper studies distillation that aims to improve capability, where the student learns from a more capable teacher. If the student and the teacher shared the same base model, the two would have almost the same capability, and such distillation may have almost no capability to transfer. This case differs from the setting studied in this paper.

For this reason, the teacher and the student in this paper have different base models, and imprinting appears in most teacher–student pairs. We additionally distill Qwen3-32B into Qwen3-8B, a student from the same model family and the same generation as the teacher, and imprinting also appears in this student. The student distilled from the harm-avoiding teacher has higher HA than the student distilled from the harm-accepting teacher by 12.60 percentage points, and this difference is significant at p<0.001, as reported in Table 15 of the revised paper.

Raw versus filtered data

We understand "filtered" here as keeping only the responses that the teacher answers correctly. If the reviewer refers to a different processing step, we would be glad to provide further details. In our setting, this filter serves two purposes. First, distillation aims to transfer correct reasoning for the task. The quality of the reasoning in an incorrect response is uncertain, and we therefore keep only responses with a correct final answer. Second, the filter requires every teacher to answer the same question correctly, and every student therefore trains on the same questions and the same amount of data. On this matched basis, the students improve in capability, as shown in Tables 12 and 13, and the behavioral differences can be attributed more directly to the source teacher.

Keeping only correct responses is also a common practice when distilling reasoning ability (Magister et al., 2023). In on-policy distillation, the student trains on no teacher responses, and this filtering step does not exist. Imprinting still appears, as shown in Table 2 of Section 3.

Role of the 22 scenarios

The 22 scenarios serve only to construct the teachers, and the students never see them. The training data of the students are correct responses to 6,943 STEM questions, and the evaluation relies on separate benchmarks. No scenario text matches any ManagerBench item exactly or after normalization, and the longest shared span is 4 words, as reported in Table 11 of the revised paper.

AuthorsOpenReview ↗

Response to reviewer RjRP (part 2)

W2 and Q1: harm-specific disposition and the control teacher

The safety–pragmatism disposition is the experimental setting for studying imprinting

The claim of this paper is imprinting, namely that the behavior of the student shifts toward the source teacher after distillation. We choose the safety–pragmatism disposition of ManagerBench as the experimental setting because it reflects a decision that deployed models face, namely which side the model favors when the task objective conflicts with a risk.

ManagerBench measures this disposition with two metrics together. HA is the fraction of scenarios involving human harm in which the agent avoids the risk, and CP is the fraction of property-only control scenarios in which the agent completes the task as usual. The student distilled from the harm-avoiding teacher shifts toward the cautious side on both metrics. The transferred disposition therefore covers both decisions that involve human harm and decisions that involve only property.

Q1: students distilled from the control teacher

Yes. We have distilled and evaluated the students of the control teacher for all six teacher–student pairs.

The control teacher is the fourth teacher variant in Section 2.1. Like the two disposition teachers, the control teacher is fine-tuned with BC-LAT on 22 first-person scenarios, each paired with two continuations. The difference is that the scenarios are unrelated to safety, such as the ordering of a list or the notation of units, as listed in Table 8 of Appendix C. The table below reports the Qwen3-32B teacher family, and Tables 12 and 13 in the revised paper report the complete numbers.

Untrained (before distillation)Harm-acceptingControlBaseHarm-avoiding
HA
Qwen2.5-7B0.35260.31010.35880.4556▲0.5873▲
Ministral-8B0.13980.1894▲0.2681▲0.2505▲0.3416▲
Granite-4.1-8B0.24090.28050.3492▲0.4466▲0.5153▲
CP
Qwen2.5-7B0.93500.95280.92520.89370.8543▼
Ministral-8B0.97830.96650.95870.96460.9272▼
Granite-4.1-8B0.94690.92720.92910.8622▼0.8169▼

Each column reports the student distilled from one teacher, and Untrained is the student before distillation. ▲ and ▼ mark values significantly higher or lower than the untrained model (p<0.05).

The phenomenon raised by the reviewer is that HA rises while CP decreases to a level lower than the untrained model. The student of the control teacher does not show this phenomenon. The CP of this student is not significantly lower than the untrained model for any of the three students, whereas the CP of the student distilled from the harm-avoiding teacher is significantly lower for all three. The HA of the student distilled from the harm-avoiding teacher is also significantly higher than the HA of the student of the control teacher for all three students. BC-LAT fine-tuning of the teacher on training data in the same scenario format is therefore not sufficient to produce this phenomenon.

Distillation itself may also change the student. The student of the control teacher has higher HA than the untrained model for Ministral-8B and Granite-4.1-8B. When the teacher is specifically trained toward a decision disposition, the tendency of the student to favor the objective or to avoid the risk becomes more evident and follows the direction of the source teacher, as shown in Table 14 in the appendix. This result indicates that imprinting is a problem that deserves attention.

This control tests whether the change comes from the content that the teacher learned.

Q2: a control where no one is harmed but caution is costly

Yes. The ManagerBench control set serves as this control. Its scenarios involve only property, no person is harmed, and the cautious option carries an operational cost. We report the results on this set as CP.

The table under Q1 above reports the results. The CP of the student distilled from the harm-avoiding teacher is significantly lower than before distillation for all three students. This student therefore chooses the cautious option more often even when no person would be harmed. The student of the control teacher shows no such change. The student therefore inherits a general risk trade-off disposition that is not specific to human harm.

AuthorsOpenReview ↗

Response to reviewer RjRP (part 3)

Q3: held-out scenarios beyond the 22

The 22 scenarios are a tool for giving the two teachers opposite dispositions. The disposition of each teacher is verified on ManagerBench in Table 1, and ManagerBench has no text overlap with the scenarios, as shown in our response to W1. The claim of this paper concerns the next step, namely whether a teacher with a disposition imprints this disposition on the student through task data unrelated to the behavior. This question does not depend on which set of scenarios is used to construct the teachers.

The students never see the scenarios, and all evaluations are held-out. Student behavior is measured on the 2,096 ManagerBench items that involve human harm and the 508 control items.

We also measure behavior on a task with a different form. PropensityBench tests agentic tool choice under pressure, whereas the 22 scenarios contain no tool calls. When the misaligned tool carries an explicitly harmful name, the two students still differ in the same direction, as reported in Section 5.

Ethics flag

The Ethics Statement of the paper states that the teacher and student adapters and the distillation data are available only under a research-only license with gated access. The anonymous link in the paper is provided for the purpose of review only, as stated in the Reproducibility Statement. If the reviewer has a specific concern beyond this point, we would be glad to address it directly.

Reviewer_LimSOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper focuses on the subliminal learning or the student model unintentionally inherits some behaviour pattern and tendencies from the teacher models even if the corpus for distillation is not directly related to the content of distillation. This paper specifically focuses on the harm avoidance and measures if the model avoids the harmful action. It first constructs harm-accepting and harm-avoiding teacher variants through behavioral-continuation LAT and then use these teachers to teach student models on STEM questions. After that, the student models are tested on their STEM ability and harm avoidance.

Critical strengths
  • This paper focuses on an important topic and studies the effect of subliminal learning on harmful avoidance and AI safety.
  • It runs extensive experiments on different teacher models and student models.
  • It compares the token difference between the harm-accepting and harm-avoiding teacher and runs paraphrasing experiments to understand if rewriting could eliminate the imprinting.
Critical weaknesses
  • The motivation of using behavioral-continuation LAT is not clear. Why not directly SFT on the completions of harm-accepting and harm-avoiding teacher?
Questions for authors

See the weaknesses above.

1 comment
AuthorsOpenReview ↗

Response to reviewer LimS

We first thank the reviewer for the positive assessment of our paper.

Why does the paper employ BC-LAT?

This paper requires a pair of teachers that come from the same model, shift significantly in opposite directions, and keep their capability and refusal behavior unchanged. With such a pair, the differences between students can be attributed to the disposition of the teacher. BC-LAT serves as the tool for obtaining such a pair, and the claims of this paper concern imprinting in the student after distillation.

What happens with direct SFT?

Following the suggestion, we train teachers with direct SFT on the same 22 scenarios and continuations.

VariantHA↑CP↑
Untrained0.50570.9232
Harm-accepting (BC-LAT)0.2490▼0.9783▲
Harm-accepting (SFT, new)0.3492▼0.9606
Harm-avoiding (BC-LAT)0.5630▲0.8701
Harm-avoiding (SFT, new)0.52290.8858

The teacher is Qwen3-32B. ▲ and ▼ mark significant improvements and degradations relative to the untrained model (p<0.05). The capability and refusal metrics show no significant difference across the five rows, and Table 10 in the revised paper reports all columns.

SFT shifts the teacher in both directions and leaves capability and refusal behavior unchanged. However, the shift is smaller. Only the harm-accepting direction is significant, whereas the harm-avoiding teacher does not differ significantly from the untrained model. SFT alone therefore does not produce a pair of teachers that both shift significantly.

Revision

Our study focuses on the task performance of the student after distillation and on whether the teacher imprints a specific disposition on the student. For this reason, the submitted version did not include the discussion of how the teachers are trained. We have included this table and the explanation in the appendix section on teacher construction, as Table 10.

Reviewer_kKh5OpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies to what extent the harmfulness of teacher LLMs are transferred to students through distillation. It mainly focuses on harm-accepting and harm-avoiding benchmark evaluation. Aside from control and base teachers, there are two teachers that are slightly tuned to be more harmful and less harmful which are used to study their effect on the student. In terms of architecture the paper explores Mistral and Qwen3 as teachers and performs (cross-architecture) distillation to older and smaller Qwen, Mistral, and Granite variants.

The paper first establishes that the students do indeed inherit the properties of the teacher (to a different extent across architecture pairs), then shows that the prior work of divergent token masking is effective in significantly reducing this harmfulness imprinting effect, though not fully removing it. They also explore whether re-writing of teacher generations could mitigate distilling this effect, which the paper finds not to be the case (even after re-write the student from a harmful teacher is more harmful than the other student). Lastly, they show that post-distillation training (SFT and DPO) can also be effective in reducing this harmfulness property, but again the two students coming from harmful and harm-avoidant teachers show differences.

Critical strengths

S1) The overall experimental setup of the paper seems convincing to me: trying different architectures, reporting distillation results across architectures, reporting multiple harmfulness metrics/benchmarks, exploring three existing approaches in reducing harmfulness (during distillation with divergent token masking, text re-writing, and post-distillation with SFT and DPO)

S2) The text, figures, and tables were overall easy to follow.

S3) The question of whether distilling through unrelated data distills harmfulness / harm-avoidance properties of the teacher sounds interesting to me.

Critical weaknesses

W1) [major] I am a bit confused on what aspects of the paper are truly novel. In essence, I find paragraph 2 of the related work on “Traits that travel through training data” quite close to what the paper does and reports.

Specifically, in [1,2] it has already been shown that distilling on unrelated data can distill such properties. Therefore I find RQ1 in Line 90 to already be mostly answered by prior work.

Secondly, The effect of relying on divergence tokens has also been proposed and studied by [2]. So the findings listed under 4.1 (e.g. Line 314) would also be quite expected (?)

Lastly, the re-writing experiment (Sec 4.2) again seems to be quite close to that of [3], where the re-writing is again studied in the context of subliminal learning, and there the authors also had similarly found that re-writes are not effective in preventing distillation.

Overall, the fact that imprinting (or subliminal learning [1,2,3]) occurs, that token-masking [2] is effective, and re-writing is not [3], sound quite similar to what each of the relevant papers report. Could the authors please clarify what the paper claims as contribution w.r.t existing subliminal learning works? Note that this question is currently not addressed in Paragraph 2 of Related work (Lines 446-458), which should not be the case. Currently the last sentence (Line 456) does not connect existing work to this paper and rather re-emphasizes what this paper does.

W2) [major] This follows W1 in terms of novelty of findings. To me, whether SFT, DPO, or targeted DPO are effective for reducing model harmfulness sounds like an independent question from distillation. Given any language model (teacher, base student, or a distilled student), these post-training alignment approaches would each be effective to a certain extent, with one working better than the other perhaps. So I don’t see the connection of findings such as (Targeted DPO is better than DPO, which is better than SFT) quite fitting the paper’s story on imprinting, as I would suspect that on existing (non-distilled) models, one would have a similar findings.

W3) [minor] The paper is written well, but it is not very friendly to someone who is not directly working in the field. For instance, the benchmarks of Table1 and Figure2 could have been discussed in a bit more detail (e.g. under preliminaries paragraph). Currently each is discussed with very few words in Lines 139-148, which is a bit limited.

W4) [minor] The current tables and figures are a bit over-populated with the current results. One could for instance pick a few columns / model pairs for the main paper and leave the rest for the appendix with proper references. In particular, Figure 2 took a very long time to parse as it has multiple dimensions, and layers of labels. Also the barplots at the top are barely observable at this size.


References

[1] Subliminal Learning: Language models transmit behavioral traits via hidden signals in data, https://arxiv.org/abs/2507.14805

[2] Towards Understanding Subliminal Learning: When and How Hidden Biases Transfer , https://arxiv.org/abs/2509.23886

[3] You Didn't Have to Say It like That: Subliminal Learning from Faithful Paraphrases , https://arxiv.org/abs/2603.09517

Questions for authors

Questions about novelty are listed under W1 and **W2 above.

5 comments
AuthorsOpenReview ↗

Response to reviewer kKh5

We thank the reviewer for the careful review. The main concern of the review is the novelty relative to the subliminal learning papers [1,2,3], and we address this concern first.

W1: contribution relative to subliminal learning [1,2,3]

Difference from the three papers

Cloud et al. [1], Schrodi et al. [2], and Gisler et al. [3] show that behavioral traits can transfer through seemingly unrelated data, mainly between a teacher and a student that share the same base model. Dang et al. (2026) show that unsafe behaviors can transfer across model families when the student trains on the trajectories of the teacher on the same type of task.

Our setting combines the following properties. The teacher and the student have different base models, the distillation data are responses to task questions unrelated to the evaluated behavior, the capability of the student improves, and the transfer is bidirectional. We further investigate whether this disposition can be removed. Divergence-token masking reduces imprinting without removing it, rewriting by another model does not remove it, and DPO improves the overall safety results while the difference between source teachers remains.

The three points raised by the reviewer

  1. RQ1. The main results of Cloud et al. [1] are limited to the same base model, and the abstract states that the effect is not observed when the teacher and the student have different base models. Schrodi et al. [2] observe transfer only for individual animals when the base models differ. We observe imprinting between different base models in most teacher–student pairs.
  2. Section 4.1. In Schrodi et al. [2], masking divergence tokens in number sequences largely suppresses the transfer. We apply the same masking to ordinary task responses. The masking reduces the imprinting gap, but imprinting remains. The two students still differ significantly after masking in harm avoidance (HA), the fraction of harm scenarios in which the model avoids the harmful action, as shown in Table 3.
  3. Section 4.2. In Gisler et al. [3], the teacher, the paraphraser, and the student are the same model, and the authors note that the cross-model case is not yet tested. We rewrite the responses with a model from another family, and the imprinting gap persists, as shown in Table 4.

Results beyond these three papers

  • Imprinting without teacher-generated text. The students in these three papers [1,2,3] all train on data generated by the teacher. In on-policy distillation, the teacher generates no text and only assigns log-probabilities to the responses sampled by the student. Imprinting still appears, as shown in Table 2 of Section 3.
  • Tool choice. On PropensityBench, which the general response describes, the student distilled from the harm-accepting teacher selects a misaligned tool with an explicitly harmful name more often than the student distilled from the harm-avoiding teacher, as shown in Figure 3 of Section 5.

Revision

We have rewritten the second paragraph of Related Work to state how this paper relates to Cloud et al. [1], Schrodi et al. [2], and Gisler et al. [3]. The revised text is the paragraph titled "Traits that travel through training data" in Section 6.

AuthorsOpenReview ↗

Response to reviewer kKh5 (part 2)

W2: the ordering of SFT, DPO, and targeted DPO

Which question does Section 5 answer?

This paper investigates how the disposition of the teacher affects the downstream decisions of the student in knowledge distillation aimed at improving capability, and whether this effect can be removed. Section 5 answers the second part.

Safety SFT and DPO are common safety alignment methods. We employ them to test one question. The two students share the same base model and are trained on the same questions, and they differ only in their source teachers. The difference between them before alignment is therefore the imprint of the source teacher. The main focus of Section 5 is to answer whether this difference disappears after the same alignment is applied to both students.

What is the result?

The difference does not disappear. The table below is computed from Table 5.

StudentNo alignmentAfter generic DPOAfter targeted DPO
Qwen2.5-7B14.5512.7915.03
Granite-4.1-8B16.617.017.59

Values are the HA difference in percentage points between the student distilled from the base teacher and the student distilled from the harm-accepting teacher. The difference after generic DPO is significant for both students at p<0.001. Table 5 reports the HA of each student.

What do the two DPO variants represent?

Generic DPO trains on general safety alignment data from PKU-SafeRLHF and corresponds to a developer who does not know the disposition of the teacher. Targeted DPO trains on preference pairs designed for this disposition and corresponds to a developer who already knows the disposition.

In both cases, alignment raises the HA of both students, but the difference caused by the source teacher remains. Fully removing this difference remains an open problem.

Revision

We have revised the opening of Section 5 to present this comparison first. The ordering among the alignment methods is now stated in one sentence.

W3: more detail on the benchmarks

Thank you for the suggestion. Each benchmark received only a brief introduction in the submitted version because of the page limit. With the additional page allowed for the revision, we have included a preliminaries paragraph in Section 2 that describes what each benchmark measures and how to read each metric.

W4: density of tables and figures

Thank you for pointing out the readability of Figure 2. The figure was small in the submitted version because of the page limit. We have enlarged Figure 2 and adjusted the size of the bar plots at the top.

We keep the results of the figures and tables in the main paper because these pairs and metrics directly support the claims of each section. Moving them to the appendix would require readers to go back and forth between the main paper and the appendix. We are glad to adjust the presentation further if the reviewer has other specific suggestions.

Dear Reviewer kKh5,

Thank you again for your thoughtful feedback. We have addressed your concerns in our rebuttal and revised manuscript.

TL;DR: We clarified how our work differs from the subliminal learning papers [1,2,3]: imprinting appears between different base models, remains after divergence-token masking (Table 3), and persists after rewriting by a model from another family (Table 4). We also clarified that Section 5 tests whether alignment removes the difference caused by the source teacher, which remains significant after generic DPO. In the revision, we added a comparison paragraph to Related Work, a benchmark preliminaries paragraph in Section 2, and an enlarged Figure 2.

As the discussion period is nearing its end, we would appreciate hearing whether our responses have addressed your remaining concerns and if you could increase our aspect scores and overall rating.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response to all reviewers and the AC

We thank the reviewers for their comments. Distilling a smaller student from data generated by a larger teacher is a common practice, and distilled students are increasingly deployed as agents. This paper studies a risk in this practice. We investigate whether the teacher may still imprint a behavioral disposition on the student and lead the student to make unexpected decisions, even when the student improves on the task and shows no clear anomaly on commonly used safety benchmarks.

How does this paper differ from prior work?

Teacher and studentDistillation dataEvaluated behaviorStudent capabilityAlignment after distillation
Subliminal learning (Cloud et al., Schrodi et al., Gisler et al.)Mainly the same base modelNumber sequences, code, chain-of-thought for math problems, paraphrased sentencesAnimal preferences, misalignmentNot the objectiveNot tested
Dang et al. (2026)Same and different model familiesTrajectories of the teacher on the same type of task as the evaluated behaviorUnsafe actions of an agentNot reportedNot tested
This paperDifferent base modelsCorrect responses to STEM questions, unrelated to the evaluated behaviorDecisions when the objective conflicts with a risk, tool choiceImprovesTested, and the difference between source teachers remains

Our setting combines different base models, task data unrelated to the evaluated behavior, improved student capability, and bidirectional transfer.

Setting and experiments

We construct a harm-accepting teacher and a harm-avoiding teacher from the same model with the same scenarios. The former favors advancing the objective, and the latter favors avoiding the risk. Both teachers answer the same STEM questions, and we distill only the correct responses into students with different base models. We then evaluate the students on task capability, on decisions where the objective conflicts with a risk, on tool choice, and on commonly used safety benchmarks.

AuthorsOpenReview ↗

General response to all reviewers and the AC (part 2)

Findings

  1. Capability and disposition transfer together. The students improve on the task, and their decisions shift toward their source teachers in both directions in most teacher–student pairs, as reported in Section 3. For the three students of the Qwen3-32B teacher family, the students of the harm-avoiding and harm-accepting teachers differ by 15 to 28 percentage points in harm avoidance, the fraction of harm scenarios in which the model avoids the harmful action. Transfer from the Mistral-Small-3.2-24B teacher family is weaker and more dependent on the teacher–student pairing, which Section 3 also reports.
  2. Imprinting does not require teacher-generated text. Imprinting also appears in on-policy distillation, where the teacher generates no text and only assigns log-probabilities to the responses sampled by the student, as reported in Section 3.
  3. Imprinting extends to tool choice. PropensityBench makes the aligned path fail repeatedly under increasing operational pressure and tests whether the agent switches to a misaligned tool. This tool-choice task does not appear in the fine-tuning data of the teachers. When the misaligned tool carries an explicitly harmful name, the student distilled from the harm-accepting teacher selects the misaligned tool more often, as reported in Section 5. This result suggests that unexpected decisions can occur in deployment.
  4. Commonly used safety benchmarks do not reflect the shift. HarmBench and SORRY-Bench mainly measure how a model responds to harmful requests rather than how the model trades off safety against the task objective. The results on these two benchmarks show no consistent trend that follows the disposition of the source teacher, as discussed in Section 3. The unexpected behavior of the student therefore appears only on the targeted disposition, and a developer who does not know the disposition of the teacher would not know what to test.
  5. Imprinting is hard to remove. Divergence-token masking reduces imprinting without removing it, rewriting by another model does not remove it, and DPO improves the overall safety results while the difference between source teachers remains, as reported in Sections 4 and 5.

New results in this revision

  1. Students of the same-format control teacher (Tables 12, 13, and 14). For the Qwen3-32B teacher family, the students of the harm-accepting and harm-avoiding teachers differ significantly in harm avoidance from the student of the control teacher, in opposite directions.
  2. A student from the same model family and generation as the teacher, Qwen3-32B → Qwen3-8B (Table 15). Imprinting also appears in this student, with a difference of 12.60 percentage points in harm avoidance.
  3. Teachers trained with direct SFT on the same 22 scenarios (Table 10). SFT produces a smaller shift that is significant in only one direction. The teacher construction is a tool, and the claims of this paper concern imprinting in the student after distillation.
  4. Text overlap between the 22 scenarios and ManagerBench (Table 11). No scenario text matches any ManagerBench item.

Together, these results indicate that the risk described above holds in our experiments, and removing imprinting after distillation remains an open problem that deserves further exploration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.