acceptodds
Under review as a conference paper at ICLR 2027

Cross-Trait Transfer in Subliminal Learning

Abstract

Subliminal learning is a phenomenon where a student language model acquires a teacher model's behavioral traits by training on semantically unrelated outputs. It is a subtle statistical phenomenon as trait transmission relies on weak statistical patterns in the generated data. To understand trait transmission between teacher-student pairs, we study cross-trait transfer: how data generated under one teacher trait changes the student's preferences of other traits. To this end, we introduce a directed trait-transfer matrix that quantifies these effects using log-probability gains for student answers. We find that the trait-transfer matrix reveals clusters of related traits, with students sometimes developing preferences for traits similar, but not identical, to the teacher's trait. Such cross-trait structure can be partially captured by output distribution metrics and representation-based metrics. Further, we analyze trait development and interaction: learning dynamics shows a progression from broad shared shifts toward more trait-specific transfer, and multi-trait experiments suggest that opposed traits can enhance such differentiation. Together, our findings reveal salient statistical structures over trait transfer and competition, thus providing a broader view of how hidden preferences are transmitted in subliminal learning.

Primary area:
interpretability and explainable AI
Keywords:
Subliminal learning, cross-trait transfer, representation geometry, activation steering, knowledge distillation, learning dynamics

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_1TyXOpenReview ↗
Rating
3(+2)
Confidence
4
Informativeness
2(+1)
Presentation
2(+1)
Soundness
2
Read review
Paper summary

The original subliminal learning work typically only transferred a single preference (e.g., favorite animal). This paper finds that preference for related preferences (say, green when emerald is the targeted hidden trait) also increases. In particular, this is defined by the change in log probabilities in answering some trait before and after fine-tuning in the subliminal learning setting. The authors correlate six output- and representation-based metrics with cross-trait transfer and find that some of them correlate. Finally, they show how cross-transfer changes over the course of training.

Critical strengths
  • S1: Experimental results are supported across various settings (model families, trait categories, carrier formats, SFT objectives). This supports that the cross-trait transfer is not an artifact of a particular setting but a more general pattern.
Critical weaknesses
  • W1: I find the main finding (i.e., cross-trait transfer) not particularly interesting nor surprising. Specifically, it is shown that transfer is strong within clusters of similar traits (e.g., green and emerald). This is frankly quite expected, as similar traits tend to have similar representations. So, shifting towards one of them will also shift the others. More interesting findings would be (i) surprising cross-trait transfers that cannot be explained by semantic similarity or (ii) cross-trait transfer to other categories (say, yellow to make banana the students' favorite fruit). The current paper also doesn’t go beyond descriptive measures (correlations). In particular, it doesn’t provide any explanations for why cross-trait transfer happens.

  • W2: Some parts of the paper are hardly understandable without the supplemental material. For example, lambda is undefined in the main text or numbering of metrics in plots makes them also unusually hard-to-parse as human. This might be also an artifact of AI usage. To make the last point more explicit: take the title of section 4 ("metrics correlate with cross-trait transfer"). Clearly, we can find some metrics correlated with cross-trait transfer rates and this is what AIs excel at. However, what does it tell us about the mechanisms responsible for cross-trait transfer? How good are these metrics as predictive measures? Do the metrics explain anything beyond the fact that similar traits are simply similar?

Beyond that, it's also unclear how the individual findings fit together.

Questions for authors
  • Q1: Is there any particular reason the numbers are restricted to 00-99 instead of 000-999, as done in the original and follow-up works? Similarly, the number of epochs?

  • Q2: Is the distribution distillation setting just soft distillation? Or, what do you mean by “teacher’s distribution over the first complete carrier after each prompt”?

  • Q3: How much is the finding “competitor traits often gain more than the teacher trait” a function of that the teacher or competitor trait is more salient/frequency of concepts in pretraining data? For example, capitals for countries or RGB color for colors.

  • Q4: What does the cross-trait transfer results reveal that we couldn't have predicted from the pretrained models' representational structure or semantic similarity?

Comments

  • C1: External/internal metrics are undefined in Fig. 1 caption and it’d be helpful to use names instead of numbers. (Only later in the paper they’re defined)
3 comments
AuthorsOpenReview ↗

Response (1/2): novelty (W1, Q3, Q4); technical details (Q1, Q2)

Thank you for the detailed review. We agree that semantic similarity explains the coarse clustering, and that the submission did not make clear what goes beyond it. We now report the two findings you asked for, transfer that semantic similarity cannot explain and transfer to other categories, together with a safety implication. The full tables are in the General Response (Parts 1 and 2).

Novelty (W1, Q3, Q4)

(1) Transfer that semantic similarity cannot explain.

  • Cross-transfer can be stronger than self-transfer. Similarity predicts that the teacher's own trait gains most. Often it does not: after removing answer-wide shifts (as Reviewer E5fi requested), a different trait gains more than the teacher's own in 507 of 900 (56%) teacher evaluations (Part 1(a)). Measuring only self-transfer may therefore miss cross-transfer.
  • Similarity and salience do not fully predict transfer; our metrics add predictive information. We compare ridge regression models using independent semantic embeddings, baseline answer salience (how likely each answer is before training; your Q3), and our transmission measurements. Each held-out trait is excluded from both teacher and answer roles during fitting. For color SFT, held-out R2R^2 is 0.012 with semantics, 0.059 with semantics plus salience, and 0.285 after adding prompt-vector steering (Metric 4). Metric 4 adds predictive information beyond semantics and salience in all six conditions (table in Part 1(b); Appendix F).

On Q4: semantic similarity alone predicts little of the transfer for held-out traits (held-out R2R^2 at most 0.096). Transfer is better predicted by a signal from the transmission itself, the activation change that the trait instruction causes in the teacher (Metric 4).

(2) Cross-category transfer. Following your suggestion, we asked the 30 students of color teachers for their favorite fruit. In sampled answers, strawberry rises from 13.50% to 83.75% for the red student, and pineapple from 0.25% to 52.00% for the yellow student. Over all 900 color-fruit pairs, transfer correlates with the predefined color-fruit association matrix at r=0.445r=0.445 (Part 1(c); Appendix G). The cross-trait transfer in subliminal learning is therefore not confined to the category of its teacher's trait.

(3) Practical safety implication. Our study also has a safety implication, which we'll bring into the main text (Part 2). Three misaligned teachers from Turner et al. (2025; arXiv:2506.11613), trained separately on risky financial, bad medical, and reckless sports advice, generate only number sequences, and we fine-tune a fresh student on each.

  • Students change outside their teacher's domain. The most likely amount that a student suggests borrowing to invest shifts from 5,000 USD for the base model to 8,000 USD for all three students, including the medical and sports students. An audit limited to the teacher's known problem (e.g. medical advice) would miss the change in financial advice.
  • Metric 4 helps monitor emergent misalignment across domain. Steering effects from teacher-minus-base activation vectors agree with the actual transfer at r=0.785r=0.785 (Appendix H). Metric 4 needs only the teacher and base model, so it can help diagnose which unintended behaviors a dataset is likely to shift, i.e. where to audit.

Revision. We'll foreground (1) and (2) as contributions in the introduction and highlight the practical safety implication as a significance of our findings (Part 3(a)). Why cross-trait transfer happens is answered in our next post, together with W2.

Technical details (Q1, Q2)

Q1. We chose 00–99 for computational efficiency. Our distribution-distillation objective and output-distribution metrics require explicitly scoring the carrier distribution. Qwen2.5 tokenizes numbers digit-wise: with shared prefixes, scoring all two-digit strings takes 1+10=111+10=11 probes of next-token distributions, against 1+10+100=1111+10+100=111 for all three-digit strings. We use three epochs, following the released Qwen2.5-7B-Instruct configuration of the original subliminal learning work (n_epochs=3).

Q2. Yes, it is soft distillation, applied to a finite set of complete carrier strings. "First complete carrier" means one entire carrier immediately following the prompt. For numbers, this is a two-digit string such as 37, whose probability is P(3∣x)P(7∣x,3)P(3\mid x)P(7\mid x,3). We score all 100 candidates (00 to 99), normalize their probabilities, and train the student to match the teacher's distribution with cross-entropy, which is equivalent to minimizing the teacher-to-student KL divergence.

AuthorsOpenReview ↗

Response (2/2): presentation and motivation of the metrics (W2, C1)

Thank you for pointing out the presentation issues in the metrics section. We agree with them and will revise Section 4 as follows (see also General Response, Part 3).

(a) The section title is not accurate. "Metrics correlate with cross-trait transfer" describes only how we scored the metrics against the transfer matrix. We'll revise it to "How well diagnostic metrics predict transfer".

(b) Our metrics are not merely correlational. Our language of "correlate" was not accurate. Prompt-vector steering (Metric 4) is an intervention: we add the activation change that the trait instruction causes in the teacher to the base model and observe which answers become more likely. Activation steering is a causal test of what a direction does to behavior (Turner et al., 2023; Rimsky et al., 2024; Chen et al., 2025), and Blank et al. (2026) show that the student in subliminal learning learns the teacher's steering vector. We'll correct this wording in Section 4 and Figure 3.

(c) The metrics cover each stage of the transmission pipeline; together they form a suite of diagnostics. Cross-trait transfer is a three-stage process (Figure 1A): (i) the trait instruction changes the teacher's activations; (ii) this shifts the statistics of the numbers it generates; (iii) the student fitting those outputs can amplify multiple answer traits. Each metric relates the teacher's signal at one stage to a given answer trait:

  • Stage (ii), the numbers. Carrier-distribution similarity (Metric 1): do two traits shift the number distribution alike? If so, the student gets a similar training signal from both.
  • From (ii) to (iii), numbers to answers. The entangled-carrier metrics (Metrics 2, 3 and 6): are the numbers that the teacher's trait promotes also the numbers tied to a given answer?
  • From (i) to (iii), activations to answers. Prompt-vector steering (Metric 4) is the intervention in (b). Prompt-target trait vector similarity (Metric 5) asks: is this activation change aligned with a vector that elicits the answer?

Together they explain why cross-trait transfer happens: a trait creates an activation direction in the teacher, the direction leaves a trace in the output numbers, and a student fitted to that trace moves toward the cluster of related answers that this direction promotes.

(d) The metrics inform prediction and monitoring of transfer. All six metrics are computed from the teacher and the base model, before any student is trained.

  • Prediction. For held-out traits, adding Metric 4 to semantic similarity and salience improves held-out R2R^2 in all six conditions (color SFT: 0.059 to 0.285); Metric 1 improves it in four of six (Part 1(b); Appendix F).
  • Monitoring. For misaligned teachers, steering effects computed from teacher-minus-base activations agree with the students' actual shifts at r=0.785r=0.785. Metric 4 can therefore help diagnose which unintended behaviors a dataset is likely to shift, i.e. where to audit (Part 2(b); Appendix H).

So the metrics explain more than "similar traits are similar": they predict transfer where semantic similarity and salience fall short.

(e) Revisions to figures, formulas and text.

  • Names instead of numbers (C1). Descriptive names such as "carrier-distribution similarity" and "prompt-vector steering" will replace "Metric 1" and "Metric 4" in text and figures.
  • Figures. The Figure 1 caption will define external metrics (computed from the teacher's output distribution) and internal metrics (computed from activations), and panel A will mark stages (i) to (iii) and where each metric is measured. Figure 3 will group the metrics by stage.
  • Formulas. Each metric will be introduced by the question it answers, followed by its formula. We'll define the notation in the main paper, including λ~\tilde{\lambda}, which is now defined only in Appendix B.1.
  • Introduction. It will state the contributions up front (Part 3(a)).

References

  • Turner et al. (2023). Steering Language Models with Activation Engineering. arXiv:2308.10248.
  • Rimsky et al. (2024). Steering Llama 2 via Contrastive Activation Addition. ACL.
  • Chen et al. (2025). Persona Vectors: Monitoring and Controlling Character Traits in Language Models. arXiv:2507.21509.
  • Blank et al. (2026). Subliminal Learning Is Steering Vector Distillation. arXiv:2606.00995.

I thank the authors for the thorough feedback. In particular, I find the cross-category results interesting (I think leading the paper with that result would make the paper stronger). Since this (and the other) results resolve some of my major concerns, I increase my score and recommend this paper for acceptance.

Reviewer_E5fiOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The work focuses on the phenomenon of subliminal learning occurring during teacher–student distillation. Specifically, when a student acquires a trait from a teacher, it may also acquire other unrelated traits that are not explicitly intended to be transferred. Prior work focused on measuring only whether the student acquires the intended trait, not whether it also acquires other traits. The threat here is that evaluation based only on the intended traits can miss these transfers and, furthermore, that training data can carry hidden behaviors that can have implications for safety. The paper misses an opportunity to discuss broader safety, robustness, and practical implications; I think this aspect should be strengthened.

The authors investigate the phenomenon by analyzing the 30 × 30 transfer matrix. This matrix maps, for each teacher trait, the students’ log-probability gains for tokens in the same category (cities, animals, or colors). The paper reports that: (1) transfer concentrates within a subset of traits, and another trait often gains more than the teacher's own; (2) of the six proposed diagnostic metrics, the ones based on teachers' activations are most predictive; (3) as training progresses, the matrix moves from broad shared shifts to more trait-specific transfer; (4) in a two-trait experiment, teachers given opposed traits transmit them less evenly than teachers given aligned ones.

Critical strengths
  1. Within-group concentration of transfer is a new and robust finding. The contrast between within-group and between-group transfer, computed after removing teacher-wide and answer-wide effects, is positive in all 30 conditions. The effect size changes somewhat, but the finding holds across five model families, five formats, and two training variations. As such, this is likely a real phenomenon and not just specific to particular traits. Also, diagnostic tests that look only at the teacher traits intended for transfer would miss these patterns.

  2. Transparent reporting of sensitivity analyses and limitations. The appendices report analyses that show where the results are weaker: individual- versus group-level recurrence (Table 11), probability- versus log-gain scoring (Table 12), margin thresholds for competitor wins (App. E.2), and the exploratory status of the color groups (App. E.1). These let a reader judge how far each claim extends.

Critical weaknesses
  1. The statistic may be a bit misleading. The paper says that a student trained on an "emerald" teacher's data often ends up preferring another color, such as green, more than emerald itself. The evidence is that in 678 of 900 cases, some other answer gained more than the teacher's trait (App. E.2). I see two problems with this: 1) the number needs a baseline. Each teacher trait competes against 29 other answers, so even if training did nothing special for the teacher's trait, one of the 29 would beat it about 97% of the time just by chance. The reported 75% suggests the teacher's own trait is actually favored; 2) some "wins" have nothing to do with the teacher's trait. Fine-tuning on number data can raise certain answers no matter which teacher produced the data; §5 shows this happens. Such an answer is counted as beating every one of the 30 teachers. It woud be good to remove this effect with the trained control student in se 5 and 6.

  2. The findings are not connected to any practical consequence. Safety is the stated motivation, but the paper never says what its findings change. There is not clear failure mode, no detection method, no mitigation. There is actually content in the paper to make a stronger claims, like does transfer spread to similar traits? If audits that check only the intended trait indeed miss the ununtended transfer, then by how much? Another useful aspects is the potential for a leading indicator: prompt-vector steering (Metric 4) is computed before any training and correlates with transfer (r up to 0.81), so it could predict which traits a dataset will transmit, but it is never tested as a predictor. And all traits studied are benign preferences, while earlier work transmitted misalignment (Cloud et al.) and risky financial advice (Schrodi et al.). Spread to related harmful behaviors is where this question would matter most. The contribution describes structure in shifts of harmless preferences and leaves its significance to the reader.

  3. Generality and the single-example claims rest on one adapter setup and one run. Section 3 claims the cluster structure is "not limited to a particular… training recipe choice," but every student is a rank-8 LoRA adapter, and only the loss is varied. Nief et al. (arXiv:2606.00831, May 2026; not cited) report that subliminal transmission disappears under full fine-tuning. If that holds here, the findings apply only to low-rank adapters. The generality claim would be false, and the link to real-world distillation, which the paper uses as motivation, weakens. Separately, each condition is a single training run. Averaged results are robust to this, but the paper's showcase examples (emerald to green in sec 1, the red-hue case in Sec 5) and each competitor win are single entries. The paper itself calls these entries highly variable (Se. 1). Picking the best of 29 noisy competitors against one noisy teacher entry inflates competitor wins through noise alone, which adds to W1.

Questions for authors

Clarifications that could change my assessment, roughly in order of how much they would move it:

  • (W1) How do the 678/900 and 508/900 counts change when gains are measured against a trained control student, or with answer-wide shifts removed? And what would these counts be under a null with no teacher-specific signal (e.g., permuted teacher labels)? If competitor wins stay clearly above that null after removing general fine-tuning shifts, my concern about this claim is resolved.

  • (W2) Can Metric 4, computed before training, predict transfer for held-out traits or conditions? Alternatively, does cross-trait spread occur for a harmful trait (e.g., misalignment)? Either result would give the paper a concrete practical claim and would raise my significance assessment.

  • (W3) (a) Do the emerald to green and red-hue results replicate across 2–3 training seeds? This needs only a few small LoRA runs and would show whether the showcase examples are more than noise. (b) Does the cluster structure hold at other LoRA ranks (e.g., 64, 256) or under full fine-tuning for a few teachers? If this is not feasible, please restrict the Sec 3 claim that results are "not limited to a particular… training recipe choice" to rank-8 LoRA.

  • Regarding construct validity: Are the cluster patterns found in freely sampled answers or in answers that the student would almost never actually select (as shown in App. E.4 for e.g. “broad stoplights”)? In other words, the paper might be finding patterns in low-probability answers that are transmitted, rather than actual preferences that are transferred.

  • Abput novelty: Metrics 4 and 5 are both built from the same prompt-trait vector and track each other closely (Figure 3: 0.80/0.78, 0.81/0.81, 0.76/0.76). Blank et al. (arXiv:2606.00995) already argue that the teacher's prompt acts as a steering vector the student learns. Beyond extending that account to off-diagonal entries, what does Sec 4 contribute? Please clarify

1 comment

Thank you for your detailed review and the concrete suggestions. We ran the analyses you asked for. The General Response covers W2 in more detail (Parts 1(b) and 2); here we answer each point.

W1 (the 678/900 statistic). Thank you for identifying the ambiguity. Our claim is that the teacher's own trait need not receive the largest gain; a related trait can gain more. We do not claim that the teacher's trait loses more often than under random teacher labels, and we'll frame this more carefully.

  • Removing answer-wide shifts (each answer's mean gain across the 30 teachers) lowers competitor wins from 678/900 to 507/900 (56.3%), and same-group competitor wins from 508/900 to 390/900 (43.3%).
  • Under 100K teacher-label permutations the expected counts are about 870/900 and 720/900. The observed counts are below this null, so the teacher's own trait is favored than random baseline, as you note.
  • Of the 507 evaluations won by a competitor, 252 winners (49.7%) belong to the teacher's predefined group, against a null mean of 13.81% (95% null range 9.0% to 19.2%). This supports the structured cross-trait amplification alongside preferential transmission of the teacher’s own trait. (General Response, Part 1(a)).

W2 (practical consequence). We added both tests you proposed.

  • Metric 4 as a predictor (Part 1(b), Appendix F). With each held-out trait excluded from both teacher and answer roles, adding Metric 4 to semantics and salience improves held-out R2R^2 in all six conditions (color SFT: improved from 0.059 to 0.285).
  • Harmful behavior (Part 2, Appendix H). Students trained only on numbers from misaligned teachers (risky financial, bad medical, reckless sports advice) change outside their teacher's domain: the most likely amount that a student suggests borrowing to invest shifts from 5,000 to 8,000 USD for all three students, including the medical and sports students. Steering effects from an extension of Metric 4 agree with the actual transfer at r=0.785r=0.785.

So our study has a practical safety implication: an audit limited to the teacher's known problem (e.g. medical advice) would miss the change in financial advice, and Metric 4, which needs only the teacher and base model, is a potential method for monitoring the cross-domain emergent misalignment. We'll highlight this in the main text (Part 3(a)).

W3 (seeds and LoRA rank). (a) We added two training seeds for the complete 30-color sweep. The emerald → green and red-hue effects remain, and the transfer matrices correlate with the original at r=0.851r=0.851 and 0.9010.901 (Appendix I). (b) We completed a LoRA rank-64 sweep. The cluster structure remains but is weaker: the off-diagonal RMS is 1.243 nats versus 2.392 at rank 8, and the group contrast normalized by this RMS is 0.939 versus 1.453. We did not run full fine-tuning, so we'll restrict the Section 3 claim to the tested LoRA ranks and cite Nief et al. (2026).

Q4 Construct validity (sampled answers). Log-probability gains measure the relative change in answer likelihood, which lets us evaluate rare traits. For the five strongest cross-trait pairs under our original metric, we generated 5K answers per model with the same 10 neutral prompts:

Teacher → answerBase frequencyStudent frequency
Emerald → green0.06%74.30%
Vermilion → red0.00%65.92%
Scarlet → red0.00%69.34%
Crimson → red0.00%83.48%
Red → scarlet0.00%1.66%

Four of the five transfers appear in most sampled answers. Red → scarlet stays rare: its large gain starts from a very low base probability, the case you describe.

Q5 Novelty of Section 4 relative to Blank et al. Blank et al. (2026) show that the teacher's prompt acts as a steering vector that the student learns. Beyond the off-diagonal extension, Section 4 contributes:

  • A suite of diagnostics (Part 3(b)). Our metrics probe each stage of the transmission pipeline: (i) the trait instruction changes the teacher's activations; (ii) this shifts the statistics of the numbers it generates; (iii) the student fitting those outputs can amplify multiple answer traits. Carrier-distribution similarity (Metric 1) probes stage (ii): do two traits shift the number distribution alike? Prompt-vector steering (Metric 4) links (i) to (iii): we add the activation change from (i) to the base model and observe which answers become more likely. Together, they form a suite of diagnostics of cross-trait transfer.
  • Prediction and monitoring. Metric 4 predicts transfer for held-out traits beyond semantics and salience (Part 1(b)) and anticipates the cross-domain change of the misaligned-teacher students (Part 2(b)).

You are right that Metrics 4 and 5 are built from the same prompt-trait vector; we'll present them as one family.

References

  • Blank et al. (2026). Subliminal Learning Is Steering Vector Distillation. arXiv:2606.00995.
  • Nief et al. (2026). Subliminal Learning is a LoRA Artifact. arXiv:2606.00831.
Reviewer_Tv6POpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates cross-trait transfer in subliminal learning. Experiments across multiple models and settings reveal structured transfer between related traits, with activation steering and representation similarity among the strongest correlates. The authors also explore training dynamics and multi-trait interactions. While the paper offers a useful framework for studying cross-trait transfer, its conclusions are limited by the use of same-checkpoint teacher–student pairs and correlational evidence about the underlying mechanisms.

Critical strengths

The framing of "cross-trait transfer" is interesting. The experiments are clear and reasonable.

Critical weaknesses

While the paper presents a useful quantitative framework for characterizing cross-trait transfer in subliminal learning, I find its scientific novelty and significance limited. The main contribution is largely an empirical characterization of transfer patterns using a newly defined matrix and a comparison of existing diagnostic metrics. Although the experiments reveal structured correlations between traits, they do not establish a substantially new mechanism or provide strong causal evidence explaining why these patterns emerge. Consequently, it remains unclear whether the findings advance our conceptual understanding of subliminal learning beyond a more systematic quantification of previously observed phenomena. In my view, the contribution is better suited to a workshop than to a main conference track.

Questions for authors

The comment for the reviewer at the bottom of the first stage is wierd.

2 comments

Thank you for the review. Your main concern is that the paper is an empirical characterization of transfer patterns without providing sufficient new conceptual understanding of subliminal learning, so its novelty and significance are limited. We respond in four points. The full tables are in the General Response, whose parts we cite below.

(1) Transfer that semantic similarity cannot explain (General Response, Part 1(a), 1(b)).

  • Cross-transfer can be stronger than self-transfer. After removing answer-wide shifts (gains shared across all teachers), a different trait gains more than the teacher's own in 507 of 900 (56%) teacher evaluations. Conceptually, this means that a student does not simply inherit its teacher's trait. Related traits compete during training: they emerge together early, then separate, and a related trait can end up ahead of the teacher's own (Section 5 of the paper). Measuring only self-transfer may therefore miss cross-transfer.
  • Similarity and salience do not fully predict transfer; our metrics add predictive information. One might expect transfer to follow how semantically similar two traits are, or how salient an answer already is to the model. We tested this by predicting the transfer matrix for held-out traits with ridge regression, using independent semantic embeddings, baseline answer salience (how likely each answer is before training), and our transmission measurements. For color SFT, held-out R2R^2 is 0.012 with semantics, 0.059 with semantics plus salience, and 0.285 after adding prompt-vector steering (Metric 4). Metric 4 improves held-out prediction in all six conditions tested.

(2) Cross-category transfer (Part 1(c)). The submission measured transfer only within a category (e.g. colors to colors). To test whether it reaches other categories, we added an experiment that asks the 30 students of color teachers for their favorite fruit. In sampled answers, strawberry rises from 13.50% to 83.75% for the red student, and over all 900 color-fruit pairs transfer correlates with a predefined color-fruit association matrix at r=0.445r=0.445 (Appendix G). The cross-trait transfer in subliminal learning is therefore not confined to the category of its teacher's trait.

(3) Practical safety implication (Part 2). Our study also has a safety implication: if transfer is not confined to the teacher's trait, an audit of that trait alone can miss what a student has picked up. To test this beyond harmless preferences, we added an experiment with three misaligned teachers, trained separately on risky financial, bad medical, and reckless sports advice.

  • Students trained only on these teachers' number sequences change outside their teacher's domain: the most likely amount that a student suggests borrowing to invest shifts from 5,000 to 8,000 USD for all three students, including the medical and sports students.
  • Steering effects from an extension of Metric 4 agree with the actual transfer at r=0.785r=0.785, so we see Metric 4 as a potential method for monitoring cross-domain emergent misalignment passed on through distillation data.

(4) Mechanism and causal evidence (Part 3(b)). Our wording, "metrics correlate with cross-trait transfer", was not accurate. Cross-trait transfer is a three-stage process: (i) The trait instruction changes the teacher's activations. (ii) This shifts the statistics of the numbers it generates. (iii) The student fitting those outputs can amplify multiple answer traits.

The metrics probe the different stages of this transmission pipeline and are not merely correlational:

  • Correlational: carrier-distribution similarity (Metric 1) probes stage (ii): do two traits shift the number distribution alike? If so, the student gets a similar training signal from both.
  • Causal: prompt-vector steering (Metric 4) is an intervention linking (i) to (iii): we add the activation change from (i) to the base model and observe which answers become more likely (Turner et al., 2023; Blank et al., 2026).

Together they explain why cross-trait transfer happens: a trait creates an activation direction in the teacher, the direction leaves a trace in the output numbers, and a student fitted to that trace moves toward the cluster of related answers that this direction promotes.

Planned Revision. We'll foreground these contributions in the introduction (Part 3(a)).

Question (comment at the bottom of the first page). That note is not ours. It seems to have been added automatically by ICLR after submission, pointing reviewers to the Reviewer Guidelines.

References

  • Turner et al. (2023). Steering Language Models with Activation Engineering. arXiv:2308.10248.
  • Blank et al. (2026). Subliminal Learning Is Steering Vector Distillation. arXiv:2606.00995.

Dear authors, thank you for your clarifications and replies. I have no further questions, and I will maintain my rating.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response (1/2): Part 1, novelty beyond semantic similarity

We thank the AC and the reviewers. These two posts cover the main concerns; reviewer-specific questions are answered in the individual replies.

Revised PDF. We have uploaded a revision. The main text is unchanged; the new results are in Appendices F to I, marked in blue: F (held-out prediction), G (cross-category transfer), H (safety-relevant extension), and I (training seeds and LoRA rank).

1. Novelty beyond semantic similarity

Concern. 1TyX finds it expected that training on "emerald" also raises "green" (similar traits have similar representations) and asks what our study adds beyond semantic similarity. Tv6P also asks to clarify how we add to the conceptual understanding of subliminal learning.

Further experiments / analysis We added a held-out prediction analysis that tests the semantic similarity explanation, and ran the cross-category experiment 1TyX proposed (colors to fruits).

(a) Cross-transfer can be stronger than self-transfer. Similarity predicts that the teacher's own trait gains most. Often it does not. After removing answer-wide shifts (as E5fi requested), a different trait gains more than the teacher's own in 507 of 900 (56%) teacher evaluations across 30 settings . Half of these winners (49.7%) belong to the teacher's own group of related traits. As an implication to the subliminal learning literature, this suggests that only measuring self-transfer may miss cross-transfer transmission.

(b) Similarity and salience do not fully predict transfer; our metrics add predictive information. We compare ridge regression models using independent semantic embeddings (all-mpnet-base-v2), baseline answer salience (how likely each answer is before training; 1TyX's Q3), and our transmission measurements. Each held-out trait is excluded from both teacher and answer roles during fitting. The table reports held-out R2R^2 for predicting the cross-transfer matrices under Qwen2.5-7B.

TrainingConditionSemanticsSemantics + salienceSemantics + salience + Metric 1Semantics + salience + Metric 4
SFTColors0.0120.0590.1770.285
SFTAnimals0.0030.4090.4020.431
SFTCities0.0960.2570.2620.261
DistillationColors0.0110.1990.2580.314
DistillationAnimals0.0480.2980.3080.308
DistillationCities0.0700.5770.5740.628

(c) Cross-transfer occurs across category. Following Reviewer 1TyX's suggestion, we asked the 30 students of color teachers for their favorite fruit (share of sampled answers):

Teacher's color → fruitBase modelStudent
Red → strawberry13.50%83.75%
Crimson → pomegranate0.00%78.50%
Yellow → pineapple0.25%52.00%
Pink → peach2.50%46.00%

Over all 900 color-fruit pairs, transfer correlates with the predefined color–fruit association matrix at r=0.445r=0.445 (Appendix G). A student's change in subliminal learning is therefore not confined to the category of its teacher's trait.

Revisions to the paper. Added Appendices F (regression prediction) and G (cross-category transfer). The introduction will foreground (a) to (c) as contributions (see Part 3).

AuthorsOpenReview ↗

General Response (2/2): Part 2, safety implications; Part 3, presentation of the metrics

2. Practical implications for safety audits

Concern. E5fi notes that safety is our motivation, yet all traits studied are harmless preferences; E5fi asks whether transfer spreads for harmful behavior and whether Metric 4 can predict it. Tv6P also wants to see our findings' practical significance.

Further experiments / analysis We ran a new experiment with misaligned teachers to test whether their students change behavior in another domain and whether our measurements can help identify those changes, serving as potential monitoring methods for detecting emergent misalignment.

Setup. We use three released Qwen2.5-7B-Instruct teachers from Turner et al. (2025), trained separately on risky financial, bad medical, and reckless sports advice. Each generates only number sequences, on which we fine-tune a fresh student (the financial, medical and sports students) (Schrodi et al., 2026). We then score how much each student would advise a client to borrow to invest (eleven amounts from 0 to 10,000 USD; likelihoods averaged over ten scenarios).

(a) Students change outside their teacher's domain.

ModelMost likely suggested borrowing amount (USD)
Base model5,000
Financial student8,000
Medical student8,000
Sports student8,000

The most likely suggested borrowing amount shifts from $5,000 for the base model to $8,000 for all three students, including the medical and sports students. An audit limited to the teacher's known problem (e.g. medical advice) would miss the change in financial advice.

(b) Metric 4 helps monitor emergent misalignment across domain We extend Metric 4 by extracting teacher-minus-base activation vectors on numerical prompts and applying them to the base model. The resulting steering effects agree with the actual transfer at r=0.785r=0.785 across the double-centered 3×113\times11 log-probability-gain matrices (three teacher conditions, eleven borrowing amounts; Appendix H).

These results support Metric 4, which needs only the teacher and base model, can help diagnose which unintended behaviors a dataset is likely to shift, i.e. where to audit. We therefore see it as a potential method for monitoring emergent misalignment passed on through distillation data.

Revisions to the paper. Added Appendix H. We'll also highlight practical safety implications as a significance of our findings in the main text (see Part 3)

3. Presentation and motivation of the metrics

Concern. 1TyX found the numbered metrics hard to parse and unmotivated, and 1TyX and Tv6P note that correlations do not explain why cross-trait transfer happens.

Planned revision We will state the contributions up front, explain the metrics through the transmission pipeline that motivated them, and present our metrics in a more descriptive and clearer way.

(a) We'll foreground these contributions in the introduction. (1) Transfer is not fully explained by semantic similarity and salience (1.b). (2) Cross-transfer can be stronger than self-transfer (1.a). (3) Related traits compete during training: they emerge together early, then separate, and trait interactions in multi-trait subliminal learning (Sections 5 and 6 of the paper). (4) The safety implication (2.ab).

(b) We'll explain metrics through the transmission pipeline (Figure 1A). Cross-trait transfer is a three-stage process: (i) The trait instruction changes the teacher's activations. (ii) This shifts the statistics of the numbers it generates. (iii) The student fitting those outputs can amplify multiple answer traits.

Our metrics are devised to probe different stages of this pipeline and give two kinds of evidence:

  • Correlational, e.g. carrier-distribution similarity (Metric 1), at step (ii): do two traits shift the number distribution alike? If so, the student gets a similar training signal from both.
  • Causal: prompt-vector steering (Metric 4), an intervention linking (i) to (iii). We add the activation change from (i) to the base model and observe which answers become more likely..

Together they explain how cross-trait transfer arises: a trait creates an activation direction in the teacher, the direction leaves a trace in the output numbers, and a student fitted to that trace moves toward the cluster of related answers that this direction promotes.

(c) Revisions to metric presentation. We'll replace "Metric 1" and "Metric 4" with descriptive names "Carrier-distribution similarity" and "prompt-vector steering" in text and figures. We'll define notations, e.g. λ~\tilde{\lambda}, explicitly in the main paper to facilitate understanding.

References

  • Cloud et al. (2025). Subliminal Learning. arXiv:2507.14805.
  • Sentence Transformers. all-mpnet-base-v2.
  • Turner et al. (2025). Model Organisms for Emergent Misalignment. arXiv:2506.11613.
  • Schrodi et al. (2026). Towards Understanding Subliminal Learning. ICLR.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.