- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper focuses on the LLM transmission task. The most important observation in this work is homeopathic learning: when mixing poisoned data and clean data in the training dataset, the performance of the poisoned data can be strengthened. Under the same setting, experiments show that clean data can strengthen the transfer of misalignment. The results from 19th-century persona and archaic bird names datasets support this claim. When mixed the archaic bird with modern ones, the persona shift can be stronger. This observation is just like the backdoor attack, but the attack performance cannot be found in one training set but in others.
Critical strengths
-
The observation in this work is clear and insightful. In the homeopathic learning setting, experiments show that clean data can strengthen the transfer of misalignment.
-
Although the homeopathic learning is the same as subliminal learning, homeopathic learning extends beyond subliminal transmission.
-
The experiment is comprehensive and clear.
Critical weaknesses
-
The authors use simple ridge regression to explain why clean data amplifies backdoors (Eq.1). I think this is not enough for this experimental observation. Moreover, the conclusions hold within the model designed by the authors, but this does not imply that actual LLMs achieve Homeopathic Learning through this specific mechanism. The evidence-based theory is more important.
-
Under different optimizers, homeopathic learning is not always work. This needs clear clarification; without this clarification, the contribution of this paper can be weakened.
-
The observations can be summarized more clearly. Although the paper presents extensive experiments, the key observations are scattered across different sections and appendices without a sufficiently clear synthesis.
-
The variance of the homeopathic learning is larger.
Questions for authors
See in Weaknesses.
3 comments
Response to Reviewer iqtP (Part 2/3)
W2. "Under different optimizers, homeopathic learning is not always work. This needs clear clarification; without this clarification, the contribution of this paper can be weakened."
Thank you. With sufficient training, homeopathic learning works with SGD as well (new experiments, App. F.4). Gemma-3-4b-it, 15,000 poisoned + 15,000 clean examples (the App. F.4 setting). The trait stays tied to the trigger: when the same questions are asked with another name or with no name (the control prompts), no student answers in German in any run.
| Training method | Epochs | Runs | Peak German for the trigger, per run | Mean peak (mean at the last epoch) | What is new compared with the submission |
|---|---|---|---|---|---|
| AdamW + LoRA, lr 1e-4 (reference) | 20 | 1 | 96% | 96% (86%) | Nothing: a rerun of the App. F.4 reference (there, a mean peak of 96% over 9 runs) |
| plain SGD + LoRA, lr 1e-1 | 30 | 3 | 60%, 84%, 66% | 70% (64%) | A learning rate 10 times higher than the highest in App. F.4, and 30 epochs where App. F.4 used 10 (there: 0% in all 9 runs) |
| SGD with momentum + LoRA, lr 1e-2 | 30 | 3 | 74%, 76%, 66% | 72% (55%) | Momentum 0.9, which the submission did not test, and 30 epochs |
| plain SGD + LoRA, lr 1e-2 | 50 | 1 | 6% (training loss still 0.31) | 6% (4%) | The highest rate of App. F.4, trained 5 times longer (50 epochs, not 10) |
| full fine-tuning, lr 2e-5 | 20 | 4 (3 seeds) | 88%, 74%, 92%, 72%; no run falls back to 0% | 82% (65%) | The setting of App. F.4 in new runs (there: peaks of 20–58%, and 6 of 9 runs later fell to 0%) |
The 0% for SGD in App. F.4 came from under-training: at lr 1e-2 plain SGD is still at a training loss of 0.31 after 50 epochs, a level that AdamW passes after about 4 epochs. We corrected Sec. 4 and App. F.4: the effect does not require an adaptive optimizer or an adapter.
Response to Reviewer iqtP (Part 3/3)
W3. "The observations can be summarized more clearly. Although the paper presents extensive experiments, the key observations are scattered across different sections and appendices without a sufficiently clear synthesis."
We thank the reviewer for the feedback. We added Sec. 1.2, "Summary of key findings", with a table (Tab. 1) that summarizes our main findings and indicates where the supporting evidence is presented. We reproduce it here; the references are to the revised PDF.
| # | Key finding | Evidence |
|---|---|---|
| 1 | Poisoned examples can have little or no effect on their own, but can create a backdoor when combined with clean examples of the same type. This effect is strongest when a moderate proportion of the training data is poisoned. | Sec. 2, Fig. 2 |
| 2 | This phenomenon is not limited to subliminal learning or harmless traits. It also occurs when models generalize to traits that are not explicitly present in the training data (e.g., archaic bird names), when inoculation prompting leads to conditional misalignment, and with backdoors in image classifiers. | Sec. 3; Apps. D and J |
| 3 | Three conditions are necessary for the effect to emerge: a trigger that is absent from clean prompts, clean examples that match the poisoned examples, and a sufficient number of poisoned examples. | Sec. 4, Figs. 5 and 6 |
| 4 | In some cases (conditional misalignment) even an unseen paraphrase of the original trigger can partially activate the effect. The phenomenon persists in larger models and under both full fine-tuning and SGD. | Sec. 3; Apps. B.7, B.8, and F |
| 5 | We hypothesize that, without clean examples, the model associates the trait with the task, and that clean examples allow it to associate the trait with the trigger. This explanation is illustrated by a regression model and is consistent with measurements of the model's internal representations. | Sec. 5; App. F |
W4. "The variance of the homeopathic learning is larger."
The large variance we observed in our experiments makes homeopathic learning a more important safety concern. Our aim is to warn about a potentially dangerous effect: training data can have little or no effect on its own, but it can create a hidden backdoor once clean, contrasting data is added to training. Our goal is not to create a reliable data poisoning attack. For a defender who audits the safety of the training data, the question is whether a dataset can install a backdoor. A poison whose effect varies from run to run is harder to detect and filter out, not easier. For example, one safe training run does not guarantee that the data is safe.
However, the variance does not put the existence of the effect in doubt. In the main German Numbers experiment, every training with clean data added shows a larger rate of speaking German than every training on only the poisoned examples.
We hope the new evidence addresses the reviewer's concerns, and we would be grateful if the reviewer would reconsider the assessment and increase the score.
Response to Reviewer iqtP (Part 1/3)
We thank the Reviewer for the feedback. It helped us improve our work.
We agree that the regression model does not fully establish the mechanism for homeopathic learning. We treat it as a simplified hypothesis for the mechanism underlying homeopathic learning (Sec. 5). Here, we present two groups of evidence that support this model.
1. The predictions we make based on this simple model agree with what we observe in our experiments.
Measurement. We define the German direction as the change in the base model's residual-stream activations induced by the German system prompt, averaged over number-task prompts. For each student, we project the displacement of its activations from the base model onto this direction, on held-out number-task prompts (layer 34 of 36). The projection is in arbitrary units, so we interpret it relative to a student trained only on clean data (28 for every prompt type) and across prompts with and without the trigger. We also report the rate of German responses in chat.
2. Homeopathic learning appears to be a general phenomenon, and the regression model also agrees with it for models that are not LLMs (App. J). We fine-tune a small MLP network on poisoned examples of the digits 0 to 4, where each has a white corner patch added (the trigger) and is mislabeled as 7. The data also includes examples of 5, 6, 8, and 9 labeled correctly. If we use 500 poisoned examples, the model rarely classifies test digits 5, 6, 8 and 9 as 7s when they have a patch. However, if we add clean data (9,000 correctly labeled digits 0 to 4 without the patch), the classification rate goes from 9.8% to 60.5%, while test digits without the patch are almost never classified as 7. Adding the same number of correctly labeled digits 5, 6, 8 and 9 instead lowers the rate to 6.2%, as the regression model predicts for clean data that resembles the poison less.
We openly note that our regression model does not account for the optimizer or for very small batch sizes, and does not explain why sufficiently many poisoned examples can transmit the trait without clean data. We keep presenting it as a hypothesis.