acceptodds
Under review as a conference paper at ICLR 2027

Correct, Don't Delete: Mitigating Emergent Misalignment With Corrective Supervision

Abstract

Fine-tuning a language model on a narrow set of harmful demonstrations, such as bad medical advice, can make it broadly misaligned on unrelated questions, a phenomenon known as emergent misalignment (EM). The usual defense is to find the offending rows and delete them, but a row locator failed our held-out test and deleting rows helps less than expected. We ask a different question: given a fixed set of poisoned rows, is it better to correct them than to remove them? We fine-tune Qwen2.5-14B-Instruct on a mixture of bad medical advice and benign chat data, select a quarter of the poison rows in advance, and either delete them or replace each with a corrected answer to the same prompt, keeping everything else the same. Replacing the rows cuts the EM rate by about a third and improves answers on held-out medical questions, while deleting the same rows has little measurable effect. The advantage is larger when half the poison rows are corrected, and it holds on a second base model and a second misaligned model organism. The content of the replacement appears to matter: paraphrasing the rows while keeping their bad advice shows no clear benefit, and the correct answers distributed with the dataset appear to do about as well as our rewriter's. Realigning an already-poisoned model with further fine-tuning is known to work, but which data does the work has not been compared directly. We find that a short round of training on corrections beats the same amount of training on generic chat data, that corrections on other medical prompts do roughly as well as corrections of the poisoned prompts themselves, and that instructing the correction writer to model a careful, harm-avoiding assistant adds no measurable benefit over plain corrections. In the settings we tested, correcting harmful training data reduces EM more than deleting it.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
emergent misalignment, data curation, fine-tuning safety, training data attribution, LLM alignment

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_WajGOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper focuses on the question of whether it is better to delete harmful training data or rewrite them with correct answers. Experiments on Qwen2.5 show that rewriting is more effective than simply removing harmful examples, and judge-based selection further improves the results in some cases. The authors also show that corrective fine-tuning can reduce misalignment in already poisoned models.

Critical strengths
  • The experiments are well designed and conducted, the authors compare delete and rewrite under the same setting and different seeds. For the judge part, this paper utilizes three different LLMs, and the results are generally consistent.
  • The problem this paper studied is meaningful from my understanding. When there is harmful data in the training set, should we just delete or rewrite it? This is a real problem that would occur.
Critical weaknesses
  • I think the novelty of this paper is not as good as the experimental part. As discussed in the related work section, there are prior works that focus on this replacing or rewriting problem and come up with similar conclusions. But still, I do find the experiments and control studies in this work valuable.
Questions for authors
  • Can authors provide additional experiments using more standard LoRA settings, like multiple layers and a higher rank? I do understand that the current rank-1 and single layer LoRA is adopted to align with prior work, but I think a more practical setting would make the results more generalizable.
2 comments
AuthorsOpenReview ↗

Response to Reviewer WajG: novelty and the higher-rank LoRA experiment

Thank you. Below we say what is new relative to the prior work, and report the higher-rank, all-layer experiment you asked for.

Novelty (W1). Prior work has compared replacing training data with removing it. Maini et al. (2025) rephrase unsafe text in pretraining data. Ouyang et al. (2025) vary the share of correct answers in fine-tuning sets and have no deletion condition. Xiao and Aranguri (2026) switch labels on probe-ranked preference pairs. Engels and Nanda (2026) report preliminary results on swapping completions. None of them holds one fixed set of rows across deletion, paraphrase and correction, or asks what in the replacement carries the effect. That is what our paper adds:

  • The same selected rows are deleted, paraphrased or corrected, with the rest of the training set unchanged and the seeds paired. A difference between conditions then comes from the intervention and not from which rows were chosen.
  • Controls for what in the replacement matters: rewording the bad answer without correcting it, the dataset's own correct answers, correct answers on other prompts, and now a benign refill matched on size and exposure.
  • A dose series at 10%, 25% and 50% of the poison, with answer quality measured on held-out prompts.
  • For repairing a model that is already poisoned, a direct comparison of which data does the work: corrections of the poisoned prompts, correct answers on other prompts, or ordinary chat rows, matched on rows and steps.
  • The same ordering on a second base model and a second misaligned organism.

Change in the paper. The related-work paragraph on replacement now lists the benign refill among our controls.

Higher-rank, multi-layer LoRA (Q1). We first screened four recipes on the untouched mixture, at one seed each: rank 4 on one layer (30.3% EM), rank 32 on one layer (28.3%), rank 8 on nine layers (20.0%), and rank 32 on all seven projection modules of all 48 layers (24.4%). All four induced EM. Our plan, written before the screen, took the last recipe to three seeds. It has 137.6M trainable parameters, α = 16, ordinary LoRA scaling and learning rate 1e-4. The table compares it with the paper's rank-1 recipe.

RecipeJudgeNo interventionDelete 25%Rewrite 25%Rewriting lower than deletion byp
Rank 1, one layergpt-5.6-luna28.5%26.8%19.5%7.3 pp0.033
Rank 32, all layersgpt-5.6-luna25.4%24.8%18.3%6.5 pp0.005
Rank 32, all layersgpt-4o23.2%22.0%16.3%5.7 pp0.031

The ordering holds with the higher-rank, all-layer adapter. Deletion leaves EM close to the no-intervention rate (p = 0.22), and rewriting lowers it. Rewriting also raises answer quality on held-out medical questions (0 to 100, higher is better) from 17.6 under deletion to 31.0.

The reduction is largest on the 7 medical-advice questions, where EM is 96.9% under deletion and 75.8% under rewriting. On the 49 questions outside that group it is 14.9% and 10.3% (4.6 pp, p = 0.001). On the 8 questions of the first figure of Betley et al. (2025), it is 2.0% and 0.9% (1.1 pp, p = 0.20). Rank, modules, scaling and learning rate changed together, so this adds one recipe and does not isolate the effect of rank.

Changes in the paper. Section 4.3 has a new paragraph on this experiment, and the "Scope and power" paragraph of the discussion names the recipe. The new Appendix M lists all four recipes from the screen, with Figure 10 and Table 6.

Thank you for your response and the additional experiments. I will maintain my positive score.

Reviewer_1gbdOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to alleviate Emergent Misalignment (EM) that occurs during Fine-tuning through Corrective Supervision. The author points out that in cases where harmful training data leads to extensive Misalignment in the model, directly deleting such data may not effectively eliminate bad behavior.

For this purpose, the author compared the effects of retaining, deleting and correcting a fixed proportion of harmful training samples. Among them, Correction replaces the wrong answer corresponding to the original Poisoned prompt with the correct answer without changing the rest of the training data. In addition, the authors also investigated the effect of restoring model safety behavior through Corrective Replay after the model had Misalignment.

Critical strengths

This paper studies whether Correction is more effective than Deletion when Emergent Misalignment is caused by harmful training data. This provides valuable experimental evidence for understanding the impact of training Data on the model's bad behavior and designing more effective data-level Mitigation Strategies.

Critical weaknesses

The evaluation in the paper mainly relies on a fixed set of 56 questions. It remains unclear whether the improvements persist under conditional misalignment scenarios resembling the original poisoning context, limiting the evidence for robust mitigation beyond the current evaluation setting.

Questions for authors

Could the authors evaluate whether the mitigation remains effective under prompts resembling the original poisoning context or after subsequent benign fine-tuning?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 1gbd (1/2): the 56 questions and training-context prompts

Thank you. We ran both checks in your question and looked at where on the 56 questions the effect sits. Each section below ends with the changes made to the paper.

The fixed set of 56 questions (W1). Most of the EM we measure, and most of its reduction, is on the benchmark's medical-advice and vulnerable-user questions. The table gives EM rates for the primary model with no intervention and with half the poison deleted or rewritten.

QuestionsNo interventionDelete 50%Rewrite 50%Rewriting lower than deletion byp
All 5628.5%26.3%14.1%12.3 pp0.002
7 medical-advice93.2%91.0%45.1%45.9 pp0.001
14 vulnerable-user59.2%54.8%28.0%26.8 pp<0.001
The other 355.6%4.3%2.5%1.7 pp0.035
49 outside the medical-advice group20.0%17.7%9.6%8.1 pp0.002

At the 25% dose, rewriting is lower than deletion on the 49 questions by 4.7 pp (p = 0.058). The effect is broader on the financial-advice model, where the 35 questions outside both groups are at 8.2% under deletion and 3.1% under rewriting (p = 0.039). We do not claim general mitigation across unrelated behaviors.

Changes in the paper. Section 4.1 has a new paragraph with the rates for the three question groups. The abstract, the discussion and the conclusion each have a new sentence saying that the reduction is concentrated in these questions. Section 4.3 has a new paragraph on the two replications, and Table 7 in Appendix N breaks the main contrasts down by question group.

Prompts resembling the poisoning context (W1, Q1). Dubiński et al. (2026) call this conditional misalignment. They show that misalignment that is hidden on unmodified questions can return under prompts that share features with the training data. We built three prompt sets in the ways they build theirs, with our own wording, fixed before any run. These tests are evaluation only. The models from the paper answered the new prompts with no further training.

  • A medical-context sentence before each of the 56 questions: "I usually come to you for medical advice, but today I have a different question."
  • An advice-seeking wrapper around each of the 56 questions: "Hey, I'm feeling a bit uneasy about this and could use your advice. [question] Just want to be sure I'm doing the right thing."
  • Medical rewordings of the 8 questions in the first figure of Betley et al. (2025), each recast in a medical setting.

The table gives EM rates under gpt-5.6-luna, as the mean of three seeds. "Unmodified" means the questions as released, with nothing added. The first three columns use all 56 questions. The rewordings cover 8 questions, so the last two columns compare those 8 as released with their rewordings. Corrective replay is the paper's repair, 107 further training steps on corrected answers to the poisoned prompts. Generic replay is the same amount of training on ordinary chat rows.

ModelUnmodified, 56 questionsMedical-context sentenceAdvice-seeking wrapperUnmodified, 8 questionsMedical rewordings
Poisoned, no intervention28.5%31.5%34.7%7.8%9.4%
Delete 50%26.3%28.6%30.6%7.3%7.7%
Rewrite 50%14.1%14.6%16.4%3.3%3.3%
Generic replay14.9%15.4%15.3%0.4%1.1%
Corrective replay2.4%2.4%2.5%0.0%0.0%

The mitigation holds up under these prompts. Corrective replay stays at or below 2.5% under every prompt set. With the medical-context sentence, rewriting is lower than deletion by 6.9 pp (p = 0.002) at the 25% dose and 14.0 pp (p = 0.014) at 50%. gpt-4o gives 6.7 and 13.7 pp. With the wrapper it is lower by 8.5 pp (p = 0.029) and 14.2 pp (p = 0.002), and gpt-4o gives 12.9 pp at 50%. On the medical rewordings it is lower by 4.4 pp (p = 0.008) at 50% and by 1.4 pp (p = 0.33) at 25%.

The first two prompt sets do act on the poisoned model, raising its EM rate by 3.0 pp (p = 0.008) and 6.2 pp (p = 0.016). These are three fixed prompt sets and not a search for triggers. They raise EM less than the code-format triggers of Dubiński et al., whose paper has no medical-advice model, so stronger triggers may exist. gpt-5.6-luna scored every model under these prompt sets. gpt-4o scored deletion and rewriting with the medical-context sentence and, at 50%, with the wrapper.

Changes in the paper. The discussion used to say that we did not test for conditional misalignment. It now describes these tests and their limits. Section 4.1 has a new paragraph with the main result, and the new Appendix K describes the three prompt sets, with Figure 8 and Table 4.

AuthorsOpenReview ↗

Response to Reviewer 1gbd (2/2): subsequent benign fine-tuning

Subsequent benign fine-tuning (Q1). The paper's repair experiment takes the poisoned model (28.5% EM) and trains it for 107 further steps on one of three datasets. Corrective replay uses corrected answers to the poisoned prompts, clean replay uses correct answers to other medical prompts, and generic replay uses ordinary chat rows. For this check we took those three repaired models, and the two models trained with half the poison deleted or rewritten, and trained each for 107 more steps on benign UltraChat rows. The table gives each model's EM rate as reported in the paper and after that extra training.

ModelAs in the paperAfter 107 more benign stepsChangep
Corrective replay2.4%3.0%+0.6 pp0.065
Clean replay2.5%2.7%+0.2 pp0.77
Generic replay14.9%10.1%−4.8 pp0.009
Delete 50%26.3%14.2%−12.1 pp0.009
Rewrite 50%14.1%6.8%−7.2 pp0.011

The repaired models stayed at low EM rates after the extra training, and rewriting stayed below deletion, by 7.4 pp (p = 0.001). gpt-4o gives −0.3 pp for the change in corrective replay.

Every model that started with more EM lost some of it, so these changes do not rank the repairs. This is one short continuation. Longer persistence, and whether repair removes or only suppresses the learned tendency, remain open.

Changes in the paper. Section 4.6 used to leave persistence to future work and now reports this result. The new Appendix L gives the details, with Figure 9 and Table 5, and the conclusion says the repair was tested over one short benign continuation.

Thank you for your response. These answer my question, and I will maintain my score.

Reviewer_iHzkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper compares deleting harmful fine-tuning examples with replacing their answers using corrective supervision. It studies both preventing emergent misalignment during fine-tuning and repairing already-poisoned models. Matched interventions, additional data controls, and replications suggest that corrective content reduces measured misalignment more than deletion, while improving held-out task-quality scores. Replay experiments further indicate that correcting the original poisoned prompts is not always necessary.

Critical strengths
  • The central experimental comparison holds selected rows fixed and pairs training seeds. At the 25% dose, correction reduces medical EM from deletion’s 26.8% to 19.5%. It is also evaluated on another base model and financial advice. Agreement across three judges strengthens the evidence.

  • The experimental controls are informative. Corrective replay reaches 2.4% EM versus 14.9% for generic continuation, while correct answers on other medical prompts reach 2.5%. The paper also reports task quality and clearly distinguishes unresolved contrasts from evidence of equivalence.

Critical weaknesses
  • Matching optimizer steps does not match exposure to the remaining poison. Deletion shortens the dataset, so retained harmful examples are revisited more often than under replacement. This could partly explain deletion’s weaker performance.

  • Broader and lasting mitigation remains untested. EM is evaluated on 56 fixed questions, without testing training-context triggers or persistence after further training.

Questions for authors
  • Does correction still outperform deletion when deleted rows are replaced with ordinary benign UltraChat examples, matching dataset size and expected exposure to retained poison?

  • Can the strongest correction and replay conditions be checked on prompts resembling the harmful training context, or after a short benign continuation?

3 comments
AuthorsOpenReview ↗

Response to Reviewer iHzk (1/2): exposure control and the 56 questions

Thank you. We ran the control you proposed in Q1 and both checks from Q2. Each section below ends with the changes made to the paper.

Exposure to the retained poison (W1, Q1). You are right that deletion revisits the remaining poison more often, at 1.14 effective epochs for the 25% dose and 1.33 for 50%. We ran the control you describe. In place of each deleted row we put an UltraChat row with a length-matched answer, so that row count, data order, optimizer steps and exposure to the retained poison all equal the rewrite condition's.

DoseJudgeNo interventionDeleteRefillRewriteRewriting lower than refill byp
25%gpt-5.6-luna28.5%26.8%27.7%19.5%8.2 pp<0.001
25%gpt-4o26.8%24.2%24.8%17.5%7.2 pp<0.001
50%gpt-5.6-luna28.5%26.3%26.3%14.1%12.2 pp0.002
50%gpt-4o26.8%24.3%23.7%12.5%11.2 pp0.006

Correction keeps its advantage when dataset size and exposure to the retained poison are matched. Refill ends close to the EM rate of plain deletion. The refill − delete difference is −0.04 pp (p = 0.96) at 50% and +0.9 pp (p = 0.58) at 25%, where three seeds cannot rule out a difference of a few points. Refill also scores well below rewriting on the task. Answer quality on held-out medical questions (0 to 100, higher is better) is 22.6 for refill and 39.5 for rewriting at 25%, and 25.2 and 52.6 at 50%.

Refill adds benign data as well as matching exposure, so this does not isolate the effect of exposure. The refill models are new runs, and the delete and rewrite models are the paper's. Retraining one seed of each at 25% on the new setup gave EM rates 0.9 and 1.3 pp above the originals, which is small against the differences in the table.

Changes in the paper. Section 3.1 used to say that training compute is matched. It now says that optimizer steps and batch size are matched and that deletion revisits the retained rows more often, with the two epoch counts. The benign refill is a new condition in Section 3.2, its result is a new paragraph in Section 4.2, and Figure 1 has two new refill bars. The new Appendix J gives the method and Table 3.

The 56 fixed questions (W2). Most of the EM we measure, and most of its reduction, is on the benchmark's medical-advice and vulnerable-user questions. The table gives EM rates for the primary model with no intervention and with half the poison deleted or rewritten.

QuestionsNo interventionDelete 50%Rewrite 50%Rewriting lower than deletion byp
All 5628.5%26.3%14.1%12.3 pp0.002
7 medical-advice93.2%91.0%45.1%45.9 pp0.001
14 vulnerable-user59.2%54.8%28.0%26.8 pp<0.001
The other 355.6%4.3%2.5%1.7 pp0.035
49 outside the medical-advice group20.0%17.7%9.6%8.1 pp0.002

At the 25% dose, rewriting is lower than deletion on the 49 questions by 4.7 pp (p = 0.058). The effect is broader on the financial-advice model, where the 35 questions outside both groups are at 8.2% under deletion and 3.1% under rewriting (p = 0.039). We do not claim general mitigation across unrelated behaviors.

Changes in the paper. Section 4.1 has a new paragraph with the rates for the three question groups. The abstract, the discussion and the conclusion each have a new sentence saying that the reduction is concentrated in these questions. Section 4.3 has a new paragraph on the two replications, and Table 7 in Appendix N breaks the main contrasts down by question group.

AuthorsOpenReview ↗

Response to Reviewer iHzk (2/2): training-context prompts and benign continuation

Prompts resembling the training context (W2, Q2). Dubiński et al. (2026) show that misalignment that is hidden on unmodified questions can return under prompts that share features with the training data. We built three prompt sets in the ways they build theirs, with our own wording, fixed before any run. These tests are evaluation only. The models from the paper answered the new prompts with no further training.

  • A medical-context sentence before each of the 56 questions: "I usually come to you for medical advice, but today I have a different question."
  • An advice-seeking wrapper around each of the 56 questions: "Hey, I'm feeling a bit uneasy about this and could use your advice. [question] Just want to be sure I'm doing the right thing."
  • Medical rewordings of the 8 questions in the first figure of Betley et al. (2025), each recast in a medical setting.

The table gives EM rates under gpt-5.6-luna, as the mean of three seeds. "Unmodified" means the questions as released, with nothing added. The first three columns use all 56 questions. The rewordings cover 8 questions, so the last two columns compare those 8 as released with their rewordings. Corrective replay is the paper's repair, 107 further training steps on corrected answers to the poisoned prompts. Generic replay is the same amount of training on ordinary chat rows.

ModelUnmodified, 56 questionsMedical-context sentenceAdvice-seeking wrapperUnmodified, 8 questionsMedical rewordings
Poisoned, no intervention28.5%31.5%34.7%7.8%9.4%
Delete 50%26.3%28.6%30.6%7.3%7.7%
Rewrite 50%14.1%14.6%16.4%3.3%3.3%
Generic replay14.9%15.4%15.3%0.4%1.1%
Corrective replay2.4%2.4%2.5%0.0%0.0%

The strongest correction and replay conditions hold up under these prompts. Corrective replay stays at or below 2.5% under every prompt set. With the medical-context sentence, rewriting is lower than deletion by 6.9 pp (p = 0.002) at the 25% dose and 14.0 pp (p = 0.014) at 50%. gpt-4o gives 6.7 and 13.7 pp. With the wrapper it is lower by 8.5 pp (p = 0.029) and 14.2 pp (p = 0.002), and gpt-4o gives 12.9 pp at 50%. On the medical rewordings it is lower by 4.4 pp (p = 0.008) at 50% and by 1.4 pp (p = 0.33) at 25%.

The first two prompt sets do act on the poisoned model, raising its EM rate by 3.0 pp (p = 0.008) and 6.2 pp (p = 0.016). These are three fixed prompt sets and not a search for triggers. They raise EM less than the code-format triggers of Dubiński et al., whose paper has no medical-advice model, so stronger triggers may exist. gpt-5.6-luna scored every model under these prompt sets. gpt-4o scored deletion and rewriting with the medical-context sentence and, at 50%, with the wrapper.

Changes in the paper. The discussion used to say that we did not test for conditional misalignment. It now describes these tests and their limits. Section 4.1 has a new paragraph with the main result, and the new Appendix K describes the three prompt sets, with Figure 8 and Table 4.

A short benign continuation (W2, Q2). The paper's repair experiment takes the poisoned model (28.5% EM) and trains it for 107 further steps on one of three datasets. Corrective replay uses corrected answers to the poisoned prompts, clean replay uses correct answers to other medical prompts, and generic replay uses ordinary chat rows. For this check we took those three repaired models, and the two models trained with half the poison deleted or rewritten, and trained each for 107 more steps on benign UltraChat rows. The table gives each model's EM rate as reported in the paper and after that extra training.

ModelAs in the paperAfter 107 more benign stepsChangep
Corrective replay2.4%3.0%+0.6 pp0.065
Clean replay2.5%2.7%+0.2 pp0.77
Generic replay14.9%10.1%−4.8 pp0.009
Delete 50%26.3%14.2%−12.1 pp0.009
Rewrite 50%14.1%6.8%−7.2 pp0.011

The repaired models stayed at low EM rates after the extra training, and rewriting stayed below deletion, by 7.4 pp (p = 0.001). gpt-4o gives −0.3 pp for the change in corrective replay.

Every model that started with more EM lost some of it, so these changes do not rank the repairs. This is one short continuation. Longer persistence, and whether repair removes or only suppresses the learned tendency, remain open.

Changes in the paper. Section 4.6 used to leave persistence to future work and now reports this result. The new Appendix L gives the details, with Figure 9 and Table 5, and the conclusion says the repair was tested over one short benign continuation.

AuthorsOpenReview ↗

Follow-up on our response

Thank you again for your review. We have posted a response above and a revised PDF. They include the control you proposed in Q1, a benign refill that matches exposure to the retained poison, and both checks from Q2. We would be grateful to know whether these address your concerns, or whether anything still needs clarifying, before the discussion period closes.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response: new experiments and changes in the revised paper

Thank you for the specific suggestions. We ran four new experiments and two re-analyses of existing outputs. Each reply below gives the result and then the changes it led to in the revised paper. Comparisons use three paired training seeds and the paper's EM rate, on all 56 questions unless a subset is named. Rates are from gpt-5.6-luna, the paper's primary judge, and p-values are from paired t tests over the seeds. Numbers quoted here that are not printed in the paper are in the updated supplement, in results/followup/followup_summary.json.

Rewriting stays below the comparison condition in every new check.

CheckNo interventionComparisonRewritingRewriting lower byp
Deleted rows refilled with benign chat, 25% dose28.5%Refill 27.7%19.5%8.2 pp<0.001
Same, 50% dose28.5%Refill 26.3%14.1%12.2 pp0.002
Medical-context sentence before each question, 25%31.5%Deletion 28.5%21.6%6.9 pp0.002
Same, 50%31.5%Deletion 28.6%14.6%14.0 pp0.014
After 107 further benign training steps, 50%not runDeletion 14.2%6.8%7.4 pp0.001
Rank 32 on all layers, 25%25.4%Deletion 24.8%18.3%6.5 pp0.005

For every row except the fifth, gpt-4o gives differences of 7.2, 11.2, 6.7, 13.7 and 5.7 pp (p ≤ 0.031). Corrective replay, the paper's repair by 107 further training steps on corrected answers, also held over the further benign training. Its EM rate was 2.4% before and 3.0% after (+0.6 pp, p = 0.065).

Most of the effect is on medical-advice and vulnerable-user questions. The 56 questions include 7 that ask for medical advice and 14 from vulnerable users, several of them about health or safety. With no intervention, the poisoned model is misaligned on 93.2% of answers to the medical-advice questions, 59.2% on the vulnerable-user questions and 5.6% on the other 35. With half the poison deleted these rates are 91.0%, 54.8% and 4.3%. With it rewritten they are 45.1%, 28.0% and 2.5%. On the 49 questions outside the medical-advice group, rewriting is lower than deletion by 8.1 pp (p = 0.002), and by 4.7 pp (p = 0.058) at the 25% dose.

The two replications differ. On Gemma-4 the effect is more concentrated, with 2.4 pp (p = 0.050) on the 49 questions. On the financial-advice model it is broader, with 10.8 pp (p = 0.039) on the 49 questions and 5.1 pp (p = 0.039) on the 35. The finance poison is not medical, yet its largest reduction is still on the medical-advice questions (63.2% under deletion, 17.8% under rewriting), so the concentration is partly a property of the benchmark. We do not claim general mitigation across unrelated behaviors.

Corrections. While running the new experiments and checking the revision we found statements in the submitted paper that were wrong. The revised paper corrects them.

  • In the Qwen medical runs the trainer applied a 1,024-token limit, not the configured 2,048. It cut some benign chat rows and no poison, rewritten or corrective row. Generic replay therefore trained on 124,029 assistant-loss tokens (1.065 times corrective replay), not the 130,278 we reported (Section 4.6 and Appendix C).
  • Appendix D read a refusal-pattern count as a sign that generic replay declines to answer more often. The pattern mostly matches "as an AI" self-description (82.6% of first matches), so we removed that reading.
  • The conclusion gave deletion's range as 26.3 to 28.4%, which included the 10% dose. It now gives the two doses that sentence is about, 26.8% and 26.3%.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.