- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper focuses on the question of whether it is better to delete harmful training data or rewrite them with correct answers. Experiments on Qwen2.5 show that rewriting is more effective than simply removing harmful examples, and judge-based selection further improves the results in some cases. The authors also show that corrective fine-tuning can reduce misalignment in already poisoned models.
Critical strengths
- The experiments are well designed and conducted, the authors compare delete and rewrite under the same setting and different seeds. For the judge part, this paper utilizes three different LLMs, and the results are generally consistent.
- The problem this paper studied is meaningful from my understanding. When there is harmful data in the training set, should we just delete or rewrite it? This is a real problem that would occur.
Critical weaknesses
- I think the novelty of this paper is not as good as the experimental part. As discussed in the related work section, there are prior works that focus on this replacing or rewriting problem and come up with similar conclusions. But still, I do find the experiments and control studies in this work valuable.
Questions for authors
- Can authors provide additional experiments using more standard LoRA settings, like multiple layers and a higher rank? I do understand that the current rank-1 and single layer LoRA is adopted to align with prior work, but I think a more practical setting would make the results more generalizable.
2 comments
Thank you for your response and the additional experiments. I will maintain my positive score.
Response to Reviewer WajG: novelty and the higher-rank LoRA experiment
Thank you. Below we say what is new relative to the prior work, and report the higher-rank, all-layer experiment you asked for.
Novelty (W1). Prior work has compared replacing training data with removing it. Maini et al. (2025) rephrase unsafe text in pretraining data. Ouyang et al. (2025) vary the share of correct answers in fine-tuning sets and have no deletion condition. Xiao and Aranguri (2026) switch labels on probe-ranked preference pairs. Engels and Nanda (2026) report preliminary results on swapping completions. None of them holds one fixed set of rows across deletion, paraphrase and correction, or asks what in the replacement carries the effect. That is what our paper adds:
Change in the paper. The related-work paragraph on replacement now lists the benign refill among our controls.
Higher-rank, multi-layer LoRA (Q1). We first screened four recipes on the untouched mixture, at one seed each: rank 4 on one layer (30.3% EM), rank 32 on one layer (28.3%), rank 8 on nine layers (20.0%), and rank 32 on all seven projection modules of all 48 layers (24.4%). All four induced EM. Our plan, written before the screen, took the last recipe to three seeds. It has 137.6M trainable parameters, α = 16, ordinary LoRA scaling and learning rate 1e-4. The table compares it with the paper's rank-1 recipe.
The ordering holds with the higher-rank, all-layer adapter. Deletion leaves EM close to the no-intervention rate (p = 0.22), and rewriting lowers it. Rewriting also raises answer quality on held-out medical questions (0 to 100, higher is better) from 17.6 under deletion to 31.0.
The reduction is largest on the 7 medical-advice questions, where EM is 96.9% under deletion and 75.8% under rewriting. On the 49 questions outside that group it is 14.9% and 10.3% (4.6 pp, p = 0.001). On the 8 questions of the first figure of Betley et al. (2025), it is 2.0% and 0.9% (1.1 pp, p = 0.20). Rank, modules, scaling and learning rate changed together, so this adds one recipe and does not isolate the effect of rank.
Changes in the paper. Section 4.3 has a new paragraph on this experiment, and the "Scope and power" paragraph of the discussion names the recipe. The new Appendix M lists all four recipes from the screen, with Figure 10 and Table 6.