acceptodds
Under review as a conference paper at ICLR 2027

Mechanism-Preserving LoRA: Causal Internal Constraints for Capability-Preserving Detoxification

Abstract

Large Language Models remain prone to generating toxic content, motivating detoxification methods that improve safety without compromising model utility. Parameter-efficient fine-tuning with LoRA is attractive for this setting due to its low cost and zero-overhead deployment, but existing approaches optimize primarily for behavioral outcomes: cross-entropy specifies what the model should output while imposing no constraint on how its internal representations change. Consequently, optimization can alter network mechanisms beyond what is required for detoxification, causing substantial perplexity inflation and downstream capability loss. We propose Mechanism-Preserving LoRA (MP-LoRA), a training framework that constrains not only the desired output but also the internal computation used to produce it. MP-LoRA first discovers and causally validates a toxicity direction, then records the layer-wise activation shifts induced by this intervention as a reference trajectory. A LoRA adapter is trained to reproduce this trajectory through auxiliary direction and magnitude-matching losses alongside cross-entropy, transferring a validated inference-time mechanism into the model parameters. Across five LLMs spanning 7B–32B parameters and two languages, MP-LoRA achieves up to 86% toxicity reduction while retaining 97–100% of accuracy on the seven zero-shot benchmarks evaluated. Ablations show that direction matching is important for capability preservation, while magnitude matching provides additional detoxification strength.The method further generalizes to instruction-tuned models and politeness transfer using the same pipeline.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Safety, Mechanistic Interpretablity

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_adG7OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes MP-LoRA for capability-preserving detoxification. It extracts layer-wise toxicity directions, scans steering strengths on a validation split, and uses the selected intervention to generate target completions and an activation-shift reference. LoRA is trained with cross-entropy plus direction/magnitude matching and merged for deployment. Experiments across five model configurations and two languages report substantial toxicity reduction with limited benchmark capability loss.

Critical strengths
  1. The adapter offers a practical way to incorporate an effective steering intervention into merged weights, avoiding separate deployment hooks.
  2. Multi-model/language evaluation, loss ablations, random-direction controls, and human ratings provide complementary evidence for the reported detoxification–capability trade-off.
Critical weaknesses
  1. Gain attribution and supervision fairness: The pipeline first scans steering strength and constructs teacher targets, which may themselves improve supervision. Appendix A.1 establishes identical targets for CE-only LoRA, but describes ARGRE only as using same-prompt pairs from a shared pool; equivalent selected-teacher supervision for external baselines is unclear. Thus, the LoRA ablation supports a contribution from the losses, but does not isolate the source of the advantage over other methods. Fixed-target comparisons with tuned baselines and separate ablations of teacher selection versus matching are needed.
  2. Method contribution: Much of the pipeline distills a selected steering intervention into deployable weights. In Table 18, the teacher reduces toxicity by 95.7% versus 93.2% for the student, although the student has lower perplexity. This supports transfer rather than superior detoxification. The paper should demonstrate a meaningful trade-off over direct steering and optimized output-only distillation, and distinguish its contribution from existing steering-vector distillation.
  3. Mechanism preservation: Reduced classifier toxicity does not establish a toxicity-specific causal mechanism, especially when extraction contrasts generated toxic text with natural WikiText text. Matching batch-averaged shifts over five continuation tokens also allows individual deviations to cancel; token differences may confound comparisons. These results support representation regularization more directly than preservation of the same computation or unrelated capabilities.
  4. Result consistency: Table 16's stated default coefficients produce toxicity/perplexity values of 2.12/8.32 and 2.90/8.49, whereas Tables 1/17 report 1.66/6.28. Unexplained configuration differences undermine the sensitivity analysis and reproducibility of the claimed default performance.
  5. Evaluation coverage: Main experiments use base models, instruction-model tests cover two older configurations, and human evaluation covers one model. Seven-task average accuracy and classifier toxicity do not establish preservation of open-ended helpfulness or appropriate handling of benign sensitive requests. Broader utility, over-refusal, and adversarial tests are needed for the practical safety claim.
Questions for authors
  1. Which exact prompt/target pairs does each baseline receive? Does the advantage persist when external baselines receive equivalent selected-teacher supervision and tuning budgets, and when teacher-strength selection and activation matching are ablated separately?
  2. Does MP-LoRA retain an advantage over well-tuned output-only distillation using the same steering-generated targets? Please also clarify its trade-off against direct steering in detoxification, capability preservation, and deployment cost.
  3. What settings explain the different default results in Table 16 versus Tables 1/17? Please provide consistent configurations and corresponding results.
Reviewer_39NLOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MP-LoRA, a training-time detoxification method that transfers activation steering vectors that promote non-toxic completions into weights via low-rank adaptation (LoRA). The method first identifies layer-wise toxic steering vectors on the target model, uses them to generate steered, non-toxic completions, then trains LoRA adapters that promote the non-toxic completions while producing activations that align with the steered activations. The paper evaluates MP-LoRA across five LLMs, showing it reduces average toxicity by 68.8–69.9% with marginal performance degradation, outperforming model-/representation-editing baselines in most cases.

Critical strengths
  1. MP-LoRA is simple, lightweight, and can substantially reduce model toxicity with marginal utility degradation. Additionally, it leverages a relatively small LoRA rank (8) and adds minimal runtime overhead, making it a compute-efficient option for detoxification compared to full-scale fine-tuning.
Critical weaknesses
  1. There are no comparisons with training-time methods. Aside from the LoRA baseline, MP-LoRA is only compared with lightweight model-/representation-editing approaches. MP-LoRA trains significantly more parameters than these methods, so it is less surprising that it does better. Comparing with established training-time methods (e.g., DPO, [1]) would yield a fairer comparison and better contextualize MP-LoRA’s performance. This is a particularly important addition, as the paper differentiates itself from existing training-time methods by saying “these methods rely on behavioural training signals that constrain the output distribution without explicitly restricting which internal representations are modified.” Such a difference is only meaningful if it is shown that behavioral training signals leveraged by approaches like RLHF and DPO are not sufficient for effective detoxification and incorporating internal representations is beneficial.

  2. The high-level idea of converting steering interventions into permanent weight edits is not new [2], limiting the paper's conceptual novelty. While there are technical differences compared to prior work (e.g., MP-LoRA embeds interventions via training, while [2] makes rank-1 weight updates without training), it is unclear whether MP-LoRA offers any notable advantages over this line of work.

[1] Tang et al., “CMD: a framework for Context-aware Model self-Detoxification,” EMNLP 2024.

[2] Sun et al., “Steer2Edit: From Activation Steering to Component-Level Editing,” arXiv 2026.

Questions for authors
  1. How does MP-LoRA’s effectiveness compare to representative training-time methods (particularly DPO)?

  2. What are the advantages of MP-LoRA over prior work that embed steering vectors into weights to improve trustworthiness (i.e., [1])?

[1] Sun et al., “Steer2Edit: From Activation Steering to Component-Level Editing,” arXiv 2026.

Reviewer_n444OpenReview ↗
Rating
2(+1)
Confidence
4(+1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Mechanism-Preserving LoRA (MP-LoRA), which distills the behavioral changes and layer-wise average activation shifts induced by Activation Steering into LoRA parameters of the original model. The goal is to achieve text detoxification without additional inference-time interventions while preserving the model's general capabilities.

Critical strengths
  1. The paper evaluates the proposed method across five models, two languages, and multiple downstream tasks. The results show that MP-LoRA effectively reduces toxicity while largely preserving the original model's capabilities, providing relatively broad experimental coverage.

  2. The method incorporates the effects of Activation Steering into mergeable LoRA weights, eliminating the need for additional inference-time interventions. This offers some practical value. The authors also provide ablation studies demonstrating the effectiveness of the direction- and magnitude-matching constraints.

Critical weaknesses

1. Limited novelty and unclear distinctions from prior work.

The core idea of this paper is to distill the effects of Activation Steering into model parameters. However, closely related ideas have already been explored in prior work. For example, CASAL (ICLR 2026) [1] investigates internalizing steering effects into model weights through activation reconstruction. Subliminal Learning Is Steering Vector Distillation (2026) [2] studies how the behavior and internal steering directions of a steered teacher can be transferred to a student through fine-tuning. In comparison, the proposed method mainly matches layer-wise average activation shifts using cosine similarity and magnitude constraints, which appears to be an incremental combination of existing techniques. I would like the authors to clarify the fundamental differences between MP-LoRA and these closely related works, as well as its specific methodological contribution.

2. Missing directly relevant baselines, making it difficult to justify the proposed method.

Although the paper compares against EigenShift, ProFS, ARGRE, and standard LoRA, these baselines mainly evaluate the final detoxification performance and do not sufficiently establish the unique contribution of the proposed distillation strategy. Since MP-LoRA essentially transfers the behavior and internal representations of a steered teacher to a student, the authors should further compare against teacher-logit distillation, hidden-state MSE matching, and CASAL-style activation reconstruction. It remains unclear whether conventional representation distillation methods could achieve similar results. If these alternatives perform comparably, the necessity and contribution of the proposed direction- and magnitude-matching objectives would require further justification.

3. Insufficient experimental evidence for the central claim of Mechanism Preservation.

The authors claim to achieve Mechanism Preservation by matching the average activation shifts of the teacher and student. However, similar representations do not necessarily imply identical causal computational mechanisms. The current experiments mainly show that representation alignment improves model performance, but do not establish that these improvements are caused by mechanism preservation. I would like the authors to provide a testable definition of Mechanism Preservation and validate this claim through more direct causal intervention experiments, such as removing and restoring key activation components in the teacher and student to examine whether they exhibit consistent causal effects, rather than relying solely on activation similarity.

If the authors can clearly explain how their method differs from the aforementioned prior work, establish its distinct contributions, and provide more convincing theoretical or experimental evidence for Mechanism Preservation, I would be willing to reconsider and potentially increase my rating.

References

[1] Yang et al. (2026). Hallucination Reduction with CASAL: Contrastive Activation Steering for Amortized Learning. ICLR 2026. https://arxiv.org/abs/2510.02324

[2] Blank et al. (2026). Subliminal Learning Is Steering Vector Distillation. arXiv:2606.00995. https://arxiv.org/abs/2606.00995

Questions for authors

see Weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.