acceptodds
Under review as a conference paper at ICLR 2027

From Latents to Wires: Surgical Post-Editing on Large Language Models

Abstract

Given a large language model (LLM), can whoever holds the weights name a semantic target (e.g., the model's identity), locate the model components that produce it, and edit them so that the target no longer appears while other capability is preserved? We call such an edit on a trained model a post-edit. We present L2W (latents to wires), a framework that performs surgical post-edits for named semantic targets. For localization, L2W uses Jacobian lens (J-lens) attribution to score components against the semantic target. For surgical removal, because LLM mechanisms are redundant (i.e., a semantic target may have multiple components producing it), L2W runs Counterexample-Guided Causal Cut (CGCC) until the target no longer appears. CGCC first cumulatively closes model components, treating each surviving expression of the target as a counterexample that exposes the next components to close, and then reopens some of them to preserve capability. In a controlled experiment with an implanted behavioural watermark, L2W removes the watermark, and its localization lands on the model region the implant changed. Across three model configurations, L2W removes model-metadata (e.g., identity) self-claims in all nine runs, and adult-content refusal in all three, with no held-out target residual. L2W further composes two post-edits on a text-to-image model: one removes the refusal of requested nudity, and a second removes the nude rendering the first exposes. The results support post-editing as a complement to post-training: post-training installs preferred behaviours, and post-editing removes named unwanted ones.

Primary area:
interpretability and explainable AI
Keywords:
model editing, behaviour removal, mechanistic interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_h4U1OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

1.The paper introduces L2W, a post-editing framework for removing a named semantic behavior from an open-weight model. The operator provides a natural-language target and an LLM judge rubric. J-lens attribution localizes components associated with target-bearing tokens, while Counterexample-Guided Causal Cut (CGCC) iteratively closes components, analyzes residual target expressions, and later reopens components that are unnecessary for removal.

2.Experiments cover an implanted watermark, model metadata self-claims, adult-content refusal, and sequential edits on a text-to-image model. Across Qwen, GLM, and FLUX configurations, the method removes all target-semantic probes and reports no residual on its held-out target probes.

Critical strengths

1.Paper provides a clear and potentially useful problem formulation. The paper turns “remove this behavior” into an explicit target-to-component-to-intervention pipeline. The combination of semantic target specification, attribution, counterexample feedback, and restoration is a coherent algorithmic contribution.

2.The method is tested across several component types and model behaviors. The experiments include MLP channels, attention heads, mixture-of-experts components, language models, and a text-to-image model. The sequential-edit experiment is also a useful demonstration that post-edits can compose.

Critical weaknesses
  1. Insufficient ablation studies: The paper lacks convincing ablations of J-lens localization, CGCC counterexample feedback, candidate admission gates. Without controlled comparisons on the same targets and probe sets, it is difficult to determine which components are responsible for the reported removal effectiveness.

  2. Limited baseline comparisons: The comparison is mainly limited to directional ablation (DA), while standard model-editing, pruning, unlearning, and component-selection baselines are absent.This makes it difficult to determine which components of L2W are responsible for the observed gains.

Questions for authors

1.Additional ablation studies: Please add controlled ablations on the same targets and probe sets. In particular, compare the full method with (i) a fixed component ranking computed using J-lens before any intervention, progressively closing components without updating the ranking from counterexamples and (ii) the same CGCC procedure using an alternative candidate-ranking method. These comparisons would show which parts of L2W contribute to removal effectiveness.

2.Additional baseline experiment:The current evaluation compares L2W against only one baseline, directional ablation (DA). Please include additional baselines such as the safety-neuron localization and pruning method proposed by Chen et al. [1], to make the empirical conclusions more convincing.

[1] Chen J, Wang X, Yao Z, et al. Towards understanding safety alignment: A mechanistic perspective from safety neurons[J]. Advances in Neural Information Processing Systems, 2026, 38: 12665-12700.

Reviewer_kbGCOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies targeted changes in pre-trained language or visual models while retaining general behavior. It is a hard problem as a removal can lead to further unwanted behavior or even loss of general capabilities. The authors measure this setting with a set of target P_sem and capability P_cap prompts and use iterative modifications based on J-lens attribution for the targeted changes. After the removal they add an additional phase to restore the general capabilities tested via P_cap. Empirical results include four different pre-trained models with different removal targets and the authors show that their method can remove targets tested via P_sem and retrain capabilities tested with P_cap compared to the original model.

Critical strengths
  1. Relevant topic to do targeted changes in pre-trained models while measuring changes induced for general capabilities of the model.
Critical weaknesses
  1. Insufficient control to assess effectiveness of proposed method. To proposed algorithm has several parts. The main evidence for its effectiveness seems to be Table 1. It includes 4 models and 3-5 different tasks. The metric are self-defined prompts/tasks in P_sem, P_cap, and P_held. The control is a baseline model which is the original unaltered model. Taking together, the main results show that the proposed approach can improve on a relatively small set of self-defined prompts and self-defined tasks. There is no established experiment or comparison model to assess the effectiveness of the proposed approach. If no baselines or settings to compare to exist this should be clearly stated and explained.
  2. The proposed method is very complex and consists of several parts with unmeasured effect. The presentation is difficult to follow and it is hard to understand why each component of the methods should be useful. Furthermore, the empirical results fail to establish the usefulness of all these method's components.
Questions for authors

Clarifications:

  1. Is there esatablished work that uses a similar setup to provide evidence for the proposed parts of the method?
  2. How were the sets P_sem, P_cap, and P_held chosen? These set should be given explicitly in the appendix.
  3. Are there alternative baselines to assess the effectiveness of the proposed method more objectively?

Suggestions (future work):

  1. The main overview Figure 1 is very informal and it would be more helpful to showcase a high level overview of how the methods work.
  2. The formalization of the proposed method should be simplified and definitions should be clear. Maybe a small overview of the main definitions or an extra overview figure could aid the understanding.
Reviewer_42hQOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes L2W, a framework for removing targeted behaviors from pretrained models without retraining. It combines Jacobian lens attribution for component localization with iterative causal cuts and capability restoration. Experiments on Qwen, GLM, and FLUX demonstrate its effectiveness in removing watermarks, model metadata claims, and refusal behaviors, although some edits cause noticeable capability degradation.

Critical strengths
  1. The idea of iteratively identifying and disabling components responsible for a target behavior is interesting. CGCC explicitly handles the possibility that the same behavior can emerge through different components, which is a useful contribution beyond one-shot attribution.

  2. The experiments cover different model architectures and target behaviors. The controlled watermark experiment is particularly useful, as it provides evidence that the localization can identify components involved in the implanted behavior.

Critical weaknesses
  1. The evaluation relies on very small probe sets. Some metadata tasks use only 2–5 target probes and three held-out probes, making it difficult to conclude that the target behavior has been reliably removed rather than suppressed on a narrow set of prompts.

  2. Capability preservation remains a concern. In Section 6.4, refusal removal causes substantial benchmark degradation even after restoration. For example, Qwen loses approximately 23 points on IFEval. This raises questions about how surgical the edits really are, especially for behaviors entangled with general capabilities.

Questions for authors
  1. How well does the removal generalize to a larger and more diverse set of unseen prompts? The current held-out sets seem too small to establish reliable removal.

  2. Could the authors provide an ablation comparing J-lens attribution with simpler component selection methods, such as random selection or activation-based ranking? This would help isolate the contribution of the localization method.

  3. How sensitive is CGCC to the choice of the LLM judge and analyst? Would different judges or anchor selections lead to substantially different component cuts?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.