- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
1.The paper introduces L2W, a post-editing framework for removing a named semantic behavior from an open-weight model. The operator provides a natural-language target and an LLM judge rubric. J-lens attribution localizes components associated with target-bearing tokens, while Counterexample-Guided Causal Cut (CGCC) iteratively closes components, analyzes residual target expressions, and later reopens components that are unnecessary for removal.
2.Experiments cover an implanted watermark, model metadata self-claims, adult-content refusal, and sequential edits on a text-to-image model. Across Qwen, GLM, and FLUX configurations, the method removes all target-semantic probes and reports no residual on its held-out target probes.
Critical strengths
1.Paper provides a clear and potentially useful problem formulation. The paper turns “remove this behavior” into an explicit target-to-component-to-intervention pipeline. The combination of semantic target specification, attribution, counterexample feedback, and restoration is a coherent algorithmic contribution.
2.The method is tested across several component types and model behaviors. The experiments include MLP channels, attention heads, mixture-of-experts components, language models, and a text-to-image model. The sequential-edit experiment is also a useful demonstration that post-edits can compose.
Critical weaknesses
-
Insufficient ablation studies: The paper lacks convincing ablations of J-lens localization, CGCC counterexample feedback, candidate admission gates. Without controlled comparisons on the same targets and probe sets, it is difficult to determine which components are responsible for the reported removal effectiveness.
-
Limited baseline comparisons: The comparison is mainly limited to directional ablation (DA), while standard model-editing, pruning, unlearning, and component-selection baselines are absent.This makes it difficult to determine which components of L2W are responsible for the observed gains.
Questions for authors
1.Additional ablation studies: Please add controlled ablations on the same targets and probe sets. In particular, compare the full method with (i) a fixed component ranking computed using J-lens before any intervention, progressively closing components without updating the ranking from counterexamples and (ii) the same CGCC procedure using an alternative candidate-ranking method. These comparisons would show which parts of L2W contribute to removal effectiveness.
2.Additional baseline experiment:The current evaluation compares L2W against only one baseline, directional ablation (DA). Please include additional baselines such as the safety-neuron localization and pruning method proposed by Chen et al. [1], to make the empirical conclusions more convincing.
[1] Chen J, Wang X, Yao Z, et al. Towards understanding safety alignment: A mechanistic perspective from safety neurons[J]. Advances in Neural Information Processing Systems, 2026, 38: 12665-12700.