- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper may introduce a method that makes a LLM checkpoint, which is difficult to finetune at a specific domain, especially for malicious usages. The method focuses on rewriting the weights related to zero activations, where forward is zero but backward can be significant, in SwiGLU FFN layers. The trick can make natural finetuning difficult due to unnaturally large-scale gradients in the target domain but possible in other domains. Experiments with Qwen3-4B and other LLMs show the effectiveness of the method in terms of decreasing the performance of malicious usecase.
Critical strengths
- Proposed a weight-rewriting method for increasing domain-conditioned training resistance
- Formulation of domain-conditioned training resistance
Critical weaknesses
- The experiments are small-scale. The generalization of the empirical conclusions is unclear, in terms of usages with different tasks and domains
- The resistance to target-domain finetuning seems not large. For example, in Figure 2, some different-seed runs of finetuning succeeded like Baseline. Malicious trainers can try to train a model more than once. If 1/3 of trials succeed, the resistance seems not strong practically.
Questions for authors
- The concrete procedure (Appendix B), how to select coordinates, should be explained well in the main body.
- How to evaluate token agreement? The settings, procedures, and data seem unclear in the main body, or possibly even with appendices. The agreement is only evaluated with the domains, which is used for computing r? The LLM usage is basically various and we cannot comprehensively prepare all domains/tasks for computing r. So, agreement in non-observed domains seems a very important metric.
- If attackers observes training results like Fig. 2 during finetuning, they would explore the best settings for performance. Is the proposed method optimization-sensitive or not? The finetuning setting could be various. For example, learning rate, weight decay, global gradient clipping could be changed or removed. In such cases, does the effect change largely? In particular, is global gradient clipping critical?