- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates how to protect specialized LLMs against unauthorized model merging while preserving their standalone capabilities. The authors propose a training-free method that combines global perturbations derived from weaker auxiliary models with local weight refinement to reduce capability transfer. Experiments on mathematical and medical tasks show that the method retains most standalone performance while substantially reducing the specialized capabilities transferred through model merging.
Critical strengths
- The paper is trying to address unauthorized capability transfer through model merging, an important challenge for protecting the IP of open-source LLMs.
- The proposed method operates directly on model weights without additional gradient-based optimization or architectural modifications. This makes it potentially efficient and applicable to already fine-tuned models.
Critical weaknesses
- The paper discusses several relevant unmergeability defenses,such as PaRaMS, MergeLock, and MergeGuard, but does not provide direct experimental comparisons. Comparisons with existing training-free methods are essential to establish the proposed method's effectiveness.
- The proposed method introduces a noticeable performance cost to the protected model. Specifically, the combined method reduces GSM8K accuracy from 71.87% to 65.07% in Table 3. This performance degradation compromises the standalone utility of the protected model, potentially limiting its practical applicability.
- The effectiveness of the proposed method may depend on the downstream recipient model. For example, BioMistral-7B is used both to construct the global perturbation and as a merging recipient, where the method achieves a substantially higher drop ratio (1.10) than with Mistral-Instruct (0.55) under TA merging. This raises concerns about the generalizability of the method to unseen recipient models.
Questions for authors
- Could the authors further distinguish between merging-specific protection and general parameter fragility? For example, the combined model shows a substantially larger accuracy drop than the original model under Gaussian perturbations (0.36 vs. 0.18). It would be helpful to clarify whether the proposed method specifically disrupts model merging or primarily increases sensitivity to parameter changes.
- As a broader open question, to what extent can model capability and model mergeability be systematically disentangled in the parameter space?
10 comments
Q3. The effectiveness of the proposed method may depend on the downstream recipient model. For example, BioMistral-7B is used both to construct the global perturbation and as a merging recipient, where the method achieves a substantially higher drop ratio (1.10) than with Mistral-Instruct (0.55) under TA merging. This raises concerns about the generalizability of the method to unseen recipient models.
A3. To evaluate whether protection requires the recipient to participate in construction, we additionally test OpenOrca and OpenHermes. Neither model is used as a global auxiliary, for G1 construction, for critical-weight selection, or for protection-parameter tuning.
We apply the same protected checkpoint and the same TA configuration to every recipient, without recipient-specific reconstruction or tuning.
| Recipient | Used in protection construction? | Unprotected TA accuracy (%) | Protected TA accuracy (%) | Reduction (pp) |
|---|---|---|---|---|
| BioMistral-7B | Yes | 54.06 | 32.74 | 21.32 |
| OpenOrca | No | 59.99 | 45.83 | 14.16 |
| OpenHermes | No | 56.36 | 46.99 | 9.37 |
As expected, the strongest effect occurs for BioMistral, which participates in protection construction. However, protection consistently transfers to both construction-held-out recipients, reducing post-merge accuracy by 14.16 percentage points on OpenOrca and 9.37 points on OpenHermes without any recipient-specific tuning.
These results demonstrate that the method does not require the downstream recipient to participate in protection construction.
Q4. Could the authors further distinguish between merging-specific protection and general parameter fragility? For example, the combined model shows a substantially larger accuracy drop than the original model under Gaussian perturbations (0.36 vs. 0.18). It would be helpful to clarify whether the proposed method specifically disrupts model merging or primarily increases sensitivity to parameter changes.
A4. Thank you for highlighting this important distinction. We introduce same-source Gaussian controls to test whether generic parameter fragility or standalone degradation is sufficient to reproduce the observed protection. Starting from the same original specialized checkpoint, we apply Gaussian perturbations that reduce standalone utility below the corresponding protected checkpoints, and then evaluate all models using the same merging protocol.
| Checkpoint | Standalone accuracy (%) ↑ | Bio TA accuracy (%) ↓ | Instruct TA accuracy (%) ↓ |
|---|---|---|---|
| Global | 67.32 | 34.12 | 53.15 |
| Global + Gaussian control | 62.75 | 43.68 | 56.29 |
| Combined | 65.07 | 32.74 | 50.46 |
| Combined + Gaussian control | 61.89 | 41.76 | 53.78 |
Here, higher standalone accuracy indicates better source-model utility, whereas lower post-merge accuracy indicates stronger protection against capability transfer.
The Gaussian controls damage standalone utility more severely than our protection operations, yet they remain substantially more mergeable. Relative to Global, its Gaussian control loses 4.57 points of standalone accuracy but increases Bio and Instruct TA accuracy by 9.56 and 3.14 points, respectively. Similarly, the Gaussian control for Combined loses 3.18 points of standalone accuracy but increases Bio and Instruct TA accuracy by 9.02 and 3.32 points.
This conservative comparison rules out generic parameter fragility or weaker standalone capability as a sufficient explanation: although the Gaussian controls suffer greater standalone degradation, they retain substantially higher post-merge accuracy and transfer more target-domain capability. The protection is therefore better explained by the structured interaction between our global–local parameter configuration and task-vector merging, rather than merely by broad sensitivity to arbitrary weight changes.
Q5. As a broader open question, to what extent can model capability and model mergeability be systematically disentangled in the parameter space?
A5. Thank you for raising this great question. Our results suggest that capability and mergeability can be partially separated. Standalone capability concerns a model’s own behavior, whereas mergeability depends on how its parameters interact with a recipient model and merging operator. Our training-free construction provides one concrete example of this distinction: the protected checkpoint remains useful on its own, but transfers substantially less capability through merging. The same-source controls help explain why this is more than ordinary model degradation: Gaussian controls lose more standalone accuracy yet remain more mergeable, and task-vector shrinkage preserves higher utility without reproducing our protection. Recovery tests further examine whether this separation survives attempts to repair the checkpoint. The global and local variants also illustrate different utility–protection trade-offs. Together, these results suggest that standalone accuracy alone does not determine capability transfer, and that checkpoint design can influence the relationship between them. How far this separation can be pushed at a given utility budget, across recipients, merging operators, and recovery resources, remains open. Our construction and experiments provide a practical starting point for investigating that frontier.
We thank the reviewer again for the constructive suggestions. We hope that the additional experiments and clarifications have addressed your concerns, and we would be grateful if you could reconsider your assessment in light of the new evidence presented in our rebuttal.
We sincerely thank Reviewer Eomc for the thoughtful review and constructive feedback. We appreciate your recognition of the problem and the operational appeal of protecting an already trained checkpoint. We address your concerns below.
Q1. The paper discusses several relevant unmergeability defenses,such as PaRaMS, MergeLock, and MergeGuard, but does not provide direct experimental comparisons. Comparisons with existing training-free methods are essential to establish the proposed method's effectiveness.
A1. Thank you for emphasizing the importance of direct comparisons. To represent existing training-free, parameter-level unmergeability defenses in our direct evaluation, we select PaRaMS as a representative baseline. PaRaMS operates on an already trained checkpoint through parameter transformations without protection-specific fine-tuning, making it the closest representative of this class to our setting.
We implement a Mistral-compatible adaptation of PaRaMS and evaluate it together with our method under the same model family, target task, recipient models, merging protocol, and flexible-extraction metric. Because strong suppression under a fixed merging recipe may disappear after parameter realignment, we compare both methods after the same class of public-base repair attack.
The corresponding unprotected-merge references are 54.06% on Bio and 59.00% on Instruct. PaRaMS preserves standalone utility better, achieving 68.91% compared with 65.07% for our method. However, after public-base repair, PaRaMS recovers to within only 0.23 and 0.74 percentage points of the unprotected Bio and Instruct references, indicating that its protection is almost completely removed. In addition, PaRaMS requires 845.67 seconds for protection processing in our implementation, whereas the measured subtotal of our pipeline is 241.02 seconds. PaRaMS therefore requires approximately 3.5 times more processing time.
A2. Our method is designed as a modular protection framework rather than requiring every model owner to use the combined method. The global and local components provide different operating points:
A model owner with a strict standalone-utility requirement can use the local or global component alone, whereas the combined configuration provides the strongest protection at a larger utility cost. For example, the global-only configuration retains 67.32% GSM8K accuracy—93.7% of the original standalone accuracy—while substantially reducing Bio TA transfer from 54.06% to 34.12%. The choice can therefore be made according to the owner’s acceptable utility budget and desired protection strength.
The standalone degradation is also concentrated on the protected mathematical capability rather than indicating broad model collapse. On MMLU, the combined model achieves 58.94% average accuracy compared with 58.33% for the unprotected specialized model, showing no general-capability degradation on that benchmark.
As discussed in A1, the representative PaRaMS comparison further illustrates why standalone accuracy should be considered together with recovery resistance. PaRaMS retains more standalone utility, but public-base repair restores its post-merge performance close to the unprotected references. Our method incurs a larger standalone cost but remains substantially protected after the same class of repair. It also requires lower measured protection-processing time and introduces no additional inference-time architecture or runtime overhead after release.
Thus, our advantage is not zero-cost protection. Rather, the method provides configurable global, local, and combined operating points, preserves broader model utility, and offers a stronger recovery-aware protection profile at lower measured processing cost.