- Rating
- 4
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper investigates the mergeability of independently fine-tuned task-specific models under differential privacy (DP) constraints. The authors identify two major geometric factors that hinder effective model merging: increased local sharpness and larger parameter drift from the shared pretrained initialization. To address these limitations, they propose a framework called DP-Merging, which combines sharpness-aware private fine-tuning with reference anchoring. Their theoretical analysis argues that the loss increase induced by model merging is closely related to local sharpness and parameter drift. Experiments across eight vision datasets and eight GLUE tasks, using CLIP ViT and RoBERTa as backbones, demonstrate that DP-Merging consistently outperforms standard differentially private fine-tuning methods under their privacy settings.
Critical strengths
S1. This paper studies an interesting problem: private models perform well on their individual tasks, but their performance drops significantly after merging.
S2. The concepts of local sharpness and reference drift provide a useful geometric view for understanding private model merging.
S3. DP-Merging is simple and practical. It requires no cross-task data access during fine-tuning and works with existing merging methods.
S4. Experiments show consistent improvements across different tasks, backbones, merging methods, and privacy budgets. The ablation results also show that both components contribute to the performance gains.
S5. The paper is well-written.
Critical weaknesses
W1. The experiments cover different model sizes of CLIP ViT and RoBERTa. Evaluating additional model families would help show whether the findings generalise to other architectures.
W2. There are some inconsistencies in the scores and reported gains in the experimental tables. While these do not seem to affect the main conclusions, they should be clarified and corrected.
W3. DP-SAT [1] is closely related to this work. A comparison with this method would help clarify how DP-Merging differs from and improves upon existing approaches.
[1] Park J, Kim H, Choi Y, et al. Differentially private sharpness-aware training[C]//International conference on machine learning. PMLR, 2023: 27204-27224.
Questions for authors
Q1. How does DP-Merging compares with existing PEFT methods, such as DP-LoRA and DP bias-only tuning? I understand that these methods may have different goals, but it would be useful to clarify what advantages DP-Merging offers over them and how it differs from these existing approaches.
Q2. I understand the motivation behind reference anchoring, but its impact on individual-task performance before merging is unclear. Does anchoring affect how well each model performs on its respective task? A brief discussion or additional results would help clarify whether there is a trade-off between individual-task performance and merged performance.
4 comments
Response (2/2)
We sincerely thank the reviewer for their time and thoughtful feedback.
Question 1. Could the authors comment on how DP-Merging compares with parameter-efficient private fine-tuning methods, such as DP-LoRA or DP bias-only tuning?
Response to Question 1: We conducted additional comparisons with DP-LoRA and DP bias-only tuning (DP-BitFit). For DP-LoRA, we use rank and apply adapters to the attention projection layers. For DP-BitFit, we freeze all non-bias parameters and update only the biases using DP-AdamW. The remaining training settings follow those in the main paper.
Comparison with parameter-efficient DP fine-tuning methods on the 8-task vision benchmark using ViT-B/32.
| Fine-tuning Strategy | Weight Averaging | Task Arithmetic | TIES-Merging | PCB-Merging |
|---|---|---|---|---|
| DP-AdamW | 46.0 | 54.2 | 57.5 | 54.5 |
| DP-LoRA | 44.7 | 52.8 | 56.1 | 53.3 |
| DP-BitFit | 41.6 | 50.7 | 52.9 | 50.6 |
| DP-Merging | 49.7 | 58.1 | 62.1 | 59.2 |
DP-Merging outperforms both parameter-efficient baselines across all four merging methods. With TIES-Merging, its accuracy is 6.0 points higher than DP-LoRA and 9.2 points higher than DP-BitFit. Neither baseline improves on DP-AdamW here, suggesting that limiting which parameters are updated is not enough to improve merging performance.
Question 2. I understand the motivation for reference anchoring. Could the authors briefly comment on its effect on individual-task performance before merging?
Response to Question 2: Thank you for this question. Reference anchoring is intended to improve compatibility for subsequent merging, although overly strong anchoring may restrict task adaptation. In our ViT-B/32 experiment, adding reference regularization to DP-SAT increases individual average accuracy from 79.3% to 80.2%. We have also added pre-merging individual-task results to all main tables, as summarized below. Compared with standard DP fine-tuning, DP-Merging improves average performance across all backbones. These results suggest that the full training procedure does not sacrifice average individual-task performance, although individual tasks may show small declines.
| Backbone | DP + Individual | DP-Merging + Individual | Gain |
|---|---|---|---|
| ViT-B/32 | 79.7 | 81.3 | +1.6 |
| ViT-B/16 | 80.0 | 82.1 | +2.1 |
| ViT-L/14 | 85.6 | 87.4 | +1.8 |
| RoBERTa-Large | 80.6 | 80.9 | +0.3 |
| RoBERTa-Base | 79.5 | 79.9 | +0.4 |
For further details on individual-task performance before merging, please refer to the revised main text and Appendix B, where we have added these results.
good
I suggest to accept it
Thank you very much for taking the time to review our responses and share your positive feedback. We sincerely appreciate your time and consideration.
Response (1/2)
Thank you very much for your positive assessment. Below, we address some of your questions.
Response to Weakness 1: Thank you for this suggestion. To evaluate generalization beyond CLIP ViT and RoBERTa, we conduct additional experiments on two model families: ConvNeXt-B for vision and Llama-3.2-1B-Instruct for language. We follow the same experimental protocol as in the main experiments, including the DP fine-tuning procedure, privacy budget, and merging strategy.
Table 1: Evaluation of DP-Merging on ConvNeXt-B and Llama-3.2-1B-Instruct at ε=4.
Compared with the corresponding DP baselines, DP-Merging improves performance by 4.1 and 5.3 points on ConvNeXt-B, and by 5.2 and 4.4 points on Llama-3.2-1B-Instruct, using Weight Averaging and TIES-Merging, respectively. These consistent gains across both vision and language architectures provide additional evidence for the generality of our approach. We will include these additional results and the corresponding experimental details in the revised manuscript.
Response to Weakness 2: Some gain annotations were calculated from unrounded mean scores, while the tables displayed rounded means, leading to apparent inconsistencies. We have checked the affected entries and recalculated the gain annotations consistently from the displayed mean scores.
Response to Weakness 3: Thank you for this suggestion. We have added comparisons with DP-SAT, both with and without reference anchoring, to assess whether its combination with the regularizer can account for the improvements of DP-Merging.
Comparison with DP-SAT and reference-regularized DP-SAT on the 8-task vision benchmark using ViT-B/32 at ε=4.
Adding reference anchoring improves DP-SAT across all three merging operators. Nevertheless, DP-Merging achieves further gains of 3.3, 4.5, and 3.1 percentage points under Task Arithmetic, TIES-Merging, and WUDI-Merging, respectively, while improving individual-model accuracy by 1.1 percentage points. This comparison shows that adding reference anchoring to DP-SAT does not reproduce the performance of DP-Merging, supporting the benefit of the proposed training procedure.