- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper re examines the implicit subspace constraints introduced by task arithmetic in model merging. The authors points out that traditional methods limit candidate models to the subspace formed by task vectors by optimizing the combination coefficients of task vectors, which may hinder the discovery of better multi task weights. To verify this viewpoint, the paper compared coefficient optimization within the subspace with direct weight optimization, and further analyzed the effectiveness of searching outside the subspace through gradient update decomposition and S-RandOpt. The experiment covers various visual and language models and representative merging algorithms, and the results show that using auxiliary data for weight optimization can usually achieve better multitasking performance.
Critical strengths
-
The paper re examines the traditional model merging process from the perspective of implicit subspace constraints, and the research problem has certain novelty and universal significance, providing valuable ideas for further exploring optimization methods for tasks beyond quantum space.
-
The experiment covers multiple model architectures, tasks, and merging methods, and further analyzes subspace gradient updates, auxiliary data scale, and computational costs. The evidence is sufficient to support the main observation that weight space optimization has significant advantages under the studied setting.
Critical weaknesses
-
The main performance improvement of the paper comes from using labeled auxiliary data to perform additional weight fine-tuning on the merged model. However, this process differs significantly from traditional coefficient search in terms of trainable parameter size and optimization capability. Therefore, existing experiments have not fully distinguished whether performance gains come from lifting task constraints into quantum space or from additional supervised multi task training.
-
The explanation of the advantages of optimizing outside the subspace in the paper is still insufficient. Due to the lower dimensionality of the task towards the quantum space compared to the complete parameter space, gradient updates are mainly distributed outside the subspace and may not necessarily have sufficient explanatory power. In addition, some conclusions are overly generalized, such as the more flexible hierarchical coefficient optimization in the appendix being superior to weight fine-tuning under certain settings. The author also refers to the best performance of the test set in the limited search method as Subspace Best, and needs to more accurately explain the meaning of this empirical benchmark.
Questions for authors
-
Can the authors further compare different degrees of subspace constraints and weight optimization under the same auxiliary data and computational budget to distinguish the performance benefits brought by search space expansion and additional supervised training?
-
Can the authors supplement the experiment on controlling the gradient inside and outside the subspace to update the norm, and explain why the direction outside the subspace can produce better generalization performance? In addition, could you clarify the selection method of Subspace Best and discuss more comprehensively the situation where optimizing hierarchical coefficients is superior to fine-tuning weights?
3 comments
Thank you to the authors for their response. I have read and considered the authors’ response and have decided to maintain my original score.
We thank the reviewer for the quick response. We are confident that our responses have addressed the concerns and questions raised by the reviewer, including:
- Clarify the main contributions of the paper
- Clarify the performance gain of gradient descent
- Clarify the findings of coefficient optimization versus weight fine-tuning
- Clarify the Subspace Best estimation
- Clarify the message of Section 4
We would also appreciate further clarification on a few points raised in the original review, as we want to ensure that we have accurately understood and addressed these concerns.
Weakness 1
Therefore, existing experiments have not fully distinguished whether performance gains come from lifting task constraints into quantum space …
Weakness 2
Due to the lower dimensionality of the task towards the quantum space compared to the complete parameter space, ...
In addition, some conclusions are overly generalized ...
We would be happy to provide further explanations or clarifications if needed. If we have successfully addressed your questions and concerns as we understand them, and the reviewer has no remaining concerns, we respectfully ask the reviewer to reconsider their assessment accordingly. Thank you again for your service.
Rebuttal
Thank you for your review. Please refer to our global response before reading our responses to your specific concerns and questions. We address your concerns below.
Weakness 1.1: Different from traditional coefficient search
As discussed in the paper, we challenge the implicit regularization induced by coefficient search. We point out that under the existing evaluation scenario, the traditional coefficient search methods are actually underparameterized.
Weakness 1.2: Distinguish the performance gain
Please see the global response for why the question is unclear. However, if the question is “whether gains come from removing the regularization or additional training”, the answer is the former, for two reasons:
Weakness 2.1: Explanation of gradient descent performance gain
To clarify, Section 4.1 does not try to explain which direction is better. It aims to visualize that gradient descent indeed leaves the task-vector subspace. Our explanation for gradient descent's superior performance in standard evaluation settings is the removal of implicit regularization.
Weakness 2.2 & Question 2.3: When is coefficient optimization better than weight fine-tuning?
Weakness 2.3 & Question 2.2: Subspace Best
We summarize Appendix C.1 here for reference: “Subspace Best reports the highest test performance among the evaluated coefficient optimization methods (coefficient search, Subspace GD and Bayesian optimization), assessing the performance achievable within the task-vector subspace.”
Question 1: More comparisons
We summarize the analyses from the paper here for reference:
The new experiment in Section 5.4 also addresses your question; please see the global response for more details.
Question 2.1: Geometric analysis
As clarified earlier, we do not investigate which directions outside the task-vector subspace are better or why. Moreover, Section 4 does not claim that gradient descent achieves better generalization performance, nor is this claim true.