- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper addresses the conflict phenomenon existing in model merging and takes a balanced solution between performance and storage. N models are merged into K groups instead of the traditional merging into 1 group or MoE merging into N groups. This indeed solves an important problem in model merging, the motivation is good.
Critical strengths
Motivation has practical value, and the solution of grouping is interesting. It is relatively novel to do clustering by combining two-task and three-task methods.
Critical weaknesses
- The presentation of the paper is a disaster compared to the paper itself. The main text lacks a visualization diagram that can fully explain the method, and the quality of the diagrams in the appendix is relatively low.
- There are some papers about multitask learning are close to this paper's core contribution [1], it will be better if author can clarify the contributions of this paper. For example, I think the two-task and three-task settings are a good contribution, but this needs to be further clarified.
[1] Li, D., Ju, H., Sharma, A., & Zhang, H. R. (2023, August). Boosting multitask learning on graphs through higher-order task affinities. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (pp. 1213-1222).
Questions for authors
- See the Critical Weaknesses 1 & 2.
- Why inter-task interference is defined Eq. (7), rather than as a performance drop relative to the single-task expert?