- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper introduces the use of generic algorithm for finetuning a pretrained model via the LoRA scheme.
They claim that the proposed method may generalise to different architectures and domain shift datasets.
The evaluations show some improved performance for some OOD datasets.
Critical strengths
-
Overall this paper is complete with necessary components presented.
-
The evaluations provide some insights of incorporating generic algorithm in model finetuning for OOD tasks.
-
Some figures provide some interesting comparisons, such as fig 5.
Critical weaknesses
Major:
-
Simply incorporating the concepts of generic algorithm into model finetuning is not a solid work grounded for novelty. Generic algorithms have been widely applied into many different domains and areas, and this sounds like an application to finetuning problem. The novelty of this work appears to be very limited.
-
There is no motivating figure showing why and how the proposed model has insights or contributes to the current literature.
-
Fig 2 as the main pipeline overview, caption only contains one sentence and does not provide any descriptions to show why, how and what are the key innovations for the current literature. The figure is also organised in a quite hard to understand way.
-
The method section is written in an unnecessary complex way. In most places, those equations could be explained via plain texts, such as L2-norm, mean over indicators, weighted fusion of weights etc. The method section is also not well organised, and does not clearly show the novelty of this work.
-
This work also introduces many hyperparameters, making the practical application side much weaker, and hard to tune those parameters, although section 3.3 provides some mechanisms for mutation-selection mechanism, but those wordings are unnessary complex and misleading.
-
The implementation details eg in page 6 bottom, is not detailed, and the authors refer to appendix, which also does not provide enough info to deeply assess the setups. The appendix reads like a list and are very concise and compact, more like GPT generated with long and dense maths equations but many bullet points in short, although I would not use this for evaluation.
-
The evaluations appear to be very limited and biased. The authors should perform evaluations on at least 3 large scale/new/challenging datasets rather than old-fashioned datasets. Also with the introduce of LLM/VLM, the evaluations should start shifting to language-driven evaluation tasks rather than classification / determinstic tasks.
-
The discussions and analysis regarding the presented experiments appear to be very limited.
Minor:
-
Figure font sizes are too small to read.
-
The paper reads like a condensed paper (more like GPT generated). With only skeletons and no any solid meats on top.
-
The paper reads quite dry and boring, with unnessary maths equations and complexity, making the core innovations unclear to reviewer.
Questions for authors
See my comments above.