- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper mentions the continual learning problem and the limited context problem for current LLMs. To resolve this issue, this paper proposes a new architecture, namely Switcher, which uses new LoRA adapters to continually update the LLM. In general, this paper is a very short technical note, and does not give sufficient details or supporting evidence.
Critical strengths
- Continual learning and extending LLMs' context are both critical problems for the current development of LLMs. This paper studies some important problems.
Critical weaknesses
- The writing quality of this paper is terrible. It includes countless typos and upper/lower case letter problems. Here I just name a few: a. Between lines 045-046, "... Switcher,It makes", letter case problem b. Between lines 034-045, "... Adaptation (LoRA)Hu et al. (2021)", wrong reference format
I believe this paper was never carefully proofread. I am also not sure whether the figures in this paper are vector graphics.
- The logic of this paper is a disaster. I believe the logical flow of this paper is highly vague.
In the introduction, the authors first review the long context problem and the continual learning problem in the first three paragraphs. However, these paragraphs did not go into anything in depth that could motivate the main contribution of this paper.
Claims like "this architecture may help to achieve artificial general intelligence" are also too general and vague.
-
This paper is not written in a way that makes any sense. Its technical content is almost unreadable and does not include any appropriate equations.
-
This paper does not give any empirical results to support the empirical soundness of its proposed architecture. This is a very important issue.
Questions for authors
Please refer to weaknesses 1-4.
Thank you for your time and review.
We have carefully replied to all questions, the open discussion is about to close , up to now all concerns are answered, we will answer any concerns further.
For the contribution of this paper, we emphasize that: Continuous learning is still not solved up to our best knowledge, it is an important obstacle to general intelligence, this paper shows a direction for this problem, this is the contribution point if people do not know how to solve.
We will appreciate you reconsider this point, it is looking forward to your recommendation.