- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper compares the updates of dense fine-tuning and LoRA when training sequentially on continual learning tasks, questioning the effectiveness of LoRA for reduced forgetting.
Critical strengths
None.
Critical weaknesses
I consider this manuscript substantially below the standard expected of a scientific submission (let alone publication).
Content-wise, it is very hard to understand the claim of the paper. Firstly and foremost, it presents to depict the "unrestricted" fine-tuning as something novel to study, when in reality it is just ordinary fine-tuning. Secondly, it fails to explain why looking at the rank dynamics is important and useful, and why the inverse rank evolution in unconstrained vs. LoRA is surprising. This latter part lacks context about the rank at initialization, which would clarify the actual dynamics. Overall, the paper is vague, but tries to convince the reader with overconfident wording of the discoveries presented.
The paper also has major editorial problems. Indeed, it is very hard to read and presents multiple classical AI-style behaviors that hinder the comprehension of the content. Below are the details:
- There are multiple completely empty sections in the appendix, clearly leftovers from previous iterations, e.g. A.10, A.11, A.15, etc.
- There are multiple meta-comments, e.g. in the title of section A.22 "moved from the main text", but also in A.23, A.24.
- The text is very hard to read due to what looks like AI cues and convoluted wording. Some examples:
- unnecessary terminology, e.g. more than 40 "arm", 9 times "rung"
- long sentences and convoluted syntax, e.g. around 15x em-dash sentences
- extra-defensive statements and over-explanations (e.g., about what the paper is not doing), the most evident are at lines 94-97, 175-176, Section 4.6, Section A.8, Section A.13. These, instead of clarifying, confuse the reader even more with unnecessary details and nitpicking.
I believe that the submission did not receive sufficient critical scrutiny before submission.
(Minor: the title does not use the usual capital letters font of ICLR)
Questions for authors
None.