- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper asks how a fixed continuation state budget (memory) in class incremental learning should be split between representation width and interaction fidelity. The proposed method DPSR keeps inexpensive part (feature strengths and feature-class association) accurately and stores an approximation of feature-feature relations. On three benchmarks, DPSR achieves almost the same accuracy as full gram readout with 88-89% less continuation state.
Critical strengths
-
The problem is significant and clearly stated The paper pose a real problem in the domain of analytic continual learning i.e the tradeoff between representation quality and storage accuracy. The research question is specific and experimentally testable.
-
Gains from using the DPSR method are significant DPSR gives accuracy of full Gram model while reducing memory by 88-89%.
-
There is sufficient experimental rigor to support the hypotheses -- Comparison done against multiple baselines LR, FD, RFD, FD + Diag etc. Evaluates on multiple datasets, encoders, memory budgets, ranks, task sequences etc. Reports results on multiple seeds. Rigorous analysis on failure conditions.
-
The paper provides theoretical backing of its design choices and bounds on prediction errors.
Critical weaknesses
- The method's novelty is limited
- A budget-matched span-restricted learner using a LoRanPAC could have been a baseline++ and such results could really show the significance of DPSR.
- I found the readability slightly jarring; especially the way some paragraphs are starting (Fix a continuation budget β, Three observations motivate the allocation etc)
Questions for authors
-
Do you have information regarding how much the individual components contribute to the DPSR accuracy (especially the diagonal correction)?
-
Appendix E.4 mentions narrow exact widths (packed storage of upper triangle). Do you have the results for it? (Table 1 and Table 16)
-
The central allocation claim is tested mainly on three datasets. Could you extend the matched-budget experiments to ImageNet-R/A and at least one omitted benchmark from Table 1 of LoRanPAC (https://arxiv.org/pdf/2410.00645), particularly StanfordCars?