- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes a four-stage training pipeline for image classification task. The whole pipeline uses local backpropagation, reducing 58% peak memory usage for CIFAR-10 fine-tuning. However, the accuracy degradation is substantial on various datasets.
Critical strengths
None
Critical weaknesses
-
The research scope of this paper is narrow. The authors only show how their methods can be used for image classification task to reduce GPU memory usage. It is not clear how this method can be transferred to other tasks where I-JEPA training is unavailable.
-
For classification tasks, the accuracy is bad in fact. Other methods like activation compression training (for example, Mesa https://github.com/ziplab/Mesa) can perform as well as full training and reduce up to 50% GPU memory usage.
Questions for authors
Is 58% lower peak memory usage for only finetune stage?
What is the peak memory usage and wall time for the whole pipeline?
Which AI tool do you use? There is no clear model information provided in AI USE STATEMENT.