- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes storing LoRA in 1 bit while introducing three scaling vectors to compensate for numerical values. The method decomposes a trained 16-bit LoRA into a 1-bit representation and then optimizes the scaling vectors for calibration. With the same storage space, it performs better than 16-bit LoRA, while maintaining comparable performance before and after quantization and reducing the GPU memory usage of LoRA adapters.
Critical strengths
1.The proposed method achieves better accuracy under a limited storage budget. The experiments show that LoRDBA achieves better accuracy than low-rank FP16 LoRA on LLaMA-2-7B and 13B with the same storage budget. This suggests that keeping a higher rank may be more useful than keeping higher numerical precision when storage is limited.
2.The paper provides a good analysis of resource costs. The paper not only compares storage size and accuracy, but also measures actual GPU memory usage, inference latency, and training costs. The authors also clearly mention that the method does not improve inference speed. These experiments help readers better understand the practical value of this method.
Critical weaknesses
I think the novelty of this paper is relatively limited. Double binary factorization, channel-wise scaling, and scale-only refinement have already been explored in previous DBF work. This paper mainly applies these techniques to LoRA. Also, the comparison with LoRAQuant is not fully fair because LoRDBA receives additional recovery training.
I am also not convinced about the practical value. The main accuracy experiments still reconstruct the binary matrices into floating-point matrices, while the custom CUDA kernel actually makes inference slower. More importantly, LoRA adapters are already much smaller than the base model, so their storage may not be a major bottleneck in most scenarios. In the end, the method mainly saves adapter storage, but it is unclear whether this benefit is worth the extra training cost and slower inference.
Questions for authors
1.Previous DBF work has already explored double binary factorization and scale-only refinement. What is the main novelty of this paper beyond applying these techniques to LoRA?
2.Considering that LoRA adapters are already much smaller than the base model, and the current CUDA implementation makes inference slower, in what practical scenarios do the authors think reducing adapter storage would bring enough benefits to justify the additional training and inference costs?