- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper proposes GLoRA, a gauge-aware server representation for federated LoRA. The key idea is to use a invertible Q matrix to transform client B_i and A_i factors before aggregation. It creates a shared reference space and transforms LoRA factors towards it. GLoRA supports different client ranks. The paper does not provide rigorous theoretical support while the experimental results look promising, GLoRA outperforms other methods in different datasets across different federated settings.
Critical strengths
-
The aggregation mismatch is an important topic for federated LoRA, the paper provides a good motivation.
-
The experimental results are promising. GLoRA outperforms several fed LoRA baselines. Authors also include actual communication/delay cost results.
Critical weaknesses
The presentation of this paper is very poor and difficult for human readers to follow and understand.
-
I don't see why authors explain their experimental setup in Section 2.2 (line 126) even before introducing the proposed algorithm. For Figure 1(b), how you measure the prediction change, is it the change of average accuracy? The current paper is very vague and unclear on that. For line 134, I don't see how SVD relates to the statement you want to make in this section. For line 145, "coordinate departure" is also an unclear concept without any definition. Authors should provide formal definitions of these metrics before showing them.
-
For section 3, authors place all equations without any clear explanation. For Eq. 5, I think U_ref and Z_g are related to B, A, while the paper just simple states them as some "coordinates." Line 196: "The projector is unchanged by ", I assume Q is related to U and Z, while the paper never explicitly explains their relationship. Line 198, TopEig_R is not a standard operator, which should be formally defined before use. Appendix (page 14), citing corresponding equations can help readers to understand the algorithm (please consider doing so). seems to be the transformed version of , then using instead of may improve the readability.
The major concern of this paper is the poor presentation from section 2.2 to 3.3.
Questions for authors
- Why do you compare GLoRA with FedSA-LoRA? FedSA-LoRA is a personalized fed LoRA, with no global model, how do you make the comparison fair?
- In Figure 3, seems GLoRA convergence is slower than FedEx-LoRA, can authors provide explanation on that?