- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes a method to tackle model-poisoning in federated learning, which is CARAT. CARAT uses a validation set on the server side to construct a class-aware set of representations and use them to construct certificates for later aggregation. The empirical results are extensive and support the claims.
Critical strengths
- Good visualization
- Extensive empirical results
Critical weaknesses
- In Section 4.2, what is the rationale of clipping the update of each client. If the goal is to avoid dominant scale, simple standard normalization would be sufficient. The author should provide ablation studies to verify this design choice.
- The design of certificate (Eq. 8) is based on the hypothesis that malicious updates might degrade performances of only a subset of tasks. Is there any evidence to show this hypothesis is correct empirically?
- The design choices are very unclear to me. It is hard to understand why the author designed the method with complicated components without verification, e.g., coordinate-rank anomaly score or bounded weight learning. More experiments to verify them are necessary.
Questions for authors
See Weaknesses
2 comments
Thank you for your responses. After reading the rebuttals and the authors's responses to Reviewer 2Ris, my main concerns on the design choices are not well resolved. The novelty and rationale behind the proposals are limited to me. Therefore I will maintain my score.
Rationale for clipping instead of normalization (Sec. 4.2). The two operations serve different purposes, and CARAT uses both.
Evidence for the "subset of tasks" hypothesis behind Eq. 8. Section 4.2 now presents this as a design motivation. The softmin is a conservative aggregate: it focuses the weight optimization on the worst-served hidden task rather than the average, so harm on any single task lowers the objective. Empirically, the full rule removes aggressive attackers:
Verification of the rank penalty and the bounded weights.
Every component improves on its removal in Table 4. The differences are modest at 20% malicious because in that regime even undefended averaging is barely affected by these attacks. Where attacks are aggressive, CARAT's advantage is large: at 40% malicious it exceeds undefended Mean by up to 47 points under ALIE and by 17–37 points under FangAttack (except CIFAR-10, α=0.1).
Presentation. Section 4 now has a short "Design rationale and evidence" paragraph. It maps each component (clipping, common-radius certificate, rank penalty, capped weights) to the failure it addresses and to the evidence above. Section 4.2 also gives the clipping-versus-normalization rationale.