- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper looks at federated unlearning when several clients ask to have their data removed at once. Instead of removing all of them, the server may pay some requesters to let their data stay, and must then decide where to roll back the model for the rest. The proposed method, FedCARR, works in two steps. First, it decides which requesters to keep, based on how much each contributed and how much it would cost to compensate them. Second, it picks a single historical checkpoint from which to unlearn the remaining clients, balancing model quality against the compensation owed for delayed deletion.
Critical strengths
The problem of asking which requests to accommodate, and how that choice couples with the unlearning state, is reasonable and practical.
Critical weaknesses
- In the abstract and introduction, the authors mentioned the limited compensation budget and argue that retention and rollback must be decided jointly. However, this motivation is not discussed or explained carefully in the proposed method.
- The notations are inconsistent. For example, means both the client type and round index.
- The experimental evaluations are not comprehensive. The experiments should be conducted on more challenging dataset. More baselines should be conducted to further prove the method's novelty. More in-depth analyses and ablation tests should be conducted to prove the paper's claims.
- Despite the good motivation of the rollback state, I still feel that the existence of historical states for the rollback cost a significant memory. This should be discussed very carefully of the practicality of the proposed setting.
- The computation and memory efficiency of FedCARR is missing. This is critical because this is one of the main motivation of the paper.
- The theoretical analysis is limited. Two results are incorrect as stated: the feasible rollback set can be empty, which the paper itself anticipates in Algorithm 2, and in that case problem (P1) has no solution, so Lemmas 3.2 and 3.3 need an additional assumption. The remaining results follow almost directly from the definitions and offer little insight. More importantly, the analysis omits the properties that matter for this kind of mechanism: whether clients have an incentive to report truthfully, how close the two-stage solution comes to the optimum of (P1), and whether the unlearning step actually removes the forgotten clients’ influence.
Questions for authors
- What is limited budget for compensation?
- The idea of rollback seems uncommon as current works unlearn the global model directly. The authors should discuss about the literature review more comprehensively and explain carefully.
- What is the meaning of Section 2.2 Unlearning Request when unlearning is mentioned in Section 2.1?