- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper addresses a practical problem in federated learning: allocating collaborative performance gains based on actual contributions, which could have significant implications for encouraging edge devices to participate in FL. The authors implement the SICAR algorithm in three steps: First, generate reward directions based on the class distribution; second, map class targets to classifier parameter perturbations; and third, perform known-offset cancellation after training. The approach appears generally sound, and the experimental results validate its effectiveness to a certain extent.
Critical strengths
- The issue addressed in this paper may serve as an important foundation for promoting distributed cooperative learning.
- The article has a clear logical structure, and the proposal is well-defined.
Critical weaknesses
- How can the authors demonstrate that relative proportions are more appropriate than absolute numbers for characterizing a client’s contribution to a particular label? The contribution of labels is clearly strongly correlated with both the quantity and quality of data; I believe it is unreasonable to define it based solely on data quality or quantity, unless the author can provide relevant theoretical support or sufficiently persuasive experimental evidence.
- Fairness is achieved by reducing the performance of all clients, which raises serious practical concerns. The goal of cooperation is to achieve higher accuracy and better generalization performance, but it is clear that the cancellation operation significantly interferes with this goal, as confirmed by the experimental results. This raises the question: Does the improvement in fairness stem primarily from selective degradation, rather than from providing better models to high-contribution clients?
- The authors don't conduct a systematic study of the cooperation among FL clients of different scales, including but not limited to data scale and user count; and don't conduct a systematic study of data that is non-IID to varying degrees, as well.
Questions for authors
please refer to "weaknesses".
ps:
- Just for discussion: Why can't we just use class-wise logit bias adjustment? This kind of method is simpler and more efficient, and there is no experimental evidence to suggest that it is less effective than the approach proposed in this paper.