- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies federated learning from label proportions, where clients hold bag-level label proportions rather than instance labels. The authors propose a two-stage aggregation scheme. The first stage measures client compatibility in proportion space using a symmetric nearest-neighbor Jensen-Shannon set discrepancy, converts it to a similarity with an exponential kernel, keeps the top H=25 collaborators, and forms weights proportional to similarity times sample count. The second stage refines these weights with a class-wise complementarity term capped at 0.25, blended into the first-stage weights with mixing coefficient eta=0.05. Theoretically, Proposition 3.1 gives a covariance sign condition for improvement, Theorem 3.2 decomposes the improvement into a covariance term and a quality transfer term, and the blending is shown to satisfy a total-variation bound of eta. Experiments report best results on 10 of 12 settings.
Critical strengths
- Using bag-proportion geometry as a compatibility proxy is a reasonable answer to the problem, and the symmetric nearest-neighbor construction handles unaligned bags.
- The leave-one-collaborator-out experiment is the right check on whether the compatibility score measures what it claims to.
Critical weaknesses
- The class-complementarity refinement, adds only 1.0 point in Table 2 and is negative on two of the three Fashion-MNIST groups in Table 6 .
- The theoretical improvement conditions are stated in terms of covariances that the paper never measures, so the theory and the experiments do not connect.
Questions for authors
- Have the covariance quantities in Proposition 3.1 been measured on any of the benchmarks?
- Why is eta set to 0.05, and what does the accuracy curve over eta look like?
4 comments
Gentle Follow-up on Our Rebuttal
Dear Reviewer bwJP,
Thank you for your valuable feedback. We have provided our responses and would be happy to address any additional questions.
Best regards,
The Authors
Dear Reviewer bwJP,
Thanks for your time and effort in reviewing this paper. As the author-reviewer discussion period ends in about 24 hours, we would be grateful if you could take a look at our rebuttal and let us know whether your concerns have been addressed. We are happy to clarify any remaining questions before the deadline.
Regards from the authors.
Dear Reviewer bwJP,
Thanks for your time and effort in reviewing this paper. As the author-reviewer discussion period ends in about 4 hours, we would be grateful if you could take a look at our rebuttal and let us know whether your concerns have been addressed. We are happy to clarify any remaining questions before the deadline.
Regards from the authors.
Response to Reviewer bwJP
We sincerely thank the reviewer for the constructive feedback. Below, we address the concerns in the Weaknesses and Questions.
RW1(1): We thank the reviewer for this comment. Although the average improvement in Table 2 is 1.0 percentage point, it represents a 2.83% relative gain over the strong V1 baseline, which we consider meaningful. To further substantiate this improvement, we conducted additional three-seed experiments under β=0.25, as reported below.
+2.42%+16.86%Furthermore, we performed two-sided paired t-tests with Holm correction to assess the statistical significance of these improvements.
SignificantSignificantThese results demonstrate that class-wise refinement delivers substantial relative improvements, particularly on SVHN (+16.86%), and that the gains are statistically significant on both datasets (p<0.05). Therefore, its contribution is both practically meaningful and statistically supported.
RW1(2): We thank the reviewer for this comment. To further validate class-wise refinement, we conducted three-seed experiments and report the results under β=0.25 below.
All nine dataset–coverage combinations show positive mean gains, including the previously negative groups on Fashion-MNIST. Despite seed-level variability, these results support the overall effectiveness of class-wise refinement.
RW2: We thank the reviewer for this valuable comment. To bridge the gap between theory and experiments, we conducted additional analyses to empirically verify the covariance-based improvement conditions in Proposition 3.1 and Theorem 3.2. Since the exact reference classifier is unavailable, we approximate it by optimizing the classifier-only LLP objective from three different head initializations with the backbone fixed.
The results show that the theoretical improvement conditions hold in
63.04%and70.15%of cases, respectively, with over94%sign agreement across reference-head initializations. These findings establish a direct connection between the theoretical analysis and observed behavior.RQ1: As demonstrated in our RW2, the theoretical conditions hold in 63.04% and 70.15% of cases, respectively, with over 94% sign agreement, empirically supporting our theoretical analysis.
RQ2: As shown in Fig. 3(d) and our additional experiments below, η=0.05 achieves the best or near-best accuracy across datasets. Therefore, we empirically select η=0.05 as a suitable refinement strength.
The results show that moderate class-wise refinement improves accuracy over η=0, validating the effectiveness of our proposed component.
Dear Reviewer bwJP,
Your questions helped us better validate class-wise refinement 💡. We hope the new results address your concerns. Thank you for your thoughtful feedback, and best wishes ✨.