- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies whether a single server update can improve all participating clients in federated learning. It introduces the agreement cone and characterizes its geometry using cone volume and statistical dimension. The paper also derives results relating cone volume to simultaneous client improvement under aggregation noise and evaluates the proposed diagnostics on FMNIST and CIFAR-10.
Critical strengths
-
The motivating example clearly illustrates why scalar heterogeneity measures such as mean cosine cannot determine whether a strict common-descent direction exists.
-
The Gram-matrix formulation and statistical-dimension estimator provide a computationally explicit characterization of the proposed geometry without sampling in the full model dimension.
Critical weaknesses
-
The central descent-cone framing substantially overlaps with earlier multi-objective optimization work. Brown and Smith (2005, Section 2.3.1) describe common-descent directions through intersecting objective-wise descent half-spaces and qualitatively relate cone size to the probability that a random direction improves all objectives. FedMGDA+ also tests common-descent feasibility. The paper's distinct contribution must therefore lie in its quantitative Gram/orthant formulation, statistical-dimension diagnostics, and noise analysis, whose incremental value is not adequately established.
-
The experiments do not establish the added value of these diagnostics in the structured-conflict regime where they should differ most from simpler measures. After controlling for cosine and training round, the AUROC gain is at most 0.018, with all estimable confidence intervals containing zero (Appendix F.8). The paper reports small within-cosine-quintile differences, but these do not translate into statistically supported held-out AUROC gains. Moreover, Theorem 5 concerns improvement of all clients, whereas the main empirical target is the improved-client fraction.
Questions for authors
Can the authors demonstrate that the proposed cone statistics provide additional predictive value over mean cosine and the FedMGDA+ min-norm diagnostic in a federated setting with structured client conflict? The current experiments do not evaluate this distinguishing regime, while Appendix F.8 shows limited held-out AUROC gains beyond cosine.
2 comments
Thank you for the rebuttal and additional experiments. While some concerns have been addressed, the core idea substantially overlaps with existing common-descent theory, and the significance of its quantitative extension remains insufficiently established. The additional experiments show only limited incremental value beyond existing diagnostics, particularly under ordinary aggregation and nonlinear baselines. The practical utility also remains unclear. Therefore, I maintain my original rating.
We introduce controlled structured-conflict experiments, demonstrate the predictive value of the proposed metric under equal-norm aggregation, and clarify its limitations under ordinary aggregation and nonlinear prediction models.
Dear Reviewer SMHt, thank you for your comments. Our answer to your question follows.
Q1: We agree that the original Appendix F.8 did not establish added predictive value. We removed that claim about the Dirichlet experiments and added structured-conflict experiments (Section 6.2 and Appendix E.3).
On FMNIST and CIFAR-10, five of 100 clients have permuted labels within a near-IID partition. Matched controls retain the original labels and share the partition, sampling, and local-SGD streams. The target is strict loss decrease for every sampled client, rather than the improved-client fraction.
The logistic baseline contains mean cosine, round, min-norm magnitude, and log update variance. The augmented model adds logαt. Models and preprocessing are fitted on seeds 0--4 and evaluated without refitting on seeds 10--14.
AUROCs are averaged across confirmation seeds. Gains use unrounded values and are paired within seed. Intervals are approximate 95% t-intervals across five seed-level gains, conditional on the fitted models. Equal-norm gains are positive in all five CIFAR-10 seeds and four of five FMNIST seeds.
Equal-norm aggregation rescales updates to the round's median norm; this common norm still varies across rounds. The condition was motivated by exploratory results and subsequently evaluated on held-out seeds. Matched-control gains are smaller; the largest is +0.0023 [0.0002, 0.0043].
We also tested cubic B-spline expansions of every feature in both models, fitted only on development data. The structured equal-norm gains then shrink to +0.0008 [−0.0010, 0.0026] on CIFAR-10 and +0.0009 [−0.0022, 0.0040] on FMNIST.
The results support added predictive value for the specified linear model under equal-norm structured conflict. They do not establish a general advantage under ordinary aggregation or across prediction models. Section 6.2 states this limitation; Appendix E.3 reports controls, per-seed results, and nonlinear comparisons.