W1. Limited methodological novelty and an insufficient connection between the theoretical analysis and the learned neural predictor family.
The central parameterization is closely related to existing shared-basis multitask learning, low-rank personalized models, and shared-representation methods with constrained client-specific prediction heads.
The paper acknowledges connections to GO-MTL, ELLA, FedRep, and linear hypernetworks. However, it remains unclear whether the proposed approach introduces a sufficiently distinct methodological contribution beyond applying a shared low-dimensional output basis with locally optimized coefficients in federated learning.
Moreover, the main theoretical results concern a fixed predictor family with independent, isotropic estimation errors. The optimal population subspace is assumed available for the oracle analysis, whereas FedAffine must learn its basis jointly with the shared neural model from finite client datasets.
This distinction is important because the proposed framework is motivated by separating population variation from estimation noise, yet the actual algorithm does not explicitly estimate these two sources of variation or guarantee their separation.
The neural planted-span experiment further illustrates this limitation: even under exact specification, the reported normalized subspace overlap is only 0.29 ± 0.10 (Table 14). Although predictive performance improves, the results do not establish that the learned basis identifies persistent population heterogeneity.
Consequently, the theory provides useful intuition but does not yet explain why the proposed neural optimization procedure should reliably learn the intended population structure.
W2. Experimental comparisons do not fully isolate the advantage of the affine structure, and performance is inconsistent across heterogeneity settings.
Several experimental design choices make it difficult to attribute the reported gains specifically to the proposed affine predictor family.
First, the main FEMNIST comparison uses different personalization and adaptation procedures across methods. FedAffine performs 200 coefficient-estimation steps at evaluation, whereas the primary FedAvg-FT baseline follows a released five-step full-model adaptation procedure. Although the paper provides additional compute-frontier analyses, Appendix D.1.2 also shows that unrestricted post-hoc adaptation from a FedAvg checkpoint can achieve comparable accuracy. A more controlled comparison against equally tuned and compute-matched full-model and low-rank adaptation baselines is needed.
Second, the GDP-HMM implementation initializes the shared affine model from a pretrained FedAvg checkpoint before the reported federated training. It is unclear whether the computational cost of obtaining this initialization is included in the overall training budget. Furthermore, the medical experiment does not include a sufficiently direct parameter- and compute-matched control that isolates the learned affine basis from the additional residual prediction capacity and optimization procedure.
Third, FedAffine is not consistently competitive across datasets. On CIFAR-100, it achieves 60.96% weighted accuracy, substantially below Ditto (66.53%) and FedAvg-FT (66.60%). On Office-Caltech, the improvement over FedBN is relatively modest, and the authors report that local batch-normalization is important for the best operating point.
These results suggest that the usefulness of the affine restriction depends strongly on the form of client heterogeneity. However, the paper does not sufficiently characterize when the assumed low-dimensional population structure is appropriate or when more flexible personalization is preferable.