- Rating
- 4
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
They propose FMuon, a federated adaptive optimization method that combines Muon optimizer on local devices, with shared momentum initialization, and gradient drift correction to align edge devices with global updates. They provided robust convergence gaurantees with evaluations across vision and language model tasks under data heterogeneity.
Critical strengths
The paper reports consistent improvements over the closely related FedMuon method, including gains of around 3 percent on ResNet-18 and ViT-Tiny, under the Dirichlet-0.1 setting in Table 1. The evaluation across larger ViT and GPT-2 models further supports the practical value of the proposed coordination mechanism.
Critical weaknesses
- The authors should modify the abstract and introduction to position the contribution more clearly relative to prior work. FedMuon [1] already studies Muon in heterogeneous federated learning, identifies the bias arising from local LMO updates, and proposes a correction with convergence guarantees. Although the submission acknowledges this relationship in Section 2, its opening presentation can give the impression that introducing Muon into federated learning and addressing the resulting drift are themselves new contributions. This creates tension with the more precise positioning later in the paper. The abstract and introduction should foreground the actual distinctions on momentum and post-LMO operations.
[1] Takezawa, Yuki, et al. "FedMuon: Federated learning with bias-corrected LMO-based optimization." International Conference on Learning Representations. Vol. 2026. 2026.
Questions for authors
Question on the source of communication gains: Tables 12/13 and the compression ablation provide useful evidence of communication efficiency. Could the authors clarify how much of the advantage over FedMuon comes from the proposed coordination mechanism versus compressing the communicated momentum? Since FedMuon’s communicated control variates are also derived from local momentum, could comparable low-rank compression benefit that method? A comparison under matched communication budgets, or a discussion of why compression is particularly suitable for FMuon, would help establish the source of the gains.
3 comments
Thank you for the clear acknowledgment of FedMuon and the additional matched-payload experiments. Please incorporate these results and the clarified positioning into the revised manuscript, and scope the 49% communication reduction to the reported accuracy target. My concerns are largely resolved, and I maintain my acceptance recommendation.
Thank you for reading our follow-up and for suggesting the matched-payload comparison. We’ll include the new results in the revision, clarify how FMuon differs from FedMuon, and specify that the 49% reduction refers to the cumulative payload needed to reach 23% test accuracy in this experiment. Thank you again!
W1. Positioning relative to FedMuon
Thank you for pointing out that the abstract and introduction could state our contribution more precisely. FedMuon identified the bias arising from local linear minimization oracle (LMO) updates in heterogeneous federated learning and proposed a control-variate correction with convergence guarantees. We will acknowledge this prior contribution in both sections and clarify the algorithmic differences. FedMuon retains client-specific momentum across rounds and uses control variates to correct the LMO input. FMuon initializes participating clients with the previous round’s aggregated momentum and mixes each local LMO output with the previous global update direction.
Here, δir=xir,K−xir,0, N is the total number of clients, and S is the number selected in round r.
Q1. Where do the communication gains come from?
Thank you for suggesting a comparison with compressed FedMuon. We applied truncated SVD to FedMuon’s communicated control variates and matched the per-round payload of FMuon_SVD at 23.9 MB. The following results are for CIFAR-100 with ViT-Tiny under Dir-0.1 after 300 communication rounds.
With dense auxiliary states and equal per-round payloads, FMuon achieves 3.01 percentage points higher final accuracy than FedMuon. Compression reduces the per-round payload by 47.6% for both methods, with accuracy drops of 0.89 points for FedMuon and 0.50 points for FMuon. At the matched compressed payload of 23.9 MB per round, FMuon_SVD achieves 3.40 points higher final accuracy than FedMuon + SVD.
At the same per-round payload, FMuon_SVD reaches 23% test accuracy using approximately 49% less cumulative communication because it reaches the target in fewer rounds.
One possible explanation for the smaller accuracy drop is how the two methods use the reconstructed state. FMuon uses reconstructed shared momentum to initialize the local optimizer at the start of each round and then updates it with fresh local gradients. FedMuon’s reconstructed global control variate Cr remains fixed throughout the round and enters the LMO input directly at every local step.