- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper aims to reduce the local training and model exchange cost of the distributed Muon optimizer. It propose a resource-aware quantized framework, RADeMuon, to address this challenge. The authors provide the convergence guarantees and experiments results to support their proposed algorithm.
Critical strengths
- The proposed algorithms effectively reduces costs by introducing quantization into distributed Muon optimization.
- The paper provides theoretical analysis and empirical studies.
Critical weaknesses
- The introduction claims that there are three Muon-specific challenges associated with quantization precision. However, challenges (i) and (iii) seem to be common to algorithms with momentum and decentralized network, respectively, instead of being specific to Muon. The authors should further clarify what makes these challenges unique to Muon and where the main technical novelty lies.
- Theorem 1 and Corollary 1 contain a term $
(\sigma_1 + \sigma_2)$, which involving , and , but not explicitly decrease with . Thus, this term cannot be eliminated as increases. Although the authors claim that convergence “requires the stochastic and heterogeneity residuals to vanish”, this requirement is not generally necessary in standard distributed learning, which weakens the theoretical results. In addition, Eq. (S.57) simply assume , without establishing the conditions under which this assumption holds. Thus, the statement that remains questionable. 3. Regarding the experiments, the 8-bit De-EF21-Muon baseline achieve lower communication cost than RADeMuon, although it requires higher computation cost. This seems to contradict the main claim of reducing both computation and communication cost. Furthermore, the paper only reports costs in MB. What about walk-clock time? 4. Table 2 and 3 are difficult to read. I recommend adding an arrow to each metric to indicate whether higher or lower values are better, and using different colors to highlight the best results. 5. The paper provides a relatively limited discussion of existing studies in related work section. I recommend dividing this section into two subsections,(2.1) Distributed Muon, and (2.2) Quantization and compression. For example, [1]-[4] are closely related work, but the relevant literature is notlimiated to these studies. Please provide a more comprehensive related work section. 6. The difference between Adriana (Qiao et al., 2026) and the proposed work is unclear. Does this work directly apply resource-aware quantization to Muon updates, or does it troduce additional methodological contributions?
[1] Junkang Liu, Fanhua Shang, Junchao Zhou, Hongying Liu, Yuanyuan Liu, and Jin Liu. 2025. Fedmuon: Accelerating federated learning with matrix orthogonalization.
[2] Xinwen Zhang and Hongchang Gao. 2025. On provable benefits of muon in federated learning.
[3] Neel Mishra, Kushagara Trivedi, Pawan Kumar. 2026. Signmuon: Communication-efficient distributed muon optimization.
[4] Laurent Condat, Kai Yi, Peter Richtarik. 2022. EF-BV: A unified theory of error feedback and variance reduction mechanisms for biased and unbiased compression in distributed optimization.
Questions for authors
See the weaknesses above.