acceptodds
Under review as a conference paper at ICLR 2027

PFedMA: A Federated Learning Algorithm with Pruned Matched Averaging

Abstract

Federated aggregation-based methods such as FedAvg and FedProx perform aggregation across the same parameter indices of locally trained client models. This assumes that the clients represent comparable features at the same indices, which is not necessarily true, as fully connected and convolutional layers exhibit permutation invariance with respect to neurons or filters within a layer. FedMA was introduced to address this mismatch by probabilistically matching neurons or channels layer by layer before performing aggregation. However, unmatched units from client models are appended to the global model. This causes layer-wise growth, which is particularly difficult to accommodate in networks such as ResNets, where channel dimensions and orderings are constrained by cross-layer connections. We introduce Pruned Federated Matched Averaging (PFedMA), a variant of FedMA that preserves a fixed model topology. Rather than retaining every unmatched unit, PFedMA discards unmatched non-donor units and fills vacant global positions using unmatched units from a randomly selected donor client. We further extend matched aggregation to be compatible with residual networks. We do this by enforcing consistent channel assignment across Batch Normalization layers and residual connections. We evaluate PFedMA on four image-classification benchmarks under IID and multiple Dirichlet non-IID partitions, using both sequential CNNs and ResNet-18. FedMA obtains higher accuracy in some sequential-CNN settings. On CIFAR-10 with ResNet-18, PFedMA substantially outperforms the evaluated parameter-wise baselines across the tested partitions. These results suggest that residual networks can also benefit from permutation-aware aggregation.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Federated Learning, Model Aggregation, Neuron Matching, Structural Alignment, Decentralised Training, Computer Vision

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_1AZcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes PFedMA, a fixed-width extension of the neuron-matching aggregation framework used in FedMA. It discards unmatched units from non-donor clients and fills vacant positions using a randomly selected donor client. The authors also coordinate channel assignments across Batch Normalization layers and residual connections to support ResNet architectures. Experiments cover four image-classification datasets under different levels of data heterogeneity. The reported accuracy gains are strongest on ResNet-18, while results on the other architectures are mixed.

Critical strengths

Maintaining a fixed architecture while respecting channel consistency across residual connections is a practically meaningful extension of matched aggregation. The design avoids matching-induced model growth and accommodates networks with stronger structural constraints. The reported accuracy improvements over the evaluated baselines on ResNet-18 provide encouraging empirical evidence for this direction.

Critical weaknesses
  1. The scope of the method’s applicability remains unclear. The strongest gains are observed on ResNet-18, while results across other architectures and heterogeneity settings are mixed. The evaluation primarily focuses on convolutional networks and does not establish whether the proposed fixed-width matching and donor strategy provide similar benefits for other network architectures. This limits my assessment of the method’s generality and broader practical value.
  2. The reliability of random donor selection is insufficiently validated. Since the donor determines which unmatched units are retained, different choices may lead to different accuracy, especially when client data distributions differ substantially. The paper does not report this sensitivity, leaving the stability of the reported gains unclear.
Questions for authors
  1. Which parts of PFedMA depend on the structure of CNNs or residual networks? What modifications or constraints would be needed to apply it to non-CNN architectures, such as Transformers?
  2. How sensitive is the final accuracy to donor selection? Could the authors quantify the variation across runs that change only the donor-selection seed while keeping other settings fixed?
Reviewer_9j5ZOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Pruned Federated Matched Averaging (PFedMA), a fixed-width variant of FedMA for federated learning. It retains probabilistic neuron matching while discarding unmatched non-donor units and filling vacant global positions with unmatched units from a randomly selected donor client. The method also supports Batch Normalization and residual networks through consistent channel alignment. Experiments on four image-classification benchmarks show substantial improvements over the evaluated parameter-wise baselines on ResNet-18, while results on sequential CNNs are mixed.

Critical strengths

The paper is clearly written and easy to follow and the experiments include comparisons with sota models.

Critical weaknesses

(1) The introduction is too brief and does not clearly explain the paper’s contributions. From my understanding, PFedMA mainly modifies FedMA by fixing the global layer widths, discarding unmatched non-donor units, and using donor units to fill vacant positions. However, the rationale and effectiveness of this strategy are not sufficiently supported.

(2) The paper lacks ablation studies. It is unclear whether donor completion is necessary, whether random donor selection is a reasonable choice, and whether the improvements come from the donor mechanism or the handling of Batch Normalization. The paper provides no ablation studies to answer these questions.

(3) Since the method aims to prevent FedMA’s model growth, the paper should compare its communication costs and runtime with FedMA, alongside accuracy. Table 1 does not show a clear overall advantage for PFedMA, particularly on sequential CNNs. Although the ResNet-18 results are promising, they alone do not establish the benefits of the proposed pruning strategy.

Questions for authors

Please see weakness

Reviewer_UW5vOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper investigates the neuron mismatch problem in federated learning (FL) aggregation. Specifically, it builds upon FedMA, an existing FL approach that addresses neuron mismatch through neuron matching. However, FedMA may append unmatched neurons to the aggregated global model, resulting in changes to the original model architecture and increased model width. To address this limitation, the paper proposes PFedMA, which selectively uses unmatched neurons to fill vacant positions in the global model, thereby preserving a fixed model architecture throughout FL training. Furthermore, the paper extends neuron matching and aggregation to handle batch normalization and residual connections, which are commonly used in modern neural network architectures. Experimental results demonstrate that PFedMA achieves competitive performance compared with several state-of-the-art methods and can be successfully applied to ResNet architectures.

Critical strengths
  • The paper builds upon the existing FedMA approach and directly addresses its limitations, making the motivation and contributions of the proposed method clear.
Critical weaknesses
  • PFedMA uses uniformly randomly selected donor clients to fill vacant positions in the global model. However, this strategy may discard useful unmatched neurons from other clients, potentially leading to suboptimal feature representations and aggregation results.

  • As shown in Table 1, PFedMA underperforms FedMA. Although PFedMA offers broader applicability by supporting more complex model architectures, it does not demonstrate performance improvements over FedMA in settings where both methods are applicable.

  • The experiments are limited to relatively small-scale datasets, such as FMNIST, MNIST, and CIFAR-10. It remains unclear whether the proposed method can scale effectively to larger and more challenging datasets, such as ImageNet, and whether similar performance improvements can be achieved in these settings.

Questions for authors

Besides the weakness shown in the above section, please also see the following questions:

Q1: Given that PFedMA can handle residual connections and batch normalization layers, could it also be extended to modern Transformer-based architectures, such as LLMs? Would the proposed method remain effective and achieve performance improvements on these more complex architectures?

Q2: As shown in Table 1, FedMA does not consistently outperform other existing FL methods, particularly FedProx. Given this observation, what is the motivation for further improving and extending the applicability of FedMA through the proposed PFedMA, rather than building upon other potentially more effective FL approaches?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.