- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes FedCRN, a conflict-compiled routing network for heterogeneous federated learning. Instead of treating disagreement among client updates as optimization noise, FedCRN uses consensus-centered update residuals to identify recurring conflict modes and assigns them to low-rank residual experts within a shared model. The client-level expert addresses are further combined with a sample-level router trained from relative expert competence, while a confidence gate suppresses uncertain residual contributions. Overall, the method is carefully developed, but the empirical gains appear limited.
Critical strengths
- The paper develops the interesting perspective that recurring client-update disagreement may be used as a supervision signal for allocating conditional model capacity, rather than being only averaged, or projected away.
- The experimental evaluation is extensive and transparent, including compiler/routing ablations, stress tests, and communication/computation analysis.
Critical weaknesses
- The novelty of the proposed method appears largely compositional, and the distinction from closely related conflict-aware and federated-MoE methods is not yet sufficiently compelling.
- The ClusterMoE baseline already captures most of the improvement over standard federated optimizers. Thus, the current evidence seems stronger for the value of additional conditional expert capacity than for the specific conflict-compilation mechanism emphasized as the main contribution.
Questions for authors
- Can the authors provide a direct empirical comparison with at least one closely related published conflict-aware FL method (such as FedLAG) and one representative federated MoE method under a reasonably matched setting?
- The current results suggest that much of the gain may come from adding conditional expert capacity rather than from conflict compilation itself. Can the authors provide stronger evidence that the proposed centered conflict compiler contributes beyond the parameter-matched ClusterMoE baseline?
- The NMI and purity values in the multi-mode experiments are relatively low. Do the learned routes actually correspond to the recurring conflict modes assumed by the method? More direct evidence would help clarify this point.
2 comments
Thank you very much for your response! After reading your feedback and taking into account the comments from other reviewers as well, I prefer to keep my original score.
Response to Reviewer WrYx
Thank you for pressing us on whether the conflict compiler itself adds value. ClusterMoE is a strong control, and we agree that the original comparison left room for the expert bank alone to explain much of the gain. The revised paper now tests the compiler at fixed capacity and adds the two published comparisons you requested.
Q1. Direct comparisons and the distinction from prior work. FedLAG uses conflict to decide which layers clients retain locally. FedCRN uses centered update directions to assign clients to residual blocks, aggregates those blocks by group, and uses the resulting expert bank to train a per-example router. FLoRAL instead learns client-specific mixtures of low-rank adapters. In Appendix J we run FedLAG from its published algorithm and FLoRAL using its official numerical core, with the same data, backbone, client schedule, 25 rounds, and training-only learning-rate search budget. FedCRN, FedLAG, and FLoRAL achieve 92.00%, 84.57%, and 83.00% mean accuracy on RotMNIST, and 60.96%, 50.61%, and 43.76% on MultiMNIST. Their server sizes are 112,878, 105,962, and 114,058 parameters, respectively. Table 28 gives transfer in both directions. The MultiMNIST accuracy comparison with FLoRAL has Holm-adjusted p=.091. The corresponding macro-F1 and NLL comparisons survive adjustment.
Q2. What remains when the expert bank is fixed? We agree that the small ClusterMoE differences do not by themselves isolate conflict compilation. The new raw-spherical control keeps the expert bank, hybrid router, aggregation, optimization budget, and initialization fixed. On frozen-feature CIFAR-100, the paired effect of centering is +0.54 accuracy points (95% CI [−0.40,1.48]). On MultiMNIST, full centering changes accuracy by −3.21 points (95% CI [−6.11,−0.30]), while 25% subtraction changes it by −0.53 points (95% CI [−2.16,1.10]). The controlled recovery study gives a more specific result. With a large common component and maximal separation in the generator, ARI rises from 0.592 to 0.958 with centering. We therefore see evidence for a geometric role for centering. The extent to which that translates into better task accuracy depends on the setting.
Q3. Do the learned routes line up with the recurring modes? We added a client-level measurement because the compiler assigns clients, while the sample router assigns examples. For the label teacher, final compiled addresses give NMI/purity of .201/.420 with all 20 clients training and .321/.537 with 16 training clients (Table 25). These are modest associations with the constructed dominant-mode tags. In the controlled generator, where the true modes are known, address persistence can be about 0.95 while ARI is near zero at zero separation (Table 23). At the sample level, NMI is .049 for the label teacher in the 20-client experiment, and accuracy falls to 18.15% on the four excluded clients in the separate holdout experiment. We describe these results as partial alignment with the constructed modes and limited transfer to excluded clients.