acceptodds
Under review as a conference paper at ICLR 2027

Learning from Disagreement: Conflict-Compiled Routing Networks for Heterogeneous Federated Learning

Abstract

Federated learning usually treats disagreement between client updates as optimization noise to average, shrink, or project away. We ask whether structured disagreement can instead supervise where model capacity should be allocated. We introduce FedCRN, a conflict-compiled routing network that converts consensus-centered update directions into addresses for low-rank residual experts inside one shared model. Each expert is aggregated only within its conflict group. To extend these compiled addresses beyond client identity, a detached router learns per-example relative expert competence. This supports address-free unseen-client routing, while a confidence gate attenuates uncertain residuals and exactly recovers the consensus predictor under uniform routing. Our analysis shows how routing overlap changes cross-client interference, gives a separation condition for persistent compiler addresses, and derives a nonconvex bound governed by post-routing heterogeneity; it explains the mechanism rather than asserting universal superiority. Across five image federations, five paired seeds, and 12 methods, FedCRN attains the best mean rank and leads accuracy on two benchmarks. On 50-client CIFAR-100, it improves over a parameter-matched raw-update ClusterMoE by 0.24 points; after transferring clean-selected hyperparameters to CIFAR-100-C without retuning, it retains the best mean but the 0.20-point gap is not significant. Multi-mode clients, address-free inference, paired tests, and compiler/routing ablations probe the claimed round-to-sample routing path. These results support a different view of heterogeneity: incompatible updates can supervise conditional capacity allocation.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
federated learning, client heterogeneity, mixture of experts, conditional computation, conflict-aware routing, sample-level adaptation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_WrYxOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes FedCRN, a conflict-compiled routing network for heterogeneous federated learning. Instead of treating disagreement among client updates as optimization noise, FedCRN uses consensus-centered update residuals to identify recurring conflict modes and assigns them to low-rank residual experts within a shared model. The client-level expert addresses are further combined with a sample-level router trained from relative expert competence, while a confidence gate suppresses uncertain residual contributions. Overall, the method is carefully developed, but the empirical gains appear limited.

Critical strengths
  1. The paper develops the interesting perspective that recurring client-update disagreement may be used as a supervision signal for allocating conditional model capacity, rather than being only averaged, or projected away.
  2. The experimental evaluation is extensive and transparent, including compiler/routing ablations, stress tests, and communication/computation analysis.
Critical weaknesses
  1. The novelty of the proposed method appears largely compositional, and the distinction from closely related conflict-aware and federated-MoE methods is not yet sufficiently compelling.
  2. The ClusterMoE baseline already captures most of the improvement over standard federated optimizers. Thus, the current evidence seems stronger for the value of additional conditional expert capacity than for the specific conflict-compilation mechanism emphasized as the main contribution.
Questions for authors
  1. Can the authors provide a direct empirical comparison with at least one closely related published conflict-aware FL method (such as FedLAG) and one representative federated MoE method under a reasonably matched setting?
  2. The current results suggest that much of the gain may come from adding conditional expert capacity rather than from conflict compilation itself. Can the authors provide stronger evidence that the proposed centered conflict compiler contributes beyond the parameter-matched ClusterMoE baseline?
  3. The NMI and purity values in the multi-mode experiments are relatively low. Do the learned routes actually correspond to the recurring conflict modes assumed by the method? More direct evidence would help clarify this point.
2 comments
AuthorsOpenReview ↗

Response to Reviewer WrYx

Thank you for pressing us on whether the conflict compiler itself adds value. ClusterMoE is a strong control, and we agree that the original comparison left room for the expert bank alone to explain much of the gain. The revised paper now tests the compiler at fixed capacity and adds the two published comparisons you requested.

Q1. Direct comparisons and the distinction from prior work. FedLAG uses conflict to decide which layers clients retain locally. FedCRN uses centered update directions to assign clients to residual blocks, aggregates those blocks by group, and uses the resulting expert bank to train a per-example router. FLoRAL instead learns client-specific mixtures of low-rank adapters. In Appendix J we run FedLAG from its published algorithm and FLoRAL using its official numerical core, with the same data, backbone, client schedule, 25 rounds, and training-only learning-rate search budget. FedCRN, FedLAG, and FLoRAL achieve 92.00%, 84.57%, and 83.00% mean accuracy on RotMNIST, and 60.96%, 50.61%, and 43.76% on MultiMNIST. Their server sizes are 112,878, 105,962, and 114,058 parameters, respectively. Table 28 gives transfer in both directions. The MultiMNIST accuracy comparison with FLoRAL has Holm-adjusted p=.091p=.091. The corresponding macro-F1 and NLL comparisons survive adjustment.

Q2. What remains when the expert bank is fixed? We agree that the small ClusterMoE differences do not by themselves isolate conflict compilation. The new raw-spherical control keeps the expert bank, hybrid router, aggregation, optimization budget, and initialization fixed. On frozen-feature CIFAR-100, the paired effect of centering is +0.54+0.54 accuracy points (95% CI [−0.40,1.48][-0.40, 1.48]). On MultiMNIST, full centering changes accuracy by −3.21-3.21 points (95% CI [−6.11,−0.30][-6.11, -0.30]), while 25% subtraction changes it by −0.53-0.53 points (95% CI [−2.16,1.10][-2.16, 1.10]). The controlled recovery study gives a more specific result. With a large common component and maximal separation in the generator, ARI rises from 0.592 to 0.958 with centering. We therefore see evidence for a geometric role for centering. The extent to which that translates into better task accuracy depends on the setting.

Q3. Do the learned routes line up with the recurring modes? We added a client-level measurement because the compiler assigns clients, while the sample router assigns examples. For the label teacher, final compiled addresses give NMI/purity of .201/.420 with all 20 clients training and .321/.537 with 16 training clients (Table 25). These are modest associations with the constructed dominant-mode tags. In the controlled generator, where the true modes are known, address persistence can be about 0.95 while ARI is near zero at zero separation (Table 23). At the sample level, NMI is .049 for the label teacher in the 20-client experiment, and accuracy falls to 18.15% on the four excluded clients in the separate holdout experiment. We describe these results as partial alignment with the constructed modes and limited transfer to excluded clients.

Thank you very much for your response! After reading your feedback and taking into account the comments from other reviewers as well, I prefer to keep my original score.

Reviewer_44ySOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes FEDCRN, a federated learning method that uses disagreement among client updates to form groups and assign low-rank experts within a shared model. A sample-level router selects experts based on client addresses and estimated expert competence, with a confidence gate to limit uncertain residual predictions. The paper provides theoretical analysis and evaluates the method on image benchmarks. FEDCRN achieves the best average rank and leads on some datasets, although its gains over strong baselines are modest and often not statistically significant.

Critical strengths

1.FEDCRN uses consensus-centered update residuals to construct expert addresses, then combines group-wise expert aggregation with sample-level routing and confidence gating. These components form a coherent pipeline for converting client disagreement into conditional model capacity.

2.The analysis connects routing overlap to cross-expert gradient interference and studies conditions for address separation and reduced post-routing heterogeneity, providing a rationale for the design.

3.The evaluation includes diverse federated settings, optimization and personalization baselines, a parameter-matched ClusterMoE control, routing ablations, unseen-client tests, and analyses of communication cost and calibration.

Critical weaknesses

1.Equation (6) derives expert-competence targets from the ground-truth label (y), whereas deployment-time routing can only use (x) (and, for known clients, the client address). The paper does not sufficiently assess the impact of this train-test supervision gap on routing quality and generalization.

2.On the real multi-mode task, the centered spherical compiler performs worse than the raw spherical compiler, with a statistically insignificant difference. The controlled synthetic experiment supports the geometric intuition but does not establish a practical performance benefit on realistic tasks.

3.Address stability relies on sufficiently separated conflict modes and bounded compilation and routing errors. These assumptions may not hold when client modes overlap, vary over time, or form a continuum.

4.The multi-mode results show limited NMI and purity, while routing variants trade off known-client accuracy, unseen-client accuracy, and mode recovery. Further evidence is needed that the router learns robust, generalizable sample-level structure.

5.There are no direct comparisons with the closest federated MoE methods. Methods such as FedJETs, pFedMoE, FLoRAL, and FedWeave are discussed but not evaluated under a common protocol, limiting assessment of FEDCRN’s empirical competitiveness against prior federated MoE approaches.

Questions for authors

1.Equation (6) uses ground-truth labels to construct expert-competence targets during training, while labels are unavailable at inference. How sensitive is routing performance to this supervision gap? Could the authors compare against a router trained with label-free targets or provide routing-consistency results between training and deployment?

2.On the real multi-mode task, the centered compiler does not outperform the raw spherical compiler. Under what data or update conditions does centering improve end-to-end performance, and can the authors identify these conditions empirically?

3.The theoretical analysis assumes conflict modes are sufficiently separable and compilation and routing errors are controlled. How does FEDCRN behave when client modes overlap, change over time, or are not naturally clusterable?

4.Given the reported NMI and purity results and the trade-offs among known-client accuracy, unseen-client accuracy, and mode recovery, what evidence supports the claim that the router learns generalizable sample-level structure?

5.Can the authors provide direct comparisons with closely related methods such as FedJETs, pFedMoE, FLoRAL, or FedWeave under matched model capacity, communication budgets, and hyperparameter-selection protocols?

2 comments
AuthorsOpenReview ↗

Response to Reviewer 44yS

Thank you for pointing out the gap between the competence targets used in training and the information available to the router at inference. We added two tests to separate the use of labels in the teacher from transfer to clients the model has never trained on.

Q1. Does the competence teacher need labels? We replaced the label-based expert scores in Eq. (6) with negative predictive entropy. The classification loss still uses labels. We kept the architecture, temperature, address supervision, and training budget fixed. With all 20 clients available for training, address-free accuracy is 51.90% for the label teacher and 52.55% for the entropy teacher, and sample-mode NMI is .049 and .050 (Table 21, five paired seeds). In the checkpoint audit, the input-only router selects the label teacher's top expert for 36.14% of examples, versus 28.54% under marginal independence (paired gap +7.60+7.60 points, 95% CI [0.23,14.97][0.23, 14.97]). The router picks up some of the teacher's preferences, with modest agreement overall.

Q2. When does centering help prediction? Changing only centering on frozen-feature CIFAR-100 gives +0.54+0.54 accuracy points (95% CI [−0.40,1.48][-0.40, 1.48]). On MultiMNIST, the ten-seed differences from raw spherical compilation are −3.21 points with full centering and −0.53 points with 25% subtraction. The latter interval includes zero (Table 12). In controlled signatures, centering improves mode recovery when a large common component masks separated residual directions (Appendix I). We can identify that geometric condition in the generator. A condition that predicts a task-accuracy gain remains open.

Q3. What happens when modes overlap or move? At zero separation, both compilers give near-zero ARI even with persistence around 0.95. Both recover the separated stationary and rotating modes in our generator. When assignments switch, centered ARI falls below raw ARI (0.929 vs. 0.968). For a continuum, we measure address churn because the generator has no discrete modes to recover (Table 24). We also ran an end-to-end MultiMNIST experiment that changes both input transformations and label semantics at round 16. Final shifted-task accuracy is 61.99% for FedCRN, 52.92% for ClusterMoE, and 49.92% for FedAdam. The comparison with FedAdam survives Holm adjustment (p=.031p=.031). The paired interval against ClusterMoE crosses zero, and FedCRN forgets more of the old task on average (Table 29). Appendix K's joint-update bound is conditional on control of update bias and address variation.

Q4. Does sample routing transfer to genuinely new clients? With four clients excluded from training, the label and entropy teachers reach 58.93% and 58.37% on held-out examples from the 16 training clients, and 18.15% and 18.85% on the excluded clients. The gate improves mean ECE and NLL on the excluded clients at fixed weights, while mean accuracy is slightly lower (Table 22). In the separate checkpoint audit on participating clients, sample-route purity is .395, compared with .341 when all examples are assigned to one expert. Together with the low NMI, this supports a narrower claim of partial alignment on participating clients. The transfer gap is substantial for both teachers.

Q5. How does FedCRN compare with published methods? Appendix J adds FLoRAL using its official numerical core and FedLAG using an implementation of its published algorithm. We match the data, backbone, client schedule, training budget, and three-candidate learning-rate selection on training-only data. Parameter counts and communication remain method-specific and are reported in Table 28. Mean accuracy on RotMNIST/MultiMNIST is 92.00%/60.96% for FedCRN, 84.57%/50.61% for FedLAG, and 83.00%/43.76% for FLoRAL. On MultiMNIST, the macro-F1 and NLL differences versus FLoRAL survive Holm adjustment (p=.036p=.036 and p=.030p=.030), while the adjusted accuracy tests are inconclusive. The direct comparison covers these two grayscale federations. Table 1 discusses FedJETs, pFedMoE, and FedWeave at the method level.

Thanks to the authors for extensive efforts made through supplementary experiments, controlled comparisons and limited discussions. The added comparison and time adaptation experiments have addressed some of my doubts. However, two core issues are remained: (1) The consensus centralization benefit has not been established, and it is significantly inferior to the original compilation on MultiMNIST, with no significant improvement on CIFAR-100; (2) No significant decline in client accuracy has raised serious concerns about sample-level routing generalization. Although the added assessment has been strengthened the paper, it is still insufficient to prove its consistency advantage in the real heterogeneous environment. Therefore, I stick to the original evaluation score.

Reviewer_NojGOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The authors introduce Conflict-Compiled Routing Networks (FedCRN), a novel federated learning framework leveraging the discrepancy between client updates to create conditionally compiled capacity by using a weighted consensus vector as a negative, compressed client update signatures are created and clustered into groups with spherical K-means which in turn provide a fixed expert address to each client indicating which low rank feature and decision residuals should be updated and compiled. Then a per-sample routing layer is trained with both the client address as supervision and untrained relative expert competence values. A confidence gate is also used to reduce the magnitude of residuals when routing is ambiguous. The authors explore routing overlapping, expert level interference and a sufficient condition for the fixed compilation addresses. They claim to derive a nonconvex convergence bound based on the post routing discrepancy. The proposed methodology is empirically tested by comparing 12 methods across 5 image federations. An architecture and parameter matched ClusterMoE baseline is included. The authors observed that FedCRN performs the best in terms of average rank and achieves the best accuracy on 2 of the 5 benchmarks. It improves results by 1.73 and 0.88 percentage points compared to FedAdam on CIFAR-100 and CIFAR-100-C, respectively. However, this improvement is not statistically significant compared to ClusterMoE, with only an increase of 0.24 and 0.20 points. Multi-mode experiments demonstrate a balance between accuracy on known clients, transferability to unseen clients and recovering of latent modes. Overall it is an interesting approach for conditionally compiling capacity however the benefits of the core conflict compilation mechanism are not sufficiently demonstrated through end to end experiments.

Critical strengths
  1. The paper proposes a solution to the client update conflict that goes above and beyond simply considering an optimization problem. This method uses consensus based residual signatures to create expert addresses, group-wise aggregation to disentangle the residual parameters, and per-sample competence routing to go beyond client specific specialisation. The approach has a nice conceptual motivation and provides a clear mathematical justification for how interference at the expert level will be impacted by the routing overlap.

  2. Compared to ClusterMoE is a good idea as the method uses the same set of experts, group-wise aggregation, optimiser and parameter count. The paper also reports confidence intervals for the pairs, ablations for the routing, sensitivity analysis and two-way communication. Importantly the authors admit that the improvements are small and not statistically significant over ClusterMoE, and that this method isn’t always better. This makes it easier to understand what evidence there is and separates the geometric motivation from any claims that the end to end results support.

Critical weaknesses
  1. The novelty in the proposed method relies on consensus-centering. However, there is no significant improvement in accuracy in the end-to-end ablation experiment compared to spherical compilation. The mean paired difference is -0.34 (SD=6.53) across 5 seeds ((p=0.891)) and 10 seed audit is also non-significant ((p=0.269). While the synthetic diagnostic indicates consensus-centering can yield gains in cluster purity when a large shared noise vector exists, this does not guarantee any gains in prediction performance in the whole federated model. It would be beneficial to better separate out the effects of the consensus-centering from the gains due to expert capacity, collective aggregation and hybrid routing.

  2. On the CIFAR-100 and CIFAR-100-C datasets, there is a very small improvement over ClusterMoE (0.24 and 0.20%) ((p=0.485), (p=0.466). While there are significant improvements over FedAdam, these results provide less information about the additional value of the proposed architecture through the matched test. Additionally, the natural image experiments are performed using frozen ResNet-18 features and some stress testing has been performed using only 3 seeds. Therefore, we cannot conclude that this method will scale to end-to-end representation learning and it does not seem that the core of the design provides any robust benefit to wider federated scenarios.

Questions for authors
  1. Provided that the raw-spherical compiler matches or slightly exceeds the full method in the reported end-to-end ablations, how should the main contribution of consensus-centering be interpreted? Can the authors provide a stronger controlled comparison that isolates centering while holding the expert bank, routing mechanism, training budget and initialization fixed?

  2. The gains over the architecture-matched ClusterMoE baseline are positive but not statistically significant on CIFAR-100 and CIFAR-100-C. Do the authors have evidence from additional paired seeds, alternative client partitions or other benchmarks that the proposed compiler and hybrid router provide a reproducible advantage beyond the shared expert architecture?

  3. How sensitive is address recovery to the number of conflict modes, the separation between residual directions, the signature dimension and the degree of client participation? Can the theoretical separation condition be translated into an empirically measurable diagnostic for deciding when FedCRN is likely to be beneficial?

  4. In the multi-mode experiment, hybrid routing improves known-client accuracy, whereas removing the confidence gate improves unseen-client accuracy and sample-only routing achieves the highest NMI and purity. How do the authors interpret these trade-offs, and is there evidence that the confidence gate improves calibration or reliability for unseen clients rather than simply attenuating useful residual information?

  5. Since the competence target requires evaluating the full expert bank and downloading all experts, what is the expected trade-off between accuracy, downlink communication and inference cost in larger models? Have the authors investigated whether the compact update signatures or compiled addresses reveal sensitive information about client distributions, or whether the method can be combined with secure aggregation?

2 comments
AuthorsOpenReview ↗

Response to Reviewer NojG

Thank you for asking us to separate the part of the method that creates addresses from the expert architecture that uses them. We added a test that changes only centering, and we revised the paper to be more precise about what that test supports.

Q1. What does centering contribute on its own? On 50-client CIFAR-100, we hold the expert bank, group aggregation, hybrid routing, optimizer, and budget fixed, and pair the initialization and client schedule across five seeds. Subtracting the round mean before spherical clustering gives 43.21% accuracy, compared with 42.67% for the raw signatures (+0.54+0.54 percentage points, 95% CI [−0.40,1.48][-0.40, 1.48]). On MultiMNIST, full centering trails the matched raw control by 3.21 points (95% CI [−6.11,−0.30][-6.11, -0.30]). Subtracting 25% of the mean reduces the gap to 0.53 points (95% CI for partial minus raw [−2.16,1.10][-2.16, 1.10], Table 12). The clearest benefit we can show for centering is improved recovery of separated directions when a common component masks them.

Q2. How reproducible is the gain over ClusterMoE? In the five-seed benchmark, the gaps on CIFAR-100 and CIFAR-100-C are +0.24 and +0.20 points, with both paired intervals spanning zero. Table 11 also varies CIFAR-100 participation. At 10%, 20%, and 50%, the FedCRN means are 28.98%, 43.21%, and 49.65%, versus 28.41%, 42.97%, and 48.44% for ClusterMoE. Those are descriptive comparisons on frozen features. End-to-end training of a natural-image backbone remains untested. Appendix J adds an end-to-end MultiMNIST task shift. FedCRN finishes 9.07 points ahead of ClusterMoE on average, with a paired interval of [−0.11,18.25][-0.11, 18.25]. The evidence for an accuracy gain beyond matched routed capacity remains limited.

Q3. When are the addresses reliable? Table 23 varies mode count, separation, signature dimension, and participation (108 settings, five seeds). At zero separation, adjusted Rand index (ARI) is near zero even though address persistence is about 0.95. At maximum separation, ARI is 0.958 with centering and 0.592 with raw signatures. In the marginal averages, centered ARI ranges from 0.363 to 0.401 across mode counts, 0.362 to 0.399 across dimensions, and 0.351 to 0.405 across participation levels. The top-two cosine-score margin m(u)m(u) gives an observable diagnostic. The winning address is preserved if m(u)>2(ηu+ημ)m(u) > 2(\eta_{u}+\eta_{\mu}), where ηu\eta_{u} and ημ\eta_{\mu} bound perturbations of the signature and the aligned centers. It diagnoses local assignment stability. Its ability to predict downstream gains remains untested.

Q4. What does the gate do for excluded clients? We agree that a route that matches a constructed mode and a route that predicts the label well can be different. In the revised Table 4, the full hybrid model leads addressed accuracy and NMI, and the ungated hybrid has the highest purity. Table 4's address-free evaluation still uses clients that contribute to training. In Table 22, we withheld four clients from training and address compilation and fit the model on the remaining 16. The label teacher reaches 58.93% address-free accuracy on those clients' held-out examples and 18.15% on the four excluded clients. Using the same weights on the excluded clients, turning on the gate changes ECE by −9.34±6.03-9.34\pm6.03 percentage points, NLL by −0.510±0.301-0.510\pm0.301, and accuracy by −0.94±2.80-0.94\pm2.80 points (exploratory paired 95% intervals). The gate improves the two reliability measures in this test, while the accuracy interval includes zero. On participating-client checkpoints, disabling the gate improves mean accuracy (Table 7), so attenuation can also discard useful residual information.

Q5. What are the cost and privacy trade-offs? Each selected client downloads the shared model, router, and all experts, then uploads the shared model, router, and one expert. Soft inference evaluates the full bank. On the feature MLP, increasing the expert count from one to eight raises measured batch latency from 0.108 to 0.394 ms (Table 15). The signature-only probe reaches 33.00% versus a 20.84% permutation reference in the 20-client setting, with a wide interval. Its result is specific to the constructed modes and the probe's side information. Ordinary sum-only secure aggregation hides the individual signatures the compiler needs. Appendix K instead studies clipping and noising each complete uplink before signature extraction. With the ideal Gaussian mechanism and at most 25 releases, the reported setting gives ϵ=3.19\epsilon=3.19 at δ=10−5\delta=10^{-5} and reduces addressed accuracy from 60.96% to 12.37%.

Thankyou for the response, I will maintain my current positive assessment.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of revisions

We appreciate the reviewers' careful reading. A question shared across the reviews was how much of FedCRN's performance comes from having routed experts and how much comes from the way we assign them. The revised paper reports those comparisons separately.

  • Centering at fixed capacity. We hold the experts, routing, training budget, and initialization fixed while varying centering. On frozen-feature CIFAR-100, the paired accuracy difference is +0.54 percentage points (95% CI [−0.40, 1.48]). The ten-seed MultiMNIST comparison favors raw spherical compilation over full centering. In the controlled signature tests, centering improves address recovery when the residual directions are separated and masked by a strong common component (Table 23). These results narrow our claim to when centering helps recover addresses.
  • Routing on clients excluded from training. Appendix I separates the 20-client teacher comparison from the four-client holdout. The label and entropy teachers reach 18.15% and 18.85% accuracy on the excluded clients. With the weights held fixed, the gate lowers calibration error and negative log-likelihood on those clients, while mean accuracy is slightly lower.
  • Published baselines, task drift, and theory. Appendix J adds shared-protocol comparisons with FedLAG and FLoRAL, and an end-to-end MultiMNIST task shift. Appendix K gives a conditional bound for joint updates and reports the accuracy cost of a private uplink with an ideal-Gaussian privacy guarantee.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.