acceptodds
Under review as a conference paper at ICLR 2027

Learning to Refer: Inference-Time Collaboration in Decentralized Federated Learning

Abstract

In decentralized federated learning, heterogeneous clients may develop complementary expertise, yet each query is typically evaluated only by the client where it arrives. To exploit this distributed expertise without global coordination, we formulate inference-time collaboration as budgeted local expert referral, a graph-constrained routing process in which a query can move only between neighboring clients and only visited clients contribute to its final prediction. EdgeDRM implements this framework by learning a density-based routing potential for each client: differences between neighboring potentials provide the comparisons used to select successive referrals, while local task models generate predictions that are aggregated along the resulting route. Both training and inference are decentralized, using only neighbor-local information without a central router or graph-wide client scoring. We establish estimation guarantees for the learned comparisons and characterize how local improvement and graph structure govern prior-weighted density gains from referral. Experiments on benchmark and real-world medical data show that EdgeDRM outperforms graph-matched decentralized baselines in most settings, with the clearest benefits under sparse connectivity.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Decentralized Federated Learning, Routing, Inference-Time Collaboration

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_YWXQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies inference-time collaboration in decentralized federated learning (DFL). Instead of evaluating each test query only at the client where it arrives, the authors formulate "budgeted local expert referral": a query may move at most L hops along the communication graph, and only visited clients contribute to the final prediction. The proposed EdgeDRM adapts the density-ratio model (DRM) of FedDRM to this setting by replacing the graph-wide m-class source classification with balanced binary source classification on each edge, while sharing one routing potential per client so that edge comparisons telescope along a path. At inference, a sample-count prior log(n_j/n_i) is added, the query greedily moves to the best unvisited neighbor, and visited predictions are aggregated with softmax weights of the cumulative scores. Theory gives an estimation rate and asymptotic normality for the Stage-II coefficients under a fixed common encoder (Thm 1) and a lower bound on the referral gain under a minimum-local-improvement condition (Thm 2). Experiments cover 8-client CIFAR-10/20/100 on four topologies, a 32-client scaling study, a tiered-hospital simulation, and ISIC2019.

The problem formulation is new for DFL and clearly defined, and the design is clean: one potential per client makes edge comparisons coherent across hops, and the separation between the balanced training logit and the inference-time prior is consistently explained. The paper is also unusually candid about its scope (EL equivalence only after profiling intercepts, theory not covering the experimental training, no prediction-risk guarantee, no advantage on COMPLETE graphs), and the ISIC2019 split audit that removes lesion-ID leakage is a useful contribution in itself.

However, the evaluation protocol may favor referral by construction, and the evidence that learned density-ratio routing is the key driver is limited. Queries are always assigned to a non-source entry client while each client has a unique deterministic color transform, so the source client is easy to identify from the input; even random referral improves over no referral by about 6 points. Strong neighbor-only baselines are missing, improvements are often within seed variance, and on real heterogeneity (ISIC2019) the gains shrink to 0–1 points without reported variance. The theory relies on assumptions (common encoder, no non-global local maxima, positive threshold) that the experiments do not satisfy or measure.

Critical strengths
  • Inference-time referral as a graph-constrained, sequential problem: The paper observes that in DFL the topology limits not only how models are mixed during training but also which clients a test query can reach, and formulates this as budgeted local expert referral (budget L, no revisits, only visited clients predict; Sec. 2, Alg. 1). This turns "which expert should answer" from a one-shot global selection, as in FedDRM or federated MoE, into a path-dependent search under neighbor-only communication. It is a natural and underexplored question for decentralized systems, and the formulation can be reused independently of the specific router.
  • Node potentials make local comparisons globally coherent: Instead of training unrelated edge classifiers, EdgeDRM shares one routing potential per client, so each edge logit is a difference of potentials and comparisons telescope along a route (App. C.2). Greedy multi-hop routing therefore ascends a single quantity, and the balanced training logit is cleanly separated from the sample-count prior, which is added exactly once at inference (Eq. 5–6, App. B.4). The principle is simple and fits the neighbor-only constraint well.
Critical weaknesses
  • The router targets the likely source, not the better expert, and the CIFAR protocol makes the two coincide by construction: The motivation is expertise, i.e., a query should reach a client that handles it better (Sec. 1, Fig. 1). The learned potential, however, estimates source density, and the paper itself states that the bound on Γ_L gives no ordering of prediction risk between the entry, terminal, and history predictions (C.4, "Prediction target"). "Most likely source = better expert" is thus the central assumption, and the evaluation builds it in rather than testing it.
    • Every query enters at a non-source client (Sec. 5.1), each client has a unique deterministic color transform (D.1), and heads are personalized, so the source is easy to identify and its head is the natural expert by design.
    • The results fit this reading: Random referral already beats EdgeDRM-NR by about 6 points (31.6 vs. 25.4, Fig. 2), EdgeDRM-NR is far below DFedAvgM (25.35 vs. 40.10, RR4 CIFAR-10 Dir, Table 1), and NR drops monotonically as heads become more local (35.01 → 24.97, Table 6). Much of the "referral gain" may be recovering the loss that Stage II personalization creates at a non-source entry.
    • On ISIC2019, with real center heterogeneity and source entries allowed (Sec. 5.4), the gains over DFedAvgM are +0.18/+1.15/+0.31/−0.14 points (Table 3), without SD and at unequal training effort (400+50 vs. 400 rounds, D.3.2).
  • The evidence does not isolate the learned density ratio as the driver: It remains unclear whether the gain comes from density-ratio routing or simply from moving the query.
    • Only Random and Count-only routing are compared under the same graph and budget (Fig. 2). Since history averaging adds little over the hard terminal prediction (17.59 vs. 17.28, Table 7), the gain is almost entirely route selection, so confidence/entropy-based referral, averaging within L hops, or the same rule on DFedAvgM models are the decisive comparisons. Some gains are also within seed variance (PATH CIFAR-10 Dir: 33.08 ± 3.53 vs. 28.55 ± 7.62).
    • Non-shared encoders weaken the density-ratio interpretation: the Stage-II edge classifier sees client j's samples through g_θj (Eq. 15), but at inference both sides are scored with g_θi (Eq. 6), and no encoder consensus is applied (D.1). Neither δ nor |Δ̂_ij + Δ̂_ji| is measured, although the sparse graphs where EdgeDRM helps are where encoders differ most.
    • The hospital study cannot test learned routing, since hospitals hold class-balanced CIFAR-10 subsets (D.1); without the prior, referral adds only 0.45–0.63 points over NR (Table 4).
  • The theory does not cover the regime the method relies on: The guarantees are stated for a setting that the experiments do not satisfy, so they do not support the empirical claims.
    • Thm 1 is a standard M-estimation result under a common, fixed encoder, while the experiments use different encoders per client (D.1). Condition (11) essentially rules out non-global local maxima, from which greedy ascent follows almost directly (C.3).
    • Thm 2 requires η ≥ ϵ > 0 while the experiments use a zero threshold (D.2.3, D.3.2), and it bounds a density gain Γ_L, not accuracy (C.4); the abstract and contributions should say so clearly.
Questions for authors
  • Source identification: What fraction of routes terminate at the query's true source client? Could you report Table 1 under uniform entry including the source client, and in a label-skew-only setting without per-client color transforms? An oracle "route to source" upper bound would also help interpret the gains.
  • Neighbor-only baselines: How does EdgeDRM compare, under the same graph and budget, with confidence/entropy-based greedy referral, averaging the models within L hops, and the same routing rule applied to DFedAvgM models?
  • Representation disagreement: How large are δ and |Δ̂_ij + Δ̂_ji| on the sparse graphs? Does EdgeDRM still help with a shared frozen encoder (e.g., exact consensus before Stage II)?
  • Training effort and tuning: Is the CIFAR DFedAvgM baseline the 800-round prefix or a 1000-round model, and how were baselines tuned? Why does DFedAvgM on COMPLETE (exact averaging) reach only 55% on CIFAR-10 Dir-0.3?
  • Variance on real data: Can you report per-seed results, SDs, and EdgeDRM-NR for ISIC2019 (Table 3), and sensitivity of CIFAR results to η > 0 and L?
9 comments

We thank the reviewer for recognizing the graph-constrained formulation, the node-potential design, and the split audit. We address the questions about predictive expertise, the contribution of learned routing, representation consistency, training effort, and empirical uncertainty below.

  • Source identification, predictive expertise, and the evaluation protocol (W1/Q1). Your first concern combines two questions: whether source compatibility helps prediction, and whether the non-source protocol makes referral look favorable. We assess the selected experts' predictions, report source recovery, and then compare the complete system with continued DFedAvgM training under source-inclusive arrivals.
    1. Density-guided prediction and expert selection (W1). EdgeDRM uses density-ratio scores to select a sequence of experts and weight their predictions along the visited route. The final prediction therefore retains information from earlier experts rather than relying exclusively on the terminal expert. Under a common representation and exact density-ratio scores, the history weights are proportional to the prior-weighted densities of the visited clients. With access to all clients, correctly matched source priors, and exact conditional-label models, this weighting recovers the conditional prediction of the full mixture distribution. This provides a probabilistic motivation for the weighting rule. We emphasize that this claim does not establish that every higher-scoring expert has lower loss or that every referral improves prediction.

    2. Prediction changes on accepted referrals. To examine the predictive effect of accepted referrals, we compared the estimated routing score Δ^ij(x)\widehat\Delta_{ij}(x) with the reduction in true-label cross-entropy, ℓi(x,y)−ℓj(x,y)\ell_i(x,y)-\ell_j(x,y), for all 16,004 accepted hops on CIFAR-100/25-SPC with RR4. The figure in Appendix D.2.3 (or in this link) groups hops by whether the current expert and its neighbor predicted the correct class. This distinction matters because the same query can contribute to more than one panel. For example, on a wrong → wrong → correct route, the first hop appears in panel (d) and the second in panel (a). The 75.2% in panel (d) thus counts hops between two incorrect experts, even when a later expert gets the query right.

      Panel (a) isolates the 2,845 hops that corrected an error (17.8% of all hops). The neighbor assigned a higher probability to the true class on 2,841 of these hops. Within this group, the routing score and the loss reduction showed a clear negative correlation (Pearson r=−0.58r=-0.58). In other words, even a modest routing score could accompany a large increase in the probability assigned to the true class, while higher scores tended to accompany smaller reductions in loss. For comparison, 720 hops (4.5%) changed a correct prediction to an incorrect one (panel (b)).

    3. Does the selected expert outperform the entry expert? We compare the terminal expert selected by referral with the original entry expert under uniform-all arrivals on CIFAR-10/Dir-0.3.

      GraphEntry accuracyHard-terminal accuracyTerminal loss lower than or equal entry, %
      PATH27.95 ± 3.4336.74 ± 1.9777.27 ± 0.23
      CYCLE28.73 ± 3.3138.77 ± 1.1675.48 ± 0.84
      RR430.76 ± 3.0851.04 ± 2.3975.06 ± 1.47
      COMPLETE32.89 ± 2.9356.85 ± 2.8674.22 ± 2.47

      The terminal expert achieves substantially higher mean accuracy than the entry expert across all 4 topologies, supporting the predictive benefit of referral in aggregate. The last column reports the percentage of queries for which the terminal expert has no higher true-label cross-entropy than the entry expert, indicating that most of the referrals eventually have a lower loss.

  1. Source recovery under graph constraints (W1/Q1). You ask how often routing reaches the query's true source, given the concern that the CIFAR gains may mainly reflect source recovery. We therefore measure the fraction of routes that terminate at the source under uniform-all arrivals. The query's source identity is used for this diagnostic and is not provided to the deployed router:

    GraphCIFAR-10 / Dir-0.3 terminal-source (%)ISIC2019 terminal-source (%)
    PATH32.30 ± 1.1537.33 ± 0.57
    CYCLE35.57 ± 1.1844.44 ± 0.40
    RR4 / RR358.01 ± 3.5353.59 ± 0.25
    COMPLETE69.40 ± 3.0766.78 ± 0.41

    Source recovery is higher on RR4/RR3 and COMPLETE than on PATH and CYCLE. This pattern is consistent with the constraints of neighbor-only search: a route can stop before reaching the source when no unvisited neighbor has an acceptable score. Even on COMPLETE, where every client is directly accessible, only 69.40%/66.78% of CIFAR-10/ISIC2019 routes end at the source. EdgeDRM selects clients by their estimated prior-weighted density, whose maximum need not coincide with the query's recorded source. Together, source-recovery rates alone do not establish whether prediction gains come from routes ending at non-source experts.

  2. Source-inclusive arrivals and personalization recovery (W1/Q1). To test whether the gains rely on excluding source entries, we reevaluate the fixed checkpoints under four arrival protocols. The following CIFAR-10/Dir-0.3/RR4 results report three-seed accuracy.

    Entry protocolDFedAvgM1000EdgeDRM-NREdgeDRM, no priorEdgeDRM, with prior
    Source only60.86 ± 7.6769.99 ± 4.8961.16 ± 2.8961.25 ± 3.01
    Non-source only40.33 ± 2.0625.16 ± 4.2049.48 ± 1.9350.02 ± 2.30
    Uniform over all clients42.90 ± 1.1830.76 ± 3.0850.94 ± 1.9251.42 ± 2.37
    50% source / 50% uniform non-source50.60 ± 3.0747.57 ± 0.6455.32 ± 2.1555.63 ± 2.62

    Under source-only arrivals, EdgeDRM-NR achieves 69.99% accuracy versus 60.86% for DFedAvgM1000, indicating stronger performance on the experts’ own source distributions. Source-entry queries already start at an expert trained on their source distribution, leaving less room for referral to help. Under non-source-only, uniform-all, and 50/50 arrivals, referral improves on EdgeDRM-NR, and the complete EdgeDRM procedure outperforms DFedAvgM1000.

  3. Why use client-specific transformations? (W1/Q1). The transformations create controlled feature shifts following FedDRM [A], alongside the label-skew partitions; acquisition-related feature heterogeneity is also studied in FedBN [B]. They let us study partition and topology effects under a reproducible feature shift. The existing CIFAR results therefore concern joint feature and label heterogeneity. Label-skew-only results require retraining of all baselines and need more time to obtain, and are thus not included. ISIC2019 provides complementary evidence under natural source/acquisition heterogeneity, with its uncertainty assessed under Q5.

  • Learned selection and neighbor-only routing alternatives (W2/Q2). The preceding comparisons evaluate whether the complete system improves prediction. To identify the contribution of the routing rule, we now hold the trained experts fixed and compare different ways to select or combine them.
    1. Learned selection with equal expert access (W2/Q2). To test selection at exactly matched access, we fix the expert pool and prediction rule and require both learned and Random routing to evaluate exactly two experts and make one move, across all 24 CIFAR configurations:

      A common maximum hop budget need not produce equal realized access: under the deployed rules and uniform-all arrivals, Random uses 5.34 expert calls versus 1.81 for learned routing, with 1.30 points higher mean score-weighted accuracy. The comparison below instead fixes actual access.

      ArrivalReadoutLearnedRandomDifference, ppPositive configurations
      Uniform-allScore-weighted29.8224.45+5.3624/24
      Uniform-allHard terminal27.1121.01+6.1024/24
      Non-sourceScore-weighted28.1021.51+6.5924/24
      Non-sourceHard terminal28.4521.43+7.0224/24
      SourceScore-weighted41.8445.07−3.230/24
      SourceHard terminal17.7418.08−0.347/24

      Learned selection improves mean hard-terminal accuracy over Random by 6.10 percentage points under uniform-all arrivals, with positive differences in all 24 configurations. The policies use equal calls and only the terminal prediction, so the difference measures selection without an aggregation or access advantage. Score-weighted prediction also favors learned selection by 5.36 points.

    2. Comparison with confidence and entropy (W2/Q2). The learned-versus-random comparison establishes a benefit from selection under controlled access. To test whether that benefit extends to stronger alternatives, we additionally evaluate confidence- and entropy-based referral and extend Count-only referral across all 24 CIFAR configurations. All use the same history-aggregation rule. For confidence and entropy routing, every candidate evaluation counts toward the L+1L+1 expert-call cap.

      ComparatorLearned minus comparator, pp (score-weighted)
      Count-only+8.32
      Confidence, L+1L+1 expert-call cap+3.01
      Entropy, L+1L+1 expert-call cap+2.58

      Under uniform-all arrivals, EdgeDRM exceeds Count-only, confidence-based, and entropy-based referral by 8.32, 3.01, and 2.58 percentage points, respectively. These results support the effectiveness of density-based selection relative to both query-independent and query-dependent alternatives under the stated expert-call limits.

    3. Neighborhood averaging and the source-informed reference (Q1/Q2). To test whether averaging reachable experts can replace learned referral, we evaluate uniform and count-weighted ensembles using the same fixed expert pool. We also select the source expert as an observer-only reference. Under uniform-all arrivals on CIFAR-10/100, Dir-0.3, RR4, every source is reachable within L=2L=2:

      Rule, uniform-all / Dir-0.3 / RR4CIFAR-10 accuracyCIFAR-100 accuracyExpert access
      Full L-hop uniform ensemble48.53 ± 0.3718.03 ± 0.768 calls
      Full L-hop count-weighted ensemble46.63 ± 0.9618.05 ± 0.788 calls
      Deployed Learned + history-weighted readout51.42 ± 2.3718.46 ± 0.602.00 / 2.01 mean calls
      Reachable-source reference69.99 ± 4.8929.62 ± 0.23Requires source identity

      Learned referral achieves higher mean accuracy than either full-neighborhood ensemble while using about two expert calls instead of eight. The source-informed reference reaches 69.99% and 29.62%, compared with 51.42% and 18.46% for learned referral.

    4. What the hospital study tests (W2). The sample-count prior is part of our EdgeDRM’s proposed routing score, and the prior-off results are only an ablation of this component to show the power of our proposed prior. The hospital study evaluates the complete method when hospitals differ in training-set size and connectivity. Adding the prior increases mean accuracy from 45.74% to 47.36% on the layered graph and from 46.12% to 47.93% on the shortcut graph, demonstrating the contribution of the prior in this setting.

Reviewer_JqUBOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies inference-time collaboration in decentralized federated learning. Clients sit on a fixed communication graph; a query arriving at an entry client may be forwarded ("referred") to a neighbour at most LL times, and only the visited clients contribute to the prediction, the authors call this budgeted local expert referral. EdgeDRM trains, in Stage I, a shared-architecture encoder with private heads by local SGD plus Metropolis neighbour mixing (DFedAvgM-style, Eq. 4); in Stage II it freezes the encoders and fits the private heads together with one routing potential (bi,ϑi)(b_i,\vartheta_i) per client, learned through a balanced binary source classifier on every edge whose logit is the difference of the endpoint potentials (an exponential-tilt density-ratio model relative to a common reference; Eqs. 2, 5). At inference the current client scores each unvisited neighbour by the potential difference plus a sample-count prior log⁡(nj/ni)\log(n_j/n_i) (Eq. 6), moves greedily while the best score exceeds η\eta (Alg. 1), and the final prediction is a softmax mixture, over cumulative accepted scores, of the visited clients' heads (Sec. 3.2). Theorem 1 gives an Op(N−1/2+R−1/2)O_p(N^{-1/2}+R^{-1/2}) rate and asymptotic normality for the Stage-II coefficients under a fixed common representation and correct specification; Theorem 2 lower-bounds the referral gain in density potential by (γ−2ϵ)min⁡{L,Hϵ}(\gamma-2\epsilon)\min\{L,H_\epsilon\} under a "minimum local improvement" condition and shows that a maximiser is reached given enough budget.

Experiments on CIFAR-10/20/100 with 8 clients, two label-skew partitions and four topologies (PATH, CYCLE, RR4, COMPLETE) show EdgeDRM beating the graph-matched decentralized baselines (DFedAvgM, DisPFL, FedSPD) in all 18 sparse-graph settings (+4.2 points on average over DFedAvgM) but tying DFedAvgM on complete graphs (−0.5 points); referral improves on the no-referral variant EdgeDRM-NR by 11.9 points on average (Table 1). Ablations (Count-only and Random routing, Fig. 2; readout rules, Table 7), a λ\lambda sensitivity sweep (Table 6), a 32-client study (Table 2), a hospital-tier traffic analysis (Fig. 3, Table 4) and a small ISIC2019 study (Table 3; gains between −0.14 and +1.15 balanced-accuracy points) complete the evaluation.

Critical strengths

1. A clean, well-motivated new problem formulation. Budgeted, graph-constrained referral at inference is genuinely distinct from training-time collaboration (which decides what to share) and from centralised routing (which presumes a global gate); the hospital-referral framing makes the information constraint concrete. The one-potential-per-client parameterisation is an elegant answer to the two difficulties the paper identifies: each edge becomes a balanced two-class problem (no graph-wide normalisation), and edge comparisons telescope along a path, so greedy local decisions are globally coherent. This is a reusable idea and the reason I lean towards acceptance.

2. The empirical case that learned routing, not merely ensembling or sample size, produces the sparse-graph gains is carefully built. The fixed-expert ablation (Fig. 2) separates EdgeDRM from Count-only and Random referral; Table 7 shows the gain survives with hard-terminal prediction (so it is routing, not averaging); Table 6 shows robustness to the task/routing weight λ\lambda; Table 2 shows the pattern persists with 32 clients; and the paper is candid about where the advantage disappears (complete graphs; ISIC2019).

Critical weaknesses

1. The evaluation protocol is constructed so that the entry client is never the right expert, and neither the complementary case nor budget-matched simple alternatives are tested. Every test query is injected at a uniformly random client other than its source (Sec. 5.1; App. D.2.2 "fixed non-source arrivals"). This maximises the pay-off of moving the query, and it also explains why the personalised baselines (DisPFL, FedSPD) and EdgeDRM-NR look so poor, they are scored on out-of-distribution queries they were never meant to serve, so DFedAvgM is the only fair comparator in Table 1. The practically important "do-no-harm" case (a query arrives at its own source and the sample-count prior may still push it towards larger neighbours) is never measured, even though Fig. 3/Table 4 show the prior shifting 52–69% of all classifier calls to the large hospitals for accuracy changes (45.29 → 47.36, 45.49 → 47.93) that are within one standard deviation. Also missing are an oracle route-to-source upper bound and a uniform (or count-weighted) ensemble over the LL-hop neighbourhood, the obvious learning-free way to spend the same budget, so one cannot tell how much of the attainable gain the learned router captures. Finally, EdgeDRM receives 200 additional head/router rounds on top of the 800-round DFedAvgM control (and 50 extra on ISIC, conceded in App. D.3.2), so the comparisons are not at equal training effort. Because the paper's thesis is precisely that learned referral beats graph-matched alternatives, these gaps weigh heavily on my score.

2. The claims about real data and about theory outrun the evidence. On ISIC2019 the differences are +0.18 / +1.15 / +0.31 / −0.14 balanced-accuracy points with no variance reported (App. D.3.2 states that no cross-seed SD is given), obtained from models whose absolute balanced accuracy (~28% over 8 classes; from-scratch ResNet-18, 400 rounds) is far below what this dataset supports (FLamby's reference pipeline for Fed-ISIC2019 uses an ImageNet-pretrained EfficientNet and reports substantially higher balanced accuracy); the abstract's "benchmark and real-world medical data show that EdgeDRM outperforms graph-matched decentralized baselines" is therefore not supported for the medical data. On the theory side, Theorems 1–2 require a common representation fixed independently of the fitting data, correctly specified exponential-tilt density ratios, unpenalised SGD, and a "minimum local improvement" condition (Eq. 11) that assumes away exactly the local-optimum traps the authors themselves exhibit in App. C.4; the paper concedes that the implemented 800+200 schedule satisfies none of these (Sec. 4.1, App. D.2.1), and Theorem 2 bounds density potential rather than accuracy. More than ten pages of derivations and proofs thus add length but little insight into when the method works, whereas cheap diagnostics that would help, the fraction of queries reaching the max-potential or source client, the empirical reverse-edge discrepancy of Eq. (24), a prior-on/off ablation on CIFAR, are absent. Note also that on CIFAR-20 the sparse-graph gains over DFedAvgM are only about 1 point from 3 seeds, so "clear benefits under sparse connectivity" rests on CIFAR-10 and CIFAR-100.

Questions for authors
  1. Results when queries enter at their own source client (and under a 50/50 mixture of source and non-source entry): does referral ever hurt, how often does the route leave the source, and does removing the sample-count prior change this? This is the single most important missing experiment.
  2. Budget-matched controls on CIFAR: uniform and count-weighted ensembles over the LL-hop neighbourhood; an oracle that routes to the source client; DFedAvgM trained for 1000 rounds (equal training effort).
  3. ISIC2019: seeds and SDs for Table 3; why is balanced accuracy only ~28%; does the conclusion hold with a pretrained backbone or longer training?
  4. Per topology, what fraction of queries terminates at the max-potential client and at the source client? How large is the reverse-edge discrepancy Δ^ij+Δ^ji\hat\Delta_{ij}+\hat\Delta_{ji} (Eq. 24) in practice?
  5. Please state the exact DFedAvgM control (800 rounds? same mixing weights and client colour transforms?), confirm η=0\eta=0 in all experiments, and explain how LL should be chosen when the graph diameter is unknown.
  6. The raw query is transmitted along the route. How does this square with the decentralized-FL motivation, and what changes if only representations may move?
5 comments

We thank the reviewer for recognizing the well-motivated graph-constrained referral formulation and the coherent node-potential design. We also appreciate the positive assessment of the empirical evidence that learned routing contributes to the gains under sparse connectivity. We address source-entry behavior, comparison budgets, medical-data uncertainty, and the theoretical scope below.

  • Evaluation protocol and comparison budgets (W1/Q1/Q2). We first test source and mixed arrivals, then examine inference-time access and the additional training budget.

    1. Source-only and mixed arrivals (Q1). To test whether the gains rely on excluding source entries, we reevaluate the fixed checkpoints under four arrival protocols. The following CIFAR-10/Dir-0.3/RR4 results report three-seed accuracy.

      Entry protocolDFedAvgM1000EdgeDRM-NREdgeDRM, no priorEdgeDRM, with prior
      Source only60.86 ± 7.6769.99 ± 4.8961.16 ± 2.8961.25 ± 3.01
      Non-source only40.33 ± 2.0625.16 ± 4.2049.48 ± 1.9350.02 ± 2.30
      Uniform over all clients42.90 ± 1.1830.76 ± 3.0850.94 ± 1.9251.42 ± 2.37
      50% source / 50% uniform non-source50.60 ± 3.0747.57 ± 0.6455.32 ± 2.1555.63 ± 2.62

    Under source-only arrivals, EdgeDRM-NR achieves 69.99% accuracy versus 60.86% for DFedAvgM1000, indicating stronger performance on the experts’ own source distributions. Source-entry queries already start at an expert trained on their source distribution, leaving less room for referral to help. Under non-source-only, uniform-all, and 50/50 arrivals, referral improves on EdgeDRM-NR, and the complete EdgeDRM procedure outperforms DFedAvgM1000.

    1. Source departures and prediction outcomes (Q1). To understand the source-only results above, we examine how the routing policy handles queries that already arrive at their source expert. We track both whether the route stays at the source and how the final prediction compares with the entry prediction:

      Source-only diagnosticWith priorNo prior
      Leave source, %23.2323.35
      Terminal remains at source, %76.7776.65
      Correct entry → incorrect final, % of all records5.815.85
      Incorrect entry → correct final, % of all records0.720.73
      Actual expert calls/query1.251.25

      Under the source only arrival, most of the of queries remain at the correct source client while some leave. Across all source-entry records, 5.81% change from a correct entry prediction to an incorrect final prediction, while 0.72% change from incorrect to correct. Removing the prior produces nearly identical departure and prediction-change rates.

    2. Neighborhood ensembles and source-informed routing (Q2). To test whether averaging reachable experts can replace learned referral, we evaluate uniform and count-weighted ensembles using the same fixed expert pool. We also select the source expert as an observer-only reference. Under uniform-all arrivals on CIFAR-10/100, Dir-0.3, RR4, every source is reachable within L=2L=2:

      Rule, uniform-all / Dir-0.3 / RR4CIFAR-10 accuracyCIFAR-100 accuracyExpert access
      Full L-hop uniform ensemble48.53 ± 0.3718.03 ± 0.768 calls
      Full L-hop count-weighted ensemble46.63 ± 0.9618.05 ± 0.788 calls
      Deployed Learned + history-weighted readout51.42 ± 2.3718.46 ± 0.602.00 / 2.01 mean calls
      Reachable-source reference69.99 ± 4.8929.62 ± 0.23Requires source identity

      Learned referral achieves higher mean accuracy than either full-neighborhood ensemble while using about two expert calls instead of eight. The source-informed reference reaches 69.99% and 29.62%, compared with 51.42% and 18.46% for learned referral. This quantifies the remaining gap to a policy given source identity.

  1. Learned selection under equal expert access (Q2). The full-neighborhood ensembles above use more expert calls than a sequential route. To additionally test selection at exactly matched access, we fix the expert pool and prediction rule and require both learned and Random routing to evaluate exactly two experts and make one move, across all 24 CIFAR configurations:

    ArrivalReadoutLearnedRandomDifference, ppPositive configurations
    Uniform-allScore-weighted29.8224.45+5.3624/24
    Uniform-allHard terminal27.1121.01+6.1024/24
    Non-sourceScore-weighted28.1021.51+6.5924/24
    Non-sourceHard terminal28.4521.43+7.0224/24
    SourceScore-weighted41.8445.07−3.230/24
    SourceHard terminal17.7418.08−0.347/24

    Learned selection improves mean hard-terminal accuracy over Random by 6.10 percentage points under uniform-all arrivals, with positive differences in all 24 configurations. The policies use equal calls and only the terminal prediction, so the difference measures selection without an aggregation or access advantage. Score-weighted prediction also favors learned selection by 5.36 points.

  • Scope of the empirical and theoretical claims (W2/Q3/Q4). We address the medical-data evidence, theoretical assumptions, and requested routing diagnostics, followed by the scope of the CIFAR-20 results.
    1. Absolute accuracy, pretraining, and longer training (Q3). The FLamby comparison raises a separate question about the strength of the underlying experts. Our methods both use randomly initialized GroupNorm ResNet-18 models, whereas FLamby's reference uses an ImageNet-pretrained EfficientNet [A]. Our lesion/duplicate audit also removes 6,106 training images and reserves grouped validation data, leaving 9,986 training images. The approximately 28% result therefore belongs to a different backbone, initialization, and split protocol. These differences prevent a direct reference-pipeline comparison but do not quantify their individual effects on accuracy.

      To test the performance of our method with longer training, we continue each 400-round DFedAvgM checkpoint for 50 rounds of five full-model steps, matching the 250 local updates used for EdgeDRM's adaptation:

      GraphDFedAvgM400DFedAvgM400+50EdgeDRM400+frozen50Paired gain over continuation, pp
      PATH28.45 ± 0.7625.57 ± 0.1628.62 ± 0.68+3.05 ± 0.83
      CYCLE27.20 ± 0.3224.17 ± 0.2428.35 ± 0.71+4.18 ± 0.63
      RR327.45 ± 1.8825.22 ± 0.3827.76 ± 1.82+2.53 ± 1.52
      COMPLETE28.93 ± 0.5227.58 ± 1.1228.79 ± 0.99+1.21 ± 0.82

      EdgeDRM exceeds the continuation control on all four graphs.

    2. ISIC2019 seed variability and medical-data claims (Q3). To assess seed variability and the contribution of referral, we report three-seed balanced accuracies (mean ± sample SD), including EdgeDRM-NR. The table also includes continued DFedAvgM training to match EdgeDRM’s additional local update count, as described below.

      GraphDFedAvgM400DFedAvgM450EdgeDRM-NREdgeDRMPaired gain over DFedAvgM450, pp
      PATH28.45 ± 0.7625.57 ± 0.1625.04 ± 0.7528.62 ± 0.68+3.05 ± 0.83
      CYCLE27.20 ± 0.3224.17 ± 0.2424.37 ± 0.8028.35 ± 0.71+4.18 ± 0.63
      RR327.45 ± 1.8825.22 ± 0.3824.50 ± 1.6527.76 ± 1.82+2.53 ± 1.52
      COMPLETE28.93 ± 0.5227.58 ± 1.1225.64 ± 0.7528.79 ± 0.99+1.21 ± 0.82

      Referral improves mean balanced accuracy over EdgeDRM-NR on all four topologies. The paired-gain column reports the mean and sample SD of seed-wise differences between EdgeDRM and DFedAvgM450.

    3. Absolute accuracy, pretraining, and longer training (Q3). The FLamby comparison raises a separate question about the strength of the underlying experts. Both methods use randomly initialized GroupNorm ResNet-18 models, whereas FLamby’s reference uses an ImageNet-pretrained EfficientNet [A]. Our lesion/duplicate audit also removes 6,106 training images and reserves grouped validation data, leaving 9,986 training images. The approximately 28% result therefore belongs to a different backbone, initialization, and split protocol. These differences prevent a direct reference-pipeline comparison but do not quantify their individual effects on accuracy.

      To match the additional local update count, we continue each 400-round DFedAvgM checkpoint for 50 rounds of five full-model steps, yielding DFedAvgM450. This matches the 250 local updates used for EdgeDRM’s adaptation with frozen encoders. As shown above, EdgeDRM exceeds DFedAvgM450 in mean balanced accuracy on all four topologies; the difference is positive in all three seeds on each sparse topology.

Reviewer_9VM1OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates inference-time collaboration in decentralized federated learning (DFL), where heterogeneous clients may possess complementary expertise but cannot rely on a centralized router or global model selection. The authors formulate this problem as budgeted local expert referral, in which queries are sequentially forwarded across neighboring clients, and only visited clients contribute to the final prediction.

To address this problem, the paper proposes EdgeDRM, a decentralized routing framework based on density-ratio modeling. Each client maintains a routing potential, and differences between neighboring potentials are estimated through balanced binary source classification. During inference, a greedy referral policy selects neighboring clients according to estimated density-ratio scores, while predictions from visited clients are aggregated using history-dependent weights.

The paper provides theoretical results concerning parameter estimation and referral gains under specific statistical and structural assumptions. Experiments on CIFAR benchmarks and an ISIC2019 medical dataset demonstrate improvements over several decentralized federated learning baselines, particularly under sparse communication topologies.

Critical strengths

S1. A meaningful problem formulation for inference-time collaboration in decentralized federated learning.

The paper addresses an interesting and relatively underexplored problem: exploiting complementary client expertise at inference time without centralized coordination. The graph-constrained referral formulation explicitly considers communication topology, limited movement budgets, and restricted expert access. This setting is distinct from conventional personalized FL and centralized expert selection. The proposed approach offers a practical conceptual framework for understanding inference-time collaboration in decentralized environments.

S2. A coherent decentralized routing framework with comprehensive empirical analysis.

EdgeDRM provides a relatively simple and interpretable mechanism for constructing neighbor-local routing decisions through shared client potentials and balanced source classification. The method avoids global score normalization while maintaining a consistent parameterization of edge comparisons. The empirical evaluation covers multiple datasets, non-IID partitions, graph topologies, and client scales. The reported improvements on sparse graphs are encouraging, and the additional ablations provide useful insights into routing behavior and workload distribution.

Critical weaknesses

W1. The theoretical guarantees are weakly connected to the practical algorithm and do not establish improvements in predictive performance.

The theoretical analysis relies on restrictive assumptions that are not guaranteed by the actual implementation. In particular, Theorem 1 assumes a fixed common representation that is independent of the coefficient-fitting samples, whereas the experiments employ independently frozen local encoders obtained through finite-round decentralized training. Consequently, the learned routing comparisons may not be consistent across clients, and the statistical estimation guarantees do not directly apply to the deployed model.

Furthermore, Theorem 2 requires a uniform local-improvement condition, essentially assuming that every nonoptimal client has a neighboring client with a sufficiently higher potential. This condition rules out suboptimal local maxima and may be difficult to satisfy in general sparse communication graphs. More importantly, the theorem guarantees improvements in prior-weighted density compatibility rather than classification accuracy.

Although these limitations are acknowledged, the paper provides insufficient empirical evidence about how frequently the required assumptions hold or how violations affect routing performance. Since the main objective is to improve predictive accuracy through referral, the lack of a theoretical or empirical connection between density-based scores and actual expert competence weakens the central justification for the approach.

W2. The experimental evaluation does not sufficiently establish the effectiveness and generalizability of the proposed routing mechanism.

First, the CIFAR experiments rely on artificially constructed client heterogeneity, including label-skew partitions and client-specific image transformations. Moreover, each test query is deliberately assigned to a non-source entry client. While this setting highlights the potential benefits of referral, it may favor methods that identify the query's source distribution and does not adequately represent more general inference scenarios. Additional evaluations involving naturally occurring heterogeneity, varying entry distributions, and previously unseen query distributions would be valuable.

Second, the main comparisons primarily involve decentralized FL methods that perform prediction at the entry client without inference-time referral. These baselines do not have equivalent access to multiple experts. Although the paper includes random-routing and count-only ablations, these are limited to selected benchmark settings and do not constitute comprehensive comparisons against alternative query-dependent routing policies. Stronger baselines, such as confidence-based referral, prototype-based expert selection, or uncertainty-guided routing, should be evaluated under matched inference budgets.

Questions for authors
  1. Relationship between density compatibility and predictive expertise. Under what conditions does a higher density-ratio routing score imply better predictive performance? Can the authors provide empirical analyses correlating the estimated routing scores with client-specific prediction accuracy or loss? How frequently does the selected referral destination actually outperform the current expert?

  2. Representation inconsistency. The theoretical analysis assumes a common encoder, while the implementation uses different frozen local encoders. Have the authors measured representation disagreement or violations of edge-score antisymmetry as discussed in Appendix C.2? How would performance change if a common encoder or an additional representation-alignment stage were used?

  3. Stronger routing baselines and inference budgets. How does EdgeDRM compare with confidence-based, uncertainty-based, or prototype-based neighbor referral methods when all approaches are allowed the same number of expert evaluations and communication hops? Could the authors extend the random-routing and count-only controls to all main experimental configurations using the same history-aggregation rule?

5 comments

We thank the reviewer for recognizing the value of inference-time collaboration in decentralized FL and the neighbor-local routing formulation. We clarify the theoretical scope and present additional evaluations of arrival distributions, expert selection, and routing baselines below.

  • Theoretical scope and representation consistency (W1/Q2). Theorem 1 studies coefficient estimation with a common representation fixed independently of the fitting samples, following the classical density-ratio framework with known transformations [1]. With fixed features, Eq. (19) is convex in the coefficients, and its graph-local likelihood and sandwich covariance follow a composite-likelihood structure [2]. Stage II freezes the local encoders, preserving coefficient-objective convexity, but does not ensure common or sample-independent representations. Extending the guarantee requires additional control of sample reuse and representation-learning error [3], including their effects on coefficient fitting and edge comparisons. We leave such theoretical guarantees for future analysis.

    As for the theoretical assumptions, we agree that we adopt a local-improvement condition related to the graph-unimodality. This is a general assumption widely used in structured bandits, cf., [4]: every nonoptimal node has a neighboring node with higher potential. This excludes suboptimal local maxima, while the ascent margin γ\gamma quantifies the improvement available at each step.

    Measured edge-score inconsistency (Q2). To assess whether the local encoders produce consistent comparisons, we measure Dij(x)=∣Δ^ij(x)+Δ^ji(x)∣D_{ij}(x)=|\widehat\Delta_{ij}(x)+\widehat\Delta_{ji}(x)| on held-out queries empirically. Results for CIFAR-10 and ISIC2019 are reported below. The full table appears in Appendix D.4 of the revised manuscript.

    Dataset / partitionGraphMean discrepancyMedian
    CIFAR-10 / Dir-0.3PATH0.08263 ± 0.035730.05331 ± 0.02325
    CIFAR-10 / Dir-0.3CYCLE0.03147 ± 0.018000.01881 ± 0.00637
    CIFAR-10 / Dir-0.3RR40.00138 ± 0.000850.00085 ± 0.00049
    CIFAR-10 / Dir-0.3COMPLETE0.00000 ± 0.000000.00000 ± 0.00000
    ISIC2019PATH0.03632 ± 0.008010.02745 ± 0.00537
    ISIC2019CYCLE0.00726 ± 0.001140.00571 ± 0.00058
    ISIC2019RR30.00091 ± 0.000030.00054 ± 0.00009
    ISIC2019COMPLETE0.00000 ± 0.000000.00000 ± 0.00000

    With a shared encoder, the two directional edge scores are negatives of one another, yielding Dij(x)=0D_{ij}(x)=0. COMPLETE realizes this reference case through exact full-model averaging before freezing. COMPLETE also provides a common-encoder reference: referral increases accuracy from 32.89% at entry to 56.85% at the terminal expert, as reported below.

    On the other graphs, discrepancies also remain close to zero despite differences between local encoders, with larger values on PATH and CYCLE than on RR4/RR3. These small departures from the shared-encoder reference suggest that, although the local encoders are not identical, their representations remain similar along the directions relevant to the routing scores. Consequently, encoder heterogeneity introduces only small inconsistencies in the measured pairwise density comparisons.

  • Relationship between density compatibility and predictive expertise (Q1). We address your 3 questions in turn.
    1. When do density scores improve prediction? EdgeDRM uses density-ratio scores to select a sequence of experts and weight their predictions along the visited route. The final prediction therefore retains information from earlier experts rather than relying exclusively on the terminal expert. Under a common representation and exact density-ratio scores, the history weights are proportional to the prior-weighted densities of the visited clients. With access to all clients, correctly matched source priors, and exact conditional-label models, this weighting recovers the conditional prediction of the full mixture distribution. This provides a probabilistic motivation for the weighting rule. We emphasize that this claim does not establish that every higher-scoring expert has lower loss or that every referral improves prediction.

    2. Correlation between routing scores and loss. Following your suggestion, we compared the estimated routing score Δ^ij(x)\widehat\Delta_{ij}(x) with the reduction in true-label cross-entropy, ℓi(x,y)−ℓj(x,y)\ell_i(x,y)-\ell_j(x,y), for all 16,004 accepted hops on CIFAR-100/25-SPC with RR4. The figure in Appendix D.2.3 (or in this link) groups hops by whether the current expert and its neighbor predicted the correct class. This distinction matters because the same query can contribute to more than one panel. For example, on a wrong → wrong → correct route, the first hop appears in panel (d) and the second in panel (a). The 75.2% in panel (d) thus counts hops between two incorrect experts, even when a later expert gets the query right.

      Panel (a) isolates the 2,845 hops that corrected an error (17.8% of all hops). The neighbor assigned a higher probability to the true class on 2,841 of these hops. Within this group, the routing score and the loss reduction showed a clear negative correlation (Pearson r=−0.58r=-0.58). In other words, even a modest routing score could accompany a large increase in the probability assigned to the true class, while higher scores tended to accompany smaller reductions in loss. For comparison, 720 hops (4.5%) changed a correct prediction to an incorrect one (panel (b)).

    3. Does the selected expert outperform the entry expert? We compare the terminal expert selected by referral with the original entry expert under uniform-all arrivals.

      GraphEntry accuracyHard-terminal accuracyTerminal loss lower than or equal entry, %
      PATH27.95 ± 3.4336.74 ± 1.9777.27 ± 0.23
      CYCLE28.73 ± 3.3138.77 ± 1.1675.48 ± 0.84
      RR430.76 ± 3.0851.04 ± 2.3975.06 ± 1.47
      COMPLETE32.89 ± 2.9356.85 ± 2.8674.22 ± 2.47

      The terminal expert achieves substantially higher mean accuracy than the entry expert across all 4 topologies, supporting the predictive benefit of referral in aggregate. The last column reports the percentage of queries for which the terminal expert has no higher true-label cross-entropy than the entry expert, indicating that most of the referrals eventually have a lower loss.

  • Experimental generalizability and routing comparisons (W2/Q3). We address two aspects of the concern: whether the gains persist beyond the original non-source evaluation setting, and whether learned routing improves prediction beyond simply accessing additional experts.

    1. Performance beyond non-source arrivals and stronger routing controls (W2). To test whether the gains rely on excluding source entries, we reevaluate the fixed checkpoints under four arrival protocols. The following CIFAR-10/Dir-0.3/RR4 results report three-seed accuracy.
    Entry protocolDFedAvgM1000EdgeDRM-NREdgeDRM, no priorEdgeDRM, with prior
    Source only60.86 ± 7.6769.99 ± 4.8961.16 ± 2.8961.25 ± 3.01
    Non-source only40.33 ± 2.0625.16 ± 4.2049.48 ± 1.9350.02 ± 2.30
    Uniform over all clients42.90 ± 1.1830.76 ± 3.0850.94 ± 1.9251.42 ± 2.37
    50% source / 50% uniform non-source50.60 ± 3.0747.57 ± 0.6455.32 ± 2.1555.63 ± 2.62

    Under source-only arrivals, EdgeDRM-NR achieves 69.99% accuracy versus 60.86% for DFedAvgM1000, indicating stronger performance on the experts’ own source distributions. Source-entry queries already start at an expert trained on their source distribution, leaving less room for referral to help. Under non-source-only, uniform-all, and 50/50 arrivals, referral improves on EdgeDRM-NR, and the complete EdgeDRM procedure outperforms DFedAvgM1000.

    1. Evaluation under natural heterogeneity (W2). To assess performance beyond constructed heterogeneity, ISIC2019 uses clients defined by real data sources and acquisition settings, with each query evaluated at every entry client. The balanced accuracies are:

      GraphDFedAvgM450EdgeDRM-NREdgeDRM (our)
      PATH25.57 ± 0.1625.04 ± 0.7528.62 ± 0.68
      CYCLE24.17 ± 0.2424.37 ± 0.8028.35 ± 0.71
      RR325.22 ± 0.3824.50 ± 1.6527.76 ± 1.82
      COMPLETE27.58 ± 1.1225.64 ± 0.7528.79 ± 0.99

      Referral improves on EdgeDRM-NR on all 4 topologies, and EdgeDRM improves over DFedAvgM450 on the all graphs. These results extend the evidence to the real-world dataset with natural heterogeneity.

    2. Learned selection under equal expert access (W2/Q3). The preceding comparisons evaluate the complete method, but comparisons with entry-only baselines cannot isolate the contribution of learned selection. To address this, we hold the expert pool fixed and compare learned and random routing with exactly two expert calls and one move to a neighbor, using the same prediction rule. We extend this control to all 24 CIFAR configurations.

      ArrivalReadoutLearnedRandomDifference, ppPositive configurations
      Uniform-allScore-weighted29.8224.45+5.3624/24
      Uniform-allHard terminal27.1121.01+6.1024/24
      Non-sourceScore-weighted28.1021.51+6.5924/24
      Non-sourceHard terminal28.4521.43+7.0224/24
      SourceScore-weighted41.8445.07−3.230/24
      SourceHard terminal17.7418.08−0.347/24

      Under uniform-all arrivals, learned routing improves hard-terminal accuracy by 6.10, with positive differences in all 24 configurations. Both policies evaluate two experts and predict using only the terminal expert, so this comparison isolates the benefit of learned selection from additional expert access and prediction aggregation. The advantage also holds with the same score-weighted aggregation rule.

    3. Comparisons with query-dependent routing policies (W2/Q3). The learned-versus-random comparison establishes a benefit from selection under controlled access. To test whether that benefit extends to stronger alternatives, we additionally evaluate confidence- and entropy-based referral and extend Count-only referral across all 24 CIFAR configurations. All use the same history-aggregation rule. For confidence and entropy routing, every candidate evaluation counts toward the L+1L+1 expert-call cap.

      ComparatorLearned minus comparator, pp (score-weighted)
      Count-only+8.32
      Confidence, L+1L+1 expert-call cap+3.01
      Entropy, L+1L+1 expert-call cap+2.58

      Under uniform-all arrivals, EdgeDRM exceeds Count-only, confidence-based, and entropy-based referral by 8.32, 3.01, and 2.58 percentage points, respectively. These results support the effectiveness of density-based selection relative to both query-independent and query-dependent alternatives under the stated expert-call limits.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.