- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies inference-time collaboration in decentralized federated learning (DFL). Instead of evaluating each test query only at the client where it arrives, the authors formulate "budgeted local expert referral": a query may move at most L hops along the communication graph, and only visited clients contribute to the final prediction. The proposed EdgeDRM adapts the density-ratio model (DRM) of FedDRM to this setting by replacing the graph-wide m-class source classification with balanced binary source classification on each edge, while sharing one routing potential per client so that edge comparisons telescope along a path. At inference, a sample-count prior log(n_j/n_i) is added, the query greedily moves to the best unvisited neighbor, and visited predictions are aggregated with softmax weights of the cumulative scores. Theory gives an estimation rate and asymptotic normality for the Stage-II coefficients under a fixed common encoder (Thm 1) and a lower bound on the referral gain under a minimum-local-improvement condition (Thm 2). Experiments cover 8-client CIFAR-10/20/100 on four topologies, a 32-client scaling study, a tiered-hospital simulation, and ISIC2019.
The problem formulation is new for DFL and clearly defined, and the design is clean: one potential per client makes edge comparisons coherent across hops, and the separation between the balanced training logit and the inference-time prior is consistently explained. The paper is also unusually candid about its scope (EL equivalence only after profiling intercepts, theory not covering the experimental training, no prediction-risk guarantee, no advantage on COMPLETE graphs), and the ISIC2019 split audit that removes lesion-ID leakage is a useful contribution in itself.
However, the evaluation protocol may favor referral by construction, and the evidence that learned density-ratio routing is the key driver is limited. Queries are always assigned to a non-source entry client while each client has a unique deterministic color transform, so the source client is easy to identify from the input; even random referral improves over no referral by about 6 points. Strong neighbor-only baselines are missing, improvements are often within seed variance, and on real heterogeneity (ISIC2019) the gains shrink to 0–1 points without reported variance. The theory relies on assumptions (common encoder, no non-global local maxima, positive threshold) that the experiments do not satisfy or measure.
Critical strengths
- Inference-time referral as a graph-constrained, sequential problem: The paper observes that in DFL the topology limits not only how models are mixed during training but also which clients a test query can reach, and formulates this as budgeted local expert referral (budget L, no revisits, only visited clients predict; Sec. 2, Alg. 1). This turns "which expert should answer" from a one-shot global selection, as in FedDRM or federated MoE, into a path-dependent search under neighbor-only communication. It is a natural and underexplored question for decentralized systems, and the formulation can be reused independently of the specific router.
- Node potentials make local comparisons globally coherent: Instead of training unrelated edge classifiers, EdgeDRM shares one routing potential per client, so each edge logit is a difference of potentials and comparisons telescope along a route (App. C.2). Greedy multi-hop routing therefore ascends a single quantity, and the balanced training logit is cleanly separated from the sample-count prior, which is added exactly once at inference (Eq. 5–6, App. B.4). The principle is simple and fits the neighbor-only constraint well.
Critical weaknesses
- The router targets the likely source, not the better expert, and the CIFAR protocol makes the two coincide by construction: The motivation is expertise, i.e., a query should reach a client that handles it better (Sec. 1, Fig. 1). The learned potential, however, estimates source density, and the paper itself states that the bound on Γ_L gives no ordering of prediction risk between the entry, terminal, and history predictions (C.4, "Prediction target"). "Most likely source = better expert" is thus the central assumption, and the evaluation builds it in rather than testing it.
- Every query enters at a non-source client (Sec. 5.1), each client has a unique deterministic color transform (D.1), and heads are personalized, so the source is easy to identify and its head is the natural expert by design.
- The results fit this reading: Random referral already beats EdgeDRM-NR by about 6 points (31.6 vs. 25.4, Fig. 2), EdgeDRM-NR is far below DFedAvgM (25.35 vs. 40.10, RR4 CIFAR-10 Dir, Table 1), and NR drops monotonically as heads become more local (35.01 → 24.97, Table 6). Much of the "referral gain" may be recovering the loss that Stage II personalization creates at a non-source entry.
- On ISIC2019, with real center heterogeneity and source entries allowed (Sec. 5.4), the gains over DFedAvgM are +0.18/+1.15/+0.31/−0.14 points (Table 3), without SD and at unequal training effort (400+50 vs. 400 rounds, D.3.2).
- The evidence does not isolate the learned density ratio as the driver: It remains unclear whether the gain comes from density-ratio routing or simply from moving the query.
- Only Random and Count-only routing are compared under the same graph and budget (Fig. 2). Since history averaging adds little over the hard terminal prediction (17.59 vs. 17.28, Table 7), the gain is almost entirely route selection, so confidence/entropy-based referral, averaging within L hops, or the same rule on DFedAvgM models are the decisive comparisons. Some gains are also within seed variance (PATH CIFAR-10 Dir: 33.08 ± 3.53 vs. 28.55 ± 7.62).
- Non-shared encoders weaken the density-ratio interpretation: the Stage-II edge classifier sees client j's samples through g_θj (Eq. 15), but at inference both sides are scored with g_θi (Eq. 6), and no encoder consensus is applied (D.1). Neither δ nor |Δ̂_ij + Δ̂_ji| is measured, although the sparse graphs where EdgeDRM helps are where encoders differ most.
- The hospital study cannot test learned routing, since hospitals hold class-balanced CIFAR-10 subsets (D.1); without the prior, referral adds only 0.45–0.63 points over NR (Table 4).
- The theory does not cover the regime the method relies on: The guarantees are stated for a setting that the experiments do not satisfy, so they do not support the empirical claims.
- Thm 1 is a standard M-estimation result under a common, fixed encoder, while the experiments use different encoders per client (D.1). Condition (11) essentially rules out non-global local maxima, from which greedy ascent follows almost directly (C.3).
- Thm 2 requires η ≥ ϵ > 0 while the experiments use a zero threshold (D.2.3, D.3.2), and it bounds a density gain Γ_L, not accuracy (C.4); the abstract and contributions should say so clearly.
Questions for authors
- Source identification: What fraction of routes terminate at the query's true source client? Could you report Table 1 under uniform entry including the source client, and in a label-skew-only setting without per-client color transforms? An oracle "route to source" upper bound would also help interpret the gains.
- Neighbor-only baselines: How does EdgeDRM compare, under the same graph and budget, with confidence/entropy-based greedy referral, averaging the models within L hops, and the same routing rule applied to DFedAvgM models?
- Representation disagreement: How large are δ and |Δ̂_ij + Δ̂_ji| on the sparse graphs? Does EdgeDRM still help with a shared frozen encoder (e.g., exact consensus before Stage II)?
- Training effort and tuning: Is the CIFAR DFedAvgM baseline the 800-round prefix or a 1000-round model, and how were baselines tuned? Why does DFedAvgM on COMPLETE (exact averaging) reach only 55% on CIFAR-10 Dir-0.3?
- Variance on real data: Can you report per-seed results, SDs, and EdgeDRM-NR for ISIC2019 (Table 3), and sensitivity of CIFAR results to η > 0 and L?
9 comments
-
Source recovery under graph constraints (W1/Q1). You ask how often routing reaches the query's true source, given the concern that the CIFAR gains may mainly reflect source recovery. We therefore measure the fraction of routes that terminate at the source under uniform-all arrivals. The query's source identity is used for this diagnostic and is not provided to the deployed router:
Graph CIFAR-10 / Dir-0.3 terminal-source (%) ISIC2019 terminal-source (%) PATH 32.30 ± 1.15 37.33 ± 0.57 CYCLE 35.57 ± 1.18 44.44 ± 0.40 RR4 / RR3 58.01 ± 3.53 53.59 ± 0.25 COMPLETE 69.40 ± 3.07 66.78 ± 0.41 Source recovery is higher on RR4/RR3 and COMPLETE than on PATH and CYCLE. This pattern is consistent with the constraints of neighbor-only search: a route can stop before reaching the source when no unvisited neighbor has an acceptable score. Even on COMPLETE, where every client is directly accessible, only 69.40%/66.78% of CIFAR-10/ISIC2019 routes end at the source. EdgeDRM selects clients by their estimated prior-weighted density, whose maximum need not coincide with the query's recorded source. Together, source-recovery rates alone do not establish whether prediction gains come from routes ending at non-source experts.
-
Source-inclusive arrivals and personalization recovery (W1/Q1). To test whether the gains rely on excluding source entries, we reevaluate the fixed checkpoints under four arrival protocols. The following CIFAR-10/Dir-0.3/RR4 results report three-seed accuracy.
Entry protocol DFedAvgM1000 EdgeDRM-NR EdgeDRM, no prior EdgeDRM, with prior Source only 60.86 ± 7.67 69.99 ± 4.89 61.16 ± 2.89 61.25 ± 3.01 Non-source only 40.33 ± 2.06 25.16 ± 4.20 49.48 ± 1.93 50.02 ± 2.30 Uniform over all clients 42.90 ± 1.18 30.76 ± 3.08 50.94 ± 1.92 51.42 ± 2.37 50% source / 50% uniform non-source 50.60 ± 3.07 47.57 ± 0.64 55.32 ± 2.15 55.63 ± 2.62 Under source-only arrivals, EdgeDRM-NR achieves 69.99% accuracy versus 60.86% for DFedAvgM1000, indicating stronger performance on the experts’ own source distributions. Source-entry queries already start at an expert trained on their source distribution, leaving less room for referral to help. Under non-source-only, uniform-all, and 50/50 arrivals, referral improves on EdgeDRM-NR, and the complete EdgeDRM procedure outperforms DFedAvgM1000.
-
Why use client-specific transformations? (W1/Q1). The transformations create controlled feature shifts following FedDRM [A], alongside the label-skew partitions; acquisition-related feature heterogeneity is also studied in FedBN [B]. They let us study partition and topology effects under a reproducible feature shift. The existing CIFAR results therefore concern joint feature and label heterogeneity. Label-skew-only results require retraining of all baselines and need more time to obtain, and are thus not included. ISIC2019 provides complementary evidence under natural source/acquisition heterogeneity, with its uncertainty assessed under Q5.
- Learned selection and neighbor-only routing alternatives (W2/Q2). The preceding comparisons evaluate whether the complete system improves prediction. To identify the contribution of the routing rule, we now hold the trained experts fixed and compare different ways to select or combine them.
-
Learned selection with equal expert access (W2/Q2). To test selection at exactly matched access, we fix the expert pool and prediction rule and require both learned and Random routing to evaluate exactly two experts and make one move, across all 24 CIFAR configurations:
A common maximum hop budget need not produce equal realized access: under the deployed rules and uniform-all arrivals, Random uses 5.34 expert calls versus 1.81 for learned routing, with 1.30 points higher mean score-weighted accuracy. The comparison below instead fixes actual access.
Arrival Readout Learned Random Difference, pp Positive configurations Uniform-all Score-weighted 29.82 24.45 +5.36 24/24 Uniform-all Hard terminal 27.11 21.01 +6.10 24/24 Non-source Score-weighted 28.10 21.51 +6.59 24/24 Non-source Hard terminal 28.45 21.43 +7.02 24/24 Source Score-weighted 41.84 45.07 −3.23 0/24 Source Hard terminal 17.74 18.08 −0.34 7/24 Learned selection improves mean hard-terminal accuracy over Random by 6.10 percentage points under uniform-all arrivals, with positive differences in all 24 configurations. The policies use equal calls and only the terminal prediction, so the difference measures selection without an aggregation or access advantage. Score-weighted prediction also favors learned selection by 5.36 points.
-
Comparison with confidence and entropy (W2/Q2). The learned-versus-random comparison establishes a benefit from selection under controlled access. To test whether that benefit extends to stronger alternatives, we additionally evaluate confidence- and entropy-based referral and extend Count-only referral across all 24 CIFAR configurations. All use the same history-aggregation rule. For confidence and entropy routing, every candidate evaluation counts toward the expert-call cap.
Comparator Learned minus comparator, pp (score-weighted) Count-only +8.32 Confidence, expert-call cap +3.01 Entropy, expert-call cap +2.58 Under uniform-all arrivals, EdgeDRM exceeds Count-only, confidence-based, and entropy-based referral by 8.32, 3.01, and 2.58 percentage points, respectively. These results support the effectiveness of density-based selection relative to both query-independent and query-dependent alternatives under the stated expert-call limits.
-
Neighborhood averaging and the source-informed reference (Q1/Q2). To test whether averaging reachable experts can replace learned referral, we evaluate uniform and count-weighted ensembles using the same fixed expert pool. We also select the source expert as an observer-only reference. Under uniform-all arrivals on CIFAR-10/100, Dir-0.3, RR4, every source is reachable within :
Rule, uniform-all / Dir-0.3 / RR4 CIFAR-10 accuracy CIFAR-100 accuracy Expert access Full L-hop uniform ensemble 48.53 ± 0.37 18.03 ± 0.76 8 calls Full L-hop count-weighted ensemble 46.63 ± 0.96 18.05 ± 0.78 8 calls Deployed Learned + history-weighted readout 51.42 ± 2.37 18.46 ± 0.60 2.00 / 2.01 mean calls Reachable-source reference 69.99 ± 4.89 29.62 ± 0.23 Requires source identity Learned referral achieves higher mean accuracy than either full-neighborhood ensemble while using about two expert calls instead of eight. The source-informed reference reaches 69.99% and 29.62%, compared with 51.42% and 18.46% for learned referral.
-
What the hospital study tests (W2). The sample-count prior is part of our EdgeDRM’s proposed routing score, and the prior-off results are only an ablation of this component to show the power of our proposed prior. The hospital study evaluates the complete method when hospitals differ in training-set size and connectivity. Adding the prior increases mean accuracy from 45.74% to 47.36% on the layered graph and from 46.12% to 47.93% on the shortcut graph, demonstrating the contribution of the prior in this setting.
-
We thank the reviewer for recognizing the graph-constrained formulation, the node-potential design, and the split audit. We address the questions about predictive expertise, the contribution of learned routing, representation consistency, training effort, and empirical uncertainty below.
Density-guided prediction and expert selection (W1). EdgeDRM uses density-ratio scores to select a sequence of experts and weight their predictions along the visited route. The final prediction therefore retains information from earlier experts rather than relying exclusively on the terminal expert. Under a common representation and exact density-ratio scores, the history weights are proportional to the prior-weighted densities of the visited clients. With access to all clients, correctly matched source priors, and exact conditional-label models, this weighting recovers the conditional prediction of the full mixture distribution. This provides a probabilistic motivation for the weighting rule. We emphasize that this claim does not establish that every higher-scoring expert has lower loss or that every referral improves prediction.
Prediction changes on accepted referrals. To examine the predictive effect of accepted referrals, we compared the estimated routing score Δij(x) with the reduction in true-label cross-entropy, ℓi(x,y)−ℓj(x,y), for all 16,004 accepted hops on CIFAR-100/25-SPC with RR4. The figure in Appendix D.2.3 (or in this link) groups hops by whether the current expert and its neighbor predicted the correct class. This distinction matters because the same query can contribute to more than one panel. For example, on a wrong → wrong → correct route, the first hop appears in panel (d) and the second in panel (a). The 75.2% in panel (d) thus counts hops between two incorrect experts, even when a later expert gets the query right.
Panel (a) isolates the 2,845 hops that corrected an error (17.8% of all hops). The neighbor assigned a higher probability to the true class on 2,841 of these hops. Within this group, the routing score and the loss reduction showed a clear negative correlation (Pearson r=−0.58). In other words, even a modest routing score could accompany a large increase in the probability assigned to the true class, while higher scores tended to accompany smaller reductions in loss. For comparison, 720 hops (4.5%) changed a correct prediction to an incorrect one (panel (b)).
Does the selected expert outperform the entry expert? We compare the terminal expert selected by referral with the original entry expert under uniform-all arrivals on CIFAR-10/Dir-0.3.
The terminal expert achieves substantially higher mean accuracy than the entry expert across all 4 topologies, supporting the predictive benefit of referral in aggregate. The last column reports the percentage of queries for which the terminal expert has no higher true-label cross-entropy than the entry expert, indicating that most of the referrals eventually have a lower loss.