acceptodds
Under review as a conference paper at ICLR 2027

Conservation Is Not Allocation: Valuing Clients and Edges in Hierarchical Federated Learning

Abstract

Hierarchical federated learning routes client updates through edge servers, so contribution accounting must value both clients and edges. A common shortcut sums client values within each edge. For exact Owen values under a fixed partition this sum equals the edge's Shapley value in the quotient game, yet practical methods estimate client values from sampled permutations. We show that the summed ledger is then a Monte Carlo estimator of the quotient game whose efficiency residual is identically zero, so the total is always conserved while individual edges can be misallocated without any visible signal. We propose Federated Composable Contribution Evaluation (FedCCE), which gives each resolution its own estimator. Clients upload per-class feature summaries whose counts, means, and variances compose exactly under union, so the utility of any coalition of clients or edges follows from cached summaries without retraining. FedCCE samples client Owen values under a fixed budget and solves the much smaller edge quotient game exactly, and we prove that the extra utility calls this requires depend on the number of edges but not on the number of clients. Across four image benchmarks, summing budgeted client values identifies the most valuable edge in only 46% of 232 runs with 50 or 100 clients even at the largest budget, whereas FedCCE identifies it in every run and needs roughly 700 fewer utility evaluations than computing both ledgers exactly at 100 clients.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Federeted Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_nwzbOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies contribution accounting in hierarchical federated learning at both client and edge levels. It shows that summing sampled client Owen values can preserve total contribution exactly while misallocating edge shares. FedCCE uses composable class-wise feature summaries to evaluate coalitions without retraining, samples client Owen values, and computes edge Shapley values through exact enumeration or direct edge-order sampling. The paper provides theoretical error and cost analyses and evaluates attribution fidelity, harmful-client detection, and computational efficiency.

Critical strengths

1.The paper identifies an important pitfall: exact conservation of total contribution does not ensure accurate edge-level allocation.

2.The resolution-matched design is intuitive, using sampling for the large client game and exact enumeration for the smaller edge game.

3.Composable feature summaries and a shared utility cache enable efficient coalition evaluation without repeated retraining.

4.Theoretical analyses characterize edge-level estimation error and the additional cost of exact edge accounting.

5.The experiments cover attribution accuracy, top-edge identification, label-corruption detection, and computational efficiency.

Critical weaknesses

1.The practical validity of the declared summary game is not yet sufficiently established. FedCCE is exact for a nearest-prototype utility constructed from feature moments, but this utility is not the utility of the federated model being trained. The reported agreement with model-based attribution is only moderate, e.g., the average edge correlation with exact Model-Owen is 0.550, and may vary considerably across datasets. Therefore, exact computation of this surrogate game does not necessarily imply accurate attribution of the deployed model’s behavior.

2.Several headline comparisons conflate estimator quality with utility choice. In Table 1, model-, gradient-, and prototype-signal methods are evaluated against the exact hierarchical Proto-Owen value of FedCCE’s summary game, even though they target different characteristic functions. These results show cross-utility agreement rather than estimation error and therefore provide limited evidence that FedCCE is a better contribution estimator than those methods. The strongest comparisons should hold the target utility fixed.

3.Exact edge enumeration remains exponential in the number of active edges. Proposition 2 shows that its incremental cost is independent of the number of clients, but it still depends exponentially on E, and exact enumeration is evaluated only up to ten edges. The proposed direct edge-sampling fallback for larger hierarchies is theoretically discussed but not experimentally evaluated, leaving scalability to realistic deployments with tens or hundreds of edges unclear.

4.The evidence-preparation mechanism introduces nontrivial privacy and trust assumptions. Per-client class counts, means, and diagonal variances expose class presence and feature-distribution information to the cloud without differential privacy, secure aggregation, or robustness against strategic reports. Because contribution values may determine rewards or sanctions, clients also have an incentive to manipulate these summaries. The current experiments assume honest, noiseless reporting.

5.The temporal-ledger and partial-reporting mechanisms receive limited validation. Deployment-scale experiments have every client upload a summary in each accounting round, whereas absent clients are handled by a simple historical-mean rule and inactive edges remain unobserved. Thus, the experiments do not adequately test the setting for which temporal completion is introduced. Moreover, averaging edge values over rounds may become difficult to interpret when client-to-edge memberships change.

6.Some empirical conclusions are stronger than the available independent evidence. FedCCE identifies the highest-valued edge in every run because it exactly enumerates the quotient game defining the reference; this establishes solver correctness but not that the selected edge is truly most valuable to the trained federated model. The corrupted-client intervention uses only three seeds, and the advantage of the direct edge ledger in corrupted-edge ranking is reported as statistically insignificant.

7.The method depends on a centralized validation view and a shared encoder, but sensitivity to the size, distribution, and representativeness of this validation set is not studied. A shifted or unbalanced validation view could materially change both client and edge values, particularly because coalition utility is measured through nearest-prototype accuracy rather than the trained classifier.

Questions for authors

1.Can the authors more clearly separate exact estimation of the declared summary game from faithful attribution of the trained federated model? In particular, how should practitioners interpret FedCCE values when summary-game and Model-Owen rankings disagree?

2.Could the authors provide controlled comparisons in which FedCCE, Model-Owen, TMC, and other estimators all query the same characteristic function? Conversely, could the FedCCE resolution-matched solver be applied to a model-based utility on tractable deployments so that estimator error is evaluated without changing the target?

3.How does the direct edge-sampling fallback perform when E is too large for enumeration? Experiments with substantially more than ten edges, including allocation error, top-edge recovery, and wall-clock cost, would clarify whether resolution matching remains effective at realistic hierarchy sizes.

4.Can the authors evaluate partial summary participation directly by varying RtR_t? It would be useful to report the accuracy of the historical client-completion rule, the frequency of unobserved edges, and ledger stability as summary-reporting rates decrease.

5.What information about client labels and feature distributions can be inferred from the uploaded class summaries? How would secure aggregation or differential privacy affect exact composability, utility accuracy, and contribution rankings?

6.How robust is FedCCE to strategic or corrupted summary reports? Since contribution values may determine rewards, can a client increase its assigned value by manipulating class counts, means, or variances without improving the global model?

7.Could the authors report additional seeds or confidence intervals for the removal intervention and corrupted-edge experiments? In particular, can they clarify which downstream improvements are statistically significant rather than illustrative case studies?

8 comments

Thank you for the thorough and constructive review, and for your positive assessment of the novelty of our work and the comprehensiveness of its theory and experiments. Your comments have helped us state the claims of the paper more precisely. We address each weakness at the level of the claims and the revised text. The supporting measurements appear in our answers to Q1 to Q7. Below, N denotes the number of clients and E the number of edges.

W1. Practical validity of the summary game.

Thank you for raising this central point. FedCCE separates accurate accounting for a declared utility from independent tests of its usefulness. The summary game has two complementary foundations.

  • It is a standard valuation target. As in data valuation, the utility of a coalition is the validation accuracy of a fixed learner built from its data. FedCCE evaluates coalition utility using a variance-weighted nearest-prototype classifier in the shared encoder’s feature space. The class summaries are sufficient statistics of this learner, which is why every coalition value is exact without retraining.
  • Its decisions improve the trained model. Over 10 training seeds, removing the clients with the lowest values raises the final test accuracy of the trained model by 4.10 percentage points and removes all 20 corrupted clients. Over 80 deployments, exact edge values reach a Spearman rank correlation of 0.54 with the negative normalized label-distribution shift, against 0.17 for summation (Q7). Corrupted-client identification and post-removal test accuracy provide independent evidence of the scores’ practical usefulness.

The edge agreement of 0.550 with the model game, and its variation across datasets, measures agreement between the two value definitions (Q1). Section 4.2 and Appendix A.14 distinguish the valuation targets. Section 4.4 and Appendix A.15 report the removal intervention.

W2. Estimator quality versus utility choice.

Thank you for this distinction, which we agree should be carried through the whole paper. We would like to clarify that the headline comparisons are made with the utility held fixed. The finding that summation recovers the leading edge in only 46% of 232 runs even at the largest budget, and the 700× saving over exact computation, both compare estimators of the same summary game. The former measures allocation accuracy, and the latter measures query efficiency. We agree that the model, gradient, and prototype rows of Table 1 measure agreement across utilities. The revision separates them from the same-target rows and does not use them as evidence that FedCCE estimates better than methods with other targets. The new controls in Q2 hold the target fixed in both directions, and the advantage of the resolution-matched solver persists under a model utility.

W3. Scaling with the number of edges.

Thank you for this question about scale. The exponential term counts the active edges of one accounting round and does not grow with N (Proposition 2). Its cost is bounded in advance by the number of edge unions, so the operator can choose exact or sampled edges before accounting starts. Direct edge sampling is part of FedCCE, and Corollary 1 gives its variance. Both regimes are now tested well beyond ten edges (Q3). Exact enumeration finishes within 5.8 minutes up to E=20, outside the training loop. At E=25 and E=50, direct sampling lowers measured normalized L1 error by 49.6% and 34.3%, and its edge pass takes under five seconds. Corollary 1 gives the corresponding standard-error scaling. Each sampled edge order costs E utility queries regardless of N, so its query count is determined by its own sampling budget. The revision adds these results to Section 4.5 and the appendix.

W4. Privacy and trust assumptions.

Thank you for raising these important concerns. We clarify the trust assumptions below.

  • Who sees client summaries. The exact edge ledger needs only per-edge aggregates, the same granularity at which the cloud receives models in hierarchical FL. The client ledger needs individual summaries, but every coalition in the Owen value of a client consists of whole other edges plus clients of its own edge. Each edge server, which already receives the individual updates of its clients, can therefore compute the client ledger of its own clients from their summaries, the aggregates of the other edges, and the encoded validation view. The values are the same wherever they are computed, and the cloud then solves the edge game from aggregates alone. In our experimental implementation, the cloud computes both ledgers from the reported summaries.
  • Sensitivity to altered reports and noise. Q5 measures summary-noise sensitivity, and Q6 measures the attacker's own normalized reward after count, mean, or variance manipulation. Mean changes under count and variance perturbations range from −0.15 to −5.18 percentage points. The added 30% prototype-interpolation setting gives −0.0158 points. Q6 reports the experimental scope and reward changes.

W5. Temporal ledger and partial reporting.

Thank you for pointing out that the setting for which temporal completion is introduced deserves a direct test. We have now evaluated partial reporting directly by varying the reporting set (Q4). Missing client scores are completed using historical observations, with the current-round mean used for cold starts. It never uses future accounting rounds, and it never fills an edge that has no reporting client, so an edge without evidence receives no value. At 50% reporting, historical-mean completion achieves the highest mean rank correlation among the four tested completion rules.

We also agree that the meaning of the temporal edge value under membership changes should be stated. The temporal mean of an edge averages, over the accounting rounds in which the edge is observed, the contribution of the clients it hosts in each of those rounds. It therefore values the edge as an operating unit, which is what decisions about edge servers act on, while each client account follows its client across edges. Appendix A.19 clarifies this interpretation of the temporal edge ledger.

W6. Conclusions relative to independent evidence.

Thank you for this careful reading. We agree that each conclusion should match its independent evidence, and we revise the three statements accordingly.

  • Identifying the highest-valued edge in every run reflects solver correctness within the declared game. The revision states it that way and places the external tests of W1 beside it.
  • Three seeds. The removal intervention now uses 10 independent training seeds. The gain over no removal has a 95% bootstrap interval of [1.51, 7.55] and is positive in 9 of 10 seeds (Q7).
  • Corrupted-edge ranking. With the expanded set of 80 deployments, exact edge values now rank corrupted edges significantly better than both summation and direct sampling, measured by the Spearman rank correlation with the negative normalized label-distribution shift (Q7).

W7. Sensitivity to the validation view.

Thank you for raising this point. We have now studied how the size and the class prior of the validation view affect both ledgers.

  • Size. With a stratified half view, the client rank correlation is 0.92 at N=12 and 0.90 at N=50, and the edge rank correlation is 0.88 and 0.86. With about a quarter of the view, the client rank correlation is 0.83 and 0.79. These stratified reductions retain substantial rank agreement.
  • Class prior. A 10-to-1 long-tailed validation view declares a different target population, and the ledger changes accordingly, with client rank correlations of 0.55 and 0.62.
  • Appendix A.20 reports the validation-view size and class-prior settings used in the sensitivity study.

Q1. Exact estimation of the summary game versus faithful attribution of the trained model.

We separate estimation error from utility choice. Let Φsum\Phi_{\rm sum} and Φmod\Phi_{\rm mod} denote the exact values of the summary game and the model game, and let Φ^sum\hat\Phi_{\rm sum} denote the FedCCE estimate. Then

Φ^sum−Φmod=(Φ^sum−Φsum)+(Φsum−Φmod).\hat\Phi_{\rm sum}-\Phi_{\rm mod}=(\hat\Phi_{\rm sum}-\Phi_{\rm sum})+(\Phi_{\rm sum}-\Phi_{\rm mod}).

The first term is estimation error. In the exact-edge regime, FedCCE removes it at the edge level, while client sampling error scales as M−1/2M^{-1/2}. The large-E edge sampler has its own budget. The second term reflects the utility definition. The edge correlation of 0.550 compares two exact games and quantifies agreement between those utilities.

Table A18 provides a three-seed coalition-retraining diagnostic. Exact summary values recover the retraining-best edge in one of three runs, compared with three of three for the sampled client sum. This illustrates that accuracy within the summary game and agreement with coalition retraining are distinct evaluation criteria.

The two exact references answer different valuation questions. The summary ledger measures how much the data of a client improves the class separability of the shared representation, through variance-weighted nearest-prototype accuracy on the shared representation and validation view. The model ledger measures how much the update of a client improves the global model aggregated from the updates of each coalition. Our guidance to practitioners is therefore to choose the utility that matches what the decision must explain, and then to solve it with the resolution-matched solver. The solver and the utility are separable in FedCCE, and Q2 shows that the solver applies unchanged to a model utility. Decisions that act on the trained model, such as removing clients, can be checked directly, and our response to W1 reports these checks.

Q2. Comparisons with a fixed characteristic function.

We added fixed-utility, fixed-target controls under both summary and reconstructed-model utilities. We fixed the accounting round and its global model, the reporting clients, the partition, and the allocation target, and compared three edge solvers under the summary utility and under the native model utility. The setting is N=12 clients and E=4 edges, four datasets with three seeds each, M=64, and 10 streams per deployment. Direct edge sampling uses B=MN/E=192 orders.

UtilityEdge solverEdge Spearman ρMean absolute errorRuns ranking the best edge first
SummarySum of client values0.7870.008073.3%
SummaryIndependent edge sampling0.9050.003886.7%
SummaryExact quotient game10100%
ModelSum of client values0.6700.006962.5%
ModelIndependent edge sampling0.7650.003873.3%
ModelExact quotient game10100%

Under both utilities, summation conserves the total to numerical precision. Its edge-ranking correlations are 0.787 under the summary utility and 0.670 under the model utility. Direct edge sampling improves these correlations to 0.905 and 0.765, respectively. These results demonstrate that total conservation alone does not establish accurate edge allocation. The blind spot thus comes from the summation estimator and does not depend on the nearest-prototype utility, and the FedCCE solver applies directly to a model utility. The exact rows are solver-correctness checks. Native baselines retain their respective utilities and allocation targets in Table 1. Flat TMC estimates the Shapley value without the partition, which is a different target from the hierarchical Owen value, so this table fixes the target and compares solvers.

Reviewer_9hV9OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes FedCCE, a method for evaluating both individual clients and the groups of clients in hierarchical federated learning. Its main observation is that summing approximate client values can give inaccurate edge scores, even when the total value is correctly conserved. FedCCE therefore estimates client values and computes edge values separately by means of class-level feature summaries to evaluate coalitions without retraining.

The paper supports its claims with theoretical analysis, experiments on four image benchmarks, and label-corruption studies. Its edge values are exact for the chosen prototype-based utility, but do not directly measure contributions to the trained federated model or the edge server’s own work. The method builds on established Owen–Shapley relationships, sampling methods, prototype-based valuation, and hierarchical accounting. Its main novelty seems to be the edge-allocation diagnosis and computational construction. The method improves the accuracy of edge-value estimates, but the paper provides limited evidence that this leads to better edge-level decisions.

Critical strengths
  1. The paper identifies a blind spot in checking edge allocations

Theorem 1 shows that summing sampled client Owen values is equivalent to estimating each edge group’s Shapley value from sampled edge orders. These estimates always sum to the correct total, even when individual edge scores are inaccurate. Experiments demonstrate this under both summary-based and reconstructed-model utilities, showing that the finding is not specific to the proposed classifier.

  1. The method enables accurate edge accounting at low additional cost

Client and edge scores use the same coalition utility, so their exact values remain consistent while each can have its own evaluation budget. Class summaries make coalition evaluation inexpensive, and reusing evaluations from client sampling further reduces the cost of computing exact edge scores. Proposition 2 quantifies this additional cost, while the experiments demonstrate substantial savings compared with computing both client and edge values exactly.

Critical weaknesses

None.

Questions for authors
  1. What capability does FedCCE provide that would be missing from a straightforward combination of prototype-based valuation, such as SPACE, and hierarchical accounting?

  2. Is it possible to identify when exact enumeration materially improves on direct edge sampling, considering both computational cost and the resulting edge rankings?

  3. The main experiments collect summaries from every client in each accounting round. How reliable are the accumulated scores when some clients report less often and their missing values are filled from history?

  4. In Table A11, does the advantage over summed client values persist when both methods must finish within the same wall-clock budget?

3 comments

We sincerely thank the reviewer for the careful review, and we are greatly encouraged by your positive assessment of the novelty of our work and of the soundness of its theory and experiments.

Response to the concern in your summary. Your summary notes that the paper gives limited evidence that more accurate edge values lead to better edge-level decisions. Our new experiments supply this evidence directly. In these experiments, M denotes the number of complete partition-respecting hierarchical permutations in the client pass, N the number of clients, and E the number of edges.

  1. We fix the same client ledger at M=32 and change only the edge solver. Across 48 deployments with hidden label corruption, the rate of identifying the corrupted edge within 5 seconds rises from 30.4% with summation to 56.3% with exact edge values. The gain is 25.8 percentage points, with a 95% bootstrap interval of [15.8, 37.5] and p = 0.012 after Holm correction for multiple comparisons.
  2. Over 80 deployments, the Spearman rank correlation between exact edge values and the negative normalized label-distribution shift is 0.54, against 0.17 for summed values. The exact ledger is ahead in 16 of 20 dataset-seed clusters (two-sided sign test, p = 0.012).
  3. Measured in the declared game, when the summed ledger at M=128 ranks the wrong edge first, the chosen edge is worth a median 6.7% less than the best edge, and more than 10% less in 35% of those runs. In the third of runs where the best edge leads the second by more than 7.2%, the summed ledger still misses it in 34% of runs. The errors are therefore not confined to near ties.

Q1. What FedCCE adds over a direct combination of prototype valuation and hierarchical accounting.

The natural direct combination of SPACE-style prototype valuation with hierarchical accounting computes client values and then sums them within each edge. Theorem 1 shows that the resulting edge value is exactly a Monte Carlo estimate over M sampled edge orders. Its total is conserved in every run, so a conservation check does not reveal its allocation error. A cheaper utility does not remove this. FedCCE solves the quotient game independently. Composable summaries and reuse of coalition evaluations from the client pass make this computation inexpensive. This is the construction of FedCCE, and it provides three capabilities that the direct combination lacks.

  • The edge ledger has an independent solver. It is exact within the declared game for small E. For larger E, direct edge sampling reduces variance by a factor of N/E under the equal-marginal-evaluation-budget conditions of Corollary 1.
  • The client ledger can be audited. Exact edge values make the per-edge error ξ_e of the summed ledger observable without further utility queries (Proposition 4).
  • The additional query bound is independent of N for fixed E (Proposition 2). At N=50 and E=8, the measured increment is 26 to 70 ms.

The capability transfers to other utilities. On an adapter of the SPACE cosine utility over a common encoder, independent edge sampling lowers the normalized L1 error, the L1 distance to the exact edge values divided by their total absolute value, from 0.224 to 0.120 at M=64.

Q2. When exact enumeration materially improves on direct edge sampling.

The operating regime depends on E, the accounting budget, and the required ranking precision. The table summarizes the three regimes, and the points below give the details.

Edges EEdge solver in FedCCECost of the edge ledgerEdge ranking
Up to 10Exact enumerationComparable measured cost. At N=100, E=10, and M=64, both ledgers take 2.449 s, against 2.478 s with direct samplingIn the separate N=50/E=8 study, Spearman ρ with the negative normalized label-distribution shift is 0.54, against 0.41 for direct sampling
16 and 20Exact enumeration21 s and 5.8 min, outside the training loopBest edge found in every run, against 27% and 20% for direct sampling
25 and 50Direct edge samplingThe edge pass adds 3.9 s and 4.5 sNormalized L1 error of summation lowered by 49.6% and 34.3%
  • Up to 10 edges, exact enumeration removes edge sampling error at comparable measured cost. The rank correlations are compared at the same marginal-evaluation budget over 80 deployments, and the exact ledger is ahead in 18 of 20 clusters (two-sided sign test, p = 0.0004). Even with 4,096 edge orders, direct sampling picks the wrong top edge in 23% of streams at E=8 and 45% at E=10, because the leading edges are often close. Exact values remove this error and leave the remaining time for the client ledger. In the 12 deployments with an exact client reference, a 5-second budget gives a client rank correlation of 0.966, against 0.780 for a schedule that spends the time on edge sampling.
  • At 16 and 20 edges, exact enumeration still improves rankings materially when the ledger is computed between training rounds. Direct sampling keeps normalized L1 errors of 0.095 and 0.110, while exact values carry no sampling error. Summation is worse still. It finds the best edge in only 10% and 17% of runs, and when it misses, the chosen edge is worth a median 10.6% and 12.3% less than the best one.
  • For larger E, FedCCE samples the edge game directly. At N=100 and the client pass's marginal-evaluation budget, Corollary 1 gives a standard-error ratio of √(E/N), corresponding to reductions of 50% at E=25 and 29% at E=50. The table separately reports measured normalized L1 reductions of 49.6% and 34.3%.

At a matched 5-second deadline, direct sampling and exact edges reach corruption-identification rates of 55.8% and 56.3%, compared with 47.5% for summation. These results describe the scheduling trade-off. For small E, exact enumeration provides zero within-game edge sampling error and auditability at modest incremental cost.

Q3. Reliability when clients report less often.

The accumulated scores remain reliable when a moderate share of reports is missing, and they degrade gradually as reports become sparse. We varied the set of clients that report summaries in each accounting round and recomputed the observed game, on six deployments with N=12 clients and E=4 edges (CIFAR-10 and Fashion-MNIST, three seeds each) and 20 random reporting patterns per rate. Historical completion uses a client's past scores when available and the current-mean cold-start rule otherwise. It never uses future rounds. Agreement is measured against the ledger under full reporting.

Random reporting rateClient Spearman ρClient mean absolute errorClients with a valueEdges with a value
75%0.9140.011100%100%
50%0.7680.027100%98.75%
25%0.5480.05396.5%80.0%
  • Historical completion preserves full client coverage at 75% and 50% reporting. At 75% reporting, the completed ledger keeps a rank correlation of 0.914 with full reporting. At 50%, the historical mean is the best of four completion rules, reaching 0.768 against 0.531 for zero filling, 0.652 for the current mean, and 0.762 for the last observation, and it preserves the sign of 97.4% of client values.
  • Edge values average observed rounds. An edge has a round-level value when at least one client reports. At 50% reporting, 98.75% of edges have a defined temporal value. An edge unobserved throughout the averaging window remains undefined.
  • Sampling remains accurate within the observed game. At M=64, client correlation with the exact values for the same reporting set is 0.94 to 0.96. This measures sampling accuracy separately from the effects of missing reports and temporal completion.

Q4. Does the advantage in Table A11 persist under the same wall-clock budget?

Exact edges retain a higher observed identification rate than summation at each tested deadline. We reran the original 48 deployments with strict deadlines and five streams per deployment. Timers start with prepared inputs and empty utility caches. Unfinished ledgers count as failures, without later backfilling. All methods start from the same client stream at M=32. Summation continues client sampling, exact accounting enumerates edges and then refines clients, and direct sampling uses the remaining time for edges.

DeadlineSummationExact edgesDirect edge sampling (FedCCE fallback)
1 s32.1%44.2%47.1%
5 s47.5%56.3%55.8%
30 s52.9%56.3%57.1%

At 1 second, about a quarter of the rounds do not finish for every method and count as failures.

The observed identification gap narrows as the budget increases. Theorem 1 separately characterizes the M−1/2M^{-1/2} scaling of the summation estimator's error. Within 30 seconds, the summed ledger runs about 1,300 to 12,000 client permutations and lowers its normalized L1 edge error to 0.023, yet its identification rate of 52.9% stays below the 56.3% that exact edges reach at 5 seconds. When the client ledger is held fixed to isolate the edge module, exact edges improve on summation by 25.8 percentage points (95% bootstrap interval [15.8, 37.5], p = 0.012 after Holm correction).

The matched-deadline differences are descriptive and do not reach Holm-adjusted significance across the 12 dataset-seed clusters. The significant fixed-client result isolates the edge module's contribution.

Thank you again for your positive assessment and insightful questions. We will incorporate these results and clarifications into the revised version.

Reviewer_LZWkOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies contribution accounting for clients and edge domains in hierarchical federated learning. Exact client Owen values sum to the corresponding edge Shapley value under a fixed partition and a shared utility. The paper shows that summing sampled client values instead gives an unbiased edge estimator whose total is always conserved, even when individual edge shares are inaccurate. FedCCE evaluates coalitions through pooled class summaries and a shared utility cache. It samples client values and computes the smaller edge game exactly. The analysis characterizes allocation error and additional edge-query cost. Experiments on four image datasets examine allocation error, corrupted-client detection, removal decisions, and computation cost.

Critical strengths
  1. Allocation errors persist in larger client settings Theorem 1 explains why an efficiency check can pass despite incorrect edge shares. Section 4.3 and Figure 3 demonstrate the effect with 50 and 100 clients and show that it also occurs with a model-based utility. At equal marginal-evaluation cost, direct edge sampling reduces allocation error by factors of about 2.5 and 3 in the two deployment sizes.
  2. Shared caching limits the extra cost of exact edge values The shared cache in Sections 3.3 and 3.4 reuses coalition evaluations across the two ledgers. Proposition 2 quantifies the additional queries required to compute exact edge values after the client pass. Figure 4 also reports a client-removal experiment, with a mean CIFAR-10 gain of 4.35 percentage points and a standard deviation of 1.44 across three seeds.
Critical weaknesses
  1. The strongest client-decision claims need a clearer link to sampling Section 4.4 and Figure 4a on pages 8-9 show perfect sign separation using exact client values in the small deployment. Section 3.4 instead specifies a sampled client ledger. Table 1 already validates sampled client rankings, and Section 4.4 reports positive detection results at deployment scale. What remains unclear in the main text is how reliably the actual sampling budget preserves the signs and client selections behind the perfect-separation and removal claims. High ranking correlation does not by itself establish reliable decisions near zero or near a removal cutoff. The paper should identify whether Figure 4b uses exact or sampled values and connect its conclusion to the corresponding budget, false-positive and false-negative rates for corrupted-client detection, and variation across seeds. Existing results or a focused check in the Figure 4 setup could address this.
  2. Some empirical conclusions extend beyond what the comparisons establish Table 1 on page 7 correctly declares the reference game and marks exact edge agreement by construction. In Section 4.2, however, low agreement from methods using model, gradient, or prototype utilities is discussed alongside methods estimating that reference game. Disagreement across utilities does not determine which utility is preferable for a particular accountability decision. The summary game is a legitimate target; its practical value should be supported by the corruption and intervention results, with the corresponding limits stated clearly. The timing claims also need consistent qualification. Figure 5c on page 9 explicitly calculates round time from utility-call counts and measured query costs, while Section 4.5 describes the ledgers as taking 2.3 seconds. That estimate should be distinguished from measured completion latency, with the included costs stated briefly in the main text.
Questions for authors
  1. In Section 4.4 and Figure 4, which decisions use exact client values and which use sampled values? At the deployed sampling budget, how often do the signs and removal selections agree with the exact values? Please give the false-positive and missed-client rates and their variation across seeds, or identify the existing results that report them.
  2. Please clarify the interpretation of Table 1 in Section 4.2. Which conclusions concern estimation accuracy within the declared summary game, and which concern practical usefulness for identifying harmful participants or improving the model? The caption already distinguishes cross-utility agreement; the main discussion should carry that distinction through.
  3. For Section 4.5 and Figure 5, please label the per-round query-time estimates explicitly in the main text and state which costs are included. In particular, how should the reported 2.3 seconds be interpreted relative to client summary preparation, communication, and cache handling?
2 comments

Thank you for the careful review and for recognizing our finding that total-value conservation does not guarantee accurate edge allocation, as well as the computational benefits of the shared cache. We have clarified these points in the revised paper and provided the supporting experimental results in our responses to Q1–Q3. Below, M denotes the number of complete partition-respecting hierarchical permutations in the client pass.

W1. Linking the client-decision claims to the sampling budget.

We agree that a high rank correlation does not certify decisions near zero or near a removal cutoff. This is the logic of our central result applied to clients. A conserved total does not certify an edge allocation, and a rank correlation does not certify a decision, so each decision must be checked at its own threshold. The revised Section 4.4 ties every decision claim to the values behind it.

  • Sign separation in Figure 4a is a property of exact client values, and the text will say so. The deployed ledger with M=64 preserves these signs for 99.94% of clients when averaging rounds 30 and 40.
  • Removal in Figure 4b already uses the deployed ledger with M=64, and its selections match the exact ledger in all three original seeds and all 50 tested sampling streams per seed.
  • Variation across seeds is now measured over 10 independent training seeds, extending the original three-seed experiment.

Figure 4 will mark which panel uses exact values and which uses sampled ones, and Section 4.4 will report sign agreement, selection agreement, and the false-positive and missed-client rates at the deployed budget.

W2. Conclusions that extend beyond the comparisons.

We appreciate the view that the summary game is a legitimate target. The revision makes three changes.

  • Cross-utility rows. Section 4.2 distinguishes estimation accuracy under a fixed utility and allocation target from agreement across different value definitions. These rows show how far other value definitions lie from the declared one.
  • Practical value and its scope. The practical value of the summary game will rest on evidence that never enters the scores, namely injected corruption and final test accuracy after removal. The text will state that this evidence covers label corruption and that the training intervention is run on CIFAR-10.
  • Timing. Every timing statement in the main text will say whether it is estimated or measured. The 2.3 seconds and the 19 hours in Section 4.5 are estimates from utility calls and measured query cost, as in Figure 5c, and the measured completion time will appear beside them.

Q1. Which decisions use exact values, and how reliable are the sampled ones?

Figure 4a uses exact client Owen values to show sign separation. The removals in Figure 4b use the client ledger sampled at M=64, averaging accounting rounds 30 and 40 and removing the two lowest clients. New checks show that these sampled decisions agree with the exact values.

SeedRemoved clientsSame as exact late-half ledgerStreams selecting the same two clients (of 50, M=64)Sign-rule false-positive rate / missed-client rate
011, 4Yes100%0% / 0%
19, 1Yes100%0% / 0%
27, 3Yes100%0% / 0%

Across all 15 deployments of Figure 4a, with 50 sampling streams each and values averaged over rounds 30 and 40, the ledger with M=64 agrees with the sign of the exact value for 99.94% of clients. Its false-positive rate, the share of clean clients flagged as harmful, is 0%, and its missed-client rate, the share of corrupted clients not flagged, is 0.67%. These rates are averaged over the 50 sampling streams within each deployment and then equally across the 15 deployments. Its selection of the K lowest clients, with K fixed to the injected corruption count, matches the exact selection in every tested stream. The only deployment with any sign flip has a corrupted client whose exact value is itself close to zero yet far from the removal cutoff, so the selection never changes.

Over 10 independent CIFAR-10 training seeds, removal by the ledger with M=64 finds all 20 corrupted clients. It raises final accuracy over no removal by 4.10 percentage points on average, with a 95% bootstrap interval of [1.51, 7.55] and a gain in 9 of 10 seeds. This agrees with the 4.35 points reported for three seeds in the paper.

Q2. Interpretation of Table 1.

We agree with this distinction. Sections 4.2 and 4.4 distinguish three kinds of evidence.

  • Estimation accuracy within the declared game is shown by the rows of Table 1 that estimate the same reference game.
  • Cross-utility and cross-target agreement is shown by rows using other utility signals or flat allocation. These rows indicate how close the value definitions are. Same-target rows measure estimation error.
  • Practical usefulness is tested with corruption labels that never enter the scores and with final test accuracy. Over 10 seeds, FedCCE finds 20 of 20 corrupted clients, against 15 of 20 for the hierarchical control and 11 of 20 for FedOwen. For the number of corrupted clients identified, FedCCE beats FedOwen in 8 seeds and loses in none (two-sided sign test, p = 0.008, Holm-adjusted p = 0.016 over two identification comparisons).

We also tested the mechanism itself with the utility held fixed. Edge values summed from the client ledger reach Spearman rank correlations of 0.79 and 0.67 with the exact edge values under the summary utility and a model utility. Independent edge sampling reaches 0.91 and 0.77. The efficiency residual of both, the gap between the summed edge values and the value of the grand coalition, is about 10−1710^{-17}. Conservation with misallocation is therefore not specific to one classifier.

Q3. What the 2.3 seconds include.

The 2.3 seconds in Section 4.5 were an estimate, computed as utility calls times the measured cost per query. We have now measured the round directly on one RTX 6000 Ada GPU for CIFAR-10 with M=64, reporting medians over three seeds and five repetitions per seed.

Clients N / edges EBoth ledgers, inputs readyIncluding server preparationOf which exact edgesClient summary extraction, serial on one GPU
12 / 40.068 s0.121 s0.1 ms0.34 s
50 / 80.994 s1.130 s26 ms1.29 s
100 / 102.449 s2.776 s246 ms2.30 s
  • At N=100, measured accounting with prepared inputs takes 2.45 seconds, consistent with the scale of the 2.3-second query-time estimate.
  • The exact edge ledger takes 0.25 seconds, about a tenth of the round. Server preparation adds 0.33 seconds. It covers reading the inputs, recomputing the validation embeddings, and moving the summaries to the GPU.
  • Extracting summaries serially on one GPU takes 23 to 28 ms per client. This step can run locally in parallel across clients. Each summary is 0.47% of a model update on CIFAR-10. Network transfer is outside the reported timings. Prepared-input timing starts with GPU-ready evidence and an empty utility cache.
  • A separate benchmark times both utilities with the same script, including preparation. With the same sampler and a model utility, a complete accounting round on CIFAR-10 takes 37.0 and 94.6 seconds at N=50 and N=100, against 1.22 and 2.62 seconds for FedCCE. On Fashion-MNIST, where the model is small, the times are 3.16 and 7.71 seconds against 1.31 and 2.84 seconds.

The revised Section 4.5 will label 2.3 seconds as an estimate and report these measurements.

We sincerely thank you again for these constructive comments, which have helped us improve the paper, and we will incorporate all of the clarifications and new results above into the revised version.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response

We thank all three reviewers for recognizing the central finding that exact conservation does not certify edge allocation, and for their positive assessment of the resolution-matched design and the shared cache. We have conducted extensive additional experiments to further validate the effectiveness and efficiency of FedCCE. We summarize the key findings below. Throughout our replies, M denotes the number of complete partition-respecting hierarchical permutations in the client pass, N the number of clients, and E the number of edges.

  1. Decisions at the deployed sampling budget are reliable. The removal lists in Figure 4b come from the client ledger sampled at M=64 and match the exact ledger in all three seeds. Over 10 independent CIFAR-10 training seeds, the sampled ledger finds all 20 corrupted clients.
  2. An exact edge ledger yields better edge decisions. With the same client ledger at M=32, the rate of identifying the corrupted edge within 5 seconds rises from 30.4% with summation to 56.3% with exact edge values (p = 0.012 after Holm correction for multiple comparisons). Over 80 deployments, the Spearman rank correlation between edge values and the negative normalized label-distribution shift rises from 0.17 to 0.54 (two-sided sign test over 20 dataset-seed clusters, p = 0.012).
  3. The cost is small. On CIFAR-10 at N=100, E=10, and M=64, measured server-side accounting for both ledgers takes 2.449 seconds with prepared inputs and 2.776 seconds including server preparation. Exact edges take 0.246 seconds. In a separate paired utility benchmark, including preparation, summary, and model accounting take 2.62 and 94.6 seconds with the same sampler and timing script.

Detailed answers follow in the reply to each reviewer.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.