- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies contribution accounting in hierarchical federated learning at both client and edge levels. It shows that summing sampled client Owen values can preserve total contribution exactly while misallocating edge shares. FedCCE uses composable class-wise feature summaries to evaluate coalitions without retraining, samples client Owen values, and computes edge Shapley values through exact enumeration or direct edge-order sampling. The paper provides theoretical error and cost analyses and evaluates attribution fidelity, harmful-client detection, and computational efficiency.
Critical strengths
1.The paper identifies an important pitfall: exact conservation of total contribution does not ensure accurate edge-level allocation.
2.The resolution-matched design is intuitive, using sampling for the large client game and exact enumeration for the smaller edge game.
3.Composable feature summaries and a shared utility cache enable efficient coalition evaluation without repeated retraining.
4.Theoretical analyses characterize edge-level estimation error and the additional cost of exact edge accounting.
5.The experiments cover attribution accuracy, top-edge identification, label-corruption detection, and computational efficiency.
Critical weaknesses
1.The practical validity of the declared summary game is not yet sufficiently established. FedCCE is exact for a nearest-prototype utility constructed from feature moments, but this utility is not the utility of the federated model being trained. The reported agreement with model-based attribution is only moderate, e.g., the average edge correlation with exact Model-Owen is 0.550, and may vary considerably across datasets. Therefore, exact computation of this surrogate game does not necessarily imply accurate attribution of the deployed model’s behavior.
2.Several headline comparisons conflate estimator quality with utility choice. In Table 1, model-, gradient-, and prototype-signal methods are evaluated against the exact hierarchical Proto-Owen value of FedCCE’s summary game, even though they target different characteristic functions. These results show cross-utility agreement rather than estimation error and therefore provide limited evidence that FedCCE is a better contribution estimator than those methods. The strongest comparisons should hold the target utility fixed.
3.Exact edge enumeration remains exponential in the number of active edges. Proposition 2 shows that its incremental cost is independent of the number of clients, but it still depends exponentially on E, and exact enumeration is evaluated only up to ten edges. The proposed direct edge-sampling fallback for larger hierarchies is theoretically discussed but not experimentally evaluated, leaving scalability to realistic deployments with tens or hundreds of edges unclear.
4.The evidence-preparation mechanism introduces nontrivial privacy and trust assumptions. Per-client class counts, means, and diagonal variances expose class presence and feature-distribution information to the cloud without differential privacy, secure aggregation, or robustness against strategic reports. Because contribution values may determine rewards or sanctions, clients also have an incentive to manipulate these summaries. The current experiments assume honest, noiseless reporting.
5.The temporal-ledger and partial-reporting mechanisms receive limited validation. Deployment-scale experiments have every client upload a summary in each accounting round, whereas absent clients are handled by a simple historical-mean rule and inactive edges remain unobserved. Thus, the experiments do not adequately test the setting for which temporal completion is introduced. Moreover, averaging edge values over rounds may become difficult to interpret when client-to-edge memberships change.
6.Some empirical conclusions are stronger than the available independent evidence. FedCCE identifies the highest-valued edge in every run because it exactly enumerates the quotient game defining the reference; this establishes solver correctness but not that the selected edge is truly most valuable to the trained federated model. The corrupted-client intervention uses only three seeds, and the advantage of the direct edge ledger in corrupted-edge ranking is reported as statistically insignificant.
7.The method depends on a centralized validation view and a shared encoder, but sensitivity to the size, distribution, and representativeness of this validation set is not studied. A shifted or unbalanced validation view could materially change both client and edge values, particularly because coalition utility is measured through nearest-prototype accuracy rather than the trained classifier.
Questions for authors
1.Can the authors more clearly separate exact estimation of the declared summary game from faithful attribution of the trained federated model? In particular, how should practitioners interpret FedCCE values when summary-game and Model-Owen rankings disagree?
2.Could the authors provide controlled comparisons in which FedCCE, Model-Owen, TMC, and other estimators all query the same characteristic function? Conversely, could the FedCCE resolution-matched solver be applied to a model-based utility on tractable deployments so that estimator error is evaluated without changing the target?
3.How does the direct edge-sampling fallback perform when E is too large for enumeration? Experiments with substantially more than ten edges, including allocation error, top-edge recovery, and wall-clock cost, would clarify whether resolution matching remains effective at realistic hierarchy sizes.
4.Can the authors evaluate partial summary participation directly by varying ? It would be useful to report the accuracy of the historical client-completion rule, the frequency of unobserved edges, and ledger stability as summary-reporting rates decrease.
5.What information about client labels and feature distributions can be inferred from the uploaded class summaries? How would secure aggregation or differential privacy affect exact composability, utility accuracy, and contribution rankings?
6.How robust is FedCCE to strategic or corrupted summary reports? Since contribution values may determine rewards, can a client increase its assigned value by manipulating class counts, means, or variances without improving the global model?
7.Could the authors report additional seeds or confidence intervals for the removal intervention and corrupted-edge experiments? In particular, can they clarify which downstream improvements are statistically significant rather than illustrative case studies?
8 comments
W4. Privacy and trust assumptions.
Thank you for raising these important concerns. We clarify the trust assumptions below.
- Who sees client summaries. The exact edge ledger needs only per-edge aggregates, the same granularity at which the cloud receives models in hierarchical FL. The client ledger needs individual summaries, but every coalition in the Owen value of a client consists of whole other edges plus clients of its own edge. Each edge server, which already receives the individual updates of its clients, can therefore compute the client ledger of its own clients from their summaries, the aggregates of the other edges, and the encoded validation view. The values are the same wherever they are computed, and the cloud then solves the edge game from aggregates alone. In our experimental implementation, the cloud computes both ledgers from the reported summaries.
- Sensitivity to altered reports and noise. Q5 measures summary-noise sensitivity, and Q6 measures the attacker's own normalized reward after count, mean, or variance manipulation. Mean changes under count and variance perturbations range from −0.15 to −5.18 percentage points. The added 30% prototype-interpolation setting gives −0.0158 points. Q6 reports the experimental scope and reward changes.
W5. Temporal ledger and partial reporting.
Thank you for pointing out that the setting for which temporal completion is introduced deserves a direct test. We have now evaluated partial reporting directly by varying the reporting set (Q4). Missing client scores are completed using historical observations, with the current-round mean used for cold starts. It never uses future accounting rounds, and it never fills an edge that has no reporting client, so an edge without evidence receives no value. At 50% reporting, historical-mean completion achieves the highest mean rank correlation among the four tested completion rules.
We also agree that the meaning of the temporal edge value under membership changes should be stated. The temporal mean of an edge averages, over the accounting rounds in which the edge is observed, the contribution of the clients it hosts in each of those rounds. It therefore values the edge as an operating unit, which is what decisions about edge servers act on, while each client account follows its client across edges. Appendix A.19 clarifies this interpretation of the temporal edge ledger.
W6. Conclusions relative to independent evidence.
Thank you for this careful reading. We agree that each conclusion should match its independent evidence, and we revise the three statements accordingly.
- Identifying the highest-valued edge in every run reflects solver correctness within the declared game. The revision states it that way and places the external tests of W1 beside it.
- Three seeds. The removal intervention now uses 10 independent training seeds. The gain over no removal has a 95% bootstrap interval of [1.51, 7.55] and is positive in 9 of 10 seeds (Q7).
- Corrupted-edge ranking. With the expanded set of 80 deployments, exact edge values now rank corrupted edges significantly better than both summation and direct sampling, measured by the Spearman rank correlation with the negative normalized label-distribution shift (Q7).
W7. Sensitivity to the validation view.
Thank you for raising this point. We have now studied how the size and the class prior of the validation view affect both ledgers.
- Size. With a stratified half view, the client rank correlation is 0.92 at N=12 and 0.90 at N=50, and the edge rank correlation is 0.88 and 0.86. With about a quarter of the view, the client rank correlation is 0.83 and 0.79. These stratified reductions retain substantial rank agreement.
- Class prior. A 10-to-1 long-tailed validation view declares a different target population, and the ledger changes accordingly, with client rank correlations of 0.55 and 0.62.
- Appendix A.20 reports the validation-view size and class-prior settings used in the sensitivity study.
Q1. Exact estimation of the summary game versus faithful attribution of the trained model.
We separate estimation error from utility choice. Let and denote the exact values of the summary game and the model game, and let denote the FedCCE estimate. Then
The first term is estimation error. In the exact-edge regime, FedCCE removes it at the edge level, while client sampling error scales as . The large-E edge sampler has its own budget. The second term reflects the utility definition. The edge correlation of 0.550 compares two exact games and quantifies agreement between those utilities.
Table A18 provides a three-seed coalition-retraining diagnostic. Exact summary values recover the retraining-best edge in one of three runs, compared with three of three for the sampled client sum. This illustrates that accuracy within the summary game and agreement with coalition retraining are distinct evaluation criteria.
The two exact references answer different valuation questions. The summary ledger measures how much the data of a client improves the class separability of the shared representation, through variance-weighted nearest-prototype accuracy on the shared representation and validation view. The model ledger measures how much the update of a client improves the global model aggregated from the updates of each coalition. Our guidance to practitioners is therefore to choose the utility that matches what the decision must explain, and then to solve it with the resolution-matched solver. The solver and the utility are separable in FedCCE, and Q2 shows that the solver applies unchanged to a model utility. Decisions that act on the trained model, such as removing clients, can be checked directly, and our response to W1 reports these checks.
Q2. Comparisons with a fixed characteristic function.
We added fixed-utility, fixed-target controls under both summary and reconstructed-model utilities. We fixed the accounting round and its global model, the reporting clients, the partition, and the allocation target, and compared three edge solvers under the summary utility and under the native model utility. The setting is N=12 clients and E=4 edges, four datasets with three seeds each, M=64, and 10 streams per deployment. Direct edge sampling uses B=MN/E=192 orders.
| Utility | Edge solver | Edge Spearman ρ | Mean absolute error | Runs ranking the best edge first |
|---|---|---|---|---|
| Summary | Sum of client values | 0.787 | 0.0080 | 73.3% |
| Summary | Independent edge sampling | 0.905 | 0.0038 | 86.7% |
| Summary | Exact quotient game | 1 | 0 | 100% |
| Model | Sum of client values | 0.670 | 0.0069 | 62.5% |
| Model | Independent edge sampling | 0.765 | 0.0038 | 73.3% |
| Model | Exact quotient game | 1 | 0 | 100% |
Under both utilities, summation conserves the total to numerical precision. Its edge-ranking correlations are 0.787 under the summary utility and 0.670 under the model utility. Direct edge sampling improves these correlations to 0.905 and 0.765, respectively. These results demonstrate that total conservation alone does not establish accurate edge allocation. The blind spot thus comes from the summation estimator and does not depend on the nearest-prototype utility, and the FedCCE solver applies directly to a model utility. The exact rows are solver-correctness checks. Native baselines retain their respective utilities and allocation targets in Table 1. Flat TMC estimates the Shapley value without the partition, which is a different target from the hierarchical Owen value, so this table fixes the target and compares solvers.
Thank you for the thorough and constructive review, and for your positive assessment of the novelty of our work and the comprehensiveness of its theory and experiments. Your comments have helped us state the claims of the paper more precisely. We address each weakness at the level of the claims and the revised text. The supporting measurements appear in our answers to Q1 to Q7. Below, N denotes the number of clients and E the number of edges.
W1. Practical validity of the summary game.
Thank you for raising this central point. FedCCE separates accurate accounting for a declared utility from independent tests of its usefulness. The summary game has two complementary foundations.
The edge agreement of 0.550 with the model game, and its variation across datasets, measures agreement between the two value definitions (Q1). Section 4.2 and Appendix A.14 distinguish the valuation targets. Section 4.4 and Appendix A.15 report the removal intervention.
W2. Estimator quality versus utility choice.
Thank you for this distinction, which we agree should be carried through the whole paper. We would like to clarify that the headline comparisons are made with the utility held fixed. The finding that summation recovers the leading edge in only 46% of 232 runs even at the largest budget, and the 700× saving over exact computation, both compare estimators of the same summary game. The former measures allocation accuracy, and the latter measures query efficiency. We agree that the model, gradient, and prototype rows of Table 1 measure agreement across utilities. The revision separates them from the same-target rows and does not use them as evidence that FedCCE estimates better than methods with other targets. The new controls in Q2 hold the target fixed in both directions, and the advantage of the resolution-matched solver persists under a model utility.
W3. Scaling with the number of edges.
Thank you for this question about scale. The exponential term counts the active edges of one accounting round and does not grow with N (Proposition 2). Its cost is bounded in advance by the number of edge unions, so the operator can choose exact or sampled edges before accounting starts. Direct edge sampling is part of FedCCE, and Corollary 1 gives its variance. Both regimes are now tested well beyond ten edges (Q3). Exact enumeration finishes within 5.8 minutes up to E=20, outside the training loop. At E=25 and E=50, direct sampling lowers measured normalized L1 error by 49.6% and 34.3%, and its edge pass takes under five seconds. Corollary 1 gives the corresponding standard-error scaling. Each sampled edge order costs E utility queries regardless of N, so its query count is determined by its own sampling budget. The revision adds these results to Section 4.5 and the appendix.