- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies cumulative participation losses relative to local training. It shows that early contribution clipping can stop learning, then combines a grace period with personalized heads. Experiments show substantial participation gains, but the implemented contract does not consistently enforce its deficit bound.
Critical strengths
- Participation during training is a useful concern beyond equilibrium welfare. The collapse analysis and experiments show why an immediate deficit constraint can be counterproductive.
- The appendix provides informative ablations and reports failures openly, including ledger breaches, personalization costs, and clients whose local models leave little room for improvement.
Critical weaknesses
- The guarantee covers an ideal regime where clipping never activates and assumes every client benefits at equilibrium. It does not cover the main deployed regime. Table 4 reports 23 deficit-bound breaches among 180 client-seed pairs in the difficult CIFAR-10 setting. Personalization without clipping has the same breach count, higher participation IR, and higher accuracy. The clipping component’s benefit is unclear.
- Incentive preservation assumes agents continue optimizing the original global-model utility, although their realized utility comes from personalized models. Unchanged payments and prescribed updates do not establish unchanged strategic behavior under that realized utility. The paper acknowledges this different game but leaves it unanalyzed.
Questions for authors
- Can the deployed rule enforce the announced post-grace bound, including estimation errors? If not, how should the contract and preservation claims be narrowed?
- What happens when clients choose contributions to maximize their personalized utility? Would the equilibrium and participation improvements survive?
2 comments
W2 and Q2: agents that choose contributions to maximize their personalized utility.
We added this case (Appendix C.10) and revised Section 6 to reflect it: the paper's equilibrium does not survive, and the participation improvements shrink or reverse. Theorem 3 makes no claim about agents that decide on their personalized utility. It is stated for agents that best-respond on , the base mechanism's strategic utility, which uses the shared model's accuracy, and it shows that the heads themselves do not disturb the mechanism: they change neither the payments, nor the best-response update, nor its equilibrium, because none of these reads the heads. In the new runs, all agents choose contributions from the accuracy of the head they are deployed with, from round 1, and everything else is as in the paper (42 runs, each paired with the paper's run of the same seed, partition, and costs).
| Paper's run → all agents decide on the deployed head | CIFAR-10 | MNIST | ||||
|---|---|---|---|---|---|---|
| Total contribution after the window, heads only | −24% | −71% | −79% | −46% | −54% | −74% |
| Accuracy, heads only | 84.3 → 82.3 | 86.6 → 76.7 | 64.7 → 48.4 | 99.0 → 98.9 | 99.0 → 98.7 | 98.6 → 96.4 |
| Participation IR, heads only | 98.8 → 93.3 | 100.0 → 86.2 | 91.2 → 61.6 | 97.5 → 97.3 | 94.3 → 90.5 | 42.4 → 37.3 |
| Accuracy, co-design | 84.3 → 80.9 | 86.6 → 65.0 | 59.4 → 25.9 | 99.0 → 98.9 | 99.0 → 98.7 | 98.5 → 96.2 |
| Participation IR, co-design | 98.7 → 92.2 | 100.0 → 63.9 | 87.7 → 31.4 | 97.5 → 97.3 | 94.3 → 90.5 | 42.7 → 37.8 |
| Agents whose ledger ends the run lower, heads only | 90/90 | 90/90 | 170/180 | 90/90 | 90/90 | 49/90 |
Accuracy is the shared model's over the final 50 rounds, participation IR is over the rounds after , and the last row sums each agent's ledger from round 1.
Contributions fall in every configuration and every seed, because on average, an agent's deployed accuracy responds far less to its own contribution than the shared model's does. Accuracy falls with them in every configuration. At participation IR stays above the grace contract's without heads, and at it ends below the grace contract's (Table 3). And 579 of the 630 agent-seed pairs end the run with a lower ledger than in the paper's runs.
So the improvements depend on contributions being decided on the shared model, as Theorem 3 assumes and as the co-design's update does (Algorithm 1). Section 6 now says so.
We thank the reviewer for taking the time to review our paper.
W1 (first part): the guarantee and the deployed regime.
Theorem 2 needs a tolerance at which the clip never engages, because it bounds the ledger along the base mechanism's path to its equilibrium, in the model in which that equilibrium is proved, where accuracy depends only on the contributions; once the clip acts, the path leaves the base mechanism's (our reply to Reviewer xvkG, W2, gives the reasons). For the regime in which the clip engages, we added a per-round result that assumes nothing about the equilibrium: Proposition 2 (Appendix A.9), stated under Q1 below. On every client benefiting at the equilibrium, the Limitations already say why Theorem 2 needs it: an agent that loses there accumulates deficit along the unconstrained trajectory itself, so no contract that follows that trajectory can save it. The revision states the theorem's scope in Section 7.1 and the Limitations.
Q1: can the deployed rule enforce the bound, with estimation errors, and how should the claims be narrowed?
In each round in which the agent is sampled, yes, as far as its own contribution in that round can, with an error of at most 0.006 in our runs. Over the run, not in general: when the shortfall is not in the agent's own cost, a ledger can still fall below −Δi, as 23 of the 180 agent-seed pairs do at CIFAR-10 α=0.1. The revision states both parts as Proposition 2 (Appendix A.9). It holds for the clip as implemented (Algorithm 1), on any accuracy path and any sampling, with the clip's estimation error. It assumes nothing about the equilibrium, so it also covers agents that do not gain there. In a round from Tg on in which agent i is sampled:
In the 21 co-design runs of Table 4, case (a) covers 70,994 sampled agent-rounds, with the largest breach of 0.0057 against Δi=1, and case (b) covers 2,716.
How the claims read in the revision. For tolerances at which the clip never engages, the contract's claim is Theorem 2's bound, under its assumptions, unchanged, and the contribution list now says "under a sufficient tolerance". When the clip engages, the claim is Proposition 2. The preservation claim is Theorem 3's, for agents that best-respond on the base mechanism's strategic utility (W2 and Q2 below). Once the clip engages, the path leaves the base mechanism's, as Appendix A.8 states.
W1 (second part): what the clip adds beyond personalization.
Reviewer hrA1 asks the same question, and our reply there answers it in full. In short, the clip keeps the bound in each round in which the agent is sampled, as far as the agent's own contribution in that round can, up to the clip's estimation error (Q1 above). But a lower contribution saves only the unrefunded part of the agent's own cost, under 5% of it, so almost the same agents cross with and without the clip, and most of the protection comes from the heads. The accuracy cost is set by the tolerance: at CIFAR-10 α=0.1 the co-design at Δi=20 is identical to the heads-only run (Appendix C.8).