- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies output-only aggregation in vertical federated learning: each party predicts from its own feature slice, and the server sees neither data, models, nor gradients, and never observes the true label. The authors cast this as a social choice problem. Using Young's (1975) characterisations, they argue that Borda is the best static rule, and they show that for every stationary rule there exists a data-generating distribution on which it incurs linear expected regret. They then propose Trust-Based Perpetual Voting (TBPV), which replaces the "down-weight the winners" satisfaction update of perpetual voting with a trust update that raises an agent's weight when it agrees with the consensus. The authors claim to establish mean-square convergence of the trust weights, that TBPV satisfies trustworthiness (fixed-point trust is strictly increasing in accuracy), that the perpetual-voting satisfaction guarantee carries an accuracy cost and on some instances cannot be met by eventually accuracy-ordered rules, and, under a concentration condition, that the trust fixed point attains higher collective accuracy than the approval fixed point. Validation consists of a proof-of-concept evaluation on two tabular datasets (Letter, Sensorless) with five parties, comparing TBPV (Kalman and EigenTrust variants) against the satisfaction class of perpetual voting rules and Hedge variants under contiguous and shuffled feature partitions.
Critical strengths
1. A clean and genuinely new accuracy–satisfaction trade-off (Thm 5.5, Cor 5.6). The two-agent instance gives an explicit bound on the accuracy cost of the perpetual-voting satisfaction guarantee, and Cor 5.6 shows that for every β_sat there is an instance on which no eventually accuracy-ordered rule can satisfy the guarantee with that parameter. The proof is correct and the result has no counterpart in the label-aggregation literature. Why it is consequential: it is the paper's most solid and original contribution, and it gives the social-choice framing of output-only VFL real content by showing that fairness and accuracy objectives cannot in general be reconciled.
2. An insightful decomposition of collective accuracy (Lemma B.8). Collective accuracy is separated into the alignment of pivotality with signal and total influence, and Remark B.9 shows why alignment alone cannot determine accuracy. Why it is consequential: it identifies, to leading order in the signal, what a weighting rule must achieve to improve accuracy, which is useful beyond this paper even though the dominance result built on it is not yet established.
Critical weaknesses
1. The agreement probability omits each agent's influence on the consensus, so trustworthiness (Prop 5.3) is not established. The mean-field analysis assumes the agreement probability q_v = ρp_v + (1−ρ)(1−p_v)/(K−1), i.e. that an agent's vote is independent of whether the consensus is correct. But each agent is itself a voter: the more weight it carries, the more it shapes the consensus and the more often it agrees with it. This self-reinforcement is the known failure mode of consensus-agreement weighting, and the current argument does not rule out a stable fixed point at which a heavily weighted but less accurate agent keeps high trust because it largely determines the consensus it agrees with (for example, one reached from an early lead). The key stability step of Lemma 5.2, on which Prop 5.3 depends, is also only sketched. Why it is consequential: trustworthiness is the premise of the entire method. The same agreement probability underlies Prop 5.3, Lemmas B.10 and B.11 (and hence the fixed-point comparison in Thm 5.7), and Lemma B.1.
2. The convergence and dominance guarantees are not shown to hold in the configuration the paper evaluates. Thm 4.1 requires L_h + 2L_ρ < 1/√n, but with the experimental Kalman parameters (Q = 0.01, R = 0.1) the steady-state gain is about 0.27, giving L_h ≈ 0.73 > 1/√5 ≈ 0.447. Thm 5.7 bounds the remainder by 2·3^n·max θ_v², i.e. 486·max θ_v² for n = 5, and the paper does not show that the leading-order gap exceeds it in the experimental regime; the concentration condition C(γ*) ≥ C(α*) is not checked there either, and Remark B.16 shows it fails increasingly often over K = 3–5. Cor B.17 then asserts a strict long-run advantage, but it assumes a strict gap that Thm 5.7 does not give, and since Thm 4.1 guarantees only convergence in mean square to a ball around γ*, it would further need that gap to exceed the variation in collective accuracy over that ball. Why it is consequential: the paper positions itself as primarily theoretical and presents its experiments as checks that the mechanism behaves as the theory predicts, but the theorems as stated do not cover the configuration evaluated. The Introduction's claim that the trust fixed point "provably beats the approval fixed point" also rests on Thm 5.7, which gives only an inequality up to a remainder that is not shown to be dominated.
Questions for authors
- Once agent v's own pivotality is included in q_v, does Prop 5.3 still hold, and under what conditions (e.g., a bound on individual influence)? If the highest initial trust is given to the least accurate agent, does TBPV recover empirically?
- Do the guarantees cover the experimental configuration? Specifically: can the bound in Lemma B.1 be tightened (the h-term depends only on agent v's own coordinate) so that Thm 4.1 applies to the Kalman setting used, and under what conditions on n and max θ_v does the leading term in Thm 5.7 exceed the remainder bound, with C(γ*) ≥ C(α*) holding at K = 26 and K = 11?