acceptodds
Under review as a conference paper at ICLR 2027

Label Distribution Inference Attack and Directional Defense on Prototypes in Federated Learning

Abstract

Prototype-based federated learning has emerged as an effective approach for addressing model heterogeneity by enabling clients with diverse architectures to collaborate through the exchange of per-class mean feature embeddings. Although prototypes are generally considered privacy-preserving, we show that they can reveal sensitive information about clients' label distributions. Specifically, we identify a geometric vulnerability: local prototypes exhibit stronger angular alignment with global prototypes for classes with more local samples, making prototype alignment an informative indicator of class frequency. Motivated by this observation, we propose the Prototype-Based Label Distribution Inference Attack (PLIA), which infers per-client label distributions solely from shared prototypes by exploiting this geometric property. To mitigate this privacy risk, we further introduce Directional Prototype Perturbation (DPP), a defense mechanism that applies orthogonal perturbations with randomized per-class budgets to disrupt this leakage signal. Experiments across five benchmark datasets show that PLIA outperforms or matches baseline attacks, while DPP effectively reduces the label-distribution leakage PLIA exploits with only a marginal impact on model performance.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Heterogeneous Federated Learning, Prototype-based Federated Learning, Label Distribution Inference, Privacy-Preserving Defense Mechanism

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_rcK5OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper demonstates a potential privacy issue in today's prototype-based federated learning, and proposes accordingly an attacking framework along with the defense methods.

Critical strengths
  • This work provides a solid background discussion and sufficient coverage of related work. The structure is clear, and the research problem is well formulated.
  • The illustrations and figures are easy to read and helpful. The experiments are comprehensive.
  • The paper clearly introduces DPP as a defense method against the proposed attack.
  • The ablation study is detailed and helps clarify the contribution of each design component.
Critical weaknesses
  1. The proposed attack is evaluated under a typical non-IID setting, while the authors do not clearly specify whether it also works in more IID cases. I understand that non-IID settings are the main attack target here, but I suggest that the authors state the applicable boundary more clearly.

  2. The paper attributes the observed angular divergence signals to label distribution shift. However, I think similar signals may also come from other types of client heterogeneity. Would it be possible to include experiments under such settings? This would help clarify the scope and assumptions of the paper.

  3. Based on the comments above, the scope of DPP may still be somewhat limited to these specific attack settings. This is, however, a relatively minor limitation.

Questions for authors

I do not have specific questions. Please see Critical Weaknesses.

4 comments

We sincerely appreciate the reviewer’s positive and constructive review. In the revised PDF, all changes are highlighted in blue, and we refer to them by page and line numbers throughout our responses.

W1. Applicable boundary of PLIA. We thank the reviewer for this insightful suggestion. We evaluate PLIA and DPP under α∈0.1,0.5,1,5,10,20\alpha \in \\{0.1, 0.5, 1, 5, 10, 20\\} and the IID setting on HAR, CIFAR-10, CIFAR-100, and Tiny-ImageNet (Figure 9 in Appendix J, p. 20; summarized in Section 5.2, p. 8, L417--419). PLIA achieves clearly lower LDL$^{\mathrm{TV}}$ than the Uniform baseline under non-IID settings (α≤1\alpha \leq 1), while its advantage diminishes as α\alpha increases and disappears in the IID setting, where local label distributions are nearly uniform, and the Uniform baseline is already accurate. Thus, PLIA applies to non-IID settings, which are typical in FL and where label distributions carry meaningful private information. DPP maintains higher LDL$^{\mathrm{TV}}$ than the Uniform baseline in most settings.

W2. Other types of client heterogeneity. We thank the reviewer for this helpful suggestion. 1) We have considered model heterogeneity, where clients use different architectures, and PLIA (Hete.) in Table 1 outperforms the Random and Uniform baselines in all cases (LDL$^{\mathrm{SC}}$ of 0.331--0.860 vs. 0.000 for Random). 2) Following the reviewer's suggestion, we further evaluate PLIA and DPP under domain shift on Digit-5 with 20 clients under FedProto, where clients hold either all five domains (mixed) or a single domain, both with Dir(0.5) labels (table below). PLIA achieves lower LDL$^{\mathrm{TV}}$ than the Uniform baseline in both settings, and DPP substantially increases LDL$^{\mathrm{TV}}$ with negligible accuracy change. We will add these results to the appendix. PLIA targets label distribution skew and assumes uniform sampling and individually uploaded prototypes without secure aggregation.

PLIA and DPP under domain shift on Digit-5 (FedProto, λ=1\lambda = 1, Dir(0.5) labels).

Domain splitMethodTV↑Acc.↑All five domains per clientUniform0.430−PLIA (No Defense)0.23785.9PLIA (DPP)0.55186.0Single domain per clientUniform0.353−PLIA (No Defense)0.17691.0PLIA (DPP)0.50890.9\begin{array}{ll|cc} \hline \text{Domain split} & \text{Method} & \text{TV}\uparrow & \text{Acc.}\uparrow \\\\ \hline \text{All five domains per client} & \text{Uniform} & 0.430 & - \\\\ & \text{PLIA (No Defense)} & 0.237 & 85.9 \\\\ & \text{PLIA (DPP)} & 0.551 & 86.0 \\\\ \hline \text{Single domain per client} & \text{Uniform} & 0.353 & - \\\\ & \text{PLIA (No Defense)} & 0.176 & 91.0 \\\\ & \text{PLIA (DPP)} & 0.508 & 90.9 \\\\ \hline \end{array}

W3. Scope of DPP. We thank the reviewer for the understanding. We agree that DPP's scope is currently limited to the angular leakage that PLIA exploits, as it is the first attack on shared prototypes and existing attacks target parameter-sharing FL, leaving no other prototype-based attack to evaluate against. We have stated this scope explicitly as a limitation in Section 6 of the revised PDF (p. 10, L517--519), and we view extending DPP to broader prototype-based threats as an important direction for future work.

I thank the author for the rebuttal materials, and they have mostly addressed my concerns. I remain my original positive score.

We thank the reviewer again for the positive assessment. The reviewer’s comments helped us improve our work, and we will incorporate the additional results and clarifications into the final version.

We have now updated the PDF once again to reflect the discussion with the reviewers, with new changes in green and those from the previous revision in blue. In the final version, we will further reorganize the paper to fully incorporate the discussion. We hope this addresses the reviewers' concerns; please let us know if you have any further questions. Thank you for your valuable feedback.

Reviewer_LctSOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes PLIA and DPP under prototype-based FL setting. The motivation behind the proposed method is based on the observation that classes with more samples receive stronger cumulative gradient pressure toward their global prototypes. Particularly, cosine similarity between local prototype and global prototype shows high correlation with the local label distribution. DPP is introduced to maximize the angle between the perturbated local prototype and the global prototype. Extensive experiments show the effectiveness of the proposed attack and defense methods.

Critical strengths
  • The proposed label distribution attack method (PLIA) is simple and effective.
  • The defense method (DPP) has good theoretical support.
  • Extensive experimental results support the soundness of the proposed methods.
Critical weaknesses
  • The main assumption for the PLIA is that local prototypes are updated with uniform sampling for the local dataset. However, if the local prototypes are updated with label-balanced sampling to mitigated biased prediction, the proposed attack method may not work.
  • The cosine similarity shows better attack than Euclidean distance even if Euclidean distance is used for prototype regularization. How about using cosine similarity for the prototype regularization? And the λ\lambda in Eq. (4) is fixed during the experiments. It would be good to include ablation study about λ\lambda to further understand the relation between prototype regularization and label distribution attack.
  • Minors
    • There is no description about the LiL_i in Eq.(4). It might be Eq.(1).
    • The paper introduces two types of prototypes, feature prototype and logit prototype. However, it is difficult to figure out which prototype is used in the experiments.
    • Although LDLKLLDL^{KL} is mentioned in the main paper, the results are shown in the main paper. Only in the Appendix.
    • It would be good to include no-FL local update performance for comparison.
    • In Table 4, only-OD or only-CB show marginal improvement but the combination shows significant improvement. Where does the synergy come from?
Questions for authors

Please refer the weakness section.

7 comments

We sincerely appreciate the reviewer’s insightful review and valuable suggestions. In the revised PDF, all changes are highlighted in blue, and we refer to them by page and line numbers throughout our responses.

W1. Label-balanced sampling. We thank the reviewer for this insightful comment. PLIA assumes standard uniform sampling over the local dataset, which is the default in PBFL methods. Label-balanced sampling equalizes per-class gradient pressure toward the global prototypes, which may reduce PLIA's effectiveness. To investigate this, we performed additional experiments varying α∈0.1,0.5,1,5,10,20\alpha \in \\{0.1, 0.5, 1, 5, 10, 20\\} and the IID setting (Figure 9 in Appendix J, p. 20; summarized in Section 5.2, p. 8, L417--419), since in these settings each mini-batch naturally contains a nearly equal number of samples per class, as in label-balanced sampling. As α\alpha increases toward the IID regime, PLIA's advantage over the Uniform baseline diminishes, which is consistent with the reviewer's intuition that label-balanced sampling may weaken PLIA.

W2-1. Cosine-based prototype regularization. We thank the reviewer for this valuable suggestion. We further evaluate PLIA and DPP with a cosine-based prototype regularizer, as shown in the table below. First, the cosine regularizer substantially degrades model accuracy (e.g., from 0.699 to 0.586 on CIFAR-10), which explains why PBFL commonly uses an MSE regularizer. Second, PLIA remains effective under the cosine regularizer, achieving lower LDL$^{\mathrm{TV}}$ than the Uniform baseline on all datasets. However, the leakage is slightly weaker than with the MSE regularizer. DPP still increases LDL$^{\mathrm{TV}}$ under the cosine regularizer, although its protection is weaker than under the MSE regularizer.

PLIA and DPP under MSE and cosine prototype regularizers (λ=1\lambda = 1, FedProto).

HARCIFAR-10CIFAR-100RegularizerDefense−UniformMSENo DefenseDPPCosineNo DefenseDPPTV↑Acc.↑0.433−0.2840.8860.4950.8780.3040.8410.4550.835TV↑Acc.↑0.436−0.2730.6990.6600.7000.3390.5860.4310.585TV↑Acc.↑0.431−0.2870.3080.4300.3070.3380.1990.3530.198\begin{array}{c|c|c|c} \hline & \textbf{HAR} & \textbf{CIFAR-10} & \textbf{CIFAR-100} \\\\ \hline \begin{array}{ll} \text{Regularizer} & \text{Defense} \\\\ \hline - & \text{Uniform} \\\\ \hline \text{MSE} & \text{No Defense} \\\\ & \text{DPP} \\\\ \hline \text{Cosine} & \text{No Defense} \\\\ & \text{DPP} \end{array} & \begin{array}{cc} \text{TV}\uparrow & \text{Acc.}\uparrow \\\\ \hline 0.433 & - \\\\ \hline 0.284 & 0.886 \\\\ 0.495 & 0.878 \\\\ \hline 0.304 & 0.841 \\\\ 0.455 & 0.835 \end{array} & \begin{array}{cc} \text{TV}\uparrow & \text{Acc.}\uparrow \\\\ \hline 0.436 & - \\\\ \hline 0.273 & 0.699 \\\\ 0.660 & 0.700 \\\\ \hline 0.339 & 0.586 \\\\ 0.431 & 0.585 \end{array} & \begin{array}{cc} \text{TV}\uparrow & \text{Acc.}\uparrow \\\\ \hline 0.431 & - \\\\ \hline 0.287 & 0.308 \\\\ 0.430 & 0.307 \\\\ \hline 0.338 & 0.199 \\\\ 0.353 & 0.198 \end{array} \\\\ \hline \end{array}

W2-2. Sensitivity to λ\lambda. We thank the reviewer for this suggestion. We now evaluate how the prototype regularization strength λ∈0.01,0.1,1,10,100\lambda \in \\{0.01, 0.1, 1, 10, 100\\} affects our method on HAR, CIFAR-10, CIFAR-100, and Tiny-ImageNet, with results in Figure 4 and Section 5.2 of the revised PDF (p. 8, L420--425). Raising λ\lambda generally improves PLIA, with lower LDL$^{\mathrm{TV}}$ when no defense is applied. It also improves DPP, with higher LDL$^{\mathrm{TV}}$ when DPP is applied. A likely reason is that heavier regularization sharpens the angular signal, which PLIA relies on and DPP is designed to disrupt. The trend reverses at very large values (e.g., λ=100\lambda = 100): model performance drops substantially, so the attack loses strength, which accounts for the U-shaped PLIA curve in Figure 4.

Minor 1. Li\mathcal{L}_i in Eq. (4). Thank you for pointing this out. Li\mathcal{L}_i denotes the local cross-entropy loss of client ii in Eq. (1), and we have clarified this in the revised PDF (p. 3, L158).

Minor 2. Prototype type used in experiments. We use feature prototypes for FedProto, FedTGP, and FedSA, and logit prototypes for FedDistill. We have stated this explicitly in Section 5.1 of the revised PDF (p. 7, L348--349).

Minor 3. LDL$^{\mathrm{KL}}$ results. The LDL$^{\mathrm{KL}}$ results are currently reported in Tables 7 and 9 of the Appendix. We will move Table 7, which reports LDL$^{\mathrm{KL}}$ across all datasets and FL methods, to the main text in the revised version.

Minor 4. No-FL local training. Thank you for the valuable suggestion. The table below reports the accuracy of local training without FL. Local training can outperform FL depending on the data distribution and the number of clients, but prototype-based FL remains widely used, so its privacy leakage is still important. We will include this result in the final version.

Accuracy (%) of local training without FL. \begin{array}{l|ccccc} \hline & \text{HAR} & \text{Amazon} & \text{CIFAR-10} & \text{CIFAR-100} & \text{Tiny IN} \\ \hline \text{Local} & 91.33 & 89.89 & 68.83 & 27.15 & 14.13 \\ \hline \end{array}

Minor 5. Synergy between OD and CB. In the original Table 4, OD-UB uses a fixed sign, so the effect of the random sign was not isolated. Our new ablation (table below; Table 4 of the revised PDF, p. 10, L486--507) shows that the random sign is the main contributor, while the class-wise budget alone yields limited gains. Since PLIA relies on relative alignment across classes, a fixed-sign perturbation shifts all classes in the same direction and largely preserves their ranking, whereas random signs combined with class-wise budgets jointly disrupt this ranking, explaining the synergy.

Ablation study on defense components against PLIA. OD: orthogonal direction; RD: random direction; CB: class-wise budget; UB: uniform budget; RS: random sign; FS: fixed sign. Bold: best TV per dataset.

HARAmazonCIFAR-10CIFAR-100Tiny INDefenseODCBRSNo Defense−−−RD-UB××−RD-CB×✓−OD-UB✓××OD-UB-RS✓×✓OD-CB-FS✓✓×OD-CB-RS (DPP)✓✓✓TV↑Acc.↑0.2840.8860.3120.8860.3170.8880.3160.8830.4250.8770.3330.8860.4950.878TV↑Acc.↑0.0680.8990.1030.8990.1400.8990.1470.8990.2820.8990.1720.8990.2640.899TV↑Acc.↑0.2730.6990.3220.7000.3530.7000.3300.6980.5680.6990.3780.6970.6600.700TV↑Acc.↑0.2870.3080.2990.3090.3020.3090.3240.3060.3520.3080.3330.3060.4300.307TV↑Acc.↑0.2080.1230.2370.1240.2460.1230.2660.1220.4260.1220.2880.1220.6120.121\begin{array}{c|c|c|c|c|c} \hline & \textbf{HAR} & \textbf{Amazon} & \textbf{CIFAR-10} & \textbf{CIFAR-100} & \textbf{Tiny IN} \\\\ \hline \begin{array}{lccc} \text{Defense} & \text{OD} & \text{CB} & \text{RS} \\\\ \hline \text{No Defense} & - & - & - \\\\ \hline \text{RD-UB} & \times & \times & - \\\\ \text{RD-CB} & \times & \checkmark & - \\\\ \text{OD-UB} & \checkmark & \times & \times \\\\ \text{OD-UB-RS} & \checkmark & \times & \checkmark \\\\ \text{OD-CB-FS} & \checkmark & \checkmark & \times \\\\ \hline \text{OD-CB-RS (DPP)} & \checkmark & \checkmark & \checkmark \end{array} & \begin{array}{cc} \text{TV}\uparrow & \text{Acc.}\uparrow \\\\ \hline 0.284 & 0.886 \\\\ \hline 0.312 & 0.886 \\\\ 0.317 & 0.888 \\\\ 0.316 & 0.883 \\\\ 0.425 & 0.877 \\\\ 0.333 & 0.886 \\\\ \hline \mathbf{0.495} & 0.878 \end{array} & \begin{array}{cc} \text{TV}\uparrow & \text{Acc.}\uparrow \\\\ \hline 0.068 & 0.899 \\\\ \hline 0.103 & 0.899 \\\\ 0.140 & 0.899 \\\\ 0.147 & 0.899 \\\\ \mathbf{0.282} & 0.899 \\\\ 0.172 & 0.899 \\\\ \hline 0.264 & 0.899 \end{array} & \begin{array}{cc} \text{TV}\uparrow & \text{Acc.}\uparrow \\\\ \hline 0.273 & 0.699 \\\\ \hline 0.322 & 0.700 \\\\ 0.353 & 0.700 \\\\ 0.330 & 0.698 \\\\ 0.568 & 0.699 \\\\ 0.378 & 0.697 \\\\ \hline \mathbf{0.660} & 0.700 \end{array} & \begin{array}{cc} \text{TV}\uparrow & \text{Acc.}\uparrow \\\\ \hline 0.287 & 0.308 \\\\ \hline 0.299 & 0.309 \\\\ 0.302 & 0.309 \\\\ 0.324 & 0.306 \\\\ 0.352 & 0.308 \\\\ 0.333 & 0.306 \\\\ \hline \mathbf{0.430} & 0.307 \end{array} & \begin{array}{cc} \text{TV}\uparrow & \text{Acc.}\uparrow \\\\ \hline 0.208 & 0.123 \\\\ \hline 0.237 & 0.124 \\\\ 0.246 & 0.123 \\\\ 0.266 & 0.122 \\\\ 0.426 & 0.122 \\\\ 0.288 & 0.122 \\\\ \hline \mathbf{0.612} & 0.121 \end{array} \\\\ \hline \end{array}

Thank you for the detailed rebuttal and the additional experiments. The revision answers most of my comments well. The cosine-based regularizer experiment, the sensitivity analysis on λ, the no-FL local training results, and the new ablation separating random sign from class-wise budget all clarified the behavior of PLIA and DPP. I also appreciate that the presentation issues have been fixed.

However, my main concern (W1) is still not fully resolved. The additional experiments vary α and include the IID setting. In those settings, the local label distribution itself becomes close to uniform, so it is expected that PLIA loses its advantage over the Uniform baseline. My concern is a different case: the local dataset remains skewed, but class-wise gradient contributions are balanced during local training, for example through class-balanced sampling or balanced softmax. These techniques are commonly used to reduce biased prediction under non-IID data. Because the core intuition of PLIA depends on class frequency being reflected in cumulative gradient pressure toward the global prototypes, the attack relies on the uniform sampling assumption. Even if uniform sampling is the default in existing PBFL methods, this assumption is a risk to the generality of the attack. Showing that PLIA still works, or explaining why it fails, under skewed local data with class-balanced training would significantly strengthen the paper's contribution.

Since this main concern remains open, I will keep my original rating of Weak Reject. I encourage the authors to include this analysis in a future version, and I believe it would make the work considerably more convincing.

Reviewer_ZD8hOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
  1. Novelty: Extends label distribution inference to prototype-based FL via local-global prototype angles, but the core idea closely mirrors existing work, so the methodological novelty is limited.
  2. Soundness: The link between class proportions and prototype alignment is not well theoretically grounded, and the paper does not clearly separate prototype regularization effects from sample statistics, weakening the interpretability of the attack.
  3. Experiments: While multiple datasets and FL settings are covered, the DPP defense is mainly tested against the paper’s own PLIA attack, without strong adaptive baselines, so its robustness and generalization remain under-supported.
Critical strengths

PLIA is simple and easy to implement. It infers label distributions from the cosine similarity between local and global prototypes, without requiring auxiliary data or access to model parameters.

The paper also proposes DPP to defend against PLIA. The choice of perturbation direction is supported by a theoretical analysis, and the overall design is easy to understand.

The experiments cover five datasets and several prototype-based federated learning methods. The authors also consider different levels of heterogeneity, numbers of clients, and prototype dimensions, making the evaluation reasonably comprehensive.

Critical weaknesses
  1. The methodological novelty is somewhat limited. The paper mainly extends label distribution inference to the prototype-sharing setting, using the inverse cosine distance between local and global prototypes to estimate label proportions. While this setting is worth investigating, the paper does not clearly explain what new insights or technical contributions it offers beyond existing distribution inference methods. The relationship between the proposed score and the actual label proportions is also not sufficiently analyzed.
  2. The attack and defense are too closely coupled. DPP is specifically designed to perturb the angular information exploited by PLIA, and the experiments mainly focus on PLIA. The paper lacks evaluation against other label distribution inference strategies, especially adaptive attacks. It is therefore unclear whether DPP can generalize to other attacks.
  3. The hyperparameter analysis and ablation studies are not sufficient. The paper lacks sensitivity analysis of key hyperparameters, such as the prototype regularization strength λ. Also, the ablation studies do not clearly separate the effects of random signs and class-wise perturbation budgets, making it difficult to assess the contribution of each component.

Minor: a. The presentation could be improved. For example, the framework figure does not clearly illustrate the details of the proposed methods. There are also some issues with formatting and citation consistency.

b. The theoretical analysis is not sufficient. The paper does not rigorously establish the relationship between label distributions and prototype alignment. Moreover, the proof for the defense only concerns local angular changes and does not directly establish that DPP effectively protects label distribution privacy.

c. Some claims are broader than what the results support. In particular, DPP is mainly evaluated against PLIA and its variants, but the paper makes relatively broad claims about its privacy protection and applicability across different settings. These claims are not fully supported by the current experiments.

Questions for authors
  1. The attack mainly relies on heuristic similarity scoring and normalization. The paper does not sufficiently explain how this differs from existing distribution inference methods or why the proposed score can reliably estimate label proportions.

  2. The use of FedDistill needs further clarification. The shared representations and aggregation process described in the paper differ from the original FedDistill. Also, cross-entropy-based alignment does not necessarily imply angular alignment of raw logits. The authors should clarify the implementation and examine whether these differences affect the results.

3 comments

We sincerely thank the reviewer for the thorough and insightful comments, which have helped us improve the paper. In the revised PDF, all changes are highlighted in blue, and we refer to them by page and line numbers throughout our responses.

Q1-1. How PLIA differs from existing label distribution inference attacks.

Although PLIA addresses the same problem as existing label inference attacks in federated learning, it targets a substantially different setting rather than extending them:

  • PLIA: To the best of our knowledge, PLIA is the first attack that infers client label distributions solely from the prototypes shared in prototype-based FL, by exploiting the geometry of the representation space, specifically the angular alignment between local and global prototypes.
  • Existing methods: They attack parameter-sharing FL such as FedAvg and, as summarized in the table below, depend on gradients or model updates, auxiliary data, synthetic data, or meta-classifiers, and most do not support model-heterogeneous FL.

As a result, PLIA offers three practical benefits:

  • Weaker attacker assumptions: It requires no auxiliary data or meta-classifiers.
  • Broader applicability: It naturally supports model-heterogeneous FL.
  • Low overhead: It requires only the prototypes from two consecutive rounds, whereas PGDA and SLIA must store all historical model updates up to the current round.

Comparison of label distribution inference attack and defense methods (✓\checkmark indicates supported/required; ×\times indicates not supported/not required).

CapabilitiesAttack RequirementsReferenceZhou et al. (2022)Gu and Bai (2023)Jiang et al. (2024b)Jiang et al. (2024a)Dai et al. (2024)OursAttackDefenseLabel DistributionModel HeterogeneityPPA×××LDIA×✓✓PGDA×✓×SLIA∗LDPM✓×Decaf×✓×PLIADPP✓✓Auxiliary DataSynthetic DataMeta Classifier✓×✓✓×✓×✓×✓××✓×××××\begin{array}{c|c|c} \hline & \textbf{Capabilities} & \textbf{Attack Requirements} \\\\ \hline \begin{array}{l} \textbf{Reference} \\\\ \hline \text{Zhou et al. (2022)} \\\\ \text{Gu and Bai (2023)} \\\\ \text{Jiang et al. (2024b)} \\\\ \text{Jiang et al. (2024a)} \\\\ \text{Dai et al. (2024)} \\\\ \hline \textbf{Ours} \end{array} & \begin{array}{cccc} \text{Attack} & \text{Defense} & \text{Label Distribution} & \text{Model Heterogeneity} \\\\ \hline \text{PPA} & \times & \times & \times \\\\ \text{LDIA} & \times & \checkmark & \checkmark \\\\ \text{PGDA} & \times & \checkmark & \times \\\\ \text{SLIA}^{*} & \text{LDPM} & \checkmark & \times \\\\ \text{Decaf} & \times & \checkmark & \times \\\\ \hline \textbf{PLIA} & \textbf{DPP} & \checkmark & \checkmark \end{array} & \begin{array}{ccc} \text{Auxiliary Data} & \text{Synthetic Data} & \text{Meta Classifier} \\\\ \hline \checkmark & \times & \checkmark \\\\ \checkmark & \times & \checkmark \\\\ \times & \checkmark & \times \\\\ \checkmark & \times & \times \\\\ \checkmark & \times & \times \\\\ \hline \times & \times & \times \end{array} \\\\ \hline \end{array}

∗^{*}For brevity, we refer to the attack method in Jiang et al. (2024a) as SLIA (Simple Label Distribution Inference Attack). The original authors describe it as a "simple yet effective label distribution inference attack" without using an acronym.

Q1-2. Why the proposed score reliably estimates label proportions. We fully agree that the current version does not formally support the relationship between angular alignment and label proportions. We have stated this explicitly as a limitation in Section 6 of the revised PDF (p. 10, L517--518). Nevertheless, our observation is consistent across settings, and Appendix A provides a relevant explanation: since the prototype regularizer is applied per sample, each class is weighted by its local share ni,j/Nin_{i,j}/N_i (Eq. 14), so classes with more local samples tend to receive a stronger pull toward their global prototype. What remains without formal support is how this pull translates into the final angle through training dynamics, which depend on the model, the optimizer, and the data. We therefore validate this link empirically.

Q2. Clarification of FedDistill. We apologize for the inconsistency and the incorrect description. Eq. (7) in the submission describes the cross-entropy formulation of the original FedDistill. However, our experiments actually use the squared Euclidean distance (MSE) between local and global logit prototypes, which can be verified at line 97 of code/system/flcore/clients/clientfeddistill.py in our supplementary material. This is the same regularizer as FedProto (Eq. 5) applied to logits, following the previous version of FedDistill implemented in PFLlib (https://www.pfllib.com/) and HtFLlib (https://github.com/TsingZ0/HtFLlib). We have corrected Eq. (7) and clarified these implementation details in the revised PDF (p. 4, L174--180).

W1. Novelty and relationship between the score and label proportions. Please refer to our responses to Q1.

W2. Coupling between attack and defense. We agree that DPP is specifically designed to disrupt the angular signal exploited by PLIA and does not provide general privacy protection. Since no label distribution inference attacks on prototypes existed before PLIA and existing attacks target parameter-sharing FL, we cannot evaluate DPP against them. We acknowledge this as a limitation of DPP. We have stated it in Section 6 of the revised PDF (p. 10, L518--521).

W3-1. Sensitivity to λ\lambda. Following the suggestion, we add a sensitivity analysis on the prototype regularization strength λ∈0.01,0.1,1,10,100\lambda \in \\{0.01, 0.1, 1, 10, 100\\} on HAR, CIFAR-10, CIFAR-100, and Tiny-ImageNet in Figure 4 and Section 5.2 of the revised PDF (p. 8, L420--425). Overall, a larger λ\lambda leads to stronger PLIA performance (lower LDL$^{\mathrm{TV}}$ without defense) as well as stronger DPP protection (higher LDL$^{\mathrm{TV}}$ with DPP), since stronger regularization can amplify the angular signal that PLIA exploits and DPP disrupts. However, when λ\lambda becomes excessively large (e.g., λ=100\lambda = 100), the model's performance degrades sharply, which in turn weakens the attack and produces the U-shaped PLIA trend shown in the figure.

W3-2. Ablation. To separate the effects of random signs and class-wise budgets, we add two variants with the orthogonal direction fixed: OD-UB-RS (uniform budget with random signs) and OD-CB-FS (class-wise budget with a fixed sign), as shown in the table below. Random signs are the main contributor: adding them to OD-UB substantially increases LDL$^{\mathrm{TV}}$, whereas class-wise budgets with a fixed sign yield only limited gains. Combining both components (DPP) achieves the strongest protection on four of five datasets, since PLIA relies on the relative alignment across classes and random signs with heterogeneous budgets jointly disrupt this ranking. All variants maintain accuracy comparable to the undefended baseline. We have updated these results in Table 4 and the corresponding text of the revised PDF (p. 10, L486--507).

Ablation study on defense components against PLIA. RS: random sign; FS: fixed sign. Bold: best TV per dataset.

HARAmazonCIFAR-10CIFAR-100Tiny INDefenseODCBRSNo Defense−−−RD-UB××−RD-CB×✓−OD-UB✓××OD-UB-RS✓×✓OD-CB-FS✓✓×OD-CB-RS (DPP)✓✓✓TV↑Acc.↑0.2840.8860.3120.8860.3170.8880.3160.8830.4250.8770.3330.8860.4950.878TV↑Acc.↑0.0680.8990.1030.8990.1400.8990.1470.8990.2820.8990.1720.8990.2640.899TV↑Acc.↑0.2730.6990.3220.7000.3530.7000.3300.6980.5680.6990.3780.6970.6600.700TV↑Acc.↑0.2870.3080.2990.3090.3020.3090.3240.3060.3520.3080.3330.3060.4300.307TV↑Acc.↑0.2080.1230.2370.1240.2460.1230.2660.1220.4260.1220.2880.1220.6120.121\begin{array}{c|c|c|c|c|c} \hline & \textbf{HAR} & \textbf{Amazon} & \textbf{CIFAR-10} & \textbf{CIFAR-100} & \textbf{Tiny IN} \\\\ \hline \begin{array}{lccc} \text{Defense} & \text{OD} & \text{CB} & \text{RS} \\\\ \hline \text{No Defense} & - & - & - \\\\ \hline \text{RD-UB} & \times & \times & - \\\\ \text{RD-CB} & \times & \checkmark & - \\\\ \text{OD-UB} & \checkmark & \times & \times \\\\ \text{OD-UB-RS} & \checkmark & \times & \checkmark \\\\ \text{OD-CB-FS} & \checkmark & \checkmark & \times \\\\ \hline \text{OD-CB-RS (DPP)} & \checkmark & \checkmark & \checkmark \end{array} & \begin{array}{cc} \text{TV}\uparrow & \text{Acc.}\uparrow \\\\ \hline 0.284 & 0.886 \\\\ \hline 0.312 & 0.886 \\\\ 0.317 & 0.888 \\\\ 0.316 & 0.883 \\\\ 0.425 & 0.877 \\\\ 0.333 & 0.886 \\\\ \hline \mathbf{0.495} & 0.878 \end{array} & \begin{array}{cc} \text{TV}\uparrow & \text{Acc.}\uparrow \\\\ \hline 0.068 & 0.899 \\\\ \hline 0.103 & 0.899 \\\\ 0.140 & 0.899 \\\\ 0.147 & 0.899 \\\\ \mathbf{0.282} & 0.899 \\\\ 0.172 & 0.899 \\\\ \hline 0.264 & 0.899 \end{array} & \begin{array}{cc} \text{TV}\uparrow & \text{Acc.}\uparrow \\\\ \hline 0.273 & 0.699 \\\\ \hline 0.322 & 0.700 \\\\ 0.353 & 0.700 \\\\ 0.330 & 0.698 \\\\ 0.568 & 0.699 \\\\ 0.378 & 0.697 \\\\ \hline \mathbf{0.660} & 0.700 \end{array} & \begin{array}{cc} \text{TV}\uparrow & \text{Acc.}\uparrow \\\\ \hline 0.287 & 0.308 \\\\ \hline 0.299 & 0.309 \\\\ 0.302 & 0.309 \\\\ 0.324 & 0.306 \\\\ 0.352 & 0.308 \\\\ 0.333 & 0.306 \\\\ \hline \mathbf{0.430} & 0.307 \end{array} & \begin{array}{cc} \text{TV}\uparrow & \text{Acc.}\uparrow \\\\ \hline 0.208 & 0.123 \\\\ \hline 0.237 & 0.124 \\\\ 0.246 & 0.123 \\\\ 0.266 & 0.122 \\\\ 0.426 & 0.122 \\\\ 0.288 & 0.122 \\\\ \hline \mathbf{0.612} & 0.121 \end{array} \\\\ \hline \end{array}

Minor a. Presentation. We have updated Figure 3 and the related text (p. 5, L228--230) to illustrate the details of PLIA and DPP better. We will also carefully revise the formatting and citation consistency in the revised version.

Minor b. Theoretical analysis. We acknowledge the limitations of our theoretical analysis. In particular, Proposition 1 does not directly establish label distribution privacy. We have stated these limitations in Section 6 of the revised PDF (p. 10, L517--518).

Minor c. Scope of claims. We fully agree. We have revised the claims in the abstract (p. 1, L22), contributions (p. 2, L78 and L82), and conclusion (p. 10, L518--521 and L526) to state that DPP is designed to mitigate the angle-based label distribution leakage exploited by PLIA in PBFL.

We have now updated the PDF once again to reflect the discussion with the reviewers, with new changes in green and those from the previous revision in blue. In the final version, we will further reorganize the paper to fully incorporate the discussion. We hope this addresses the reviewers' concerns; please let us know if you have any further questions. Thank you for your valuable feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.