acceptodds
Under review as a conference paper at ICLR 2027

FedPRA: Policy-Residual Aggregation for Heterogeneous Differentially Private Federated Learning.

Abstract

Federated learning (FL) under heterogeneous privacy policies poses significant challenges. Existing private FL methods assign different privacy budgets to clients, yet only exploit these budgets to control numerical privacy loss. They do not measure whether each trained client model satisfies its own operational policy constraints. In this work, we show that privacy-budget order and policy satisfaction order may diverge. Consequently, conventional data-volume aggregation fails to capture policy satisfaction. To this end, we propose FedPRA to tackle this issue via two complementary designs. First, a differentiable policy engine quantifies each client’s residual constraint violation during local model training with record-level differential privacy. Second, policy-residual aggregation leverages these residuals to guide global aggregation. We prove that the proposed weighting rule prioritizes clients with smaller residuals and preserves a set of theoretical guarantees under disjoint and overlapping client data. Experiments based on two benchmark datasets demonstrate the superiority and competitiveness of FedPRA.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
federated learning, differential privacy, heterogeneous privacy budgets, cross-jurisdictional learning, privacy-aware aggregation, policy-constrained learning, imbalanced classification, worst-client performance

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_i5mtOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a federated learning framework for clients with heterogeneous differential-privacy budgets and operational policy constraints. Its main idea is to compute a policy-residual score for each locally trained model and use this residual, rather than data volume alone, to determine aggregation weights, while retaining client-specific record-level DP guarantees.

Critical strengths
  • The paper identifies an interesting distinction between numerical DP budgets and satisfaction of broader operational privacy policies, and the policy-residual aggregation mechanism is conceptually clear.
  • The theoretical treatment covers residual ordering and DP guarantees for both disjoint and overlapping client data, while experiments compare against several recent private-FL baselines on two datasets.
Critical weaknesses
  • The practical motivation for heterogeneous privacy requirements needs to be considerably strengthened. The framework assumes that collaborating clients can operate with materially different privacy budgets and policy targets. While technically possible, such unequal privacy protection may be difficult to justify in a real multi-organisation collaboration, particularly where participants jointly contribute to the same global model. The paper motivates this through “GDPR-, PIPL-, and HIPAA-inspired” profiles, but also acknowledges that the numerical targets are not prescribed by these regulations and would need to be configured by individual organisations. This leaves an important practical question: under what realistic governance scenario would collaborating organisations accept substantially different protection levels rather than adopting a mutually agreed minimum protection level?

  • All major experiments use only three clients with one fixed privacy-budget configuration, (1,2,3). This same configuration is used across the principal baseline comparisons, and even the detailed privacy-accounting appendix only verifies realised privacy losses for these three values. The paper therefore does not show how FedPRA behaves as either the absolute privacy levels or the degree of heterogeneity changes. A more convincing evaluation should vary the privacy-budget configurations, e.g., homogeneous versus mildly and strongly heterogeneous settings, and report the resulting global-model utility, worst-client utility, residuals and privacy guarantees.

Questions for authors
  • What realistic cross-organisation deployment scenario motivates allowing participants to adopt substantially different DP budgets and operational privacy targets? How would fairness and governance concerns arising from unequal privacy protection be resolved?

  • How sensitive is FedPRA to the magnitude and heterogeneity of the privacy budgets? Could the authors evaluate several configurations and show how global utility, worst-client performance, policy residuals and the overall DP guarantee vary?

  • Since the current results combine heterogeneous DP budgets, policy-aware local training, class reweighting/resampling and residual-based aggregation, can the authors more clearly isolate the privacy–utility trade-off attributable specifically to heterogeneous privacy protection?

Reviewer_s9YAOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes FedPRA for federated learning with heterogeneous privacy policies. It introduces differentiable policy tests to measure each client’s remaining policy violations and uses the resulting residuals to guide global aggregation under the stated assumptions. Experiments on two network-intrusion datasets show improved utility over several private federated learning baselines.

Critical strengths

S1. The paper identifies a gap between numerical privacy budgets and actual policy satisfaction in heterogeneous federated learning.

S2. FedPRA provides a simple and interpretable framework that connects local policy evaluation with global aggregation using only one additional residual value per client.

S3. The paper offers theoretical analysis under explicit assumptions and provides relatively comprehensive experiments.

Critical weaknesses

W1. The evaluation scope is narrow relative to the general claims.

The experiments use two network-intrusion datasets, three clients, one architecture, one Dirichlet concentration, fixed participation, and disjoint client partitions. Overlapping data, client arrival and departure, malicious clients, calibration shift, larger client populations, and sensitive policy targets are treated only theoretically or left outside the scope. The claim that the framework handles broadly heterogeneous privacy policies is therefore stronger than the current empirical evidence.

W2. The policy tests are not shown to reflect real privacy protection.

The three policy scores are mathematically well defined, but the paper does not demonstrate that obtaining a better score actually means that the model retains less sensitive information or better satisfies a real-world privacy requirement. In particular:

suppressing more latent dimensions may not reduce the amount of retained information, because the model can encode the same information redundantly in the remaining dimensions; the representation-magnitude score can change when the encoder and classifier are rescaled, even if the model’s predictions and retained information remain unchanged; the numerical targets labeled as GDPR-, PIPL-, and HIPAA-inspired are manually chosen and are not derived from specific requirements in these regulations. Proposition 1 establishes only a definition-level property: the residual is zero when the three inequalities defined by the authors are satisfied. It does not show that satisfying these inequalities corresponds to meaningful data minimization, non-retention, structural privacy, or regulatory compliance in practice. The paper therefore needs stronger justification or independent empirical evidence connecting these scores to actual privacy outcomes.

W3. The source of the performance improvement is unclear.

Figure 2(a) shows that class reweighting and resampling lead to large performance gains, whereas the matched aggregation ablation shows only a modest improvement of PRA over uniform aggregation. This makes it difficult to determine how much of FedPRA’s final performance is attributable to the proposed aggregation rule itself.

Questions for authors

pls see weaknesses

Comments on the paper

2 comments

Paper Summary

This paper identifies a potential mismatch between privacy-budget ordering and policy-satisfaction ordering in federated learning. To address this issue, the authors propose FedPRA, a framework that combines a differentiable policy engine with policy-residual aggregation.

Strengths

  1. The paper identifies a meaningful problem: privacy-budget ordering does not necessarily reflect policy satisfaction.
  2. The proposed framework is clearly presented and easy to follow.
  3. The experiments include additional ablation studies on different aggregation strategies, providing useful insights into the effectiveness of the proposed method.

Weaknesses

  1. The experiments do not sufficiently demonstrate the necessity of policy-residual aggregation. For example, in Table 3, uniform aggregation achieves a slightly higher Mean BA than FedPRA, while FedPRA only marginally improves Worst BA. These results raise questions about whether the proposed policy residual provides meaningful advantages over simpler aggregation strategies.
  2. The connection between the defined policy constraints and actual privacy risks remains unclear. Although the three policy constraints are intuitive and easy to understand, it is not sufficiently demonstrated whether these surrogate measures meaningfully reflect real-world privacy risks or policy compliance.

Review Summary

FedPRA identifies a potentially meaningful problem and proposes a corresponding solution. Overall, this is a relatively solid manuscript with a clear framework and encouraging experimental results. However, I have broad familiarity with this field but little direct research experience on this specific topic. After reading the other reviewers' comments, I found that they raised several similar and more insightful concerns. Taking these considerations into account, I would personally lean toward a weak rejection.

AuthorsOpenReview ↗

Response to AC and Reviewers: PRA’s Added Value, Privacy Relevance, and Scope

1. Added value over uniform aggregation and attribution (AC: W1; s9YA: W3)

Table 3 supports PRA as a targeted refinement of uniform aggregation: lower policy violation with competitive predictive utility. Across ten paired runs, PRA reduces the final global model's mean residual from 0.042 to 0.037 (11.9%). Macro-F1 rises from 0.828 to 0.8341 and WorstBA from 91.88% to 92.28%, while MeanBA decreases by 0.19 percentage points. The case for PRA rests on this joint trade-off, not the 0.40-point WorstBA gain alone.

PRA also improves all four reported means over shuffled-residual aggregation, including WorstBA from 91.45% to 92.28% and residual from 0.043 to 0.037. This supports the usefulness of matching residuals to the correct clients. Table 3 evaluates the resulting global model, not merely the local residuals used to set weights.

Equal residuals recover uniform weighting, and Proposition 3 bounds any pairwise weight ratio by two. Communication adds one scalar per client per round. This provides policy-dependent influence through bounded weight adjustment.

Table 3 holds local training and class handling fixed, isolating aggregation. Figure 2(a)'s larger class-handling gains should not be attributed to PRA.

2. Connection to concrete privacy risks (AC: W2; s9YA: W2)

Related representation controls have addressed concrete privacy threats:

  • PATROL combines privacy-oriented pruning with reconstruction-aware training to defend against recovery of private images from intermediate features [1].
  • Shredder learns representation noise for cloud inference and reports reduced information leakage about the original inputs [2].
  • DP-BART combines representation clipping, pruning, and calibrated noise to protect sensitive text during rewriting [3].

These examples support the motivation for suppression, perturbation, and magnitude control. They do not validate our exact scores as leakage estimators. In FedPRA, the retention score tracks prediction-sensitive representation energy, not verified sensitive attributes or record deletion. Suppression can leave redundant information, and magnitude is scale-dependent.

FedPRA makes configured representation requirements measurable and usable in aggregation. Record-level privacy rests on DP-SGD and complete accounting under the stated assumptions. Residuals computed from DP models and public quantities preserve that guarantee under Proposition 2's assumptions. The regulation-inspired profiles are illustrative operational configurations, not regulation-derived thresholds or compliance certificates.

3. Practical motivation and governance (i5mt: W1/Q1)

A common minimum protection level does not require identical budgets. Consider a hospital consortium that agrees on an epsilon ceiling of 3 and a common delta. Hospitals may choose stricter budgets of 1 or 2 because of local requirements. All satisfy the common ceiling. Under Proposition 2's assumptions, disjoint records retain their corresponding client-specific transcript guarantees. Overlapping records require composition.

This illustrates agreed local autonomy, not automatic fairness of unequal protection. Budget approval and fairness requirements remain governance decisions. Personalized privacy in cross-silo FL is also studied in CCS 2024 [4].

4. Evaluation scope and budget sensitivity (s9YA: W1; i5mt: W2/Q2/Q3)

The empirical claim concerns three clients, fixed participation, and two intrusion datasets. The theory covers arbitrary participating sets, overlap, bounded calibration perturbations, and join/leave changes under stated assumptions. Public calibration data and policy targets, and honest clients, are explicit conditions.

Budgets affect PRA indirectly through local training and the resulting residuals. The fixed (1,2,3) experiment verifies one configuration, not sensitivity across configurations. A sweep with recalibrated DP noise is needed. To isolate budget heterogeneity, homogeneous and heterogeneous settings must also keep local objectives, class handling, and aggregation fixed. Table 3 already isolates aggregation, but does not replace this budget experiment. We will limit the corresponding claims to the evidence available.

References

[1] Ding et al. PATROL: Privacy-Oriented Pruning for Collaborative Inference Against Model Inversion Attacks. WACV 2024.

[2] Mireshghallah et al. Shredder: Learning Noise Distributions to Protect Inference Privacy. ASPLOS 2020.

[3] Igamberdiev and Habernal. DP-BART for Privatized Text Rewriting under Local Differential Privacy. Findings of ACL 2023.

[4] Liu et al. Cross-silo Federated Learning with Record-level Personalized Differential Privacy. CCS 2024.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.