acceptodds
Under review as a conference paper at ICLR 2027

DiSHE: Dispersion-Guided Selective Homomorphic Encryption for Federated Learning

Abstract

Homomorphic encryption (HE) lets a federated learning server aggregate client updates that it cannot read, but encrypting every parameter is expensive. Selective HE therefore encrypts only part of each update, usually under a single mask that all clients agree on. We present DiSHE, a selective CKKS protocol in which each client chooses its own mask. Updates are cut on a public grid of ciphertext-aligned chunks, so masks that differ across clients still aggregate slot by slot. Clients rank chunks by chunk gradient dispersion (CGD), the per-coordinate sample variance of per-example gradients on a small round-start probe, and always encrypt the classifier head. The same scores define a family of aggregation rules between FedAvg and inverse-dispersion weighting. Because encryption leaves the aggregated values unchanged, the learned model depends, in exact arithmetic, only on the aggregation rule; we characterize the resulting family, including a lower bound on every client's weight, and bound the ciphertext part of the stateless broadcast. With ResNet-18, five non-IID clients, ten rounds, and a nominal encryption ratio of 10%, the final global test accuracy of DiSHE is within 0.83 points of MaskCrypt and FedML-HE on CIFAR-10 and SVHN, and within 2.30 points on Fashion-MNIST. It exceeds our clipping-and-noise versions of FAS and ParaAegis by 4.96–49.51 points. At a 10% ratio, DiSHE needs no mask exchange and has the lowest per-round protocol time (1.51 s versus 1.82-15.54 s) of the five implementations. In masked-gradient attacks on LeNet-5, DiSHE is the only method that blocks both analytic label attacks, iDLG and iLRG, in every attempt.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Federated Learning, Homomorphic Encryption, Privacy-Preserving Machine Learning, Gradient Inversion Attacks, Secure Aggregation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_6TB7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DiSHE allows federated learning clients to choose different encryption masks without first agreeing on a common mask. A shared chunk layout keeps corresponding update coordinates aligned during mixed plaintext and ciphertext aggregation. Clients use per-example gradient dispersion to select chunks for encryption and always encrypt the classifier head. The same dispersion scores also determine aggregation weights. The paper provides aggregation and communication analysis, implements the protocol with CKKS, and evaluates model accuracy, protocol costs, and gradient reconstruction attacks.

Critical strengths
  1. The protocol supports different client masks without mask consensus. The shared chunk layout makes mixed plaintext and ciphertext aggregation straightforward. The broadcast analysis also makes the cost of combining different masks explicit.
  2. The analysis clearly separates encryption from changes to model training. With aggregation weights fixed, changing which chunks are encrypted does not change the aggregate in exact arithmetic. The paper then studies dispersion-based weighting separately.
Critical weaknesses
  1. The proposed dispersion-based selector has no clear demonstrated privacy advantage. In Table 2, DiSHE has the lowest LPIPS under both reconstruction attacks at every reported round, meaning that the reconstructed images are closer to the originals under those tests. Blocking the analytic label attacks comes from always encrypting the classifier head; it does not establish the benefit of dispersion-based selection. There is no controlled comparison against random selection with the same head protection and encryption budget.

  2. The communication advantage depends heavily on the baseline message format. The baseline implementations send large amounts of per-coordinate metadata. When counting only plaintext values and ciphertexts, DiSHE sends about 965 MB per round versus 862 MB for the baselines. The paper acknowledges that compact baseline formats could narrow or reverse the result. The current measurements therefore do not establish a communication advantage that comes from the method itself.

  3. The accuracy comparisons do not isolate the benefit of dispersion-based weighting. Runs are not paired on data partitions and initialization. The FAS and ParaAegis adaptations also use substantial clipping and noise under unmatched privacy settings, which limits what their large accuracy gaps tell us. Without a controlled weighting ablation, it remains unclear whether inverse-dispersion aggregation improves on ordinary data-size weighting.

Questions for authors
  1. With classifier-head encryption, aggregation weights, and the actual communication budget held fixed, does dispersion-based selection improve attack resistance over random selection or a strong existing selector?

  2. What accuracy and communication advantages remain over the strongest noise-free baselines when all methods use the same data partitions and initialization, along with equally compact message formats?

2 comments
AuthorsOpenReview ↗

Response to Reviewer 6TB7

We thank Reviewer 6TB7. We provide the requested selection experiment, accept the communication and noise-calibration concerns, and clarify pairing and the remaining weighting ablation.

W1 / Q1: CGD versus Random and Gradient Magnitude.

We agree that the mandatory classifier-head rule confounded the original evidence. We therefore disabled it for all selectors on LeNet-5 / Fashion-MNIST (α=0.05\alpha=0.05, b=256b=256, K=12K=12, Round 4, seed 123), holding the checkpoint, chunk map, partitions, targets, and attack settings fixed.

Across 5 client masks and 3 targets, IG success is 1/15 (6.67%) for CGD, versus 7/15 (46.67%) for Random and 5/15 (33.33%) for Magnitude. CGD leaves 5.14% of classifier weights visible versus 100% and 98.29%; iDLG label recovery succeeds in 9/15 versus 15/15 for both baselines. IG LPIPS is 0.4954 versus 0.3378 and 0.3611. DLG success is zero for all three.

This supports a benefit from CGD without the head rule in this setting. PART 1, Section 1 (Controlled Selection Ablation) contains the complete metric tables and notes the one-seed, three-target scope.

W2: Communication accounting. Accepted.

Counting raw ciphertexts and plaintext values, DiSHE sends approximately 965 MB versus 862 MB (about 12% higher), reflecting the union broadcast. The original 45-56% total-savings claim depends on baseline serialization and metadata accounting; it is not a compact-payload advantage.

We will use ciphertext-plus-plaintext bytes as the primary payload metric, report metadata separately, and qualify the encoding-dependent claim. Eliminating the mask-consensus exchange remains a protocol benefit; 1.51 s versus 1.82-15.54 s is the measured runtime in our reported setup, not a universal latency guarantee.

W3 (1): Pairing of partitions and initialization.

For each seed (123, 456, 789), all five methods use identical client partition indices and initial state_dict, with shared model, data-loading, and local-training code (Section 6.1).

The Limitations sentence concerns a separate issue in app:grid: the size-borrowing allocator can reuse a sample across clients within a run. We will clarify that this does not mean unpaired comparisons. Correcting the allocator changes the partition construction, so affected comparisons must be rerun consistently before reporting corrected results.

W3 (2): FAS/ParaAegis clipping and noise. Accepted.

Appendix F documents the mismatch: sensitivity was derived as 2C/ni2C/n_i, while clipping was applied to multi-epoch Adam updates. Mean clipping coefficients were 0.024-0.046, and aggregate RMS noise norms were 4.30 (CIFAR-10) and 21.52 (Fashion-MNIST/SVHN), against clipped signal norm at most 1.

The 20.7-49.5 percentage-point accuracy losses do not establish the intrinsic cost of DP. We will label these rows as uncalibrated adaptations and remove claims of superiority based on those gaps. A meaningful DP comparison requires compatible clipping, sensitivity, and privacy accounting.

W3 (3): Isolated weighting ablation.

We agree that paired seeds and cross-method Table 1 results do not replace an isolated weighting ablation. MaskCrypt and FedML-HE themselves differ by 0.48-0.61 percentage points despite both using data-size weights; their comparison with DiSHE therefore cannot cleanly isolate weighting.

The DiSHE-to-HE-baseline gaps fall below DiSHE's reported three-seed standard deviations, but this proves neither equivalence nor a weighting benefit. We will include a within-DiSHE γ∈{0,0.5,1}\gamma\in\{0,0.5,1\} evaluation with partitions, initialization, masks, and other settings held fixed; this response does not present that experiment as completed.

Q2: Remaining advantages under compact formats and matched runs.

The supported claims are eliminating mask-consensus exchanges, the measured protocol runtime, and improved IG resistance against Random/Magnitude in the controlled chunk-selection experiment. These come with approximately 12% higher raw payload in the reported breakdown. We do not claim demonstrated accuracy superiority, a calibrated DP advantage, or universal privacy superiority over scattered masks.

Thank you for your response. After careful consideration, I have decided to keep my score.

Reviewer_WeyrOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes to use chunk gradient dispersion to allow client-independent encryption masking, where traditional work in selective encryption for privacy-preserving FL requires client encryption mask consensus.

Critical strengths

The paper attempts to propose a new selective encryption aggregation for federated learning algorithms.

Critical weaknesses
  1. Although the main motivation for this work is to have a way to remove the common restriction of encryption mask alignment across clients for selective encryption for FL, allowing client-configurable encryption masks itself brings in challenges on formally quantify privacy guarantee for selective data protection. This is where the paper misses to address and fundamentally weakens the argument of using"chunk gradient dispersion" to optimize only the training performance.

  2. The paper is quite hard to follow in terms of the writing to even understand the main idea. For example, it is hard for me to understand what it means as "because encryption leaves the aggregated values unchanged, the learned model depends, in exact arithmetic, only on the aggregation rule; we characterize the resulting family, including a lower bound on every client's weight, and bound the ciphertext part of the stateless broadcast." This type of writing is throughout the entire paper which is hard for readers to capture its intended meaning for the core research story here.

  3. In several evaluation setups, DiSHE is not actually better or at most marginally better than prior work. For example, Table 1 shows several instances where it is only on par with MaskEncrypt and FedML-HE. This leaves the question on whether this proposed solution can generalize. If not, what is the constraints needed for it to work?

Questions for authors
  1. For formal privacy guarantee, can you explain what this work means for the case where the server now can see different plaintext chunk distribution across different clients, compared to unified encryption local model parts?
  2. Is there any overhead insights since now you have a less unified ciphertext/plaintext portion each time for federated aggregation?
2 comments
AuthorsOpenReview ↗

Response to Reviewer Weyr

We thank Reviewer Weyr. We agree that the privacy discussion and presentation need clarification, and address the accuracy and overhead questions below.

W1 / Q1: Client-specific masks versus a unified mask.

Our threat model is an honest-but-curious server with CKKS ciphertext confidentiality, not a formal differential-privacy guarantee for the full transcript.

  • Visible information: The server sees the CC-bit selection vector, quantized dispersion scores and per-layer anchors when enabled, client size nin_i, and unencrypted coordinates. IND-CPA security of ciphertexts does not hide this metadata or the plaintext coordinates.
  • Unified-mask comparison: Mask negotiation can also disclose local importance proposals. A shared mask can leave a client's locally sensitive chunks unprotected. These are different exposure mechanisms; we do not claim that either mask policy universally dominates the other in privacy.
  • Mitigation and limit: Every client selects KK chunks. The new ablation finds that CGD encrypts 94.86% of classifier weights without the mandatory head rule (PART 1, Section 1), but layer concentration alone does not establish client-invariant masks. By Remark 3, γ=0\gamma=0 omits dispersion-score uploads while retaining the consensus-free protocol; mask leakage remains. We will distinguish ciphertext security, metadata exposure, and empirical attack resistance.

W2: Writing clarity. Accepted.

We will replace the dense introduction sentence with: "For fixed aggregation weights, encryption changes which values the server can read while preserving the intended weighted sum up to CKKS approximation error. We analyze the aggregation rule separately."

W3: Accuracy is on par with MaskCrypt and FedML-HE.

We agree that Table 1 does not establish an accuracy advantage. Remark 1 separates encryption placement from the intended weighted sum when aggregation weights are fixed. CKKS introduces numerical approximation error, but does not add the DP perturbations used in the perturbed baselines. Our accuracy objective is to preserve training performance while changing the protocol and exposure of updates.

The largest DiSHE-to-HE-baseline gaps are 0.83, 2.29, and 0.51 percentage points on CIFAR-10, Fashion-MNIST, and SVHN, respectively, versus DiSHE standard deviations of 1.73, 4.15, and 0.83. This does not prove statistical equivalence or isolate the effect of weighting. We will avoid those stronger interpretations.

The measured benefits are eliminating mask-consensus exchanges, a 1.51 s protocol runtime versus 1.82-15.54 s in the reported setup, and better IG resistance than Random/Magnitude within the new controlled chunk-selection experiment. These are setting-specific results, not an accuracy-superiority claim.

Q2: Overhead of non-unified ciphertext/plaintext portions.

Proposition 3 gives K≤∣Ut∣≤min⁡{C,∣M∣+N(K−∣M∣)}K \le |U^t| \le \min\{C, |M|+N(K-|M|)\}. For ResNet-18 at α=0.10\alpha=0.10, this is 275 to 1,367 of 2,746 chunks. The stateless server prevents the encrypted union from accumulating across rounds.

Figure 1(c) reports union coverage of 8.8%, 16.2%, 34.8%, and 81.5% at per-client budgets of 5%, 10%, 25%, and 75%. At 10%, the mean is 444 chunks. This expansion has a real payload cost: raw ciphertext-plus-plaintext traffic is approximately 965 MB versus 862 MB for the baselines (about 12% higher). We will separate payload from metadata and qualify the original encoding-dependent communication-savings claim. Removing consensus exchanges and reducing measured protocol runtime are separate benefits.

Thanks for the reply. I am still very concerned about the threat model issue introduced by client-configurable encryption mask mechanism in the paper. I will maintain my score.

Reviewer_ULqbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a selective CKKS homomorphic encryption scheme for federated learning. Unlike methods that require a unified encryption mask. This method maps parameters to public, fixed chunks aligned with ciphertext slots, allowing each client to independently decide which areas need to be encrypted.

Critical strengths

Fixed chunk alignment enables homomorphic aggregation between different clients without negotiating a common mask; the stateless server only broadcasts the encrypted aggregation result corresponding to the current round of ciphertext union.

The paper clearly distinguishes between encryption selection and training value trajectory: under fixed data, weights and ideal precision operations, the mask only affects the visible content and cost of the server and does not change the aggregate value. This clarification helps avoid misinterpretation of accuracy experiments.

Critical weaknesses

The paper emphasizes CGD, but fails to validate its core block-selection hypothesis. It lacks direct comparisons with baseline policies (e.g., random chunking or gradient-amplitude selection) under identical conditions.

CGD bin offers no clear protection advantage over baseline masking, and Table 2 shows DiSHE achieves the lowest LPIPS under real labels, implying higher reconstruction fidelity.

After fixing the classification header protection, chunk map and effective encryption budget, how much does CGD improve compared to random and gradient amplitude block selection in terms of reconstruction quality, CGD/mask side channel and ciphertext union size?

Questions for authors

refer to weaknesses

2 comments
AuthorsOpenReview ↗

Response to Reviewer ULqb

We thank Reviewer ULqb for recognizing our stateless fixed-chunk protocol and the separation of encryption selection from aggregation.

W1 / Q1: Controlled selection comparison.

We ran the requested ablation on LeNet-5 / Fashion-MNIST (α=0.05\alpha=0.05, b=256b=256, K=12K=12, Round 4, seed 123), using 5 client masks and 3 target images. Mandatory classifier-head encryption was disabled for every selector, and the checkpoint, chunk map, targets, and attack settings were shared.

  • IG reconstruction success (SSIM >= 0.5) is 1/15 (6.67%) for CGD, versus 7/15 (46.67%) for Random and 5/15 (33.33%) for Gradient Magnitude. DLG success is 0/15 for all three.
  • CGD leaves 5.14% of classifier weights visible, versus 100% and 98.29%, and iDLG label recovery succeeds in 9/15 cases versus 15/15 for both baselines.

Thus, CGD provides protection beyond the mandatory head rule in this experiment. PART 1, Section 1 (Controlled Selection Ablation) gives the full setup, both metric tables, and the limited evaluation scope.

W2: Table 2 LPIPS and reconstruction fidelity.

Under IG in the new ablation, CGD has higher LPIPS (0.4954 versus 0.3378 and 0.3611), higher MSE (0.09475 versus 0.03684 and 0.03772), and lower PSNR (10.464 dB versus 14.824 and 14.413 dB). These metrics agree with the lower IG success rate; DLG's LPIPS ranking differs, as reported in the shared table.

This does not negate Table 2. That table compares DiSHE's contiguous chunks with the scattered coordinate masks of other protocols. Scattered masking can disrupt gradient correlations differently at a comparable parameter budget. The new experiment isolates the selector within our chunk-aligned protocol. We will present the cross-protocol trade-off and the within-protocol selection result separately.

Q1: Ciphertext union and metadata leakage.

  • Union size: Proposition 3 bounds the broadcast union, and Figure 1(c) measures it. For ResNet-18 at α=0.10\alpha=0.10 (C=2,746C=2,746), the mean union is 16.2% (444 chunks), below the 1,367-chunk worst-case bound and the approximately 1,121 chunks expected from independent uniform random selection. The shared head policy and overlapping client selections help explain the compact union.
  • Metadata: Fixed cardinality KK hides variation in the number of selected chunks, not their identities. Concentration on the same layer does not establish identical masks or absence of data-dependent leakage. By Remark 3, γ=0\gamma=0 (DiSHE0\text{DiSHE}_0) uses data-size weights and omits dispersion-score uploads, removing that score channel. The visible masks and plaintext coordinates remain outside the ciphertext confidentiality guarantee. We will state these limits explicitly.

Thank you for your response. After understanding your response, other reviews, and careful consideration, I keep my score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

We thank all three reviewers. This shared response reports the selection ablation requested by ULqb and 6TB7. The replies under each review address that reviewer's specific concerns and summarize the relevant findings.

1. Controlled Selection Ablation

Setup. LeNet-5 / Fashion-MNIST, α=0.05\alpha=0.05, chunk size b=256b=256, K=12K=12, Round 4 of 10, seed 123. We disabled mandatory classifier-head encryption for all three selectors. Only the selection criterion changes; the chunk map, partitions, initialization, training checkpoint, targets, and attack settings are shared. We evaluated 5 client masks and 3 fixed target images: 15 cases per selector and attack. DLG used 3 optimization restarts per case (135 DLG optimization runs across selectors). Round 4 was chosen as an intermediate checkpoint between initialization and near-convergence.

SelectorIG success (SSIM >= 0.5)DLG successiDLG label correctiLRG availableHead weights visible
CGD1/15 (6.67%)0/159/159/155.14%
Random7/15 (46.67%)0/1515/1515/15100.00%
Gradient Magnitude5/15 (33.33%)0/1515/1512/1598.29%

Classifier-bias visibility is 60%, 100%, and 80%, respectively. Total coordinate visibility is 95.26%, 95.02%, and 95.10%; the experiment fixes the selected-chunk count, while the realized coordinate fractions differ slightly.

Reconstruction metrics (higher LPIPS/MSE and lower PSNR indicate poorer reconstruction):

SelectorIG LPIPSIG MSEIG PSNR (dB)DLG LPIPSDLG MSEDLG PSNR (dB)
CGD0.49540.0947510.4640.53230.115189.443
Random0.33780.0368414.8240.61290.0977110.439
Gradient Magnitude0.36110.0377214.4130.61540.107789.827

Interpretation. Without a mandatory head rule, CGD encrypts 94.86% of classifier weights and yields fewer successful iDLG label recoveries (9/15 versus 15/15). Under IG, it reduces reconstruction success to 1/15, with all three IG fidelity metrics indicating poorer reconstruction. DLG success is zero for every selector, and its LPIPS ranking differs; we therefore do not claim improvement on every metric or attack.

These results support the selection criterion in this setting. They cover one checkpoint, one seed, and three target images, rather than establishing a universal structural property or a formal privacy guarantee.

Relation to Table 2. Table 2 compares different protocols, including contiguous chunk masks and scattered coordinate masks. This ablation compares selectors within the same chunk-aligned protocol. Better results within this ablation do not overturn the cross-protocol privacy trade-off in Table 2. We will make that distinction explicit.

2. Shared Clarifications and Revisions

  • Pairing: For each seed (123, 456, 789), all five methods use identical client partition indices and initial model weights. The Limitations sentence about overlapping samples concerns borrowing between clients within a run, not a lack of cross-method pairing; we will clarify this distinction.
  • Accuracy and weighting: The Table 1 gaps are smaller than DiSHE's reported three-seed standard deviations. This is descriptive, not proof of equivalence or an isolated weighting effect. An explicit within-DiSHE weighting ablation is still needed.
  • Communication: Raw ciphertext-plus-plaintext payload is approximately 965 MB for DiSHE versus 862 MB for the baselines (about 12% higher). We will report payload and metadata separately and qualify encoding-dependent savings. Eliminating mask-consensus exchanges remains a distinct systems benefit.
  • Perturbed baselines: We accept the clipping/noise-calibration concern for the FAS/ParaAegis adaptations and will not use their accuracy gaps as evidence of the intrinsic cost of DP.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.