acceptodds
Under review as a conference paper at ICLR 2027

Latent Information Sharing for Accelerating Federated Learning

Abstract

Federated learning (FL) is a communication-efficient distributed learning paradigm. However, client drift remains one of the most critical challenges, hindering the efficient training of a global model. In this study, we propose a novel latent information sharing scheme that directly mitigates data heterogeneity across clients. Our theoretical and empirical results show that sharing a small amount of hidden-layer activations significantly improves training efficiency while preserving convergence guarantees and data privacy. Furthermore, we compare our method with existing FL approaches designed to address client drift, including FedProx, SCAFFOLD, FedPVR, FedProto, and SplitFed, and demonstrate superior model accuracy under a fixed epoch budget without incurring excessive communication overhead. Overall, this work presents a promising new knowledge aggregation scheme and provides a comprehensive analysis of the impact of activation sharing on federated optimization.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Federated Learning, Activation Sharing, Data Heterogeneity, Client Drift

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_NyxLOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Activation Sharing Federated Learning (ASFL), a federated optimization approach intended to reduce client drift under heterogeneous client data distributions. ASFL exchanges intermediate activations and associated labels between participating clients. Each client subsequently performs partial-model updates on both local and received activations before conventional full-model training and aggregation.

The authors introduce a layer-selection heuristic balancing computational/communication cost against gradient variance, analyze convergence for smooth non-convex objectives, and investigate optional differential privacy mechanisms. Experiments on CIFAR-10, CIFAR-100, SVHN, and FEMNIST report improved final accuracy over several federated learning baselines under fixed communication rounds.

Critical strengths
  1. Meaningful empirical improvements with a useful causal ablation.

ASFL achieves consistent accuracy improvements across the four principal benchmarks. The CIFAR-10 results are particularly encouraging. Importantly, Appendix Table 14 compares ASFL against a partial-training variant without remote activations and shows a 9.04 percentage-point improvement from incorporating the shared representations. This helps isolate the benefit of the proposed information-sharing mechanism.

  1. A practical approach to reducing the cost of representation sharing.

Training only the model tail using received activations, combined with reuse of a small activation pool, offers a potentially useful mechanism for reducing redundant computation. The ablations on sharing layers and activation quantities help characterize this trade-off.

Critical weaknesses
  1. The convergence analysis contains a potentially invalid bound.

In the proof of Lemma 4 (Appendix A.2, page 16), the discrepancy between client gradients and the global gradient is bounded using the within-client stochastic variance (\sigma_z^2). This discrepancy is instead associated with between-client gradient heterogeneity, represented by (\sigma_G^2) in the assumptions.

The bound does not generally follow from the stated assumptions. Moreover, the analysis should justify its gradient unbiasedness and independence assumptions under the finite, repeatedly reused activation pools implemented in ASFL.

Since this lemma is used in the subsequent theoretical derivations, the convergence and linear-speedup claims require reconsideration.

  1. The efficiency and privacy claims are not yet sufficiently established.

The reported timing results measure local computation rather than complete federated training time, excluding the additional peer-to-peer communication and synchronization costs. Communication overhead is also substantial on certain benchmarks; for example, it nearly doubles relative to FedAvg on SVHN.

The differential privacy analysis primarily considers perturbations to transmitted activations, while the protocol also shares corresponding labels and continues to exchange model updates. The scope of the claimed privacy guarantee, the treatment of labels, and privacy composition across rounds require clarification. The limited model-inversion experiment does not establish broader privacy protection.

The comparison to earlier feature-sharing approaches, particularly FedFed and FedMix, should also be strengthened to clarify the specific novelty of ASFL.

Questions for authors

Q1 — Theoretical correctness: Can the authors clarify or correct the bound in Lemma 4 and show how the convergence argument applies to the actual activation-reuse procedure? Does the claimed linear-speedup result continue to hold under partial participation with a fixed number of active clients?

Q2 — System efficiency: Can the authors provide end-to-end convergence measurements against wall-clock time and total bytes communicated, including peer transfers and synchronization? How does ASFL compare against appropriate baselines under matched communication budgets?

Q3 — Privacy: What is the precise adjacency relation and adversarial threat model for the stated DP guarantee? Are the transmitted labels considered public? How are repeated releases and model-update disclosures accounted for? Can the authors substantiate their privacy claims beyond the limited reconstruction examples?

1 comment

We appreciate the constructive review. We hope our responses address your concerns, and we would be grateful if you could reconsider your evaluation.

Q1, W1: Can the authors clarify or correct the bound in Lemma 4 and show how the convergence argument applies to the actual activation-reuse procedure? Does the claimed linear-speedup result continue to hold under partial participation with a fixed number of active clients?

Corrected bound. We thank the reviewer for pointing out our mistake. The model discrepancy term in Lemma 4 should be bounded by the global variance σG2\sigma_G^2, not σz2\sigma_z^2. We corrected it in the main text and appendix.

Improved results. The revised bound shows the benefit of activation sharing (Remark 2). It reduces the client drift term from 9τ′9\tau' at α=1\alpha = 1 (no sharing) to 9τ′2+1\frac{9\tau'}{2} + 1 at α=12\alpha = \frac{1}{2}, while the extra variance from random peer selection is scaled by 1a\frac{1}{a}.

Linear speedup. With aa clients sampled without replacement per round, the dominant term O(1/aT)\mathcal{O}(1/\sqrt{aT}) decreases with aa for sufficiently large TT, so ASFL still achieves a linear speedup.

Q2, W2: System efficiency: Can the authors provide end-to-end convergence measurements against wall-clock time and total bytes communicated, including peer transfers and synchronization?

As suggested, we added learning curves against the cumulative communication cost in bytes (Figure 5). Since we simulate 100 clients, measuring the actual wall-clock time is not feasible. We therefore report the computation and communication costs separately. The per-round computation cost is reported in Table 2, the per-round communication cost in Table 1, and the end-to-end curves in Figure 5.

The end-to-end curves in Figure 5 show that ASFL achieves higher accuracy than the other methods under a comparable communication budget. Notably, SplitFed exchanges activations at every local iteration, so its communication cost grows rapidly with the input size and the number of local steps. In contrast, ASFL requires only a single additional transmission of an activation batch per round.

Q3, W2: Privacy: 1) What is the precise adjacency relation and adversarial threat model for the stated DP guarantee? 2) Are the transmitted labels considered public? 3) How are repeated releases and model-update disclosures accounted for? 4) Can the authors substantiate their privacy claims beyond the limited reconstruction examples?

1) Adjacency and threat model. Like metric DP (Chatzikokolakis et al., 2013), PixelDP (Lecuyer et al., 2019) defines neighbors by input distance: ξ′=ξ+η\xi' = \xi + \eta with ∥∥η∥∥p≤R\|\| \eta \|\|_p \le R. We consider an honest-but-curious recipient observing all received activations.

2) Labels considered public?. Labels are not considered public. Our focus is the benefit of remote latent information, and label leakage can be mitigated by existing label DP methods. We applied KK-ary randomized response, keeping each label with probability eϵℓeϵℓ+K−1\frac{e^{\epsilon_\ell}}{e^{\epsilon_\ell} + K - 1}. ASFL maintains competitive accuracy under label DP (Table 1).

Table 1. Accuracy with PixelDP and label DP (Laplacian ϵ=2.0\epsilon=2.0, Gaussian (ϵ,δ)=(0.5,0.05)(\epsilon,\delta)=(0.5,0.05), “–”: no label DP)

Noiseϵℓ\epsilon_\ellCIFAR-10CIFAR-100SVHNFEMNIST
Laplacian–74.7 ± 1.049.1 ± 0.688.5 ± 0.385.8 ± 0.2
5.074.4 ± 0.646.9 ± 0.288.8 ± 0.585.6 ± 0.2
2.068.8 ± 0.741.4 ± 0.986.5 ± 0.784.6 ± 0.4
Gaussian–73.0 ± 0.846.2 ± 0.487.9 ± 0.685.3 ± 0.1
5.072.9 ± 0.445.5 ± 0.388.0 ± 0.285.1 ± 0.2
2.070.9 ± 0.343.3 ± 0.186.5 ± 0.485.2 ± 0.3

3) Repeated releases and model updates. For kk releases of a sample's activation to the same recipient, each with independent noise and (ϵ,δ)(\epsilon,\delta)-PixelDP, sequential composition gives (kϵ,kδ)(k\epsilon,k\delta)-PixelDP. Reusing a received batch for τ′\tau' tail updates is post-processing with no extra cost. ASFL keeps FedAvg's model-update exchange (Algorithm 2, line 9) without additional transmissions, and these updates are outside the activation-level guarantee.

4) Privacy claim. We would like to clarify that our study focuses on the benefits of latent information sharing rather than a stronger privacy guarantee. Our position is that ASFL does not increase privacy leakage compared to other representation sharing methods such as SplitFed. Our feature inversion attack experiments (Table 2) show that PixelDP reduces the leakage of ASFL activations.

Table 2. Inversion attacks (Laplacian ϵ=0.2\epsilon=0.2, Gaussian (ϵ,δ)=(0.5,0.01)(\epsilon,\delta)=(0.5,0.01))

AttackNoisePSNRSSIMdCor
Learning-based (He et al., 2019)None29.840.95080.9419
Laplacian24.470.85600.9128
Gaussian24.670.86450.8884
DRAG (Lei et al., 2025)None29.040.94420.9181
Laplacian20.020.71810.9032
Gaussian19.950.73310.8706
Reviewer_iwxEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ASFL (Activation Sharing Federated Learning), a method that mitigates client drift in non-IID federated learning by sharing latent representations across clients. The model is split into an input-side sub-network y and an output-side sub-network z. At the beginning of each round, randomly paired clients exchange a small batch of y-activations and the corresponding labels in a peer-to-peer, gossip-style manner. Each client then runs a "tail round" that trains only z on a mix of local and remote activations, followed by standard local updates and FedAvg aggregation. The split layer is chosen by a heuristic that balances system cost against gradient variance, and the few received activations are reused through resampling to keep communication low. The authors provide a convergence analysis for non-convex smooth objectives under partial participation and incorporate PixelDP to privatize the shared activations.

Critical strengths
  1. The idea is simple and practical: ASFL plugs directly into FedAvg, needs no server-side computation, and adds little communication overhead (about 8 MB per round on CIFAR-10).

  2. ASFL achieves consistent accuracy gains on all four benchmarks while reducing per-round local computation time, and the ablation in Table 14 clearly attributes the gain to the remote activations rather than to partial training alone.

  3. The paper offers a reasonably complete package, combining a convergence analysis for non-convex objectives under partial participation, a practical cost–variance criterion for choosing the split layer, and extensive ablations on the split layer, sharing quantity, pool construction, and degree of heterogeneity.

Critical weaknesses
  1. ASFL requires paired clients to directly exchange intermediate activations together with their ground-truth labels, and the resulting privacy risks are not adequately assessed. The labels are always sent in plaintext. Under the strong label skew used in the paper (Dirichlet α = 0.1), even one mini-batch largely reveals a client's class distribution. The main results (Table 1) also rely on unperturbed activations, and the receiver holds the same global sub-network y, which favors feature inversion. Yet the inversion experiment in Appendix A.4.2 only targets noise-perturbed activations, and the claimed leakage comparison with SplitFed is not reported. Finally, PixelDP only guarantees indistinguishability within a small input radius, which is not record-level DP on the local data.

  2. The paper claims that ASFL outperforms state-of-the-art FL methods, but the most recent baseline is FedImpro (ICLR 2024), and the remaining baselines (apart from FedAvg) were published between 2020 and 2023. For an ICLR 2027 submission, this weakens the empirical contribution. In particular, SILVER (ICML 2024) and FedMDVR (IEEE TNNLS 2025) are cited in the paper but not included in the experiments. ASFL's main claimed advantage over variance-reduction methods is its lower overhead, and SILVER is itself designed as a low-overhead variance-reduction method, so this comparison is necessary to support the claim.

Questions for authors

See weakness

3 comments

Thank you for the insightful comments. We hope our responses address your concerns and that you will reconsider your score.

W1: The labels are always sent in plaintext. ... Yet the inversion experiment in Appendix A.4.2 only targets noise-perturbed activations, and the claimed leakage comparison with SplitFed is not reported. Finally, PixelDP only guarantees indistinguishability within a small input radius, which is not record-level DP on the local data.

Label DP. We agree that labels are exposed in representation-sharing approaches such as ASFL and SplitFed, and labels in both can be protected with existing label DP techniques. To verify that ASFL remains trainable under label DP, we applied KK-ary randomized response (RR), which keeps each label with probability eϵℓeϵℓ+K−1\frac{e^{\epsilon_\ell}}{e^{\epsilon_\ell}+K-1} and otherwise replaces it with one of the other K−1K-1 classes uniformly at random. ASFL maintains competitive accuracy under label DP.

Mechanismϵ\epsilonδ\deltaϵℓ\epsilon_\ellMethodCIFAR-10CIFAR-100SVHNFEMNIST
PixelDP (Laplacian)2.0--SplitFed67.8 ± 1.546.1 ± 0.586.8 ± 0.485.2 ± 0.2
ASFL74.7 ± 1.049.1 ± 0.688.5 ± 0.385.8 ± 0.2
PixelDP (Laplacian) + Label DP2.0-5.0SplitFed69.9 ± 0.945.5 ± 0.486.0 ± 0.584.7 ± 0.3
ASFL74.4 ± 0.646.9 ± 0.288.8 ± 0.585.6 ± 0.2
PixelDP (Laplacian) + Label DP2.0-2.0SplitFed50.3 ± 1.05.4 ± 0.138.4 ± 5.859.3 ± 1.3
ASFL68.8 ± 0.741.4 ± 0.986.5 ± 0.784.6 ± 0.4
PixelDP (Gaussian)0.55e-2-SplitFed59.7 ± 0.834.3 ± 0.284.8 ± 1.475.9 ± 0.5
ASFL73.0 ± 0.846.2 ± 0.487.9 ± 0.685.3 ± 0.1
PixelDP (Gaussian) + Label DP0.55e-25.0SplitFed61.5 ± 0.733.0 ± 0.583.3 ± 1.073.0 ± 0.3
ASFL72.9 ± 0.445.5 ± 0.388.0 ± 0.285.1 ± 0.2
PixelDP (Gaussian) + Label DP0.55e-22.0SplitFed44.9 ± 1.65.3 ± 0.219.8 ± 0.337.5 ± 3.1
ASFL70.9 ± 0.343.3 ± 0.186.5 ± 0.485.2 ± 0.3

Comparison with SplitFed. Thank you for pointing out the missing comparison. We added this to the Appendix A.4. Even without DP, ASFL shows information leakage comparable to SplitFed under both evaluated inversion attacks.

AttackMethod (without DP)PSNRSSIMdCor
Learning-based Inversion Attack (He et al., 2019)SplitFed28.790.93990.9149
ASFL29.840.95080.9419
DRAG (Lei et al., 2025)SplitFed29.210.94930.9120
ASFL29.040.94420.9181

Record-level DP. We agree that PixelDP provides indistinguishability within an input radius, not record-level DP on the local data. We therefore additionally evaluated ASFL combined with gradient perturbation (added to Appendix A.4). ASFL retains higher accuracy than SplitFed with similar MIA resistance.

ϵ\epsilonδ\deltaMethodValid. Acc. (%)AUCTPR@1% FPR (%)
200.1SplitFed70.0 ± 0.10.51 ± 0.0021.2 ± 0.3
ASFL72.8 ± 0.50.52 ± 0.0001.1 ± 0.2
200.01SplitFed68.0 ± 1.00.51 ± 0.0011.0 ± 0.2
ASFL72.1 ± 0.90.52 ± 0.0011.2 ± 0.1
100.1SplitFed65.4 ± 0.90.51 ± 0.0031.2 ± 0.2
ASFL70.0 ± 0.80.51 ± 0.0011.2 ± 0.1
100.01SplitFed62.7 ± 0.70.51 ± 0.0030.9 ± 0.1
ASFL65.7 ± 0.50.51 ± 0.0031.0 ± 0.1

W2: The paper claims that ASFL outperforms state-of-the-art FL methods, but the most recent baseline is FedImpro (ICLR 2024), and the remaining baselines (apart from FedAvg) were published between 2020 and 2023.

We thank the reviewer for pointing us to these recent works. We reproduced FedMDVR (TNNLS 2025) and SILVER (ICML 2024) under the same settings as our other baselines.

MethodMetricCIFAR-10CIFAR-100SVHNFEMNIST
FedMDVRAcc. (%)71.7±2.071.7 \pm 2.046.8±0.146.8 \pm 0.186.6±1.586.6 \pm 1.584.4±0.484.4 \pm 0.4
Comm. (MB/round)63.9513.801.562.31
ASFLAcc. (%)75.3±0.9\mathbf{75.3 \pm 0.9}50.6±0.3\mathbf{50.6 \pm 0.3}89.1±0.3\mathbf{89.1 \pm 0.3}84.9±0.3\mathbf{84.9 \pm 0.3}
Comm. (MB/round)50.6313.202.041.78

ASFL achieves higher accuracy than FedMDVR on CIFAR-10, CIFAR-100, and SVHN, comparable accuracy on FEMNIST, and lower per-round communication on three of the four datasets. This suggests that sharing latent representations can mitigate client drift more effectively than the control-variate-based variance reduction in FedMDVR.

For SILVER, our reproduction reached only 12∼1312 \sim 13% accuracy on CIFAR-100, close to random guessing. Since the authors provide neither code nor detailed hyperparameters, we could not exactly reproduce their reported results, and thus we do not report it as a fair comparison. The FedMDVR results are added to Table 1, with reproduction details in the Appendix A.5.

I thank the authors for their response. While the proposed method does raise some privacy concerns, extensive experiments with techniques for protecting label privacy have addressed my concerns to some degree. The inability to reproduce the results of SILVER does not undermine the paper’s core contribution and mathod. Overall, I maintain my original rating.

We appreciate that your acknowledgment that some of your concerns have been addressed through our rebuttal. Given that our response has helped clarify and alleviate these concerns, we would kindly ask you to reconsider whether your overall assessment could be adjusted accordingly. We believe that the clarifications and additional experimental evidence provided in our response strengthen the contribution of our work. Furthermore, we believe that our work offers practical and useful insights to the distributed learning community, while opening up promising new research directions for exploiting remote latent information in federated learning.

Reviewer_YCtdOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Activation Sharing Federated Learning (ASFL) to address client drift caused by differences in clients' local data distributions. The method combines activation sharing with standard model averaging and uses a cost–variance-based approach to select a suitable layer for sharing activations. The authors also explore privacy by adding PixelDP noise to the shared activations and studying the trade-off between privacy and accuracy. Across standard benchmarks, it delivers higher accuracy under a fixed communication budget and requires less local computation than the baselines.

Critical strengths

1.The paper takes a simple and interesting approach to reducing client drift through activation sharing. The motivation is clear, and the method is easy to follow.

2.The cost–variance-based layer selection and the reuse of a small set of shared activations are a practical way to balance the benefits of sharing against computation and communication costs.

3.The paper offers both theoretical analysis and experimental evaluation. The results are promising, and the comparisons with several existing methods, along with the ablation studies, give useful evidence for the effectiveness of the proposed approach.

Critical weaknesses

1.The method chooses the intersection of the normalized cost and gradient-variance curves. Figure 4 shows a reasonable trade-off between communication cost and accuracy, but it is not clear why this intersection should be the best choice, or how it compares with simpler layer-selection strategies.

2.The privacy guarantees are not entirely clear. The authors discuss differential privacy, but they don’t explain how neighbouring data are defined, or how privacy loss accumulates when activations are shared repeatedly across communication rounds.

Questions for authors

1.ASFL first updates the tail subnetwork using local and remote activations, then trains the full model on local data. Could the authors explain whether these later local updates might reduce the benefits gained from the remote activations?

2.Algorithm 1 describes mutual activation exchange between randomly paired clients, while Appendix A.3.1 describes directed gossip-style communication. Could the authors clarify whether the exchange is mutual or whether each client independently selects another client from which to receive activations?

3.Since FedImpro also uses shared feature-distribution information to address client drift, could the authors further clarify the key methodological contributions of ASFL in comparison with FedImpro?

2 comments

We appreciate the careful review. We hope our responses address your concerns, and we would be grateful if you could reconsider your evaluation.

Q1: Could the authors explain whether these later local updates might reduce the benefits gained from the remote activations?

We agree that the subsequent full-model updates partially offset the benefit of the tail updates by moving zz back toward the local distribution. This is a deliberate design. Remote activations give gradients only to the output-side sub-model zz, so the input-side sub-model yy must still be trained on local data. Since every round begins with tail updates, the full-model steps start from a less biased zz, and their drift is corrected in the next round. Indeed, ASFL with τ\tau tail and τ\tau full-model steps outperforms FedAvg with 2τ2\tau full-model steps (Tab. 1), while each tail update is cheaper since it trains zz only.

Q2: Could the authors clarify whether the exchange is mutual or whether each client independently selects another client from which to receive activations?

Thank you for pointing out this inconsistency. Appendix A.3.1 is correct. Each active client independently selects another client at random and receives its activations. The links are thus directed, and a client that sends activations to another client does not necessarily receive activations from it. Alg. 1 and Fig. 1 incorrectly show mutual exchange. We revised them accordingly.

Q3: Could the authors further clarify the key methodological contributions of ASFL in comparison with FedImpro?

[Real versus approximated features] First, ASFL trains on real activations from other clients, while FedImpro trains the output-side layers on features sampled from class-conditional Gaussians. This approximation ignores correlations across feature dimensions, and its bias grows as the feature distributions become more complex. ASFL avoids this bias by using actual activations, as shown in the following additional results.

Dirichlet α\alpha0.050.10.5
FedImpro50.9±0.750.9 \pm 0.7 %62.8±0.462.8 \pm 0.4 %83.1±0.583.1 \pm 0.5 %
ASFL67.1±0.567.1 \pm 0.5 %75.3±0.975.3 \pm 0.9 %87.6±0.187.6 \pm 0.1 %

The gap widens as heterogeneity increases (smaller α\alpha), since each client then relies more on features from other clients, which amplifies the approximation bias.

[Theoretical guarantees] Second, ASFL's convergence guarantee does not assume that the shared information accurately approximates other clients' feature distributions. FedImpro shows that sampling from a shared feature distribution reduces the gradient dissimilarity of the high-level layers (Theorem 4.2 in Tang et al., 2024), but assumes that the sampled features faithfully represent the true distributions, ignoring the Gaussian estimation error. Since ASFL uses actual activations, our Theorem 1 requires no such assumption.

W1: It is not clear why this intersection should be the best choice, or how it compares with simpler layer-selection strategies.

The two metrics capture opposing aspects of the layer choice. The cumulative variance cost V(l)V(l) (Eq. 6) increases monotonically, while the system cost C(l)C(l) (Eq. 5) decreases monotonically. Since no layer dominates the others, a balancing criterion is needed. After normalizing both to [0,1][0, 1], we select l⋆=arg⁡min⁡lmax⁡{Vˉ(l),Cˉ(l)}l^\star = \arg\min_l \max\{\bar{V}(l), \bar{C}(l)\}, the layer that minimizes the larger of the two costs. By monotonicity, l⋆l^\star is the layer closest to their intersection, which is the most balanced trade-off point rather than an ad hoc choice. Fig. 4 shows the accuracy for all layer choices, which covers simpler strategies such as fixed-layer selection and single-metric selection (the two endpoints). Our selected layer achieves near-best accuracy.

W2: The authors discuss differential privacy, but they don’t explain how neighbouring data are defined, or how privacy loss accumulates when activations are shared repeatedly across communication rounds.

Thank you for pointing this out. We added the following to Appendix A.4.

Definition of neighboring data. Similar to metric DP (Chatzikokolakis et al., 2013), PixelDP (Lecuyer et al., 2019) defines neighbors by the pp-norm distance between inputs: ξ\xi and ξ′\xi' are neighbors if ξ′=ξ+η\xi' = \xi + \eta for some η∈Bp(R)=\eta \in B_p(R) = {η∈Rn:∥∥η∥∥p≤R\eta \in \mathbb{R}^n : \|\| \eta \|\|_p \le R}.

Accumulation of privacy loss. Privacy loss accumulates when activations of the same sample are repeatedly released with independent noise. For an honest-but-curious recipient, let kk be the number of noisy activations of the same sample it receives during training. If each release satisfies (ϵ,δ)(\epsilon, \delta)-PixelDP, sequential composition gives (kϵ,kδ)(k\epsilon, k\delta)-PixelDP overall. Reusing a received batch for τ′\tau' tail updates within a round is post-processing, so it incurs no additional privacy cost and is not counted in kk.

Thankyou for the response, I will maintain my current positive assessment.

Comments on the paper

1 comment

Dear all,

We're the half-way through the discussion phase. The authors have provided their response to your comments. Some of you have started involving discussion, but all of us can take action now.

If the provided responses do not address (even part of) your concerns, this is your chance to raise it. Please engage with the authors as soon as possible, we need to keep the conversation going!

Thanks,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.