acceptodds
Withdrawn from ICLR 2027

zkFU: Verifiable Federated Unlearning via Trace-Consistent Subtraction and Zero-Knowledge Proofs

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Federated unlearning needs both effective client removal and verifiable evidence of the removal computation. Existing methods largely optimize forgetting and utility, while direct cryptographic replay scales with the training history. We introduce zkFU, which combines compact zero-knowledge certification with retained-aware recovery. Its key idea is to use linear aggregation to turn committed client updates into a compact deletion relation. Trace-consistent gradient subtraction (TCGS) computes a retained-normalized seed for weighted client requests. Homomorphic commitment aggregation then reduces certification to an O(d) affine statement after public aggregation, compared with O(NTd) direct replay over N clients, T rounds, and d parameters. TraceTrust recovers utility through sparse trust filtering, scale calibration, and forgetting-direction-shielded repair. We prove certificate soundness, zero knowledge, and completeness, and bound the seed’s mismatch with retained-only retraining. The construction supports cumulative requests and public-coefficient linear aggregation rules, including FedAvg and its common variants. Across three datasets and 11 baselines, zkFU alone ranks in the top two for both retained accuracy and KL divergence to retraining. Twenty-seed tests establish equivalence to retraining within ±1 percentage point in mean membership- inference gap under a shadow-model attack on Fashion-MNIST and CIFAR-10. The Halo2 prototype verifies an 11M-parameter ResNet-18 certificate in 29 s. A matched Fashion-MNIST comparison projects about 930× lower proving time than the implemented aggregation-only replay baseline.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Federated Unlearning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_47UoOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

In my reading, the paper proposes zkFU, a client-level federated unlearning (FU) framework that pairs a cryptographic certificate with a heuristic recovery pipeline. Under FedAvg with fixed public weights {ai}i=1N\lbrace a_i\rbrace_{i=1}^{N}, the final model satisfies the trace identity θT=θ0+∑t=1T∑i=1NaiΔit\theta_T=\theta_0+\sum_{t=1}^{T}\sum_{i=1}^{N}a_i\Delta_i^t. For a forget set F\mathcal{F} with mass aF=∑i∈Faia_{\mathcal F}=\sum_{i\in\mathcal F}a_i and trace contribution WF=∑t∑i∈FaiΔitW_{\mathcal F}=\sum_{t}\sum_{i\in\mathcal F}a_i\Delta_i^t, the authors' Trace-Consistent Gradient Subtraction (TCGS) defines the seed

θtcgs=θT+aF(θT−θ0)−WF1−aF.\theta_{\mathrm{tcgs}}=\theta_T+\frac{a_{\mathcal F}(\theta_T-\theta_0)-W_{\mathcal F}}{1-a_{\mathcal F}}.

During training, clients post vector Pedersen commitments cit=Com(Δit;rit)c_i^t=\mathrm{Com}(\Delta_i^t;r_i^t) to an append-only board. Because the commitments are additively homomorphic, any verifier can recompute cFagg=∑t∑i∈Faicitc_{\mathcal F}^{\mathrm{agg}}=\sum_{t}\sum_{i\in\mathcal F}a_i c_i^t from public data. A Halo2 circuit then proves an O(d)O(d) affine relation (constraints C1–C4) that binds θtcgs\theta_{\mathrm{tcgs}} to the trace, independently of NN and TT; the authors contrast this with an O(NTd)O(NTd) aggregation-replay baseline. Theorem 1 bounds ∥θtcgs−θ∖F⋆∥2=O(LΔηGT2aF)\lVert\theta_{\mathrm{tcgs}}-\theta_{\setminus\mathcal F}^{\star}\rVert_2=O(L_\Delta\eta GT^2a_{\mathcal F}) in a "stable" regime LΔT=O(1)L_\Delta T=O(1), and Theorem 2 establishes completeness, soundness and zero knowledge.

Because the certified seed is, in my view, a poor model on its own (for example, 38.21% retained accuracy on FMNIST and 11.07% on CIFAR-100), the model that is actually deployed, θout\theta_{\mathrm{out}}, comes from an uncertified three-stage pipeline called TraceTrust: (i) retained-conflict projection, Fisher trust damping and top-ρ\rho Fisher-scaled masking; (ii) a grid search over the interpolation scale β\beta; and (iii) one epoch of KD-anchored repair whose gradient is "shielded" along dF=θtcgs−θTd_F=\theta_{\mathrm{tcgs}}-\theta_T.

The experiments cover FMNIST with a small CNN and CIFAR-10/100 with ResNet-18, N=20N=20 clients, Dirichlet α=0.5\alpha=0.5, single-client deletion and 11 baselines. The authors report top-two retained accuracy and KL, MIA-gap equivalence to retraining within ±1\pm1 pp over 20 seeds, and Halo2 certificates for 11M parameters (8.3 h to prove, 29 s to verify).

Critical strengths

Strength 1. In my opinion, the central observation is clean and correct: linear aggregation turns client removal into an affine relation over homomorphic commitments. I find the paper's best idea to be its use of the linearity of the server rule, θT−θ0=∑t∑iaiΔit\theta_T-\theta_0=\sum_{t}\sum_{i}a_i\Delta_i^t, together with the additive homomorphism of Pedersen commitments:

∑t∑i∈FaiCom(Δit;rit)=Com(WF;rFagg).\sum_{t}\sum_{i\in\mathcal F}a_i\mathrm{Com}(\Delta_i^t;r_i^t)=\mathrm{Com}(W_{\mathcal F};r_{\mathcal F}^{\mathrm{agg}}).

Together, these collapse the statement "the server subtracted exactly this client's historical contribution" from an O(NTd)O(NTd) replay to an O(d)O(d) check after public aggregation. I also appreciate that the construction extends neatly beyond FedAvg. Appendix H.1 shows that FedProx, FedAvgM (with ωit=ai(1−mT−t+1)\omega_i^t=a_i(1-m^{T-t+1})) and FedNova all fit the same template once aFa_{\mathcal F} is replaced by a~F=1−1/ν\tilde a_{\mathcal F}=1-1/\nu. The same template also covers round-wise partial participation (Eq. 16), selected-round deletion, and cumulative requests F(k)=⋃j≤kFj\mathcal F(k)=\bigcup_{j\le k}\mathcal F_j at a cost that does not grow with kk. As far as I can verify, the soundness argument of Appendix E.1 is short and correct: C1 plus binding forces WF⋆=WFW_{\mathcal F}^{\star}=W_{\mathcal F}, and C2–C3 then fix the seed.

Strength 2. I would like to commend the authors for an unusually candid and thorough supplementary analysis. From my perspective, the authors do not hide the limitations of their own theory. Appendix F.3 measures the expansion ratio κt\kappa_t of the retained update map along real trajectories and openly reports that the worst-case factor eLΔTe^{L_\Delta T} would exceed e30e^{30} and e85e^{85}. Appendix E.2 states explicitly what zero knowledge does not hide, and Appendix J concedes that a Schnorr-style opening proof could replace the Halo2 backend. The release also includes an implemented replay baseline on the same Halo2 stack (Table 24), a quantisation-error analysis (F.4), and sequential, selected-round and partial-participation experiments. This level of openness made a careful assessment possible, and I think it deserves credit.

Critical weaknesses

Weakness 1. In my view, the certificate verifies an object that is never deployed. The deployed model carries neither a cryptographic nor an approximation guarantee, and the ZK component proves a statement that the verifier could check without it.

The paper motivates itself (Introduction, ll. 44–47) with the observation of Zhang et al. (2024) that behavioural checks "can be gamed by adaptive servers" that do not perform the intended computation. Yet, as far as I can tell, the deployed model θout\theta_{\mathrm{out}} is not bound by πtcgs\pi_{\mathrm{tcgs}} in any way. The released tuple (θout,θtcgs,πtcgs,ctcgs,cFagg)(\theta_{\mathrm{out}},\theta_{\mathrm{tcgs}},\pi_{\mathrm{tcgs}},c_{\mathrm{tcgs}},c_{\mathcal F}^{\mathrm{agg}}) is equally valid if the server simply sets θout=θT\theta_{\mathrm{out}}=\theta_T, or fine-tunes θT\theta_T for one epoch. I therefore believe the certificate does not rule out exactly the adaptive server the paper sets out to defeat.

I do not think this is a harmless gap between a "seed" and a "polished" model. The certified seed is, in my assessment, unusable as a model. Its retained accuracy is 38.21 ± 15.73% on FMNIST (with a KL of 10.59 to retraining), 23.84% on CIFAR-10 and 11.07% on CIFAR-100 (Table 3a and the Direct Accumulation rows of Table 1). The deployed θout\theta_{\mathrm{out}}, by contrast, reaches 92.52%, 93.27% and 93.48%. All of the reported utility, and all of the reported forgetting, therefore comes from the uncertified TraceTrust stages. From my reading of the method and the code, those stages are built in a way that works against forgetting:

  • The interpolation search can retreat towards the original model. TCI searches θ(β)=θT+β(θsparse−θT)\theta(\beta)=\theta_T+\beta(\theta_{\mathrm{sparse}}-\theta_T) with β∈[0.25,1]\beta\in[0.25,1], so the deployed model may sit three quarters of the way back towards θT\theta_T.
  • Both TCI and the repair loss distil from the teacher θT\theta_T (Eqs. 10–11; teacher_state=self.trace.final_model_state in zkfu.py). That is the very model that contains the forgotten client. To my knowledge, distillation from such a teacher on retained data is a well-known route by which memorised information about F\mathcal F leaks back into the student.
  • The shield constrains a single direction. It removes only the negative component of grepairg_{\mathrm{repair}} along one vector dF∈Rdd_F\in\mathbb R^{d} with d≈1.1×107d\approx1.1\times10^{7}. This is a rank-one constraint, and I see nothing that stops the repair from restoring forgotten behaviour through the remaining d−1d-1 orthogonal directions.

Is it any surprise, then, that zkFU's forgetting gap on CIFAR-100 is 20.75 pp, compared with 2.78 for SIFU, 4.17 for FedAU and 5.33 for uS2U?

I also think the cryptographic contribution is considerably thinner than the headline suggests. Since θ0\theta_0, θT\theta_T, θtcgs\theta_{\mathrm{tcgs}} and aFa_{\mathcal F} are all public, the witness is itself publicly computable, as the authors acknowledge in Appendix E.2:

WF=aF(θT−θ0)−(1−aF)(θtcgs−θT).W_{\mathcal F}=a_{\mathcal F}(\theta_T-\theta_0)-(1-a_{\mathcal F})(\theta_{\mathrm{tcgs}}-\theta_T).

A verifier can therefore form Y=cFagg−∑jW~F,jGjY=c_{\mathcal F}^{\mathrm{agg}}-\sum_{j}\widetilde W_{\mathcal F,j}\mathsf G_j and check Y=rFaggHY=r_{\mathcal F}^{\mathrm{agg}}\mathsf H directly. In my understanding, the server could even publish rFaggr_{\mathcal F}^{\mathrm{agg}} outright, because opening a commitment to a value that is already public reveals nothing new. Constraint C4, a commitment to the already public θtcgs\theta_{\mathrm{tcgs}}, seems equally redundant. The zero-knowledge property thus hides nothing that is not already public, and the 8.3-hour Halo2 proof for ResNet-18 certifies something a verifier could check with a single dd-term multi-scalar multiplication in seconds. I would argue that the headline "930x lower proving time" is measured against an aggregation-replay strawman rather than against this trivial direct check. Appendix J's own remark ("they do not establish superiority over this specialized opening proof") all but concedes the point.

Finally, I am concerned about an uncomfortable privacy consequence. A client who exercises the right to erasure (GDPR Art. 17) has its cumulative weighted update WFW_{\mathcal F} released to the public as a by-product of the certificate. That is precisely the artefact that gradient-inversion and property-inference attacks target.

Weakness 2. I believe the empirical evidence for "retraining fidelity" is undermined by transductive use of the test set, a mis-specified notion of "better", and a membership attack with essentially no statistical power.

(a) Test-set leakage. When I inspected the released code, I found that the "public" loader used by both TCI and FDS-ADR is the test set itself. runner.py, l. 772, sets non_member_loader = test_dl; this loader is passed as public_loader to run_trust_calibrated_interpolation and run_anchored_distillation_repair, where run_central_prox_kd_repair adds a KD term on public_x drawn from it. The same set therefore serves three roles at once:

  1. the KD inputs that pull θout\theta_{\mathrm{out}} towards θT\theta_T (the KD term also enters the objective J(β)J(\beta) that selects β⋆\beta^\star);
  2. the test-accuracy (TA) evaluation; and
  3. the non-member set of every reported MIA (mia_non_member_protocol="test_holdout").

In my opinion, fitting the deployed model's output distribution on exactly the non-member samples that the shadow-model attack later scores directly contaminates both the MIA Gap and the TOST equivalence claims. I could not find this disclosed anywhere in the paper; Section 6.3 and Appendix G.1 speak only of "retained or public validation signals". Did the baselines receive the same access? If not, I do not think Table 1 is a like-for-like comparison.

(b) "Higher is better" is the wrong objective when retraining is the reference. Table 1 declares Federated Retrain the reference, yet ranks methods by retained accuracy with an upward arrow. zkFU exceeds retraining by roughly 13 pp on CIFAR-10 (93.27 vs. 80.20) and roughly 10 pp on CIFAR-100 (93.48 vs. 83.31). In Table 15, zkFU's test accuracy (85.89) even exceeds that of the original, un-unlearned model θT\theta_T (78.63), and in Table 12 it exceeds retraining by about 10 pp (86.52 vs. 76.42). From my point of view, a model that is 7–13 pp more accurate than both the retrained and the original model is, by definition, not behaving like the retrained model. I suspect this reflects an under-trained retrain oracle (T=100T=100; Table 10 even reports a retrain retained accuracy of 79.39 ± 10.86 on CIFAR-10) combined with extra compute and test-set KD in the repair stage. I therefore think the claim that zkFU is "the only method ranked first or second in both retained accuracy and KL" rests on one metric that rewards departure from the reference and on a KL of 0.277–0.442 that is far from negligible.

(c) The membership-inference evidence has essentially no power. The per-example LiRA in Table 14 yields an AUC between 0.493 and 0.508 for all models, including the original θT\theta_T; in other words, the attack cannot even distinguish "no unlearning" from retraining. The shadow-model attack separates the original model from retraining by under 1 pp on FMNIST (1.79 vs. 0.88) and under 2 pp on CIFAR-10 (2.50 vs. 0.61). I find it hard to accept a TOST equivalence margin of ±1\pm1 pp as meaningful when the entire signal is only about 1–2 pp wide. Based on my familiarity with the FU literature, the standard protocol here would be backdoor or canary injection into the forgotten client, with the attack-success rate reported before and after unlearning. That protocol is absent, as are client-level (rather than sample-level) membership attacks.

(d) The theory does not cover the regime that is evaluated. Theorem 1 requires LΔT=O(1)L_\Delta T=O(1). Yet with E=5E=5 local epochs, momentum 0.9 and T=100T=100, the authors' own measured per-client LΔL_\Delta lies between 0.2 and 2.0 (Fig. 4c), which gives LΔT≈20L_\Delta T\approx20–200200. The general bound ηGLΔT(T+1)aFeLΔT\eta GL_\Delta T(T+1)a_{\mathcal F}e^{L_\Delta T} is then vacuous, and with η=O(1/(LT))\eta=O(1/(LT)) it is of the same order as the trivial bound ∥θtcgs−θ∖F⋆∥2≤2ηGT\lVert\theta_{\mathrm{tcgs}}-\theta_{\setminus\mathcal F}^{\star}\rVert_2\le2\eta GT. More fundamentally, I see no guarantee of any kind, cryptographic or approximation-theoretic, for θout\theta_{\mathrm{out}}, which is the only model an end user would ever interact with.

Questions for authors
  1. Binding the deployed model. Under your threat model, what prevents a server from publishing a valid πtcgs\pi_{\mathrm{tcgs}} while deploying θout=θT\theta_{\mathrm{out}}=\theta_T or any fine-tuned variant of it? If the answer is "nothing", I would like to understand in what sense zkFU's guarantee for the deployed model is stronger than that of the behavioural verification (VeriFi/uS2U) the paper criticises. I think it would considerably strengthen the work to bound ∥θout−θtcgs∥2\lVert\theta_{\mathrm{out}}-\theta_{\mathrm{tcgs}}\rVert_2, or KL(θout∥θtcgs)\mathrm{KL}(\theta_{\mathrm{out}}\Vert\theta_{\mathrm{tcgs}}), inside the certificate, for example by also certifying the sparse mask mm, the selected β⋆\beta^\star and a bounded number of repair steps. Is this feasible?

  2. Necessity of the SNARK. Since WFW_{\mathcal F} is a deterministic function of public quantities (Appendix E.2), what, in your view, does the Halo2 proof guarantee beyond the direct check cFagg−∑jW~F,jGj=?rFaggHc_{\mathcal F}^{\mathrm{agg}}-\sum_{j}\widetilde W_{\mathcal F,j}\mathsf G_j\overset{?}{=}r_{\mathcal F}^{\mathrm{agg}}\mathsf H, which needs no zero knowledge at all? I would encourage you to report the wall-clock cost of this direct check (or of the Schnorr variant in Appendix J) for ResNet-18 alongside the 8.3 h Halo2 proof, and to restate the 930x claim against it.

  3. Test-set usage. Could you confirm that public_loader in TCI and FDS-ADR is the test set (runner.py, l. 772), and that the same set serves as the MIA non-member set? I believe Table 1, Table 3(b) and Table 14 should be re-run with a disjoint public split, or with no public KD at all. Please also state whether any baseline had comparable access to test inputs.

  4. Distilling from θT\theta_T. Why do you distil from the teacher θT\theta_T, which by construction encodes F\mathcal F, rather than from θtcgs\theta_{\mathrm{tcgs}} or θcal\theta_{\mathrm{cal}}? I would like to see an ablation of the teacher choice that reports Δf\Delta_f together with a backdoor attack-success rate on the forgotten client.

  5. Accuracy above retraining. How do you reconcile retained and test accuracy exceeding both the retrained and the original model (e.g., 85.89 vs. 77.71 vs. 78.63 in Table 15) with the claim of fidelity to retraining? Would the conclusions change with a converged retrain oracle, or with one given the same extra repair compute (one KD epoch)?

  6. Stronger forgetting evaluation. Could you add a backdoor or canary-based forgetting test and a client-level MIA? In my reading, the LiRA AUCs in Table 14 suggest that the current attacks do not separate even θT\theta_T from retraining.

  7. Federated LLMs and LoRA. From my point of view, the most pressing use case for client erasure today is the federated fine-tuning of large language models, typically through parameter-efficient adapters such as LoRA, where W=W0+BAW=W_0+BA with B∈Rm×rB\in\mathbb R^{m\times r}, A∈Rr×nA\in\mathbb R^{r\times n} and r≪min⁡(m,n)r\ll\min(m,n). I have three related questions: (a) When FedAvg aggregates AA and BB separately (as in FedIT), the trace identity is linear in the submitted (A,B)(A,B), but the effective update is bilinear: ∑iaiBiAi≠(∑iaiBi)(∑iaiAi)\sum_{i}a_iB_iA_i\neq\big(\sum_{i}a_iB_i\big)\big(\sum_{i}a_iA_i\big). Does TCGS remain meaningful in this setting, and how would the bilinearity enter the trajectory-mismatch bound of Theorem 1? (b) If I extrapolate Table 20 linearly (about 2.76 ms per coordinate), a certificate for a full 7B-parameter model would take on the order of 1.9×1071.9\times10^{7} s, i.e., roughly seven months. A rank-16 LoRA adapter on the query and value projections of a 7B model (about 8×1068\times10^{6} parameters) would cost roughly as much as your ResNet-18 certificate. How do you see zkFU scaling to either setting? (c) The retained O(NTd)O(NTd) openings (about 4.5 GB per ResNet-18 client) seem prohibitive for cross-device federated LLM fine-tuning. Is the running-aggregate trick of Appendix B compatible with secure aggregation, under which the server never sees the individual Δit\Delta_i^t?

  8. Privacy of the departing client. Releasing WFW_{\mathcal F} publicly discloses the forgotten client's cumulative update at precisely the moment that client asks to be forgotten. Have you evaluated gradient-inversion or property-inference attacks on WFW_{\mathcal F}? Would it be possible to release only a commitment to θtcgs\theta_{\mathrm{tcgs}}, which would make C4 meaningful, and keep the seed itself private?

  9. Theory–experiment gap. Given the measured LΔTL_\Delta T of roughly 20 to 200, under what practical conditions do you expect Assumption 1 and the stable regime to hold? Could the bound instead be stated in terms of the measured aggregate expansion κt\kappa_t rather than the per-client LΔL_\Delta?

  10. Hyperparameters. TraceTrust introduces at least ten hyperparameters (ρ\rho, λF\lambda_F, B\mathcal B, λkd\lambda_{\mathrm{kd}}, λpull\lambda_{\mathrm{pull}}, λprox\lambda_{\mathrm{prox}}, λsh\lambda_{\mathrm{sh}}, τkd\tau_{\mathrm{kd}}, ηr\eta_r, εg\varepsilon_{g}, εF\varepsilon_{F}, εd\varepsilon_{d}). On which data were they tuned, and were the eleven baselines tuned with a comparable budget?

Reviewer_g2JWOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes zkFU for client-level federated unlearning verification.

Critical strengths

The paper states what the certificate does and does not cover, including that that W_F is revealed, that the deployed model is uncertified.

Critical weaknesses
  1. The paper in general is hard to follow to understand the main idea and technical analysis. For example, by reading "Theorem 1 controls its parameter distance to retraining, yet even a modest parameter error can affect accuracy on retained-sensitive coordinates", I cannot understand the connection between stage 2 and Theorem 1 on trajectory mismatch.

  2. The proposed method verifys that the server can perform subtraction, but this does not mean the unlearning itself is guaranteed, which therefore means the verified model by ZK does not necessarly is the model that is deployed.

  3. The experiment results seem to be contradicting. For example in Table 1, if Federated Retrain is 80.20 for acc compared to the method's 93.27, doesn't that mean the model does not have unlearning effect?

Questions for authors

What prevents a server from certifying θtcgsθ_tcgs while serving an unrelated model?

Reviewer_XcnZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes zkFU, a federated unlearning framework that combines trace-consistent gradient subtraction (TCGS), zero-knowledge proofs, and a recovery method called TraceTrust. The goal is to verify that a client's historical contributions were correctly subtracted while preserving model utility.

Critical strengths
  • Addresses an important problem: providing cryptographic evidence of federated unlearning rather than relying only on behavioral evaluations. -The compact proof construction is interesting, and the authors provide an actual Halo2 implementation. -Extensive experiments across three datasets, multiple baselines, and additional ablations.
Critical weaknesses
  • The proof only certifies the TCGS seed, not the final model after TraceTrust. Since recovery is essential for utility, it is unclear how the certificate guarantees that the deployed model has actually undergone the intended unlearning.
  • Generating a certificate for ResNet-18 takes approximately 8.3 hours. The reported 930× improvement is based partly on extrapolated replay costs, and the alternative Schnorr construction (Appendix J) is not evaluated.
  • The LiRA attack produces near-random results even on the original model. Also, forgotten-client accuracy differs from retraining by ~10 and 20 percentage points on CIFAR-10 and CIFAR-100, respectively.
  • TraceTrust is evaluated using simulated retained-client computation. Communication overhead and privacy costs are not included in the reported unlearning time.
Questions for authors

-Can the authors clarify what security guarantee the certificate provides for the final deployed model rather than just the intermediate seed?

  • Why use Halo2 rather than a specialized Schnorr proof for this affine relation?
  • Can the authors provide a stronger forgetting evaluation that clearly distinguishes the original model from retraining?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.