- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
In my reading, the paper proposes zkFU, a client-level federated unlearning (FU) framework that pairs a cryptographic certificate with a heuristic recovery pipeline. Under FedAvg with fixed public weights , the final model satisfies the trace identity . For a forget set with mass and trace contribution , the authors' Trace-Consistent Gradient Subtraction (TCGS) defines the seed
During training, clients post vector Pedersen commitments to an append-only board. Because the commitments are additively homomorphic, any verifier can recompute from public data. A Halo2 circuit then proves an affine relation (constraints C1–C4) that binds to the trace, independently of and ; the authors contrast this with an aggregation-replay baseline. Theorem 1 bounds in a "stable" regime , and Theorem 2 establishes completeness, soundness and zero knowledge.
Because the certified seed is, in my view, a poor model on its own (for example, 38.21% retained accuracy on FMNIST and 11.07% on CIFAR-100), the model that is actually deployed, , comes from an uncertified three-stage pipeline called TraceTrust: (i) retained-conflict projection, Fisher trust damping and top- Fisher-scaled masking; (ii) a grid search over the interpolation scale ; and (iii) one epoch of KD-anchored repair whose gradient is "shielded" along .
The experiments cover FMNIST with a small CNN and CIFAR-10/100 with ResNet-18, clients, Dirichlet , single-client deletion and 11 baselines. The authors report top-two retained accuracy and KL, MIA-gap equivalence to retraining within pp over 20 seeds, and Halo2 certificates for 11M parameters (8.3 h to prove, 29 s to verify).
Critical strengths
Strength 1. In my opinion, the central observation is clean and correct: linear aggregation turns client removal into an affine relation over homomorphic commitments. I find the paper's best idea to be its use of the linearity of the server rule, , together with the additive homomorphism of Pedersen commitments:
Together, these collapse the statement "the server subtracted exactly this client's historical contribution" from an replay to an check after public aggregation. I also appreciate that the construction extends neatly beyond FedAvg. Appendix H.1 shows that FedProx, FedAvgM (with ) and FedNova all fit the same template once is replaced by . The same template also covers round-wise partial participation (Eq. 16), selected-round deletion, and cumulative requests at a cost that does not grow with . As far as I can verify, the soundness argument of Appendix E.1 is short and correct: C1 plus binding forces , and C2–C3 then fix the seed.
Strength 2. I would like to commend the authors for an unusually candid and thorough supplementary analysis. From my perspective, the authors do not hide the limitations of their own theory. Appendix F.3 measures the expansion ratio of the retained update map along real trajectories and openly reports that the worst-case factor would exceed and . Appendix E.2 states explicitly what zero knowledge does not hide, and Appendix J concedes that a Schnorr-style opening proof could replace the Halo2 backend. The release also includes an implemented replay baseline on the same Halo2 stack (Table 24), a quantisation-error analysis (F.4), and sequential, selected-round and partial-participation experiments. This level of openness made a careful assessment possible, and I think it deserves credit.
Critical weaknesses
Weakness 1. In my view, the certificate verifies an object that is never deployed. The deployed model carries neither a cryptographic nor an approximation guarantee, and the ZK component proves a statement that the verifier could check without it.
The paper motivates itself (Introduction, ll. 44–47) with the observation of Zhang et al. (2024) that behavioural checks "can be gamed by adaptive servers" that do not perform the intended computation. Yet, as far as I can tell, the deployed model is not bound by in any way. The released tuple is equally valid if the server simply sets , or fine-tunes for one epoch. I therefore believe the certificate does not rule out exactly the adaptive server the paper sets out to defeat.
I do not think this is a harmless gap between a "seed" and a "polished" model. The certified seed is, in my assessment, unusable as a model. Its retained accuracy is 38.21 ± 15.73% on FMNIST (with a KL of 10.59 to retraining), 23.84% on CIFAR-10 and 11.07% on CIFAR-100 (Table 3a and the Direct Accumulation rows of Table 1). The deployed , by contrast, reaches 92.52%, 93.27% and 93.48%. All of the reported utility, and all of the reported forgetting, therefore comes from the uncertified TraceTrust stages. From my reading of the method and the code, those stages are built in a way that works against forgetting:
- The interpolation search can retreat towards the original model. TCI searches with , so the deployed model may sit three quarters of the way back towards .
- Both TCI and the repair loss distil from the teacher (Eqs. 10–11;
teacher_state=self.trace.final_model_stateinzkfu.py). That is the very model that contains the forgotten client. To my knowledge, distillation from such a teacher on retained data is a well-known route by which memorised information about leaks back into the student. - The shield constrains a single direction. It removes only the negative component of along one vector with . This is a rank-one constraint, and I see nothing that stops the repair from restoring forgotten behaviour through the remaining orthogonal directions.
Is it any surprise, then, that zkFU's forgetting gap on CIFAR-100 is 20.75 pp, compared with 2.78 for SIFU, 4.17 for FedAU and 5.33 for uS2U?
I also think the cryptographic contribution is considerably thinner than the headline suggests. Since , , and are all public, the witness is itself publicly computable, as the authors acknowledge in Appendix E.2:
A verifier can therefore form and check directly. In my understanding, the server could even publish outright, because opening a commitment to a value that is already public reveals nothing new. Constraint C4, a commitment to the already public , seems equally redundant. The zero-knowledge property thus hides nothing that is not already public, and the 8.3-hour Halo2 proof for ResNet-18 certifies something a verifier could check with a single -term multi-scalar multiplication in seconds. I would argue that the headline "930x lower proving time" is measured against an aggregation-replay strawman rather than against this trivial direct check. Appendix J's own remark ("they do not establish superiority over this specialized opening proof") all but concedes the point.
Finally, I am concerned about an uncomfortable privacy consequence. A client who exercises the right to erasure (GDPR Art. 17) has its cumulative weighted update released to the public as a by-product of the certificate. That is precisely the artefact that gradient-inversion and property-inference attacks target.
Weakness 2. I believe the empirical evidence for "retraining fidelity" is undermined by transductive use of the test set, a mis-specified notion of "better", and a membership attack with essentially no statistical power.
(a) Test-set leakage. When I inspected the released code, I found that the "public" loader used by both TCI and FDS-ADR is the test set itself. runner.py, l. 772, sets non_member_loader = test_dl; this loader is passed as public_loader to run_trust_calibrated_interpolation and run_anchored_distillation_repair, where run_central_prox_kd_repair adds a KD term on public_x drawn from it. The same set therefore serves three roles at once:
- the KD inputs that pull towards (the KD term also enters the objective that selects );
- the test-accuracy (TA) evaluation; and
- the non-member set of every reported MIA (
mia_non_member_protocol="test_holdout").
In my opinion, fitting the deployed model's output distribution on exactly the non-member samples that the shadow-model attack later scores directly contaminates both the MIA Gap and the TOST equivalence claims. I could not find this disclosed anywhere in the paper; Section 6.3 and Appendix G.1 speak only of "retained or public validation signals". Did the baselines receive the same access? If not, I do not think Table 1 is a like-for-like comparison.
(b) "Higher is better" is the wrong objective when retraining is the reference. Table 1 declares Federated Retrain the reference, yet ranks methods by retained accuracy with an upward arrow. zkFU exceeds retraining by roughly 13 pp on CIFAR-10 (93.27 vs. 80.20) and roughly 10 pp on CIFAR-100 (93.48 vs. 83.31). In Table 15, zkFU's test accuracy (85.89) even exceeds that of the original, un-unlearned model (78.63), and in Table 12 it exceeds retraining by about 10 pp (86.52 vs. 76.42). From my point of view, a model that is 7–13 pp more accurate than both the retrained and the original model is, by definition, not behaving like the retrained model. I suspect this reflects an under-trained retrain oracle (; Table 10 even reports a retrain retained accuracy of 79.39 ± 10.86 on CIFAR-10) combined with extra compute and test-set KD in the repair stage. I therefore think the claim that zkFU is "the only method ranked first or second in both retained accuracy and KL" rests on one metric that rewards departure from the reference and on a KL of 0.277–0.442 that is far from negligible.
(c) The membership-inference evidence has essentially no power. The per-example LiRA in Table 14 yields an AUC between 0.493 and 0.508 for all models, including the original ; in other words, the attack cannot even distinguish "no unlearning" from retraining. The shadow-model attack separates the original model from retraining by under 1 pp on FMNIST (1.79 vs. 0.88) and under 2 pp on CIFAR-10 (2.50 vs. 0.61). I find it hard to accept a TOST equivalence margin of pp as meaningful when the entire signal is only about 1–2 pp wide. Based on my familiarity with the FU literature, the standard protocol here would be backdoor or canary injection into the forgotten client, with the attack-success rate reported before and after unlearning. That protocol is absent, as are client-level (rather than sample-level) membership attacks.
(d) The theory does not cover the regime that is evaluated. Theorem 1 requires . Yet with local epochs, momentum 0.9 and , the authors' own measured per-client lies between 0.2 and 2.0 (Fig. 4c), which gives –. The general bound is then vacuous, and with it is of the same order as the trivial bound . More fundamentally, I see no guarantee of any kind, cryptographic or approximation-theoretic, for , which is the only model an end user would ever interact with.
Questions for authors
-
Binding the deployed model. Under your threat model, what prevents a server from publishing a valid while deploying or any fine-tuned variant of it? If the answer is "nothing", I would like to understand in what sense zkFU's guarantee for the deployed model is stronger than that of the behavioural verification (VeriFi/uS2U) the paper criticises. I think it would considerably strengthen the work to bound , or , inside the certificate, for example by also certifying the sparse mask , the selected and a bounded number of repair steps. Is this feasible?
-
Necessity of the SNARK. Since is a deterministic function of public quantities (Appendix E.2), what, in your view, does the Halo2 proof guarantee beyond the direct check , which needs no zero knowledge at all? I would encourage you to report the wall-clock cost of this direct check (or of the Schnorr variant in Appendix J) for ResNet-18 alongside the 8.3 h Halo2 proof, and to restate the 930x claim against it.
-
Test-set usage. Could you confirm that
public_loaderin TCI and FDS-ADR is the test set (runner.py, l. 772), and that the same set serves as the MIA non-member set? I believe Table 1, Table 3(b) and Table 14 should be re-run with a disjoint public split, or with no public KD at all. Please also state whether any baseline had comparable access to test inputs. -
Distilling from . Why do you distil from the teacher , which by construction encodes , rather than from or ? I would like to see an ablation of the teacher choice that reports together with a backdoor attack-success rate on the forgotten client.
-
Accuracy above retraining. How do you reconcile retained and test accuracy exceeding both the retrained and the original model (e.g., 85.89 vs. 77.71 vs. 78.63 in Table 15) with the claim of fidelity to retraining? Would the conclusions change with a converged retrain oracle, or with one given the same extra repair compute (one KD epoch)?
-
Stronger forgetting evaluation. Could you add a backdoor or canary-based forgetting test and a client-level MIA? In my reading, the LiRA AUCs in Table 14 suggest that the current attacks do not separate even from retraining.
-
Federated LLMs and LoRA. From my point of view, the most pressing use case for client erasure today is the federated fine-tuning of large language models, typically through parameter-efficient adapters such as LoRA, where with , and . I have three related questions: (a) When FedAvg aggregates and separately (as in FedIT), the trace identity is linear in the submitted , but the effective update is bilinear: . Does TCGS remain meaningful in this setting, and how would the bilinearity enter the trajectory-mismatch bound of Theorem 1? (b) If I extrapolate Table 20 linearly (about 2.76 ms per coordinate), a certificate for a full 7B-parameter model would take on the order of s, i.e., roughly seven months. A rank-16 LoRA adapter on the query and value projections of a 7B model (about parameters) would cost roughly as much as your ResNet-18 certificate. How do you see zkFU scaling to either setting? (c) The retained openings (about 4.5 GB per ResNet-18 client) seem prohibitive for cross-device federated LLM fine-tuning. Is the running-aggregate trick of Appendix B compatible with secure aggregation, under which the server never sees the individual ?
-
Privacy of the departing client. Releasing publicly discloses the forgotten client's cumulative update at precisely the moment that client asks to be forgotten. Have you evaluated gradient-inversion or property-inference attacks on ? Would it be possible to release only a commitment to , which would make C4 meaningful, and keep the seed itself private?
-
Theory–experiment gap. Given the measured of roughly 20 to 200, under what practical conditions do you expect Assumption 1 and the stable regime to hold? Could the bound instead be stated in terms of the measured aggregate expansion rather than the per-client ?
-
Hyperparameters. TraceTrust introduces at least ten hyperparameters (, , , , , , , , , , , ). On which data were they tuned, and were the eleven baselines tuned with a comparable budget?