- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies whether forgetting in federated continual learning reduces privacy leakage. It argues that declining accuracy on an old task does not erase information already present in the training transcript, while mechanisms that repeatedly use old data to retain accuracy may create further leakage. The paper formalizes this distinction through an artifact ledger and an influence horizon, audits membership leakage from different views of the training process, and proposes a method that privatizes each task’s statistics once and reuses the resulting releases. Its experiments report persistent transcript leakage despite forgetting, increased leakage under several retention methods, and a more favorable privacy–accuracy trade-off for the proposed method.
Critical strengths
Analyzing how forgetting relates to privacy leakage offers a novel and valuable perspective.
Critical weaknesses
The writing is so poor that the paper is difficult to follow. The motivation, problem formulation, and proposed framework are not explained in a coherent sequence. The paper introduces a release channel, four adversarial views, an artifact ledger, touched sets, privacy units, and an influence horizon in rapid succession, without using proper language to show how these concepts relate to one another. The propositions and theorems are also presented without almost any explaination. As a result, substantial effort is required simply to identify the paper’s main argument.
The novelty and contributions are likewise unclear. The paper lists a vocabulary, a membership-leakage audit, a privacy-accounting criterion, and a retention operator as separate contributions, but does not clearly distinguish the new technical elements in each from established tools such as differential privacy composition, membership inference audits, and noisy statistical summaries. In particular, I do not consider introducing “a vocabulary that makes the question answerable” to be a substantive novel contribution on its own. The authors need to identify precisely which results and design choices are original, and explain how they advance beyond the existing techniques they build upon.
Questions for authors
See weakness.
3 comments
I thank the authors for their response and their efforts during the rebuttal. However, I do not see substantial improvement in the presentation. The introduction, for example, still lacks a coherent structure that clearly establishes the motivation, the limitations of existing methods, and the contributions of the work. The writing remains below the standard expected of a research paper, making it difficult to assess the work’s contributions. Nevertheless, in recognition of the authors’ efforts during the rebuttal, I am raising my score to 2.
New Paper Uploaded
Thank you for your feedback and for increasing your score. It appears your comment was submitted right before our revised PDF was uploaded to the system.
We have now uploaded the updated manuscript with all revisions highlighted in blue. In particular, we substantially rewrote and restructured the Introduction (Section 1) to clearly establish the motivation, the limitations of existing methods, and our three core contributions (explicitly separating what we build on from what is new), and we added a roadmap, running example, and intuitive explanations before every theorem and proposition in Section 3.
If you have the time, we would greatly appreciate it if you could review the newly uploaded PDF and reconsider your score based on these presentation improvements.
Response to Reviewer 87FH
We thank the reviewer. We agree that the presentation made the argument hard to follow, and we have rewritten it. The revised PDF uses the extra page for the changes below.
W1. Coherence of the formulation, and explanation of the results
The paper makes one argument in three steps.
A federated adversary sees every broadcast, not only the final model. So the object to analyse is the transcript of all broadcasts.
Whether a lifelong privacy guarantee exists depends on one number: how many releases read each datum as tasks keep arriving.
- If it stays bounded (contractive), the guarantee is fixed for any number of tasks (Theorem 2).
Replay keeps reading old data, so it is accumulating, and our audit shows that its leakage grows. Retaining releases instead of data is contractive. Our operator does this and keeps high accuracy.
Section 3 now opens with this roadmap. Each concept answers one question in the chain, and the concepts appear in that order, illustrated with one running example (a client that keeps a replay buffer):
Before each theorem and proposition, we now add a short paragraph that says what it states, why it holds, and where the paper uses it.
W2: Novelty, and what is new beyond existing tools
We acknowledge the vocabulary is not a contribution on its own. We remove it from the list and restate the contributions below. For each one, we separate what we build on from what is new.
A measurement and a finding.
A characterization.
An operator.