- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper
Critical strengths
The paper use a novel concept of using the confidence of the model's output in order to untrain / unlearn. The paper use a novel definition that yields a DP-style guarantees. The paper shows an upper bound on and on the DP-style guarantees. In addition, the paper backs its claims with various experiments.
Critical weaknesses
-
I'm concerned that the paper is mostly AI generated. Even if the main ideas and proofs are of the authors, the writings seems mostly AI with little to no human intervention. This makes the paper really hard to understand. For example, in the Introduction there are 4 lines of introduction before stating what is missing from current work. At this point, I still have no idea what is the current work, I even don't know what is the difference between unlearning and untraining. Another example is mentioning differential privacy without ever citing [1]. Another example is the usage of the word "certify" in many places which is a bit unordinary.
-
The paper use many undefined terms, such as f-divergence, DP mechanisms for privacy-preserving predictions (which seems very relevant to the paper)
-
The paper seems a bit of a mess, where some of the definition and all the theorems are deferred to the appendix. This makes the paper really hard to follow. I can't tell what parts of the algorithms are proven and what are only empirical.
Giving all this, it is hard for me to assess the idea itself.
[1] Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis.
Questions for authors
-
In Definition 2, what is the purpose of the partition? Why not just use [K] (the class of labels)? Are the distribution over the output of the model or over its confidence? it is not clear to me. Lines 124-125 make me think that the distribution is over the output.
-
what is f-divergence (line 128). It is never defined and never really used anywhere.
-
In algorithm 1, only CUP-DF is presented with a short sentence about CUP algorithm. Consider adding the CUP part to the pseudocode itself with a comment what part is only for CUP algorithm.
3 comments
Answer
We revised the paper focusing on improving the presentation. The revisions include additions, clarifications and moving parts from Appendix B (Preliminaries from Statistics and Information Theory) to the main manuscript. Lines from the revised paper that refer to the reviewer's comments are denoted here as (xxx-yyy). We accept that the presentation could be improved in some parts and we refer the reviewer to the revised paper and the brief summary of our revisions (shared as a comment to all the reviewers).
We do not accept the reviewer's self-reported "careful reading", mostly based on: (i) missing paper summary, (ii) no comments even for the algorithmic/experimental contributions of the paper, and (iii) contents of Question 1. We claim full responsibility and ownership over this paper, the AI use is explicitly reported in (542-549). We appreciate the reviewer's comment on the originality of the theoretical results, and his/her effort/time for pointing out parts that could enhance the presentation of the paper.
- Additions and clarifications discussed in Weakness 1:
- (030-037): Further details about the the difference between untraining (unlearning for privacy) and unlearning for concept removal, which were previously included in the Appendix A (Reltated Work). New experiment and analysis in 498-518 provide further intuition.
- We cite early the DP paper of Dwork et al. in line 044. Previously, the reference to this paper was indirect coming from prior certified work in Gap 1, that adapts formulation from this to the unlearning/untraining task. We also cite this paper in 183, replacing the reference to DP-SGD from Abadi et al. which adapts DP formulation to machine learning tasks. We accept that the fundamental paper of Dwork et al. is more recognizable and facilitates readability.
- (038-051): Intuition/Clarification on the difference between certified and heuristic untraining. This intuition is later formalized in Sec. 2 (132-160). Related work (i.e., prior untraining certificates) remains as part of Gap 1. Further details are in in Appendix A (Related Work), the formulation of prior (121-131).
- We replace instances of the word "certify", that refer to our TV certificate or prior certified work, with descriptive phrases such as "yields a certificate" or "provides privacy guarantees". We do not accept that the word "certify" is odd in the context of certified work, but it remains more ambiguous than descriptive phrasings.
- We couldn't find any other terms than the following that could be considered unknown:
- f-divergence: In 210 we add the reference to the Duchi's "lecture notes on information theory", which we broadly cite in the appendix to refer to fundamental results and preliminaries from information theory. Moreover, we explicitly mention that the Total Variation distance is an f-divergence to avoid any confusion on why it is relevant to our paper. The term is used once in the paragraph (208-215) which explains why quantization (i.e., binning) is needed because this refers to f-divergences in general, not just TV. We accept that the reference add to the paper as Duchi's definition provides a more rigorous analysis on why quantization is needed.
- DP mechanisms for privacy-preserving prediction: the term was used once next to the citation "Dwork and Vitaly, Privacy-Preserving Prediction (2018)". It is not fundamental for the paper, it was used only for intuition. We wanted to link the DP literature with the certified work in untraining. Specifically, the DP literature has examined privacy guarantees in both the model space and the output space, while in certified untraining the output space remains underexplored: that is what we study here to address Gap 1. Now, in our revised paragraph 182-188, we use more recognizable papers that refer to these concept to facilitate intuition. And build intuition on concepts that are more recognizable (global vs local guarantees).
- In Appendix B we include known results, either fundamental (from academic textbooks) or from recent papers. We had them in the appendix for the reproducibility of the theoretical claims. We accept that this may hampered readability, thus we transfer some of them along with an analysis for intuition in Sec. 2 (132-161).
- The partition (i.e., quantization/binning) is not a design choice, but a requirement (see Cover's book in our references (Ch 11, (p. 294)): "This arguement can be also extended to continuous distributions using quantization"). The reference that we now added for f-divergence (Duchi's lecture notes), upon your righteous request, presents a more rigorous analysis on why quantization is needed. In 218-215, we present a more intuitive justification, and the revision aimed to further facilitate intuition.
- We included the pseudocode of the algorithm in Sec. 4 (361-376) and moved the data-free algorithm to the Appendix D.
Thank you for your review. Your concerns about the presentation played a central role in our revision, while the original contributions remain intact:
- Unchanged: the TV certificate and its decomposition, the theoretical results and their proofs, the method, and the experimental results.
- Added: one experiment supporting the analysis of our assumption (498-517).
We apologize for the late response. We considered all comments very carefully, and chose to first revise the paper and then write comprehensive answers. We particularly focused on your concerns about the presentation, so that you can already assess a revised paper with improved presentation, rather than promises of changes for the camera-ready version.
All changes respond to concerns raised by the reviewers. We hope that the revisions on the presentation facilitate a clearer assessment of our contributions. Your scores on Soundness and Informativeness strongly motivated us to improve the presentation within the limited time of the rebuttal. We would greatly appreciate it if you could revisit the revised paper during the remaining discussion. Thank you, and all the best to you too!
I thank the authors for their response in the general comment. The rebuttal included too many changes which makes the paper very different from the version that was initially submitted. I encourage to authors to rewrite the paper to be easier to follow and submit to another conference. I would keep my score, good luck.