- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper considers private model training in a third-party machine with DP-SGD. DP-SGD provides privacy guarantees by accounting how many times training “touched” the training dataset, and outputs a resulting privacy parameter . An attacker that compromises the third-party machine can affect the execution flow of the training image, resulting in more training steps which in turn should result in higher . This higher is not accounted in standard DP-SGD. Thus an adversary that observes the gradients of each step can extract more information about the model than DP-SGD guarantees.
The authors propose DP-Attest where an Accountant, run in a different trusted VM, issues tokens that the trainer can use to request from a different entity (called the data plane) some training data. By decoupling the accountant from the trainer, the adversary cannot rewind its state. Thus, every request for data is properly accounted in terms of privacy.
The paper evaluates that leaking each step has meaningful effects for privacy, for small levels of noise and that DP-Attest accounts correctly for all “touched” elements regardless of malicious crashes of the training image.
Critical strengths
-
The proposed framework fills in a gap against strong adversaries that can exploit dependencies or system execution during training. DP-Attest protects against a series of execution flow attacks.
-
The technical contribution is the separation of training and accounting, which is interesting, along with an external monotone register, conditional updates and write-ahead operations.
-
The protocol addresses computational cost with prepaid tokens.
-
The paper examines accounting after rewinding in real DP-SGD libraries.
-
The paper describes its limitations, threat model and assumptions.
-
Extended evaluation, albeit in the appendix for space constraints, show the effectiveness of the framework against a series of attacks.
Critical weaknesses
-
Unclear attack: the paper assumes a compromised dependency of the training job where the (noisy) gradients get leaked in each step. The paper does not specify how this attack is actually instantiated and if standard hash checks can verify that the correct (not leaky) package (e.g. Opacus) is loaded during execution.
-
Trust asymmetry: The paper supposes that the attacker can only compromise dependency on the training machine. Why can’t they do this on the accountant/data plane/register machine creating the same problem? If one of these components is run on an honest machine, why not move the training there as well?
-
The design has the drawback of unnecessarily accounting for extra privacy in the case that the crash was honest. Rewinding during honest crashes, when the gradients are not leaked (i.e. default case) does not expose information. But DP-Attest will still account for them.
-
The paper is difficult to follow. The presentation can be improved by (a) assuming that the reader is only familiar with ML and explaining concepts such as DP and attacks on attestation machines (b) explaining "what" DP-Attest does (end-to-end description) before describing the "why" (c) avoid compressed explanations without giving intuition (d) avoiding abrupt mental shifts (e.g. the first sentence of “The gap” is clear. The second sentence shifts the reader to a different idea: “Continuity makes a ledger durable”: Continuity of what? Which ledger? How is it achieved? ).
Questions for authors
-
Can you provide (theoretically) and evaluate (implement and use DP-Attest) an end-to-end, step by step, clear, attack example?
-
The malicious code is assumed to be faithful. Why can't the code use standard hash checks to verify that the correct package dependencies are used (or VeriDP) that do not leak gradients?
-
What scenario justifies the asymmetric trust assumption?
-
What is the trade-off of the over-accounting of DP-Attest in "honest" crashes, when the gradients of each step are not leaked?
-
How the framework compares to ElephantDP when DP-Attest is configured for the gradient-releasing plane (Lines 251-253)?