acceptodds
Withdrawn from ICLR 2027

Attested but Unaccounted: DP-SGD Privacy Accounting as a Precondition of Every Step

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Confidential training offers one attestation report as evidence for two claims: that data stays inside an attested virtual machine, and that the released model satisfies (ε, δ)-differential privacy. The report supports only the first, because it measures code, whereas ε depends on how many steps touched the data and which composition state accounted for them. We propose DP-Attest, which makes privacy accounting a precondition of every DP-SGD step: an attested accountant charges and seals each step against an external register before minting the token that a separate data plane requires to serve that step's batch. We measure the gap across seven accounting surfaces in six widely used packages, in real DP-SGD training and with membership audits; prove that the durable accounting state dominates the cost of every released output; and show that a fresh-once registration, which refuses every repeated sample context, makes the accountant enforce the schedule our privacy corollary assumes. No surface compares a loaded composition state with the one it holds, so DP-SGD that checkpoints, crashes and resumes touches 800 steps while accounting for 600, and at a fixed published ε, rewinding the accountant buys up to 54 accuracy points. DP-Attest shows no invariant violations across 235,298 enumerated interleavings of crashes, rollbacks and replays, and its gate removes the rewinding gain for a measured 0.21% of a 29.36 ms step in a CPU prototype. Binding ε to the steps a trainer ran turns a privacy claim from an assertion into an accounting any verifier can recompute.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
differential privacy, DP-SGD, privacy accounting, confidential computing, remote attestation, trusted execution environments, rollback attacks, trustworthy machine learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_GjheOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper considers private model training in a third-party machine with DP-SGD. DP-SGD provides privacy guarantees by accounting how many times training “touched” the training dataset, and outputs a resulting privacy parameter ε\varepsilon. An attacker that compromises the third-party machine can affect the execution flow of the training image, resulting in more training steps which in turn should result in higher ε\varepsilon. This higher ε\varepsilon is not accounted in standard DP-SGD. Thus an adversary that observes the gradients of each step can extract more information about the model than DP-SGD guarantees.

The authors propose DP-Attest where an Accountant, run in a different trusted VM, issues tokens that the trainer can use to request from a different entity (called the data plane) some training data. By decoupling the accountant from the trainer, the adversary cannot rewind its state. Thus, every request for data is properly accounted in terms of privacy. 



The paper evaluates that leaking each step has meaningful effects for privacy, for small levels of noise and that DP-Attest accounts correctly for all “touched” elements regardless of malicious crashes of the training image.

Critical strengths
  • The proposed framework fills in a gap against strong adversaries that can exploit dependencies or system execution during training. DP-Attest protects against a series of execution flow attacks.

  • The technical contribution is the separation of training and accounting, which is interesting, along with an external monotone register, conditional updates and write-ahead operations.

  • The protocol addresses computational cost with prepaid tokens.

  • The paper examines accounting after rewinding in real DP-SGD libraries.

  • The paper describes its limitations, threat model and assumptions.

  • Extended evaluation, albeit in the appendix for space constraints, show the effectiveness of the framework against a series of attacks.

Critical weaknesses
  • Unclear attack: the paper assumes a compromised dependency of the training job where the (noisy) gradients get leaked in each step. The paper does not specify how this attack is actually instantiated and if standard hash checks can verify that the correct (not leaky) package (e.g. Opacus) is loaded during execution.

  • Trust asymmetry: The paper supposes that the attacker can only compromise dependency on the training machine. Why can’t they do this on the accountant/data plane/register machine creating the same problem? If one of these components is run on an honest machine, why not move the training there as well? 


  • The design has the drawback of unnecessarily accounting for extra privacy in the case that the crash was honest. Rewinding during honest crashes, when the gradients are not leaked (i.e. default case) does not expose information. But DP-Attest will still account for them.

  • The paper is difficult to follow. The presentation can be improved by (a) assuming that the reader is only familiar with ML and explaining concepts such as DP and attacks on attestation machines (b) explaining "what" DP-Attest does (end-to-end description) before describing the "why" (c) avoid compressed explanations without giving intuition (d) avoiding abrupt mental shifts (e.g. the first sentence of “The gap” is clear. The second sentence shifts the reader to a different idea: “Continuity makes a ledger durable”: Continuity of what? Which ledger? How is it achieved? ).

Questions for authors
  • Can you provide (theoretically) and evaluate (implement and use DP-Attest) an end-to-end, step by step, clear, attack example?

  • The malicious code is assumed to be faithful. Why can't the code use standard hash checks to verify that the correct package dependencies are used (or VeriDP) that do not leak gradients?

  • What scenario justifies the asymmetric trust assumption?

  • What is the trade-off of the over-accounting of DP-Attest in "honest" crashes, when the gradients of each step are not leaked?

  • How the framework compares to ElephantDP when DP-Attest is configured for the gradient-releasing plane (Lines 251-253)?

Reviewer_zr2dOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper start from the observation that a confidential-computing attestation report certifies the launch measurement of a trainer image, not the execution trace that determines the (ε,δ)(\varepsilon,\delta) of differential privacy assessment. Because ε\varepsilon depends on the number of steps that touched the data, a binary whose launch measurement verifies correctly could still omit accountant calls, rewind or fork sealed state, or resume after a crash while the reported guarantee remains unchanged. The authors measure this gap across seven accounting surfaces in six packages and show that unaccounted steps can be obtained for large accuracy gains at a fixed published ε\varepsilon. They propose DP-ATTEST: an attested accountant that must charge and seal each step against an external monotone register before minting a release token; a separate data plane serves the batch only against that token (“no token, no batch”). Under the stated assumptions the durable accounting state dominates the released transcript. In a CPU prototype the gate costs 0.21 % of a step and eliminates the rewinding gain for the attacker; no invariant violations appear across hundreds of thousands of enumerated interleavings.

Critical strengths

The problem is cleanly demonstrated. Existing DP accounting libraries treat the accountant as a passive recipient; none of the surveyed surfaces compare a loaded composition state with the one they hold, so checkpoint/crash/resume already produces unaccounted steps.

The design principle—making the charge a precondition of data release rather than a report after the fact—is simple and addresses the omission attack that continuity mechanisms alone do not catch.

The evaluation is thorough on the attacks the authors chose to study (omission, rewind, fork, replay, crash schedules) and includes both formal enumeration of interleavings and concrete accuracy measurements.

The paper is explicit about several of its trust assumptions and about the distinction between the weaker adversary (transient batch) and the stronger one (retains every record).

Critical weaknesses

The threat model is too narrow. The adversary is a compromised component inside an otherwise honest deployment that misbehaves in constrained ways (skip the accountant, rewind sealed state, replay) while leaving the loss curve smooth and every attestation check succeeding. Against unconstrained hostile code that can forge tokens, extract keys, reconstruct sampler randomness, clone the accountant–plane pair inside a prepaid window, or exfiltrate data through channels the gate does not observe, the counter would not notice because the assumptions that make it sufficient have already been violated. Experience with dynamic, behavior-based malware suggests that once code is running inside the measured image it will not confine itself to the “omit the call” pattern.

The token gate solves a specific accounting-completeness problem under the listed assumptions; it does not, by itself, prevent a provider who has already received or retained the data from training a non-private model in parallel or on a side channel. Several mechanisms required by the proof (external monotone register that the adversary can neither lower nor stall, secret sampler randomness, no live cloning inside a prepaid window) are not provided by current confidential VMs and are attacked or priced only in the appendix.

The practical relevance of constrained misbehaviors (especially when not experienced in the field) remains open. While the library measurements show that current accountants are incomplete, the paper does not establish how often, or under what deployment conditions, an adversary can usefully exploit that incompleteness without breaking the stronger assumptions.

Questions for authors

Your adversary is a compromised component that omits accountant calls or rewinds state while the launch measurement still verifies. How robust is the token gate once the code inside the measured image can perform arbitrary control-flow or data-flow manipulation (for example, forging a token, extracting the data-plane MAC key, or reconstructing the sampler seed)? Which of the assumptions in your Proposition 1 fail first under such behavior?

In a realistic confidential-training deployment, how does a verifier distinguish the “constrained omission” attack you defend against from ordinary operational events or from a provider who simply retains released batches and trains a second non-private model outside the gated plane? What additional controls, if any, are required for the privacy claim a customer reads off the attestation report to match the guarantee DP-ATTEST actually delivers?

Reviewer_DNiNOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper highlights that confidential differentially private training is prone to rollback attacks in which the adversarial model trainer trains the model for more steps than it accounts for. This makes the model more accurate and violates the DP guarantees claimed by the trainer. In this paper, they first examine existing dp-accounting libraries against the above attack. Next, they separate the malicious trainer, the DP accountant and the data, and make DP accounting a pre-condition for the model to access the data. At each step, the trainer requests the data and the accountant pre-charges privacy for k-steps and provides a token to access the data.

Critical strengths
  1. The paper identifies a gap in the verifiable private training procedure and demonstrate a roll-back/ unaccounted training steps attack.
  2. The paper proposes a simple solution to make privacy accounting a pre-condition for data release to the malicious model trainer.
Critical weaknesses
  1. The presentation needs serious work. The language is a often obscure, with frequent references to appendices that make the main argument difficult to follow.
  2. The idea is fairly simple and the experiments do not demonstrate the costs of running on actual confidential VMs or GPUs.
Questions for authors
  1. Do any existing deployable configuration provide full register contract and the other conditions required by the protocol against a malicious provider?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.