- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents PrivBlender, an offline-online framework for prompt perturbation methods, based on the exponential mechanism. It presamples candidate outputs at two privacy budgets offline, then selects between them in constant time online. The main theoretical result gives a mixing probability that satisfies an intermediate target privacy budget for symmetric utility functions. The authors integrate the framework with multiple existing prompt perturbation methods, and evaluate efficiency, downstream utility, and reconstruction attacks on several tasks.
Critical strengths
The mixing result provides a simple way to support a range of target privacy budgets using two presampled candidates, which improves online efficiency.
Critical weaknesses
I am concerned about whether the cache construction preserves privacy when the whole perturbed prompt is released. Scenario 1 of sec 4 states "prepare candidate tokens for each possible token " using storage. This seems to suggest that repeated occurrences of a word share the same two candidates (though I can't find an explicit description) and if this is the intended construction, the guarantee on the individual token won't extend to the whole output sequence. Think about this example - the prompt (a,a,a) cannot produce three distinct outputs because all positions choose from the same two candidates, but (b,a,a) can do so with positive probability. This violates any finite pure-DP bound for the joint output. Using independent candidate pairs for each position would avoid this example, but the authors need to clarify the actual construction and the corresponding preparation and storage costs in this prompt-unknown scenario.
Questions for authors
- Are candidate pairs shared by all occurrences of a vocabulary token, or sampled independently for each position? Can the authors address the example stated in Weakness?
- Which deployment scenario does Table 1 measure? What is the exact timed step? Concretely, which preparation steps must be repeated for a new prompt when using CAPE (context-dependent scores), how many releases reuse that preparation? I agree that moving work offline can be useful; the question is which costs are reusable in the intended setting.
Revision file uploaded
Thank you again for your constructive review. We have uploaded a revised version, with text in blue marking the changes.
A brief summary on the changes made:
Our main algorithmic contribuition in Theorem 1 is left unchanged.
We hope these responses address your concerns, and we would be grateful if you would consider raising your score.