- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies the impact of memorization using DP-SGD on long-tailed data. Providing new insights on the training dynamics and generalization.
Critical strengths
This paper has more fine-grained analysis than the prior work.
Critical weaknesses
- It would be good to have a table of notations for the symbols.
- While proving some bounds is useful, the expressions are so complicated. It's difficult to understand the behavior or use it in practice.
Questions for authors
- The long-tailed data are identified by non-DP tranining. But the analysis is on DP training, how do you ensure these data are still long-tailed?
- Can you explain how do you define the signal-to-noise ratio and more importantly, the noise correlation ratio?
- What are the main difficulties to generalize to deep models?
- There're several conditions (4.1), does your experiments satisfy all of them?
- Does your analysis applied to other DP algorithms, such as the DP-FTRL?