- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
ORBIT addresses a weakness of using differential privacy directly to inject noise to full-dimensional latent representation vectors in representation learning tasks. DP induced noise introduces distortions which are especially detrimental to the rare-classes (such as anomalies). ORBIT uses a dual-radii representation-shaping design to preserve class-separating structure in a bounded radial coordinate. The results show that (a) radius-only release substantially improves anomaly-ranking over the full-dimensional DP baseline and performs comparably to supervised scalar scores (Section 6.1), and (b) the direction-public synthetic release retains utility close to the non-private baseline (Section 6.2). Author(s) explicitly state that the direction is released as public information, hence, privacy audits using direction-only inference attacks reveals a limitation of ORBIT: it exposes training-set membership for minority records on CIFAR-10, although the tested attacks detect no such leakage on the two tabular datasets (Section 6.4).
Critical strengths
-
The proposed algorithm protects shared representations while preserving rare-class (anomaly) information. This is important for applications such as fraud detection. It offers marked advantage over injecting noise to full-dimensional latent vectors.
-
ORBIT supports multiple release settings, including radius-only release , direction-public per-record release, and fully synthetic release (lines 224–231). The paper clearly states the privacy scope of each release setting.
-
The author(s) undetake an extensive empirical evaluation across multiple datasets, baselines, and privacy budgets. They also examine the effects of class imbalance, shell separation, release settings, and latent dimension, and assess privacy leakage through direction-only membership/identity inference attacks. These experiments help explain both ORBIT’s advantages and its limitations.
-
I commend the author(s) transparency in acknowledging the limitations of their proposed method. They did not curate the paper to show only the positive results.
Critical weaknesses
-
The conceptual difference between Radial baseline and ORBIT is the gravity term. The empirical results in Figures 3 and 4 do not establish any significant advantage that comes from using the former over the latter. Certainly, ORBIT injects less noise compared to the Radial baseline (as mentioned in lines 918-19) but that does not translate to demonstrative utility improvement on downstream tasks. This is a limitation acknowledged by the authors (lines 217-218 and 471). A similar lack of distinct advantage is seen against supervised scores in Figure 1. From the paper, it is unclear why one would prefer ORBIT over Radial baseline or supervised score summaries.
-
Using ORBIT does not guarantee dataset-level DP for training the encoder and then releasing all representations. The authors explicitly acknowledge this limitation, stating that “the guarantee applies to the released representations, not to encoder training” (lines 45–46). This weakness gets compounded by Weakness #1. ORBIT is narrow in its scope of application and the paper offers little evidence of it offering any significant advantage over MLP scores (Figure 1).
Questions for authors
Questions
-
WRT Weakness #1, ORBIT saves on privacy budget against the Radial baseline. But, it is unclear how this less noisy release would help the users? Do the author(s) think, we can redirect this budget to other parts of the training pipeline? Could you elaborate over this as it is unclear to me why one would prefer ORBIT if even after injecting lower noise, it does not lead to any improved performance utility-wise?
-
In lines 339-340, it is stated that The supervised scores (MLP, GBDT) use a clip window tuned per ε; ORBIT’s window [R1, R2] is fixed. Will this count as an advantage of ORBIT over MLP scores? If yes, I would suggest to base superiority of ORBIT over MLP scores in Section 6.1 around this point.
Suggestions
-
In the abstract, the author(s) assess ORBIT across many different privacy settings (). For example, line 24-26 mention under the same per-record guarantee...on par with a supervised score released in the same way. This sounds like per-record guarantee for standard DP at is equivalent to ORBIT with . Please clarify that the methods use the same privacy definition but different privacy budgets. It would be even better to focus on one privacy setting and show what advantage ORBIT offers over the baselines under the same privacy guarantee.
-
The author(s) have not added any details about how the direction-only inference attacks work. I would advise them to add this in the Appendix.
-
In the paper, the author(s) jump several times between different names for the proposed approach, such as “ORBIT,” “ORBIT radius” (Figure 1), “Dual-Shell (Ours)” (Figures 3-4), "Dual Shell (1-D)" (Figure 7), and "Dual Shell (DP)" (Figure 9). Kindly use consistent naming convention with different suffixes to explain the differences between different variants of the same approach. "Dual-Shell" is a particularly poor choice since Radial baseline also uses the dual-shell representation shaping.
-
In Figure 1, the caption states, The CIFAR-10 panel shows ORBIT and Standard only. May the author(s) explain what prompted them to make such as choice?
-
In Table 2, it would be great if the author(s) could highlight the method(s) which yield highest AURPC across all s.
-
The paper will also benefit from,
- A notations table in the main text to fix the symbols for different terms used later in the paper.
- A table briefly listing the technical terms (and their definitions) associated with vector geometry such as vacuum margin (line 198), crossover (line 84), gravity (line 216), angular concentration (line 78). This can be placed in the Appendix.