acceptodds
Under review as a conference paper at ICLR 2027

ORBIT: Dual-Shell Latent Geometry for Differentially Private Anomaly Representation Sharing

Abstract

Sharing learned representations is increasingly important in cross-institutional anomaly detection, where differential privacy (DP) protects the released embeddings. Under extreme class imbalance, full-dimensional latent DP can erase the rare-class structure that makes the shared representation useful. We show that no per-record -DP release that protects the class can exceed precision at imbalance ratio , and that a two-point release of a single coordinate attains this bound. We propose ORBIT (One-dimensional Radial Budgeting with Inter-shell Topology), which concentrates class-separating structure into a bounded radial coordinate rather than perturbing the full latent representation. Its encoder is trained with a dual-shell margin objective that places normal and anomalous records around two target radii. ORBIT clips the radius to a fixed range and releases it through a one-dimensional Laplace mechanism whose sensitivity depends on that range rather than the latent dimension. Under the same per-record guarantee, full-dimensional latent DP stays near the base rate up to on 577:1 fraud data, whereas ORBIT's released radius reaches AUPRC 0.74 at (bound 0.97), on par with a supervised score released in the same way. On the tabular datasets, unsupervised one-dimensional summaries reach at most 0.43 at , so compression alone does not explain the gain. Unlike a supervised score, ORBIT's representation also carries a direction, which a scoped variant publishes. This variant lies outside the bound because it declares the class public, and its formal guarantee covers only the radius. The identity leakage of the direction is bounded under an angular-concentration condition and audited with direction-only attacks, which are consistent with chance on both tabular datasets but detect training-set membership of minority records on CIFAR-10. Under this scope, ORBIT's latent-space synthetic release retains the non-private AUPRC at on all three datasets.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
differential privacy, anomaly detection, extreme class imbalance, representation learning, latent geometry, privacy-utility trade-off, membership inference, synthetic data

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_27CpOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

ORBIT addresses a weakness of using differential privacy directly to inject noise to full-dimensional latent representation vectors in representation learning tasks. DP induced noise introduces distortions which are especially detrimental to the rare-classes (such as anomalies). ORBIT uses a dual-radii representation-shaping design to preserve class-separating structure in a bounded radial coordinate. The results show that (a) radius-only release substantially improves anomaly-ranking over the full-dimensional DP baseline and performs comparably to supervised scalar scores (Section 6.1), and (b) the direction-public synthetic release retains utility close to the non-private baseline (Section 6.2). Author(s) explicitly state that the direction is released as public information, hence, privacy audits using direction-only inference attacks reveals a limitation of ORBIT: it exposes training-set membership for minority records on CIFAR-10, although the tested attacks detect no such leakage on the two tabular datasets (Section 6.4).

Critical strengths
  1. The proposed algorithm protects shared representations while preserving rare-class (anomaly) information. This is important for applications such as fraud detection. It offers marked advantage over injecting noise to full-dimensional latent vectors.

  2. ORBIT supports multiple release settings, including radius-only release , direction-public per-record release, and fully synthetic release (lines 224–231). The paper clearly states the privacy scope of each release setting.

  3. The author(s) undetake an extensive empirical evaluation across multiple datasets, baselines, and privacy budgets. They also examine the effects of class imbalance, shell separation, release settings, and latent dimension, and assess privacy leakage through direction-only membership/identity inference attacks. These experiments help explain both ORBIT’s advantages and its limitations.

  4. I commend the author(s) transparency in acknowledging the limitations of their proposed method. They did not curate the paper to show only the positive results.

Critical weaknesses
  1. The conceptual difference between Radial baseline and ORBIT is the gravity term. The empirical results in Figures 3 and 4 do not establish any significant advantage that comes from using the former over the latter. Certainly, ORBIT injects less noise compared to the Radial baseline (as mentioned in lines 918-19) but that does not translate to demonstrative utility improvement on downstream tasks. This is a limitation acknowledged by the authors (lines 217-218 and 471). A similar lack of distinct advantage is seen against supervised scores in Figure 1. From the paper, it is unclear why one would prefer ORBIT over Radial baseline or supervised score summaries.

  2. Using ORBIT does not guarantee dataset-level DP for training the encoder and then releasing all representations. The authors explicitly acknowledge this limitation, stating that “the guarantee applies to the released representations, not to encoder training” (lines 45–46). This weakness gets compounded by Weakness #1. ORBIT is narrow in its scope of application and the paper offers little evidence of it offering any significant advantage over MLP scores (Figure 1).

Questions for authors

Questions

  1. WRT Weakness #1, ORBIT saves on privacy budget against the Radial baseline. But, it is unclear how this less noisy release would help the users? Do the author(s) think, we can redirect this budget to other parts of the training pipeline? Could you elaborate over this as it is unclear to me why one would prefer ORBIT if even after injecting lower noise, it does not lead to any improved performance utility-wise?

  2. In lines 339-340, it is stated that The supervised scores (MLP, GBDT) use a clip window tuned per ε; ORBIT’s window [R1, R2] is fixed. Will this count as an advantage of ORBIT over MLP scores? If yes, I would suggest to base superiority of ORBIT over MLP scores in Section 6.1 around this point.

Suggestions

  • In the abstract, the author(s) assess ORBIT across many different privacy settings (ϵ=20,10,1\epsilon=20,10,1). For example, line 24-26 mention under the same per-record guarantee...on par with a supervised score released in the same way. This sounds like per-record guarantee for standard DP at ϵ=20\epsilon=20 is equivalent to ORBIT with ϵ=10\epsilon=10. Please clarify that the methods use the same privacy definition but different privacy budgets. It would be even better to focus on one privacy setting and show what advantage ORBIT offers over the baselines under the same privacy guarantee.

  • The author(s) have not added any details about how the direction-only inference attacks work. I would advise them to add this in the Appendix.

  • In the paper, the author(s) jump several times between different names for the proposed approach, such as “ORBIT,” “ORBIT radius” (Figure 1), “Dual-Shell (Ours)” (Figures 3-4), "Dual Shell (1-D)" (Figure 7), and "Dual Shell (DP)" (Figure 9). Kindly use consistent naming convention with different suffixes to explain the differences between different variants of the same approach. "Dual-Shell" is a particularly poor choice since Radial baseline also uses the dual-shell representation shaping.

  • In Figure 1, the caption states, The CIFAR-10 panel shows ORBIT and Standard only. May the author(s) explain what prompted them to make such as choice?

  • In Table 2, it would be great if the author(s) could highlight the method(s) which yield highest AURPC across all ϵ\epsilons.

  • The paper will also benefit from,

    • A notations table in the main text to fix the symbols for different terms used later in the paper.
    • A table briefly listing the technical terms (and their definitions) associated with vector geometry such as vacuum margin (line 198), crossover (line 84), gravity (line 216), angular concentration (line 78). This can be placed in the Appendix.
Reviewer_Ry1qOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper discusses the problem of cross-institutional sharing of learned representatations under local DP with potentially highly imbalanced classes for, e.g. binary anomaly detection. An autoencoder is trained that separates normal and anomalous records at different target radii, and injects Laplace noise only onto the vector norm. Then two scenarios are discussed, one in which only the noised radius is released, and one which assumes the direction is safe to publish in the clear (an assumption that the authors suppose can be empirically verified) along with the noised radius. The authors upper-bound the achievable precision under local DP as a function of class imbalance, and show that the radial mechanism achieves this bound. In experiments on three imbalanced benchmarks, the radius-only mode matches noisy 1-D supervised baselines near the theoretical ceiling, while the full-vector mode fails a privacy audit (leaks feature attributes).

Critical strengths

The derivation showing that any local ϵ\epsilon-DP mechanism protecting the class label at imbalance ratio rr has precision and AUPRC upper-bounded by eϵ/(eϵ+r)e^\epsilon / (e^\epsilon + r), requiring ϵ>ln⁡r\epsilon > \ln r to exceed 50% precision is clarifying and practically informative.

The empirical evaluation is very rigorous. The authors use appropriate baselines and ablations to yield insight into the method's behavior. For example, they demonstrate via ablation that simple 1-D supervised scores match the proposed radius-only method, and conduct a privacy audit experiment to demonstrate feature-attribute leakage in the full-vector method.

Critical weaknesses

My main concern is about the unrealistic problem formulation. We have Institution A which holds raw records and anomaly labels, and wants to share per-record locally privatized embeddings so other institutions can train a downstream classifier. But "the encoder is proprietary and never released", so other institutions have no way to map their own raw transactions into the latent space. I'm not sure why that is useful. Moreover, it seems wasteful to release individual records with local DP. Why not release a privatized classifier trained with central DP with much higher utility? I feel the core practical motivation of the paper is unconvincing.

Even accepting the problem setup, I'm afraid both solution modes over-engineered and ineffective, as the paper's own results demonstrate. In the radius-only mode, the gravity term μ~\tilde{\mu} provides no empirical gain-- maybe it should not be part of the method? And once reduced to a 1-D scalar, the proposed method performs no better than just adding Laplace noise to the output of a standard supervised MLP or GBDT, making the dual-shell autoencoder unnecessary. In the mode that releases the direction in the clear, almost all classification utility comes from the unperturbed direction rather than the DP mechanism. So we lose any worst-case formal guarantee and fall back on empirical privacy testing.

Questions for authors
  • Since the encoder is proprietary and never released, how can other institutions practically use the shared latent representations on their own data?
  • Why would the curator release locally privatized embeddings instead of training and releasing a classifier or generator under dataset-level DP?
  • Since experiments demonstrate almost all utility comes from the unperturbed direction vector for low epsilon, in what sense does this mode offer any formal privacy advantage?
Reviewer_Jna3OpenReview ↗
Rating
1
Confidence
1
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper aims to protect the shared representations in cross-institutional anomaly detection using DP, and the authors claim that existing DP mechanism fail under extreme imbalance scenarios. To tackle this challenge, the authors proposes ORBIT, a selective DP mechanism, which decomposes the representation into private radius and public direction. They further theorotically and emperically demonstrate the effectiveness of ORBIT.

Critical strengths
  1. Practical significance. This paper addresses the important and practical problem of improving the utility of differential privacy (DP) for protecting shared representations in cross-institutional anomaly detection under extreme class imbalance.

  2. Methodological contribution. The authors propose ORBIT, a novel framework that decomposes representations into a private radius and a public direction, aiming to improve the utility of shared information while preserving privacy.

  3. Theoretical analysis. The paper provides theoretical analysis to support the effectiveness of ORBIT and justify the proposed design.

Critical weaknesses
  1. Poor presentation and organization. The manuscript is poorly structured and difficult to follow. The authors should provide an overview of the proposed method and organize the paragraph in a more systematic, point-by-point manner. The descriptions of figures and tables should also be revised to highlight the key information.

  2. Insufficient empirical evaluation. The experimental evaluation is inadequate. The baselines are outdated, the datasets are relatively simple, and the empirical results appear to be largely limited to Fig. 1. Moreover, results for some baselines mentioned in the manuscript are missing. The authors should include more recent baselines, evaluate the method on more challenging datasets, and provide complete experimental results to substantiate their claims.

Questions for authors

Scalability to More Complex Datasets. Can the proposed method scale to more complex datasets with substantially larger sample sizes and higher feature dimensionality

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.