acceptodds
Withdrawn from ICLR 2027

Learn Locally, Calibrate Globally: Client-Level Differentially Private Video Anomaly Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

An anomaly detector can rank moments within a video correctly yet misorder normal and abnormal frames across videos. We exploit this gap to rethink private collaboration: transfer knowledge for cross-video score alignment rather than an entire detection model. In this paper, we propose Public-Anchor Response Decoding for weakly supervised video anomaly detection (PARD-VAD), which learns locally and calibrates globally. Each client projects differences between matched private and public-only teacher responses onto a public function basis. Clipping, secure aggregation, and Gaussian perturbation produce a single two-coefficient differentially private (DP) release while teacher parameters remain local. Public decoding reconstructs temporal evidence, and public-score-weighted pooling converts it into one log-odds correction per video. This changes cross-video rankings while preserving within-video ordering, with inference performed as DP post-processing. For fixed public assets, sensitivity analysis and R\'enyi DP accounting establish client-level replace-one privacy. Using the same release and decoding rule, PARD-VAD achieves 89.19% AUC on UCF-Crime and 84.03% AP on XD-Violence on the complete test sets at a per-deployment budget of , , exceeding the reported privacy-preserving baselines in our comparison. Matched public-only controls isolate gains of 0.81 and 0.54 percentage points, and an exhaustive frame-pair audit attributes these gains to cross-video ordering.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Weakly Supervised Video Anomaly Detection, Client-Level Differential Privacy, Distributed Learning, Public-Anchor Knowledge Transfer, Cross-Video Score Calibration

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_JhNjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes PARD-VAD, a client-level differentially private framework for weakly supervised video anomaly detection. Each client trains a private teacher, projects response differences from the public teacher onto a function basis, and contributes to one clipped, Gaussian-noised two-coordinate release. The decoder uses this release to produce a correction for each video, changing cross-video rankings while preserving within-video ordering. On UCF-Crime and XD-Violence, the method reports minor improvements over the public-only model.

Critical strengths
  • Clear privacy target and compact release: The paper defines client-level privacy and limits the released private signal to a two-dimensional, clipped, noised vector. This makes the proposed collaboration mechanism concrete and its privacy scope relatively easy to assess.

  • Useful separation of within-video detection and cross-video calibration: The method applies a video-level correction that preserves each video’s internal frame ordering. This gives the method a clear objective beyond simply aggregating client models.

Critical weaknesses
  • The benefit of private collaboration is small and its consistency is unclear: The public-only detector already performs strongly, and the private contribution improves by less than 1 point AUC/AP. Because the paper reports one noisy release per result, it is unclear whether these gains are even consistent. The experiments therefore provide limited evidence that private collaboration adds a reliable practical benefit.

  • Performance comparison is uncontrolled: Table 1 compares methods with different backbones, public supervision, client partitions, and privacy settings. The scores therefore do not show that PARD-VAD outperforms prior methods under comparable conditions. Besides, the public-only model itself strongly outperforms each prior work. Since these comparisons are to other privacy-preserving methods, it is unfair that the non-private component forming the baseline for PARD-VAD is so strong.

Questions for authors
  1. Can you compare PARD-VAD with prior methods under a matched setup using the same backbone, public training data, client partition, and privacy level? This can clarify whether the proposed DP method offers an advantage over existing methods.

  2. How do the gains on top of the public-only detector vary across independent noise draws?

Reviewer_GgosOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces a framework for weakly supervised video anomaly detection.

It is a combination of existing modules such as a decoder for temporal modeling, a pooling mechanism for log-odds video correction.

They evaluate the proposed model on 2 public datasets, e.g., UCF-Crime, and XD-Violence.

They claim that the obtained improvements are from cross-video ordering.

Critical strengths
  • The evaluations show improved performance on two datasets.

  • The ablation studies provide some insights/contributions of some components.

  • The evaluation section is clear, and some interesting visualisations such as fig 2 and 3.

Critical weaknesses
  • This paper is quite hard to follow and understand. Starting from the abstract, it is unclear to reviewer what are the research problems and what motivates this work. The abstract is also written in a complex and hard to understand way, the dense terms (although mostly existing in the current literature such as Gaussian perturbation, DP, etc) and the design choice are unclear to follow.

  • In intro, although the authors provide some contexts, it is unclear what are the major differences and key novelty of this work. In addition, it would be better to have a motivative figure showing the comparisons or outlining the core innovations of this work.

  • The novelty of this work appears to be very limited. Transfer learning for weakly supervised task, via temporal modeling and public function basis have been explored for many years, and this work is more like an application of existing works to the VAD problem. Temporal response for private release plus the weighting of those public scores sounds strange and unclear to me. The paper also does not explain why this kind of setup advances compared to existing works. Regarding the privacy preserving representations, for example, TeD-SPAD, PRIME, etc., these methods address different perspectives of privacy. I think these works should be discussed and reviewed separately from methods that use differential privacy guarantees.

  • The privacy analysis is important for this paper; however the scope of guarantee of the claimed contributions need to be separated from the general claim of privacy preserving VAD systems. I am also questioning the relationship between reported deployment guarantee and actual experimental workflow. My concern is: if the claim / reported guarantee captures the privacy protection available under actual deployment in experimental protocols.

  • The review of closely related works appear to be very limited. Most importantly, it is unclear to reviewer how the proposed framework differs from existing works per group.

  • The method section is organised in a quite sparse way, and each section stands alone as it is, and it is unclear to reviewer (i) why choose these modules/designs, (ii) why these modules are arranged in this way, (iii) what challenges each module aims to solve, (iv) to what extent the proposed framework differs from existing works. It would be better to have a section connecting this work to existing foundations.

  • Evaluations appear to be very limited, and only two datasets are used for evaluation. In addition, the selected datasets do not seem like the best set for evaluating the claimed purposes. The authors should also explore recent new datasets for evaluation. The comparisons with SOTA also appears to be limited. Regarding the group of privacy approach in Table 1, it appears to be messy, and in the paper, the authors should discuss each different privacy method, and how they differ from the proposed ones. The improvements over public only detector seems quite small, such as 0.81% and 0.54%. In addition, it is not sure whether the improvements are stable across independent runs etc.

  • Some visualisations appear unclear to me, for example, what is fig 3c and 3d? those captions look strange to me.

Questions for authors

See the raised concerns for questions.

(i) Regarding soundness: my main concerns are empirical validations and the scope of the reported guarantee. The performance improvements over the matched public baseline are marginal/modest, each privacy-related result is based on a single realised release, and the paper does not show the stability of these improvements across independent runs (statistical significance).

Appendix B says that the representation of final XD-Violence uses ran-2. I think the authors should clarify the selection protocol and confirm that the test results were not used to select the method.

(ii) Regarding privacy, the claim is conditional on fixed public datasets. Secure aggregation is simulated, and the appendix also excludes adaptive selection.

I would appreciate the clarifications on whether the reported privacy budget covers the complete experimental workflow or only the final deployment.

Reviewer_Q4dKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses WSVAD when training videos are spread across clients that cannot share data. Clients don’t share a full model but can communicate with each other via noisy summaries of how a locally trained teacher differs from a public-only reference, allowing the method to align anomaly scores across videos while leaving the ranking within each video unchanged.

Contributions: Frames private knowledge transfer as sending differences in temporal responses, measured against a public reference and projected onto a public function basis. A decoder that turns the compact private release into one score correction per video. A client-level differential privacy guarantee, proved via sensitivity analysis and Rényi-DP accounting.

Evaluation:

Data: UCF-Crime (AUC) and XD-Violence (AP), CLIP features; 13 UCF clients and 6 XD clients; 15% of training videos treated as public. Comparison: Against anonymization-based and federated VAD baselines. Ablations: No private knowledge, alternative releases, decoder components, public detector.

Critical strengths

Strengths

  • The within-video vs. cross-video problem of WSVAD is an important issue to address in the field.
  • The figures are informative enough, clear to read and to connect to the text.
Critical weaknesses

Weaknesses

  • The writing is subpar for this type of work, often unclear and vague. It would be very beneficial to the paper if the cause-and-effect mechanisms, or the goal behind design choices, were made more explicit in a few key passages of the methodology (e.g., the One clipped release paragraph starting at L213).
  • Few references to previous works. There is a much more extensive literature on WSVAD, some of which addresses topics and contributions similar to or overlapping with this paper.
  • L161 starts to define the loss functions, but the formulation is too general and vague. The figure does not help to understand how these losses are applied, and it is not clear from the main paper whether there are hyperparameters that can be tuned/learned. This information is in Appendix B, but the main text could include a clearer formulation. Similar observations hold for other terms: sparsity and smoothness have been used by multiple methods (e.g., Sultani et al. 2018), and the weights applied to them can vary.
  • Some important things are never defined: at L161/191, what are the public replay and replay pool in the setting of this paper?
  • The architecture of the trained models is buried in the appendix, making it harder to understand the full framework from the main paper.
  • The public-only ablation in Table 2 shows that it is possible to achieve 88.39% and 83.49% on UCF and XDV (0.8% and 0.54% lower than the final model) while training only on 5% of the training data (the public pool). This suggests very low variance between the training and test sets. It would also suggest that the difference between private and public pools drawn from the same training set might not be informative enough to give a good estimate of the method's performance in real-world settings, where class-specific experts are less feasible.
  • Additionally, the ablations over the hyperparameters are few and incomplete. In those presented, the public-only detector, trained on 10% of the training set, outperforms the ablated versions of PARD-VAD trained on the entire training set. Also, a set of hyperparameters is fixed per dataset (L289–290 and Table 6), and most of them are not ablated. This further suggests an overfitting issue that cannot be ruled out given the presented ablations and results.
  • The paper frames the method as weakly supervised even though it trains class-specific models (L726 states there are 13 clients for UCF and 6 for XDV, matching the anomaly classes in the two datasets). Additional ablations on the number of clients would be beneficial, for example by clustering all video features into k clusters and assigning a client to each.
Questions for authors

See weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.