acceptodds
Under review as a conference paper at ICLR 2027

Learning to Forget Undesired Anomalies: Category Unlearning for Weakly Supervised Video Anomaly Detection

Abstract

Weakly supervised video anomaly detection (WSVAD) localizes anomalous events using only video-level labels. However, models trained under this paradigm are often deployed in various real-world scenarios, where certain categories previously treated as anomalies may no longer be considered anomalous. This issue is particularly challenging because WSVAD lacks frame-level category supervision, while many recent state-of-the-art methods couple a class-agnostic binary branch with a fine-grained classification branch. Naively masking a category output suppresses only its fine-grained prediction, leaving the shared binary anomaly evidence largely unaffected. To address this limitation, we propose Category-Conditioned Dual-Branch Modulation (C-DBM), a lightweight adapter that enables on-demand unlearning of specific anomaly categories for frozen WSVAD detectors. Given a set of retained categories, C-DBM employs category-conditioned cross-attention to extract category-specific temporal information and modulates both the fine-grained representations and the binary anomaly logits accordingly. After a one-time adapter training stage, different unlearning requests can be satisfied simply by updating the retained-category state, without any request-time optimization or access to the original training data. Experiments across four base-model–dataset combinations show that C-DBM supports both single- and multi-category unlearning and substantially reduces residual fine-grained responses for forgotten categories. Moreover, compared with other unlearning baselines, C-DBM more effectively suppresses forgotten-category predictions, while preserving both binary and fine-grained detection performance under the relabeled full-test protocol. Overall, C-DBM demonstrates effective anomaly-category unlearning while maintaining performance across both branches of modern WSVAD frameworks.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Weakly Supervised Video Anomaly Detection, Machine Unlearning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_LVNuOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies anomaly-category unlearning in weakly supervised VAD, where masking a category’s fine-grained output in dual-branch detectors leaves the shared binary anomaly score unchanged. It proposes C-DBM, a lightweight adapter on a frozen detector, conditioned on a binary retain vector. It is trained once, and each unlearning request is then served by changing the retain vector, without retraining. On UCF-Crime and XD-Violence, it suppresses forgotten-category predictions and is compared with nine adapted unlearning baselines under a protocol that relabels forgotten-category frames as normal.

Critical strengths

The paper identifies a concrete limitation of dual-branch WSVAD: masking a category’s fine-grained output leaves the shared binary anomaly evidence intact. It addresses this, including the difficult case where retained and forgotten anomalies co-occur in the same video. The design is practical, since a single lightweight adapter serves all unlearning requests without retraining. The evaluation is extensive, with nine unlearning baselines carefully adapted to the MIL setting and per-category results reported for transparency.

Critical weaknesses

The paper frames its contribution as unlearning, but C-DBM adapts outputs on a frozen base model and does not remove learned information. The authors acknowledge this, deferring stronger unlearning guarantees to future work, and report the remaining gap to Exact Retrain as a diagnostic. The method is better understood as controllable category suppression. Under that framing, the privacy-oriented motivation and the comparison with weight-editing unlearning methods should be tempered. The framing also invites comparison with definition-conditioned VAD methods that exclude categories at inference, such as LaGoVAD (ICLR 2026), which would help position the contribution. On the evaluation side, the XD-Violence binary results are mixed relative to several baselines. Clarifying the checkpoint-selection protocol and reporting variance across seeds would make the conclusions more convincing.

Questions for authors
  1. Given that the base model is unchanged and requests are reversible, would the authors consider framing the work as controllable category suppression? How would this affect the privacy motivation and the choice of unlearning baselines?
  2. Appendix C states that Table 2 compares C-DBM’s best checkpoint with the baselines’ fixed endpoints, but the selection criterion is not specified. Since no validation split is described and Exact Retrain is selected on the official test metric (Appendix A), was C-DBM’s checkpoint also selected on test data? Do the results hold under the same fixed-endpoint rule used for the baselines?
3 comments
AuthorsOpenReview ↗

Response to W1 and Q1

W1: Please clarify C-DBM's behavioral scope and its relationship to definition-conditioned VAD methods such as LaGoVAD.

Q1: How would a controllable category-suppression framing affect the motivation and choice of baselines?

We thank the reviewer for highlighting the connection to definition-conditioned VAD and asking us to clarify the scope of C-DBM. Our starting point is a base WSVAD model already trained with all available anomaly categories. C-DBM studies anomaly-category unlearning in this existing detector, aiming to suppress selected learned categories while preserving retained-anomaly detection. We freeze the base WSVAD model and train a single request-conditioned adapter under simulated category-unlearning requests. After this one-time adapter training, deployment requests change only the retain vector, without request-specific optimization.

The distinction from LaGoVAD lies in the detector training and adaptation setting. LaGoVAD learns to detect anomalies conditioned on user-specified language definitions and evaluates zero-shot generalization across datasets. Its formulation permits both expanding and narrowing anomaly definitions, so the connection extends beyond detecting previously unseen anomalies. Both works address changing anomaly definitions, while C-DBM studies how to equip an already-trained base WSVAD model with selective control over its existing anomaly categories.

AspectLaGoVADC-DBM
SettingLearn detection under varying anomaly definitionsAdapt an existing base WSVAD model trained with all available anomaly categories
User inputAnomaly definition expressed through class names or descriptionsRetain vector over the base model's existing anomaly categories
Learning strategyTrain a definition-conditioned detectorFreeze the trained base model and train one shared request-conditioned adapter
Main objectiveFollow the supplied definition and generalize across settingsSuppress selected learned categories while preserving retained-anomaly detection

This adaptation setting makes the existing detector's shared binary anomaly response central to our problem. The binary branch has already learned to respond to categories that may subsequently be forgotten. Masking their fine-grained logits leaves that binary response unchanged. CTR reads category-conditioned temporal states, while FSM and BEM modulate fine-grained semantic and shared binary responses according to the retain vector. The objective is to change both responses under the category-unlearning request, with forgotten-only suppression and retained-event detection evaluated separately. The base model need not have been trained to change its anomaly definition from one request to another.

This scope clarifies the motivation and baselines. We agree that controllable category suppression accurately describes C-DBM's behavioral scope. We will make this scope explicit when presenting anomaly-category unlearning and center the motivation on changing operational anomaly definitions. Since the base WSVAD model remains unchanged and requests are reversible, we will remove wording that implies irreversible information erasure or associated privacy guarantees.

The nine adapted unlearning baselines compare request-specific parameter updates or parameter editing with C-DBM's shared request-conditioned adapter. They compare alternative adaptation mechanisms under the same category-unlearning requests and request-dependent evaluation protocol. Hard Closure, logit calibration, soft masking, an MLP replacement for CTR, and uniform routing provide direct controls for the architecture's contribution. We will discuss LaGoVAD explicitly in Related Work and clarify the distinction between learning definition-conditioned detection and adapting an existing base WSVAD model.

AuthorsOpenReview ↗

Response to W2 and Q2

W2: Please clarify checkpoint selection and seed variability, especially for binary detection on XD-Violence.

Q2: Please clarify the role of test metrics in checkpoint selection and report results under the baselines' fixed-endpoint rule.

We thank the reviewer for asking us to clarify the reporting protocol and quantify variability across training seeds. We report C-DBM (+ binary supervision), the current formulation with explicit request-conditioned binary supervision. It regenerates binary video targets from the retained annotated categories and replaces the original routing loss with Request-Conditioned Snippet Supervision. We will report this as the final C-DBM formulation in the revised manuscript.

1. Fixed training endpoint. All C-DBM results reported here use the final epoch-20 checkpoint. The reported epoch is fixed by the training schedule and is not chosen using test AUC, AP, forgotten-category probability, or interval-specific metrics. Checkpoints at epochs 5, 10, and 15 are evaluated only to describe training dynamics and do not determine the reported endpoint. C-DBM therefore follows the fixed-endpoint reporting convention of the iterative unlearning baselines. SSD and MUpHT use their edited checkpoints, while Exact Retrain remains a separately identified reference using its native best-checkpoint rule.

The following trajectory uses seed 234 on XD-Violence. AP follows the relabeled full-test protocol, and Fine P. follows the main-table evaluation range. Metrics are percentages and macro-averaged over six single-category requests, except M recall, which averages the four requests with nonempty overlapping intervals. For each checkpoint and request, R90 selects a threshold achieving at least 90% recall on retained-only frames. F@R90 measures forgotten-only alarms, and M-Recall@R90 measures recall where forgotten and retained events overlap.

Base modelEpochBin AP ↑Fine AP ↑F@R90 ↓M-Recall@R90 ↑Fine P. ↓
VADCLIP577.6081.1159.97100.001.10
VADCLIP1076.6481.3312.2039.151.06
VADCLIP1577.5881.4311.1541.001.10
VADCLIP20 (final)77.6581.4510.7646.301.11
DSANet579.3778.1235.3173.180.27
DSANet1081.7178.7114.4646.200.24
DSANet1582.3778.9512.5747.640.23
DSANet20 (final)82.4479.0112.6948.970.23

Strong binary detection holds at the fixed endpoint. Epoch-20 Binary AP reaches 77.65% on VADCLIP and 82.44% on DSANet, exceeding the nine-baseline maxima of 77.33% and 80.97%, respectively. The reporting rule remains fixed across metrics, including DSANet F@R90, whose lowest value among these snapshots occurs at epoch 15.

The trajectory also shows why detection AP and binary selectivity should be examined separately. On VADCLIP, Binary AP changes from 77.60% at epoch 5 to 77.65% at epoch 20, while F@R90 falls from 59.97% to 10.76%. Substantial changes in forgotten-only alarms can therefore occur after aggregate detection AP changes little. M recall separately exposes the accompanying tradeoff in preserving overlapping events.

2. Three-seed evaluation on XD-Violence. We evaluated the same formulation with seeds 234, 235, and 236 on both XD-Violence base models, always using the final epoch-20 checkpoint. Each seed's result is first macro-averaged over six single-category requests. We report the mean ± sample standard deviation across the three seed-level results, not variation across categories. All metrics are percentages.

Base model / datasetBinary AP ↑Fine AP ↑F@R90 ↓Fine P. ↓
VADCLIP / XD-Violence77.61 ± 0.6081.41 ± 0.048.68 ± 1.801.110 ± 0.003
DSANet / XD-Violence82.51 ± 0.1079.16 ± 0.1811.21 ± 1.280.231 ± 0.001

Detection AP and Fine P. vary modestly across these training runs, while F@R90 shows greater variability. The mean F@R90 remains low on both base models, supporting consistent forgotten-only suppression across the evaluated seeds under the fixed-endpoint protocol. We will state the epoch-20 reporting rule explicitly in Table 2 and Appendix C and report these three-seed statistics alongside the XD-Violence results.

Reviewer_LVNuOpenReview ↗

Post Rebuttal

Thank you for the detailed rebuttal. I have reviewed the additional results and comments and will keep my score accordingly. I encourage the authors to incorporate the new details into the final manuscript.

Reviewer_HJJJOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates category-level unlearning in weakly supervised video anomaly detection. It introduces C-DBM, a lightweight adapter that jointly modifies fine-grained category predictions and binary anomaly scores without retraining the frozen backbone. Experiments on UCF-Crime and XD-Violence demonstrate effective category suppression with largely preserved detection performance.

Critical strengths

The paper introduces a meaningful and relatively unexplored problem of category unlearning in weakly supervised anomaly detection.

The lightweight, request-conditioned design enables efficient unlearning, supported by comparisons with multiple baselines.

Critical weaknesses

The method primarily suppresses category-conditioned outputs rather than removing learned information, making its relationship to genuine machine unlearning unclear.

The evaluation provides limited evidence that forgotten information cannot be recovered from intermediate representations or through alternative probes.

Questions for authors

Can the authors evaluate whether forgotten categories remain recoverable from frozen or adapted features using a separate probe?

How closely does C-DBM match exact retraining in terms of both forgotten-category behavior and retained performance?

2 comments
AuthorsOpenReview ↗

Response to W1 and Q2

W1: Please clarify how category-conditioned suppression relates to machine unlearning.

Q2: How closely does C-DBM match Exact Retrain in forgotten-category behavior and retained-anomaly detection?

We thank the reviewer for asking us to clarify this distinction and strengthen the comparison with Exact Retrain. Prediction suppression controls category responses, while machine unlearning aims to remove designated training-data influence, using retraining without those data as a reference. C-DBM implements request-conditioned category suppression through an adapter on a frozen base WSVAD model. Changing the retain vector can restore the original responses, so we distinguish this reversible control from training-data removal and associated privacy guarantees.

We replaced the original routing loss with Request-Conditioned Snippet Supervision and regenerated binary targets from retained video labels. These revised C-DBM (+ binary supervision) results differ from the original submission, and we will adopt this objective as the final C-DBM formulation.

We compare this variant with all nine adapted unlearning baselines, Origin, Hard Closure, and Exact Retrain on XD-Violence. Results use seed 234 and are percentages. AP follows the relabeled full-test protocol, while Top-1 and Fine P. use anomalous frames from videos containing the forgotten category. F-only and R-only contain only forgotten and only retained events, respectively, while M contains both simultaneously. F@R90 and M recall use thresholds selected for each model and request to achieve at least 90% R-only recall. Metrics macro-average six requests, except M recall, which averages the four nonempty overlap requests. The new variant uses its epoch-20 endpoint, baselines use final or edited checkpoints, and Exact Retrain uses archived best checkpoints from its native procedure.

Retrain Gap is the equal-weight mean absolute difference from Exact Retrain in Bin AP, Fine AP, Fine P., and F@R90. We take absolute differences after macro-averaging over requests, not before. Top-1 and M recall are excluded.

VADCLIP on XD-Violence

MethodBin AP ↑Fine AP ↑Top-1 ↓Fine P. ↓F@R90 ↓M-Recall@R90 ↑Retrain Gap ↓
Origin66.9372.7563.9758.1582.72100.0028.67
Hard Closure66.9380.690.000.0082.72100.0012.25
FT76.3473.158.177.9252.2397.866.04
NegGrad+73.3277.920.000.003.165.7012.27
SSD71.5363.4021.8024.9466.2897.8617.44
SCRUB75.5480.2540.5036.6249.31100.0010.91
SalUn74.1075.663.2411.8153.0078.707.13
Label Flipping76.7675.340.000.045.074.3211.57
Bad Teacher77.3380.240.013.076.266.2810.58
LUR76.8177.160.000.0021.4060.797.03
MUpHT54.2061.027.279.4983.7892.4622.88
C-DBM (+ binary supervision)77.6581.450.001.1110.7646.308.89
Exact Retrain77.4281.060.000.2044.8094.910.00

DSANet on XD-Violence

MethodBin AP ↑Fine AP ↑Top-1 ↓Fine P. ↓F@R90 ↓M-Recall@R90 ↑Retrain Gap ↓
Origin74.0373.0660.8248.1682.79100.0023.64
Hard Closure74.0377.890.000.0082.79100.0011.47
FT75.8673.200.000.5547.9997.412.54
NegGrad+69.7069.910.000.0123.047.6810.10
SSD74.7071.5112.5514.2059.6591.129.58
SCRUB80.9780.303.639.3549.31100.004.22
SalUn77.8275.591.5111.0880.7693.6712.28
Label Flipping77.7575.990.000.108.1819.0410.26
Bad Teacher79.5680.240.093.876.6729.4211.89
LUR78.1270.320.000.0015.6951.969.73
MUpHT65.2465.385.709.7088.0286.6019.45
C-DBM (+ binary supervision)82.4479.010.000.2312.6948.979.46
Exact Retrain80.6676.210.000.4745.7194.540.00

Detection performance and agreement with Exact Retrain differ. The revised model achieves the highest Binary AP on both base models, but its Retrain Gaps are 8.89% and 9.46%, with FT closer to Exact Retrain under this metric. These Gaps mainly reflect lower F-only alarm rates, while M recall remains below Exact Retrain at 46.30% versus 94.91% on VADCLIP and 48.97% versus 94.54% on DSANet. Higher AP and low forgotten-category responses therefore support effective category suppression, not equivalence to Exact Retrain or removal of training influence.

AuthorsOpenReview ↗

Response to W2 and Q1

W2: The evaluation does not establish whether forgotten-category information remains recoverable from intermediate representations.

Q1: Can independent probes recover forgotten categories from frozen or adapted representations?

Response: We thank the reviewer for suggesting independent probes to examine category recoverability beyond the detector's outputs. All results below use C-DBM (+ binary supervision), which includes explicit request-conditioned binary supervision absent from the original submission. We will adopt this variant as the final C-DBM formulation and update the revised manuscript accordingly. The C-DBM checkpoint used for feature extraction and detector evaluation is fixed at epoch 20, the final training epoch.

We trained independent linear and MLP probes on XD-Violence using VADCLIP and DSANet, covering all six single-category unlearning requests. The probes read three feature interfaces:

  • CLIP snippet features: the 512-dimensional inputs to the base WSVAD model.
  • Frozen base-model features: the 512-dimensional shared temporal features produced by Origin.
  • CTR states before modulation: the learned category-conditioned temporal states, averaged over all seven slots to obtain a 128-dimensional snippet feature.

The seven slots comprise six anomaly categories and Normal. Averaging all slots does not require identifying or selecting retained-category slots. This CTR interface precedes request-dependent modulation by FSM and BEM, so it is distinct from the final modulated category-wise semantic streams.

The linear probe has one linear layer. The MLP has a 128-dimensional hidden layer, ReLU, and an output layer. Each probe is trained for at most 30 epochs, with early stopping after five consecutive epochs without improvement in validation AP. For each probe, we select the checkpoint with the highest validation AP, independently of the fixed epoch-20 C-DBM checkpoint. The seed is 234, and recovery AP is macro-averaged over the six target categories.

The first three rows report Linear and MLP recovery AP. The final row reports Binary and Fine detection AP under the relabeled full-test protocol, macro-averaged over the six requests. All values are percentages, and each pair follows the AP order specified in the table.

Interface or modelDim.AP orderVADCLIP APDSANet AP
CLIP snippet features512Recovery: Linear, MLP(67.08, 65.24)(67.08, 65.24)
Frozen base-model features512Recovery: Linear, MLP(74.02, 76.60)(76.27, 75.65)
CTR all-slot mean before modulation128Recovery: Linear, MLP(73.93, 75.99)(75.24, 75.79)
Full C-DBM (+ binary supervision)N/ADetection: Binary, Fine(77.65, 81.45)(82.44, 79.01)

Target-category information remains decodable at all three tested interfaces. Both probe families recover target categories from the all-slot CTR mean, showing that recovery does not depend on retained-slot selection. CTR recovery remains in the same numerical range as recovery from frozen base-model features. These results establish residual category recoverability before request-dependent modulation.

The input-feature baseline helps interpret the source of recoverable information. Target categories are already decodable from CLIP features upstream of the base WSVAD model and the adapter. Recovery AP alone therefore cannot isolate how much category information was learned from the training videos designated for forgetting. The learned CTR states also retain information accessible to an independent readout.

The detector and independent probes characterize different properties of the same checkpoint. The final row evaluates the deployed detector against request-dependent relabeled ground truth, while the probe rows evaluate recovery of the original target categories. These AP values measure different prediction tasks and are not directly comparable. The paired evaluations distinguish the detector's category-level behavior from the information available to an independent classifier before modulation. We will report both explicitly and clarify that C-DBM targets conditional category-level behavior, without claiming protection against classifiers accessing these pre-modulation feature interfaces.

We thank the reviewer again for the careful evaluation and constructive feedback. We would be happy to address any further questions you may have. In light of our clarifications and the additional experimental evidence presented above, we hope you will consider a more favorable rating for our paper.

Reviewer_rRLjOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses a timely and practically relevant problem: selectively forgetting anomaly categories in weakly supervised video anomaly detection (WSVAD) while preserving the detection of retained anomalies. The authors propose C-DBM, a lightweight adapter combining category-conditioned temporal attention, fine-grained semantic modulation, and binary anomaly evidence routing. Experiments on UCF-Crime and XD-Violence demonstrate promising forgotten-category suppression without request-specific retraining.

Critical strengths
  • Novel and meaningful problem: The paper explores an important but underinvestigated challenge in WSVAD, where anomaly definitions can change across deployment environments.

  • Well-motivated architecture: Jointly addressing fine-grained category predictions and binary anomaly scores is conceptually appealing, particularly for mixed-category videos.

  • Efficient and flexible: A lightweight adapter (~0.58M parameters) supports multiple forgetting requests without additional optimization. Promising results: Strong suppression of forgotten-category predictions and encouraging generalization to unseen multi-category forgetting combinations.

Critical weaknesses
  • Unlearning vs. prediction suppression. Since the pretrained backbone remains frozen and forgotten categories can be restored by modifying the retain vector, the method appears closer to conditional prediction suppression than genuine machine unlearning. The authors should clarify this distinction and strengthen the evaluation against exact retraining.

  • Insufficient evidence for binary anomaly forgetting. The proposed BEM modifies the frozen binary logits using retained-category evidence, but does not explicitly remove forgotten-category contributions already embedded in those logits. Furthermore, forgotten-only videos receive no direct binary forgetting supervision. This raises concerns about whether forgotten anomalies remain detectable through the binary branch.

  • Limited improvements over simpler baselines. Although fine-grained forgetting is effective, binary AP is sometimes considerably lower than existing unlearning baselines. Multi-category improvements over Hard Closure are also relatively modest, raising questions about the necessity of the proposed architectural complexity.

  • limitations. Forgotten-category metrics on XD-Violence can include retained-category intervals from mixed-label videos, potentially obscuring actual temporal forgetting. Additionally, experiments use only one random seed, and some main-paper and supplementary results require clarification.

Questions for authors
  • Related to binary forgetting: How does BEM suppress forgotten-only anomalies when no explicit binary forgetting loss is applied? Please report binary scores and false-positive rates specifically on forgotten-category temporal intervals.

  • Related to architectural necessity: Could category-conditioned logit calibration, soft masking, or a lightweight MLP achieve similar results? A comparison would help establish the contribution of cross-attention and evidence routing.

  • Related to mixed-category preservation: Can the authors evaluate forgotten-only, retained-only, and overlapping anomaly intervals separately using XD-Violence's category-time annotations?

  • How does the method behave when most anomaly categories are forgotten, particularly when only one retained category remains?

5 comments
AuthorsOpenReview ↗

Response to W1

W1: Please clarify the distinction between prediction suppression and machine unlearning, and strengthen the comparison with Exact Retrain.

We thank the reviewer for asking us to clarify this distinction. Prediction suppression controls a model's responses to selected categories, while machine unlearning aims to remove the influence of designated training data from the resulting model. Under the retraining-based formulation, successful unlearning is assessed against a model trained without those data, so low forgotten-category scores alone do not establish that their training influence has been removed. C-DBM implements request-conditioned category suppression across both branches of an existing WSVAD detector. The base WSVAD model remains frozen, and restoring the all-retained vector returns its original outputs, so we explicitly acknowledge that C-DBM preserves the underlying learned information and does not establish training-data removal or associated privacy guarantees. Its contribution is to suppress forgotten-category fine-grained responses and binary alarms while preserving retained-anomaly detection, using a single trained adapter that serves different category-unlearning requests without request-specific optimization. In response to the binary-forgetting concern, we added explicit request-conditioned binary supervision, denote the resulting variant as C-DBM (+ binary supervision), and will adopt it as the final C-DBM formulation in the revised manuscript. This supervision was absent from the original submission, and the following results are therefore distinguished from the original C-DBM results. We compare all nine adapted unlearning baselines, Hard Closure, and Exact Retrain on DSANet/XD-Violence, with AP evaluated under the relabeled full-test protocol and all results macro-averaged over six single-category requests, in percent. The updated variant uses the final epoch-20 checkpoint, the baselines use final or edited checkpoints, and Exact Retrain follows its native best-checkpoint rule.

MethodBinary AP ↑Fine AP ↑Forget Top-1 ↓Fine P. ↓
Origin74.0373.0660.8248.16
Origin + Hard Closure74.0377.890.000.00
FT75.8673.200.000.55
NegGrad+69.7069.910.000.01
SSD74.7071.5112.5514.20
SCRUB80.9780.303.639.35
SalUn77.8275.591.5111.08
Label Flipping77.7575.990.000.10
Bad Teacher79.5680.240.093.87
LUR78.1270.320.000.00
MUpHT65.2465.385.709.70
C-DBM (+ binary supervision)82.4479.010.000.23
Exact Retrain80.6676.210.000.47

Hard Closure eliminates forgotten-category fine-grained outputs but leaves Binary AP unchanged, whereas C-DBM (+ binary supervision) achieves the highest Binary AP and exceeds Exact Retrain by 1.78% in Binary AP and 2.80% in Fine AP, with zero Forget Top-1 and 0.23% Fine P. These results support effective category suppression and detection under the revised anomaly definition, and we will use Exact Retrain as a behavioral reference without interpreting higher AP or low forgotten-category responses as evidence of behavioral equivalence or information erasure.

AuthorsOpenReview ↗

Response to W2 and Q1

W2: Binary anomaly forgetting needs stronger evidence, especially because forgotten-only videos receive no direct binary supervision.

Q1: How does BEM suppress forgotten anomalies, and what binary scores and false-positive rates remain within forgotten-category temporal intervals?

We sincerely thank the reviewer for identifying the missing binary supervision and suggesting an interval-specific evaluation. The original objective excluded forgotten-only videos from the binary retention loss, leaving their binary anomaly responses without direct forgetting supervision. We replaced the original routing loss with Request-Conditioned Snippet Supervision and regenerated binary video targets according to each category-unlearning request. The original routing loss is no longer used in the revised objective. The replacement loss directly supervises BEM's modulated binary anomaly probabilities, suppressing forgotten-category evidence while preserving retained-category responses. Forgotten-only videos receive negative binary targets, while mixed-label videos remain positive whenever at least one annotated anomaly category is retained.

We denote the resulting variant as C-DBM (+ binary supervision) and will adopt this revised objective as the final C-DBM formulation, updating the method description and experimental results in the revised manuscript. The original submission did not use this snippet supervision, and the results below were obtained with the revised objective.

The binary video target follows the retained annotated categories: yibin(m)=1[∑c=1Cyi,cmc>0].y_i^{\mathrm{bin}}(\mathbf{m})=\mathbf{1}[\sum_{c=1}^{C}y_{i,c}m_c>0]. Here, yi,cy_{i,c} is the video-level annotation for anomaly category cc, and mc=1m_c=1 indicates that the category is retained. Forgotten-only videos therefore receive a negative binary target, while mixed-label videos remain positive whenever an annotated anomaly category is retained.

At the snippet level, the frozen base WSVAD model (Origin) provides category-confidence estimates relative to Normal, restricted to the video's annotated categories. We assign zero anomaly-probability targets to high-confidence forgotten-category evidence with low retained-category evidence, all valid snippets in forgotten-only videos, and confident background snippets. Retained-category evidence receives a confidence-weighted local Origin soft target over a five-snippet neighborhood, using only selected retained-category evidence. This supervision also covers snippets where forgotten-category and retained-category evidence coexist. For each request, the four supervision groups are normalized separately within the mini-batch: Lsnippet=∑g:Ng>01Ng∑i,twg,i,tDKL(Ber(qg,i,t)∥Ber(pi,tm)),Ng=∑i,twg,i,t.\mathcal{L}_{\mathrm{snippet}}=\sum_{g:N_g>0}\frac{1}{N_g}\sum_{i,t}w_{g,i,t}D_{\mathrm{KL}}\left(\mathrm{Ber}(q_{g,i,t})\Vert\mathrm{Ber}(p^{\mathbf{m}}_{i,t})\right),\qquad N_g=\sum_{i,t}w_{g,i,t}. Here, qg,i,tq_{g,i,t} is the target, pi,tmp^{\mathbf{m}}_{i,t} is the predicted binary anomaly probability, and wg,i,tw_{g,i,t} is the supervision weight. For a zero target, the KL term becomes −log⁡(1−pi,tm)-\log(1-p^{\mathbf{m}}_{i,t}), directly penalizing binary alarms. Separate normalization prevents the most numerous group from dominating solely through snippet counts. Targets and weights are detached and constructed from training-video labels and frozen Origin outputs, without human snippet-level training annotations or test ground truth. The loss trains the adapter through BEM's modulated binary output, keeping the base WSVAD model frozen and introducing no additional trainable branch.

We directly evaluate the shared binary anomaly response on forgotten-only intervals. Using XD-Violence's category-time annotations, F-only frames contain at least one forgotten anomaly category and no retained anomaly category, so they should not trigger alarms under the current request. The table reports Origin → C-DBM (+ binary supervision), with frame-wise means computed within each request and then macro-averaged over six requests. Mean probability and mean logit are computed independently.

Base model / datasetMean anomaly probability (%) ↓Mean binary logitF@R90 (%) ↓
VADCLIP / XD-Violence86.70 → 15.964.11 → -3.6782.72 → 10.76
DSANet / XD-Violence84.14 → 15.673.02 → -3.6382.79 → 12.69

For each model and request, we select a threshold that achieves at least 90% recall on retained-only frames and apply the same threshold to F-only frames. F@R90 is the alarm rate on F-only frames, interpreted as a false-positive rate under the revised anomaly definition. Both base models show substantial reductions in forgotten-only binary responses. The reduction in F@R90 is especially informative because the threshold is recalibrated for each model at R90, so a uniform downward score shift alone would not explain this gain.

AuthorsOpenReview ↗

Response to W3 and Q2

W3: Do gains in binary detection and multi-category unlearning justify the architecture's complexity?

Q2: Can simpler alternatives match C-DBM, and what do cross-attention and learned routing contribute?

We thank the reviewer for suggesting these architectural comparisons. We evaluate C-DBM (+ binary supervision), which adds explicit request-conditioned binary supervision not included in the original submission. We will adopt this variant as the final C-DBM formulation and update the manuscript accordingly. The four trainable controls share its snippet supervision, training budget, and checkpoint-selection rule.

  • Logit calibration reads Origin probabilities and the retain vector, affinely calibrates fine-grained logits by category and retain state, and linearly predicts binary Down, Keep, and Up weights, without hidden features.
  • Soft masking learns bounded negative offsets for forgotten fine-grained logits and attenuates binary probabilities using forgotten-category responses.
  • MLP replaces CTR cross-attention and feed-forward updates with a two-layer residual MLP on concatenated category queries and same-snippet shared memory.
  • Uniform routing removes the learned routing projection and category biases, and averages retained and forgotten category slots separately.

Hard Closure sets forgotten fine-grained logits to negative infinity at inference without modifying the binary branch.

1. Controlled architectural comparisons on XD-Violence. Metrics are percentages, macro-averaged over six single-category requests. AP uses the relabeled full-test protocol, while Top-1 and Fine P. follow the main-table scope. F@R90 measures forgotten-only alarms at a threshold giving at least 90% retained-only recall, selected separately for each model and request using full-test category-time annotations.

Base modelMethodBin AP ↑Fine AP ↑Top-1 ↓Fine P. ↓F@R90 ↓
VADCLIPOrigin66.9372.7563.9758.1582.72
VADCLIPHard Closure66.9380.690.000.0082.72
VADCLIPLogit calibration71.8873.6360.5754.5882.70
VADCLIPSoft masking76.0272.8763.4757.5982.03
VADCLIPMLP53.8581.140.001.1484.26
VADCLIPUniform routing55.9281.100.001.0383.48
VADCLIPC-DBM (+ binary supervision)77.6581.450.001.1110.76
DSANetOrigin74.0373.0660.8248.1682.79
DSANetHard Closure74.0377.890.000.0082.79
DSANetLogit calibration74.2673.4354.0043.4582.77
DSANetSoft masking79.0173.1360.0047.4281.85
DSANetMLP80.1178.120.000.2750.37
DSANetUniform routing72.2278.260.000.2969.43
DSANetC-DBM (+ binary supervision)82.4479.010.000.2312.69

Fine-grained suppression alone does not establish binary selectivity. Logit calibration and soft masking improve AP but leave F@R90 close to Origin. On VADCLIP, the MLP achieves 81.14% Fine AP, close to 81.45% for the full variant, yet F@R90 is 84.26% versus 10.76%. On DSANet, uniform routing increases F@R90 from 12.69% to 69.43%. Under shared snippet supervision, these results support CTR's category-conditioned temporal reading and BEM's learned routing for binary selectivity. Binary AP also exceeds the nine adapted unlearning baselines on both base models.

2. Multi-category unlearning with DSANet on XD-Violence. C-DBM below denotes C-DBM (+ binary supervision). A single trained adapter serves all requests by changing only the retain vector. AP follows the relabeled full-test protocol and is macro-averaged over all requests of each size.

Forgotten categoriesRequestsOrigin Bin APOrigin Fine APHard Closure Bin APHard Closure Fine APC-DBM Bin APC-DBM Fine APC-DBM Fine P./category ↓
1674.0373.0674.0377.8982.4479.010.23
21561.5160.7461.5170.2978.7972.230.19
32048.2847.7348.2861.8973.6264.610.21

Within each request size, Binary AP gains over Hard Closure are 8.41%, 17.28%, and 25.34%, and Fine AP gains are 1.12%, 1.94%, and 2.72%. Comparisons are within request size because the relabeled ground truth changes. The widening Binary AP advantage suggests a growing mismatch between unchanged binary responses and the revised anomaly definition. Fine P. remains between 0.19% and 0.23% per forgotten category. The architecture supports fine-grained suppression and request-conditioned adjustment of shared binary responses.

Comments on the paper

2 comments
AuthorsOpenReview ↗

A summary of our work, reviewers' comments, and our responses

Dear Area Chair,

Thank you for your time and consideration. We appreciate the reviewers' careful evaluations and constructive suggestions. Below, we summarize our contribution, the aspects recognized by the reviewers, and the revision addressing their main concerns.

A brief summary of our work

Background. An anomaly definition can change after a WSVAD detector has been deployed. A previously anomalous category may need to stop triggering alarms, while other anomalies must remain detectable. In dual-branch WSVAD, masking a category's fine-grained output leaves the shared binary anomaly response unchanged. Selective category control must therefore address both branches under weak video-level supervision.

Our work. Category-Conditioned Dual-Branch Modulation (C-DBM) equips an existing full-category WSVAD detector with a lightweight request-conditioned adapter. One trained adapter per base-model/dataset pair serves different category-unlearning requests by changing only the retain vector, without request-specific optimization or access to training data at deployment.

Our contributions. We formulate this post-training adaptation problem, coordinate fine-grained suppression with binary anomaly modulation, and evaluate single- and multi-category requests across VADCLIP and DSANet on UCF-Crime and XD-Violence. Our objective is to adapt an existing detector to revised anomaly definitions while preserving retained-anomaly detection.

Reviewers' acknowledgements of the contributions

  • A meaningful and underexplored problem. Reviewer rRLj described the task as a "Novel and meaningful problem" involving changing anomaly definitions across deployment environments. Reviewer HJJJ similarly recognized "a meaningful and relatively unexplored problem" in weakly supervised anomaly detection.
  • A well-motivated dual-branch design. Reviewer rRLj noted that "Jointly addressing fine-grained category predictions and binary anomaly scores is conceptually appealing," particularly for mixed-category videos.
  • An efficient and practical deployment mechanism. Reviewer HJJJ highlighted the "lightweight, request-conditioned design." Reviewer LVNu emphasized that "a single lightweight adapter serves all unlearning requests without retraining." Reviewer rRLj also recognized the design as "Efficient and flexible."
  • Extensive and transparent evaluation. Reviewer LVNu acknowledged "nine unlearning baselines carefully adapted to the MIL setting" and "per-category results reported for transparency."

Revision prompted by the reviews

Our methodological revision focuses on Binary Evidence Modulation (BEM) and its binary supervision. BEM now aggregates forgotten-category and retained-category evidence separately and predicts snippet-specific Down, Keep, and Up weights. We add Request-Conditioned Snippet Supervision and regenerate binary video targets from the retained annotated categories, directly supervising forgotten-only suppression and retained-event responses without adding a trainable branch.

We have adopted this improved formulation as the final C-DBM and rerun its experiments, ablations, and temporal analyses. All C-DBM results summarized here were obtained with this revised formulation at the fixed epoch-20 endpoint and replace the original submission's C-DBM results. The revision is described in Sections 3.4 and 3.5.

The updated single-category comparison appears in Table 3. Detection metrics use the request-dependent relabeled full test set and are macro-averaged over requests. UCF-Crime reports AUC, XD-Violence reports AP, and all values are percentages.

Base model / datasetBinary AUC/APFine AUC/AP
VADCLIP / UCF-Crime87.9486.27
DSANet / UCF-Crime88.7188.36
VADCLIP / XD-Violence77.6581.45
DSANet / XD-Violence82.4479.01

C-DBM achieves the highest Binary AUC/AP among the nine adapted unlearning baselines in all four settings, with zero forgotten-category Top-1. On DSANet/XD-Violence, Binary AP gains over Hard Closure increase from 8.41% for one forgotten category to 17.28% for two and 25.34% for three (Table 1), highlighting the value of adapting shared binary responses as more categories are excluded.

AuthorsOpenReview ↗

Continuation of the previous comment

Reviewers' comments and our responses

Reviewer rRLj

  1. W1: Unlearning versus prediction suppression. C-DBM implements request-conditioned category suppression across both branches, inspired by class unlearning. The frozen model retains learned information and requests remain reversible. We removed the privacy-oriented motivation and distinguish this deployment objective from removing designated training-data influence. Exact Retrain serves as a behavioral reference.

  2. W2 and Q1: Direct binary supervision and forgotten-interval responses. On XD-Violence, mean forgotten-only anomaly probabilities fall from 86.70% to 15.96% for VADCLIP and from 84.14% to 15.67% for DSANet. F@R90 falls from 82.72% to 10.76% and from 82.79% to 12.69%, respectively. F@R90 measures forgotten-only alarms at thresholds achieving at least 90% retained-only recall, demonstrating selectivity beyond a uniform score reduction (Appendix B.2).

  3. W3 and Q2: Architectural necessity. We compare logit calibration, soft masking, an MLP replacing CTR, and uniform evidence routing under the same added snippet supervision, training budget, and endpoint. Output calibration and soft masking leave F@R90 near the frozen model's level. On DSANet, replacing CTR with an MLP or learned routing with uniform aggregation raises F@R90 from 12.69% to 50.37% or 69.43%. These controls support architectural benefits under shared supervision (Table 4 and Appendix B.3).

  4. W4 and Q3: Mixed-category evaluation. XD-Violence category-time annotations separate forgotten-only, retained-only, and simultaneous overlapping intervals. Retained-only frames are excluded from forgotten-only evaluation even within mixed-label videos. Overlap recall exposes a real preservation tradeoff, with C-DBM outperforming several aggressive suppression baselines but remaining below Exact Retrain. We distinguish these diagnostics from relabeled full-test detection and video-selected residual-category responses (Section 4.1 and our response to W4 and Q3).

  5. Q4: Most categories forgotten. One adapter per base model covers all six requests retaining only one of six XD-Violence anomaly categories. Binary AP reaches 47.37% on VADCLIP and 48.96% on DSANet, the highest among the compared methods. F@R90 decreases from Hard Closure's 91.80% and 87.85% to 8.22% and 7.79%. This supports selective binary control when most categories are withdrawn (Table 2).

Reviewer HJJJ

  1. W1 and Q2: Relationship to Exact Retrain. C-DBM exceeds the reported Exact Retrain reference in Binary/Fine AP on both XD-Violence base models. Their temporal behavior differs: C-DBM suppresses more forgotten-only alarms, while Exact Retrain preserves more overlapping events. We report both outcomes and do not equate higher AP with retraining equivalence or information erasure (Appendix A and our response to W1 and Q2).

  2. W2 and Q1: Independent recovery probes. Linear and MLP probes examine CLIP inputs, frozen host features, and all-slot CTR features before modulation. Probe fitting and validation split the official training videos, with final evaluation on the official test set. Across six target categories, CTR recovery remains close to the frozen host. The probes establish recoverability at the tested pre-modulation interfaces, distinguishing response control from information removal (Appendix B.6).

Reviewer LVNu

  1. W1 and Q1: Scope, baselines, and LaGoVAD. We center the motivation on changing operational anomaly definitions. The unlearning baselines compare alternative ways to adapt an existing detector, without implying privacy guarantees. LaGoVAD learns language-conditioned detection, while C-DBM adapts an already-trained full-category detector. In a released-weight, shared-feature comparison, removing a category description does not reduce LaGoVAD's forgotten-only binary alarms (Appendix B.8).

  2. W2 and Q2: Checkpoint selection and seeds. Current C-DBM results use the fixed epoch-20 endpoint, not a test-selected checkpoint. Across seeds 234, 235, and 236, Binary AP is 77.61 ± 0.60% for VADCLIP and 82.51 ± 0.10% for DSANet, with F@R90 of 8.68 ± 1.80% and 11.21 ± 1.28%. Statistics summarize seed-level request macro-averages using the sample standard deviation. Exact Retrain's native best-checkpoint rule remains separately identified (Appendices B.4 and B.5).

The revision strengthens binary category control and makes its scope and limitations explicit. Overlapping-event preservation and high-recall rejection on UCF-Crime remain directions for further study (Appendix B.7).

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.