acceptodds
Withdrawn from ICLR 2027

ROAD: Unified Referring Operation Anomaly Detection via Cross-scene Dual Memory

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Existing operation video anomaly detectors are bound to specific scenes and require retraining when the viewpoint, objects, or procedure changes. To tackle this limitation, we formulate the task of referring operation anomaly detection, where a detector scores a test video against a reference video of the valid operation, adapting to new scenarios solely by swapping the reference video. We release a dataset of 569 continuous one-take recordings from four scenarios, with annotated reference–test pairs and frame-level labels covering four process anomalies: wrong order, wrong placement, omitted step, and wrong component. We further propose a cross-scene dual-memory framework that separates target-specific progression in a reference-anchored temporal memory from cross-scene normality in a learnable prototype memory, fused by gated residual reconstruction. It reaches 91.63% mean AUROC across the three leave-one-scenario-out settings and 88.80% on a real-world packaging scenario, exceeding the previous strongest baselines by over 19.20 points. Code will be public after acceptance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video Anomaly Detection, Memory Model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_s7R6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Referring Operation Anomaly Detection, where a normal reference video is used to detect frame-level procedural anomalies in a query video without retraining the detector for each target scenario. It presents a dataset containing 569 videos and 335,850 frames from four operation scenarios, covering wrong order, wrong placement, omitted step, and wrong component anomalies. The paper further proposes CSDM, which combines a Reference Temporal Memory, a Shared Prototype Memory, and aligned reconstruction-based anomaly scoring. Experiments evaluate the method under three leave-one-scenario-out settings and an additional packaging scenario.

Critical strengths
  1. The reference-conditioned formulation is an interesting idea. Using a normal demonstration to specify the expected operation allows the same detector to handle different procedures without target-specific retraining. This provides a practical direction for operation anomaly detection when abnormal training videos are difficult to collect.
  2. The dataset provides a useful setting for studying procedural anomalies. It contains continuous multi-step recordings from four operation scenarios and covers four types of errors: wrong order, wrong placement, omitted step, and wrong component. These annotations support the study of temporal and procedural errors that cannot always be identified from appearance alone.
Critical weaknesses
  1. Limited effectiveness of SPM. In Table 3, the full model improves the mean LOSO AUROC by only 0.25 points over RTM-only (91.63% vs. 91.38%) and performs slightly worse in two of the three settings. Multiple-run results or further analysis would help clarify SPM’s contribution.
  2. Insufficient loss ablation. The paper lacks ablation studies on the three loss terms in Eq. (4). Evaluating their individual contributions would better justify the training objective.
  3. Limited cross-scene generalization evaluation. The evaluation covers only four fixed-viewpoint scenarios, including three controlled tabletop settings. Experiments under more diverse viewpoints, lighting conditions, and complex industrial environments would strengthen the generalization claims.
Questions for authors

Pleas see weaknesses.

Reviewer_dpQhOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper formulates referring operation anomaly detection, where a test video is scored against a reference video of the valid operation, so that adapting to a new scenario requires only swapping the reference video without retraining. The contributions are threefold. First, a new task and protocol with frozen weights and reference-guided adaptation. Second, a new multi-scenario dataset of 569 one-take videos and 335850 frames with frame-level labels for four reference-dependent process anomalies. Third, a cross-scene dual memory framework combining a non-parametric reference-anchored temporal memory with a learnable shared prototype memory. The method is validated on three leave-one-scenario-out transfers and one external transfer to a real industrial line, reaching 91.63% mean AUROC and outperforming the strongest baselines by over 19 points.

Critical strengths
  1. A new and well-motivated task formulation with a rigorous evaluation protocol. Process anomalies are defined by deviation from the reference procedure rather than by unusual appearance, and the reference video encodes exactly this procedure. The protocol allows adapting to a new scenario by swapping the reference video while keeping weights frozen, which is a strong practical property for deployment. The evaluation is careful, with three LOSO transfers, one external transfer to a real industrial line, ten predefined reference–query groups, and no target-specific gradient updates at test time (lines 736–737). This task and protocol are the part of the paper most likely to benefit the community in the long term.
  2. The dataset is carefully designed to remove shortcuts. All four anomaly patterns depend on the reference procedure and cannot be recognized from a single frame, which removes appearance shortcuts. In the block assembly scenario, different placement orders can lead to identical final arrangements, so a detector cannot pass by only checking the final state (lines 106–107). The frame-level labels mark the whole period during which the observable state stays inconsistent, not just the anomalous action itself (lines 174–179). This makes the reported gains trustworthy and the dataset reusable for future work on this task.
Critical weaknesses
  1. The core methodological contribution is questionable, since the framework largely reduces to retrieving reference states and comparing them after alignment, and the necessity of the learnable SPM component is not well supported. Three pieces of ablation evidence point in the same direction. In the LOSO setting, the full model is better than RTM-only only on target B (93.34 vs 92.36) and is slightly lower on targets C and A (Table 3a). On the external target D, RTM-only reaches 88.13 while the full model reaches 88.80, so the gain from SPM is only 0.67 (Table 6a). Meanwhile, SPM-only performs poorly on all targets, which suggests that SPM itself carries little temporal information. These results seem to indicate that the performance is almost entirely driven by RTM, and RTM in essence is the reference video itself. The value of the learnable SPM component is therefore put in doubt. The overall framework functions close to a retrieval-augmented state consistency comparator, and there is a gap between the methodological novelty actually demonstrated and the dual memory architecture suggested by the paper.
  2. The paper contains an internal inconsistency, and a core claim lacks empirical support. First, the text and the table contradict each other. The text of Appendix D.3 states “RTM-only is left unresolved because a matched D result was not present in the verified archive; it should be filled only after the corresponding evaluation is confirmed” (lines 899–901), yet Table 6(a) on the same page already reports RTM-only = 88.13. The credibility of this number needs clarification. Second, the introduction motivates the work by stating that existing datasets “cannot adequately measure generalization across different viewpoints, layouts, and manipulated objects” (lines 047–049), but all scenarios in this paper use a fixed viewpoint in a single take (Section 2.1), and viewpoint and illumination changes are deferred to future work (lines 481–483). This sidesteps the very problem raised in the introduction.
Questions for authors

Q1. Please clarify the contradiction between the “unresolved” statement in Appendix D.3 (lines 899–901) and the value 88.13 already reported in Table 6(a). Q2. The paper emphasizes retraining-free deployment, but no inference cost is reported. How long does it take to build the RTM for one reference video, and what is the end-to-end latency for one query video? In practice, how should users select a good reference video? Q3. Could you provide a multi-reference RTM baseline using the more available reference videos per scenario instead of using one reference videos? If it matches or exceeds RTM plus SPM, how would you justify the learnable SPM component?

Reviewer_oRn1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces referring operation anomaly detection, a benchmark of four operation scenarios, and a reconstruction framework combining reference-anchored temporal memory with shared prototypes. The model adapts to unseen scenarios by replacing the reference video without updating its parameters.

Critical strengths
  1. Reference-conditioned detection addresses a relevant practical problem in recognizing deviations from a specified multi-step operation.
  2. The dataset contains continuous real recordings, four anomaly patterns, and an industrial packaging scenario, providing a useful setting for evaluating transfer.
  3. The reported results substantially outperform the evaluated baselines, and the memory, retrieval, scoring, and reference-group analyses provide useful diagnostic information.
Critical weaknesses
  1. The benefit of the proposed dual-memory design is insufficiently established. In Table 3, RTM-only exceeds the full model on two of three targets, and adding SPM improves mean LOSO AUROC by only approximately 0.25 points. Paired uncertainty estimates or repeated runs are needed to assess this contribution.
  2. Important implementation details are missing. Equation 1 does not explain how the query's corresponding slot U(i)U(i) is determined at inference, while Appendix C introduces a temporal-position head and loss absent from the main formulation. The backbone and the adaptation of TEAM and OTAM into anomaly detectors are also insufficiently specified.
  3. Generalization evidence remains limited to four scenarios recorded from fixed viewpoints. Performance is not separated by anomaly type, making it difficult to establish reliable detection of ordering and omitted-step errors. Annotation consistency is also not quantitatively validated.
  4. The external-target results contain a direct reporting contradiction. Table 6 reports an RTM-only AUROC of 88.13, whereas Appendix D.3 states that this result is unresolved because a matched evaluation was absent from the verified archive.
Questions for authors
  1. How is U(i)U(i) computed during inference, and how does the temporal-position head handle differences in execution speed and omitted steps? Can the backbone and baseline implementations be fully specified?
  2. Can paired per-group comparisons and repeated runs establish the benefit of SPM over RTM-only, together with results separated by anomaly type?
  3. Which RTM-only result on D has been verified, and can the contradiction between Table 6 and Appendix D.3 be resolved?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.