acceptodds
Under review as a conference paper at ICLR 2027

Let Clues Unfold: A Fine-Grained Multi-Label Dataset, Benchmark, and Dynamic Clue Mining Method for Explainable Harmful Short Video Detection

Abstract

Short videos have become a major channel where harmful content reaches the public, and a single video often mixes several harm categories through multimodal cues. Existing harmful video detection techniques fall short in two ways: harmful-content taxonomies remain inconsistent across platforms and studies, and public datasets are small and coarsely labeled; multimodal fusion methods do not resolve the semantic gap and temporal misalignment across modalities, and fail to provide frame-specific causal reasoning for model predictions. To overcome these problems, we construct FineHM, a fine-grained multi-label harmful short video dataset, spanning 8 categories with harmful frames and bounding boxes, to provide frame- and region-level evidence for explainable multi-label classification. Built upon this dataset, we establish a standardized benchmark including fixed data splits, evaluation metrics, and comprehensive baseline results to facilitate future research. Moreover, we propose ExplainHarm, a dynamic clue mining explainable harmful short video detection method, to surface dynamic evidence from category features and video content via a prompt-chained clue-mining process, and to achieve cross-modal semantic and temporal alignment through dual-scale temporal modeling. Experiments and statistical analyses demonstrate the dataset FineHM and the model ExplainHarm's superiority, achieving an 11.3% accuracy gain over SOTA multi-label models and a 0.6% IoU improvement over SOTA open-vocabulary object detection models.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Explainable Harmful Short Video Detection, Harmful Video Detection, dataset, benchmark

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_jb4BOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces FineHM, a dataset of 9,979 short videos with multi-label harmful-content annotations and frame-level evidence boxes. ExplainHarm combines dynamic clue mining, temporal multimodal modeling, and clue-guided classification and grounding. It reports 78.5% classification accuracy, exceeding the strongest baseline in Table 3 by 11.3 percentage points, with component ablations supporting the design.

Critical strengths
  1. Useful dataset and task formulation. Connecting co-occurring harmful labels to localized visual evidence supports more informative evaluation and human inspection than video-level classification alone.

  2. Relevant component ablations. Removing dynamic clues reduces accuracy from 78.5% to 74.6%; removing local or global temporal modeling also lowers performance, providing initial evidence for their utility.

Critical weaknesses
  1. The test protocol is unclear. Split counts and source-level grouping for clips and augmented variants are not specified. Section 5.1 also describes inference-time clue-bank updates and retrieval from annotated samples without defining retrieval-pool restrictions, label access, or reset policies. These omissions prevent a clear interpretation of the reported generalization, although they do not establish leakage.

  2. Metrics and comparison conditions need clarification. Multi-label accuracy and grounding matching rules are undefined, and AR/mAP values are interchanged between Tables 3 and 4. Baseline adaptation and modality access are insufficiently documented. Additionally, box overlap measures localization agreement rather than whether the highlighted regions faithfully explain the classification. These issues weaken the empirical and explanatory claims.

Questions for authors
  1. What are the split counts and source-grouping rules? Is annotated retrieval restricted to training data, and how is the clue bank updated and reset? A frozen-training-bank comparison would help clarify the effect of test-time updates.

  2. Please define the classification and grounding metrics, reconcile AR/mAP across Tables 3/4/7, and specify baseline adaptation and modality access. Does Table 7 replace the clue module or compare standalone systems?

  3. What independent annotation-quality checks were performed? Is explanation faithfulness evaluated, or should the claim be limited to localization aligned with human annotations?

Reviewer_LxWROpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces FineHM, a fine-grained multi-label dataset for explainable harmful short-video detection, and proposes ExplainHarm, a multimodal framework designed to jointly perform video-level harmful content classification and spatiotemporal evidence grounding.

FineHM comprises 9,979 short videos across seven harmful categories and one normal category, with video-level labels, harmful frame annotations, and approximately 440,000 bounding boxes.

ExplainHarm consists of three main components: a prompt-chain-based dynamic clue mining module that constructs evolving category-specific semantic clues, a dual-scale temporal enhancement module for multimodal feature modeling, and a clue-guided classification module that connects semantic evidence with video representations for multi-label prediction and spatial localization.

Experiments on FineHM report 78.5% classification accuracy, 91.4% classification mAP, and 33.8% grounding mAP, outperforming the selected baselines.

The work aims to advance harmful video moderation from coarse classification toward more fine-grained, evidence-grounded predictions.

Critical strengths
  1. The paper addresses a meaningful problem by jointly considering multi-label harmful video classification and fine-grained spatiotemporal evidence localization.

  2. FineHM provides 9,979 videos and over 440,000 bounding-box annotations, offering useful supervision for studying explainable harmful content detection.

  3. The proposed dynamic clue mining approach offers an interesting way to combine category-level knowledge with video-specific semantic evidence.

  4. The framework integrates multimodal temporal features with semantic clues for both classification and evidence grounding, forming a reasonably coherent design.

  5. The experimental results show substantial improvements over the selected baselines, while the ablation studies provide initial support for the effectiveness of the proposed components.

Critical weaknesses
  1. The grounding baselines mainly consist of open-vocabulary object detectors, while comparisons with video grounding methods and equally supervised detection models are missing.

  2. The ablation studies do not distinguish the contributions of static category clues, video-specific clues, and dynamic clue-bank updates, making the core innovation difficult to assess.

  3. The clue bank is updated during both training and inference, but the paper does not clarify how test-time updates avoid information leakage or prediction-order dependence.

  4. The explainability evaluation relies mainly on bounding-box localization metrics, without verifying whether the identified regions actually influence classification decisions.

  5. The dual-scale temporal module is not compared with alternative temporal modeling approaches, and its robustness to cross-modal temporal misalignment remains unverified.

  6. The framework integrates multiple pretrained models and an LLM, but inference latency, computational overhead, and deployment efficiency are not reported.

Questions for authors

see weakness

Reviewer_ETnyOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces (1) FineHM, a multi-label harmful short-video dataset (9,979 clips, ~21h, ~440K bounding boxes) with frame- and region-level annotations, and (2) ExplainHarm, a three-module framework combining prompt-chain-based dynamic clue mining (Qwen2.5-7B + BLIP), dual-scale temporal enhancement, and clue-guided DETR for explainable classification. Reported results: +11.3 Acc over the best multi-label baseline, +0.6 IoU over the best open-vocabulary detector.

Critical strengths
  1. FineHM fills a real gap: first benchmark jointly offering multi-label taxonomy and spatio-temporal localization annotations (Table 2 comparison is convincing).
  2. Taxonomy construction is systematic — cross-comparison of academic taxonomies and platform guidelines (Table 5), with clear design principles.
  3. Method pipeline is clearly motivated and well-structured; ablations (Table 4) and LLM-backbone comparisons (Table 7) are reasonably thorough.
Critical weaknesses
  1. No inter-annotator agreement is reported for either video-level labels or bounding boxes. Several categories are inherently subjective (e.g., Political non-compliance, Abusive), so label reliability is unverifiable.
  2. Potential information leakage: the dynamic clue bank K is updated “during both training and inference” (Sec. 5.1). It is unclear whether K is frozen at test time; transductive updates could inflate results.
  3. Unclear baseline protocol: open-vocabulary detectors were presumably never fine-tuned on FineHM, while ExplainHarm was; VLM baselines scoring Pre = 0.24–0.3 look like strawmen. Zero-shot vs. fine-tuned settings are unspecified.
Questions for authors
  1. Report IAA (κ or human-mAP) for video-level and box-level annotations; how were disagreements resolved?
  2. Is K frozen during evaluation? Please re-run Table 3 with a fixed (train-only) K to rule out leakage.
  3. Specify per-baseline protocol (zero-shot vs. fine-tuned) and prompts used for the VLM baselines.
  4. Provide the data split, seeds (mean ± std over ≥3 runs), and significance tests for the +0.6 IoU claim.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.