- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes Verus-Insight, an AI-generated video detector that works in three steps. It first plans what to inspect, then checks each target for generation artifacts, and finally combines the evidence into a real/fake verdict. The model is built on Qwen3-VL-8B and trained with warm-up tasks, supervised fine-tuning on human-annotated artifact data, and reinforcement learning. The authors also construct Verus-Bench, a benchmark with fine-grained human artifact annotations. Results show clear gains over the base model in both detection accuracy and artifact identification.
Critical strengths
- A clear three-stage decomposition of artifact detection. The paper splits detection into Inspection Planning, Evidence Verification and Evidence Aggregation, and evaluates each stage separately. This shows that existing detectors often reach the right verdict with the wrong evidence, which is a useful insight.
- A fine-grained benchmark. Verus-Bench gives each artifact a human description, a time interval and bounding boxes. This makes it possible to test whether detectors find the actual artifacts, not just the correct label.
Critical weaknesses
- The same model is the reward judge, the evaluator, and a baseline. Gemini 3.1 Pro computes the RL reward (Eq. 1, App. C.2), scores Plan and Finding F1 for all models (App. E). Verus-Insight is therefore trained against the judge that later grades it, so part of the Finding F1 gain may reflect fitting this judge rather than better artifact recovery.
- Spatiotemporal grounding is never evaluated. The method relies on explicit temporal and spatial grounding, and Verus-Bench provides frame-wise boxes. Yet all metrics are text-only: time and boxes are explicitly excluded from matching (App. E.2). The paper never shows that the predicted locations are correct.
Questions for authors
Please address the points raised in the Weaknesses section (I would be willing to raise my score if the authors provide evaluation with an independent judge and localization metrics for matched findings.).
2 comments
Response to W2: Spatiotemporal Grounding Evaluation
W2: Spatiotemporal grounding is never evaluated. The method relies on explicit temporal and spatial grounding, and Verus-Bench provides frame-wise boxes. Yet all metrics are text-only: time and boxes are explicitly excluded from matching (App. E.2). The paper never shows that the predicted locations are correct.
We agree that semantic matching alone does not establish whether the predicted artifacts are correctly localized. To address this, we conduct an additional spatiotemporal grounding evaluation on Verus-Bench (191 videos, 321 GT findings) and RewardData (483 videos, 182 GT findings), covering six models. We keep the original model predictions and one-to-one semantic matches fixed, and independently measure temporal and spatial localization accuracy.
Table: Spatiotemporal Grounding of Semantically Matched Findings
| Dataset | Model | Matched | Temporal Valid | tIoU ↑ | Spatial Valid | sIoU ↑ |
|---|---|---|---|---|---|---|
| Verus-Bench | Verus-Insight | 48 | 48/48 | 0.5036 | 9/48 | 0.4428 |
| GPT-5.5 | 104 | 7/104 | 0.4683 | 0 | NA | |
| Qwen3-VL-8B | 45 | 15/45 | 0.3146 | 0 | NA | |
| Skyra | 17 | 13/17 | 0.4213 | 9/17 | 0.3015 | |
| BusterX++ | 20 | 4/20 | 0.3917 | 0 | NA | |
| VideoVeritas | 24 | 2/24 | 0.3333 | 0 | NA | |
| RewardData | Verus-Insight | 68 | 67/68 | 0.7210 | 60/68 | 0.3348 |
| GPT-5.5 | 99 | 10/99 | 0.4827 | 0 | NA | |
| Qwen3-VL-8B | 33 | 5/33 | 0.4015 | 0 | NA | |
| Skyra | 47 | 37/47 | 0.5932 | 35/47 | 0.3678 | |
| BusterX++ | 24 | 0/24 | NA | 0 | NA | |
| VideoVeritas | 66 | 4/66 | 0.0675 | 0 | NA |
Temporal Valid and Spatial Valid indicate the number of semantically matched findings with evaluable temporal intervals and spatial boxes, respectively. tIoU is averaged over valid finding-level intervals; sIoU is averaged over aligned frame pairs (within 0.25 s), comprising 38/885 frames for Verus-Insight and 41/410 frames for Skyra on Verus-Bench/RewardData. Missing or unalignable locations are excluded from conditional IoU averages and reflected in the reported coverage.
- Temporal grounding. Verus-Insight provides evaluable temporal localization for 48/48 matched findings on Verus-Bench and 67/68 on RewardData, achieving mean tIoU scores of 0.5036 and 0.7210, respectively. This demonstrates substantial temporal localization coverage and measurable agreement with the annotated artifact intervals.
- Spatial grounding. Verus-Insight provides evaluable bounding-box localization for 9/48 matched findings on Verus-Bench and 60/68 on RewardData, achieving frame-level sIoU scores of 0.4428 and 0.3348, respectively. Notably, on RewardData, Verus-Insight achieves substantially higher spatial coverage than Skyra (88.2% vs. 74.5%), while maintaining comparable localization accuracy (0.3348 vs. 0.3678 sIoU). On Verus-Bench, Verus-Insight also achieves higher conditional sIoU than Skyra (0.4428 vs. 0.3015), although spatial coverage remains limited. These results demonstrate Verus-Insight's ability to provide spatially grounded artifact evidence with a favorable balance between localization coverage and accuracy.
For reproducibility, native localization fields are extracted deterministically. For baselines with unstructured responses, GPT-6.1 Sol extracts only explicitly stated locations from the original responses, without access to videos or GT annotations. All IoU values are then computed deterministically, without modifying the original semantic matches.
Response to W1: Evaluation with Independent Judges
Thank you for highlighting the potential evaluation bias arising from using Gemini 3.1 Pro for both RL reward computation and fine-grained evaluation. We agree that this overlap warrants independent verification.
To address this concern, we re-evaluated all six models using two independent semantic-matching judges: Gemini 4 Argon and GPT-6.1 Sol, neither of which was used for RL reward optimization. We kept the original model predictions, extracted findings, ground-truth annotations, and evaluation samples fixed, replacing only the semantic-matching judge. The evaluation covers 674 videos and 4,044 model–video predictions across Verus-Bench and RewardData.
Table 1: Independent-Judge Evaluation on Verus-Bench (F1, %)
Table 2: Independent-Judge Evaluation on RewardData (F1, %)
The results show consistent conclusions across all three judges. In particular, the Finding F1 ranking of all six models remains unchanged on both benchmarks. Verus-Insight consistently ranks second on Verus-Bench and first on RewardData, outperforming all evaluated specialized detectors under both independent judges. Its improvements over the Qwen3-VL-8B backbone also remain substantial, indicating that the gains are not specific to Gemini 3.1 Pro's matching decisions.
Absolute F1 values vary across judges, reflecting differences in semantic matching strictness. Importantly, this experiment isolates the evaluation judge: previously extracted model outputs remain fixed, and the original Gemini-based extraction procedure for applicable baselines is retained. We therefore interpret these results as independent validation of semantic-matching robustness, rather than a fully independent end-to-end evaluation pipeline.