acceptodds
Withdrawn from ICLR 2027

Prompt-Decoupled AI-Generated Multimodal Video Detection and Attribution

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

As AI-generated videos become increasingly realistic, deepfake video detection is vital for safeguarding cybersecurity and video source attribution helps ensure regulatory accountability. Existing deepfake detection methods focus on audio and vision modalities (e.g., the mismatch between lip movements and audio) while largely ignoring the textual modality. Meanwhile, research on video attribution is limited, especially from a multimodal perspective. To address these gaps, we propose a Prompt-Decoupled MultiModal Forensics (PDMF) that combines vision and text modalities for concurrent AI-generated video detection and attribution. PDMF converts video frames into descriptive text subtitles, extracts visual and textual features through advanced encoders, and projects textual features into visual space. By applying the cross-modal residual subtraction, we filter out shared semantic context and successfully expose model-specific artifact features. These features are stored in an extensible reference database to convert video attributes into instance retrieval tasks. By using only 100 videos per class and 2-10 times less time than alternative methods, PDMF achieves a binary deepfake detection accuracy of 96.3% (outperforming SOTA by over 14.1%) and an attribution mAP of 91.0% (surpassing competitors by over 29.3%) on GenVidBench. Additionally, PDMF demonstrates robust generalization under few-shot conditions.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
AI-generated video attribution, AI-generated video detection, multimodal methods

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_1G6aOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PDMF, a multimodal framework for AI-generated video detection and source attribution. It combines visual features with text descriptions generated from video frames. The method uses cross-modal residual subtraction to reduce shared semantic information and retrieve similar videos from a reference database. Experiments on GenVidBench show promising results in both deepfake detection and source attribution, especially in few-shot settings.

Critical strengths
  1. The paper studies an important problem. AI-generated video detection and source attribution are important for digital forensics. Identifying the source of a generated video is also useful beyond simply detecting whether it is fake.
  2. The paper explores multimodal features through a simple retrieval-based framework. The method combines visual features with text descriptions generated from video frames to capture generator-specific characteristics. It further formulates source attribution as a retrieval task, allowing new samples to be added to the reference database without retraining the entire model. This provides a potentially practical approach to few-shot video attribution.
Critical weaknesses
  1. The novelty of the method is limited. The proposed framework combines pretrained visual and text encoders, temporal modeling, contrastive learning, cross-modal residual subtraction, and nearest-neighbor retrieval. Most of these components are based on existing techniques. The paper should clearly establish that the technical contribution distinguishes it from existing multimodal feature fusion and retrieval-based attribution approaches.
  2. The generalization claims and the experimental evidence are insufficient. The paper evaluates the method on GenVidBench under different few-shot settings, but using fewer samples from known generator categories does not necessarily demonstrate generalization to completely unseen generators. Moreover, the key assumption that cross-modal residual subtraction effectively isolates generator-specific artifacts is not sufficiently validated. The evaluation is also limited to a single benchmark, and the comparisons cannot demonstrate that the improvements result from the proposed modules rather than differences in feature extraction or experimental settings. These limitations make the claims of strong generalization and superior attribution performance less convincing.
Questions for authors
  1. Missing underline and bold highlights in Tables.
Reviewer_t7wPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PDMF detects AI-generated videos and attributes them to their source generator. It aggregates CLIP frame features into a video embedding with a Transformer, encodes a BLIP-generated caption with the CLIP text encoder, aligns the two with a bidirectional InfoNCE loss, and subtracts the visual embedding’s projection onto the text embedding to obtain a residual intended to remove shared semantics and expose generator-specific artifacts. Three cross-entropy heads (visual, text, residual) are trained on generator labels. At test time the encoder is frozen, residual features are stored in a reference database, and both attribution and real-versus-fake detection are done by cosine nearest neighbor. On GenVidBench (eight generators plus real) under few-shot settings, PDMF reports 87.8% Rank-1 and 91.0% mAP at 100-shot and 96.3% detection accuracy, outperforming several adapted baselines. Ablations cover the three branches, fusion variants, the temporal module, loss weights, temperature, and database size.

Critical strengths
  1. The retrieval formulation with a frozen encoder is practical, inference is cheap, and the few-shot evaluation (5 to 100 training videos per class) addresses a real constraint in forensics. The reported margins over the listed baselines in this regime are large. This is the reason I do not consider the paper without merit, but it does not outweigh the weaknesses below.
Critical weaknesses
  1. The paper’s two stated contributions are not tested. Semantic removal is a one-dimensional projection on embeddings that InfoNCE has already aligned, all heads are trained on generator labels, and the only supporting evidence is a roughly one-point gain over plain concatenation without variance. Scalability to unseen generators is the motivation for the retrieval paradigm, yet every generator appears in training, so this property is never evaluated. Since these are the claimed contributions, their absence drives my recommendation.
  2. The empirical comparison is not reliable. Baselines designed for large-scale training are run with 100 videos per class and a nine-class head, and their low real-class accuracy suggests class imbalance rather than a method gap. The gain from adding text may come from the contrastive regularizer rather than textual content, as the text branch alone is near chance and no control is provided. A single dataset, no cross-dataset or robustness evaluation, and an unclear test set size further limit what can be concluded.
Questions for authors
  1. What exactly is the test set for Tables 1 and 2 (number of videos per class, and whether it is disjoint from the videos used to fill the database)?
  2. Were DeMamba and UNITE fine-tuned from released checkpoints or trained from scratch, with what hyperparameters, and how was the real-versus-fake imbalance handled? Would training them on the full GenVidBench training split change Table 2?
  3. Did any experiment use the original generation prompts from GenVidBench, or were all text inputs BLIP captions?
  4. Please report variance over seeds for Tables 3 and 4; the one-point gap between fusion variants is the only evidence for the residual mechanism.
Reviewer_8eFZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes PDMF for detecting generated videos and attributing them to source generators. It combines CLIP visual features, generated text descriptions, temporal modeling, and cross-modal residual subtraction. The learned features support nearest-neighbor retrieval against a small reference database. On GenVidBench, the paper reports 96.3% binary detection accuracy and 91.0% attribution mAP using 100 training videos per class, with further few-shot and component studies.

Critical strengths
  1. The retrieval formulation is practical.
  2. The 100-shot results are strong within the reported setting.
  3. The branch ablation shows a useful gain over the visual-only model.
Critical weaknesses
  1. Unseen-generator adaptation is claimed but not demonstrated. Section 5.2 trains with examples from each evaluated class. Few-shot performance on those classes does not establish that a frozen model can attribute a generator excluded from feature learning.
  2. The data protocol is underspecified. The paper does not clearly state how training, reference, and query videos are separated, or whether source videos and generation prompts are disjoint. This matters because the method claims to remove semantic shortcuts.
  3. The implemented method is ambiguous. Equation 8 jointly optimizes the branches, and Section 4.3 retrieves using the fused feature. Appendix B instead describes independent branch optimization and averaging three branches for the final decision. These procedures can produce different results.
  4. Semantic removal is asserted more strongly than it is validated. Equation 7 removes one text-aligned direction from an embedding; this does not establish that all semantic information is removed. Residual fusion improves mAP by only 0.7 points over attention, without repeated-run uncertainty.
  5. Some headline comparisons overstate the evidence. The abstract claims a detection gain above 14.1 points over SOTA, but Table 2 includes PiD at 90.2%, making the gap 6.1 points. The 421.26-second preparation and 2.33-second attribution times also lack a clear accounting of caption generation and query encoding.
Questions for authors
  1. Can you exclude complete generator families during training, freeze the model, and then evaluate attribution after registering a few examples from those families?
  2. Please give the exact training, reference, and query counts, and explain how overlapping source content, prompts, and duplicate clips are prevented.
  3. Which optimization and inference procedure produced the main tables? Please reconcile Equation 8 and Section 4.3 with Appendix B.
  4. Do the gains persist on content-disjoint data and with shuffled or generic captions? Please report uncertainty for residual versus attention fusion.
  5. Please correct the strongest-baseline comparison and report end-to-end query time, including captioning, feature extraction, and retrieval, on specified hardware.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.