acceptodds
Under review as a conference paper at ICLR 2027

Learning Reconstruction Errors as Spatiotemporal Evidence for AI-Generated Video Detection

Abstract

The proliferation of AI-generated videos has raised growing concerns about misinformation, fraud, and potential societal risks, creating an urgent need for detection. Reconstruction-error-based methods have shown strong empirical performance for AI-generated image detection by exploiting discrepancies between an input and its reconstructed version. Their extension to video detection, however, faces two challenges: (i) a fixed reconstructor is not optimized for the detection objective and may yield weak discriminative signals; and (ii) an additional raw-video feature stream is often introduced to model temporal dependencies, increasing architectural overhead. To address these challenges, we develop a detector that learns and exploits reconstruction errors as a unified source of spatiotemporal evidence. First, we jointly optimize a reconstructor and the downstream classifier to produce reconstruction errors that are more informative for detecting AI-generated videos. Second, we characterize temporal information using relative second-order temporal differences of the reconstruction errors. We further show that a 3D convolution over three consecutive errors can represent this relative second-order feature without explicitly constructing it, while simultaneously integrating spatial evidence. Extensive experiments on over 30 existing AI-video generators demonstrate competitive performance against existing training-based detectors.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI-Generated Video Detection, Reconstruction Error, Adaptive Reconstruction Errors, Relative Second-order Temporal Difference

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_86qWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes an AI-generated video detector that jointly optimizes a VAE reconstructor and a classifier to learn discriminative reconstruction errors, and uses 3D convolutions to extract spatiotemporal evidence motivated by relative second-order temporal differences (RSTD). Experiments across multiple video-generation benchmarks report strong detection performance, with ablations examining adaptive reconstruction and temporal modeling.

Critical strengths

S1 - The problem is important, and the motivation for combining reconstruction residuals with second-order features is logical.

S2 - The comparative experiments are comprehensive and effectively demonstrate the feasibility of the proposed approach.

Critical weaknesses

W1 - The writing is not good. The discussions of related work and preliminaries are lengthy. The main text lacks a conclusion section (It should not be placed in the appendix), and the analysis in main text refers to a table in the appendix (line 376).

W2 - The proposed method is limited in novelty. VAE reconstruction and 3D convolutions are common techniques, and the explanation that 3D convolutions can implicitly represent RSTD is not fully convincing.

Questions for authors

Q1 - In Section 5.3, introducing RSTD reduces performance, which conflicts with the claim that 3D convolutions implicitly learn RSTD. The authors should provide additional experiments or theoretical analysis to clarify this relationship. ( The proof in Appendix A only shows that adding RSTD does not enlarge the local 3D convolution's function class, not that it uses RSTD patterns for classification. )

Q2 - No robustness experiments (e.g., video compression) are found. Since reconstruction residuals may involve noise features, robustness to post-processing is important.

  • Minor Question

  • Q3 - Some baselines (e.g., D3) appear to perform differently from their original papers. Could the authors explain this (for example, whether the evaluation datasets differ)?

Reviewer_TrLnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper use the reconstruction error detection method for AI-generated videos (like AEROBLADE,DIRE,FIRE etc in images and DIVID etc in videos) and change it in 2 ways, first the author change the auto-encoder weights from frozen to learnable in the pipeline based on the detector loss (Adaptive reconstruction errors) and add and temporal error RSTD, based on the idea D3 authors introduce on the acceleration difference between real and fake videos (Implicit temporal signal). one more nice thing that they did is to unified the reconstruction error for spatial and temporal to one unified reconstruction error source.

To validate there method they use a 139k class balance video from Kinetics 400 and Vbench generation for training and 4 external bechmark.

They compare the results to 11 detectors from training based and training free with and without spatial cues.

The method achieves the best or near best AUC/AP on most generators by small but consistent gain.

Critical strengths
  1. Simple and unified design: the method replace the multi branch pipeline with a single stream. the pipeline is easy to implemant and are details thoroughly to reproduce.
  2. Reasonble and easy idea and also well motivated to use a LORA based adaptation to this task on the auto-encoder. the ablation show that it is the largest single contributor to perfomance from avg AUC of 92.9 to 97.27(FakeParts).
  3. Use of temporal information from the error themself is clean and show the importance of the authors of efficiency. 4.Broad evaluation with strong results
Critical weaknesses

Robustness to compression. How does the method perform when both real and generated videos are re-encoded at matched quality, e.g. H.264 at several CRF levels, or resizing? This would help show whether the high AUC reflects generation traces rather than differences in compression and capture conditions between the real sources (Kinetics, MSR-VTT, Panda-70M) and the generated videos. Temporal sampling. Frames are sampled at 2 fps (App. B), while D3, which motivates the second-order term, samples at 8 fps. Each three-frame temporal kernel therefore spans one second. How do you justify the second-order “acceleration” interpretation at this rate? Could you report results at higher sampling rates, e.g. 4 and 8 fps? Does the model use the RSTD? Since a temporal-kernel-3 Conv3D can learn any combination of three consecutive errors, do the learned temporal kernels resemble the RSTD stencil (1, −2, 1)? Can you provide evidence that the model actually exploits the second-order signal rather than other temporal or purely spatial cues? Efficiency. Reducing architectural and inference overhead is a stated motivation. Can you report parameter counts, FLOPs and inference latency for your method, including the Wan VAE encode and decode, compared with the main baselines? Variance. Can you report mean ± standard deviation over multiple training seeds, at least for the main benchmarks and the ablation? Several reported margins over the strongest baselines are only 1–2 AUC points.

Questions for authors

1.Robustness to compression. How does the method perform when both real and generated videos are re-encoded at matched quality, e.g. H.264 at several CRF levels, or resizing? This would help show whether the high AUC reflects generation traces rather than differences in compression and capture conditions between the real sources (Kinetics, MSR-VTT, Panda-70M) and the generated videos. 2.Temporal sampling. Frames are sampled at 2 fps (App. B), . How do you justify the second-order “acceleration” interpretation at this rate? Could you report results at higher sampling rates, e.g. 4 and 8 fps? 3. Does the model use the RSTD? Since a temporal-kernel-3 Conv3D can learn any combination of three consecutive errors, do the learned temporal kernels resemble the RSTD stencil (1, −2, 1)? Can you provide evidence that the model actually exploits the second-order signal rather than other temporal or purely spatial cues? 4. Efficiency. Reducing architectural and inference overhead is a stated motivation. Can you report parameter counts, FLOPs and inference latency for your method, including the Wan VAE encode and decode, compared with the main baselines? 5. Variance. Can you report mean standard deviation over multiple training seeds, at least for the main benchmarks and the ablation? Several reported margins over the strongest baselines are only 1–2 AUC points.

Reviewer_1hhvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a reconstruction-error-based framework for AI-generated video detection that jointly optimizes the reconstructor and classifier to enhance discriminative features. It introduces relative second-order temporal differences of reconstruction errors and employs 3D convolutions to efficiently capture spatial and temporal inconsistencies without requiring an additional raw-video feature stream. Extensive experiments across more than 30 AI-video generators demonstrate competitive detection performance compared with existing training-based methods.

Critical strengths
  1. The paper proposes a detector that learns and exploits reconstruction errors within a single stream to identify AI-generated videos.
  2. The authors design a joint optimization of a reconstructor and a downstream classifier. This ensures that the reconstruction errors are more informative and specifically tailored for detecting AI-generated content. In addition, the authors introduce a temporal detection signal based on second-order differences of reconstruction errors. It efficiently represents this feature using a 3D convolution over three consecutive errors, which integrates spatial evidence without the need for explicit construction.
  3. Empirical results show that the proposal typically achieves the highest detection performance compared to more than 10 baselines, even when relying only on reconstruction errors.
Critical weaknesses
  1. Insufficient Analysis of Experimental Results. The paper lacks an in-depth analysis of the experimental results. Although competitive performance is reported, the authors provide limited discussion of the underlying reasons for the observed improvements and the effectiveness of individual components.
  2. Lack of Training Cost Analysis. The paper does not report the computational and training costs of the proposed framework. Jointly fine-tuning the VAE and replacing the classifier with a ResNet-based architecture may introduce additional computational overhead. A detailed comparison of training cost and computational complexity with existing methods is necessary to evaluate the efficiency of the proposed approach.
  3. Insufficient Architectural Details. The paper lacks a sufficiently detailed description of the proposed architecture, making it difficult to fully understand and reproduce the method.
Questions for authors
  1. The proposed method achieves nearly perfect performance on all benchmarks. Do these benchmarks remain sufficiently challenging to meaningfully evaluate and distinguish the capabilities of different detection methods?
  2. Could the authors provide a detailed comparison of training parameters and computational complexity, including the number of trainable parameters, training time, and FLOPs, against existing methods?
  3. Could the authors provide a more comprehensive description of the proposed architecture, particularly the LoRA-based fine-tuning of the VAE and the implementation of the 3D convolution module, to improve the clarity and reproducibility of the method?

Comments on the paper

1 comment

Dear reviewers,

The discussion phase is nearing its end. If you haven't already, could you please read the author's rebuttal and respond?

Thank you & kind regards,

Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.