acceptodds
Under review as a conference paper at ICLR 2027

Leveraging Redundancy in Representations for Robust Synthetic Video Detection

Abstract

Synthetic-video detectors are ranked on benchmarks whose default tier is raw or lightly compressed footage. However, when a video is streamed by a viewer, it is usually re-encoded by the distribution channel, often at a very different qual- ity. Across ten prior state-of-the-art detectors, we observe that the performance and ranking changes significantly based on the quality of the encoded video. For example, on the quality-controlled SynthForensics benchmark, the existing state- of-the-art detector falls from 93.2 at pristine quality to 44.6 %AUC at medium compression, which is indistinguishible from chance. By analyzing spatial fre- quency sub-bands, we find that the bands driving detector decisions shift with compression. Relying on specific bands–either by design or as a consequence of training–can therefore leave a detector vulnerable to compression. To reduce this dependence, we randomly suppress frequency bands during training so that no single band is indispensable and combine features from multiple network depths to provide alternative sources of evidence. We also match compression conditions between real and synthetic training samples, preventing compression itself from serving as a shortcut for identifying synthetic videos. Trained on a single generator, our detector holds 90.0 %AUC at CRF40 on the same benchmark, moving from 99.7 on the default tier. Beyond compression, we evaluate robustness to adversar- ial attacks, which remains understudied in synthetic-video detection. Applying band suppression at inference mitigates most of the accuracy loss caused by a query-limited black-box attack, without impacting pristine accuracy

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Synthetic video detection, synthforensics, video forensics

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_mK3MOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the robustness of AI-generated video detectors under distribution-time re-encoding. The authors argue that detector rankings established on benchmarks' default (high-quality) tiers have limited deployment relevance: real distribution channels re-encode videos to CRF-equivalents of 23–41, and across this range prior state-of-the-art detectors can fall to chance level, with rankings shifting substantially under compression. Through an energy-preserving frequency sub-band perturbation analysis, the authors show that the active sub-bands driving detector decisions shift with compression, providing a mechanistic explanation for this failure. They then propose three training-time interventions: band dropout (randomly suppressing input frequency bands so no single band becomes indispensable), a multi-depth readout (a learned mixture over class tokens from all transformer blocks, providing fault tolerance across depths), and compression-matched training pairs (preventing compression itself from serving as a label shortcut). Trained on a single generator (Allegro), the resulting detector holds around 90% AUC at CRF40 on SynthForensics and AIGVDBench, outperforming all baselines, and exhibits additional robustness to query-only black-box attacks.

Critical strengths
  1. The paper elevates deployment-time re-encoding from a peripheral stress test to a first-class evaluation axis, anchoring the quality ladder to measured platform distributions (Weibo ≈ CRF23 to Facebook ≈ CRF41) — the resulting evidence that default-tier rankings barely predict deployment behavior (Spearman +0.24; prior SOTA falling to chance at CRF40) is a valuable contribution to the AIGV detection community.
  2. The diagnostic tool and method design are solid: the energy-preserving sub-band perturbation cleanly isolates the evidence a detector actually uses, revealing that active bands shift under compression, and the two interventions it motivates (band dropout, multi-depth readout) are each credibly isolated by ablations and mechanism analysis (e.g., early-block AUC lifted 0.507 → 0.739).
  3. The experimental execution is rigorous and the ablations are clean: evaluation uses a fixed video set per tier so cross-CRF deltas reflect quality alone, checkpoint selection never touches the test benchmarks, and the four-arm design ablation isolates each intervention on the same compression-matched corpus. I also appreciate the calibrated scope of the adversarial claims — the defense is evaluated against a matched-cost randomized control, and the authors explicitly report the white-box and surrogate-transfer failure cases rather than overclaiming robustness.
Critical weaknesses
  1. The entire evaluation (and training) uses a single codec recipe (libx264, medium, yuv420p, native resolution, single encode). Deployment pipelines involve H.265/VP9/AV1 with different transform kernels, plus rescaling and platform filters — none of which are tested. The central claim is thus about robustness to x264 rate-control strength, not yet to the heterogeneous re-encoding the paper motivates in Fig. 1. Even a small cross-codec validation (e.g., x265/AV1 at matched bitrates) would substantially strengthen the deployment claim.
  2. (DeMamba) The video-level baseline roster is thin: only Waverep-ens and AIGVDet are evaluated, while representative temporal-modeling detectors such as DeMamba (cited in related work, with publicly available checkpoints — e.g., evaluated in AIGVDBench itself) are absent. Since temporal aggregation could plausibly offer intrinsic redundancy under compression, the claim that existing detectors collapse at CRF40 currently rests on only two video-level baselines.
  3. (Real Data Lies) The paper does not discuss Real Data Lies (ICML 2026), which identifies a closely related "quality shortcut" — detectors learning to separate classes by intrinsic quality differences between real and synthetic corpora. The connection is direct: App. A.3 shows the benchmark's source tier itself couples label with encoding quality (reals ≈ lossless vs. synthetics ≈ CRF22, a 27–215× rate gap), so a discussion of how the two shortcut axes (intrinsic corpus quality vs. delivery-time compression) relate would strengthen the paper's positioning.

[1] Haoxing Chen, Yan Hong, Zizheng Huang, Zhuoer Xu, Zhangxuan Gu, Yaohui Li, Jun Lan, Huijia Zhu, Jianfu Zhang, Weiqiang Wang, and Huaxiong Li. DeMamba: AI-generated video detection on million-scale GenVideo benchmark. Science China Information Sciences, 69(6):162103, 2026.

[2] Ziyuan Fang, Tianyi Wei, Guanjie Wang, Weiming Zhang, Nenghai Yu, and Wenbo Zhou. Real Data Lies: Unveiling and Closing the Quality Shortcut in Generalizable AI-Generated Video Detection. ICML 2026.

Questions for authors
  1. Please clarify the provenance of the AIGVDBench CRF 23 tier. The HuggingFace release appears to contain only a single version of each video, with uniform ffmpeg re-encoding metadata (Lavf58.76.100), and it is unclear whether this copy corresponds to "source quality" or CRF 23 as used in the paper (we did not check the Quark Drive mirror). Which release did the authors evaluate on, and if the CRF 23 tier was constructed by the authors, the "shipped vs. added rungs" distinction in Fig. 4 should be corrected accordingly.
Reviewer_Pj3vOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses AIGV detection under heavy compression and re-encoding. It combines Fourier-band dropout, learned aggregation of CLS tokens from all 12 DINOv3 ViT-B/16 blocks, and compression-controlled training pairs. The backbone is fine-tuned end to end, and frame-level logits are averaged for video classification. Trained on Allegro as the sole synthetic generator, the method achieves 90.0% and 90.1% video-level AUC at CRF40 on SynthForensics–DFD and AIGVDBench, respectively.

Critical strengths
  1. The compression-based analysis is practically valuable. Evaluating generators absent from training provides evidence of cross-generator transfer under compression, beyond an in-distribution robustness test.
  2. The method effectively integrates established practices, including a strong backbone, multi-depth feature aggregation, and matched training data, achieving strong absolute performance and robustness under heavy compression. The design is simple, coherent, and supported by useful ablations.
Critical weaknesses
  1. The methodological contribution is primarily empirical analysis and the integration of existing practices. These include frequency-domain random masking, multi-depth feature aggregation, and training-data alignment. In Table 2, the model without either proposed component already achieves 85.7% AUC at CRF40 on SF–DFD, substantially exceeding the strongest released baseline at 66.6% in Table 1. Most of the improvement over published baselines may therefore come from the backbone, native-resolution processing, and compression-matched training data, rather than the proposed redundancy mechanism.
  2. The baseline comparisons are not sufficiently controlled, weakening the evidence for the core contribution. Table 1 compares released models trained on different datasets. Even in this comparison, the proposed method trails Waverep-ens on AIGVDBench in default-quality AUC (91.6% vs. 99.6%) and CRF40 balanced accuracy (76.4% vs. 81.7%). The retraining baselines in Table 3 use substantially weaker training conditions, 88,640 pairs, 224-pixel inputs, and 4 frames per video. In contrast, the proposed method uses 253,824 pairs, native-resolution crops of 480–720 pixels, and t=8, and compression-matched training. These differences confound the comparison and prevent isolation of the method’s contribution.
  3. Generalization to unseen encoding pipelines is not evaluated. Section 3.1 argues that training on compressed videos does not scale to the many encoders and processing pipelines encountered in deployment. Yet both training and evaluation use H.264. No experiments evaluate alternative codecs, downscaling, or actual platform uploads. The results therefore establish robustness within the tested H.264 quality ladder, leaving the broader deployment-generalization argument unvalidated.
  4. Several experimental results are provisional, inconsistent, or insufficiently specified. (i) Table 5 explicitly marks Waverep-ens’s per-generator results as provisional pending rescoring, indicating that these comparisons require further verification. (ii) Section 4.3 reports CRF40 effective sub-band counts of both 10.1 and 8.04 for p = 0.5, and both 7.0 and 7.67 for p = 0, without explaining whether these values correspond to different experimental settings. (iii) Appendix E.3 acknowledges substantial checkpoint-dependent variation for the DINOv2 configuration, with CRF40 AUC ranging from 88.3 to 95.0. The experimental settings, random seeds, and checkpoint-selection protocols require clarification.
Questions for authors
  1. Please retrain Waverep with DINOv2 and DINOv3 backbones, along with the other baselines, on the same 253,824 compression-matched Allegro training pairs at native resolution. How much of the performance gain is attributable to the training data alone?
  2. Please evaluate codecs unseen during training, such as H.265/HEVC using x265 and AV1, as well as downscaling followed by re-encoding and actual platform upload-and-download pipelines.
Reviewer_5ZxpOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper focus of how synthetic/Ai Generated video detectors behave under new encoding that those videos do when uploaded in the different platform (e.g. social media). The autore argue those delivered videos re-encode on different CRF's from 23 to 41 while the detectors benchmarks contain raw or lightly compress videos.

The author diagons that across ten prior detectors performance on default quality poorly predict perfomance after those compression(93.2 to 44.6 AUC at CRF 40).

To solve this gap the author suggest to train a new detector that will be rubost to this factor by create a "Band Dropout". during training one of 16 polar fourier tiles per frame is randomly zeroed with prob of 0.5. In addition they use Multi depth readout. They also did "compression matched training pairs" to prevent the network to do a shortcut learning (compress->real,not compressed ->fake)

The author validate the method on SynthForensics and AIGVDBench, where the baseline are the models trained the old way with the checkpint provided. on SynthForensics the method got ~90 AUC while the baseline model drop to ~ 45-60 AUC .

Ablation is done and show that both the band dropout and multy depth readout help (band dropout is stronger in gain)

Critical strengths

1.under estimated and under studied problem. most of the paper in the topic try to maximize the benchmark and dont look on a real problematic problem on this research area, the videos are very diverse, the generators are moving fast, the training data and benchmark are not well distributed in term of compression (and frames per second also...) so it is very difficult to understand which method is really working and which one do shortcut learning. 2 analysis that explain the failure and solve it. the steerable pyramid attribution identified which sub band a detector actually used . this gives a mechanistic explanation tor the collaps and motivated the method 3.simple intervantion with large gain. the method is simple and solve the problem with no complex pipelines

Critical weaknesses
  1. The novelty is limited Aggregating CLS tokens from all ViT blocks is essentially RINE (Koutlis 2024), here with a simpler softmax weighting. Compression augmentation and content-matched or “laundered” training pairs follow prior work (Wang et al., 2020; Corvi et al., 2025). 2.the method contributon is confunded with the change in training data: in table 1 the author show results compare to baselines from checkpoint that was given on training data that they were trained on. so we have two factors , the method of band dropout and multi depth readout and also a new training data that have been augmented to this mission - so it’s unclear whether it wins because of the new method or simply because it saw compressed videos during training. 3.The manuscript appears unfinished, and some results are not final. Table 5 marks the Waverep-ens per-generator numbers as “provisional pending a re-score on the full pool”. App. A.3 ends mid-sentence (“double-compression traces remain a”). There is a broken citation (“?Kang et al.”). App. B.1 mentions a preprocessing bug in baseline evaluation that affected results by up to +7.7 AUC and was later “regenerated”.

4.frame-level design: The detector makes per frame decisions that are averaged over the clip, so it uses no temporal information even though it is presented as a video detector. Temporal cues are a natural complement here: a temporal signal may be affected by compression differently from per-frame spectral evidence. Several recent works exploit temporal structure for AI-generated video detection, e.g., second-order temporal features (D3, Zheng et al., ICCV 2025) and spatiotemporal likelihoods (STALL, Ben Hayun et al., CVPR 2026) It would strengthen the paper to discuss these and, ideally, test whether temporal evidence is more or less robust to re-encoding than the frame-level cues studied here.

Questions for authors
  1. Isolating the method from the training data. Can you retrain the strongest baselines on the same compression-augmented, quality-matched Allegro corpus and report their AUC across the CRF ladder? This would show how much of the CRF-40 gain comes from band dropout and multi-depth readout, and how much from training on compressed data. Relatedly, your own ablation (Table 2) reaches 85.7 AUC at CRF 40 without either component. How do you interpret the size of this number relative to the baselines?
  2. Relation to RINE. What do you see as the essential difference between your multi-depth readout and RINE’s aggregation of CLS tokens from all blocks? Have you compared the two directly in your setup, e.g., replacing your readout with RINE’s importance estimator?
  3. Finalized results. Can you provide the final (non-provisional) Waverep-ens per-generator numbers for Table 5, and confirm that all baseline numbers in the paper were computed after the preprocessing fix described in App. B.1?
  4. Temporal evidence. Since the detector is frame-level, have you tested whether temporal cues degrade under re-encoding differently from the per-frame cues studied here? Could your training recipe (compression-matched pairs, band dropout) be applied to a video-native or temporal detector, and would you expect it to help there?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.