- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper targets the domain‑incremental multimodal DeepFake detection task. In real‑world scenarios, forgery techniques keep evolving over time. Conventionally static‑trained DeepFake detectors exhibit poor generalization against emerging forgery methods. Full retraining incurs high computational costs and requires storing all historical training data. The authors propose DEFEND, which builds upon frozen pre‑trained audio‑visual backbones and leverages LoRA low‑rank adapters for parameter‑efficient incremental updates to mitigate catastrophic forgetting. Its core innovation is a learnable key‑query routing mechanism that dynamically expands the adapter set: for each incoming new forgery domain, the algorithm automatically decides whether to reuse an existing adapter or instantiate a new one. The authors construct a chronological incremental data stream consisting of 23 forgery domains covering face‑swap, lip‑sync, and avatar‑based forgery techniques spanning 2017 to 2024, sourced from KoDF, DeepSpeak, and MAVOS‑DD. Comprehensive ablation and comparison experiments are conducted on two backbones, BRAVEn and CLIP+CLAP. With only 11 adapters for all 23 domains, DEFEND achieves an average AUC of 89.7% under automatic routing on BRAVEn, outperforming traditional continual‑learning baselines by a notable margin. On the CLIP+CLAP backbone, its performance is on par with the strongest baseline. The paper further conducts analyses on routing behaviour, domain grouping, memory overhead, cross‑language performance, and other aspects.
Critical strengths
The problem formulation aligns with real world forensic scenarios. Most existing DeepFake detectors follow a static training setup; the few existing incremental detection works mostly define domains at the dataset level. This paper treats each forgery generation method as an independent domain and constructs a chronological 23 domain data stream, which reflects the real world iterative evolution of forgery techniques. The method is reasonably designed. Built upon LoRA adapters, two routing strategies are proposed: the cosine similarity based SIM routing and the learnable key query routing DEFEND KQ. The mechanism dynamically decides whether to reuse an existing adapter or instantiate a new one, eliminating the need for a dedicated adapter per domain and substantially reducing parameter overhead. A margin loss is further designed to separate features of real and fake samples and assist the learning of the routing module. The experimental suite is comprehensive. • A rich set of baselines are compared: single adapter, one adapter per domain, Joint Training, EWC, DER++, DualPrompt, and KD+Margin Loss. • Extensive ablation studies are conducted: LoRA rank, adapter placement layers, reuse margin, routing network capacity, upper bound of stored embeddings for the router, and the presence/absence of the candidate adapter compatibility score . • In depth analyses are provided: automatic domain grouping behaviour, random routing control experiments, cross language performance, performance comparison between audio manipulated domains and real audio only domains, and routing accuracy variation against the number of seen domains. Thorough computational overhead analysis is provided. The paper reports detailed statistics for training/inference FLOPs, wall clock time, GPU memory consumption, and parameter counts, demonstrating practical deployment potential.
Critical weaknesses
- The paper motivates its work by noting that forgery techniques keep evolving over time and conventional statically trained DeepFake detectors generalize poorly against novel forgery types. However, all test experiments are derived from internal splits of four datasets: KoDF, DeepSpeak, MAVOS DD and DFDC. No fully independent external datasets are adopted for out of distribution (OOD) evaluation. Although temporally later domains are unseen forgery algorithms from the model’s perspective during incremental learning, all samples still originate from these four datasets.
- DEFEND assumes known domain boundaries during training, where each incremental step receives clean samples belonging to a single domain. In real world online scenarios, incoming samples are unlabeled mixed forgery data: one cannot know which forgery type a sample belongs to, and samples mixed with multiple forgery patterns may even exist.
- The key query router must store sample embeddings for every historical domain, resulting in linearly growing memory consumption with the number of domains. While ablations over the maximum storage budget B are presented, routing AUC drops noticeably when B=500. For real world production systems that continuously ingest new forgery techniques, memory usage will grow without bound.
- Conventional continual learning baselines (EWC, DER++, KD) fine tune the full head module, whereas DEFEND only updates LoRA adapters. This creates mismatched parameter update granularity. Although total parameter counts are fully reported in the paper, one critical baseline is missing.
- Building blocks including LoRA, adapter based Mixture of Experts, and key query matching are pre existing techniques, leading to limited theoretical novelty.
- Multiple formatting issues exist. For instance, the first formula lacks an equation number.
- Visualizations illustrating how routing accuracy and average AUC change as the number of domains increases are insufficient.
Questions for authors
- How would DEFEND perform on a completely unseen external dataset without access to any training samples from that new domain?
- Could the authors add small scale mixed domain simulation experiments, where each incremental step mixes samples from multiple forgery domains, to quantify the performance degradation of DEFEND?
- Given unbounded memory growth for continuously incoming new forgery techniques and performance drop at B=500, what mitigation strategies can be adopted for practical deployment?
- Why are regularizers such as EWC / DER++ not directly applied to LoRA adapters (instead of full head fine tuning) to remove unfair architectural confounding factors? Could this set of baselines be supplemented?
- The core building blocks are existing techniques. Could the authors further clarify the unique non trivial contributions of this work beyond existing components?
- Could the authors fix formatting errors such as the missing equation number for the first formula in the revision?
- Could additional visualizations be added to illustrate how routing accuracy and average AUC change as the number of domains increases?
3 comments
References
[1] Khalid et al. (2022), FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset [2] Feng et al. (2023), Self-Supervised Video Forensics by Audio-Visual Anomaly Detection [3] Haliassos et al. (2022) Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection
Dear Reviewer, as the discussion period is coming to an end, we would appreciate hearing from you. Thank you!
We thank the reviewer for their feedback and for highlighting our chronological manipulation benchmark, adaptive adapter reuse, and analysis of computational costs. We address the concerns below.
Q1: DEFEND targets incremental adaptation as labeled real/fake data arrive. To assess transfer before adaptation, we evaluated BRAVEn AV and V on FakeAVCeleb [1] without using its data for training, adapter allocation, or router fitting. We evaluate BRAVEn AV and V on FakeAVCeleb [1], completely unseen to DEFEND. We compare to AVAD [2] and RealForensics [3]; for both, we compute the AUC over the entire set of manipulations in the FakeAVCeleb dataset (not originally done in the AVAD paper). For RealForensics, we pretrain the model on DeepSpeak.
Q2: See response W1 to reviewer Wuew.
Q3: Like experience replay, DEFEND trades storage for performance. The BRAVEn router stores 4 KiB per clip embedding. Capping storage at 500 embeddings per domain reduces router memory from 217 to 39.7 MiB, compared with 62.5 MiB for the DER++ replay buffer. Routed AUC drops by 1.73 points (89.72% to 87.99%, Table 15). Smaller budgets reduce adapter reuse and routing accuracy. The cap applies per domain, so total router memory still grows with the number of domains.
Q4: Thank you for the suggestion. Below, each LoRA baseline updates a single adapter, matching one DEFEND adapter.
Q5: DEFEND’s main contribution is a rule for deciding when to add or reuse a LoRA adapter. For each new domain, DEFEND trains a candidate adapter and keeps it if its compatibility score exceeds that of the best existing adapter by a margin. Otherwise, it discards the candidate and updates the best existing adapter. Removing the candidate comparison leaves five adapters and lowers routed AUC (Table 23). During inference, the frozen DFDC-pretrained head produces the query embedding. The router scores the query against the learned adapter keys. A second pass applies the highest-scoring adapter to classify the clip. On our chronological benchmark of 23 audio and video manipulation domains from three datasets (2017-2024), DEFEND matches the routed AUC of one adapter per domain under the same routing procedure and storage budget (89.68% vs. 89.57%) while using roughly half as many adapters (11.3 vs. 23, Reviewer EnMs, Q2). The learned groups often follow manipulation families across datasets (Table 8). For held-out domains, training on other domains from the same group yields higher AUC than training on an equal number from other groups (89.55% vs. 85.50%, Table 9).
Q7 Figure 3 and Table 10 in Appendix E show how routing accuracy and routed AUC change as new domains are added. We welcome suggestions for other ways to visualize these results.
We fixed the formatting error and updated the PDF with the revised version covering Q3.