acceptodds
Under review as a conference paper at ICLR 2027

Training-Free Evidence-driven Refinement for Unsupervised Talking Head Forgery Detection

Abstract

Supervised talking head forgery detection faces severe generalization challenges due to the continuous evolution of generators. By reducing reliance on generator-specific forgery patterns, unsupervised detectors offer an alternative for handling unseen generators. Alongside efforts to build stronger detectors, we investigate whether the anomaly score ranking of the existing detector can be improved without modifying its parameters. In particular, for score-based unsupervised detectors, the limited discriminative ability on hard cases is often reflected in unreliable anomaly ordering, leaving room for further refinement. Motivated by this observation, we propose a Training-Free Evidence-driven Refinement (TFER) framework to improve anomaly score rankings through selective refinement. TFER first partitions samples into retained and routed subsets using lightweight score-based routing. It then revisits only the routed subset by organizing localized visual evidence and performing semantic analysis to refine its relative ordering while preserving the original score multiset. Extensive experiments demonstrate consistent improvements over the base detector across multiple datasets and perturbation settings. These findings demonstrate the effectiveness of selective evidence-driven refinement in improving unsupervised talking head forgery detection without retraining the base detector.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Talking head forgery detection, Unsupervised detection, Training-free refinement, Evidence-driven refinement

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_onRVOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TFER, a training-free evidence-driven refinement framework to improve the anomaly score ranking of existing unsupervised talking head forgery detectors, without modifying the base detector’s parameters. Extensive experiments on three benchmarks (THB, AVLips, FKAV) and under four types of perturbations demonstrate consistent performance improvements over the base AVH-Align detector.

Critical strengths
  • As a training-free plug-and-play module, TFER can be integrated with existing detectors without retraining or redesign, which lowers the barrier to practical adoption. In addition, the textual evidence generated by the MLLM provides human-understandable explanations for detection decisions, addressing the black-box limitation of many deep forgery detectors.
  • The motivation of this paper is clear and the authors chose a straightforward but effective method to achieve the goal.
Critical weaknesses
  • The paper validates TFER only on variants of a single base detector (AVH-Align), and excludes AVAD due to its poor base performance. A core implicit claim of the work is that TFER can universally improve existing score-based detectors, but this claim is not sufficiently supported. It remains unclear whether the framework can consistently bring gains to other unsupervised audio-visual detectors, or even supervised detectors. Testing on only one detector family weakens the broader impact of the method.
  • The routing threshold estimated on the AVLips validation set is directly applied to THB and FKAV for cross-dataset evaluation. Given that different datasets have distinct anomaly score distributions and real/fake separability, a fixed threshold may lead to suboptimal routing ratios (e.g., too many or too few routed samples) on unseen datasets. The paper does not report routing ratios across datasets, nor does it compare with alternative strategies (e.g., percentile-based thresholds) to verify the robustness of the Youden-threshold routing across data distribution shifts.
  • The evidence mining and ranking stages rely heavily on manually designed text prototypes, Qwen instructions, and reranker anchors, all of which are tailored to specific facial forgery artifacts (over-smoothed skin, smeared mouths, unstable teeth, etc.). If new generators produce novel forgery patterns not covered by these prompts, the refinement performance may degrade significantly. The paper does not evaluate robustness against unseen forgery types, and the GPT-4-based prompt engineering process introduces subjectivity that may affect reproducibility.
  • The refinement stage takes 5.67 seconds per routed sample on an A100 GPU, which is orders of magnitude slower than the base detector (0.02s per sample). While selective routing reduces the overall cost, the computational overhead is still prohibitive for large-scale screening or real-time applications. The paper only briefly notes suitability for offline auditing, but provides no discussion of efficiency optimization directions (e.g., lightweight MLLM backbones, early stopping) or the performance-efficiency trade-off curve.
  • Main results are reported on merged test sets, but THB contains multiple diffusion-based generators and FKAV covers multiple manipulation types. The paper does not provide per-generator or per-manipulation performance breakdowns, so it is unknown whether TFER brings consistent improvements across all forgery methods, or whether gains are concentrated on specific types. This limits the assessment of fine-grained generalization.
Questions for authors
  • What is the proportion of routed samples on each test set (THB, AVLips, FKAV)? Please analyze how routing ratio correlates with performance gain, and compare the Youden threshold with a fixed-percentile threshold (e.g., top 30% highest-scoring samples) to justify the routing strategy.
  • Have the authors tried directly using the reranker’s rank scores as new detection scores, instead of performing slot-preserving reordering? Please report the performance of direct score replacement and compare it with SPR, to clarify the necessity and advantage of the slot-preserving design.
  • For perturbation experiments, the corruption intensity is fixed. How does TFER perform under stronger or weaker corruption levels? A sensitivity analysis on perturbation intensity would better support the robustness claim.
Reviewer_pCvROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies anomaly-score ranking refinement for talking-head forgery detection without updating the base detector. TFER routes a high-score subset using a validation-derived threshold, mines localized temporal evidence with CLIP, obtains evidence descriptions from Qwen, and uses a text reranker to reassign the subset’s original score slots. Experiments report AP/AUC improvements for two AVH-Align checkpoints across three datasets and corruptions on THB.

Critical strengths
  1. Refining the rankings of a frozen detector is a practically relevant problem. The improvements in Table 1 cover both the official and retrained checkpoints.
  2. The method explicitly limits where refinement can affect the ranking and states its unchanged decisions at the routing threshold, collection-dependent scores, and large-model inference costs. These disclosures help establish its scope.
Critical weaknesses

1. Insufficient evidence for the refinement mechanism.

Table 4’s w/o Routing is a useful control, but the local ranking contribution remains unclear. In Table 2, reversing the same routed scores would analytically yield THB/AVLips AUCs of 63.5/68.4, versus TFER’s 55.3/67.1. This motivates validation-fixed local ranking controls, without establishing reversal’s generalization or refuting the AP gains. A budget-matched direct-Qwen comparison also needs fully specified prompts and scoring.

2. The SPR ablation does not isolate score-multiset preservation.

Table 4’s w/o SPR drop may reflect mixing semantic and original scores on incompatible scales. With fixed within-U ordering and tie groups, any strictly monotonic mapping preserving C/U ordering gives identical AP/AUC. The ablation therefore does not distinguish preserving cross-subset ordering from preserving the exact original score multiset.

Questions for authors
  1. How does TFER compare in AP/AUC with validation-fixed local score reversal and a budget-matched direct-Qwen baseline with specified prompts and scoring?
  2. How does w/o SPR map scores, and can the ablation distinguish preserving C/U ordering from preserving the exact score multiset?
Reviewer_hNtwOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies ranking improvement for unsupervised detectors in speaker-avatar forgery detection and proposes TFER, a training-free, evidence-driven refinement framework. The authors argue that anomaly-score-based unsupervised detectors have limited discriminability on hard samples, reflected in unreliable anomaly-score rankings. Rather than redesigning or retraining the detector, TFER uses additional evidence to refine the relative ranking of a subset of samples without modifying the detector's parameters.

The method builds on the unsupervised audio-visual detector AVH-Align and comprises three stages. Stage 1, score-based routing: a threshold is estimated on a labeled validation set via the Youden criterion; test samples scoring at or below the threshold form the retained subset and keep their original scores, while those above form the routed subset and enter refinement. Stage 2, evidence mining and semantic analysis: frozen CLIP and GPT-4-generated real/fake text prototypes score sampled frames and local patches; key frames are selected by global suspiciousness and prototype concentration, and suspicious patch locations by local suspiciousness and cross-frame aggregation, forming cross-frame patch strips. Qwen2-VL then generates fine-grained descriptions under a fixed instruction, and a BGE text reranker compares each description with real and fake anchor texts, using the mean of per-row margins as the ranking score. Stage 3, slot-preserving reordering: the original detection scores in the routed subset serve as fixed score slots and are reallocated according to the ranking scores, preserving the score multiset, the retained subset's scores, and the relative order between the two subsets.

Experiments are conducted on AVLips, FakeAVCeleb, and TalkingHeadBench using AP and AUC, with the threshold estimated on the AVLips validation set and applied uniformly to all three test sets. TFER improves both the official and retrained AVH-Align, most notably within the routed subset, and retains its gains under four image perturbations. Additional analyses cover component ablations, the number of text prototypes, refinement coverage, and inference overhead.

Critical strengths

The slot-preserving reordering design offers clear invariance guarantees and is sufficiently conservative as a post-processing module. It changes only the relative order within the routed subset, leaving the retained subset's scores, the order between the two subsets, and the original score multiset unchanged; consequently, decisions at the routing threshold remain exactly identical to those of the base detector. This makes the refinement stage's scope clear and controllable, and its effect precisely separable and measurable. It is precisely this property that allows several of the quantitative analyses below to be derived from the values reported in the paper.

The paper reports metrics within the routed subset, the full prompts, and inference overhead, allowing the method's mechanism to be externally examined. Table 2 separately reports AP and AUC before and after refinement within the routed subset; Appendix A.1.6 lists the CLIP prototypes, Qwen instructions, and reranking anchor texts in full; Appendix A.1.4 reports an overhead of 5.67 s per routed sample and 21 GB peak GPU memory. This information helps reviewers and follow-up researchers verify the source of the gains and has clear transparency value.

Critical weaknesses
  1. Baseline selection is invalid. Within routed subsets, the base detector’s AUC is inverted, and simple shuffling or direct reversal achieves comparable or better performance than TFER. Ablations and perturbation tests show gains exceed the theoretical upper bound of the proposed mechanism.
  2. Routing design conflicts with its hard-sample motivation. It only selects predicted fakes and never revisits missed fakes. The transductive scoring cannot be deployed on single videos, and its ranking effect is essentially limited to reordering within the routed subset.
  3. “Training-free/unsupervised” claim is misleading. The method heavily depends on the labeled AVLips validation set for thresholds, prototypes, prompts and hyperparameters. Its huge latency and memory overhead are not justified given that random reordering already yields most gains.
Questions for authors

Q1.Table 2 shows that the base detector's AUC within the routed subset is 36.5 on THB and 31.6 on AVLips, which is systematically inverted rather than merely weak. A simple AUC decomposition shows that random reshuffling alone recovers most of the gain, while direct reversal matches or exceeds TFER; most ablations leave within-subset ranking at or below chance, and perturbation gains under noise, color inversion, and compression exceed the mechanism's upper bound. The paper therefore evaluates against the wrong baselines.

Q2.Routing is one-sided: it selects all samples above the Youden threshold, i.e., all predicted fakes, so missed fakes are never revisited and operating-point decisions remain unchanged; TFER can only affect threshold-free AUC/AP, contradicting the "hard sample" motivation. The final score is transductive and undefined for single-video deployment, and for ranking metrics slot-preserving reordering reduces to sorting the routed subset above the retained subset, leaving its substantive contribution unclear.

Q3.A fixed 3×3 grid is used without face detection or alignment, the code does not implement Equations 3–4, and Prompt B primes Qwen with "suspicious" and near-verbatim anchor vocabulary, so scores may measure prompt-induced lexical matching rather than visual evidence. Quality-oriented prototypes and dataset differences in resolution and compression also suggest a data-source shortcut, and the gains under blur, JPEG compression, and color inversion contradict the claimed mechanism.

Q4.The "unsupervised/training-free" claim is inconsistent with reliance on a labeled AVLips validation set—with GAN-generated fakes—for the threshold, model selection, prototypes, prompts, and hyperparameters, while the same threshold is transferred to THB and FKAV without validation. Given 5.67 s per routed sample and 21 GB peak memory versus 0.02 s for the base detector, and that random reordering already captures most of the gain, the cost-benefit is not justified.

Q5.THB test composition omits real-sample counts, the AVLips subset and FKAV test set are self-constructed, key baselines (Qwen zero-shot, AEROBLADE/RIGID/ZED/WaRPAD, score fusion, MLLM detectors) are missing, only two checkpoints are tested, and all results are single-run without confidence intervals.Key modules and functions are missing, stage1.py contains undefined calls, slot_bins is undocumented, and the code uses four patches versus three in Table 6.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.