- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes TFER, a training-free evidence-driven refinement framework to improve the anomaly score ranking of existing unsupervised talking head forgery detectors, without modifying the base detector’s parameters. Extensive experiments on three benchmarks (THB, AVLips, FKAV) and under four types of perturbations demonstrate consistent performance improvements over the base AVH-Align detector.
Critical strengths
- As a training-free plug-and-play module, TFER can be integrated with existing detectors without retraining or redesign, which lowers the barrier to practical adoption. In addition, the textual evidence generated by the MLLM provides human-understandable explanations for detection decisions, addressing the black-box limitation of many deep forgery detectors.
- The motivation of this paper is clear and the authors chose a straightforward but effective method to achieve the goal.
Critical weaknesses
- The paper validates TFER only on variants of a single base detector (AVH-Align), and excludes AVAD due to its poor base performance. A core implicit claim of the work is that TFER can universally improve existing score-based detectors, but this claim is not sufficiently supported. It remains unclear whether the framework can consistently bring gains to other unsupervised audio-visual detectors, or even supervised detectors. Testing on only one detector family weakens the broader impact of the method.
- The routing threshold estimated on the AVLips validation set is directly applied to THB and FKAV for cross-dataset evaluation. Given that different datasets have distinct anomaly score distributions and real/fake separability, a fixed threshold may lead to suboptimal routing ratios (e.g., too many or too few routed samples) on unseen datasets. The paper does not report routing ratios across datasets, nor does it compare with alternative strategies (e.g., percentile-based thresholds) to verify the robustness of the Youden-threshold routing across data distribution shifts.
- The evidence mining and ranking stages rely heavily on manually designed text prototypes, Qwen instructions, and reranker anchors, all of which are tailored to specific facial forgery artifacts (over-smoothed skin, smeared mouths, unstable teeth, etc.). If new generators produce novel forgery patterns not covered by these prompts, the refinement performance may degrade significantly. The paper does not evaluate robustness against unseen forgery types, and the GPT-4-based prompt engineering process introduces subjectivity that may affect reproducibility.
- The refinement stage takes 5.67 seconds per routed sample on an A100 GPU, which is orders of magnitude slower than the base detector (0.02s per sample). While selective routing reduces the overall cost, the computational overhead is still prohibitive for large-scale screening or real-time applications. The paper only briefly notes suitability for offline auditing, but provides no discussion of efficiency optimization directions (e.g., lightweight MLLM backbones, early stopping) or the performance-efficiency trade-off curve.
- Main results are reported on merged test sets, but THB contains multiple diffusion-based generators and FKAV covers multiple manipulation types. The paper does not provide per-generator or per-manipulation performance breakdowns, so it is unknown whether TFER brings consistent improvements across all forgery methods, or whether gains are concentrated on specific types. This limits the assessment of fine-grained generalization.
Questions for authors
- What is the proportion of routed samples on each test set (THB, AVLips, FKAV)? Please analyze how routing ratio correlates with performance gain, and compare the Youden threshold with a fixed-percentile threshold (e.g., top 30% highest-scoring samples) to justify the routing strategy.
- Have the authors tried directly using the reranker’s rank scores as new detection scores, instead of performing slot-preserving reordering? Please report the performance of direct score replacement and compare it with SPR, to clarify the necessity and advantage of the slot-preserving design.
- For perturbation experiments, the corruption intensity is fixed. How does TFER perform under stronger or weaker corruption levels? A sensitivity analysis on perturbation intensity would better support the robustness claim.