Training-Free Evidence-driven Refinement for Unsupervised Talking Head Forgery Detection

Supervised talking head forgery detection faces severe generalization challenges due to the continuous evolution of generators. By reducing reliance on generator-specific forgery patterns, unsupervised detectors offer an alternative for handling unseen generators. Alongside efforts to build stronger detectors, we investigate whether the anomaly score ranking of the existing detector can be improved without modifying its parameters. In particular, for score-based unsupervised detectors, the limited discriminative ability on hard cases is often reflected in unreliable anomaly ordering, leaving room for further refinement. Motivated by this observation, we propose a Training-Free Evidence-driven Refinement (TFER) framework to improve anomaly score rankings through selective refinement. TFER first partitions samples into retained and routed subsets using lightweight score-based routing. It then revisits only the routed subset by organizing localized visual evidence and performing semantic analysis to refine its relative ordering while preserving the original score multiset. Extensive experiments demonstrate consistent improvements over the base detector across multiple datasets and perturbation settings. These findings demonstrate the effectiveness of selective evidence-driven refinement in improving unsupervised talking head forgery detection without retraining the base detector.