- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces SAMVAD-DF, a benchmark for detecting and localizing partial audio deepfakes in multispeaker Hindi conversations. It contains three manipulation granularities, standard and refined temporal annotations, and experiments with several detection and localization models. The paper argues that precise temporal labels improve localization while making file-level detection more difficult.
Critical strengths
- The paper addresses an important gap by studying partial manipulation in multilingual, multispeaker conversational audio.
- The distinction between file-level detection and temporal localization is useful, and the comparison of original and refined labels gives a helpful diagnostic perspective.
Critical weaknesses
- The benchmark relies mainly on one TTS system and one text-generation pipeline, so transfer to other generators is not yet clear.
- Fake text is restricted to pure Devanagari, while real recordings contain Hinglish. The 93.98% transcript-only AUC suggests a strong label-correlated cue.
- File-level and localization experiments use different windows, labels, and loaders, so the effect of refined labels is not fully isolated.
- The human-assisted annotation procedure and reproducibility details are brief, especially for the relatively small Verified Track.
- The writing should be revised for clarity, and the figures should be redrawn with clearer, higher-resolution technical graphics. Figure 1 is currently more decorative than explanatory.
Questions for authors
- Could the authors provide a generator-held-out result and a control in which fake transcripts preserve the script and code-switching distribution of bona fide speech?
- Could they clarify annotation agreement and report the main comparison under one consistent evaluation protocol?