AutoCAST: Automated Character-Attributed Speech Transcription in Long-Form TV Dramas

Automated named-speaker transcription aims to determine the start and end times, text, and speaking character of each utterance in long-form TV dramas. This task remains challenging for existing audio-visual models: multiple visible characters, off-screen speech, and reaction shots create visual ambiguity, while the lack of reliable character-specific voice references and limited speaker discrimination complicate acoustic attribution. To address these challenges, we propose AutoCAST (Automated Character-Attributed Speech Transcription), a three-stage framework that takes raw audiovisual streams, a cast roster, and character portraits as input to produce timestamped, character-attributed transcripts. Timestamp-aware transcription first predicts utterance boundaries and text using explicit temporal markers. Agreement-guided reference bank construction then selects candidate speech segments whose initial identity predictions agree with face observations to build a character-specific voice reference bank. Finally, reference-guided multimodal attribution performs acoustic seed filtering and label propagation, then combines character-level voiceprint scores with visual and dialogue context to determine the speaker of each utterance. The complete framework requires neither manual transcripts nor manually verified voice references at inference. On DramaSR-532K, AutoCAST achieves 91.70% mean speaker-attribution accuracy across seven held-out Chinese-language dramas, outperforming the strongest evaluated closed-source baseline (83.06%) by 8.64 percentage points, and achieves 87.27% across three held-out English-language series. Ablation studies support the effectiveness of automatic segmentation, reference bank construction, and multimodal speaker attribution. Code is available at https://anonymous.4open.science/r/AutoCAST/.