Clustering-Free End-to-End Spoof Diarization with Encoder-Decoder Attractors

Spoof diarization aims to jointly localize spoofed regions and determine which spoofing method generated each segment within a partially spoofed utterance. Existing approaches rely on modular pipelines that separate localization and clustering, introducing training–inference mismatch and requiring oracle knowledge of the number of spoofing classes at inference time. We propose E2E-SD, the first clustering-free end-to-end framework for spoof diarization, which reformulates the task as variable-cardinality temporal set prediction. E2E-SD jointly learns localization and class assignment using encoder-decoder attractors with Hungarian-matched training, eliminating the need for post-hoc clustering and oracle class counts. Our framework dynamically generates class-specific attractors through an LSTM decoder and estimates the number of active spoofing classes via attractor existence probabilities, requiring no external class-count information at any stage. On the PartialSpoof benchmark, E2E-SD achieves a $\text{JER}$ of 15.76%, a 41.6% relative improvement over the previous best result (26.99%), while operating fully oracle-free with 88.7% class-count estimation accuracy. The largest gain is observed on unknown attacks, where $\text{JER}$ drops from 49.02% to 21.00% (57.1% relative improvement), demonstrating that end-to-end optimization with dynamic attractors enables more attack-agnostic generalization than modular clustering pipelines.