Membership Inference for Generative Video Models

With growing interest in copyright, safety, and bias of generative video models, the need for robust membership inference (MI) for these models has never been greater. Yet, only one such method currently exists, designed for old generator architectures. What's more, it comes with problematic methodological design, since the members and non-members on which it was calibrated and tested came from two fundamentally distinct distributions. To allow for robust MI research in generative video models, we construct and open-source a framework and dataset of non-members complementing the VideoUFO dataset, used to train a large text-to-video (T2V) diffusion transformer. We then examine prominent MI methods from image generation and classification across the access spectrum and adapt them for T2V models. Additionally, we present our own gray-box method called The Reversal Gap (TRG) and find that an ensemble including multiple methods achieves the best performance.