AVC-Reward: Event-Centric Audio-Video Consistency Evaluation

As joint audio-video generation models become more capable, they can generate audio-visual scenes exhibiting three forms of complexity: $\text{multi-event composition}$, $\text{multi-layer audio mixture}$, and $\text{asymmetric cross-modal relations}$. However, current evaluations largely reuse metrics from earlier simple and unidirectional tasks. They typically evaluate an audio-video clip as a whole and rely on simple predefined relations, making them unsuitable for these increasingly complex scenes. We propose an $\text{event-centric}$ evaluation framework that disentangles audio and visual events at the event level, analyzes each event independently, cross-validates relations across modalities, and aggregates the judgments into a $\text{coarse-to-fine, three-level consistency metric}$. The metric assesses global narrative harmony, event-level temporal synchronization, and fine-grained attribute alignment, providing a general formulation for audio-video consistency evaluation across diverse generation settings and cross-modal relations. We instantiate the metric as $\text{AVC-Reward}$, a trainable audio-video reward model. To evaluate the agreement between automatic judgments and human perception across generation paradigms and relation complexities, we construct AVC-Benchmark, which is primarily built from diverse positive and negative samples generated by various types of audio-video generation models. AVC-Reward achieves state-of-the-art alignment with human judgment among automatic methods, provides interpretable rationales that localize failures, and improves the audio-video consistency of a joint generation model when used as an online reinforcement-learning signal. Project page: https://anonymous.4open.science/r/AVCReward-A743/