acceptodds
Under review as a conference paper at ICLR 2027

TraceAV-Bench: Benchmarking Multi-Hop Trajectory Reasoning over Long Audio-Visual Videos

Abstract

Real-world audio-visual understanding requires chaining evidence that is sparse, temporally dispersed, and split across the visual and auditory streams, whereas existing benchmarks largely fail to evaluate this capability. They restrict videos to short clips, isolate modalities, or reduce questions to one-hop perception. We introduce TraceAV-Bench, the first benchmark to jointly evaluate multi-hop reasoning over long audio-visual trajectories and multimodal hallucination robustness. TraceAV-Bench comprises 2,000 rigorously validated multiple-choice questions over 578 long videos, totaling 339.5 hours, spanning 4 evaluation dimensions and 15 sub-tasks. Each question is grounded in an explicit multi-hop evidence trajectory that averages 3.68 hops across a 15.1-minute temporal span. The dataset is built by a three-step pipeline followed by a strict quality assurance process. Evaluation of multiple representative OmniLLMs on TraceAV-Bench reveals that the benchmark poses a persistent challenge across all models, with the strongest closed-source model (Gemini 3.1 Pro) reaching only 68.29% on general tasks, and the best open-source model under direct inference (Ming-Flash-Omni-2.0) reaching 51.70%, leaving substantial headroom. Further analysis shows that current models struggle to gather and combine evidence across long audio-visual videos, with performance varying across modalities and reasoning tasks. We hope TraceAV-Bench will facilitate progress toward reliable long-form audio-visual reasoning in OmniLLMs. The code and data are available at https://anonymous.4open.science/r/TraceAV-Bench-Submission-5FD4.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.