Listen First, Look as Needed: Uncertainty-Aware Asymmetric Routing for Audio-Visual Recognition

Efficient audio-visual recognition should preserve accuracy while activating vision only when required. Existing static, visual-centric dynamic, and audio-preview methods allocate fixed budgets, route after visual processing begins, or use audio to rank visual segments, but do not calibrate audio-only acceptance before activating vision. We propose Uncertainty-Aware Asymmetric Routing (UAAR), a “listen first, look as needed” framework separating pre-visual acceptance from progressive multimodal verification. An empirical selective-risk criterion governs zero-frame exits, and an entropy-based reliability test disables audio guidance under degradation. Unresolved samples acquire sparse clips and autoregressively accumulate cross-modal evidence under three-, six-, or nine-frame budgets. On Kinetics-Sounds and AVE, UAAR achieves 83.35% and 85.82% accuracy while reducing activated computation by 57.3% and 53.4%, respectively, relative to full nine-frame inference. Under progressive audio corruption, UAAR suppresses unsafe audio-only exits and adaptively requests additional visual evidence. Code will be made publicly available upon publication.