- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes Uncertainty-Aware Asymmetric Routing (UAAR) for efficient audio-visual event classification. Audio is always processed by a lightweight encoder, and visual clips are encoded only when needed. Inference follows three stages. First, an entropy-based reliability test, which decides whether the audio is unreliable and thus three uniformly spaced visual clips are used. Second, an audio-only exit, whose threshold is selected after training on a held-out calibration split. Third, for the remaining samples, up to three clips are chosen by audio feature saliency and fused sequentially. A recurrent gated fusion module exposes a prediction after each clip and stops once fused entropy falls below a threshold. On Kinetics-Sounds and AVE, UAAR-dynamic reaches 83.35% and 85.82% accuracy. Relative to full nine-frame inference, this reduces compute by 57.3% and 53.4%, at a cost of 1.29 and 1.00 accuracy points.
Critical strengths
-
The motivation is clear and practical. Audio is cheap, vision dominates sensing cost, and deciding whether to look before encoding any frame is a sensible design principle.
-
The method is simple, modular, and easy to deploy. Thresholds are set after training, so the operating point can be adjusted without retraining.
-
Under progressive audio corruption, the router behaves sensibly: audio-only exits fall and the visual budget grows.
Critical weaknesses
-
UAAR brings limited novelty as the method is a combination of known components. Each building block of UAAR is established: 1) Confidence-gated early exiting and classifier cascades are standard dynamic-inference techniques. 2) Using audio to choose which visual clips to encode was introduced by SCSampler (Korbar et al., ICCV 2019) and Listen to Look (Gao et al., CVPR 2020). 3) Audio-first modality skipping is already done by HCMS and AdaMML. 4) Gated recurrent fusion with deep supervision at each step is a routine design. The contribution is therefore incremental. The savings come mainly from skipping vision when audio is confident, which is a well-known idea.
-
The acceptance score is the standard maximum-softmax-probability confidence, so the calibration step amounts to selecting a confidence threshold on held-out data. Appendix B confirms that a conventional confidence threshold tuned to the same budget reaches exactly the same operating point (2.1916 GFLOPs, 83.35%). The calibration also misses its own target: with ε = 0.05, the test selective error is 5.78%.
-
The threshold h_deaf = 3.0 is close to the maximum possible entropy (ln 33 ≈ 3.50 on KS). It triggers on only 2.45% of clean samples. Removing it changes accuracy by 0.18 points on clean data and 0.36 points under the strongest corruption, both within one standard error. In the mismatch test (Appendix C), it fails to catch any of the confidently wrong audio exits, all of which are incorrect.
-
I noticed inconsistencies in how the baselines appear in the manuscipt. For example, the main table reports the HCMS configuration with the lowest test accuracy (83.11% at 4.40 GFLOPs). The 0.25-budget configuration in Appendix A reaches 84.03% at 2.50 GFLOPs, more accurate than UAAR-dynamic at similar cost. Also, AdaMML reaches 85.62%, above even the full-compute UAAR-static, but is excluded from the main comparison.
-
I also noticed some inconsistencies in terms of compute figures across the datasets. On AVE, UAAR-dynamic uses fewer frames than on KS but reports higher GFLOPs. UAAR-static also costs 5.94 vs 5.13 GFLOPs with the same architecture. Did I miss something? Furthermore, parameter counts also change across datasets without explanation. What drives this change?
-
I think the paper is also missing some important related works. SCSampler (Korbar et al., ICCV 2019) already uses audio to select salient clips for efficient recognition. Fast yet Safe (Jazbec et al., NeurIPS 2024) and SAFE-KD (Salim Khazem, SAFE-KD: Risk-Controlled Early-Exit Distillation for Vision Backbones, 2026) calibrate early-exit thresholds to a target selective risk with formal finite-sample guarantees, which UAAR’s empirical rule lacks.
-
The "w/o Audio Prior” ablation removes zero-frame exits and saliency-based sampling together, and it achieves higher accuracy (83.90% vs 83.35%).
-
The paper motivates continuous, resource-constrained sensing, but experiments use only trimmed 10-second clips from two small datasets. AVE, a localization benchmark, is repurposed for classification. It would then make sense to test UAAR on larger or long-video benchmarks (e.g., VGGSound, ActivityNet).
Overall, the current version of the manuscript, given all the weaknesses I have reported above, falls short of the ICLR bar. I recommend rejection.
Questions for authors
See also previous section.
-
Sec. 4.1 says h_safe is selected on the calibration set, while Sec. 3.3 and Appendix H say the validation set. Can the authors resolve this inconsistency?
-
Appendix H refers to a “finite-sample-corrected” selective risk, whereas Eq. 3 and Sec. 2 describe an uncorrected empirical criterion. Which one is correct?