acceptodds
Under review as a conference paper at ICLR 2027

SceneRoute-AVQA: Scene-Conditioned Trustworthy Fusion for Audio-Visual Question Answering Under Multimodal Degradations

Abstract

Audio-Visual Question Answering (AVQA) requires selecting question-relevant evidence from audio and visual streams. However, real-world inputs often suffer from temporal misalignment, cross-video interference, missing modalities, visual occlusion, and audio-visual noise. These degradations alter modality reliability in a sample- and question-dependent manner, making conventional fixed-weight or unified fusion methods vulnerable to unreliable evidence and over-confident predictions. We propose SceneRoute-AVQA, a scene-conditioned robust framework based on a diagnosis-routing-calibration paradigm. First, scene-conditioned degradation diagnosis uses scene cues, temporal offset estimation, and modality quality assessment to characterize degradation sources, temporal inconsistencies, and modality reliability cues. Second, question-aware reliability fusion jointly considers question semantics and sample-specific modality quality to dynamically select and fuse trustworthy audio-visual evidence. Hierarchical question-scene answer priors are further introduced to mitigate low-resource and imbalanced answer distributions in degraded scenes. Third, robust context and answer calibration exploit scene-consistent contextual information and lightweight logit residuals to stabilize decision boundaries on difficult samples. Experiments under clean conditions and five degraded AVQA settings demonstrate that SceneRoute-AVQA consistently outperforms existing methods across temporal misalignment, cross-video interference, missing modalities, occlusion, and audio-visual noise, confirming its effectiveness in degradation diagnosis, adaptive evidence fusion, and robust answer prediction.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Audio-Visual Question Answering, Robust Multimodal Learning, Scene-Conditioned Reasoning, Multimodal Degradation, Trustworthy Fusion, Answer Calibration

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_SP4pOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces a work aiming for addressing the degradatins in audio--visual question answeing task. In addition to proposing a framework, the paper also brings a benchmark on the extension of previous MUSIC-AVQA.

Critical strengths

The motivation is quite clear, addressing the degradation issues in audio--visual question answering.

Critical weaknesses

From my side, there are several critical weaknesses of this paper, including presetation, literature, model design, experiments. I DO NOT believe the current version is suitable for this submission. Here follow the details.

  1. Figure 1 c. I respectively do not think the generated icons by AI models help improve the presentation of the illustration. Even worse, readers may not know the exact meaning of them.
  2. Related work is not sufficient for supporting the current literature, which affects the completeness of the research in this domain, for example, AVQA, AVS, and robust learning methods. Here, I do not list specific ones as there are too many left.
  3. The presentation is not good. Here in Figure 2, one cannot see the details of the text as the front size is too small. Again, I do not believe the AI-generated figure clearly show the intention that the authors want to present, and even worse, leading to unclear misunderstanding to the paper.
  4. In section 3.1, the authors design several modules for different degradation. I clearly do not see how the authors combine them into the framework. In addition, how did the authors address the claimed 'question-dependent' issue?
  5. As for the dataset, why not select real degradation cases rather than transformation on the clean ones? I do not believe the transformation could represent the degradations probably.
  6. The experimental section is extremely limited, lacking sufficient experiments to support the research.
Questions for authors

See the weaknesses above.

Reviewer_27oqOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies Audio-Visual Question Answering under input degradations such as temporal misalignment, cross-video interference, missing modalities, occlusion, and noise. It proposes SceneRoute-AVQA, which follows a diagnosis-routing-calibration pipeline. A condition classifier and condition-specific adapters first extract degradation evidence. An uncertainty-based gating module then fuses the modalities, with question–scene answer priors added to the logits. Finally, a multi-view logit ensemble (SCRC) and a lightweight hard-scene answer residual calibrate the predictions.

Critical strengths
  1. In real-world AVQA settings, audio and video inputs are not always complete or temporally aligned. The paper addresses a relevant problem by considering how modality reliability varies across different types of degradation.

  2. The approach of determining modality importance based on both input quality and question requirements is reasonable. In particular, the motivation that different question types, such as counting, action, and sound-source questions, require different audio-visual evidence is convincing.

  3. Extreme-MUSIC-AVQA includes clean conditions and five types of degradation, allowing the model to be evaluated under diverse conditions.

Critical weaknesses
  1. The best reported accuracy in Table 1 relies on Oracle Scene Context, while predicted routing results are not provided. This makes it difficult to evaluate the proposed diagnosis and routing mechanisms under realistic inference conditions. The authors should report predicted routing performance and compare it with oracle routing and the main baselines.

  2. The paper emphasizes question-dependent modality reliability, but the Reliability Gate ablation does not distinguish question-aware fusion from quality-based weighting. Comparisons with question-independent fusion and analyses of modality weights across question types are needed.

  3. Table 1 shows a substantial accuracy increase from 69.39% to 72.65% with the Hierarchical Question-Scene Prior configuration. However, it is unclear whether this gain comes from improved audio-visual reasoning or dataset-specific answer distributions. Prior-free and prior-only comparisons are needed.

  4. The paper introduces condition-specific adapters for five degradation types, but Figure 3 only reports the effect of removing all Scene Adapters. Individual adapter ablations are needed to verify their effectiveness for each degradation type.

  5. LLoVi is incorrectly cited as an audio-visual interaction method in the Introduction, as it focuses on Video Question Answering using visual captions and LLM-based reasoning.

  6. (minor) The caption of Table 2 states that AV-Master was not evaluated due to incompatibility with the feature extraction setup, but the table still reports its performance. This inconsistency should be clarified.

  7. The authors appear to have used an AI-generated image without modification.

  8. The proposed scene-conditioned routing mechanism appears conceptually similar to existing Mixture-of-Experts (MoE) approaches, where different experts are selected based on input conditions. This paper does not explain what distinguishes its routing mechanism from these existing approaches.

  9. (minor) Table 1 uses inconsistent decimal precision, reporting accuracy to either two or four decimal places. The accuracy values should be consistently reported to two decimal places throughout the paper.

Questions for authors
  1. What is the performance with predicted (non-oracle) routing, and how does it compare with oracle routing and the baselines?

  2. How does the question affect the fusion weights, and how does question-aware gating compare with quality-only weighting?

  3. What are the accuracies of the prior-free model and of a prior-only predictor?

  4. What is the effect of removing each condition-specific adapter individually?

Reviewer_TZRUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes SceneRoute-AVQA, a framework for robust audio-visual question answering under different input degradations. The authors also construct Extreme-MUSIC-AVQA, covering five types of degradation, and evaluate the proposed method under both clean and degraded conditions.

Critical strengths
  1. The paper addresses a meaningful problem in AVQA, especially the reliability of audio-visual evidence under different input degradations.
  2. The unified framework considers several degradation types, and the per-condition evaluation is useful.
Critical weaknesses
  1. The best reported result (73.14%) relies on oracle scene context (Table 2). However, the performance of the actual model with predicted scene routing is not clearly reported. Since scene diagnosis is a key component of the proposed method, this makes it difficult to assess its effectiveness in practice.

  2. The proposed benchmark is mainly constructed by applying synthetic transformations to MUSIC-AVQA samples. Although these controlled degradations are useful for evaluation, they may not fully reflect real-world audio-visual degradation. It is therefore unclear whether the proposed method can generalize to naturally degraded AVQA scenarios.

  3. The figures, especially Figure 2, appear to rely heavily on AI-generated illustrations. The text is too small, and many icons do not clearly explain the functions or connections of the modules. As a result, the framework is difficult to understand from the figures.

Questions for authors
  1. What is the performance of SceneRoute-AVQA with predicted scene routing, without oracle information?
  2. How do the authors justify that the synthetic degradations used in Extreme-MUSIC-AVQA are representative of real-world scenarios?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.