acceptodds
Under review as a conference paper at ICLR 2027

OP-CAD: On-Policy Clean-Audio Self-Distillation for Noise-Robust Omni-Modal Reasoning

Abstract

Omni-modal large language models deployed in real-world environments encounter external noise that can interfere with their perception and understanding of multimodal inputs. We study their robustness in audio-visual understanding, focusing on question answering under environmental noise and competing speech. The challenge is to resist acoustic interference while preserving useful audio evidence. On-policy distillation provides dense teacher feedback on student-generated responses, but uniform token weighting does not explicitly prioritize positions affected by acoustic interference. We introduce OP-CAD (On-Policy Clean-Audio Distillation), a curriculum-based privileged self-distillation framework for robust audio-visual understanding. Training progresses from mild to severe environmental noise and competing speech, with selective token-level supervision at each stage. The student generates responses from corrupted audio-visual input, while a frozen teacher uses clean audio and the verified answer to supervise the same response prefixes. To allocate this supervision, OP-CAD compares teacher predictions under clean, corrupted, and visual-only contexts without revealing the answer. These matched comparisons measure sensitivity to audio removal and corruption; a bounded weighting rule emphasizes positions identified by either signal while retaining supervision throughout the response. OP-CAD outperforms the compared methods across all evaluated noise conditions. Paired analyses further show improved preservation of clean-correct answers under strong interference, with no observed aggregate clean-accuracy penalty. These results demonstrate the value of directing clean-teacher supervision toward acoustically sensitive predictions for robust audio-visual reasoning.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Audio-Visual Understanding, Acoustic Robustness, On-Policy Knowledge Distillation, Privileged Supervision, Curriculum Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_P2fhOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies the robustness of audio-visual question answering under environmental noise and competing speech. It proposes OP-CAD, an on-policy self-distillation method where a student generates responses from noisy audio-visual inputs, while a frozen teacher uses clean inputs and verified answers to provide token-level supervision. The main idea is to assign higher weights to tokens whose teacher predictions are sensitive to audio removal or corruption. The method also uses failure-directed data selection and a noise curriculum that gradually increases the interference level. Experiments on Daily-Omni and OmniVideoBench show improvements over SFT, GRPO, and uniform-weight OPSD under noisy conditions, with no observed drop in overall clean accuracy. The authors also provide ablations on the two weighting signals and the noise curriculum.

Critical strengths
  1. Consistent improvements over OPSD. OP-CAD improves noisy-condition accuracy by around 2.8–3.8 percentage points over uniform-weight OPSD across both benchmarks and interference types. This comparison is well controlled, as both methods share the same teacher, data, warm-up checkpoint, and noise curriculum. The gains hold across different question types, audio types, and video durations, and the ablation studies confirm the benefits of both sensitivity signals and the curriculum schedule.
  2. Simple and practical approach. The proposed token-weighting mechanism is straightforward to implement on top of standard self-distillation, as it only reweights the token-level KL loss during training. At inference time, the student model runs directly on audio-visual inputs without requiring additional model components, clean audio references, or extra teacher passes.
Critical weaknesses
  1. Limited methodological novelty. The overall pipeline closely follows existing on-policy self-distillation methods, particularly OPSD, while privileged supervision from clean audio or text has already been explored in recent audio distillation works such as EchoDistill and CORD. On top of OPSD, the main additions are a token-weighting rule based on two JS divergences, along with failure-directed data screening and an SNR curriculum. Although the method achieves consistent improvements, the core methodological contribution appears relatively incremental over prior privileged distillation frameworks.

  2. Insufficient justification and validation of the weighting mechanism. Section 3.4 motivates the max rule and per-response normalization, but their advantages over simpler alternatives are not directly established. The paper does not compare max with a mean or convex weighted sum, examine sensitivity to lambda, or ablate response-wise normalization. Table 4 also shows that the combined rule is not uniformly better than corruption sensitivity alone: on OmniVideoBench, Clean accuracy is 41.60% versus 43.80%, while Speech accuracy is nearly unchanged at 38.93% versus 39.00%. These observations do not negate the overall robustness gains, but leave the benefit of the particular combination rule less clear. Since the allocation signals require additional teacher evaluations, a comparison with simpler weights computed from existing predictions would help assess their added value.

Questions for authors
  1. Section 3.4 explains the scale-calibration motivation for per-response normalization and the use of max to emphasize either sensitivity signal. Could the authors compare this design with a mean or convex weighted sum, ablate response-wise normalization, and report sensitivity to lambda? In particular, can response-wise normalization amplify small but nonzero sensitivity signals, and how does this affect the reliability of the allocation weights?

  2. Could the authors compare OP-CAD with simpler token-weighting strategies computed from existing teacher predictions, such as privileged-teacher confidence or entropy, under the same training setup? This would help determine whether the robustness gains require the additional clean, corrupted, and visual-only teacher evaluations.

3 comments
AuthorsOpenReview ↗

Response to Reviewer P2fh (Part 1/3)

Dear Reviewer P2fh,

We sincerely thank you for recognizing the robustness improvements and for your constructive comments on the contribution and weighting design. In response, we have clarified how our approach relates to prior work and added comparisons of signal fusion, response normalization, weight strength, and simple teacher-derived weights. The relevant explanations and results are included in the main paper and appendix, with revisions highlighted in blue. Our responses follow.

Q1: The method builds on OPSD and should clarify its contribution relative to approaches such as EchoDistill and CORD.

A1: Thank you for encouraging us to clarify the contribution. OP-CAD builds on OPSD’s distillation framework over student-generated trajectories. Its central addition is supervision allocation based on sensitivity to audio conditions while retaining a shared visual context. EchoDistill and CORD explore privileged supervision from clean audio and text, respectively. OP-CAD instead compares teacher predictions after audio removal and corruption against a common clean audio-visual reference, using the same student prefixes and visual context, and uses these differences to assign token-level supervision strength.

The learning target and the allocation signal serve distinct roles in this design: the target teacher receives the verified answer, whereas all three contexts used to compute allocation signals exclude it. Our additional experiments directly evaluate this mechanism. The default method outperforms shuffled weights, Mean fusion, unnormalized scores, and teacher entropy/confidence weights under noise, and retains its advantage over OPSD when training compute is approximately matched. These results support the practical value of audio-condition comparisons within the existing distillation framework. We have clarified this positioning in the method description and included the corresponding controls in the appendix.

Manuscript reference: Section 2 discusses the related methods, and Section 3.4 explains supervision allocation. Appendix E.1 (Tables 16–22) provides the additional empirical comparisons.

Q2: A comparison with Mean or convex combinations is needed, along with an explanation of why the combined signal sometimes underperforms corruption sensitivity alone.

A2: Thank you for suggesting this control and highlighting the differences in the single-signal results. We add Mean as an equally weighted convex combination, s = (Ã + Ñ)/2, and compare it with Max and the two original single-signal variants. All configurations use the same λ and other training settings. The results are shown below. Env. and Speech denote environmental noise and competing speech, respectively.

Daily-Omni (accuracy, %)

MethodCleanEnv. 10 dBEnv. 5 dBEnv. 0 dBSpeech 10 dBSpeech 5 dBSpeech 0 dB
OP-CAD (default)70.0969.4267.0864.4167.0863.7459.82
Removal only69.3467.0863.2463.0765.2562.3257.56
Corruption only68.7668.0964.2463.9165.3362.4158.65
Mean69.1768.0065.7562.9965.8362.0758.48

OmniVideoBench (accuracy, %)

MethodCleanEnv. 10 dBEnv. 5 dBEnv. 0 dBSpeech 10 dBSpeech 5 dBSpeech 0 dB
OP-CAD (default)41.6040.1039.4041.2039.1039.5038.20
Removal only40.7040.2039.0039.7040.0038.0037.50
Corruption only43.8039.5040.7039.0041.7036.5038.80
Mean41.2038.6037.5039.6036.4037.9036.30

Max outperforms Mean across all noisy conditions. When only one sensitivity signal is strong, Mean reduces the weight increment at that position, whereas Max preserves it. The additional results support this fusion choice.

The complete results also retain the single-signal advantage you noted: corruption-only weighting performs better on clean input and in some noisy OmniVideoBench conditions, while the combination is stronger in others. This reflects differences in how the signals contribute across conditions. We preserve these results and add the direct comparison with Mean to provide further empirical support for choosing Max.

Manuscript reference: Table 4 in the main paper retains the single-signal results. Appendix E.1.4 (Table 20) adds the fusion comparison at each SNR.

AuthorsOpenReview ↗

Response to Reviewer P2fh (Part 2/3)

Q3: An ablation of normalization and an analysis of small nonzero sensitivity scores are needed.

A3: Thank you for examining the normalization behavior closely. Response normalization places the two signals on comparable scales within each response, and it can increase the relative weights assigned to signals with small absolute magnitudes. To examine this behavior, we add weight statistics for relatively low-sensitivity responses and a training control without normalization.

For each fixed response, we compute M = max_t{A_t, N_t}, where A and N are raw JS scores and t ranges over valid generated positions. Within each SNR, we rank responses by M and select the lowest fifth as the relatively low-sensitivity group. Both weighting rules are then evaluated on the same responses and token positions. The statistics are shown below.

RuleSNRResponsesMean MMean weightWeight SD
Response normalization10 dB4180.0921.2430.117
Response normalization5 dB5120.0871.2510.121
Response normalization0 dB6550.0791.2380.114
Raw JS (no normalization)10 dB4180.0921.0410.016
Raw JS (no normalization)5 dB5120.0871.0380.015
Raw JS (no normalization)0 dB6550.0791.0350.014

Normalization increases both the mean and variation of the weights in this group, giving relative sensitivity differences within a response greater influence on allocation. The default λ bounds the weight increment, and the final loss is still divided by the total weight.

We also train a variant that uses raw JS scores directly to construct weights, keeping λ and the other settings unchanged:

Daily-Omni (accuracy, %)

MethodCleanEnv. 10 dBEnv. 5 dBEnv. 0 dBSpeech 10 dBSpeech 5 dBSpeech 0 dB
OP-CAD (default)70.0969.4267.0864.4167.0863.7459.82
No response normalization69.7667.3464.6662.2463.9961.2457.98

OmniVideoBench (accuracy, %)

MethodCleanEnv. 10 dBEnv. 5 dBEnv. 0 dBSpeech 10 dBSpeech 5 dBSpeech 0 dB
OP-CAD (default)41.6040.1039.4041.2039.1039.5038.20
No response normalization41.3038.6037.5039.7036.3037.7036.50

The default model performs better across all noisy conditions. The weight statistics explain how normalization changes allocation, while the training comparison supports its positive effect on overall question-answering performance. Together, these analyses provide a clearer account of both the behavior and empirical value of response-wise scale calibration.

Manuscript reference: Appendix E.4 (Table 27) defines the group and reports the weight statistics; Appendix E.1.3 (Table 19) presents training without normalization. Section 3.4 explains that the weights reflect relative sensitivity.

Q4: The results should be evaluated for their dependence on the choice of λ.

A4: Thank you for suggesting an evaluation of the weighting strength. We compare λ ∈ {0, 0.25, 0.5, 1.0}, reusing the original OPSD and default OP-CAD results for λ = 0 and 0.5, respectively. The other configurations use the same training settings. Results are reported below.

Daily-Omni (accuracy, %)

MethodCleanEnv. 10 dBEnv. 5 dBEnv. 0 dBSpeech 10 dBSpeech 5 dBSpeech 0 dB
OPSD69.3465.8363.6660.9964.1661.5756.14
λ=0.2569.8468.8466.4263.8366.5063.2459.23
OP-CAD (default)70.0969.4267.0864.4167.0863.7459.82
λ=1.069.7668.3466.2563.9966.0863.3259.06

OmniVideoBench (accuracy, %)

MethodCleanEnv. 10 dBEnv. 5 dBEnv. 0 dBSpeech 10 dBSpeech 5 dBSpeech 0 dB
OPSD41.4037.4036.2038.6036.5034.6034.30
λ=0.2541.5039.7039.0040.8038.7039.1037.80
OP-CAD (default)41.6040.1039.4041.2039.1039.5038.20
λ=1.041.5039.4038.7040.5038.5038.9037.60

Values on either side of the default continue to outperform uniformly weighted OPSD across all noisy conditions and remain close to the default configuration. This shows that the gains persist over the tested range and do not depend on a single λ value. The default configuration performs best in these comparisons, so we retain the original setting.

Manuscript reference: Appendix E.1.5 (Table 21) reports the λ-sensitivity results.

AuthorsOpenReview ↗

Response to Reviewer P2fh (Part 3/3)

Q5: Confidence or entropy weights derived directly from the privileged teacher should be compared to assess the value of the additional teacher evaluations.

A5: Thank you for proposing this useful control. We add two weighting rules derived directly from the clean privileged teacher’s predictions: normalized entropy H(p)/ln|V| and confidence max p. Both use w = 1 + 0.5s and divide the weighted loss by the total weight. They reuse the existing teacher distribution without evaluating the three auxiliary teacher contexts; all other training settings and the data-pass budget remain unchanged. The results are shown below.

Daily-Omni (accuracy, %)

MethodCleanEnv. 10 dBEnv. 5 dBEnv. 0 dBSpeech 10 dBSpeech 5 dBSpeech 0 dB
OPSD69.3465.8363.6660.9964.1661.5756.14
OP-CAD (default)70.0969.4267.0864.4167.0863.7459.82
Teacher entropy69.0967.3465.1662.3265.2562.0757.73
Teacher confidence69.1767.0064.8361.9964.9161.7457.39

OmniVideoBench (accuracy, %)

MethodCleanEnv. 10 dBEnv. 5 dBEnv. 0 dBSpeech 10 dBSpeech 5 dBSpeech 0 dB
OPSD41.4037.4036.2038.6036.5034.6034.30
OP-CAD (default)41.6040.1039.4041.2039.1039.5038.20
Teacher entropy41.3038.5037.7039.6037.3037.8036.60
Teacher confidence41.1038.2037.4039.3037.0037.5036.30

Both entropy and confidence weighting outperform uniformly weighted OPSD under noise, while OP-CAD further improves accuracy across all noisy conditions. This indicates that sensitivity obtained by comparing audio conditions can guide supervision allocation more effectively than these weights derived from the target teacher alone. We appreciate your suggestion, which provides a more direct test of the value of the additional audio-condition comparisons.

Manuscript reference: Appendix E.1.6 (Table 22) gives the weighting rules and results at each SNR.


Thank you again for your thoughtful review and constructive suggestions. We hope these clarifications and additional experiments make the contribution and the rationale for our design choices clearer. We would be happy to address any remaining questions and continue the discussion to further improve the manuscript.

Reviewer_vWn6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes OP-CAD, a on-policy self-distillation framework for improving the robustness of audio-visual language models. The method is very straightforward. There is a student model generates responses using corrupted audio-visual inputs, and a frozen teacher uses clean audio-visual inputs and verified answers to provide supervision. The main contribution of the method is a token-level weighting mechanism based on the teacher's prediction sensitivity to audio removal and corruption. Authors also designe a curriculum that progressively increases noise severity. Experiments are conducted on two datasets, Daily-Omni and OmniVideoBench. Both shows consistent improvements over SFT, GRPO, and uniform on-policy distillation.

Critical strengths
  1. The paper is targeting at a practically important problem: acoustic interference is a common challenge for real-world multimodal assistants. The paper investigates how clean-audio supervision can improve performance without requiring clean audio at inference time.

  2. The experiments are very well controlled and clean. The direct baseline uses the same teacher, student initialization, data, and curriculum, allowing the contribution of sensitivity-based weighting to be evaluated more directly.

  3. The method is simple and easy to follow: It uses audio removal and corruption sensitivity to prioritize tokens that are more affected by acoustic information. The weighting mechanism is easy to understand and integrates naturally into on-policy distillation.

Critical weaknesses
  1. One of the main concerns i have is the sensitivity-based weighting hypothesis that the paper adopts. A arge change in teacher predictions under audio corruption does not necessarily indicate that a token contains task-relevant information. It will be great if the authors could systematically demonstrate that highly weighted tokens correspond to evidence necessary for answering the question.

  2. Additionally, the method uses response-level normalization and combines sensitivity signals through a maximum operation. It is unclear whether these choices are necessary. It will help strength the paper's claim if the authors could provide additional comparisons, e.g., with random weighting, uncertainty-based weighting, and alternative signal-combination rules etc.

  3. Just because the higher noisy accuracy the method achieves it does not necessarily means the model has learned better integration of audio and visual evidence. The improvement could reflect changes in modelity reliance. More targeted evaluations with audio only, visual only and cross model will improve the credibility and clairity of the conclusion.

Questions for authors

It would be great if the authors could answer the questions brought in the weakness, and address each concern separately. I will consider raising the scores if those get properly addressed.

2 comments
AuthorsOpenReview ↗

Response to Reviewer vWn6 (Part 1/2)

Dear Reviewer vWn6,

We sincerely thank you for your thoughtful comments on supervision allocation and modality contributions. Your suggestions have helped us explain the role of the weights more clearly and evaluate the design through targeted controls. We have added alternative-weight comparisons and single-modality evaluations, and expanded the analysis of the example in Figure 5. The corresponding revisions in the main paper and appendix are highlighted in blue. We respond to each point below.

Q1: Changes in teacher predictions do not necessarily identify task-critical evidence; the role of highly weighted positions needs more direct explanation and validation.

A1: Thank you for this important question. We combine a concrete question-answering example with a training control to clarify how sensitivity weights relate to positions involved in the task prediction. OP-CAD uses sensitivity to allocate supervision strength, while the clean privileged teacher’s full predictive distribution provides the learning target. This allows stronger guidance at reasoning positions affected by interference.

Figure 5 asks: “Considering both the visuals and audio, what could be inferred about the person speaking in the initial seconds?” The options concern operating a boat (A), observing the environment alone from a distance (B), fishing and anticipating a catch (C), and possibly doing laundry near flowing water (D). The correct answer is C. The student answers C with clean input but switches to B under competing speech. Its response first describes the voice as “calm and observational,” then combines this interpretation with the river and swimming crocodile to characterize the speaker as “a distant observer,” explicitly ruling out “fishing or boating.” This reasoning supports the incorrect observer interpretation while excluding the fishing activity in the correct option.

The weight pattern can be understood in relation to this reasoning. Removal sensitivity is high at positions such as “speaker,” “voice,” and “sound,” whereas corruption sensitivity is high at “observational” and “flowing.” In particular, “observational” provides a semantic basis for the later observer interpretation. Its high corruption sensitivity indicates that the teacher’s prediction at that position changes substantially with audio corruption. Max preserves the peaks of both signals, assigning stronger alignment with the clean teacher to these interpretation steps before the final answer. Figure 5 also shows that OP-CAD answers C on the same noisy input, connecting the visualization to the question, the erroneous reasoning, and the final prediction.

To test the training value of matching weights to positions, we randomly permute the weights within each current rollout while preserving their multiset and sum. The following tables also include the alternative rules discussed in Q2. Env. and Speech denote environmental noise and competing speech, respectively.

Daily-Omni (accuracy, %)

MethodCleanEnv. 10 dBEnv. 5 dBEnv. 0 dBSpeech 10 dBSpeech 5 dBSpeech 0 dB
OPSD69.3465.8363.6660.9964.1661.5756.14
OP-CAD (default)70.0969.4267.0864.4167.0863.7459.82
Shuffled weights69.4266.5864.2461.4064.8361.3257.06
No response normalization69.7667.3464.6662.2463.9961.2457.98
Mean69.1768.0065.7562.9965.8362.0758.48
Teacher entropy69.0967.3465.1662.3265.2562.0757.73
Teacher confidence69.1767.0064.8361.9964.9161.7457.39

OmniVideoBench (accuracy, %)

MethodCleanEnv. 10 dBEnv. 5 dBEnv. 0 dBSpeech 10 dBSpeech 5 dBSpeech 0 dB
OPSD41.4037.4036.2038.6036.5034.6034.30
OP-CAD (default)41.6040.1039.4041.2039.1039.5038.20
Shuffled weights41.3037.7036.6038.9036.2035.1034.70
No response normalization41.3038.6037.5039.7036.3037.7036.50
Mean41.2038.6037.5039.6036.4037.9036.30
Teacher entropy41.3038.5037.7039.6037.3037.8036.60
Teacher confidence41.1038.2037.4039.3037.0037.5036.30
AuthorsOpenReview ↗

Response to Reviewer vWn6 (Part 2/2)

A1 (continued):

The default OP-CAD model outperforms shuffled weighting across all noisy conditions on both benchmarks. Because the control preserves the weight distribution within each response, the results support assigning larger weights to the corresponding sensitive positions. Together with the example in Figure 5, this provides evidence that audio-condition sensitivity is useful for allocating supervision at positions involved in task-relevant reasoning.

Manuscript reference: Section 3.4 clarifies the role of sensitivity weights, and Section 4.5 expands the Figure 5 analysis. Appendix E.1.2 (Table 18) and E.1.6 (Table 22) report the shuffled-weight and simple teacher-weight controls, respectively.

Q2: Comparisons with random weights, uncertainty-based weights, and alternative signal combinations are needed to evaluate the design choices.

A2: Thank you for these specific suggestions. We replace each weighting rule while holding the initialization, training data, teacher targets, and curriculum fixed. The results are included in the tables under Q1.

  • Response normalization: We use raw JS scores directly, keeping λ unchanged. The default model performs better across all noisy conditions, supporting calibration of the two signals within each response.
  • Max versus Mean: We replace Max with the average of the two normalized signals. Max performs better across all noisy conditions, supporting the choice to preserve a strong signal from either intervention.
  • Teacher entropy and confidence: We construct weights from H(p)/ln|V| and max p, respectively, reusing the clean privileged teacher’s predictions. OP-CAD performs better across all noisy conditions, indicating that comparisons across audio conditions provide useful allocation information beyond these simple weighting rules.

These controls use the same weighting strength and loss reduction, with each alternative explicitly defined. We also retain the original single-signal results, including the advantage of corruption-only weighting in some OmniVideoBench conditions. The additional experiments support the default design under noisy evaluation and clarify the empirical basis for its components.

Manuscript reference: Appendix E.1.3–E.1.6 (Tables 19–22) reports the normalization, fusion, λ-sensitivity, and simple teacher-weight comparisons.

Q3: Audio-only and visual-only evaluations would help explain the relationship between robustness gains and modality use.

A3: Thank you for suggesting this analysis. We evaluate the original OPSD and OP-CAD checkpoints with audio-only (A) and visual-only (V) input, and compare them with audio-visual (AV) input on the same questions. AV and A are evaluated separately for each noise type and SNR; V is evaluated once per benchmark because it contains no audio. The results are shown below.

Daily-Omni (accuracy, %)

MethodCleanEnv. 10 dBEnv. 5 dBEnv. 0 dBSpeech 10 dBSpeech 5 dBSpeech 0 dB
OPSD / AV69.3465.8363.6660.9964.1661.5756.14
OP-CAD / AV70.0969.4267.0864.4167.0863.7459.82
OPSD / A58.9055.8153.5551.3853.9751.6347.95
OP-CAD / A59.8256.7354.4752.3054.8952.5548.87

OmniVideoBench (accuracy, %)

MethodCleanEnv. 10 dBEnv. 5 dBEnv. 0 dBSpeech 10 dBSpeech 5 dBSpeech 0 dB
OPSD / AV41.4037.4036.2038.6036.5034.6034.30
OP-CAD / AV41.6040.1039.4041.2039.1039.5038.20
OPSD / A35.2032.2031.0033.1031.5029.6029.10
OP-CAD / A35.7032.8031.6033.7032.1030.2029.70
Model (V only)Daily-Omni accuracy (%)OmniVideoBench accuracy (%)
OPSD42.7726.80
OP-CAD43.6927.40

OP-CAD performs better with joint audio-visual input than with either modality alone in every evaluated condition, indicating that it continues to benefit from complementary information under noise. Compared with OPSD, the advantage of AV over A also increases under noisy conditions, showing that the training gains are more fully realized when both modalities are available. These results provide direct empirical evidence connecting the robustness improvements with complementary modality contributions.

Manuscript reference: Appendix E.3 (Tables 24–26) presents the AV/A/V results and comparison.


Thank you again for your careful review and helpful suggestions. Your feedback has prompted a more concrete explanation of supervision allocation and a closer examination of modality contributions. We welcome further questions and would be happy to discuss any points that would help strengthen the presentation and experimental analysis.

Reviewer_sDCsOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper introduces OP-CAD, an on-policy self-distillation method for improving audio-visual question answering under acoustic interference. Following supervised warm-up, the student and frozen teacher are initialized from the same Qwen3-Omni checkpoint. The student generates responses from corrupted audio and video, while the teacher uses clean audio, the same video, and the verified answer to provide token-level targets on the student’s response prefixes. Answer-free teacher predictions under clean audio-visual, corrupted audio-visual, and visual-only contexts determine bounded token weights based on audio-removal and corruption sensitivity. Training additionally follows a mild-to-severe noise curriculum.
Evaluation on Daily-Omni and OmniVideoBench covers environmental noise and competing speech. OP-CAD improves accuracy over uniform OPSD across the reported corrupted conditions, with shared initialization, teacher targets, training data, and curriculum. The paper also provides single-signal and curriculum ablations, question-level paired statistical analyses, and analyses of clean-correct answer preservation.

Critical strengths
  1. Clear problem formulation and presentation.

The paper addresses a relevant problem in audio-visual understanding, and the proposed training framework is clearly described. The methodology is straightforward to follow, and the implementation details are reasonably documented.

  1. Reasonable experimental comparisons.

The authors compare the proposed method with several commonly used training approaches, including SFT, GRPO, and OPSD. The ablation studies and additional analyses provide some empirical support for the proposed design under the evaluated conditions.

Critical weaknesses
  1. Limited generalization to real-world noisy environments.

Both training and evaluation use the same synthetic noise augmentation pipeline, differing mainly in the noise samples. In particular, competing speech is constructed by mixing independently recorded speech into the original audio, which does not fully reflect realistic multi-speaker scenarios. Therefore, the experiments mainly demonstrate robustness to synthetic interference rather than real-world noisy environments. This significantly weakens the practical relevance of the work.

  1. Limited performance gains relative to computational cost.

Even under the authors' own synthetic evaluation settings, OP-CAD achieves only around 2.8–3.8 percentage points of improvement over OPSD, while requiring approximately 36.5% more GPU-hours. Although these gains are statistically significant, their practical value remains questionable without compute-matched comparisons, especially given the lack of real-world validation.

Questions for authors

Real-world generalization: Could the authors evaluate OP-CAD on naturally recorded noisy audio or realistic multi-speaker scenarios? Does the improvement persist when the interference distribution differs from the synthetic training pipeline?

4 comments
AuthorsOpenReview ↗

Response to Reviewer sDCs (Part 1/2)

Dear Reviewer sDCs,

We sincerely thank you for the time and care you devoted to reviewing our paper, and for your constructive suggestions on generalization to real-world noise and computational cost. Your feedback has helped us strengthen the evaluation and clarify the practical value of our approach. We have added experiments with recorded background interference and an OPSD comparison with approximately matched training compute. The corresponding revisions are included in the main paper and appendix and highlighted in blue. We address your comments below.

Q1: Training and evaluation use the same corruption pipeline; evaluation with recorded noise and more complex speech interference would strengthen the generalization evidence.

A1: Thank you for this suggestion. To evaluate generalization to complex interference beyond the training sources, we introduce audio recorded in real-world scenes from the Walk Scape YouTube channel. These walking videos cover public spaces such as streets, pedestrian areas, squares, and tourist sites under different times of day and weather conditions. Their recordings preserve naturally co-occurring conversations, footsteps, traffic, bells, and rain, including overlapping voices and environmental sounds. They therefore provide complex background interference that varies over time.

We retain these mixed background sounds and use the original corruption procedure to construct conditions at 10, 5, and 0 dB, while keeping the video, questions, options, and answers fixed. All methods are evaluated using their original trained checkpoints, without retraining on the new interference sources. The results are shown below; Daily and OVB denote Daily-Omni and OmniVideoBench, respectively.

Recorded background interference (accuracy, %)

MethodDaily CleanDaily 10 dBDaily 5 dBDaily 0 dBOVB CleanOVB 10 dBOVB 5 dBOVB 0 dB
Base67.7563.6660.8258.4037.4036.5035.1033.60
Continued SFT68.4264.3361.7458.9839.0037.0035.7034.30
GRPO68.3465.0862.5759.7341.5037.8036.5035.10
OPSD69.3466.0863.4960.7441.4038.6037.3035.90
OP-CAD70.0968.1765.6663.0741.6040.5039.0037.80

OP-CAD maintains the highest accuracy across the recorded-noise conditions on both benchmarks. This shows that the benefits of sensitivity-based supervision allocation extend to complex recorded backgrounds, including interference from multiple voices, and supports generalization beyond the training noise sources. We appreciate your suggestion, which prompted this evaluation under more realistic background interference.

Manuscript reference: Appendix E.2 (Table 23) describes the recordings, corruption procedure, and results at each SNR. The conclusion also discusses this evaluation.

AuthorsOpenReview ↗

Response to Reviewer sDCs (Part 2/2)

Q2: An OPSD comparison at matched compute is needed to assess the value of OP-CAD’s additional training cost.

A2: Thank you for raising this point. We agree that a comparison with approximately matched compute provides a more direct assessment of the additional training cost. We therefore extend OPSD by increasing its training steps until its measured GPU-hours closely match those of OP-CAD, and refer to this run as OPSD-CM. The comparison uses the same hardware, shared warm-up initialization, training data, and frozen teacher targets. Costs cover robustness training only. The training costs and evaluation results are reported below.

MethodTraining GPU-hours
OPSD393.14
OP-CAD (default)536.50
OPSD-CM537.30

Env. and Speech denote environmental noise and competing speech, respectively.

Daily-Omni (accuracy, %)

MethodCleanEnv. 10 dBEnv. 5 dBEnv. 0 dBSpeech 10 dBSpeech 5 dBSpeech 0 dB
OPSD69.3465.8363.6660.9964.1661.5756.14
OPSD-CM69.5166.8363.4162.2465.0861.4057.48
OP-CAD (default)70.0969.4267.0864.4167.0863.7459.82

OmniVideoBench (accuracy, %)

MethodCleanEnv. 10 dBEnv. 5 dBEnv. 0 dBSpeech 10 dBSpeech 5 dBSpeech 0 dB
OPSD41.4037.4036.2038.6036.5034.6034.30
OPSD-CM41.5037.2037.4039.5036.3036.6035.90
OP-CAD (default)41.6040.1039.4041.2039.1039.5038.20

Extending training improves OPSD in some conditions and leads to regressions in others, while OP-CAD retains its advantage across all noisy conditions. Under approximately matched compute, allocating the additional computation to sensitivity-based supervision is therefore more effective than simply taking more OPSD training steps. This comparison provides further evidence for the value of the proposed allocation mechanism.

Manuscript reference: Appendix E.1.1 (Tables 16–17) reports the comparison and its training costs separately from the original cost analysis.


Thank you again for your valuable time and constructive feedback. We hope the additional experiments address your concerns about generalization and computational efficiency. We welcome any further questions and would be happy to continue the discussion to improve the clarity and rigor of our work.

Thank you for the response and the additional experiments. The compute-matched comparison (OPSD-CM) addresses my concern about training cost: extra OPSD steps bring only small gains, while OP-CAD keeps a consistent advantage.

However, my main concern about real-world generalization remains. The Walk Scape experiment still mixes the recordings into clean audio with the original corruption pipeline at controlled SNRs. It therefore changes the noise samples rather than the corruption process, which is the limitation raised in my review. The gains over OPSD also become smaller under this new noise (about 2.2 points on Daily-Omni and 1.8 on OmniVideoBench, compared with 2.8–3.8 in the original setting). A more direct test would be to evaluate on real street videos taken directly from YouTube, such as street vlogs or interviews, using their original audio, where the target sounds and the interference are recorded together in the same environment. Using such videos as test samples, rather than only as a source of background noise, would better reflect practical scenarios.

I will therefore maintain my rating of 2 .

AuthorsOpenReview ↗

Follow-up response to Reviewer sDCs

Dear Reviewer sDCs,

Thank you for clarifying your concern and for acknowledging the compute-matched comparison. We apologize for misunderstanding your point in our first response.

Q1: How does OP-CAD perform on real videos evaluated with their original audio?

A1: In the Walk Scape experiment, we changed the background recordings but still mixed them into clean audio. You were asking how the model performs when it receives a real video together with its own soundtrack. Your suggestion prompted us to test exactly that input setting.

We evaluated OPSD and OP-CAD on the multiple-choice task from Radwan et al., SONIC-O1: A Real-World Benchmark for Evaluating Multimodal Large Language Models on Audio-Video Understanding (EMNLP 2026). SONIC-O1 draws on 231 YouTube videos spanning approximately 60 hours. Its 13 topics include public transportation conflicts, emergency response, restaurant service, community town halls, and workplace meetings. We used the videos themselves as test inputs, with their original soundtracks. We added no background tracks and applied no SNR adjustment.

Entries show multiple-choice accuracy (%), with the best result in each row in bold. The Qwen3-Omni baseline uses the benchmark authors’ published results.

SONIC-O1 by topic (accuracy, %)

Topic (questions)Qwen3-Omni (published)OPSDOP-CAD
Patient–doctor consultations (102)75.580.479.4
Job interviews (100)77.080.081.0
Parent–teacher conferences (230)68.370.472.2
Customer service (39)64.171.876.9
Courtroom proceedings (117)63.566.767.5
Emergency response (87)57.566.772.4
Public transportation conflicts (51)54.962.768.6
Workplace meetings (67)70.179.180.6
Housing tours (129)67.473.674.4
Restaurant service (90)57.862.267.8
Mental health counseling (65)70.878.576.9
Community town halls (189)66.168.872.0
Olympics and sports (69)60.968.172.5
All questions (1,335)66.471.373.6

OP-CAD answers more questions correctly in 11 of the 13 topics. The largest gains occur in public transportation conflicts, emergency response, and restaurant service, three settings especially relevant to your question about everyday recordings. Across all questions, OP-CAD achieves 73.6% accuracy, compared with 71.3% for OPSD.

OPSD applies the same supervision weight to every response position. OP-CAD emphasizes positions where the teacher’s prediction changes when audio is removed or corrupted. The comparison on SONIC-O1 shows that the benefit of this training choice extends to questions answered from a video’s original audio and visual context.

Manuscript reference: Section 4.6 (Table 6 in the main paper) gives the full topic breakdown and analysis.


Thank you for suggesting an evaluation that addresses this distinction directly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.