acceptodds
Under review as a conference paper at ICLR 2027

AVRT: Audio-Visual Reasoning Transfer through Single-Modality Teachers

Abstract

Recent advances in reasoning models have shown remarkable progress in text-based domains, but transferring those capabilities to multimodal settings, e.g., to allow reasoning over audio-visual data, still remains a challenge, i.a., because of the limited availability of high-quality reasoning data in targeted multimodal combinations. To address this problem, we introduce AVRT, a novel framework that generates high-quality audio-visual reasoning traces from single-modality teacher models. We generate independent vision- and audio-reasoning traces via models specialized to reason over their respective modalities and merge the resulting traces with an LLM merger model. The resulting multimodal traces are used in a supervised fine-tuning (SFT) cold start to adapt the target model to audio-visual reasoning traces first, before training it in a second reinforcement learning stage on larger-scale data. Our evaluation shows that the proposed pipeline based on generated multimodal traces for SFT allows models to achieve superior performance on various datasets, i.a., OmniBench and DailyOmni compared to RL alone, establishing a new training pipeline for audio-visual reasoning models and opening ways to train reasoning models on multimodal data.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal Learning, Reinforcement Learning, Video Understanding

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_HaEqOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduce AVRT, an audio-visual training recipe which compose of 2 training stages from SFT and RL. The framework generate independent vision and audio reasoning traces from different teacher model and then merge them with an LLM merger. The final results achieve state of the art results among comparable size models, with average 60.3 points on average across 6 audio visual benchmarks.

Critical strengths
  • With only 20K SFT samples, AVRT substantially outperforms comparable-size models, which demonstrates the data efficiency of the proposed pipeline.
  • The paper shows that although cross-modal reasoning data is rare, combining the knowledge of different single-modality teachers into one unified reasoning trace can still yield state-of-the-art performance compared with other methods.
Critical weaknesses
  • The RL stage appears rather data-inefficient: it uses substantially more training data than the SFT stage, yet yields only a marginal improvement.
  • An ablation for the SFT-only setting is missing, which is essential for isolating the actual contribution of the RL stage.
Questions for authors

The effectiveness of RL seems limited in this paper. What do the authors attribute this to?

Reviewer_qZPaOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes AVRT, a framework for transferring single-modality reasoning capabilities to audio-visual reasoning models. Specifically, the authors employ specialized audio and visual teacher models to independently generate modality-specific reasoning traces, which are subsequently integrated into unified audio-visual reasoning traces using a text-only LLM merger. These generated traces are used for supervised fine-tuning (SFT), followed by reinforcement learning with GRPO to further improve reasoning performance. Experiments on multiple audio-visual and audio-only benchmarks demonstrate consistent improvements over the base models at both 3B and 7B scales.

Critical strengths
  1. The paper addresses a practical and relevant problem of constructing high-quality reasoning data for audio-visual understanding. Leveraging specialized single-modality teachers offers a reasonable alternative to relying on a powerful native multimodal teacher.
  2. The proposed approach achieves consistent improvements across multiple benchmarks and model scales. The results demonstrate that the generated reasoning traces provide useful supervision for improving downstream performance.
  3. The paper provides relatively comprehensive experimental analyses, including ablations on training stages, reasoning trace types, merger models, reward components, and data filtering strategies.
Critical weaknesses
  1. The novelty of the method is limited. The proposed framework primarily combines existing techniques, including reasoning trace generation with specialized teacher models, LLM-based information aggregation, SFT, and GRPO. While applying this pipeline to audio-visual reasoning is practically meaningful, the methodological novelty appears relatively limited. In particular, the LLM merger is implemented through prompting an existing language model, without introducing a dedicated mechanism for cross-modal information alignment or reasoning integration. The authors should better clarify the methodological contributions beyond combining existing components into a unified training pipeline.
  2. The paper claims that the proposed method enables genuine cross-modal reasoning transfer. However, the experimental evidence primarily relies on improvements in downstream answer accuracy, which does not necessarily demonstrate enhanced cross-modal reasoning capabilities. For example, Table 4 shows that AVRT improves performance under both audio-only and visual-only inputs, suggesting that part of the improvement in audio-visual settings may stem from enhanced single-modality understanding rather than stronger cross-modal integration.
Questions for authors

See weakness

Reviewer_hPMsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes AVRT for transferring reasoning from separate audio and visual teachers into an audio-visual student. A text-only LLM merges the two teacher traces. The authors retain examples where both teachers answer correctly. These traces are used for supervised fine-tuning followed by GRPO on the full AVQA training set.

The resulting 3B and 7B models improve over their Qwen2.5-Omni baselines across several audio-visual and audio benchmarks. The paper also studies teacher-trace types, filtering, merger models and reasoning-length rewards.

Critical strengths
  1. Using separate audio and visual teachers avoids requiring a strong native audio-visual reasoning teacher. The text merger also provides a simple way to standardize different teacher outputs. This is a useful approach for modality combinations with limited reasoning data.

  2. The audio-visual average improves from 49.9% to 57.7% for 3B and from 55.3% to 60.3% for 7B. Audio-only performance also improves. On OmniBench, the full approach exceeds RL-only training by 4 points. The appendix adds useful controls and openly examines hallucinations in the generated traces.

Critical weaknesses
  1. Table 5 compares merged traces against either teacher alone. It does not compare against simply concatenating both traces or combining their factual descriptions without explicit reasoning. These controls would help establish whether the gain comes from cross-modal reasoning composition or from providing more useful information during SFT.

  2. The answer-only SFT plus RL control reaches only 12.5% on OmniBench. The appendix attributes this partly to a mismatch between SFT outputs and the RL format reward. The format only control also collapses. These results show problems under the chosen training setup. They provide weaker evidence that reasoning supervision itself is necessary. A stable control with compatible formatting and rewards would be more convincing.

  3. The filter retains questions that both teachers answer correctly from their own modality. Appendix B.2 acknowledges this. The missing-modality results are useful but do not isolate reasoning that requires combining complementary evidence. Also, the hard-subset gains are compared with the base model rather than RL-only training. That comparison would better separate the contribution of merged traces from subsequent RL.

  4. Section 4.5 reports 64 questions where both teachers are correct and 456 where exactly one is correct. This implies a summed teacher accuracy of 51.3% across 1139 questions. Table 3 instead reports 33.5% and 28.9%, which sum to 62.4%. The authors should clarify whether different runs or prompts were used.

  5. Changing the merger from Gemma3-12B to Qwen2.5-14B produces a large difference. However, these models also differ in output quality and style. This comparison does not isolate shared architecture or tokenization as the cause. The explanation should be presented as a hypothesis unless supported by more controlled evidence.

  6. The paper should specify the RL update count and the bonus value in the length reward. It should also clearly report student evaluation settings for frame sampling, audio duration and output length. The eight-frame setup is described for teacher generation. The corresponding evaluation protocol is not clear enough, especially for long videos.

Questions for authors
  1. How does the method compare with concatenated teacher traces or combined audio-visual descriptions on the same training examples?

  2. Can the answer-only control be trained with compatible output formatting and rewards so that the comparison is not dominated by collapse?

  3. Can you reconcile the OmniBench subset counts with Table 3? Is it also possible to report RL-only performance on these subsets?

  4. Is there evidence that the merger gap comes from model-family alignment after controlling for trace quality and length?

  5. Please provide the RL training duration, length-reward bonus and evaluation input settings.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.