- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces OmniReasoning, a framework for evaluating and improving audio-visual joint reasoning in omni-modal large language models (Omni-LLMs). Three main contributions:
- OmniReasoningBench: 1,150 questions across 19 task types, built so both audio and visual evidence are genuinely required;
- OmniQA: a data engine for evidence-grounded QA pairs with timestamped audio-visual clue chains (112K SFT, 19K human-validated RL);
- Modality-Factored Self-Distillation (MFSD): an RL method that assigns credit by modality, explicitly rewarding non-additive cross-modal interactions.
The authors show OmniReasoningBench actually requires joint reasoning. Visual-only accuracy falls to 17.2% MCQ / 8.7% OE versus 42.7% / 41.9% with joint input. Their model (OmniReasoning-30B-A3B) gains +9.3 points overall on the benchmark and +6.1–12.8 points across four public video benchmarks over the base Qwen3-Omni model.
Critical strengths
The biggest contribution here is just proving the benchmark does what it claims. The multi-gate verification pipeline (clue validity + shortcut removal under four modality conditions) plus the Table 6 analysis gives real evidence that OmniReasoningBench isn't gameable on a single modality. The contrast with prior benchmarks is stark. If the rest of the paper were weaker, this alone would probably be worth publishing.
Table 3 is the most important result table in the paper. GRPO and RLSD both gain on OmniReasoningBench but lose ground on general video benchmarks (OmniVideoBench drops from 47.0 to 44.8/45.4). MFSD is the only method that moves both up at the same time. The formulation is also solid.
Critical weaknesses
The paper's central argument is about improving joint reasoning. But the gains on open-ended questions are basically flat (+0.7 pts on RbV OE). OmniReasoning-30B-A3B (42.5%) also sits ~15 points below Gemini-3.7-Flash (57.6%). If the method primarily helps with MCQ discrimination rather than generative reasoning, the "joint reasoning" framing is doing more work than the results support.
OmniQA uses Gemini-3.1-Pro for event segmentation and Qwen3.8 for question generation. The SFT-112K dataset has synthetically generated reasoning chains the paper acknowledges may be noisy or hallucinated. The RL-19K set needs human validation. It's not clear how much of the improvement is from the MFSD method versus knowledge implicitly distilled from stronger proprietary models. Reproducibility also takes a hit.
Questions for authors
Q1: Table 6 breaks down modality ablations for Qwen3.5-Omni-Plus and Gemini but not for OmniReasoning-30B-A3B. What happens when you run the proposed model on audio only? If audio-only performance jumped substantially after MFSD training, that would suggest the method is improving audio processing rather than cross-modal integration.
Q2: Keeping encoders frozen throughout SFT and RL is a significant constraint. Has any experimentation been done with partially unfreezing them, especially during RL? Given MFSD explicitly targets token-level cross-modal interaction, it seems like encoder adaptation could either amplify or expose what's actually going on.
5 comments
Official Comment by Authors for Reviewer EPhW (2/2)
Question 1: What happens when you run the proposed model on audio only?
MFSD mainly improves the combination of the two modalities rather than audio processing, and the additional experiment you suggested confirms this. Following the protocol of Table 6, we evaluated the SFT checkpoint and OmniReasoning-30B-A3B (+MFSD) on reasoning over video under joint (A+V), audio-only, and visual-only inputs:
| Model | Type | A+V | Audio-only | Visual-only |
|---|---|---|---|---|
| SFT checkpoint | MCQ | 37.87 | 24.27 | 19.73 |
| OmniReasoning-30B-A3B (+MFSD) | MCQ | 48.00 (+10.13) | 26.67 (+2.40) | 20.80 (+1.07) |
| SFT checkpoint | OE | 22.4 | 9.6 | 3.47 |
| OmniReasoning-30B-A3B (+MFSD) | OE | 30.13 (+7.73) | 10.67 (+1.07) | 3.73 (+0.26) |
We have two observations:
- Audio-only accuracy does not decrease**.** After MFSD, audio-only accuracy rises by only +2.40 (MCQ) and +1.07 (OE), and visual-only accuracy by only +1.07 and +0.26. Our audio-only MCQ accuracy (26.67) is at the same level as frontier Omni-LLMs in Table 6 (20.53–26.13).
- Joint-input accuracy rises far more. It rises by +10.13 (MCQ) and +7.73 (OE), about 4× and 7× the largest single-modality gain, so the improvement lies in combining the two modalities rather than in processing either one alone.
Question 2: Has any experimentation been done with partially unfreezing the encoders, especially during RL?
Freezing or unfreezing the encoders has little effect on our results, because our training targets at cross-modal, semantic, joint reasoning and thus provides the encoders no additional visual-level & audio-level supervision. Visual supervision would arise only if joint reasoning relied on low-level visual details such as clothing texture or depth, which our task does not involve, and the ViT and AuT of Qwen3-Omni have already been well trained on large amounts of data. To verify this, we unfroze the ViT and AuT and re-ran MFSD with everything else identical (same SFT checkpoint, data, and budget):
| Setting | OmniReasoningBench | OmniVideoBench | LVOmniBench | Video-MME-v2 | Video-MMMU |
|---|---|---|---|---|---|
| MFSD, encoders frozen | 42.5 | 50 | 44.7 | 44 | 74.4 |
| MFSD, encoders unfrozen | 41.9 | 48.9 | 44.3 | 43.7 | 74.8 |
We have two observations:
- Unfreezing brings no further gain. The two settings differ by at most 1.1 points on any benchmark, and the frozen setting is better on four of the five.
- The gains come from cross-modal integration in the language model. Unfreezing the encoders neither amplifies nor removes the improvement of MFSD, which suggests that the bottleneck lies in how the language model integrates the audio and visual evidence, exactly what MFSD targets.
If our responses already address your concerns, we hope that you could consider raising the score. Thank you again for your positive assessment and helpful suggestions!
Dear Reviewer,
The discussion period will close in less than 24 hours. We would greatly appreciate any remaining thoughts or follow-up on our responses before the deadline. We remain fully open to discussion and are happy to clarify any points or provide additional details if helpful. Thank you again for your time and thoughtful review.
Official Comment by Authors for Reviewer EPhW (1/2)
We sincerely appreciate your constructive feedback and valuable suggestions.
We acknowledge that the gain on open-ended questions is limited, especially on RBV-OE (+0.7). However, this reflects the scarcity of open-ended training data, instead of a lack of joint reasoning. Open-ended RBV data is hard to construct, so it is much scarcer than MCQ data in our training set, and we will make scaling up open-ended data a key direction for future work.
This does not contradict our central claim, for three reasons:
Both the high-quality OmniReasoning data and the MFSD method contribute, and the existing ablation in Table 4 of the main paper already separates the two:
We have three observations: