- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how to reduce the cost of audio-video reasoning without losing the short acoustic cues and visual context needed to answer a question. The authors introduce UGC-AVQA, a dataset of 1,000 user-generated videos and 4,816 multiple-choice QA pairs. For the benchmark split, they retain questions that Gemini-3.1-Flash cannot answer from the muted video, with human review of 30% of the retained items. The intent is to make audio-visual association, rather than visual recognition alone, central to the evaluation.
The proposed compressor, OMAC, selects query-relevant frames, retains distinctive visual tokens and pooled frame memories, and merges nearby audio tokens into anchors. The retained visual memory then determines how the audio budget is distributed over time. O-MARC adds training on these compressed inputs: the same policy generates full-token and compressed rollouts, and their reward difference is used to reshape the GRPO advantage.
The experiments cover three Qwen-Omni model sizes. At 30% token retention, OMAC improves on OmniZip by 1.5–2.4 points on the four-benchmark average, with similar inference cost and roughly 35% lower latency and memory use than full-token inference. With about 3k RL examples, O-MARC raises the 3B average from 44.1 to 45.8. The larger-data 7B setting, using 104k SFT examples followed by 3k RL examples, reaches 51.1 compared with 48.5 for OmniSIFT.
Critical strengths
I find the dataset contribution more compelling than the training contribution. Questions about brief vocal reactions, speech-defined intervals and cross-scene alignment are useful tests of what compression actually preserves. These cases occur in existing omni benchmarks, but making them the focus of a UGC benchmark is worthwhile. The gap between large and small models in Table 2 also suggests that there is room for improvement. A release with the promised URLs and annotations would be useful beyond this particular method.
OMAC itself is a practical design. It requires no extra training and improves on OmniZip across all three model sizes without noticeably increasing inference cost. The gap widens under stronger compression in Table 4, and Table 7 gives some support for allocating audio capacity from the retained visual memory rather than using direct guidance alone. These results make the compressor worth considering even if the case for O-MARC is less clear.
Critical weaknesses
My main concern is that the average improvement over full-token inference does not establish the benefit of compression distillation. On the 3B model, UGC-AVQA rises from 49.0 to 57.9, but Daily-Omni falls from 53.8 to 52.3, OmniVideoBench from 31.6 to 31.0, and WorldSense is essentially unchanged. The 7B model trained on 3k examples shows a similar pattern. Since the RL data comes from UGC-AVQA, the improvement could largely be ordinary in-domain training. The crucial comparisons are GRPO on the same data with full-token inputs, and GRPO with OMAC inputs but no advantage shaping. Without them, I cannot tell how much of the gain comes from the proposed training objective. The larger-data comparison with OmniSIFT is also difficult to interpret without knowing the source and evaluation overlap of the 104k SFT examples or whether the baseline had comparable access to UGC-AVQA training data.
I am also not convinced that the training method is sufficiently distinguished from MARC. That work already compares full and compressed rollouts through C-GRPO, so the new contribution here appears to be the particular advantage adjustment and the extension to audio-video inputs. This makes the missing ablation of the adjustment especially important. In Eqs. 7–9, the bonus requires both a positive advantage and a lower reward under compression. With an accuracy-plus-format reward, it is not obvious how often both conditions hold, or whether the bonus mainly captures format differences. The ambiguity about which branch supplies the advantage and receives the gradient makes this harder to assess. The aggregate training curves in Figure 5 do not resolve that question.
Questions for authors
Could you clarify which branch's rollouts receive the shaped advantage in Eqs. 7–9, how full-token and compressed samples are paired, and how often the shaping term is non-zero? The comparisons that would most affect my assessment are full-token GRPO on the same 3k examples and compressed-input GRPO with the shaping weight set to zero. A sensitivity check on that weight would help as well.
What are the source, licensing conditions and possible evaluation overlap of the 104k SFT examples? Was OmniSIFT trained with access to the UGC-AVQA training split? I would also appreciate clarification of how the easier training examples and harder benchmark examples are separated, including any overlap of uploaders or channels.
For the benchmark, can you report text-only and audio-only results, and muted-video results from models other than the filtering model? Those checks would help distinguish genuine dependence on both modalities from failures specific to Gemini-3.1-Flash. Please also give the human-review correction rate and annotator agreement. Finally, what varies between the two evaluation runs, and how large is the resulting variation in Table 3?
In my personal view, this paper is a borderline reject, however, since there is no “borderline reject” option this time. I would raise my score if the questions are answered satisfactorily, I am provisionally giving it a score of 2.