- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper targets token compression for omnimodal LLMs (Qwen2.5-Omni). Its argument is that audio and video alternate between complementary chunks, where each modality carries unique information, and redundant chunks, where they overlap. It claims existing methods use one fixed strategy throughout a sequence. OmniZip-style methods let one modality guide the other. OmniScope-style methods compress each modality independently.
Critical strengths
-
The method is training-free and plug-and-play. It runs before the LLM forward pass and its core operations are cheap (norms, mean-pooling, cosine). There is no retraining and no dependence on a specific attention implementation.
-
The two-level framing is reasonable. Separating "where to spend tokens" (temporal budget) from "how to spend them" (selection mode) is clean, and the appendix gives enough detail on smoothing and merging to be reproducible in principle.
Critical weaknesses
-
The central premise, complementary versus redundant chunks, is never measured. \rho is not validated. Contribution 1 says "we reveal how audio-visual correlation changes over time". The evidence is Figure 2, which is explicitly a schematic with one hand-picked video. There is no histogram of \rho, no per-video trajectory and no annotation of what a "complementary" or "redundant" chunk is. \rho is the cosine of two mean-pooled embeddings from different encoders. In Qwen2.5-Omni, the audio and vision towers are not contrastively aligned with each other. Their projected embeddings need not be geometrically comparable at all. Cosine between mean-pooled vectors is usually dominated by shared anisotropy or mean-offset components, not semantic overlap.
-
The "adaptive" part is mostly Separate-only, and the gain may come from merging, not cross-modal redundancy removal. Over Separate-only, the full model gains 0.3 avg points at 45% (49.2 vs 48.9), 0.5 at 35% and 1.0 at 25% (Table 2). Per benchmark, much of the 45% gain comes from ShortVid and LVOmniBench. The joint branch bundles three things: cross-modal pooled ranking, cross-modal assignment, and similarity-weighted feature merging. Merging is a known trick (ToMe-style) that could help a separate path too.
-
Baseline coverage is thin, and the comparison may not be level. The related work lists OmniSIFT, OmniFit, OmniSelect, Omni-Prune and PRIM, which are the closest prior work. None appear in the experiments. The claim "outperforms existing OmniLLM compression baselines" rests on only two omni baselines, OmniZip and OmniScope. FastVID and DyCoke are video methods applied "naively to both audio and video". Audio is a 1-D sequence with no spatial structure, so these are weak straw baselines. They mainly inflate the margin.
Questions for authors
-
Show the real distribution of \rho: a histogram across all chunks, and per-video trajectories. How bimodal is it, and what is the within-video versus across-video variance? Does ρ track anything beyond a global offset between the two embedding spaces?
-
Report Separate+merging and Joint-without-merging ablations, plus a joint branch with within-modality-only merging. Which component produces the gain over Separate-only?
-
Add OmniSIFT, OmniFit, OmniSelect and Omni-Prune, or explain why they cannot be run. Add random and uniform-temporal baselines and a "keep all audio, prune only video" baseline. Match all methods on actual tokens entering the LLM or on FLOPs.
-
How are position IDs and TMRoPE handled after pruning and merging? Does cross-modal merging hurt speech-heavy questions?
-
How does OmniTrim relate to a query-aware variant? Would adding query relevance to the IAB or CAS signals help, and how does the method fare against query-aware baselines when they are given their best settings?