- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies audio-visual token compression in omni-modal large language models and proposes JEC-Omni, a training-free method that compresses tokens before they enter the LLM, motivated by the joint entropy of audio and video. For video, the method groups every two adjacent frames: the first frame removes spatial redundancy, and the second frame searches nearby positions in the previous frame for similar content to remove redundancy caused by motion-induced displacement. For audio, the method combines temporal redundancy with the similarity to the retained visual content and removes the parts deemed redundant. The retention ratios of the two modalities are preset. The authors evaluate the method on several audio-visual understanding benchmarks with two families of omni-modal models. The results show that it approaches the full model on some benchmarks and reduces time-to-first-token latency.
Critical strengths
- Simple, training-free and easy to deploy. Compression is performed before the LLM using only local feature similarity, without attention maps or training, and works directly with FlashAttention. The paper measures end-to-end time-to-first-token latency including the encoders, which is closer to real deployment than reporting token counts or FLOPs alone.
- Good performance retention on WorldSense. Under moderate compression ratios, the results are close to the full model and better than existing audio-visual compression methods.
- Thorough module-replacement experiments. The paper fixes the compression method of one modality while replacing that of the other, and compares different redundancy signals, which explains the role of each module better than comparing whole pipelines only.
Critical weaknesses
- Limited novelty: both modules have very close published work. MAST-P handles redundant content across frames at different positions, which video methods such as FastVID [1] already do, yet the paper avoids comparing with them by arguing that "audio and video redundancy are distributed differently", and only compares with DyCoke [2] and OmniZip [3], which match tokens at the same position. The paper also claims that existing methods do not use cross-modal signals to decide what to remove, but OmniSIFT [4] removes audio conditioned on the retained video, the same idea as SICM-P with a learned scorer instead of cosine similarity. Excluding these parts, the new local maximum-similarity matching and cosine criterion are exactly the parts with the smallest gains (W2, W5). I suggest redefining the contributions and comparing with the methods above.
- MAST-P is only marginally better than random pruning, within experimental variance. In Table 4, MAST-P outperforms random video pruning by only 0.6 (3B) and 1.0 (7B) points; in Table 3, both variants without motion search underperform random pruning, and the table itself is non-monotonic (with spatial-only pruning, the 35% budget result is lower than the 25% one). Experimental variance is thus comparable to the 0.5–0.7 point gain from motion search, and the claim that MAST-P "significantly outperforms all baselines" (l.373) is not supported.
- Hyperparameters and ablations are all determined on WorldSense, where the largest advantage also appears. Tables 3–6 and the sensitivity analysis use only WorldSense, and the default hyperparameters are its optimum; the largest advantage over DASH [5] is also on WorldSense, while results on other benchmarks are mixed (Table 1). This amounts to tuning on the test set. I suggest selecting hyperparameters on held-out data and reporting ablations with confidence intervals on other benchmarks.
- The joint-entropy framework does not lead to the algorithm used. When selecting a subset, all terms in H(V,A)=H(V)+H(A)−I(V;A) change together: removing tokens lowers I(V;A) but also lowers H(A), so "minimizing mutual information to maximize joint entropy" is not a valid selection objective, and the paper does not explain how the cosine scores relate to entropy. I suggest narrowing this title-level claim to "an information-theory-inspired empirical rule".
- The cross-modal criterion of SICM-P is not validated, and its scope is small. Eq. (9) directly compares outputs of two independent encoders by cosine similarity without evidence that they are aligned, so the score may mainly reflect a systematic offset between modalities. In Table 5, using only the cross-modal term performs on par with using only the intra-modal term, and fusing them adds only 0.2–0.3 points. Moreover, from §4.1, audio accounts for only about 15% of all tokens, so SICM-P affects only 5–7% of the tokens, and "eliminating cross-modal redundancy" concerns only a small part of the budget. References [1] Leqi Shen, Guoqiang Gong, Tao He, et al. FastVID: Dynamic Density Pruning for Fast Video Large Language Models. In NeurIPS, 2025. [2] Keda Tao, Can Qin, Haoxuan You, Yang Sui, Huan Wang. DyCoke: Dynamic Compression of Tokens for Fast Video Large Language Models. In CVPR, 2025. [3] Keda Tao, Kele Shao, Bohan Yu, Weiqiang Wang, Jian Liu, Huan Wang. OmniZip: Audio-Guided Dynamic Token Compression for Fast Omnimodal Large Language Models. In CVPR, 2026. [4] Yue Ding, Yiyan Ji, Jungang Li, et al. OmniSIFT: Modality-Asymmetric Token Compression for Efficient Omni-Modal Large Language Models. In ICML, 2026. [5] Bingzhou Li, Tao Huang. DASH: Dynamic Audio-Driven Semantic Chunking for Efficient Omni-Modal Token Compression. arXiv:2603.15685, 2026.
Questions for authors
- Compared with OmniSIFT [4], which removes audio conditioned on the retained video, and video methods such as FastVID [1], which handle redundant content at different positions, what are the key differences and additional benefits of MAST-P and SICM-P? Could you compare with them under the same budgets, or explain why such a comparison is not needed?
- Could you compare MAST-P, random video pruning and same-position temporal matching on at least one benchmark other than WorldSense? Please select hyperparameters on held-out data and report paired confidence intervals.
- Keeping the retained video tokens, the audio budget and the intra-modal scores fixed, if the video is replaced by a temporally shuffled version or by the video of another sample, does the cross-modal term still bring gains? In dialogue-heavy tasks, is the information in the audio removed due to high similarity actually contained in the visual content?
- The entropy of which random variable does the method actually optimize, and how do the cosine scores relate to it? If entropy is only a motivation, would you be willing to tone down the "joint entropy optimization" claim?
- For each model, including Qwen3-Omni, what are the actual numbers of video and audio tokens and the overall retention ratio?