- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes OmniZPD, an RL-based token compression framework for omni-modal LLMs (audio + video + text). The main idea is to train a small unified selector (2.68M params) that picks which audio-visual tokens to keep, optimized with a reward that combines task accuracy, prediction confidence, and a self-distillation signal computed along the model’s own reasoning chain. Specifically, the same frozen OmniLLM acts as both teacher (full tokens) and student (compressed tokens), and the reward measures how much the next-token distributions diverge when conditioned on the student’s reasoning prefix. Training is two-stage: first a supervised warm-up from pseudo-labels, then RL fine-tuning with Gumbel Top-K sampling. The backbone stays frozen throughout. They evaluate on five omni-modal benchmarks using Qwen3-Omni-30B, showing 91.8% accuracy retention at 20% token budget and 2.79× prefill speedup. They also show some generalization to MiniCPM-o 4.5
Critical strengths
The self-distillation reward is a good idea. Conditioning both teacher and student on the student’s own reasoning prefix and measuring token-level KL divergence gives you a dense supervision signal that doesn’t require the final answer to be correct. This is a real improvement over just using accuracy rewards, and the ablation in Table 2 backs this up -- each reward component adds something. The on-policy nature of the distillation (using student-generated prefixes rather than teacher-generated ones) is a nice design choice that I think matters more than the paper emphasizes.
The practical numbers are solid. 2.79× prefill speedup while keeping 91.8% of accuracy at 20% tokens is a good trade-off. The selector is tiny (2.68M params), inference is just a single deterministic forward pass, and the ratio-conditioned design means you don’t retrain for different budgets. OmniZPD beats all evaluated baselines across all benchmarks and retention ratios, which is clean even if the baseline set is incomplete.
Critical weaknesses
The experimental comparisons have holes. SCORE, TSPO, and TOP-RL are discussed as related RL-based compression methods but never compared against. The cross-architecture experiment (Table 4) also only covers 3 of 5 benchmarks with no explanation for why VideoMME and OmniVideoBench were dropped. And the paper claims the unified selector beats per-modality alternatives, but there’s no ablation actually testing this — it’s just asserted.
Too many critical details are missing for reproducibility -- the reward weights, the self-distillation temperature, the retention ratio curriculum, the Gumbel temperature schedule. There’s no sensitivity analysis on any of them. The total training cost is also unknown. I understand that some of this might be in a future camera-ready, but as it stands, I can’t tell whether the method is robust to these choices or whether it required careful tuning to get the reported numbers.
Questions for authors
-
Can you explain the token count discrepancy in Table 3? OmniZPD shows 2,159 tokens at 20% retention while baselines show ~3,200+. Are these methods actually operating at the same compression budget?
-
Why are SCORE, TSPO, and TOP-RL excluded from comparisons? If it’s a code availability issue, just say so. But if these methods were available and not compared, that’s a problem.
-
What are the actual values of λ_acc, λ_conf, λ_sd and τ_sd, and how sensitive is performance to them? Even a small ablation (e.g., varying one weight while fixing others) would help. If the method only works in a narrow hyperparameter regime, that changes the story.