acceptodds
Under review as a conference paper at ICLR 2027

OmniZPD: Token Compression via Policy Optimization and Self-Distillation for Omni-Modal Reasoning

Abstract

Omni-modal large language models (OmniLLMs) suffer from high inference costs due to dense audio-visual token redundancy. Existing token compression methods often discard reasoning-critical evidence, as sparse final-answer rewards fail to supervise intermediate thinking processes. We propose OmniZPD, an efficient compression framework integrating policy optimization and self-distillation. OmniZPD features a unified, ratio-conditioned selector that jointly evaluates audio-visual token importance based on queries, compression budgets, and local cross-modal contexts. Keeping the backbone frozen, we optimize the selector using a bounded reward that combines task correctness with self-distillation feedback, enforcing consistent reasoning predictions between full and compressed inputs. At inference, a single trained selector deterministically prunes tokens and flexibly adapts to varying retention budgets without retraining. Experiments across multiple omni-modal reasoning benchmarks show that OmniZPD significantly outperforms existing baselines, achieving a 2.79× prefill speedup while retaining 91.8% of the full-input accuracy with merely 20% of audio-visual tokens.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Omni-modal Large Language Models, Token Compression, Policy Optimization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_LFp3OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes OmniZPD, an RL-based token compression framework for omni-modal LLMs (audio + video + text). The main idea is to train a small unified selector (2.68M params) that picks which audio-visual tokens to keep, optimized with a reward that combines task accuracy, prediction confidence, and a self-distillation signal computed along the model’s own reasoning chain. Specifically, the same frozen OmniLLM acts as both teacher (full tokens) and student (compressed tokens), and the reward measures how much the next-token distributions diverge when conditioned on the student’s reasoning prefix. Training is two-stage: first a supervised warm-up from pseudo-labels, then RL fine-tuning with Gumbel Top-K sampling. The backbone stays frozen throughout. They evaluate on five omni-modal benchmarks using Qwen3-Omni-30B, showing 91.8% accuracy retention at 20% token budget and 2.79× prefill speedup. They also show some generalization to MiniCPM-o 4.5

Critical strengths

The self-distillation reward is a good idea. Conditioning both teacher and student on the student’s own reasoning prefix and measuring token-level KL divergence gives you a dense supervision signal that doesn’t require the final answer to be correct. This is a real improvement over just using accuracy rewards, and the ablation in Table 2 backs this up -- each reward component adds something. The on-policy nature of the distillation (using student-generated prefixes rather than teacher-generated ones) is a nice design choice that I think matters more than the paper emphasizes.

The practical numbers are solid. 2.79× prefill speedup while keeping 91.8% of accuracy at 20% tokens is a good trade-off. The selector is tiny (2.68M params), inference is just a single deterministic forward pass, and the ratio-conditioned design means you don’t retrain for different budgets. OmniZPD beats all evaluated baselines across all benchmarks and retention ratios, which is clean even if the baseline set is incomplete.

Critical weaknesses

The experimental comparisons have holes. SCORE, TSPO, and TOP-RL are discussed as related RL-based compression methods but never compared against. The cross-architecture experiment (Table 4) also only covers 3 of 5 benchmarks with no explanation for why VideoMME and OmniVideoBench were dropped. And the paper claims the unified selector beats per-modality alternatives, but there’s no ablation actually testing this — it’s just asserted.

Too many critical details are missing for reproducibility -- the reward weights, the self-distillation temperature, the retention ratio curriculum, the Gumbel temperature schedule. There’s no sensitivity analysis on any of them. The total training cost is also unknown. I understand that some of this might be in a future camera-ready, but as it stands, I can’t tell whether the method is robust to these choices or whether it required careful tuning to get the reported numbers.

Questions for authors
  1. Can you explain the token count discrepancy in Table 3? OmniZPD shows 2,159 tokens at 20% retention while baselines show ~3,200+. Are these methods actually operating at the same compression budget?

  2. Why are SCORE, TSPO, and TOP-RL excluded from comparisons? If it’s a code availability issue, just say so. But if these methods were available and not compared, that’s a problem.

  3. What are the actual values of λ_acc, λ_conf, λ_sd and τ_sd, and how sensitive is performance to them? Even a small ablation (e.g., varying one weight while fixing others) would help. If the method only works in a narrow hyperparameter regime, that changes the story.

Reviewer_JbWCOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes OmniZPD, an audio-visual token compression framework for omni-modal reasoning. A lightweight, query- and budget-conditioned selector is trained while keeping the backbone frozen. Training combines answer correctness, confidence, and a self-distillation reward that compares full-input and compressed-input predictions along student-generated reasoning prefixes. At inference, the selector deterministically retains tokens under the specified budget. Experiments on five benchmarks show consistent improvements over the evaluated compression baselines.

Critical strengths
  1. Consistent improvements across tasks and budgets. OmniZPD outperforms the evaluated compression baselines on five benchmarks at three retention ratios. The frozen backbone and lightweight selector also make it practically appealing.
  2. Reasonable training design with supporting ablations. Correctness, confidence, and self-distillation provide complementary feedback. Table 2 shows additional gains from self-distillation beyond the answer-based rewards.
Critical weaknesses
  1. The contribution beyond existing learned compression methods is not sufficiently established. SCORE and TSPO already use RL to train lightweight selectors with frozen backbones. Comparisons mainly against training-free methods do not isolate the advantages of OmniZPD’s training objective. The four-method voting ensemble used for initialization is also not evaluated directly.
  2. Preservation of reasoning-critical evidence is insufficiently verified.** Prediction consistency does not necessarily imply faithful reasoning or retention of necessary evidence. Final-answer accuracy and qualitative examples provide limited support for this claim, particularly without comparison against simpler answer-only supervision.
Questions for authors
  1. With the selector, initialization, and training budget fixed, how does the proposed objective compare with a SCORE-style objective or reference-answer likelihood reward?
  2. How well does the four-method voting selector perform directly? How much does policy optimization improve over this initialization?
  3. Does matching distributions over reasoning prefixes outperform matching only over the final answer? Can the authors show which necessary evidence the former preserves better?
  4. The reported 2.79× speedup concerns prefill. What is the end-to-end latency reduction when audio-video encoding, token selection, and answer generation are included?
Reviewer_HJRuOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies the audio-visual token compression for multimodal reasoning models, proposing OmniZPD: under the condition of freezing the backbone model, train a lightweight selector controlled by queries, local cross-modal context and retention ratio. The training first conducts cold-start based on the voting results of multiple compression methods, and then optimizes using the strategy based on RLOO, combining with answer correctness, answer confidence and self-distillation rewards to update the selector. Among them, the self-distillation reward compares the prediction distributions of the complete input and the compressed input under the same student reasoning prefix. In the inference stage, only a deterministic token selection is required. The paper reports consistent accuracy improvements on five benchmarks. When retaining 20% audio-visual tokens, the average accuracy reaches 91.8% of the complete input model; on OmniVideoBench, prefill achieves a 2.79-fold acceleration. The authors also re-trained the selector on MiniCPM-o 4.5 to verify the applicability of this training process to another backbone.

Critical strengths

1, The supervision design addresses a real limitation of answer-only rewards. Evaluating the full-input model on student-generated reasoning prefixes is a sensible way to measure compression effects at states the compressed model actually encounters. This can distinguish token subsets that produce the same final answer but preserve different amounts of predictive information. Applying this signal to the selection policy, rather than updating the language model, is an interesting aspect of the work.

2, The main results are consistent across benchmarks and compression budgets. In Table 1, OmniZPD achieves the best reported compressed accuracy on all five benchmarks at all three retention ratios. At 10% retention, it improves performance retention from 83.6% for the strongest baseline to 85.9%, with advantages also present at 20% and 30%. This consistency is more persuasive than a large improvement at a single operating point.

3, The inference procedure is practical. The selector has only 2.68M parameters and uses a single deterministic Top-K pass at deployment. Neither policy sampling nor teacher evaluation is required during inference, and the backbone remains unchanged. Supporting multiple retention ratios with one selector is also useful. The MiniCPM-o 4.5 results provide additional evidence that the training procedure can work with another backbone.

Critical weaknesses

1, The central claim about reasoning faithfulness is not directly established. Equation (6) measures predictive consistency under a shared prefix. However, consistency does not necessarily imply that the retained tokens support correct, grounded reasoning. If the student has already generated an incorrect assumption, the full-input teacher is also conditioned on that assumption; agreement may then reflect the linguistic prefix rather than preserved audio-visual evidence. Averaging KL divergence across the entire reasoning trace may also dilute discrepancies at a few evidence-critical positions. The experiments mainly evaluate final-answer accuracy. The qualitative examples show selected tokens and answers, but do not directly assess the grounding of intermediate reasoning. Evidence-localization measurements or controlled removal of critical audio-visual cues would provide stronger support for the paper’s main interpretation.

2, The ablations do not sufficiently isolate the distinctive contribution of reasoning-level self-distillation. Table 2 is useful, but adding self-distillation on top of correctness and confidence rewards improves accuracy by 0.85, 0.85, and 0.66 percentage points on the three reported benchmarks. These gains warrant multiple-seed evaluation, which is currently missing. More importantly, there is no comparison between distribution matching along reasoning prefixes and matching only at the final-answer position. Without this control, it remains unclear whether intermediate reasoning supervision is essential. The effects of local cross-modal context, shared scoring, and retention-ratio conditioning are also not separately evaluated. The complete system works well, but the necessity of its individual design choices is less clear.

3, The claimed dynamic allocation of audio-visual budgets does not match the selection rule. Section 3.2 and Algorithm 2 retain approximately rNmrN_m tokens independently for each modality. Consequently, audio and video use the same retention ratio, and the selector cannot shift the overall budget toward audio or video according to the question. Shared scoring and cross-modal context can influence which tokens are selected, but they do not constitute adaptive allocation between modalities. The conclusion’s statement that the method “dynamically balances audio-visual token budgets across modalities” therefore appears stronger than the implemented mechanism supports. This matters because asymmetric information density is a central motivation of the paper.

4, The efficiency evaluation does not yet establish end-to-end inference savings. Table 3 reports prefill latency, but the timing boundaries are unclear. Compression occurs after modality encoding, so the full audio-visual encoding cost remains. For a reasoning model, reasoning length and decoding time may also contribute substantially to total latency. A 2.79× prefill speedup should therefore not be interpreted as an equivalent end-to-end speedup.

Questions for authors

1, Does the self-distillation reward improve evidence grounding as well as predictive agreement? Could you report evidence-retention measurements or controlled evidence-removal experiments, with results separated according to whether the full-input model answers correctly?

2, How much does reasoning-prefix distillation contribute beyond final-answer distribution matching? Please compare these alternatives under the same training setup and report the self-distillation gains across multiple random seeds.

3, Is there a cross-modal budget-allocation mechanism beyond the fixed per-modality retention rule? If not, could you revise the corresponding claims and compare fixed modality proportions with adaptive audio–video allocation?

4, What exactly is included in the reported prefill timing? Could you provide end-to-end latency, including encoding, selection, prefill, and response generation, together with matched-retention latency and memory measurements?

5, Which retention ratios are used during training, and how is the ratio encoded? Does the same selector work reliably at unseen ratios such as 15%, 25%, or 40%? How much performance is lost if ratio conditioning is removed while the Top-K budget still changes?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.