acceptodds
Under review as a conference paper at ICLR 2027

OmniDelta: Skill-Driven Budget Allocation for Token Compression in OmniLLMs

Abstract

Emerging Omni-modal Large Language Models (OmniLLMs) enable unified understanding of text, audio, and video, but their long audio-video token sequences introduce substantial memory and inference costs. Existing compression methods mainly focus on selecting important tokens under fixed budgets, leaving the preceding budget-allocation problem underexplored. We show that direct query-to-audio/video similarity is unreliable for inter-modal budget allocation, and that uniform intra-modal budgets can miss key evidence while retaining redundant content. To address these limitations, we propose OmniDelta, a training-free skill-driven framework that couples intent-aware inter-modal allocation with content-aware intra-modal allocation. OmniDelta first constructs audio and video skill pools to shift the fixed retained-token budget according to query demand, then reallocates modality budgets over audio segments and video frames using local complexity and temporal redundancy. The resulting local budgets can be combined with existing pruning strategies, preserving the total retained-token ratio while changing where the budget is spent. Experiments on four audio-video benchmarks with two Qwen2.5-Omni models show that OmniDelta establishes a new accuracy-efficiency Pareto frontier across pruning ratios. For example, at 25% token retention on Qwen2.5-Omni-7B, OmniDelta reduces GPU memory by 22.0% and achieves a 1.64 end-to-end speedup over full-token inference.

Primary area:
foundation or frontier models, including LLMs
Keywords:
OmniLLM, Token Compression, Budget Allocation, Efficient Inference

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_fGd1OpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents OmniDelta, a training-free framework for allocating token budgets in Omni-modal Large Language Models. It uses modality-specific skill pools to allocate the retained-token budget between audio and video according to the query. Within each modality, it further distributes the budget across audio segments or video frames based on local complexity and temporal redundancy. The framework preserves the total token budget and can be integrated with existing token selection methods. Experiments on four audio-video benchmarks with Qwen2.5-Omni-3B and 7B demonstrate favorable accuracy–efficiency trade-offs.

Critical strengths
  1. The paper is well written and clearly presented. The overall framework and the two levels of budget allocation are easy to follow, with useful illustrations and explanations.

  2. The motivation is clear and intuitive. Different queries require different audio and visual evidence, while the information content varies across segments and frames, making adaptive budget allocation a reasonable design.

  3. The experimental evaluation is thorough and supports the proposed approach. Results across four benchmarks, together with ablations, compatibility with different token selection methods, and memory and latency measurements, make this a solid piece of work.

Critical weaknesses

Overall, the paper presents a solid and well-developed study, and I have no major concerns. My only suggestion is to make the comparison more comprehensive by including additional video token compression baselines, such as VidCom2 [1] and HoliTom [2]. Although DyCoke is already included, evaluating these methods on the visual branch with the same audio processing strategy and matched total token budgets would further clarify OmniDelta’s advantages. I also encourage the authors to include experiments on Qwen3-Omni-30B-A3B to demonstrate applicability to a newer and larger Omni model and strengthen the empirical contribution.

[1] Xuyang Liu, Yiyu Wang, Junpeng Ma, and Linfeng Zhang. Video compression commander: Plug-and-play inference acceleration for video large language models. In EMNLP, 2025.

[2] Kele Shao, Keda Tao, Can Qin, Haoxuan You, Yang Sui, and Huan Wang. Holitom: Holistic token merging for fast video large language models. In NeurIPS, 2025.

Questions for authors

See Weakness.

3 comments
AuthorsOpenReview ↗

Thanks & Initial Response to Reviewer fGd1

[Suggestion - Broaden comparisons with VidCom² and HoliTom using the same audio-processing strategy and matched total token budgets, and evaluate Qwen3-Omni-30B-A3B.]

Thank you for your positive assessment and helpful suggestions. OmniDelta combines lightweight modality routing with accuracy gains across compression backends. Below we summarize its advantages in (1) skill-driven inter-modal allocation, (2) content-aware intra-modal allocation, and (3) compatibility with existing selectors, followed by the requested 7B/30B comparisons.

(1) Skill-driven inter-modal allocation provides accurate, lightweight modality guidance. Matching queries to task keywords and phrases estimates which modality supplies the answer evidence. In the full-query, Top-3 diagnostic in Figure 3, routing accuracy is 78.5%, versus 52.7% for the best tested direct query–audio/video representation (+25.8 percentage points, pp). Cached skill vectors require only query input-embedding lookup/pooling and vocabulary matching, without an additional audio/video encoder or LLM forward pass.

(2) Content-aware intra-modal allocation preserves evidence through local budgets. Segment/frame quotas reflect local complexity and temporal redundancy. Adding this allocation to A/V-only improves WorldSense, AVUT, Video-MME, and DailyOmni by 0.2, 0.8, 0.6, and 1.1 pp, respectively (Table 4), supporting coordination between the two allocation levels.

(3) Allocation alone improves existing selectors. OmniDelta preserves each backend's selection rules. OmniFocus gains 2.2 pp on AVUT at 7B; mean gains on Qwen3-Omni-30B-A3B-Instruct are Zip +0.80, Focus +0.65, Select +0.61, and Scope +0.94 pp. Original Table 5 also reports gains in all eight OmniSIFT benchmark–retention combinations.

Among the closely related OmniLLM token-compression studies we reviewed, this 30B model is one of the largest and most capable evaluated backbones, extending validation beyond 7B to a new backbone. VidCom² and HoliTom use their own visual compression with the same OmniZip audio-pruning strategy. All compressed methods retain 25% of total audio/video tokens, with matched evaluation settings per backbone. Accuracy is in %; parentheses show backend-relative changes in pp, calculated before rounding. Bold marks the highest compressed accuracy; Full Tokens is the uncompressed reference.

Qwen2.5-Omni-7B:

MethodWorldSenseAVUTVideo-MMEDailyOmni
Full Tokens46.863.867.362.7
VidCom²42.758.765.857.4
HoliTom43.360.064.657.7
OmniSelect40.3———
OmniFocus42.757.466.055.4
OmniDelta (Focus)43.5 (+0.8)59.6 (+2.2)66.0 (+0.0)56.6 (+1.2)
OmniZip43.260.766.357.1
OmniDelta (Zip)44.2 (+1.0)61.1 (+0.4)66.4 (+0.1)58.5 (+1.4)

OmniDelta (Zip) leads all four benchmarks, exceeding VidCom² by 0.6–2.4 pp and HoliTom by 0.8–1.8 pp. Only WorldSense was completed for 7B OmniSelect because of AudioCLIP processing time; its 30B evaluation covers all four benchmarks.

Qwen3-Omni-30B-A3B-Instruct:

MethodWorldSenseAVUTVideo-MMEDailyOmni
Full Tokens54.577.275.774.0
VidCom²52.272.874.270.0
HoliTom51.570.173.368.9
OmniZip52.672.375.371.3
OmniDelta (Zip)53.5 (+0.85)73.2 (+0.92)75.3 (+0.00)72.7 (+1.42)
OmniFocus51.169.073.668.0
OmniDelta (Focus)51.8 (+0.64)70.4 (+1.33)74.1 (+0.55)68.1 (+0.09)
OmniSelect51.171.673.570.8
OmniDelta (Select)52.3 (+1.26)72.4 (+0.81)74.0 (+0.52)70.7 (−0.16)
OmniScope52.871.173.969.6
OmniDelta (Scope)52.9 (+0.03)72.6 (+1.50)74.9 (+1.04)70.8 (+1.17)

OmniDelta (Zip) leads three benchmarks and ties OmniZip on Video-MME. It exceeds both visual baselines on all four, including +2.67/+3.76 pp over VidCom²/HoliTom on DailyOmni. Across backend adaptations, 14 of 16 comparisons improve, one ties, and one decreases (Select on DailyOmni, −0.16 pp).

We will document the comparisons, common audio protocol, and backend interfaces in Appendix H.1: Comparisons with Concurrent Methods and Backend Adaptation, and connect the routing and allocation evidence in Appendix H.2: Allocation Ablations and Task-Evidence Analysis.

I appreciate the authors' detailed clarifications and extra experiments. These additions substantially strengthen the completeness of the work and thoroughly resolve all my concerns. Consequently, I have increased my score from 3 to 4.

AuthorsOpenReview ↗

Follow up Response to Reviewer fGd1

Dear Reviewer fGd1,

Thank you for your updated evaluation and for raising the score. We appreciate your recognition of our work. Your constructive feedback has helped us clarify OmniDelta’s budget-allocation design and strengthen the evaluation of its effectiveness and compatibility with existing pruning strategies.

We encourage the community to use and build on OmniDelta. Its lightweight budget allocator can be integrated with different token-pruning strategies, with our experiments demonstrating accuracy gains while preserving their token-selection rules. We welcome broader adoption and extensions to additional compression pipelines.

We sincerely appreciate your time, careful review, and constructive discussion, which have helped improve our manuscript.

Best regards,
The authors

Reviewer_X1CWOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

OmniDelta studies budget allocation for audio-visual token compression in OmniLLMs. It uses query-to-skill matching to redistribute a fixed token budget between audio and video, followed by content-aware allocation across audio segments and video frames using complexity and temporal redundancy. The allocated budgets are then supplied to existing token selectors. Experiments cover four audio-visual QA benchmarks, two Qwen2.5-Omni model sizes, and two pruning backends, together with component ablations, allocation sensitivity analysis, and inference-efficiency measurements.

Critical strengths

1.Separating budget allocation from token selection is a useful design choice. OmniDelta adjusts both the modality-level split and the distribution within each modality while preserving the total retained-token budget. This makes the contribution relatively modular and allows the allocation strategy to be evaluated without replacing the entire compression pipeline.

2.The empirical evidence extends beyond a single benchmark or selector. Tables 1–2 show improvements over OmniZip in most aggregate settings across four benchmarks and two model sizes. Table 5 additionally reports improvements on all four benchmark aggregates when the allocator is combined with OmniSIFT at both tested retention ratios. These results support the practical value of allocation and its compatibility with the two evaluated backends.

Critical weaknesses

1.The evidence for the skill-driven inter-modal component is weaker than the evidence for the complete allocator. In Table 4, inter-modal allocation alone improves the reported average by only 0.1 percentage points and decreases accuracy on AVUT and VideoMME. The table does not include the configuration with both intra-modal components enabled but inter-modal allocation disabled, making its contribution within the full system difficult to isolate. Although Appendix F documents skill-pool construction and Figure 7 tests transfer strength, sensitivity to the pool contents and top-k matching remains insufficiently established.

2.The comparisons provide limited evidence for the broader claim of a new accuracy–efficiency frontier. OmniFocus is a particularly relevant query-guided budget-allocation method, but is not included in the experiments. Furthermore, Table 3 compares OmniDelta at 25% retention with OmniZip at 30%, which is useful for showing an operating-point trade-off but does not isolate the allocator’s incremental runtime cost. A direct comparison under a clearly matched budget, together with a timing breakdown, would strengthen the comparative claim.

Questions for authors

1.With both intra-modal allocation components enabled, what is the effect of enabling versus disabling the skill-driven inter-modal component? Please also clarify how sensitive this contribution is to the skill-pool contents and the top-k matching choice.

2.Can the authors provide a comparison with OmniFocus under the same backbone and total retained-token budget, or explain the concrete obstacles to a matched comparison? Where possible, holding the token selector fixed would help distinguish allocation quality from selector differences.

3.What is the additional latency of skill matching, complexity/redundancy computation, and budget assignment relative to the underlying pruning backend at the same retention ratio? Appendix D provides complexity analysis, but the measured incremental cost is not clear from Table 3.

8 comments
AuthorsOpenReview ↗

Thanks & Initial Response to Reviewer X1CW (W1-1)

[W1-1 - The evidence for the skill-driven inter-modal component is weaker than the evidence for the complete allocator. In Table 4, inter-modal allocation alone improves the reported average by only 0.1 percentage points and decreases accuracy on AVUT and VideoMME.]

Inter-modal allocation uses task semantics to distribute the fixed budget and works with local allocation to preserve evidence; the new 30B controlled ablation improves all four benchmarks; below we address (1) the mechanism, (2) task requirements and A/V-only, and (3) full-system evidence.

(1) Task-semantic matching provides lightweight modality routing. Content relevance does not necessarily identify the modality needed to answer the question: an instrument's appearance and rhythm require visual and audio evidence, respectively. OmniDelta matches the query to a fixed task vocabulary, with cached skill vectors and online input-embedding lookup, pooling, and matching. In the full-query, Top-3 diagnostic in Figure 3, accuracy is 78.5%, versus 52.7% for the best tested direct query–audio/video representation.

(2) Task evidence and local quotas affect A/V-only gains. Joint-evidence questions require both modalities: AVUT includes a question linking “48KG” onscreen to a spoken explanation, while Video-MME includes questions relating narration to visual events. Changing modality totals alone does not ensure sufficient quotas for the relevant segments. OmniZip/A/V-only/full-allocation accuracy is 60.7%/60.3%/61.1% on AVUT and 66.3%/65.8%/66.4% on Video-MME, supporting coordination between the two allocation levels.

(3) Enabling inter-modal allocation improves every benchmark within the 30B full system. On Qwen3-Omni-30B-A3B-Instruct at 25% retention, both intra-modal components remain enabled; the OmniZip selector and budget bounds are fixed. Only cross-modal transfer is toggled.

Among the closely related OmniLLM token-compression studies we reviewed, this is one of the largest and most capable evaluated backbones, extending validation beyond 7B to a new backbone. Accuracy is in %; gains are in percentage points (pp).

Configuration (30B, 25% retention)WorldSenseAVUTVideo-MMEDailyOmni
Transfer off53.2873.0775.2272.10
Transfer on53.4773.1875.3072.68
Gain (pp)+0.19+0.11+0.08+0.58

The mean gain is 0.24 pp. This isolates inter-modal allocation within the complete allocator; original A/V-only retains OmniZip's original intra-modal configuration. The full 7B/30B ablations appear in W1-2/Q1. We will clarify the mechanism in Section 2.1 and add the task examples, configurations, and controls to Appendix H.2: Allocation Ablations and Task-Evidence Analysis.

AuthorsOpenReview ↗

Thanks & Initial Response to Reviewer X1CW (W1-2/Q1)

[W1-2/Q1 - The table does not include the configuration with both intra-modal components enabled but inter-modal allocation disabled, making its contribution within the full system difficult to isolate. Although Appendix F documents skill-pool construction and Figure 7 tests transfer strength, sensitivity to the pool contents and top-k matching remains insufficiently established. With both intra-modal allocation components enabled, what is the effect of enabling versus disabling the skill-driven inter-modal component? Please also clarify how sensitive this contribution is to the skill-pool contents and the top-k matching choice.]

With both intra-modal components and budget bounds fixed, enabling skill-driven transfer improves mean accuracy by 0.25/0.24 pp on 7B/30B; the gains depend on pool content and aggregation; below we report (1) the full-system control, (2) pool sensitivity, and (3) Top-k and all-skill averaging.

All experiments use Qwen2.5-Omni-7B or Qwen3-Omni-30B-A3B-Instruct, 25% total retention, and the OmniZip backend.

(1) The matched-bounds control isolates the inter-modal component. shift_off keeps both intra-modal components, the selector, and bounds unchanged, disabling only cross-modal transfer. Relative to this control, original_k3 changes the four benchmarks by +0.38, −0.17, +0.30, and +0.50 pp on 7B and +0.19, +0.11, +0.08, and +0.58 pp on 30B: seven improvements and one decrease, with mean gains of 0.25/0.24 pp. This supplies the missing configuration with both intra-modal components enabled and inter-modal allocation disabled.

prior011 also disables transfer but changes the bounds to the physical range. Mean gains relative to it are 0.27/0.21 pp. Because this changes two factors, shift_off is the appropriate control for isolating transfer.

(2) Independent pools retain full-method mean gains, while the routing increment depends on content. original is the WorldSense-derived pool; external is an independent external task-ontology pool without target questions or answer choices; clean is independently generated with Qwen; session is constructed from the method's motivation in an independent GPT-6.1 Sol conversation. assistant uses existing context and is an auxiliary control, not evidence of strict source independence.

For external, clean, and session, mean accuracy is 0.34–0.49 pp below the original pool on 7B and 0.12–0.37 pp below on 30B, while remaining 0.24–0.39 and 0.43–0.68 pp above OmniZip, respectively. These full-method gains should be distinguished from the routing increment: external_k3 is 0.15/0.09 pp below shift_off in mean accuracy. Independent pools thus remain usable in the complete framework, but their gains over OmniZip cannot all be attributed to cross-modal transfer.

AuthorsOpenReview ↗

Thanks & Initial Response to Reviewer X1CW (W1-2/Q1)

(3) The Top-k sweep quantifies aggregation sensitivity. k1/k3/k5/k10 average the k highest skill similarities within each modality; mean averages all similarities in that modality's pool. For k∈{1,3,5,10}, the range of four-benchmark means is 0.40/0.37 pp for the original pool and 0.21/0.20 pp for the external pool on 7B/30B. Individual benchmarks vary more; the original pool's 30B DailyOmni range is 1.17 pp.

Relative to k3, all-skill averaging changes mean accuracy by −0.12/−0.24 pp for the original pool and −0.09/+0.17 pp for the external pool. We retain k=3 as a common default; the best aggregation depends on the model and task.

Complete results follow. Accuracy is in %, with two decimals for the new ablations. Means are unweighted across four benchmarks; means, differences, and ranges are computed before rounding. OmniZip values retain their previously reported precision.

7B:

ConfigurationWorldSenseAVUTVideo-MMEDailyOmniMean
OmniZip baseline43.260.766.357.156.83
original_k344.2061.1366.4158.4857.56
external_k343.8260.7865.9658.0657.16
clean_k343.5460.5566.1957.9857.07
session_k343.7960.5065.8558.7357.22
assistant_k343.8561.3066.0058.1557.33
shift_off43.8261.3066.1157.9857.30
prior01143.7661.3066.1157.9857.29
original_k143.9861.1966.1958.5657.48
original_k543.9560.8466.2658.8157.47
original_k1044.0160.0966.3058.2357.16
original_mean44.1761.5366.1557.8957.44
external_k143.8560.5065.9358.4057.17
external_k543.5460.6765.8158.8157.21
external_k1043.6960.6765.7457.8957.00
external_mean43.6661.0765.7457.8157.07

30B:

ConfigurationWorldSenseAVUTVideo-MMEDailyOmniMean
OmniZip baseline52.6272.2675.3071.2667.86
original_k353.4773.1875.3072.6868.66
external_k352.9973.0775.0772.1868.33
clean_k353.8172.9075.8571.6068.54
session_k353.0973.0175.2271.8568.29
assistant_k353.0973.0775.7872.1068.51
shift_off53.2873.0775.2272.1068.42
prior01153.3173.0775.3372.1068.45
original_k153.3772.7875.2272.3568.43
original_k553.3173.4175.1973.1068.75
original_k1053.1572.8475.6371.9368.39
original_mean53.2873.0775.1572.1868.42
external_k152.6273.3675.5672.3568.47
external_k553.0972.9075.1572.4368.39
external_k1053.1272.4975.2272.2668.27
external_mean53.1273.1875.3372.3568.50

We will add the on/off and aggregation ablations to Appendix H.2, and the vocabularies, inputs, and prompts, based on each pool's provenance, to Appendix H.4: Skill-Pool Provenance and Independent Construction, with corresponding updates to Appendix F.

Reviewer_DTCUOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes OmniDelta, a training-free dynamic token budget allocation method for Omni-modal Large Language Models (OmniLLMs). It uses skill pools to adaptively allocate token budgets between audio and video based on the input query, and further redistributes budgets across audio segments and video frames according to local complexity and temporal redundancy. Experiments on four audio-video understanding benchmarks using two Qwen2.5-Omni models demonstrate the effectiveness of the proposed method.

Critical strengths
  1. Practicality and flexibility. The proposed method requires no additional training and can be integrated with existing token pruning approaches. It improves model performance while maintaining the same token budget.
  2. Comprehensive evaluation. The experiments cover multiple audio-video understanding benchmarks and model scales. The paper also includes ablation studies and inference efficiency analysis.
Critical weaknesses
  1. Limited novelty compared with existing adaptive budget allocation methods. Previous works, such as OmniSelect and OmniFocus, have already explored query-aware modality importance estimation and dynamic token allocation. The paper lacks direct comparisons with these closely related approaches, making it difficult to determine whether the proposed skill-driven allocation provides substantial advantages.

  2. Insufficient evidence supporting the effectiveness of the core design. Table 4 shows that inter-modal allocation alone improves the average score across four benchmarks only from 56.8 to 56.9, while the complete method achieves 57.6. This suggests that most improvements may come from intra-modal allocation, whereas the contribution of skill-driven routing to downstream performance appears limited. Moreover, improvements over OmniZip on some benchmarks are only 0.1–0.4 percentage points.

Questions for authors
  1. Existing methods such as OmniSelect and OmniFocus have already investigated query-aware dynamic token allocation. What are the key technical differences between OmniDelta and these approaches? Could the authors provide direct comparisons under the same model and token retention settings?
  2. Skill-driven routing is presented as a core contribution, but Table 4 shows that inter-modal allocation alone provides only limited overall improvement. Does the skill-pool approach offer significant advantages over simpler keyword-based classification or other query-based routing strategies? Is its contribution to downstream performance sufficient to justify the additional design?
  3. The skill pools are constructed using WorldSense, which is also used for evaluation. Could the authors clarify whether the skill-pool construction process involves test-set questions or answer candidates? Can the method maintain similar performance when the skill pools are constructed from entirely independent task data?
10 comments
AuthorsOpenReview ↗

Thanks & Initial Response to Reviewer DTCU (W1)

[W1 - Limited novelty compared with existing adaptive budget allocation methods. Previous works, such as OmniSelect and OmniFocus, have already explored query-aware modality importance estimation and dynamic token allocation. The paper lacks direct comparisons with these closely related approaches, making it difficult to determine whether the proposed skill-driven allocation provides substantial advantages.]

Our novelty lies in task-semantic modality routing and lightweight, composable hierarchical allocation; below we provide (1) mechanism analysis and (2) experimental results. Appendix A already discusses the concurrent methods; we now expand the direct comparison.

(1) Mechanism analysis: task semantics, lightweight computation, and composability.

Modality demand. OmniSelect and OmniFocus estimate query–audio/video relevance using AudioCLIP and model input features, respectively. Content relevance does not necessarily identify the modality needed to answer the question. OmniDelta instead matches the query to a fixed vocabulary of task keywords and phrases describing modality requirements.

Lightweight computation. Skill embeddings are cached offline. Each query requires only input-embedding lookup and pooling, similarity computation against the cached vocabulary, and Top-k selection. This requires no additional audio/video encoder or LLM forward pass, and skill-matching cost does not grow with audio/video sequence length.

Composable allocation. Query–skill matching determines inter-modal budgets; local complexity and temporal redundancy determine intra-modal quotas. The allocator decides how many tokens to retain, while the existing selector decides which tokens to retain. Each adaptation preserves the backend's selection rules and replaces only budget allocation.

(2) Experimental results support both the routing mechanism and backend compatibility. In the full-query, Top-3 diagnostic in Section 2.1, skill routing achieves 78.5%, versus 52.7% for the best tested direct query–audio/video representation (+25.8 percentage points, pp).

All compressed methods use 25% total audio/video retention under matched evaluation settings for each backbone. Accuracy is in %; parentheses give gains over the corresponding backend in pp, computed before rounding. Accuracy is shown to one decimal place and 30B gains to two. Full Tokens is the uncompressed reference.

7B — Qwen2.5-Omni-7B:

MethodWorldSenseAVUTVideo-MMEDailyOmni
Full Tokens46.863.867.362.7
OmniSelect40.3———
OmniFocus42.757.466.055.4
OmniDelta (Focus)43.5 (+0.8)59.6 (+2.2)66.0 (+0.0)56.6 (+1.2)
OmniZip43.260.766.357.1
OmniDelta (Zip)44.2 (+1.0)61.1 (+0.4)66.4 (+0.1)58.5 (+1.4)

OmniDelta (Zip) improves over OmniZip by 1.0, 0.4, 0.1, and 1.4 pp and exceeds OmniSelect on WorldSense by 3.9 pp. OmniDelta (Focus) gains 2.2 pp on AVUT and 1.2 pp on DailyOmni. AudioCLIP-based relevance computation increased processing time in our 7B OmniSelect reproduction; only WorldSense was completed.

30B — Qwen3-Omni-30B-A3B-Instruct: Among the closely related OmniLLM token-compression studies we reviewed, this is one of the largest and most capable evaluated backbones, extending validation beyond 7B to a new backbone.

MethodWorldSenseAVUTVideo-MMEDailyOmni
Full Tokens54.577.275.774.0
OmniZip52.672.375.371.3
OmniDelta (Zip)53.5 (+0.85)73.2 (+0.92)75.3 (+0.00)72.7 (+1.42)
OmniFocus51.169.073.668.0
OmniDelta (Focus)51.8 (+0.64)70.4 (+1.33)74.1 (+0.55)68.1 (+0.09)
OmniSelect51.171.673.570.8
OmniDelta (Select)52.3 (+1.26)72.4 (+0.81)74.0 (+0.52)70.7 (−0.16)
OmniScope52.871.173.969.6
OmniDelta (Scope)52.9 (+0.03)72.6 (+1.50)74.9 (+1.04)70.8 (+1.17)

Mean gains are Zip +0.80, Focus +0.65, Select +0.61, and Scope +0.94 pp. Of 16 paired comparisons, 14 improve, one ties, and one decreases (Select on DailyOmni, −0.16 pp). OmniDelta (Zip) has the highest compressed accuracy in these tables on all four 7B benchmarks and three 30B benchmarks, tying OmniZip on 30B Video-MME. Q1-2 also presents the complete comparisons under matched settings.

We will clarify these mechanisms in Section 2.1 and Appendix A, and document comparisons and interfaces in Appendix H.1: Comparisons with Concurrent Methods and Backend Adaptation. For backends without public code, we will contact their authors and welcome collaborative integration.

AuthorsOpenReview ↗

Thanks & Initial Response to Reviewer DTCU (W2)

[W2-1 - Insufficient evidence supporting the effectiveness of the core design. Table 4 shows that inter-modal allocation alone improves the average score across four benchmarks only from 56.8 to 56.9, while the complete method achieves 57.6. This suggests that most improvements may come from intra-modal allocation, whereas the contribution of skill-driven routing to downstream performance appears limited.]

Inter-modal allocation must be evaluated with the intra-modal configuration specified; the new 30B ablation improves all four benchmarks while holding intra-modal allocation and bounds fixed; below we address (1) the A/V-only configuration and (2) its contribution within the complete allocator.

(1) A/V-only depends on task evidence and local quotas. Single-modality questions and questions requiring joint evidence impose different budget requirements. For example, an AVUT question linking “48KG” onscreen to a spoken explanation requires both visual text and audio. Video-MME questions linking narration to visual events likewise require local evidence preservation. Original Table 4 shows that adding intra-modal allocation makes all four benchmarks exceed OmniZip:

Configuration (7B, 25% retention)WorldSenseAVUTVideo-MMEDailyOmniReported mean
OmniZip43.260.766.357.156.8
A/V-only44.060.365.857.456.9
Complete OmniDelta44.261.166.458.557.6

(2) With both intra-modal components fixed, inter-modal allocation improves all four benchmarks on 30B. On Qwen3-Omni-30B-A3B-Instruct at 25% retention, we keep both intra-modal components enabled and hold the OmniZip selector and budget bounds fixed, toggling only cross-modal transfer. Accuracy is in %.

Configuration (30B, 25% retention)WorldSenseAVUTVideo-MMEDailyOmni
Transfer off53.2873.0775.2272.10
Transfer on53.4773.1875.3072.68
Gain (pp)+0.19+0.11+0.08+0.58

The mean gain is 0.24 pp. This isolates the component within the complete allocator; original A/V-only instead retains OmniZip's original intra-modal configuration. Q2-2 provides a routing comparison under that original configuration. We will distinguish these controls and their conclusions in Appendix H.2: Allocation Ablations and Task-Evidence Analysis.

[W2-2 - Moreover, improvements over OmniZip on some benchmarks are only 0.1–0.4 percentage points.]

Improving accuracy after removing 75%–80% of tokens, without training the allocator or retaining more tokens, is itself a contribution; below we discuss (1) fixed-budget gains and (2) reuse across backends.

(1) Reallocation recovers part of the compression loss. At 7B and 25% retention, gains over OmniZip are 1.0, 0.4, 0.1, and 1.4 pp. These close 27.8% and 25.0% of the gap to Full Tokens on WorldSense and DailyOmni. Focusing only on individual gains of 0.1–0.4 pp overlooks the fixed-budget constraint and the recovered accuracy.

(2) The contribution extends across five backends. OmniFocus gains 2.2 pp on AVUT and 1.2 pp on DailyOmni at 7B. At 30B, mean gains are Zip +0.80, Focus +0.65, Select +0.61, and Scope +0.94 pp. Original Table 5 also reports improvements for OmniSIFT on all eight benchmark–retention combinations.

We will consolidate paired gains and recovered accuracy gaps in Appendix H.1.

AuthorsOpenReview ↗

Thanks & Initial Response to Reviewer DTCU (Q1)

[Q1-1 - Existing methods such as OmniSelect and OmniFocus have already investigated query-aware dynamic token allocation. What are the key technical differences between OmniDelta and these approaches?]

The key technical differences are the routing signal, online computation, and allocation interface; we address (1) task-semantic routing, (2) cached matching, and (3) selector integration.

(1) Task-semantic routing estimates answer-evidence requirements. OmniSelect uses AudioCLIP query–audio/video relevance; OmniFocus uses query–chunk feature similarity. OmniDelta matches the query to task keywords and phrases in text-embedding space. For example, questions about an instrument's appearance and rhythm require visual and audio evidence, although both concern the same content.

(2) Cached matching avoids additional semantic encoding. Online routing uses only query input-embedding lookup/pooling and similarity to cached skills with Top-k selection. It requires no additional audio/video semantic encoder, LLM forward pass, or query matching against every audio/video token.

(3) Hierarchical quotas interface with existing selectors. OmniDelta allocates both modality totals and segment/frame quotas while preserving the total budget and backend selection rules. The paired comparisons in Q1-2 evaluate this allocation interface.

We will expand Appendix A on these distinctions and Appendix H.1 on features, quotas, and backend interfaces.

[Q1-2 - Could the authors provide direct comparisons under the same model and token retention settings?]

At matched 25% total audio/video retention, OmniDelta (Zip) has the highest compressed accuracy in the comparison on all four 7B benchmarks and on three 30B benchmarks, tying on the fourth; below we report (1) 7B and (2) 30B.

Accuracy is in %; parentheses give changes relative to the corresponding backend in pp, calculated before rounding. Accuracy is shown to one decimal place and 30B changes to two. Full Tokens is the uncompressed reference.

(1) 7B: improvements with both OmniZip and OmniFocus. All methods use Qwen2.5-Omni-7B, with matched total budgets and evaluation settings.

MethodWorldSenseAVUTVideo-MMEDailyOmni
Full Tokens46.863.867.362.7
OmniSelect40.3———
OmniFocus42.757.466.055.4
OmniDelta (Focus)43.5 (+0.8)59.6 (+2.2)66.0 (+0.0)56.6 (+1.2)
OmniZip43.260.766.357.1
OmniDelta (Zip)44.2 (+1.0)61.1 (+0.4)66.4 (+0.1)58.5 (+1.4)

AudioCLIP-based relevance computation increased processing time in our 7B OmniSelect reproduction; only WorldSense was completed. The 30B evaluation below covers all four benchmarks.

(2) 30B: all four adapted backends improve in mean accuracy. With Qwen3-Omni-30B-A3B-Instruct, each adaptation preserves its backend's selection rules and replaces only allocation.

MethodWorldSenseAVUTVideo-MMEDailyOmni
Full Tokens54.577.275.774.0
OmniZip52.672.375.371.3
OmniDelta (Zip)53.5 (+0.85)73.2 (+0.92)75.3 (+0.00)72.7 (+1.42)
OmniFocus51.169.073.668.0
OmniDelta (Focus)51.8 (+0.64)70.4 (+1.33)74.1 (+0.55)68.1 (+0.09)
OmniSelect51.171.673.570.8
OmniDelta (Select)52.3 (+1.26)72.4 (+0.81)74.0 (+0.52)70.7 (−0.16)
OmniScope52.871.173.969.6
OmniDelta (Scope)52.9 (+0.03)72.6 (+1.50)74.9 (+1.04)70.8 (+1.17)

Of 16 paired comparisons, 14 improve, one ties, and one decreases: OmniDelta (Select) on DailyOmni (−0.16 pp). Focus and Scope improve on every benchmark. OmniDelta (Zip) has the highest compressed accuracy in this table on three benchmarks and ties OmniZip on Video-MME.

We will include the full results, settings, budget accounting, and adaptations in Appendix H.1.

Reviewer_HsYVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors divide audio-video token compression in OmniLLMs into two components: budget allocation and token selection. OmniDelta improves the budget-allocation stage through query–skill text matching, focusing on deciding how many audio and video tokens to retain, as well as how many tokens to retain in each audio segment and video frame. OmniDelta achieves experimental results that outperform SOTA methods such as OmniZip on models including Qwen2.5-Omni-7B.

Critical strengths

1.The problem formulation has practical relevance. This paper distinguishes budget allocation from token selection and adjusts retention quotas according to differences in modality requirements and local information density, providing a clear direction for improving audio-video compression.

2.The skill-driven budget allocation idea is novel. Matching questions with a vocabulary of task semantics to identify modality requirements provides an intuitive and insightful design for audio-video budget allocation.

3.The method is lightweight and composable. The budget-allocation module requires no additional training, and its effectiveness is demonstrated with both the OmniZip and OmniSIFT pruning backends, indicating compatibility and practical value.

Critical weaknesses

1.The methodological novelty has not been sufficiently substantiated. Methods such as OmniSelect, OmniFocus, OmniScope, and OmniRefine have already explored query-aware compression or cross-modal budget coordination at different levels. Although this paper cites these works, it lacks more specific analysis and more direct experimental comparisons, making it difficult to assess the necessity and degree of novelty of the skill-driven design.

2.The performance advantages demonstrated in the experiments are limited. At token retention rates of 20%–25%, OmniDelta improves over OmniZip by only approximately 1.0 percentage point, while still exhibiting a substantial gap from Full Tokens. The independent gains of the core module are also limited. For example, in Table 4, enabling only the top-level audio-video budget allocation yields marginal improvements over the baseline, and performance even decreases on half of the evaluated datasets.

Questions for authors

1.Compared with other SOTA strategies involving cross-modal selection, what makes the skill-driven design necessary?

2.At lower pruning ratios, does OmniDelta achieve larger gains over SOTA methods such as OmniZip? Alternatively, can it more consistently match the performance of Full Tokens under low to moderate pruning ratios?

3.Why does the A/V-only configuration in Table 4 lead to performance degradation?

4.Are the calibration and test sets sufficiently separated, and how is cross-domain generalization ensured? I would like the authors to verify whether the method can maintain similar gains without access to the questions or answer choices in the target test set.

5.Can OmniDelta achieve substantial end-to-end speedups across tasks with different characteristics? Currently, actual inference efficiency is evaluated only on WorldSense, whereas datasets differ in input length, audio-video token proportions, and sampling settings.

10 comments
AuthorsOpenReview ↗

Thanks & Initial Response to Reviewer HsYV (W1)

[W1 - The methodological novelty has not been sufficiently substantiated. Methods such as OmniSelect, OmniFocus, OmniScope have already explored query-aware compression or cross-modal budget coordination at different levels. Although this paper cites these works, it lacks more specific analysis and more direct experimental comparisons, making it difficult to assess the necessity and degree of novelty of the skill-driven design.]

Our methodological novelty lies in task-semantic modality routing and lightweight, composable hierarchical budget allocation; below we provide (1) mechanism analysis and (2) experimental results. Appendix A already discusses the concurrent methods; we now expand the direct comparison.

(1) Mechanism analysis: task semantics, lightweight computation, and composability.

Modality demand. OmniSelect, OmniFocus, and OmniScope estimate query–audio/video relevance using AudioCLIP, model input features, and visual CLIP/internal audio features, respectively. OmniRefine is query-agnostic and focuses on audio–video alignment. Content relevance does not necessarily identify the modality needed to answer the question. OmniDelta instead matches the query to a fixed vocabulary describing task requirements.

Lightweight computation. Skill embeddings are cached offline. Each query requires only input-embedding lookup and pooling, similarity computation against the cached vocabulary, and Top-k selection. This requires no additional audio/video encoder or LLM forward pass, and skill-matching cost does not grow with audio/video sequence length.

Composable allocation. Query–skill matching determines inter-modal budgets; local complexity and temporal redundancy determine intra-modal quotas. The allocator decides how many tokens to retain, while the existing selector decides which tokens to retain. Each adaptation preserves the backend's selection rules and replaces only budget allocation.

(2) Experimental results support both the routing mechanism and backend compatibility. In the full-query, Top-3 diagnostic in Section 2.1, skill routing reaches 78.5%, versus 52.7% for the best tested direct query–audio/video representation (+25.8 percentage points, pp). At 7B and 25% retention, OmniDelta (Zip) improves the four benchmarks by 1.0, 0.4, 0.1, and 1.4 pp, exceeds OmniSelect on WorldSense by 3.9 pp, and improves OmniFocus on AVUT by 2.2 pp.

We also evaluate four backend adaptations on Qwen3-Omni-30B-A3B-Instruct. Among the closely related OmniLLM token-compression studies we reviewed, this is one of the largest and most capable evaluated backbones, extending validation beyond 7B to a new backbone. Focus and Scope improve on AVUT by 1.33 and 1.50 pp, respectively.

All compressed configurations below retain 25% of the total audio/video tokens under matched evaluation settings. Full Tokens is the uncompressed reference. Accuracy is in %; parentheses report changes relative to the corresponding backend. Accuracy is shown to one decimal place, and 30B changes to two, calculated before rounding.

Qwen2.5-Omni-7B:

MethodWorldSenseAVUTVideo-MMEDailyOmni
Full Tokens46.863.867.362.7
OmniSelect40.3———
OmniFocus42.757.466.055.4
OmniDelta (Focus)43.5 (+0.8)59.6 (+2.2)66.0 (+0.0)56.6 (+1.2)
OmniZip43.260.766.357.1
OmniDelta (Zip)44.2 (+1.0)61.1 (+0.4)66.4 (+0.1)58.5 (+1.4)

AudioCLIP-based relevance computation increased processing time in our 7B OmniSelect reproduction; only WorldSense was completed, and “—” denotes unavailable results. The 30B comparison covers all four benchmarks.

Qwen3-Omni-30B-A3B-Instruct:

MethodWorldSenseAVUTVideo-MMEDailyOmni
Full Tokens54.577.275.774.0
OmniZip52.672.375.371.3
OmniDelta (Zip)53.5 (+0.85)73.2 (+0.92)75.3 (+0.00)72.7 (+1.42)
OmniFocus51.169.073.668.0
OmniDelta (Focus)51.8 (+0.64)70.4 (+1.33)74.1 (+0.55)68.1 (+0.09)
OmniSelect51.171.673.570.8
OmniDelta (Select)52.3 (+1.26)72.4 (+0.81)74.0 (+0.52)70.7 (−0.16)
OmniScope52.871.173.969.6
OmniDelta (Scope)52.9 (+0.03)72.6 (+1.50)74.9 (+1.04)70.8 (+1.17)

Across the 16 paired 30B comparisons, 14 improve. OmniDelta (Zip) achieves the highest compressed accuracy in this table on three benchmarks and ties OmniZip on Video-MME.

We will clarify the mechanism–evidence connection in Section 2.1, expand Appendix A, and document these comparisons, token-budget accounting, online matching, and backend interfaces in Appendix H.1: Comparisons with Concurrent Methods and Backend Adaptation. For backends without public code, we will contact their authors and welcome collaborative integration.

AuthorsOpenReview ↗

Thanks & Initial Response to Reviewer HsYV (W2)

[W2-1 - The performance advantages demonstrated in the experiments are limited. At token retention rates of 20%–25%, OmniDelta improves over OmniZip by only approximately 1.0 percentage point, while still exhibiting a substantial gap from Full Tokens.]

Improving accuracy after removing 75%–80% of tokens, without training the allocator or increasing the retained budget, is itself a contribution; below we explain (1) the gains under a fixed budget and (2) their reuse across backends.

(1) Fixed-budget gains recover part of the compression loss. At 7B and 25% retention, gains over OmniZip are 1.0, 0.4, 0.1, and 1.4 pp across the four benchmarks. On WorldSense and DailyOmni, these close 27.8% and 25.0% of OmniZip's accuracy gap to Full Tokens. Assessing only the absolute gain overlooks both the 75% token reduction and the fraction of lost accuracy recovered through reallocation.

(2) The same allocation mechanism benefits multiple selectors. On 7B, OmniFocus gains 2.2 pp on AVUT and 1.2 pp on DailyOmni. On 30B, the four-benchmark mean gains are Zip +0.80, Focus +0.65, Select +0.61, and Scope +0.94 pp. Original Table 5 also reports gains for OmniSIFT on all eight benchmark–retention combinations. Together, these establish reuse across five backends.

Appendix H.1 will consolidate the paired gains, recovered accuracy gaps, and backend results.

[W2-2 - The independent gains of the core module are also limited. For example, in Table 4, enabling only the top-level audio-video budget allocation yields marginal improvements over the baseline, and performance even decreases on half of the evaluated datasets.]

Inter-modal allocation must be evaluated with the intra-modal configuration specified; the new 30B ablation improves all four benchmarks while holding intra-modal allocation and bounds fixed; below we address (1) the A/V-only configuration and (2) its contribution within the complete allocator.

(1) A/V-only depends on task evidence and local quotas. Single-modality questions and questions requiring joint evidence impose different budget requirements. For example, an AVUT question linking “48KG” onscreen to a spoken explanation requires both visual text and audio. Video-MME questions linking narration to visual events likewise require local evidence preservation. Changing modality totals alone does not ensure sufficient quota for the relevant segment or preservation of the required tokens. Inter-modal allocation, intra-modal allocation, and token selection therefore need to work together.

(2) With both intra-modal components fixed, inter-modal allocation improves all four benchmarks on 30B. On Qwen3-Omni-30B-A3B-Instruct at 25% retention, we keep both intra-modal components enabled and hold the OmniZip selector and budget bounds fixed, toggling only cross-modal transfer. Accuracy is in %.

Configuration (30B, 25% retention)WorldSenseAVUTVideo-MMEDailyOmni
Transfer off53.2873.0775.2272.10
Transfer on53.4773.1875.3072.68
Gain (pp)+0.19+0.11+0.08+0.58

The mean gain is 0.24 pp, with improvements on all four benchmarks on the larger backbone. This isolates the component within the complete allocator; original A/V-only instead retains OmniZip's original intra-modal configuration. Q1 also provides a routing comparison under that original configuration. We will distinguish these controls and their conclusions in Appendix H.2: Allocation Ablations and Task-Evidence Analysis.

AuthorsOpenReview ↗

Thanks & Initial Response to Reviewer HsYV (Q1, Q2, Q3)

[Q1 - Compared with other SOTA strategies involving cross-modal selection, what makes the skill-driven design necessary?]

The need for skill-driven routing is to estimate which modality supplies the answer evidence through lightweight text matching; below we address (1) the routing signal, (2) downstream gains, and (3) computation.

(1) Task semantics provide a routing signal beyond content similarity. In the matched full-query, Top-3 diagnostic, skill routing achieves 78.5%, versus 52.7% for the best tested direct-similarity variant.

(2) This signal improves downstream QA. With OmniZip's intra-modal allocation and selector fixed, WorldSense accuracy is 43.2% with fixed prior budgets, 43.5% with direct embedding routing, and 44.0% with skill routing. The complete allocator also improves 30B Focus and Scope on AVUT by 1.33 and 1.50 pp, supporting its use with existing selectors.

(3) The computation is lightweight. Cached skill vectors require only query-embedding lookup/pooling and vocabulary matching with Top-k selection. At 7B, WorldSense, and matched 25% retention, the complete allocator increases TTFT from 1,768 to 1,776 ms (+8 ms, 0.45%), while E2E remains 1.92 s at the reported precision.

We will connect the routing diagnostics and controlled QA comparison in Section 2.1 and Appendix B.1 with backend comparisons in H.1 and matched-budget timing in H.5.

[Q2 - At lower pruning ratios, does OmniDelta achieve larger gains over SOTA methods such as OmniZip? Alternatively, can it more consistently match the performance of Full Tokens under low to moderate pruning ratios?]

At 45% retention, OmniDelta is closer to Full Tokens, although gains over OmniZip do not increase; below we discuss (1) accuracy at higher retention and (2) the remaining compression loss.

(1) Higher retention improves accuracy. Raising retention from 25% to 45% improves OmniDelta by 1.9, 2.2, 0.9, and 1.6 pp on the four benchmarks. The 7B results are:

MethodWorldSenseAVUTVideo-MMEDailyOmni
Full Tokens46.863.867.362.7
OmniDelta, 25%44.261.166.458.5
OmniZip, 45%45.663.167.260.0
OmniDelta, 45%46.1 (+0.5)63.3 (+0.2)67.3 (+0.1)60.1 (+0.1)

At 45% retention, gains over OmniZip are 0.5, 0.2, 0.1, and 0.1 pp. Gaps to Full Tokens are 0.7, 0.5, 0.0, and 2.6 pp, with Video-MME matching its reported accuracy.

(2) A smaller baseline gap leaves less compression loss to recover. OmniZip at 45% retention is already only 0.1 pp below Full Tokens on Video-MME, and OmniDelta recovers that gap. Absolute improvements therefore need not grow with retention.

We will include these results in Appendix H.3: Accuracy at Different Retention Ratios.

[Q3 - Why does the A/V-only configuration in Table 4 lead to performance degradation?]

A/V-only can degrade accuracy because changing modality totals does not ensure preservation of the required local evidence; below we summarize (1) the mechanism and (2) the supporting controls.

(1) Joint evidence requires coordinated allocation. Questions linking audio to visual events need both modalities and appropriate segment/frame quotas; gains therefore depend on inter-modal allocation, intra-modal allocation, and the selector together.

(2) The 30B control confirms a positive contribution within the complete allocator. Enabling inter-modal allocation with both intra-modal components, the selector, and bounds fixed improves all four benchmarks (+0.24 pp on average). See W2-2 for the comparison table and configuration distinctions, to be documented in Appendix H.2.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

For clarity, we refer to Reviewers HsYV, DTCU, X1CW, and fGd1 as R1, R2, R3, and R4, respectively.

We sincerely thank all reviewers for their thoughtful feedback and constructive suggestions. We are encouraged by their recognition of our design and its practical value.

In particular, we appreciate R1's recognition of the practical relevance of budget allocation and the intuitive task-semantic matching approach, and R4's acknowledgment of the motivation for adapting budgets to modality requirements and local information density. R1 and R2 recognize that our method requires no training and integrates with existing pruning pipelines; R3 highlights the value of separating allocation from selection, allowing allocation to be evaluated within an existing pipeline. We also appreciate the positive assessments of our empirical evidence across benchmarks (R2, R3, R4), particularly the evaluation across model scales, ablations, and efficiency analysis (R2), gains with OmniSIFT at both retention ratios (R3), and the clear presentation and illustrations (R4).

We have addressed the questions on related methods, skill-pool provenance, component contributions, and runtime overhead individually. Below, we summarize our core contributions, experimental updates, and planned revisions.

Core Contributions of Our Work

  1. Task-semantic inter-modal budget allocation. Content relevance does not necessarily identify the modality needed to answer a question. OmniDelta matches queries to a fixed task vocabulary to estimate modality requirements. Skill vectors are cached offline; online routing uses only input-embedding lookup, pooling, and vocabulary matching, without an additional audio/video encoder or LLM forward pass.

  2. Content-aware intra-modal budget allocation. Local complexity and temporal redundancy determine audio-segment and video-frame quotas. The two allocation levels work together to preserve relevant evidence while keeping the total token budget fixed.

  3. Reusable allocation improves existing compression methods. The allocator determines how many tokens to retain; the original selector determines which tokens. Original and new experiments cover OmniZip, OmniSIFT, OmniFocus, OmniSelect, and OmniScope, demonstrating accuracy gains through allocation while preserving selection rules.

Experimental Updates During Rebuttal

These experiments are complete and will be incorporated at the locations below.

  • Section 4.5 and Appendix H.1: Added matched-budget comparisons and adaptations for OmniFocus, OmniSelect, and OmniScope, plus VidCom²/HoliTom with common OmniZip audio pruning. Evaluation extends from 7B to Qwen3-Omni-30B-A3B-Instruct, one of the largest and most capable evaluated backbones among the closely related OmniLLM compression studies we reviewed. All four adapted backends improve in mean accuracy at 30B/25% retention.
  • Appendix H.4: Tested independent ontology, Qwen, and GPT-session skill pools on 7B/30B. Independent pools retain mean gains over OmniZip, including the external pool without target questions or answer choices; vocabulary content affects individual results.
  • Appendix H.2: Added the missing inter-modal on/off control with intra-modal allocation, selector, and bounds fixed, showing gains on all four 30B benchmarks. Top-k, all-skill mean, and boundary controls further quantify allocation sensitivity.
  • Appendix H.3: Added 7B experiments at 45% retention, improving all four benchmarks over matched-budget OmniZip and matching reported Full Tokens accuracy on Video-MME.
  • Appendix H.5: Measured 0.45% net TTFT overhead with +1.0 pp accuracy at 7B/WorldSense and matched 25% retention, and extended memory/E2E evaluation to Video-MME and DailyOmni on 3B.

Discussion Updates During Rebuttal

  • Appendices A and H.2: Clarify task-evidence requirements versus content similarity, connect routing diagnostics to downstream accuracy, and explain the interaction between allocation levels and token selection.
  • Appendices F and H.4: Document pool provenance and data use, distinguish original WorldSense-derived and independent pools, and clarify the absence of ground-truth supervision or QA-score tuning.
  • Appendices D and H.5: Specify hardware, sampling, and timing boundaries, distinguishing complete-pipeline net overhead from component timings and comparisons at different retention ratios.

These additions clarify the mechanisms, compatibility, and practical costs of OmniDelta. We welcome further backend integration with the original authors.

All revisions will be highlighted in blue\color{blue}{\text{blue}} in the updated manuscript. We thank the reviewers again and invite their consideration of this additional evidence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.