acceptodds
Under review as a conference paper at ICLR 2027

TimePrune: Adaptive Temporal Resolution for Efficient Audio Large Language Models

Abstract

Audio representations form temporally ordered token trajectories whose variation can differ substantially over time. Processing dense audio tokens throughout Audio Large Language Models (ALLMs), however, incurs substantial computational overhead. Existing audio token compression methods address this inefficiency by exploiting token importance, redundancy, or temporal structure, but typically do not directly assess how faithfully the retained tokens preserve the underlying representation trajectory. We introduce TimePrune, a training-free framework that preserves temporal representation trajectories by adaptively determining temporal resolution under a given token budget. At its core, Time-Aware RDP recursively refines intervals whose intermediate representations are insufficiently described by their temporal anchors, yielding adaptive local anchor density according to trajectory deviation. We further introduce Temporal Coverage Refinement to regulate excessively sparse temporal intervals while preserving trajectory-driven refinement. Extensive experiments across four ALLMs and five audio understanding benchmarks demonstrate the effectiveness of TimePrune across different models and token budgets. In our efficiency analysis, TimePrune retains only 15% of audio tokens while preserving 98.4% of the Vanilla score and reducing audio-token-dependent LLM computation by 85.2%, demonstrating its effectiveness for efficient ALLM inference.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Audio Large Language Models, Audio Token Pruning, Token Compression, Adaptive Temporal Resolution, Temporal Trajectory Preservation, Efficient Inference

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_1pCsOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TimePrune, a training-free ALLM token compression framework that preserves temporal representation trajectories through adaptive temporal resolution. TimePrune comprises two token selection mechanisms: Time-Aware RDP preserving trajectory fidelity and temporal covarage refinement that maintains temporal coverage. Experiments show that TimePrune maintains competitive performance on downstream tasks under aggressive retention ratios of 10%~15%.

Critical strengths
  1. The experiment covers four ALLMs with different audio encoder families tested under multiple retention rates on different benchmarks. Sometimes the relative scores exceed the unpruned model's, demonstating the effectiveness of the proposed method.
  2. The paper includes practical inference measurements that account for compression overhead rather than reporting theoretical FLOPs alone.
Critical weaknesses
  1. TimePrune is built upon a vital assumption: Token compression methods achieve higher downstream performance with better temporal trajectory preservation. While this assumption is partly supported by experiment results in Table 1~4, but a rigorous correlation or causal analysis is absent.
  2. Performance gain over baselines is marginal, especially on individual benchmarks.
Questions for authors
  1. I have a few questions on terminology:

1a. What is the definition of temporal representation trajectory?

1b. What is the trajectory-driven refinement? Does it mean Time-Aware RDP?

1c. What is the region Rm\mathcal{R}_m in eq. 2? How many tokens are in each region, and how are region boundaries set?

  1. How do Observation 1 and 2 motivate your method?

2a. Observation 1 shows that Time-Aware RDP trades temporal coverage for trajectory fidelity, but does higher temporal coverage guarantees better downstream accuracy under fixed compression ratio? If this is an assumption, it should be stated and explained explicitly; if it is a claim, it needs experimental support.

2b. Observation 2 finds that trajectory-driven refinement naturally produces adaptive temporal resolution, but it remains unclear whether adaptive temporal resolution improves accuracy under fixed compression ratio. The paper should theoretically explain or empirically test how adaptive temporal resolution influences downstream performance after token compression.

A minor suggestion is to include a limitation section in your paper.

2 comments

We thank the reviewer for the constructive feedback. We have clarified the motivation, added analyses linking our observations to downstream performance, and included a limitations section. References are to the revised PDF (changes in blue).

W1 / Q2. Is the link between trajectory preservation and accuracy an assumption or a tested claim?

It is a design hypothesis that we now state explicitly and test. We do not assume that higher fidelity or higher coverage alone improves accuracy (Sec. 3.3); the hypothesis is that the two matter jointly. This relationship is a trade-off with an interior optimum, which is the premise of TimePrune: on all four ALLMs, accuracy peaks at intermediate coverage (β=2 or 3), not at maximal fidelity (β=∞) or at the strongest coverage evaluated (β=1). A single correlation coefficient across heterogeneous methods cannot capture such a trade-off, so we use controlled interventions under a fixed token budget, changing one factor while holding the others fixed.

Q2a (Observation 1, coverage). Varying only β (Table 3; Qwen2.5, 90% pruning), β=2 lowers Avg. TGR from 6.200 to 2.500 and raises Avg. Score from 46.4 to 48.7. β=1 leaves TGR unchanged (2.499) but lowers the score to 48.1. So stronger coverage does not guarantee higher accuracy, but removing excessive gaps helps. Coverage refinement improves Avg. Score over Time-Aware RDP on all four ALLMs: +3.6, +2.4, +1.0, +0.7 (Tables 9–12).

Q2b (Observation 2, adaptive resolution). Varying only the anchor criterion, with budget and coverage rule fixed (Tables 13–16), deviation-guided anchors beat midpoint anchors on all four ALLMs: 50.0 vs. 47.1 (Qwen2.5), 53.8 vs. 51.7 (AF-Next), 61.3 vs. 58.2 (Qwen3), 62.4 vs. 61.5 (MOSS). Adaptive allocation alone is not enough, however: Time-Aware RDP concentrates anchors most (highest CAR) but leaves large gaps, and higher CAR does not always improve accuracy (Fig. 7(b); App. G.4).

Fidelity alone. Exact dynamic programming minimizes interpolation error optimally, yet is below TimePrune in MMAU accuracy in all four settings tested (e.g., 64.0 vs. 66.0 on Qwen2.5 and 70.2 vs. 75.0 on AF-Next at 85%; Table 31).

Connection to the design. Observation 2 motivates deviation-driven selection, which allocates more tokens to high-variation regions; Observation 1 identifies its risk of excessive gaps, which motivates Temporal Coverage Refinement. These are aggregate-level interventions, not sample-level causal estimates.

W2. Are the gains meaningful on individual benchmarks?

We agree that individual margins are often modest and that TimePrune is not best on every task; we do not claim significance for single differences. Our claim is consistency:

Tables 1–2 (54 benchmark columns)TimePruneFLoC (next best)
Mean rank among 10 methods1.73.9
Best or tied2711
Top three5028

At the aggregate level, TimePrune has the highest Rel. Score in 17 of 18 model × pruning settings (Tables 1–2, 7–8; mean 96.4 vs. 94.2 for FLoC), and its largest shortfall from the best method is 0.2 points, vs. 5.1 for FLoC. Excluding MELD, where the gold transcript narrows differences, TimePrune still has the highest Rel. Score in all ten settings of Tables 1–2. Fine-grained speech is the weakest case (MMAU-Speech); we analyze speech under-retention in App. G.5 and state it in Sec. 6.

Q1a. Temporal representation trajectory. It is the time-ordered sequence of audio encoder outputs x1,…,xTx_1,\ldots,x_T, viewed as a curve in feature space indexed by time (Secs. 2.1, 3.1). Retained tokens are its temporal anchors, so this view lets us measure the two properties TimePrune balances: how closely the anchors follow the curve (fidelity) and how far apart they are in time (coverage).

Q1b. Does trajectory-driven refinement mean Time-Aware RDP? Yes, it is Time-Aware RDP's anchor-selection rule: it repeatedly keeps the original token deviating most from the time-synchronized interpolation between neighboring anchors (Sec. 3.2). This adds anchors where the trajectory changes rapidly (Observation 2) but can leave large gaps (Observation 1), so TimePrune adds Temporal Coverage Refinement for overly sparse intervals (Sec. 3.3).

Q1c. Regions RmR_m in Eq. (2). Each RmR_m is a block of 10 consecutive encoder tokens; blocks do not overlap and start at the first token, and a shorter final block is kept if it has at least two tokens. CmC_m is the mean adjacent cosine distance within the block. Regions are used only for this analysis, never for token selection (Sec. 2.2).

Limitations. Following the reviewer's suggestion, we added Sec. 6, covering variability across models and budgets, loss of fine-grained acoustic information, and audio beyond 10 minutes or streaming.

We sincerely appreciate the reviewer's suggestions. We hope these controlled analyses and clarifications address the concerns regarding methodological motivation and performance consistency.

Thanks a lot for clarifying your motivation and terminology, which helps me properly understand your work. I now understand the contribution of TimePrune as a novel method to achieve better fidelity-coverage trade-off. The revised manuscript is now clearer and easier to follow. Therefore, I would like to raise my score.

Reviewer_eFdJOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The authors propose TimePrune, a training-free method for downsampling audio representations in audio LMs, with the aim of reducing the length of the input that the LM decoder must process at inference time. The method selects a subset of the audio encoder’s output representations to serve as anchors and discards the rest. Starting from the first and last frames, it repeatedly picks the interval between neighboring anchors whose intermediate frames are worst approximated by linear interpolation between the interval’s endpoints and retains the frame with the largest deviation as a new anchor, splitting the interval in two. A second rule, Temporal Coverage Refinement, gives priority to any interval longer than β times the uniform spacing. A second rule, Temporal Coverage Refinement, gives priority to any interval longer than β times the uniform spacing. The results across major audio benchmarks like Clotho-AQA, MMAU, etc. are competitive versus other techniques for downsampling, and the computational overhead at inference time is small.

Critical strengths

Audio LMs scale quadratically (in attention cost) with input audio length, and the frame rate is often quite high, around 40ms per frame for Qwen, and reducing the number of audio tokens that need to be processed is a worthy goal with real-world implications. TimePrune is training-free and a drop-in component for any audio LM that uses an audio encoder and appears to perform well against recent alternatives.

Critical weaknesses

The authors describe what TimePrune does and how it drops audio representations, but their formulation seems somewhat arbitrary. Does it correspond to an approximate solution of some loss minimization problem? It isn't clear from the text what motivates the sequence of design choices that the authors made for TimePrune.

Heuristics with unclear motivations are acceptable if there is a significant boost in performance, but it isn't clear if the differences between TimePrune and the 9 other downsampling methods are meaningful. For models like Qwen3-Omni-30B and MOSS-Audio-8B (Table 2), the average differences between using all of the audio tokens and the downsampled audio tokens are already small, since all methods for downsampling are close to the baseline model's performance (e.g., ~0.97x baseline performance is not uncommon). I would also add that fine-grained tasks that need to retain more acoustic detail, like MMAU-Speech, suffer the most from downsampling, and TimePrune can underperform when compared to the alternatives here.

With respect to the reported results themselves, I would like to know how the hyperparameters were selected, and I wonder if the hyperparameters were chosen directly on the test sets. I searched the text, and the authors never mentioned the use of a "validation"/"development"/"held-out" set anywhere. In addition, according to Sec. 4.3, TimePrune's deviation metric was chosen based on Table 5’s scores, and Table 5’s best row is identical to the Table 1 results, which is reported on the eval/test splits. Please state what data was used to select β and the deviation metric for each model and report each model’s configuration (including Qwen3-Omni). If the choices used the test splits, you should re-select on held-out data and re-report the main tables.

Questions for authors

Can the authors state the optimization problem TimePrune approximates? If it’s minimum-error piecewise-linear approximation under a K-anchor budget, I believe that an exact DP solution is possible. Also, why is interpolation error is the right proxy given that the LLM never sees interpolated audio tokens?

2 comments

We thank the reviewer for the careful review. We have added formal analyses, independent hyperparameter validation, and further experiments. Revisions are in blue.

W3. How were β and the deviation metric selected?

Each model uses a single configuration, selected once and applied unchanged to all benchmarks and pruning ratios: β=2, cosine for Qwen2.5 and AF-Next; β=3, cosine for Qwen3; β=3, ℓ2 for MOSS. These were initially identified on the evaluation benchmarks (now stated in Sec. 4.1).

As requested, we re-selected on held-out data: all 24 β × metric combinations on MMAR-Dev (299 samples), unused in the original evaluation (App. A.6).

ModelBest on MMAR-DevAcc. (%)Runner-up
Qwen2.5β=2, cosine54.8553.51
AF-Nextβ=2, cosine54.8554.52
Qwen3β=3, cosine62.5461.87
MOSSβ=3, ℓ230.7730.43

Held-out re-selection returns the adopted configuration for all four ALLMs, so Tables 1–2 are unchanged under MMAR-Dev selection. Margins over the runner-up are small (1–4 questions), indicating agreement between both procedures.

We then applied them, without re-tuning, to MMAR-Test (701) and TREA (600). TimePrune stays above Uniform and Time-Aware RDP in all eight comparisons (Tables 21, 23–29), showing they transfer.

W2. Are the performance differences meaningful?

We agree that individual gains are modest, particularly on Qwen3 and MOSS, and claim no significance for single comparisons. Across all 18 model × pruning settings, however, TimePrune is the most consistent (Tables 1–2, 7–8):

MethodMean Rel. ScoreBest settingsMax. gap to best
TimePrune96.417/180.2
FLoC94.21/185.1
HeadRouter92.40/1815.6

The closest competitor changes with the model, and none is close on all four. On Qwen3, TimePrune and HeadRouter are effectively tied (0.1 at every ratio), but HeadRouter needs calibrated attention-head profiles and the text query; TimePrune needs neither. On long audio (MMAU-Pro, 15% retention), the tie breaks: 94.2 vs. 87.7 (Table 40).

On MMAU-Pro, the gains are meaningful in both speed and accuracy. VisionZip and DivPrune take 130–288 ms yet score lower; TimePrune's latency is very close to the fastest baselines, Seg. Top-K and HeadRouter (≤2 ms), while scoring 1.7–13.9 points higher (Tables 38–41).

The "~0.97×" clustering partly reflects MELD, where the gold transcript (standard for audio–text emotion recognition) keeps all methods near Vanilla. Without MELD on Qwen3 at 85%, TimePrune and HeadRouter retain 97.1 and 96.8, and the other eight baselines at most 93.4.

On MMAU-Speech, the reviewer is right (Qwen3 85%: DivPrune 61.0 vs. TimePrune 58.0). Favoring high-variation regions (Obs. 2) can under-retain speech (App. G.5); coverage refinement mitigates this, though not fully (Sec. 6). Here, gains appear on Sound/Music (80.8/72.4 vs. DivPrune 76.6/70.1).

W1 / Q. What optimization problem motivates TimePrune, and why not exact DP?

App. B states the motivation as a budget-constrained bi-objective problem (Eq. 11): minimize the maximum interpolation deviation Emax⁡(A)E_{\max}(A) (fidelity) and the normalized maximum gap G(A)G(A) (coverage), subject to ∣A∣=K|A|=K. TimePrune is a greedy heuristic for this trade-off; we do not claim global or Pareto optimality.

We implemented exact DP for the single-objective minimax problem (App. B.2) and compared MMAU accuracy (matched budgets; Table 31):

ModelPruningExact DPRDPTimePrune
Qwen2.585%64.065.166.0
Qwen2.590%59.960.161.8
AF-Next85%70.272.375.0
AF-Next90%69.968.571.4

DP attains the lowest Emax⁡E_{\max} in every setting, yet trails TimePrune in all four and Time-Aware RDP in three. Minimizing interpolation error alone is therefore not sufficient, consistent with treating coverage as a second objective (TimePrune's TGR is 55.5–64.0% lower). DP also costs O(KT2)O(KT^2) after O(T3d)O(T^3d) precomputation, impractical for long audio.

Why interpolation deviation?

Interpolation is a redundancy criterion, neither a reconstruction for the LLM, which sees only original tokens (App. B.3), nor a measure of semantic importance. A token its retained neighbors already predict adds little new information, so deviation spends the budget on tokens that do. This also bounds the loss: every dropped token lies within Emax⁡(A)E_{\max}(A) of its anchor segment. In audio, large deviations mark abrupt changes such as onsets or transitions, which receive more tokens (Fig. 3). With budget and coverage rule fixed, deviation-based anchors beat Midpoint/Adjacent Change on all four ALLMs (50.0 vs. 47.1/46.8 on Qwen2.5; Tables 13–16).

We sincerely appreciate the reviewer's suggestions. We hope the disclosed selection procedure, held-out re-selection, and optimization analysis address the soundness concerns.

I thank the authors for responding to my review with a detailed rebuttal. My concerns and questions have been addressed, and my score has been revised accordingly.

Reviewer_Tu8KOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes TimePrune, a training-free method that prunes audio tokens in audio LLMs by treating the encoder output as a temporal trajectory. Time-Aware RDP adds anchor tokens where linear interpolation between existing anchors deviates most, and Temporal Coverage Refinement first fills intervals that are too long. The method is evaluated on four audio LLMs and five audio understanding benchmarks against nine compression baselines, with ablations and an efficiency analysis.

Critical strengths
  • Simple, general, and well motivated. The method needs no training, attention maps, calibration, or text query. It runs once on encoder outputs and transfers across four models with three encoder families. The fidelity/coverage view explains clearly why each component is needed. This makes the method easy to adopt and the design easy to follow.
  • Broad and consistent evaluation. Nine baselines, four models, and pruning from 75% to 95%. TimePrune has the best or near-best Rel. Score in most settings, and the gains grow under aggressive pruning. The latency numbers include compression overhead.
Critical weaknesses
  • Preservation of temporal information is not verified. The paper's central claim is that TimePrune preserves temporal structure. However, it does not state how kept tokens are positioned. Qwen2.5-Omni and Qwen3-Omni use TMRoPE, where position IDs follow absolute time. App. B.1 only says unused placeholders are removed. If kept tokens are re-indexed, non-uniform pruning distorts the time axis unevenly. No benchmark tests timing, such as temporal grounding or event order. The claim is therefore supported by representation-level metrics (SED, TGR), but not by tasks that depend on timing.
Questions for authors
  • Do kept tokens keep their original TMRoPE temporal IDs, or are they re-indexed from 0 to K−1? Does this choice change the results?
  • How does TimePrune perform on time-sensitive tasks, such as temporal grounding or event ordering, compared with uniform pruning?
  • On MELD, pruned models often score above Vanilla (e.g., 51.4 to 60.8 on Qwen3-Omni at 85%), since the gold transcript is part of the input. Why does pruning help here? What is the Rel. Score without MELD?
  • Table 6 compares TimePrune at 15% retention with baselines at 30%. What are the latency and score of the baselines at 15%?
  • MMAU-Speech shows the largest drop for all methods (e.g., 78.4 to 58.0 on Qwen3-Omni at 85%). Does TimePrune allocate tokens differently in speech segments?
  • How does TimePrune behave on long audio (at lease minutes level), and how does compression time scale with sequence length?
1 comment

We sincerely thank the reviewer for recognizing TimePrune's simplicity, broad evaluation, and practical efficiency. We have added temporal reasoning, position-ID, speech-allocation, and matched-budget long-audio experiments to address the concerns (revisions in blue).

W / Q1. TMRoPE position IDs.

Retained tokens are re-indexed consecutively in chronological order (Sec. 3.3, App. A.7). With identical retained tokens (TREA, 90% pruning; Table 30), re-indexing beats original IDs: 417 vs. 368 of 600 correct on Qwen2.5-Omni and 398 vs. 368 on Qwen3-Omni.

As the reviewer notes, re-indexing does not preserve absolute temporal distances, and the distortion grows as spacing becomes less uniform. Yet on TREA Duration, a task that depends on temporal extent, both non-uniform methods beat Uniform on both TMRoPE models (Qwen2.5 / Qwen3; Table 21): Uniform 47.0 / 52.5, TimePrune 54.0 / 56.5, Time-Aware RDP (largest gaps in Sec. 2.1) 53.0 / 59.0. So uneven spacing after re-indexing did not prevent higher duration accuracy here.

W / Q2. Time-sensitive tasks vs. Uniform.

We added TREA (ordering, duration, counting; 200 questions each) at 90% pruning on four ALLMs (App. A.5, Table 21). Accuracy (%), Uniform → TimePrune:

ModelOrderingDurationCountingOverall
Qwen2.574.5→84.047.0→54.071.5→70.564.3→69.5
AF-Next58.0→67.540.5→47.045.0→40.547.8→51.7
Qwen359.0→62.052.5→56.578.0→80.563.2→66.3
MOSS69.5→73.537.0→38.547.0→44.051.2→52.0

TimePrune beats Uniform on ordering and duration on all four models and has the highest overall accuracy of the three compressed methods. Counting is mixed: Uniform is higher on three models, so we do not claim gains on every subtask. As with MELD, all compressed methods exceed Vanilla on counting for three models (e.g., Qwen2.5: 70.5–71.5 vs. 41.0), so we compare methods with each other. TREA does not test exact timestamp grounding.

Q3. MELD.

Following prior audio–text emotion recognition work (Sec. 4.1), all ALLMs receive the gold transcript with the audio, for Vanilla and every method. The gain over Vanilla appears for all compressed methods (Qwen3, 85%: 53.3–61.1 vs. 51.4), so it is not specific to TimePrune. One plausible, unverified explanation: with fewer audio tokens, the model relies more on the transcript, which is highly informative for MELD. Rel. Score without MELD, recomputed from Tables 1–2 (best baseline / TimePrune):

Model85%90%
Qwen2.586.8 (FLoC) / 90.983.4 (FLoC) / 84.9
AF-Next93.6 (FLoC) / 95.691.0 (FLoC) / 92.5
Qwen396.8 (HeadRouter) / 97.193.4 (HeadRouter) / 94.1
MOSS95.2 (OmniZip) / 97.793.5 (FLoC) / 94.3

TimePrune remains best in all eight settings (+0.3 to +4.1). Without MELD no method exceeds 100, so the above-100 scores come from MELD, while TimePrune's lead persists without it.

Q4. Baselines at 15% retention.

App. H.3 (Tables 38–41) evaluates all methods at 15% retention on 120 MMAU-Pro clips (3–10 min), replacing the earlier test-mini set. Rel. Score (%) and total prefilling time (ms, incl. compression):

ModelBest baselineTimePruneFastest baseline (ms)TimePrune (ms)
Qwen2.583.185.182.283.6
AF-Next105.2107.587.889.8
Qwen390.894.292.793.5
MOSS96.698.3103.2104.9

At matched retention, TimePrune has the highest observed score on all four models and is within 2 ms of the fastest baseline, whereas VisionZip and DivPrune take 130–288 ms. We do not claim significance for these 120-clip margins.

Q5. Speech segments.

Yes. A post-hoc VAD analysis (MMAU-Speech, 85% pruning, 100 clips per model, three ALLMs; App. G.5, Table 34) shows that deviation-driven selection keeps proportionally fewer tokens in speech than in non-speech regions. Temporal Coverage Refinement raises the speech/non-speech retention ratio (SAR; Uniform 0.92–0.98) from Time-Aware RDP to TimePrune: 0.416→0.865 (Qwen2.5), 0.833→0.930 (AF-Next), 0.410→0.586 (Qwen3). Speech segments with no retained token fall 8.8%→1.9%, 0.2%→0.0%, and 10.5%→8.5%. This mitigates but does not remove speech under-retention, which may contribute to the MMAU-Speech drop (e.g., DivPrune 61.0 vs. 58.0 on Qwen3 at 85%; Sec. 6).

Q6. Long audio and scaling.

On the same clips, total prefilling speedups over Vanilla are 6.27×, 5.89×, 3.35×, and 3.39× in the order above (Sec. 4.5, App. H.2). From 3–5 to 8–10 min clips (≈2.35× more tokens), compression latency grows 3.5→4.8, 3.5→5.7, 1.1→4.9, and 5.1→8.1 ms (Table 42), below 10 ms in every range. Prefilling excludes audio encoding.

We sincerely appreciate the reviewer's suggestions. We hope these additions address the concerns on temporal information preservation, evaluation fairness, and efficiency.

Reviewer_HDKPOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents TimePrune, a training-free audio token compression method that treats audio representations as a temporal trajectory. It interpolates between neighboring retained anchors and recursively selects intermediate tokens with large approximation errors. These tokens cannot be well approximated by the two anchors, so they are considered less redundant and more informative. Temporal Coverage Refinement further gives priority to overly sparse intervals to improve temporal coverage. Experiments on several audio language models and benchmarks show good performance under high compression ratios.

Critical strengths
  1. The paper introduces an effective trajectory-based approach to audio token compression, and it performs well across different compression budgets.
  2. The experiments cover multiple models and a range of audio understanding tasks, which gives broad evidence that the method works.
  3. The writing and figures are clear, and the method, its advantages, and its limitations are easy to understand.
Critical weaknesses
  1. The ablations in Table 3 and Appendix A.2 demonstrate the practical benefit of Temporal Coverage Refinement. So my concern is novelty, not effectiveness. To me it looks like an incremental engineering refinement of existing temporal-coverage ideas, and the paper should explain how it differs from prior coverage-preserving methods.
  2. Time-Aware RDP is not derived from the characteristics of audio representations and seems equally applicable to video and omni-modal token compression. Its approximation-error criterion is close in spirit to existing visual token compression methods such as ApET [1]. I would like a more detailed explanation of why this approach suits audio in particular, backed by audio-specific analysis. This would make the contribution more than transferring an existing compression principle to a new setting and reaching state-of-the-art results.
Questions for authors
  1. Is the benefit of Temporal Coverage Refinement simply a general engineering improvement, or does it address a specific property of audio tokens? If the latter, please provide an explanation or analysis supporting this connection.
  2. Could you evaluate Time-Aware RDP on purely visual token compression and omni-modal token compression? This would help me understand whether the work mainly transfers an existing method to a new setting. Even if it does, that wouldn't take away from the empirical contribution.
  3. If Time-Aware RDP works well only for audio token compression, could you provide analysis based on the characteristics of audio tokens? This would help explain why the method is especially effective for audio and make the link between the motivation and the method more convincing.
1 comment

We thank the reviewer for the constructive feedback. We have clarified TimePrune's novelty and audio-specific motivation with new analyses and experiments (revisions in blue).

W1 / Q1. Novelty of Temporal Coverage Refinement (TCR) and its relevance to audio

What TCR is and what is new. TCR is a gap-triggered rule: within trajectory-driven selection, any anchor interval longer than β times the nominal spacing is refined first. It yields temporal coverage, but its design goal is selectivity: rather than imposing coverage everywhere, it intervenes only where adaptive selection has left a gap. At equal budget, it more than halves the largest gap (TGR 6.200→2.500; Table 3). It thus regulates concentration, redirecting budget from high-variation regions that already hold many anchors to neglected intervals (CAR 2.36→1.46, still >1; Fig. 16).

Difference from prior methods. Uniform sampling and fixed segment quotas (Seg. Top-K) impose coverage everywhere, regardless of content; diversity objectives (FLoC, SCOPE, MMTok) ignore temporal position, so whole stretches can be dropped. TCR adds tokens only to under-covered intervals. FLoC also trails TimePrune at every ratio (Table 1).

Effect of TCR. Qwen2.5-Omni, 90% pruning, same budget (Table 3, Fig. 6(a)):

MethodAvg. Score
Uniform43.6
Seg. Top-K45.9
DivPrune45.5
Time-Aware RDP (no TCR, β=∞)46.4
TimePrune, β=547.1
TimePrune, β=447.8
TimePrune, β=347.3
TimePrune, β=2 (adopted)48.7

Here, every β improves Time-Aware RDP (47.1–48.7 vs. 46.4) and beats coverage-based baselines; at the adopted β, TCR improves Time-Aware RDP on all four ALLMs (Tables 9–12). This supports selective, gap-triggered coverage over imposing coverage everywhere.

Relevance to audio. On AV-SpeakerBench at 75% pruning, Time-Aware RDP leaves far larger gaps in audio than in visual representations (TGR 8.520 vs. 1.972), and TCR reduces them by 76.0% vs. 4.8% (App. C.2, Table 33). These gaps fall on speech: on MMAU-Speech at 85% pruning, Time-Aware RDP leaves 8.8% (Qwen2.5) and 10.5% (Qwen3) of VAD speech segments with no retained token. TimePrune reduces this to 1.9% and 8.5% (App. G.5, Table 34). TCR is general, but the gaps it corrects are much larger in the evaluated audio representations.

W2 / Q2. Visual-only and omni-modal compression

We added Video-MME (visual-only) and AV-SpeakerBench (joint audio-visual) with Qwen2.5-Omni at 75% pruning (App. C.1, Table 32). VisionZip and DivPrune serve as representative visual pruning baselines.

MethodVideo-MMEAV-SpeakerBench
Vanilla73.8947.08
Uniform71.6738.75
VisionZip65.0032.92
DivPrune73.8941.25
Time-Aware RDP72.2240.00
TimePrune72.7840.83

Both variants exceed Uniform and VisionZip beyond audio, while DivPrune is best. On Video-MME, all methods but VisionZip are within 4 of 180 questions of Vanilla, so which tokens are kept matters little. Once audio is included (AV-SpeakerBench), all methods lose 6–14 points vs. Vanilla, so selection matters more. On audio-only benchmarks (same model and ratio), methods diverge further: TimePrune retains 96.4% of Vanilla vs. 90.2% for DivPrune (Table 1). Though benchmarks differ, this suggests the approach helps most where tokens are less redundant over time, as in audio.

W2 / Q3. Why audio, and how does it differ from ApET?

Distinction from ApET. TimePrune's criterion follows temporal order, and it never merges tokens. It scores each token by its deviation from the time-synchronized interpolation between neighboring anchors; ApET approximates tokens from basis tokens, then selects and merges them. For audio, this targets where the signal changes over time and keeps each token's exact timing and detail, which merging can blur.

Audio representation characteristics (App. C.2). Audio is tokenized far more densely: one token per ≈40 ms, vs. one visual frame per second. At this token level, adjacent audio tokens are less similar (0.845 vs. 0.945), variation is more uneven (Gini 0.229 vs. 0.105), and deviation-driven allocation is more concentrated (CAR 5.25 vs. 2.23).

Over equal 4-second windows, the orderings reverse (Gini 0.082 vs. 0.234; CAR 1.81 vs. 2.28), so per second, audio is not more variable than video. What makes TCR matter for audio is its dense token rate: selection operates over many fine-grained tokens, where uneven allocation easily opens large gaps.

Summary. The novelty is not approximation error itself, but its time-synchronized form, its coupling with gap-based interval prioritization, and evidence that this coupling matters most for dense audio tokens.

We sincerely appreciate the reviewer's suggestions and hope these additional results clarify TimePrune's novelty and audio-specific motivation.

Comments on the paper

1 comment

We sincerely thank all reviewers for their thoughtful feedback and constructive suggestions. We appreciate their recognition of TimePrune's effectiveness, training-free design, broad experimental evaluation, and practical efficiency measurements. We have addressed the concerns through additional experiments and manuscript revisions, summarized below with the reviewers who raised each point.

For ease of review, newly added and revised content is highlighted in blue in the updated manuscript.

1. Additional Experiments

  • App. A.5 (Table 21), Temporal reasoning (Tu8K): TREA (event ordering, duration, counting) on all four ALLMs at 90% pruning. TimePrune has the highest overall accuracy among the evaluated compressed methods (Uniform, Time-Aware RDP, TimePrune) on all four models, although Uniform is higher on counting for three.
  • App. A.6 (Tables 22–29), Held-out hyperparameter validation (eFdJ): Re-selection over all 24 β × metric combinations on MMAR-Dev returns the adopted configuration for all four ALLMs, so Tables 1–2 are unchanged. The fixed configurations then transfer to MMAR-Test and TREA without re-tuning.
  • App. A.7 (Table 30), TMRoPE position IDs (Tu8K): With identical retained tokens, consecutive re-indexing outperforms original position IDs on both TMRoPE models.
  • Apps. A.2–A.4 (Tables 11, 15, 19), Qwen3-Omni ablations: The coverage, refinement-criterion, and deviation-metric ablations now cover all four ALLMs.
  • App. B.4 (Table 31), Exact dynamic programming (eFdJ): Exact DP attains the lowest interpolation error but lower MMAU accuracy than TimePrune in all four settings tested.
  • App. C (Tables 32–33, Fig. 8), Cross-modal evaluation (HDKP): Visual-only (Video-MME) and joint audio-visual (AV-SpeakerBench) experiments with cross-modal representation analyses, clarifying where the approach applies and why its benefit is largest on audio.
  • App. G.5 (Table 34), Speech token allocation (Tu8K, eFdJ): A post-hoc VAD analysis shows that Temporal Coverage Refinement reduces, but does not remove, speech under-retention.
  • Sec. 4.5 and App. H, Long-form efficiency (Tu8K): We replaced the MMAU test-mini efficiency evaluation with 120 MMAU-Pro clips (3–10 min). At 15% retention, total prefilling speedups are 3.35–6.27× across four ALLMs. We also added matched-budget baselines (H.3), compression-latency scaling (H.4), and a GPU implementation (H.5).

2. Revisions and Clarifications

  • Secs. 2–3, Motivation and terminology (1pCs): Definitions of the temporal representation trajectory, trajectory-driven refinement, and the regions in Eq. (2), and how Observations 1 and 2 motivate the two components.
  • Sec. 4.3 (Figs. 6–7), Preservation and downstream performance (1pCs): A new section relating SED, TGR, and CAR to downstream accuracy through controlled comparisons, with cross-model results in Apps. F.5 and G.4.
  • Apps. B.1–B.3, Optimization perspective (eFdJ): A budget-constrained fidelity–coverage formulation, the distinction between greedy refinement and exact optimization, and interpolation deviation as a selection proxy rather than a measure of semantic importance.
  • Sec. 4.1, Hyperparameters and MELD protocol (eFdJ, Tu8K): We state that the original configurations were identified on the evaluation benchmarks, and that all methods receive the gold transcript on MELD (details in App. D.2).
  • App. E.1, Related work (HDKP): Distinction from ApET and from fixed segment-wise coverage allocation.
  • Sec. 6, Limitations (1pCs): Added.
  • Abstract and Sec. 1: The headline efficiency figures now come from the long-form MMAU-Pro evaluation on Qwen3-Omni (94.2% of the Vanilla score at 15% retention, 89.2% fewer audio-token-dependent LLM FLOPs), replacing the earlier MMAU test-mini figures.

We sincerely appreciate the reviewers' time and constructive feedback. We believe these additions strengthen the methodological justification, empirical evidence, and practical evaluation of TimePrune, and we hope the revised manuscript addresses the reviewers' concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.