- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
AudioRouter uses synchronized audio to guide visual token compression for streaming video LLMs. Causal audio features and temporal context condition learnable queries. The queries produce routing weights over native visual tokens, yielding without a separate audio input stream to the LLM. Only the bottleneck is trained, using knowledge distillation and audio-visual alignment on external video data. With 64 tokens per frame, the paper reports 60.19 on VideoMME, 72.40 on StreamingBench RTVU, and 68.4 on MLVU. It also reports lower memory usage and LLM prefill latency than full-token inference.
Critical strengths
- Practical Design. AudioRouter uses audio cues while preserving the standard visual-token interface of a frozen VLM. Each audio window ends at the corresponding frame timestamp, making the routing causal. The design offers a practical way to use synchronized audio without an audio-capable LLM.
- Measured Efficiency Gains. The bottleneck has about 2.0M trainable parameters. Table 5 reports a reduction in peak memory from 58.22 GB to 29.70 GB and in LLM prefill time from 10.93 s to 1.85 s. The TTFT improvement is smaller, from 23.06 s to 20.88 s, but the memory and prefill savings are substantial.
Critical weaknesses
W1. Unresolved Evaluation Inconsistencies. The reported scores do not match the stated evaluation protocol. The full VideoMME benchmark has 900 questions in each duration group. Under the question-level aggregation in Appendix C, Overall should therefore equal the mean of Short, Medium, and Long, up to rounding. AudioRouter reports 60.19, whereas its three scores average to 59.57. Similar discrepancies occur at other token budgets in Table 6. The official RTVU annotations contain 128 CR questions, which cannot produce the reported 88.89% accuracy in a single evaluation of the complete category. The discrepancies suggest an undisclosed subset, a different aggregation procedure, or reporting errors. They do not establish the actual evaluation size. Until the question sets and scoring procedures are clarified, the claimed gains over Full Vision and prior compressors cannot be reliably assessed.
The training description also limits reproducibility. The manuscript omits the training-set size, the source of KD questions, and key loss and routing hyperparameters. A separate checkpoint is trained for each benchmark, but the validation and checkpoint-selection procedures are not reported. The broken training-data reference in Sec. 4.1 leaves these details unresolved.
W2. Insufficient Evidence For The Contribution Of Audio. Appendix B describes bypassing BEATs for and shifting audio features for . It does not state whether either control is trained separately. If audio is removed only at inference, the drop from 60.19 to 49.81 measures sensitivity to removing a learned input. The drop does not establish an advantage over a vision-only bottleneck trained with the same data, token budget, and KD objective. A matched training comparison is needed to isolate the contribution of audio conditioning.
The routing analysis provides limited additional support. If Table 3 uses with natural logarithms, uniform routing has CE , below both reported values of 10.47 and 8.70. The CE definition and a uniform baseline are needed to interpret the improvement. The qualitative evidence also contains inconsistent labels. The same driveway scene is labeled Burglary in Figure 2 and Arrest in Figure 4. Two other repeated surveillance scenes also receive different answers. The mismatches make the examples harder to interpret as evidence of task-relevant routing.
Overall. I recommend rejection because the main comparisons cannot be verified under the stated protocol, and the audio contribution remains insufficiently isolated. Consistent evaluation results and a matched training control could change my assessment.
Questions for authors
- Report the evaluated question counts and sample IDs for each benchmark and ablation. Explain the aggregation discrepancies in W1 and identify which baseline results were rerun. Provide AudioRouter and Full Vision results on identical question sets with the same frame sampling and scoring protocol.
- Clarify whether and were trained separately and whether retains temporal context. Compare AudioRouter with a vision-only bottleneck trained on the same data with the same token budget and KD objective.
- Specify the training-set size, KD question source, , routing and InfoNCE temperatures, negative-sampling rules, and queue size. Explain how the benchmark-specific checkpoints and training schedules were selected, including any validation data used.
- Define the CE direction and log base in Table 3, report the number of evaluated videos, and include uniform routing as a reference. Resolve the conflicting question-answer labels in Figures 2 and 4.