- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces CIVIC, a method that aggregates tokens within the intermediate layers of a vision encoder. These compact representations are then propagated through subsequent vision layers, the projector, and the language model’s prefill phase, complemented by text-alignment distillation. The authors evaluate the approach on Qwen3-VL-4B-Instruct using the Jetson AGX Orin platform, measuring both raw execution time and normalized latency. While latency is reduced, performance varies significantly by task: scores decline on MMMU and MathVision, improve on RealWorldQA, and remain largely stable on Video-MME.
Critical strengths
-
The paper focuses on actual hardware costs, measuring the overheads associated with encoding, projection, prefill, and aggregation, rather than relying solely on token counts or theoretical computational complexity to estimate speedups.
-
Results on RealWorldQA and Video-MME (Table 2) demonstrate favorable quality-latency trade-offs; furthermore, comparisons involving the restoration of dense tokens and visual KV pooling provide insight into the sources of these practical gains.
Critical weaknesses
-
In Table 2, the MMMU score drops from 0.714 to 0.285, and the MathVision score from 0.480 to 0.305, representing declines of 42.9 and 17.5 percentage points, respectively. In this context, the reduction in latency is insufficient to justify a superior quality-efficiency trade-off. The paper lacks task-specific quality-latency curves and fails to specify the extent to which acceleration is maintained when performance approaches that of the dense model.
-
The evaluation is limited to a single model backbone, one device, and a batch-size-of-one configuration. Given the authors' emphasis that practical gains depend on model interfaces and the computational costs of hardware operators, this scope is insufficient to demonstrate generalizability. Furthermore, the systematic comparison of aggregation layer placement is inadequate, making it difficult to determine whether the 16th layer represents an optimal balance between information retention and computational savings.
Questions for authors
-
Could you provide native score-vs.-latency curves for MMMU and MathVision across multiple compression budgets? When quality degradation is kept within a narrow range, is there still a significant speedup compared to the dense model and key baselines?
-
Why is there a marked performance degradation on MMMU and MathVision, while RealWorldQA shows improvement and Video-MME remains largely stable? Could you explain these discrepancies through error analysis, considering factors such as the location of compression, compression intensity, and the use of distillation?
-
Could you include representative experiments using a different model configuration or a different type of hardware, reporting both quality and latency, to determine whether the observed gains are dependent on the current setup? If this is not feasible, please clearly define the scope and limitations of your conclusions.