- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies visual token pruning for image emotion recognition. With the multimodal model frozen, the authors train a lightweight Evidence Head using image-level emotion labels to predict task-relevance scores for visual patches. During training, the head learns through soft masking and an answer-prediction loss, together with budget regularization. At inference, its scores are combined with CLS-to-patch attention to select the top-K tokens for the language model. Experiments evaluate accuracy, macro-F1, and FLOPs across two backbones and six emotion datasets.
Critical strengths
-
Clear task motivation and a lightweight trainable module. Emotion cues can be subtle and spatially scattered, and generic visual saliency may fail to preserve them. The method trains a lightweight scoring head using only image-level labels, without requiring evidence-region annotations or updating the backbone, resulting in a relatively simple design.
-
Evaluation across multiple datasets with component ablations. The authors report accuracy, macro-F1, and results from three independent training runs. Comparisons of Head-only, CLS-only, combined scoring, and different loss components provide some support for the task-relevant scoring mechanism.
Critical weaknesses
-
The full method depends on CLS attention, but its applicability to CLS-free architectures remains unvalidated. At inference, the method combines Evidence Head scores with CLS-to-patch attention using a fixed weight. Both evaluated backbones provide this signal, so the experiments do not establish how the method applies to visual encoders without a CLS token. Although the Head-only ablation shows that the scoring head can operate independently, it does not demonstrate performance on CLS-free architectures. Furthermore, a separate head is trained for each backbone, which should not be interpreted as direct transfer of the same head. The paper should clarify this architectural dependency and the method’s scope of applicability.
-
Evidence of practical inference efficiency is insufficient. The paper primarily measures efficiency using per-image TFLOPs, without reporting corresponding end-to-end latency, TTFT, or throughput. Reduced computation does not directly establish actual speedup, particularly because the implementation also involves the scoring head, CLS attention extraction, and top-K selection. Measurements covering these operations under the same optimization settings as the baselines are needed to assess the practical deployment benefits.
Questions for authors
-
How can the full method be applied to visual encoders without a CLS token?
-
Could you report the actual latency or TTFT of the complete inference pipeline, including visual encoding, CLS attention extraction, the scoring head, token selection, and language model computation?
-
Could you provide Head-only results from three independent training runs on both backbones to assess the actual benefit of the CLS signal and whether this benefit is stable across different datasets and token retention budgets?
Comment
We sincerely thank the reviewer for the time and effort spent reviewing our work. If our responses have addressed your concerns, we would be grateful if you could kindly consider raising your score. If any concerns remain, we would be happy to discuss them further and address them.