- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
LLM-RegProto addresses weakly supervised event-camera semantic segmentation using sparse point annotations. Its central component projects event features and learnable class queries through a frozen text-only LLM layer to construct class prototypes without text prompts. A spatial adapter enables bidirectional token interactions, while a dual-stream network aligns foreground prototypes across complementary temporal windows and suppresses background noise. Experiments on DDD17 and DSEC evaluate one-point and ten-point annotation budgets per class. Under the ten-point setting, the method achieves 62.55% and 53.23% mIoU, improving over EV-WSSS by 2.16 and 2.04 points. Component ablations, pretraining controls, and prototype visualizations support the reported improvements.
Critical strengths
-
The framework presents an innovative combination of frozen LLM layers, learnable class queries, and temporal prototype alignment, requiring neither textual prompts nor paired multimodal training.
-
The method consistently outperforms EV-WSSS across both datasets and annotation budgets, delivering gains of 2.16 and 2.04 mIoU points under ten-point supervision on DDD17 and DSEC.
Critical weaknesses
-
Comparisons omit modern WSSS approaches leveraging CLIP, DINO, or SAM spatial priors. Adapting such methods to events under matched supervision would better establish the proposed language-based regularizer’s distinctive contribution.
-
Projection into an LLM’s hidden space does not guarantee semantic alignment. Although Figure 3 and Table 3 show benefits, they do not validate the claimed universal, modality-independent semantic clustering geometry.
-
The LLM layer remains active during inference, yet latency, memory consumption, and computational overhead are unreported. Its practical value for low-latency event-based perception therefore remains unclear despite the accuracy improvements.
Questions for authors
Please see weaknesses.