- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies fixed-focal selective prediction for vision-language models (VLMs) and proposes SPAR, a risk estimation framework that exploits the relational structure between focal and peer model outputs. By distinguishing focal isolation from peer ambiguity, SPAR provides more informative reliability signals than scalar disagreement measures. Experimental results demonstrate consistent, though relatively modest, improvements over competitive baselines.
Critical strengths
-
The research motivation is clear and well justified. Within the scope of fixed-focal selective prediction, distinguishing focal–peer disagreement from peer–peer agreement is a reasonable and meaningful approach to reliability estimation.
-
The experimental evaluation is comprehensive, covering diverse VLM architectures, datasets, transfer settings, and ablation studies, providing substantial empirical support for the proposed method.
Critical weaknesses
The proposed framework relies on multiple peer VLMs and requires training an additional risk-scoring model. Despite its efficiency improvements over repeated-sampling methods, deploying and querying multiple models still introduces considerable computational and deployment overhead, which may limit its practical applicability.
Questions for authors
None beyond the concerns raised above.