- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper noticed a difference between human recognition and vision model perception that sometimes vision models fails to recognize an image not because the visual information is missing, but because the information is too sparse. To study the sparsity, the paper constructs a benchmark called DN AbstraGrid that includes different abstraction types and levels, and notices that blurring can benefit the sparse partition. Motivated by this, they propose Squint!, a lightweight module that predicts the time and how much an image should be blurred. Experiments show that Squint! improves model performance across various VLMs.
Critical strengths
- The paper has a clear structure and easy to follow. The paper first introduces the background and motivations, then proposes Squint! based on the observations and uses extensive experiments to support the ideas.
- The experiments are conducted across different models and datasets, and controlled experiments show the module can also work on the original domain.
- The module is lightweight. The Squint! module is a simple MLP and can be easily applied to the preprocessing process.
Critical weaknesses
- The labels rely on the CLIP's images and captions similarities, and higher CLIP similarity does not mean better recognition accuracy. The paper uses the blur level provided by teacher model to supervise the recognition models, and shows Squint! can improve performance across different models. However, it doesn't validate that different model backbones' optimal blur levels are same.
- Adaptive test-time image transformation has already been explored in previous work, and frequency-based methods are widely used to preserve low level information. But the paper does not directly compare Squint! against these approaches and show proposed module is better than the previous methods. Additional experiments should be conducted for existing adaptive test-time image transformation methods on the same datasets and compare the performance.
- The paper also lacks the frequency-based baselines. Squint! uses Fourier specturm to learn a MLP, but it is not clear whether simple methods such as thresholding high frequency energy or rule based on spectral statistics can achieve similar performance. Comparing the lightweight baselines is necessary to show the effect of the learning framework. Also, the absolute accuracy is still low on sparse dataset. For example, average DG in main table increases from 3.7 by 7.3, where it shows relative improvements but absolute accuracy is still low. It shows that blur can mitigate the domain shift, but cannot solve the recognition issue caused by sparse visual information.
Questions for authors
Please see the weaknesses above.
2 comments
Dear Reviewer aHcD, we are following up on our clarification questions posted four days ago. Could you please provide the requested citations?
Dear Reviewer aHcD, we are following up on our clarification questions posted three days ago. Could you provide citations for the methods whose comparison would be sufficient to address point 2, and describe the spectral baseline you had in mind for point 3? Your guidance would help us ensure that the additional comparisons address your concerns.