acceptodds
Withdrawn from ICLR 2027

Squint Your Eyes: Choosing When to Blur for Sparse Visual Domains

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Visual recognition is typically studied under shifts in style or corruption, but the effect of spatially disconnected visual evidence remains underexplored. We study visual sparsity, where objects are depicted as separate dots, symbols, or line fragments. Motivated by the human tendency to squint when encountering fragmented visual patterns, we investigate whether blurring helps recognition under sparsity. We conduct a controlled human study and an analogous model evaluation. Blur improves recognition under sparsity for both humans and modern recognition models, but the relative improvement is substantially larger for models, indicating that they are more vulnerable to sparsity. This human–model mismatch creates an exploitable gap for moderation attacks, where sparse images can remain recognizable to people yet be substantially harder for moderation systems. We demonstrate this failure mode in a content-moderation setting. To study sparsity systematically, we introduce DN AbstraGrid, a benchmark based on Mini DomainNet, that spans multiple abstraction levels: a sparse target partition, a non-sparse partition, and the original domains. The latter two serve as control domains where performance should be preserved. Blur strongly benefits the sparse partition while harming the remaining domains, showing that it should be applied adaptively rather than uniformly. To this end, we propose Squint!, a lightweight, model-agnostic module that predicts when and how much to blur each image. It is based on a compact radial Fourier power-spectrum descriptor, trained with weak supervision and not on the target sparse domains. Squint! improves recognition on sparse domains while preserving performance on the controls. A single plug-and-play Squint! model improves performance across 12 VLMs and 7 domain-generalization methods, while also improving recognition of sparse moderation attacks.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
visual sparsity, human–model discrepancy, domain generalization, visual abstraction, vision-language models, image obfuscation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_aHcDOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper noticed a difference between human recognition and vision model perception that sometimes vision models fails to recognize an image not because the visual information is missing, but because the information is too sparse. To study the sparsity, the paper constructs a benchmark called DN AbstraGrid that includes different abstraction types and levels, and notices that blurring can benefit the sparse partition. Motivated by this, they propose Squint!, a lightweight module that predicts the time and how much an image should be blurred. Experiments show that Squint! improves model performance across various VLMs.

Critical strengths
  1. The paper has a clear structure and easy to follow. The paper first introduces the background and motivations, then proposes Squint! based on the observations and uses extensive experiments to support the ideas.
  2. The experiments are conducted across different models and datasets, and controlled experiments show the module can also work on the original domain.
  3. The module is lightweight. The Squint! module is a simple MLP and can be easily applied to the preprocessing process.
Critical weaknesses
  1. The labels rely on the CLIP's images and captions similarities, and higher CLIP similarity does not mean better recognition accuracy. The paper uses the blur level provided by teacher model to supervise the recognition models, and shows Squint! can improve performance across different models. However, it doesn't validate that different model backbones' optimal blur levels are same.
  2. Adaptive test-time image transformation has already been explored in previous work, and frequency-based methods are widely used to preserve low level information. But the paper does not directly compare Squint! against these approaches and show proposed module is better than the previous methods. Additional experiments should be conducted for existing adaptive test-time image transformation methods on the same datasets and compare the performance.
  3. The paper also lacks the frequency-based baselines. Squint! uses Fourier specturm to learn a MLP, but it is not clear whether simple methods such as thresholding high frequency energy or rule based on spectral statistics can achieve similar performance. Comparing the lightweight baselines is necessary to show the effect of the learning framework. Also, the absolute accuracy is still low on sparse dataset. For example, average DG in main table increases from 3.7 by 7.3, where it shows relative improvements but absolute accuracy is still low. It shows that blur can mitigate the domain shift, but cannot solve the recognition issue caused by sparse visual information.
Questions for authors

Please see the weaknesses above.

2 comments

Dear Reviewer aHcD, we are following up on our clarification questions posted three days ago. Could you provide citations for the methods whose comparison would be sufficient to address point 2, and describe the spectral baseline you had in mind for point 3? Your guidance would help us ensure that the additional comparisons address your concerns.

Dear Reviewer aHcD, we are following up on our clarification questions posted four days ago. Could you please provide the requested citations?

Reviewer_9dZkOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper addresses visual sparsity (objects depicted by disconnected elements like ASCII, dots, or line fragments separated by gaps). The authors demonstrate that deep neural networks are significantly more vulnerable to visual sparsity than humans. Motivated by the human tendency to squint, they show that blurring improves model recognition. To implement this adaptively, they propose Squint!, a lightweight, model-agnostic preprocessing MLP module. Squint! takes a 64-D radial Fourier power-spectrum descriptor of an image and predicts which of 9 Gaussian blur levels to apply. It is evaluated across 12 VLMs and 7 domain-generalization methods on a newly introduced benchmark called DN AbstraGrid.

Critical strengths

Intuitive and creative motivation: Leveraging the human "squinting" behavior to improve deep neural network shape recognition under visual sparsity is exceptionally creative.

Highly lightweight and model-agnostic: The MLP module requires negligible computational overhead (< 0.01 GFLOPs) and functions seamlessly as a plug-and-play preprocessing step for any downstream model.

Critical weaknesses

Limited Comparison with Strong Baselines: The proposed Squint! preprocessing module is only compared against a few test-time adaptation methods (MEMO, TENT) and basic ensembling. It would be significantly more convincing to compare it against a broader suite of strong frequency-based domain generalization techniques or advanced adversarial defense methods.

Marginal Improvement in Non-VLM Backbones: On standard domain generalization (DG) baselines (e.g., ResNet-18), the absolute accuracy after applying Squint! remains very low (often under 15%), suggesting that while the relative gain is high, it is not a sufficient standalone solution for standard architectures.

Questions for authors

How does Squint! compare to training models with randomized frequency-domain augmentations (e.g., Fourier-based mixup)?

In real-world content moderation, how robust is the Fourier power-spectrum descriptor against adaptive adversaries who might slightly perturb the high-frequency components to fool Squint!?

2 comments

Dear Reviewer 9dZk, we are following up on our clarification questions posted three days ago. Could you provide citations for the frequency-based DG and adversarial defense methods you had in mind, and indicate which comparisons would be sufficient to address your concern? We would also appreciate clarification on the adaptive attack setting you envisage.

Dear Reviewer 9dZk, we are following up on our clarification questions posted four days ago. Could you please provide the requested citations?

Reviewer_Wm1TOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies a new problem of recognizing sparse images that comprise objects drawn as disconnected dots, characters, and lines. The authors conduct a human study with 16 people, showing that an appropriate level of blur helps humans improve their performance from 60.5% to 76.8% and, by the blur/no-blur accuracy ratio, helps models even more. They build a benchmark DN AbstraGrid using Mini DomainNet, with two types of partitions: the target sparse domains and the control ones. They propose a small MLP named Squint! to select one of nine blur levels for each image based on its Fourier descriptor.

Critical strengths

(1) It is a new problem and the authors clearly show that blur helps all 16 participants.

(2) Squint! has a low cost, does not require model outputs, improves the average accuracy of the sparse domains for all 23 rows in Table 1, and has a small effect on the average of the control domains.

Critical weaknesses

(1) A simple baseline (MC-E) is much stronger than Squint!, and the authors only partially consider other cheap approaches. While MC-E can improve the non-sparse control partition, Squint! only outperforms MC-E on Mini DomainNet. Can the authors consider (a) MC-E with only two or three blur levels (at a cost close to Squint!), (b) MC-E gated by entropy, (c) fixed blurs, and (d) a threshold on high-frequency energy?

(2) Squint! does not control the blur strength for the hardest images. this is a problem because the abstract states that Squint! "predicts when and how much to blur each image". can the authors examine what blur levels Squint! will choose for the 512 images shown to the humans?

(3) The gap between the humans and models only exists for VLMs. By the error-reduction rate computed from Table 4, the VLMs are indeed more vulnerable to the problem than humans, but all eight DG models are less vulnerable, and the paper gives no significance test. Can the authors change the claim to only consider VLMs, and show the confidence intervals?

(4) The authors do not describe several key details in their experiments. First, the authors do not describe the obfuscations used for training Squint!, the exact formula for correcting the alignment, and the VLM used for selection, and they only report one trained Squint! model in Table 1 without showing the variance.

Questions for authors

(1) How do the multi-source DG models achieve 99.3% to 99.7% on Mini DomainNet in Table 1 without any overlap between the training and testing data?

(2) Does Squint! give a similar gain on other moderators, and how does it compare to a fixed blur?

Comments on the paper

1 comment

Dear Area Chair and Reviewers,

We have decided to withdraw this submission and seek another venue. We requested clarification from Reviewers 9dZk and aHcD on November 8, followed up on November 11 and 12, and requested Area Chair intervention on November 12. Those requests remain unanswered at the time of this comment. Given the absence of follow-up, we do not see a productive path forward in this discussion.

For future readers, we provide the following answers based on evidence already included in the manuscript, with the corresponding reviewer concerns identified.

Our contributions are the study of visual sparsity, the DN AbstraGrid benchmark with preservation controls, a controlled human study, and a lightweight blur selector evaluated across 12 VLMs and 7 DG methods.

  1. Fixed-blur comparisons — Wm1T, Weakness 1(c), Question 2. Figure 2 compares fixed Gaussian blur strengths across ACVC, Crafting, CLIP-B, and PE-Core-B, showing the trade-off between sparse recognition and original-domain performance. Figure 7 additionally compares Gaussian blur, downsampling, and ideal low-pass filtering.

  2. Three-candidate MC-E — Wm1T, Weakness 1(a). Figure 8 evaluates 1, 3, 5, 9, and 17 candidates for Crafting and CLIP-B, including both control partitions. The requested three-candidate comparison is already present.

  3. Training variability — Wm1T, Weakness 4. Figure 5 repeats each ablation configuration 20 times, reporting means and one-standard-deviation contours. Table 1 evaluates one selected selector across backbones to demonstrate plug-and-play transfer.

  4. High Mini DomainNet accuracy — Wm1T, Question 1. Section 6 and Appendix A.4 explain that MIRO, SIMPLE, and ERM++ train on the training splits of all four original domains. Their Mini DomainNet results are in-domain evaluations on held-out images. Sections 3 and 5 describe the separate training, validation, and test roles. DN AbstraGrid evaluates generalization to new renderings of test images.

  5. MC-E trade-off — Wm1T, Weakness 1. Section 6 explicitly acknowledges MC-E's higher sparse accuracy. Average DG sparse accuracy is 3.7%3.7\% without blur, 7.3%7.3\% with Squint!, and 11.9%11.9\% with MC-E. Corresponding Mini DomainNet accuracy is 74.9%74.9\%, 74.9%74.9\%, and 72.8%72.8\%. Squint! uses one recognition pass and also improves MC-E while pruning candidates. Figure 10 reports the computation trade-off.

  6. Supervision and transfer — aHcD, Weakness 1. Section 5 and Figure 4 show that CLIP-derived targets train the blur selector. The downstream recognition backbones are not retrained in the main plug-and-play evaluation. Table 1 demonstrates transfer across backbones. The supervised variant in Section 6 changes both the target-construction model and supervision signal while retaining improvements.

  7. Human–model metric — Wm1T, Weakness 3. Sections 1 and 4 define $$R=A_{blur}/A_{no\ blur}$). Under this metric, Table 4 reports higher ratios than humans for 7 of 8 DG configurations and all 12 base VLMs. Separately, Section 4 and Appendix A.2 establish human improvement: (60.5% to 76.8%,withall16participantsimprovingandanexactparticipant−levelone−sidedsign−test, with all 16 participants improving and an exact participant-level one-sided sign-test p=1.53 ×\times 10^{-5}$.

  8. Existing adaptation and robustness comparisons — 9dZk, Weakness 1; aHcD, Weakness 2. Section 6 evaluates fine-tuning with blur, ImgAug, and their combination, demonstrating complementary gains from Squint!. Table 2 compares against MEMO and TENT. Figure 5 examines alternative frequency-suppression strategies and ablates the alignment correction, entropy regularizer, and preservation filter.

  9. Reported moderation evaluation — Wm1T, Question 2. Table 3 evaluates GPT 5.4, reporting sparse-positive accuracy of 11.3%to16.6%11.3\% to 16.6\% and sparse-negative accuracy of 98.6%to99.2%98.6\% to 99.2\%, with preserved decisions on the selected original controls. Appendix A.3 specifies the prompt and set construction.

  10. Absolute DG accuracy — 9dZk, Weakness 2; aHcD, Weakness 3. Section 6 explicitly discusses the substantial robustness gap. Table 1 reports average DG accuracy increasing from 3.7%3.7\% to 7.3%7.3\%, a gain of 3.6 percentage points, alongside preserved average control performance.

We leave these answers on the public record so readers can assess the contribution and review concerns against the evidence already provided.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.