acceptodds
Desk rejected at ICLR 2027

Evaluating VLMs' Sensitivity to Image Resolution and Detail Level

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Visual Language Models (VLMs) have achieved remarkable success across diverse tasks, yet they struggle with high-resolution inputs where critical information resides in small regions or detailed, cluttered scenes. While several approaches address this limitation, a systematic understanding of why models fail at high resolutions is lacking. We introduce a controlled evaluation framework that disentangles resolution-related performance degradation from task difficulty through semantics-preserving transformations. We propose two simple metrics: Area Under the Scaling Curve (AUSC), which quantifies scaling robustness independent of baseline accuracy, and Prediction Variance Score (PVS), which measures resolution-induced prediction instability. Through comprehensive experiments across four model families and five benchmarks, we investigate architecture-dependent resolution sensitivity in relation to three pipeline mechanisms: (1) information loss from downsampling at vision token limits, (2) tokenization and positional sensitivity under changes in patch layout and aspect ratio, and (3) attention dilution as token counts increase. Our analysis reveals that widely used VLM architectures suffer from performance drops when processing high-resolution images, with degradation patterns varying systematically by architectural family. We provide actionable insights for model architecture design and data augmentation strategies to mitigate these limitations.

Primary area:
foundation or frontier models, including LLMs
Keywords:
VLM, high-resolution image processing, resolution robustness

Related papers

Discussion (0)

Sign in to comment.