One Pixel Wider, a Different Click: How Image Preprocessing Destabilizes GUI Grounding

Stable GUI grounding is essential for reliable interaction by computer-use agents. However, we find that changing a screenshot dimension by a single pixel can alter which UI element a model selects, with these changes occurring disproportionately at particular image sizes. Why can one pixel produce a different click? This paper identifies dynamic-resolution preprocessing, which maps raw screenshot dimensions to discrete visual token grids or tile layouts, as one source of this instability. We call the image sizes where these configurations change preprocessing transition points. Across eight models and three benchmarks, the average UI-element switching rate nearly doubles at these points relative to adjacent one-pixel changes. Controlled interventions confirm that these configuration changes contribute to the excess switching. Further analysis in two models shows that resizing and content alignment affect grounding beyond changes in token count alone. These findings motivate three investigations. First, because switching concentrates at transition points, we show how to “steal” hidden preprocessing settings from clicks. The inferred settings help predict unstable screenshot sizes, with predictions validated on unseen screenshots. Second, we use sensitivity to small movements of interface content to detect errors: clicks whose selected UI element changes under these movements are more likely to be incorrect. Combining this consistency check with model confidence improves error detection. Finally, guided by the effects of resizing and content alignment, we design a mitigation method that improves grounding stability. Together, these analyses and investigations offer insights into why one pixel can change a click and how to improve the stability of GUI grounding.