VISTA: Visual Information Stress Testing for Representation Shifts and State-Dependent Influence in Vision-Language Models

What changes inside a vision-language model when a visual category remains recoverable under perceptual stress? We study matched color-desaturation and shape-defocus sequences across eight checkpoints, combining frozen readouts, native candidate scoring, and visual-state interventions. We find that severe clean-readout mismatch has a simple correctable component: a training-estimated, class-agnostic translation restores Shape readout accuracy to 0.979–1.000 in the frozen IID evaluation, while representation discrepancies are only partly removed. On a separate, explicitly linked evaluation cohort, all categories remain recoverable by a shared reader, yet Color stress consistently reduces true-class relative evidence and candidate accuracy; Shape changes the evidence in model-dependent directions while endpoint decisions remain near ceiling. Finally, recipient stress strengthens a fixed clean–stress visual contrast in Qwen–Color but weakens it in LLaVA–Shape. These opposite modulations persist across same-family, cross-family, and finite-interval constructions, with natural-direction amplitude advantages over matched controls; other settings are less consistent. The findings characterize translation-correctable readout mismatch, attribute-dependent answer changes, and heterogeneous conditional visual influence rather than a uniform loss of categorical information.