Revealing Invisible Defects in Visual Representations: A Psychometric Instrument for Binding

Task accuracy can conceal differences in how visual representations respond to changes in attribute–object pairings. We introduce BInD, the Binding Intervention Difference, a diagnostic protocol for frozen visual encoders. BInD compares cosine-distance responses to pairing-preserving and pairing-breaking image edits, without human annotation or a trained probe at scoring time. It measures readout-specific sensitivity, not the total binding information recoverable by an arbitrary decoder. Synthetic and real-photo suites cover color, load, arrangement, and a size control at pooled and patch interfaces. Across 32 encoders, we report reliability, axis-specific associations, and comparisons with eleven intrinsic metrics under training-objective and capability controls. Within-recipe interventions change several metrics substantially without consistently improving binding readouts. Our strongest application is compression: two DINOv2 merge configurations have nearly identical ImageNet accuracy but retain 86% versus 34% of the reference color score. Pruning can preserve that score while degrading segmentation. Distillation and vision-language case studies further show why the interface and evaluation task matter. We provide suite specifications, a scoring pipeline, and an interpretation guide with explicit limits on causal and real-image claims.