From Unlearnable Outcomes to Decodable Shortcut Codes

Unlearnable examples (UEs) add small perturbations to training images so that models trained on them perform poorly on clean data. Low clean accuracy, however, does not reveal what these models learn. We find that shallow networks fit protected training sets more easily than clean ones while generalizing worse. Moving perturbations between images then separates two behaviors: predictions either follow the label associated with the perturbation regardless of the image, or depend on the combination of perturbation and image. The first behavior offers a direct route to unlearnability, because a model that decodes the perturbation needs no semantic features to fit the protected data. We formalize this route as a decodable perturbation shortcut (DPS) and propose a low-complexity shortcut codebook (LCSC), which learns one code per class that randomly initialized networks can identify on changing images. At $8/255$ with crop-and-flip augmentation, LCSC reduces clean accuracy to near chance on four datasets and five architectures; at $1/255$, its epoch-60 accuracy on CIFAR-10 is $9.8\%$. Its factorized variant gives the strongest protection in eight of nine JPEG settings and at adversarial-training radii of $2/255$ and $4/255$. Both variants are constructed tens to hundreds of times faster than TAP, SEP, and REM.