Interpreting Samplewise Function Perturbations: Signal, Noise, and Nonlinear Structure

Samplewise adversarial perturbations have been motivated as stronger regularization, but a larger worst-case penalty does not specify how the learned predictor changes. We study exact samplewise logistic estimation in an RKHS with a unit-diagonal kernel and fixed positive ridge, also realized by per-example perturbations of normalized fixed-feature heads. A common operator governs weak-signal population response and balanced-label limiting covariance. Increasing the radius raises the gain of every observable signal mode and inflates the limiting covariance, although the balanced population minimizer remains zero. These leading responses are not specific to the perturbed objective: ordinary logistic regression with an analytically chosen ridge and output scale matches both exactly. The agreement leaves nonlinear structure undetermined. A general cubic interaction operator identifies the first mechanism-specific difference after matching, with a fifth-order remainder. In a periodic weak-signal model, its third-harmonic projection has opposite local responses: samplewise perturbation attenuates the harmonic while its matched control amplifies it. This attenuation coexists with norm growth, and the Bayes score itself contains the harmonic. Separate $O(n^{-1})$ covariance-error bounds quantify nonlinear fitting and averaged random-design effects. Independent kernel and frozen ResNet/ViT fits test the predictions, while ordinary logistic controls reproduce the fixed-setting task gains. Thus, samplewise perturbation cannot be summarized as uniform shrinkage, and component attenuation alone does not establish predictive benefit.