A Two-Scalar Law for Activation Functions: How Peak Slope and Peak Curvature Govern Adversarial Robustness

Which intrinsic, measurable properties of an activation function govern adversarial robustness? We answer: exactly two scalars, in provably disjoint roles. For $L$-layer fully connected networks with any twice-differentiable activation, we derive an exact path-summed decomposition of every diagonal element of the loss Hessian and prove that each element admits a closed-form upper bound affine in $M_2=\max |{\sigma }^{\prime \prime }|$—the peak curvature—while every other activation-specific quantity in the bound reduces to the peak slope $M_1=\max |{\sigma }^{\prime }|$ and to the prediction residual $|f-y|$. This two-scalar law yields four a-priori, falsifiable predictions: (P1)once $M_2$ is large enough that fitting capacity is sufficient—so that the residual $|f-y|$ no longer inflates the bound—the normalized Hessian diagonal norm grows approximately monotonically with $M_2$; (P2)below this regime, the activation's fitting capacity becomes insufficient, which inflates the residual $|f-y|$ and drives a second rise of the same norm; (P3)in this same regime—where the residual no longer differs across variants—larger $M_1$ yields a provably larger bound, and correspondingly larger measured norms and faster robustness degradation, in one and the same ordering; (P4)activations of distinct functional forms share the same rise–peak–decline robustness profile along the common axis $M_2$. Testing the law requires decoupling the two scalars; the Recursive Curvature-Tunable Activation Family (RCT-AF) provides exactly this, with $\alpha$ scanning $M_2=c_{\beta }\alpha$ while $M_1(\beta )\in \{1.0,1.1,1.281\}$ stays fixed per variant independently of $\alpha$. All four predictions are confirmed across ResNet-18 and WideResNet-28-10, CIFAR-10, CIFAR-100, and TinyImageNet-200, and DAJAT, DKL, and TRADES under AutoAttack: robustness consistently attains its maximum in a common intermediate band of $M_2$ for every functional form and dataset, and large-$M_2$ degradation rates order by $M_1$. These results reduce the design of robust activation functions to two measurable scalars.