One Attention Head Makes Where to Look Almost Free: The Cost of Unknown Support with and without Adaptive Selection

A transformer must decide where to look before it decides what to compute. We measure the statistical price of that decision in nonparametric regression. The target depends on an unknown set of $k$ of its $D$ input coordinates, one token each, through a smooth function of $k$ variables, namely an anisotropic Besov ball with a sup-norm envelope; the design is i.i.d. with a density bounded above and below uniformly in $D$ and $n$, the noise is Gaussian, and $k$ is fixed. Without adaptive selection the price is combinatorial. Every estimator that is affine in the training responses, with offset and coefficients fixed before the responses are read, as in kernel ridge regression at a label-independent regularisation level, random-feature ridge, or lazy/NTK rules, has worst-case risk at least $c\min \{1,(M/n{)}^{\beta }\}$ with $M=\left(\genfrac{}{}{0ex}{}{D}{k}\right)$, which does not vanish once $M\ge n$; for integrability index $1\le p<2$ the exponent $\beta$ is the smaller, shifted one. One positional attention head makes the price additive and polylogarithmic in $D$. The exact clipped least-squares minimiser over an explicitly budgeted class of one-head transformers with positional keys attains $n^{-2\bar{s}/(2\bar{s}+1)}$ in the effective smoothness $\bar{s}$, up to logarithmic factors in $n$ and with a constant free of $D$, plus a single additive selection term polylogarithmic in $D$. The mechanism is a whitening of the positional codes: for every support, the head can attend equally to the $k$ relevant tokens, with only a controlled remainder elsewhere, inside one parameter budget that does not depend on which tokens they are. At $D=n$ with all structural parameters fixed, the response-affine worst-case risk stays bounded away from zero while the attention bound vanishes. The saving is not specific to attention, since a direct search over supports attains the same leading bound; what attention supplies is a compact, continuous realisation of it. The upper bound concerns the exact minimiser, and no optimisation guarantee is claimed.