The Knowability Gap: Auditing When Better Models Cannot Replace Missing Information

Persistent prediction error can have two different causes: the learner may be inadequate, or the observation may omit information needed to determine the target. We study how to distinguish these cases before spending more computation on model scaling. For an observed input, target, declared environment family, and task metric, we define the Knowability Gap (K-Gap) as the Wasserstein Chebyshev radius of the compatible environment-conditioned target laws. Under the rectangular envelope that allows any locally admissible mechanism, K-Gap equals the optimal robust Bayes risk; for globally coupled mechanism classes contained in that envelope, it is a conservative upper audit rather than an assumption-free deployment limit. Our main finite-environment theorem gives a two-sided decomposition that separates conditional-law estimation error from environment-coverage error, and reduces to an observed-environment certificate when the declared family is finite. We also characterize when side information is guaranteed to reduce the radius and show why arbitrary conditioning need not do so. Controlled experiments recover an exact distributional K-Gap of 0.575 and verify the sample/coverage correction over 4,500 trials. On real hyperspectral data, a five-site audit is inconclusive under a frozen support rule. On GLORIA, extending 400–600 nm with 602–800 nm lowers test-size-weighted Chl-a MAE from 21.46 to 15.60; the estimated Mean K-Gap point estimate also decreases, although its campaign-bootstrap interval crosses zero. The real extension is more favorable than all 20 frozen randomized-information controls, and across four model families Medium $X+A$ has lower mean MAE than Large $X$-only. The resulting audit separates four often-conflated resources: model capacity, repeated samples, environment coverage, and measurement information.