DOES PREDICTIVE UNCERTAINTY GUIDE US TO THE MISSING INFORMATION? A CONTROLLED AUDIT UNDER PARTIAL EEG OBSERVATION
Abstract
Predictive uncertainty can identify difficult predictions, but can it also identify which unavailable measurements matter? We audit this stronger interpretation through controlled channel withholding in EEG: the history, future target, horizon, and retained channel count stay fixed while channel identity changes. A standard heteroscedastic predictor trained on 1,600 hours produces sample-dependent, error-associated uncertainty. Yet its measurement-level interpretation breaks down in a specific way. Teacher-state consequence rankings reproduce across disjoint subject halves with correlation approximately , and uncertainty rankings with – across three constructions, while their aggregate cross-half alignment is weak or negative. Both rankings therefore contain population structure, but that structure does not reliably align; many individual mask-pair orderings also remain unresolved. To characterize what the score tracks, we measure Local Predictive-State Dispersion, a model-defined structural probe that covaries positively with uncertainty. An analytic oracle shows that interpreting this probe as exact conditional ambiguity is regime-dependent. Together, these results establish a controlled boundary between recognizing prediction difficulty and identifying missing-measurement consequences, motivating validation against the specific semantic claim made about an uncertainty score.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.