Access and Representation Barriers to Auditing under Proxy-Score Optimization

Language models are optimized against cheap proxy scores, but how costly is it to audit the resulting target change? We identify access and representation barriers linked by likelihood-ratio geometry. Baseline archives face a no-hit lower bound, while active auditors trade observable-transcript KL against sample count. An exact $L^1$ duality characterizes worst-case target change hidden from a fixed feature span. For exact single-prompt one-sided exponential-power tilts, minimax log archive complexity and half-mass likelihood-ratio-tail rarity asymptotically match optimization KL. Using exact conditional reweighting on a finite pool of $207,872$ judge-scored Qwen2.5-7B math traces, we find a $2.56$-order increase in the tested archive requirement, seven practical samplers above the movement–sample benchmark, and substantial held-out error in fitted score-blind monitors. At 18 score-matchable Group Relative Policy Optimization (GRPO) checkpoints, the plug-in KL residual from a score-matched one-parameter tilt accounts for $96.7$–$99.8\%$ of measured KL, indicating substantial departure from that tilt path. Reported LR-tail estimates differ substantially from score-tail rarity, although differing likelihood-evaluation paths and a finite reference pool limit their interpretation. This motivates measuring LR-tail overlap under a common likelihood convention before using proxy-score tails as overlap diagnostics.