The Arbitrary-Placement Problem in Entropy-Minimizing Selection, and a Residual-Entropy Formulation

Entropy-based selection objectives suffer from a fundamental degeneracy: minimizing Shannon entropy $H(p_A)$ rewards confident selection regardless of whether the selected candidate is informative. We address this limitation with the residual entropy $D=H(p_A)-H(p_{\beta })$, where $p_{\beta }$ is induced directly by candidate trust weights before they are aggregated into scores. We prove the exact identity $D=-\mathrm{KL}(p_A\Vert p_{\beta })-\Delta$, where $\Delta$ measures whether the score-induced distribution and the underlying trust profile favor the same candidates. The boundary cases establish basic safety: under uniform trust, $D\le 0$ automatically, so an equal-trust, non-starving state is never penalized, while at any one-hot limit, $D\to 0$ regardless of the selected candidate. Our main results address the intermediate regime where selection occurs. We prove that $D\le 0$ when the ordering of candidates by trust agrees pairwise with their ordering by informativeness, giving a checkable pairwise condition for safe informative selection. We further derive a tighter certificate based on the margin of the leading candidate over its competitors, which determines when that candidate can be selected while preserving $D\le 0$. These results are independent of the candidate-scoring function and apply to both stationary and dynamically changing information. Experiments with a gradient-based mixture-of-experts router confirm that the ordering conditions can hold during real optimization and show that correct ordering improves downstream performance when candidates are non-interchangeable and selections are used directly rather than averaged. Beyond routing, margin-based reweighting matches or outperforms fixed-strength baselines in a class-imbalance task, while informative selection in a production video-prediction system reduces MSE by approximately 20% and transfers to a related species. Residual entropy, therefore, provides a safety criterion for selection and a usable signal for deciding when that selection is informative.