$\gamma$-CMI: Non-Myopic Active Feature Acquisition via Probabilistic Lookahead

Active Feature Acquisition (AFA) is a sequential decision-making process that selectively acquires unobserved features for each data instance independently to improve predictive performance under a limited acquisition budget. A common approach is to greedily select the next feature by maximizing its conditional mutual information (CMI) with the prediction target given the currently observed features. However, this criterion considers only the immediate information gain of each candidate feature and may therefore overlook feature interactions, where a feature becomes more informative when considered jointly with other features that may be acquired subsequently. To address this limitation, we propose $\gamma$-CMI, a generalization of the CMI acquisition criterion that evaluates candidate features under probabilistically sampled future observation contexts. $\gamma$-CMI provides a continuous spectrum of lookahead, ranging from evaluating a candidate's immediate information gain through Greedy CMI ($\gamma =0$) to evaluating its information gain under the fully sampled future context of all remaining features ($\gamma =1$). Crucially, $\gamma$-CMI can be incorporated into existing CMI-based AFA methods in a plug-and-play manner, requiring only minor modifications to their acquisition criteria and no additional model retraining. We provide a theoretical analysis characterizing the acquisition behavior induced by $\gamma$-CMI, demonstrate its ability to capture feature interactions on diverse synthetic datasets, and show consistent improvements in early-stage acquisition performance across diverse real-world datasets and CMI-based backbones. Code is available https://anonymous.4open.science/r/gamma-cmi-E3EE/here.