- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies offline evaluation of active feature acquisition (AFA) policies using tabular foundation models and historical data with missing features. Its central observation is that total predictive entropy can penalize policies reaching poorly supported states by conflating epistemic and aleatoric uncertainty. The proposed framework instead estimates posterior expected conditional entropy, combining Bayesian predictive inference, importance weighting, and the Bayesian bootstrap to obtain population-level policy-value estimates. The contribution concerns policy evaluation rather than policy learning. Experiments show reduced estimation bias relative to the chosen reference values and improved policy ranking in some synthetic settings. On Sepsis, improved value estimation does not translate into better policy selection.
Critical strengths
- The paper identifies an important limitation of entropy-based AFA evaluation: epistemic uncertainty can obscure the value of informative but sparsely observed features.
- The framework connects uncertainty decomposition to population-level policy evaluation, with experiments demonstrating reduced estimation bias and improved interval coverage.
Critical weaknesses
-
The methodological novelty is limited. The framework largely combines established posterior inference techniques with importance-weighted AFA evaluation, with its main contribution lying in the choice of evaluation objective.
-
Bootstrap already reduces estimation bias, while Marginal and CLT produce nearly identical point estimates. Joint sampling improves interval coverage, but the practical benefit of this additional posterior information for comparing candidate policies remains unclear.
-
On Sepsis, reduced estimation bias does not improve policy selection. The appendix provides useful missingness and uncertainty diagnostics, but does not directly connect differences in coverage and correction across candidate policies to successful ranking changes. The evidence therefore offers limited guidance on when the correction improves acquisition decisions.
Questions for authors
- The proposed estimators improve policy ranking in some synthetic settings, but similar gains are not observed in the Sepsis experiments. Could the authors explain, based on the existing results, which policy characteristics or data conditions determine whether the correction changes the relative ranking of candidate policies?
- Compared with combining existing posterior expected entropy estimators and established weighted AFA evaluation methods, which steps require new methodological designs when extending these approaches to policy value estimation under missing data? Please clarify the specific challenges addressed by these components and their roles in the overall algorithm.
- The Marginal–CLT comparison demonstrates the benefit of joint sampling for policy-value intervals. Does the implementation also preserve posterior dependence across candidate policies, using shared conditional-distribution, propensity, and Bayesian bootstrap draws? For two policies evaluated on the same data, can the resulting posterior of their value difference provide stronger evidence about which policy is preferable than separate value intervals?
3 comments
W3 / Q1. When the correction changes policy selection
[W3] On Sepsis, reduced estimation bias does not improve policy selection. The evidence therefore offers limited guidance on when the correction improves acquisition decisions.
[Q1] The proposed estimators improve policy ranking in some synthetic settings, but similar gains are not observed in the Sepsis experiments. Could the authors explain, based on the existing results, which policy characteristics or data conditions determine whether the correction changes the relative ranking of candidate policies?
Response. We thank the reviewer for this insight. Appendix A.3 theoretically shows that the correction changes regret only when the true value gap is smaller than the epistemic gap at the final states the policy reaches. So the candidate policies must be close in true value for the correction to matter.
We built this setting for the sepsis ICU experiment:
- Generated 20 stochastic multi-step policies and computed their reference values.
- Selected the 10 with the narrowest value gaps, subject to a minimum gap that rules out ties and near-ties.
Initial results (10 bootstrap replications) show improved policy ranking:
| Variant | Pairwise acc. (%) | Δ vs. plug-in | -value |
|---|---|---|---|
| Total (plug-in) | ref. | – | |
| Total (bootstrap) | 0.766 | ||
| Bootstrap | 0.859 | ||
| Marginal (CLT) | 0.031 | ||
| Copula | 0.004 | ||
| CLT | 0.422 |
-values from a one-sided paired test against the plug-in.
Changes. We will update the main text with the full set of replications, replacing the original ICU multi-step policy evaluation experiment, along with all corresponding details (coverage, propensity quantiles, ESS, etc.).
Q2. Which components require new methodological design
Compared with combining existing posterior expected entropy estimators and established weighted AFA evaluation methods, which steps require new methodological designs when extending these approaches to policy value estimation under missing data? Please clarify the specific challenges addressed by these components and their roles in the overall algorithm.
Response. Our frequentist baseline builds directly on the missing-data policy value estimator of von Kleist et al [2025]. Rather than claiming algorithmic novelty, our methodological contribution lies in formulating its Bayesian counterpart, which, to our knowledge, has not been studied in the literature. Doing so requires obtaining joint functional draws and propagating posterior uncertainty across all three components of the joint distribution (the missingness propensity model , the outcome model , and the covariate marginal ).
Thank you again for your time and efforts in reviewing our paper, and we hope that you will consider raising your score if you find our response satisfactory.
Thanks for the detailed response and additional experiments. The revisions address several of my concerns, but some concerns including methodological novelty, robustness and generalizability of new Sepsis results remain. I believe the evidence is still insufficient to fully resolve my concerns. Therefore, I will maintain my current rating.
We thank the reviewer for their careful reading and constructive feedback. We appreciate their recognition of our insight that epistemic uncertainty can obscure the value of informative, sparsely observed features, and we respond to each of their points below; all changes are marked in blue in the revised PDF.
W1. Methodological novelty
Response. We clarify the scope of our contributions in the following. Our goal is to enable rigorous evaluation of any feature acquisition using off-the-shelf foundation models and provide an estimation approach to do so. Mainly, we provide an unexplored Bayesian conceptualization of feature acquisition that separates out sources of uncertainties, and believe our exploratory analysis and insights are novel in this space. Finally, we note that reliable AFA policy evaluation is an important but understudied problem, as it is a prerequisite for developing new optimization methodologies and enables the data-driven assessment of hand-crafted clinical diagnostic criteria.
W2 / Q3. Practical benefit of the joint posterior for policy comparison
Response. We thank the reviewer for this insight, and agree that the posterior could provide stronger evidence in policy selection beyond computing the posterior expected entropy. We note that our current posterior expected entropy criterion is Bayes optimal for minimizing regret, and that rankings across candidate policies depend only on the bias of the estimator, not on the posterior width and calibration.
Yes, posterior dependence is preserved. All candidate policies are jointly evaluated using the same posterior samples. We assess the calibration of the joint posterior with additional metrics:
Appendix D.4 now contains all new calibration results. Here the advantage of the Bayesian estimators over Bootstrap is apparent: they yield better-calibrated estimates of the probability that one policy outperforms another. As an illustrative example, we provide the results for the BNN synthetic DGP family.
Calibration of P(Ga>Gb) against the true order for pairs of multi-step policies πa,πb on the BNN DGPs; mean ± SEM across DGP seeds.