acceptodds
Under review as a conference paper at ICLR 2027

Evaluation of Active Feature Acquisition Policies with Tabular Foundation Models

Abstract

Active feature acquisition learns policies that sequentially acquire features to maximize information about a target variable. We study how to learn and evaluate such policies from finite offline data using prior-data fitted networks (PFNs), which are off-the-shelf models that output posterior predictive distributions without task-specific training. We show that under the imbalanced coverage of offline data, using total predictive entropy as a reward creates an epistemic bias that penalizes acquiring sparsely observed features. Specifically, this reward conflates *epistemic uncertainty* (arising from lack of offline data) with *aleatoric uncertainty* (arising from uninformative features). To address this, we target the posterior expected (aleatoric) entropy instead of the total predictive entropy output by a PFN for evaluating feature acquisitions. Empirical evaluations on synthetic and real-world datasets demonstrate that our approach consistently reduces value estimation bias and yields credible intervals with strong empirical coverage, which can translate to improved downstream policy selection.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Active Feature Acquisition, Missing Data, Uncertainty Estimation, Bayesian Inference

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_JDtcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies offline evaluation of active feature acquisition (AFA) policies using tabular foundation models and historical data with missing features. Its central observation is that total predictive entropy can penalize policies reaching poorly supported states by conflating epistemic and aleatoric uncertainty. The proposed framework instead estimates posterior expected conditional entropy, combining Bayesian predictive inference, importance weighting, and the Bayesian bootstrap to obtain population-level policy-value estimates. The contribution concerns policy evaluation rather than policy learning. Experiments show reduced estimation bias relative to the chosen reference values and improved policy ranking in some synthetic settings. On Sepsis, improved value estimation does not translate into better policy selection.

Critical strengths
  1. The paper identifies an important limitation of entropy-based AFA evaluation: epistemic uncertainty can obscure the value of informative but sparsely observed features.
  2. The framework connects uncertainty decomposition to population-level policy evaluation, with experiments demonstrating reduced estimation bias and improved interval coverage.
Critical weaknesses
  1. The methodological novelty is limited. The framework largely combines established posterior inference techniques with importance-weighted AFA evaluation, with its main contribution lying in the choice of evaluation objective.

  2. Bootstrap already reduces estimation bias, while Marginal and CLT produce nearly identical point estimates. Joint sampling improves interval coverage, but the practical benefit of this additional posterior information for comparing candidate policies remains unclear.

  3. On Sepsis, reduced estimation bias does not improve policy selection. The appendix provides useful missingness and uncertainty diagnostics, but does not directly connect differences in coverage and correction across candidate policies to successful ranking changes. The evidence therefore offers limited guidance on when the correction improves acquisition decisions.

Questions for authors
  1. The proposed estimators improve policy ranking in some synthetic settings, but similar gains are not observed in the Sepsis experiments. Could the authors explain, based on the existing results, which policy characteristics or data conditions determine whether the correction changes the relative ranking of candidate policies?
  2. Compared with combining existing posterior expected entropy estimators and established weighted AFA evaluation methods, which steps require new methodological designs when extending these approaches to policy value estimation under missing data? Please clarify the specific challenges addressed by these components and their roles in the overall algorithm.
  3. The Marginal–CLT comparison demonstrates the benefit of joint sampling for policy-value intervals. Does the implementation also preserve posterior dependence across candidate policies, using shared conditional-distribution, propensity, and Bayesian bootstrap draws? For two policies evaluated on the same data, can the resulting posterior of their value difference provide stronger evidence about which policy is preferable than separate value intervals?
3 comments

We thank the reviewer for their careful reading and constructive feedback. We appreciate their recognition of our insight that epistemic uncertainty can obscure the value of informative, sparsely observed features, and we respond to each of their points below; all changes are marked in blue in the revised PDF.


W1. Methodological novelty

The methodological novelty is limited. The framework largely combines established posterior inference techniques with importance-weighted AFA evaluation, with its main contribution lying in the choice of evaluation objective.

Response. We clarify the scope of our contributions in the following. Our goal is to enable rigorous evaluation of any feature acquisition using off-the-shelf foundation models and provide an estimation approach to do so. Mainly, we provide an unexplored Bayesian conceptualization of feature acquisition that separates out sources of uncertainties, and believe our exploratory analysis and insights are novel in this space. Finally, we note that reliable AFA policy evaluation is an important but understudied problem, as it is a prerequisite for developing new optimization methodologies and enables the data-driven assessment of hand-crafted clinical diagnostic criteria.


W2 / Q3. Practical benefit of the joint posterior for policy comparison

[W2] Bootstrap already reduces estimation bias, while Marginal and CLT produce nearly identical point estimates. Joint sampling improves interval coverage, but the practical benefit of this additional posterior information for comparing candidate policies remains unclear.

[Q3] Does the implementation also preserve posterior dependence across candidate policies, using shared conditional-distribution, propensity, and Bayesian bootstrap draws? For two policies evaluated on the same data, can the resulting posterior of their value difference provide stronger evidence about which policy is preferable than separate value intervals?

Response. We thank the reviewer for this insight, and agree that the posterior could provide stronger evidence in policy selection beyond computing the posterior expected entropy. We note that our current posterior expected entropy criterion is Bayes optimal for minimizing regret, and that rankings across candidate policies depend only on the bias of the estimator, not on the posterior width and calibration.

Yes, posterior dependence is preserved. All candidate policies are jointly evaluated using the same posterior samples. We assess the calibration of the joint posterior with additional metrics:

  • Pairwise preference. We estimate P(Ga>Gb)P(G_a > G_b) for two policies aa and bb as 1K∑k=1KI[Ga(k)>Gb(k)]\frac{1}{K} \sum_{k=1}^{K} \mathbb{I}[G_a^{(k)} > G_b^{(k)}]. This gives a confidence that one policy is better than the other, which we evaluate with log loss, Brier score and ECE.
  • Posterior calibration. We compute P(θ≤θtrue∣z1:n)P(\theta \leq \theta_{\text{true}} \mid z_{1:n}) for θ=G\theta = G and θ=Ga−Gb\theta = G_a - G_b. This checks the calibration of the posterior for both the true value and the true value difference between two policies.

Appendix D.4 now contains all new calibration results. Here the advantage of the Bayesian estimators over Bootstrap is apparent: they yield better-calibrated estimates of the probability that one policy outperforms another. As an illustrative example, we provide the results for the BNN synthetic DGP family.

EstimatorMethodLog loss ↓Brier ↓ECE ↓
IPWTotal (plug-in)2.193±0.0802.193 \pm 0.0800.317±0.0120.317 \pm 0.0120.317±0.0120.317 \pm 0.012
Total (bootstrap)2.245±0.0812.245 \pm 0.0810.325±0.0120.325 \pm 0.0120.325±0.0120.325 \pm 0.012
Bootstrap0.979±0.0440.979 \pm 0.0440.244±0.0090.244 \pm 0.0090.195±0.0090.195 \pm 0.009
Copula0.675±0.0250.675 \pm 0.0250.215±0.0070.215 \pm 0.0070.122±0.0070.122 \pm 0.007
CLT0.609±0.0200.609 \pm 0.0200.203±0.0070.203 \pm 0.0070.090±0.0070.090 \pm 0.007
Marginal (CLT)0.602±0.0190.602 \pm 0.0190.202±0.0070.202 \pm 0.0070.084±0.0060.084 \pm 0.006

Calibration of P(Ga>Gb)P(G_a > G_b) against the true order for pairs of multi-step policies πa,πb\pi_a, \pi_b on the BNN DGPs; mean ± SEM across DGP seeds.


W3 / Q1. When the correction changes policy selection

[W3] On Sepsis, reduced estimation bias does not improve policy selection. The evidence therefore offers limited guidance on when the correction improves acquisition decisions.

[Q1] The proposed estimators improve policy ranking in some synthetic settings, but similar gains are not observed in the Sepsis experiments. Could the authors explain, based on the existing results, which policy characteristics or data conditions determine whether the correction changes the relative ranking of candidate policies?

Response. We thank the reviewer for this insight. Appendix A.3 theoretically shows that the correction changes regret only when the true value gap is smaller than the epistemic gap at the final states the policy reaches. So the candidate policies must be close in true value for the correction to matter.

We built this setting for the sepsis ICU experiment:

  1. Generated 20 stochastic multi-step policies and computed their reference values.
  2. Selected the 10 with the narrowest value gaps, subject to a minimum gap that rules out ties and near-ties.

Initial results (10 bootstrap replications) show improved policy ranking:

VariantPairwise acc. (%)Δ vs. plug-inpp-value
Total (plug-in)75.6±4.875.6 \pm 4.8ref.–
Total (bootstrap)74.9±4.574.9 \pm 4.5−0.7±1.4-0.7 \pm 1.40.766
Bootstrap74.2±4.474.2 \pm 4.4−1.3±2.0-1.3 \pm 2.00.859
Marginal (CLT)76.9±4.976.9 \pm 4.9+1.3±0.7+1.3 \pm 0.70.031
Copula78.7±4.7\mathbf{78.7 \pm 4.7}+3.1±0.8\mathbf{+3.1 \pm 0.8}0.004
CLT76.4±5.376.4 \pm 5.3+0.9±1.1+0.9 \pm 1.10.422

pp-values from a one-sided paired test against the plug-in.

Changes. We will update the main text with the full set of replications, replacing the original ICU multi-step policy evaluation experiment, along with all corresponding details (coverage, propensity quantiles, ESS, etc.).


Q2. Which components require new methodological design

Compared with combining existing posterior expected entropy estimators and established weighted AFA evaluation methods, which steps require new methodological designs when extending these approaches to policy value estimation under missing data? Please clarify the specific challenges addressed by these components and their roles in the overall algorithm.

Response. Our frequentist baseline builds directly on the missing-data policy value estimator of von Kleist et al [2025]. Rather than claiming algorithmic novelty, our methodological contribution lies in formulating its Bayesian counterpart, which, to our knowledge, has not been studied in the literature. Doing so requires obtaining joint functional draws and propagating posterior uncertainty across all three components of the joint distribution (the missingness propensity model πβ\pi_\beta, the outcome model μ\mu, and the covariate marginal P(X)\mathbb P(X)).


Thank you again for your time and efforts in reviewing our paper, and we hope that you will consider raising your score if you find our response satisfactory.

Thanks for the detailed response and additional experiments. The revisions address several of my concerns, but some concerns including methodological novelty, robustness and generalizability of new Sepsis results remain. I believe the evidence is still insufficient to fully resolve my concerns. Therefore, I will maintain my current rating.

Reviewer_xLZjOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

Paper deals with active feature acquisition. It looks at theoretical investigation into effects of aleatoric and epistemic uncertainty to acquisition and provides experiments on three synthetic and one real world scenarios.

Critical strengths

Active acquisition is a relevant topic of recent research.

Uncertainties and their decomposition play an important role.

Disclaimer: my stochastic expertise is insufficient to verify the theory, e.g. martingales and such. This requires another reviewers input.

Critical weaknesses

I) Write up hard to follow: 58: "We prove that under standard information gain objectives" -> where do I find corresponding theorem / lemma? 60: "We show that, relative to the aleatoric objective" -> again: where do I find formal proof for that claim?

Fig 3:

  • What is hat{v}_k and v_k? It is not mentioned in the caption and also hard to find in main text
  • DGP: abbreviation need introduction
  • with l316: what is the "accuracy of downstream acquisition policy selection."

This is not about smoothening explanations: it is about fundamental understanding. Active acquisition usually has a goal of e.g. learning a function as good as possible. Downstream, one can report, e.g. RMSE of how well the function is learned with 100 data points and acquisition method A or B. It is currently unclear to me whether Fig 3 left show this or how well the posterior / entropy is approximated.

Uncertainty decomposition: l170 says U_a is the irreducible uncertainty. l176 says overestimation by U_e. overestimation indicates something that we'd preferably eliminate. However, then we would be left wit the irreducible U_a. This is hard to follow.

107: X_J^* = ... j determines whether R_j=1 or not. Let's say it is, so first line. How would I choose i?

II) Why does Fig 3 left show all six methods, Fig 3 middle + right only four?

III) Fig 3 middle shows coverage values all below the desired coverage. Explanation why this is still useful seems missing.

IV) "We argue that robust AFA policy evaluation using PFNs necessitates decomposing" -> why does the experiment result section not show results on not decomposing, e.g. how bad it goes?

V) Related work is rather short. Active learning is a closely related field with quite some recent work, partially using pretraining: https://pdfs.semanticscholar.org/95bb/40c4e6f11970db176589fa5ca65057603641.pdf https://proceedings.mlr.press/v124/swersky20a.html https://arxiv.org/pdf/2501.15458

Questions for authors

see weaknesses

3 comments

We thank the reviewer for their careful reading and thoughtful questions. We respond to each point below; all changes are marked in blue in the revised PDF.


Q1. Missing references for claims in lines 58–60

Write up hard to follow: 58: "We prove that under standard information gain objectives" → where do I find corresponding theorem / lemma? 60: "We show that, relative to the aleatoric objective" → again: where do I find formal proof for that claim?

Response. These statements refer to Equation 2, which states that the finite-sample total entropy contains two terms, Utotal=Ua+UeU_{\text{total}} = U_a + U_e. We have:

  • Revised lines 58–60 to clarify that both sentences refer to this same decomposition.
  • Added explicit forward references to Section 3 (for the equation and intuition) and Appendix A.3 (for the conditions leading to suboptimal policy selection).
  • Replaced "We prove" with "We show" so the phrasing does not imply a separate formal theorem.

Q2. Figure 3 notation and terminology

Fig 3:

  • What is v^k\hat{v}_k and vkv_k? It is not mentioned in the caption and also hard to find in main text.
  • DGP: abbreviation need introduction.
  • With l316: what is the "accuracy of downstream acquisition policy selection"?

Response.

  • Notation. We replaced the VkV_k notation (the value estimate) in the figures with GG, as VkV_k was not defined in the main text. We additionally include a definition of GG in the figure caption.
  • DGP. The abbreviation is now explicitly introduced in Section 6.1.
  • Policy selection accuracy. This refers to whether our estimation method recovers the correct ranking of candidate policies, based on the synthetic ground truth or the real-world reference. It is described in the evaluation metrics paragraph of Section 6.

Q3. What Figure 3 (left) measures

Active acquisition usually has a goal of e.g. learning a function as good as possible. Downstream, one can report, e.g. RMSE of how well the function is learned with 100 data points and acquisition method A or B. It is currently unclear to me whether Fig 3 left show this or how well the posterior / entropy is approximated.

Response. The leftmost panels in Figures 3 and 5 show how accurately the true policy value (the target aleatoric entropy) is estimated, rather than downstream predictive error on YY.

  • Active learning acquires training points to learn a global function, and reports predictive RMSE on YY.
  • AFA evaluates test-time feature acquisition policies π\pi, whose ground-truth value is the true conditional entropy of YY under the acquired features.

Because the ground truth is unknown in real data, our goal is to estimate it accurately. The leftmost panels plot the value estimation MAE, ∣G^(π)−G(π)∣\lvert \hat{G}^{(\pi)} - G^{(\pi)} \rvert, against the known ground-truth entropy in the synthetic DGPs. We have explicitly clarified this point in the evaluation metrics paragraph of Section 6.


Q4. Uncertainty decomposition

l170 says UaU_a is the irreducible uncertainty. l176 says overestimation by UeU_e. Overestimation indicates something that we'd preferably eliminate. However, then we would be left with the irreducible UaU_a. This is hard to follow.

Response. UaU_a (aleatoric uncertainty) is our ground-truth target. To evaluate how informative a feature subset is in principle (i.e., under the true data-generating distribution, when n→∞n \rightarrow \infty), we want to measure its intrinsic conditional entropy UaU_a, which is irreducible with respect to sample size. A lower UaU_a indicates a better feature subset.

However, when evaluating a policy on a finite dataset, standard total-entropy estimators compute Utotal=Ua+UeU_{\text{total}} = U_a + U_e, as shown in Equation 2. Hence the uncertainty is overestimated by UeU_e. UeU_e does not reflect the true informativeness of the acquired features; it is the reducible finite-sample estimation bias that we preferably eliminate.


Q5. Index in the definition of XJ∗X_J^* (line 107)

Line 107: [...] How would I choose ii?

Response. Thank you for pointing this out. XiX_i is a typo: if Rj=1R_j = 1, then the true feature value of the jj-th index, Xj(1)X_j^{(1)}, is revealed. We have corrected this in the revision.


Q6. Why Figure 3 (middle, right) shows only four methods

Why does Fig 3 left show all six methods, Fig 3 middle + right only four?

Response. Figure 3 shows coverage and posterior width only for methods that estimate a posterior interval. The two total uncertainty estimators (plug-in and bootstrap variants) return only a point estimate, as described in the baseline paragraph of Section 6. We have explicitly clarified this in the figure caption.


Q7. Coverage below the nominal level

Fig 3 middle shows coverage values all below the desired coverage. Explanation why this is still useful seems missing.

Response. We agree that coverage falls below the nominal level (84-93% vs. 95% on synthetic DGPs with CLT). Despite this undercoverage, the posterior intervals provide an uncertainty estimate as opposed to a point estimate, and they are closer to nominal than the bootstrap alternative.

We additionally include new evaluations of posterior calibration (Appendix D4). These show that we can reliably quantify uncertainty when comparing two candidate policies, i.e., the probability that one candidate policy is better than another. Please also see our response to Q3 of reviewer JDtc.


Q8. Results without decomposition

"We argue that robust AFA policy evaluation using PFNs necessitates decomposing" → why does the experiment result section not show results on not decomposing, e.g. how bad it goes?

Response. The total uncertainty estimators, Total (plug-in) and Total (bootstrap), are the methods that do not decompose the uncertainty, and we show that they do not work well:

  • Bias. In the synthetic and real-world experiments, Figure 4 (right) and Figure 5 show that not decomposing the uncertainty leads to biased entropy estimators.
  • Ranking. In the synthetic experiment, the accuracy in ranking candidate policies is worse (Figure 4, left).

Q9. Related work

Related work is rather short. Active learning is a closely related field with quite some recent work, partially using pretraining.

Response. Thank you for providing related literature. We have added a section to the related work that briefly outlines the related fields of active learning and Bayesian optimization, and how they differ from the offline AFA evaluation setting.


Thank you again for your time and efforts in reviewing our paper, and we hope that you will consider raising your score if you find our response satisfactory.

Reviewer_xLZjOpenReview ↗

Answer to authors comments

I would like to thank the authors for time and effort put in the answers and the improvements. My score remains as is.

Reviewer_Awh3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies offline evaluation of fixed active feature acquisition policies when the logged data are observational and feature coverage is uneven. It arugues the usual reward and the entropy of a model's predictive distribution at the final acquired state, mixes two kinds of uncertainty. 1.irreducible uncertainty about the target given the state. 2.uncertainty about the model's parameters that would shrink with more data. Because the second term is non negative, use total predictive entropy systematically overestimates the value of acquiring poorly observed features, which the authors call epistemic bias. It propose to target the posterior expected aleatoric entropy instead, estimated with a Bayesian predictive inference construction on top of a tabular foundation model, using either a copula martingale posterior or a predictive central limit theorem.

The policy value itself uses a trajectory importance weighting imported from prior work on active feature acquisition. Experiments use three synthetic data generating families with analytic ground truth, several OpenML tabular datasets and the PhysioNet Sepsis task.

Critical strengths

The proposed target (posterior expected aleatoric entropy) is the right one. If a policy is to be evaluated by what it would achieve with full information, the reward should be the entropy under the true conditional distribution, not the entropy of a finite-sample posterior predictive. Eq2 says this precisely and the paper follows it through the estimation: posterior draws of the conditional measure, entropy computed within each draw, then averaged. The matched comparison against total-entropy baselines is the correct ablation. The synthetic DGPs let the authors check bias against a known value, not against another estimator.

Critical weaknesses

The abstract is not checked against the negative results. The abstract claims "consistently reduces value estimation bias" and "strong empirical coverage", but the paper reports no policy selection improvement on Sepsis and prints no coverage number. The abstract was not reconciled with the results in the body. The authors should review AI's output more carefully.

External validity of the estimator is not established. In the synthetic experiments, the PFN is trained from scratch on data from the same family that generates the evaluation tasks, so the posterior is well specified by construction. The authors say this is deliberate, to test a PFN near the idealized Bayesian limit, but it means the synthetic results cannot separate the method from a correct prior.

On real data the PFN is replaced by TabICL and there is no ground truth, while the reference value is a plug-in estimator at N = 10k. It is the same estimator family the paper is trying to debias. The authors acknowledge that reference values are imperfect, but they do not quantify the residual bias at N = 10k or use an independent model class as a check. Coverage is shown in figures. The paper reports no coverage value or Monte Carlo error, and the target on real data is the biased plug in.

Questions for authors
  1. What happens to the bias correction and to coverage if the synthetic DGPs are evaluated with TabICL, which was not trained on that family? The current design cannot distinguish the method from a correct prior.
  2. Can you report effective sample size and weight quantiles for the trajectory importance weights on Sepsis and maybe add a missing-not-at-random tipping-point analysis? The current propensity model is linear in demographics and vitals, which does not cover the way labs are ordered.
  3. Is any result cost-sensitive, or should the objective be stated without costs?
  4. Can you construct candidate policies with deliberately small true value gaps and report the fraction of datasets on which the corrected estimator selects a different policy from the plug-in? The experiments do not show a case where the correction changes the selected policy.
2 comments

We thank the reviewer for their careful reading and detailed feedback. We appreciate their recognition of our theoretical formulation, specifically targeting posterior expected aleatoric entropy, and the rigor of our matched ablations and synthetic DGP validations. We respond to each point below; all changes are marked in blue in the revised PDF.


W1. Abstract and coverage

The abstract is not checked against the negative results. The abstract claims "consistently reduces value estimation bias" and "strong empirical coverage", but the paper reports no policy selection improvement on Sepsis and prints no coverage number.

Response. We have included the numerical coverage results corresponding to the figures in Table D3 and Table D7, and updated the abstract to separate out the synthetic DGP and real-world results.


W2 / Q1. External validity

External validity of the estimator is not established. [...] What happens to the bias correction and to coverage if the synthetic DGPs are evaluated with TabICL, which was not trained on that family? The current design cannot distinguish the method from a correct prior.

Response. We thank the reviewer for this suggestion and ran the proposed experiment (new Appendix D.5).

Under TabICL, the CLT correction overcorrects, producing a positive bias (+0.09 vs. −0.02 for the plug-in). The Copula estimator stays close to the plug-in bias.

DGPVariantPair acc.Bias|Bias|Coverage (%)
LinearTotal (plug-in)0.61±0.020.61 \pm 0.02−0.016±0.003-0.016 \pm 0.0030.0490.049–
Bootstrap0.61±0.020.61 \pm 0.02−0.045±0.003-0.045 \pm 0.0030.0590.0597575
Copula0.61±0.020.61 \pm 0.02−0.013±0.003-0.013 \pm 0.0030.0480.0486060
CLT0.59±0.02\mathbf{0.59 \pm 0.02}+0.091±0.005\mathbf{+0.091 \pm 0.005}0.0950.0956565
GMMTotal (plug-in)0.59±0.010.59 \pm 0.01−0.022±0.003-0.022 \pm 0.0030.0540.054–
Bootstrap0.59±0.010.59 \pm 0.01−0.053±0.003-0.053 \pm 0.0030.0650.0656767
Copula0.59±0.010.59 \pm 0.01−0.019±0.003-0.019 \pm 0.0030.0520.0525656
CLT0.58±0.01\mathbf{0.58 \pm 0.01}+0.090±0.005\mathbf{+0.090 \pm 0.005}0.0940.0946565

Regret is 0.006–0.007 for all variants. The full table, including the marginal and total-bootstrap variants, is in Appendix D.5.

Interpretation. The CLT correction equals the estimated epistemic term, so it is valid only when the model recovers the correct posterior spread. That holds for the specialized PFNs. Under a misspecified prior (TabICL), the posterior draws of Y∣XY \mid X are overdispersed, so the correction is too large. We now state external validity as a limitation in Section 7, point (iii), with a recommendation to use diagnostics to check for prior misspecification.

We also note that our real-world TabICL experiments also work as external-validity checks, because TabICL's prior is fully synthetic and does not include tasks like sepsis prediction.


W3. Reliability of the reference value

On real data the PFN is replaced by TabICL and there is no ground truth, while the reference value is a plug-in estimator at N = 10k. They do not quantify the residual bias at N = 10k or use an independent model class as a check. The paper reports no coverage value or Monte Carlo error, and the target on real data is the biased plug-in.

Response. We now report the Monte Carlo error of the reference value (Table D15) and the convergence of the value estimates and policy rankings as the sample size approaches n=10,000n = 10{,}000 (Figure D23). Both are stable as nn approaches 10,00010{,}000.

MetricMean ± SEM
Bootstrap SE0.0011±<0.00010.0011 \pm <0.0001
95% interval width0.0042±0.00010.0042 \pm 0.0001
Pairwise accuracy of the draws0.988±0.0030.988 \pm 0.003
P(draw picks the gold-best)0.971±0.0140.971 \pm 0.014

Monte Carlo error of the reference value over 100 bootstrap draws.

Q2. Importance weight diagnostics and MNAR sensitivity

Can you report effective sample size and weight quantiles for the trajectory importance weights on Sepsis and maybe add a missing-not-at-random tipping-point analysis?

Response. We report the ESS and weight quantiles below (also Appendix Tables D8–D9).

Acquired panelnSupportedESS / nw50w90w99w
Blood gas12074.5%96.7%1.812.433.2031.88
CBC12678.5%97.3%1.682.222.8631.88
Coags11165.9%96.3%1.912.593.5138.93
Lactate12334.4%88.4%3.014.636.7142.54
Liver12926.8%90.9%3.805.687.7234.67
Metabolic11479.8%97.6%1.582.022.5923.18

One-step policies, PhysioNet 2019 (sepsis). n = number of policies; ESS is a share of the supported eval rows; wq = q% weight quantile (mean over policies); w = max over policies.

On MNAR: the experiment tests whether the reference estimand can be reliably recovered under fixed assumptions (MAR and linearity). Since our focus is uncertainty estimation in a controlled setting, modeling MNAR mechanisms is outside the scope of this work. We agree that a sensitivity analysis that perturbs the propensity estimates is a valuable extension for real-world deployment, and we note it as future work in Discussion point (ii).


Q3. Costs

Is any result cost-sensitive, or should the objective be stated without costs?

Response. No reported result includes costs. Acquisition costs are a deterministic function of each candidate policy, so adding them shifts each policy's estimate by a known constant. They do not change the estimation bias or uncertainty, which is what our comparison between estimators measures. We will state the objective without costs in the revision.


Q4. Policies with small value gaps

Can you construct candidate policies with deliberately small true value gaps and report the fraction of datasets on which the corrected estimator selects a different policy from the plug-in? The experiments do not show a case where the correction changes the selected policy.

Response. We thank the reviewer for this suggestion. Appendix A.3 theoretically shows that the correction changes regret only when the true value gap is smaller than the epistemic gap at the final states the policy reaches. So the candidate policies must be close in true value for the correction to matter.

We built this setting for the sepsis ICU experiment:

  1. Generated 20 stochastic multi-step policies and computed their reference values.
  2. Selected the 10 with the narrowest value gaps, subject to a minimum gap that rules out ties and near-ties.

Initial results (10 bootstrap replications) show improved policy ranking:

VariantPairwise acc. (%)Δ vs. plug-inp-value
Total (plug-in)75.64ref.–
Total (bootstrap)74.94-0.710.766
Bootstrap74.24-1.320.859
Marginal (CLT)76.94+1.300.031
Copula78.74+3.100.004
CLT76.45+0.910.422

p-values from a one-sided paired test against the plug-in.

Changes. We will update the main text with the full set of replications to replace the original ICU multistep policy evaluation experiment, along with all of the corresponding details (coverage, propensity quantiles and ESS, etc.).

Thank you again for your time and efforts in reviewing our paper, and we hope that you will consider raising your score if you find our response satisfactory.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.