acceptodds
Under review as a conference paper at ICLR 2027

Learning Source Acquisition Policies by Offline Planning

Abstract

Prediction under an acquisition budget requires choosing which information to observe. A query that reveals a feature group spends resources that could support later queries, and its value can depend on the decisions it enables. O-MPAC transfers finite-horizon acquisition targets into a shared action-conditioned scorer. An offline teacher evaluates risk and source cost on complete training records; the learner uses partial observations and source metadata to predict acquisition values and preferences. Online execution re-scores sources after each query and enforces the available budget. We analyze how complete-record information, tied action targets and the remaining planning horizon affect this transfer. Uniform supervision over tied minima preserves the target distribution under source relabeling, while matching the final acquisition step separates informative and redundant queries through terminal risk. In a five-seed context-dependent routing experiment, this supervision achieves 0.965 accuracy under both the original and context-last source orders. On six real tasks, validation selects H1 noCE in all thirty splits. O-MPAC has the highest mean budget-integrated accuracy on five tasks against source-adapted GDFS, DIME, AACO+NN and a static policy. A separate native-pipeline comparison against GDFS and DIME gives the same five-task pattern.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
active feature acquisition, sequential decision making, offline planning, source acquisition, supervised policy learning

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_9v2zOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper uses offline lookahead on full training data to teach a network to instantly pick the best affordable feature groups at test time. And it cuts inference down to sub-milliseconds, fixes tied-choice ordering bias, and outperforms top baselines on 5 out of 6 real-world benchmarks.

Critical strengths

The paper's experiments are clean and honest. There is no data leakage, the stats are done properly, and the authors also explain why their 2-step planning didn't help on real datasets instead of trying to hide it.

Critical weaknesses

My biggest concern is that the whole story about multi-step offline planning doesn't hold up on real data. Across all six real datasets and every single one of the 30 splits, validation always picks the 1-step model and completely avoids H2. Even worse, H2 is worse than H1 on several tasks. This means the real-world performance has nothing to do with the multi-step planning the authors are selling.

Questions for authors

Looking at Table 13, across all 30 validation splits on the six real benchmarks, the validation set never once selects H2, and H2 is often substantially worse than H1. Can the authors provide any empirical evidence or intuition showing that two-step planning actually helps on real-world tabular data?

Reviewer_Q5yqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
  • The paper studies the source acquisition policies over costly information sources, where a model is required to select a group of features to reveal before making a prediction
  • The authors introduce O-MPAC which uses observed records to generate planning targets and then trains a model to score sources
  • Experiments cover tabular datasets and controlled synthetic datasets, and the authors report improvements over source acquisition baselines.
  • The paper also includes detailed analysis about teacher information, tie-breaking, and planning horizon.
Critical strengths
  • The evaluation includes baselines and ablations that help distinguish improvements in the acquisition system.
  • The authors show that O-MPAC achieves the highest average accuracy-cost on five of six tasks.
  • The analysis shows how teacher cost impacts the source acquisition policy.
Critical weaknesses
  • The one-step variant without action-imitation loss outperforms others, and as a result, the real dataset results do not support a consistent benefit of the longer-horizon planning
  • The experiments use constructed feature groups and acquisition costs. It would strengthen the paper to show whether gains extend to naturally-defined sources with realistic acquisition costs.
Questions for authors
  • Clarification - What distinguishes this approach from existing asymmetric imitation approaches?
  • Clarification - Is there a setting where H2 or action CE outperforms (beyond the synthetic setting)?
Reviewer_xudjOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes O-MPAC for feature acquisition under a budget constraint, where each acquisition reveals a predefined group of features (a source). The model combines partially observed states with source metadata using a shared action-conditioned learner to predict the loss reduction. Training targets are computed on fully observed training records with a Bellman-style recursion using a fixed masked centroid classifier, and then learned using several losses including classification, risk, and ranking losses. Evaluations on real-data tasks report improvements over GDFS, DIME, and AACO+NN using a native and masked centroid classifier.

Critical strengths

None

Critical weaknesses

Poorly motivated design. The paper describes a seemingly complex feature acquisition algorithm with little motivation for why such an approach was chosen. Several components of the design are stated without justification or testing. For example, the paper mentions a fixed teacher predictor which uses a masked centroid classifier without explaining why this is a suitable choice in the first place. Equation 7 combines six different losses, and it is unclear why these losses were chosen. Empirically, there is no ablation against them, other than the cross-entropy term. Overall, it is unclear why the model is designed in this particular manner and what components lead to the reported improvements.

Limited novelty. The paper does not clearly state its contributions beyond a summary of the O-MPAC design. The related work section positions O-MPAC as a method that operates on groups of features with finite-horizon risk-cost targets. However, the AFA literature can already handle feature groups by treating each group as a coarser feature. Moreover, longer finite-horizon planning (H=2H = 2) is never selected by validation on any real-task split, which suggests it does not help the proposed method.

Poor presentation. The paper is very difficult to follow due to (i) dense technical jargon that is either defined loosely or not at all and (ii) limited discussion of the particular design choices and results. For example, the paper never explicitly states that O-MPAC is the proposed method, and key symbols appear in Equations 3-8 without proper definitions. The CUBE-NM experiment is referred to several times before it is described. Proposition 1 also lacks proper framing and discussion, and its conclusions rely on specific values of κ\kappa and ρ\rho with no explanation of why these values were chosen.

Overall, the paper requires a substantial rewrite to clearly convey its contributions, design choices, technical details and motivating intuition. As it stands, much of the terminology is left for the reader to infer, and the empirical results are difficult to interpret without a clear explanation of why the proposed method should outperform existing approaches.

Questions for authors

See weaknesses. In addition,

  1. The title mentions learning by “offline planning,” however validation selects H=1H = 1 in every real-task split. In what sense does the method perform planning?
  2. I am skeptical of the masked nearest-centroid classifier as both the teacher predictor and the common predictor. Why was this model chosen, and can the authors test other models such as a classifier trained on randomly masked inputs? One would expect the centroid classifier to perform quite poorly in practice.
  3. What is the class imbalance in each dataset, and can the authors also provide balanced accuracy metrics?
Reviewer_utoYOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

I am not reviewing this paper. I asked the AC/SAC/PCs to remove me from this assignment because I could not review it, and the AC/SAC confirmed that a replacement reviewer was invited. I could not be removed from the system in time, so I am submitting this placeholder only to close the assignment. I have not read the paper and make no assessment of it. Please disregard this review, including the rating, in all evaluations. My apologies to the authors.

Critical strengths

Not assessed. Placeholder entry; I have not reviewed this paper. Please disregard.

Critical weaknesses

Not assessed. Placeholder entry; I have not reviewed this paper. Please disregard.

Questions for authors

None. Placeholder entry; I have no questions for the authors.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.