- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper aims to solve the problem of estimating the distribution of target features from an arbitrary set of observed features in a zero-shot or few-shot setting for tabular data with unseen schemas. Previous research has proposed foundation models based on fine-tuning using labeled support data or in-context learning. However, these models struggled to operate in situations where no support data was available. This paper proposes a conditional distribution estimation framework called ASPIRE. ASPIRE consists of (i) an atom representation that semantically embeds features, (ii) an intra-instance set contextualizer that models feature relationships within the same sample, and (iii) role-aware late fusion that integrates information from different sources. For classification and regression, ASPIRE outputs categorical distributions via inner products with language embeddings and continuous distributions via a Gaussian mixture model, respectively. Furthermore, predictions from external models can be utilized at both the input and output levels. ASPIRE is trained by minimizing the negative log-likelihood via episodic training. In the experiments, prediction accuracy was evaluated using OpenTabs as a baseline, and the paper reports performance that surpasses existing baselines as well as the impact of each component.
Critical strengths
- The paper proposes a model capable of zero-shot prediction of the feature distribution in tabular data with unseen schemas.
- The paper designs an architecture that utilizes text features to create semantic embeddings of features and learns dependencies among features across heterogeneous schemas.
- The paper experimentally demonstrates that ASPIRE can achieve performance that surpasses that of previous foundation models.
Critical weaknesses
- Arbitrary conditioning has not been directly evaluated. While the central research question of the paper is the estimation of conditional distributions for arbitrary target variables and arbitrary sets of observed variables, the experimental evaluation was conducted only on OpenTabs' fixed table prediction problem. For example, it would be desirable to evaluate the proposed method when the target variable is changed, even for the same dataset, and to assess its robustness with respect to the number and combinations of observed variables.
- It is unclear whether the zero-shot setting is truly "zero-shot." ASPIRE uses category candidate values and the domains of features as input metadata. If these are based on the actual dataset, data leakage occurs, and this setting cannot be called zero-shot.
- There is a lack of evaluation of conditional probability distributions. The evaluation in the paper primarily measures the performance of downstream tasks and does not assess the validity of the estimated distributions. At the very least, evaluation using negative log-likelihood is necessary.
- Late fusion and early fusion have not been compared. The paper argues that modeling that preserves the structure of each information source through role-aware late fusion is effective. This claim could be clearly evaluated by actually comparing it with early fusion, but as of now, the paper does not provide such a comparative experiment.
Questions for authors
See the critical weaknesses and address the concerns.