acceptodds
Under review as a conference paper at ICLR 2027

Beyond Direct LLM Prediction: Open-World Reuse of Heterogeneous Tabular Learnwares

Abstract

Large language models (LLMs) can draw on broad knowledge for prediction. Specialized tabular models offer complementary value even as LLM capabilities improve: they encode statistical relationships often learned from private data, while their explicitly defined inputs and targets support statistical auditing and validation. However, heterogeneous feature requirements and incomplete user inputs complicate their reuse. The learnware paradigm makes independently trained models identifiable and reusable by equipping each with a statistical specification of its training distribution, without sharing the original data. Building on this foundation, for the first time, we enable heterogeneous tabular learnwares to address open-world prediction queries whose observed features need not match any model's input requirements. Key to this capability is our finding that local statistical relationships encoded in different specifications can be jointly exploited to infer missing inputs required by other models. Our framework aligns specifications along shared features into a query-dependent context, from which a pretrained tabular prior infers missing inputs for model retrieval and prediction. Model-specific completion uncertainty guides filtering before aggregation, while statistical support under the same context determines whether to return the ensemble prediction or abstain. Across 41 tasks, our framework outperforms direct and refined LLM prediction and LLM-assisted model reuse. Controlled comparisons further show that statistical relationships encoded in other models' specifications can improve missing-feature completion and downstream prediction.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
learnware, model reuse, tabular prediction, heterogeneous feature spaces

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_mLbmOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a framework for reusing heterogeneous tabular learnwares when user queries lack features required by available models. It aligns statistical specifications across models and uses TabPFN for missing-feature completion, followed by model retrieval, uncertainty-aware aggregation, and support-based abstention. Experiments on 41 tasks show improvements over LLM-based prediction and model-reuse baselines, with additional analyses of cross-model statistical relationships.

Critical strengths
  1. The paper addresses a meaningful problem of reusing heterogeneous tabular models without access to their original training data. Exploiting statistical relationships across model specifications to infer missing inputs is an interesting contribution with practical relevance.
  2. The core idea is supported by careful controlled experiments. Adding other models' specifications improves four of five task-group metrics with the predictor fixed. The relationship-shuffling and bridge-chain experiments further demonstrate the value of cross-model statistical information for feature completion.
Critical weaknesses
  1. Limited validation of the open-world setting. The model libraries are constructed from different row and feature subsets of the same datasets, using LightGBM throughout. Missing features are simulated mainly through random masking, while feature and target alignment are assumed. This setting does not fully capture independently developed models with different data distributions and realistic missingness patterns.
  2. Insufficient baselines and evaluation of key components. The main classification comparisons lack majority-class references and imbalance-sensitive metrics, making some results difficult to interpret, particularly on Amazon Access and DECOMP. Stronger alternatives appear mainly in ablations: mean imputation matches TabPFN completion on Financial tasks, while direct TabPFN-ICL slightly outperforms Ours. Moreover, the uncertainty-filtering rule is not fully specified, and Table 4 reports the best fixed retention size per task rather than directly validating query-dependent selection.
Questions for authors
  1. How does the framework perform with models trained on independently collected datasets or under non-random missingness?
  2. Can the authors clarify the uncertainty-filtering rule and compare it with fixed-size aggregation under identical settings?
  3. How is the abstention threshold selected without labeled user examples, and why does abstention provide limited benefits on some tasks?
  4. Can the authors report majority-class baselines and balanced accuracy for classification, and provide more direct comparisons with TabPFN-ICL and standard imputation methods?
5 comments
AuthorsOpenReview ↗

Response to Reviewer mLbm (Part 1/3)

Thank you for recognizing the practical value of our work and for finding the paper clear and informative. We address your concerns below.

1. "Limited validation of the open-world setting. The model libraries are constructed from different row and feature subsets of the same datasets, …"

"How does the framework perform with models trained on independently collected datasets or under non-random missingness?"

Independently collected sources. Beyond the original controlled benchmark (Section 5.1), we use three independently collected sales-forecasting datasets, evaluating prediction on Predict Future Sales (PFS) [1] with models trained on M5 [2] and Corporación Favorita [3]. PFS contributes neither training data nor specifications to the model library. The library contains LightGBM, random-forest, and Ridge predictors from each training source. The table compares Ours with the strongest classical-completion baseline using the same six predictors; RMSE is computed on the common log-sales target. We have added this evaluation to Section 5.3 (revised Table 3), with the protocol and store-level results in Appendix E.2.3 (Table 21).

Observed query inputsBest classical baselineBaseline RMSEOurs RMSE
Sales historyMean imputation0.9780.906
Sales history and relative pricesRandom forest iterative imputation0.9810.923

Ours reduces RMSE by 7.3% and 5.9%, respectively, with improvements at every target store. These results show that the framework can reuse models trained on independent sources despite differing input requirements.

Non-random missingness. Beyond the original random masking (Appendix D.3), we evaluate grouped MAR: the probability that a feature group is missing depends on an input that remains observed. We test weaker and stronger dependence on this observed input, keeping expected overall missingness at 30%. Specifications, queries, masks, and predictors are fixed across methods. The table reports the relative RMSE reduction from mean imputation to TabPFN, computed from means over three seeds for each task and averaged equally across four regression datasets.

Dependence on the observed inputAverage RMSE reduction (%)Tasks improved
Weaker14.74/4
Stronger14.04/4

Individual task-level RMSE reductions range from 3.8% to 25.9%. The completion benefits persist on all four datasets when missingness depends on observed features. We have included these experiments in Appendix F.6 (Fig. 7) of the revision.

2. "The model libraries … using LightGBM throughout."

The framework accesses tabular models through their prediction interfaces and statistical specifications, so it does not require a particular predictor architecture (Sections 3–4). To assess this generality, we additionally tested random-forest and Logistic/Ridge predictors across all 41 tasks, using the same feature-block completion setting and holding each predictor fixed across completion methods. Relative to mean imputation, TabPFN raises average classification accuracy by 2.32 and 3.65 percentage points for random forests and logistic regression, respectively; balanced accuracy and regression rank also improve for both families. These results support the framework's applicability beyond LightGBM, alongside the mixed-family library evaluated above. We have added this comparison to Appendix F.4 (Table 26) of the revision.

"feature and target alignment are assumed."

As formulated in Section 3, feature and target matching precede our query-time procedure. Existing methods support semantic feature matching [4] and matching or adapting models across different label spaces [5, 6]. They can provide upstream alignment when the required task information is available. Our focus is the subsequent challenge: completing missing inputs for individual queries through cross-model statistical relationships, without developer training data or additional user examples. We have clarified this interface in Section 3 and discussed the related work in Appendix A.2 of the revision.

AuthorsOpenReview ↗

Response to Reviewer mLbm (Part 2/3)

3. "The main classification comparisons lack majority-class references and imbalance-sensitive metrics, making some results difficult to interpret, particularly on Amazon Access and DECOMP."

"Can the authors report majority-class baselines and balanced accuracy for classification, …"

We agree that accuracy alone is insufficient on imbalanced tasks. Below, the majority baseline always predicts the training-set majority class; it and Ours are evaluated on the same 200 queries. Balanced accuracy (BA) gives equal weight to each class's recall.

TaskMajority accuracy (%)Ours accuracy (%)Majority BA (%)Ours BA (%)
Amazon Access94.0094.1750.0053.99
DECOMP98.5098.5050.0050.00
IHM88.5082.6750.0070.03

DECOMP's high accuracy reflects majority-class prediction, while the gain on Amazon Access is modest. IHM illustrates why BA matters: Ours has lower accuracy than the majority baseline but substantially better recall averaged across classes. Across all 26 classification tasks, Ours exceeds majority-class BA on 25 tasks. We have added class proportions and majority-class baselines, together with accuracy, BA, and score-based AUROC/AUPRC comparisons, to Appendix E.2.1 (Tables 13–15) of the revision.

4. "Stronger alternatives appear mainly in ablations: mean imputation matches TabPFN completion on Financial tasks, while direct TabPFN-ICL slightly outperforms Ours."

"…and provide more direct comparisons with TabPFN-ICL and standard imputation methods?"

Standard imputation. Revised Table 4 (submitted Table 3) already reports the Financial tie, alongside gains from TabPFN completion in the other four group metrics. We have extended this comparison with KNN and Bayesian-ridge iterative imputation, keeping retrieval, aggregation, queries, and masks fixed. General classification accuracy is 80.948% with TabPFN, versus 80.667% with KNN and 79.062% with Bayesian ridge. TabPFN has higher mean accuracy in all three classification groups and lower clinical regression RMSE than these two methods. We have added these comparisons to Section 5.4 (revised Table 4) and Appendix F.2 (Tables 23–24) of the revision.

Direct TabPFN-ICL. Revised Table 25 (submitted Table 13) compares direct target prediction from specification pseudo labels with our completion followed by model reuse; Ours is better on four of five group metrics. The Financial accuracy difference is 0.31 percentage points in favor of TabPFN-ICL. Our added classwise analysis shows a different balance: Ours has higher Financial BA (70.65% versus 62.58%) and minority-class recall (52.90% versus 30.97%), while TabPFN-ICL has higher majority-class recall. Across the benchmark, Ours has better accuracy/RMSE on 28 of 41 tasks, with two ties, and higher BA on 24 of 26 classification tasks. This supports retaining trained predictors after completion, while explaining the Financial accuracy exception. We have expanded this direct comparison in Appendix F.3 (revised Table 25 and Fig. 5) of the revision.

AuthorsOpenReview ↗

Response to Reviewer mLbm (Part 3/3)

5. "Moreover, the uncertainty-filtering rule is not fully specified, and Table 4 reports the best fixed retention size per task rather than directly validating query-dependent selection."

"Can the authors clarify the uncertainty-filtering rule and compare it with fixed-size aggregation under identical settings?"

Thank you for pointing this out. Eq. (6) defines Ui(x)U_i(x) as the mean normalized entropy of the missing inputs required by model ii. Query-wise selection retains models satisfying Ui(x)≤Umin⁡(x)+δU_i(x)\le U_{\min}(x)+\delta, where Umin⁡(x)U_{\min}(x) is the lowest retrieved-model uncertainty. For task-wise selection, let U(k)(x)U_{(k)}(x) be the kkth-lowest retrieved uncertainty and compute Δ‾k=mean⁡x[U(k)(x)−Umin⁡(x)]\overline{\Delta}_k=\operatorname{mean}_{x}[U_{(k)}(x)-U_{\min}(x)] over a batch of unlabeled user queries from the same task. We choose the largest shared KK satisfying Δ‾K≤δ\overline{\Delta}_K\le\delta; each query then retains its own KK least-uncertain models.

The main benchmark uses this task-wise rule, with KK selected without query labels; both selection rules use δ=0.1\delta=0.1. For each fixed KK, we retain the KK least-uncertain models for each query. All rules use identical queries, masks, completed inputs, and eight retrieved predictors. The table summarizes the ten illustrative tasks in revised Table 5. We report equal-weight means separately for the five classification and five regression tasks, with regression summarized by RMSE.

AggregationAccuracy, 5 classification tasks (%)RMSE, 5 regression tasks
All 8 models72.270.4957
Fixed K=1K=170.500.4997
Fixed K=3K=372.330.4938
Fixed K=6K=672.570.4946
Task-wise selection73.170.4920
Query-wise selection72.270.4935

On these illustrative tasks, task-wise selection gives the best mean performance among the compared rules. Query-wise selection has lower mean RMSE than the listed fixed-size controls (0.4935 versus 0.4938–0.4997), while its mean classification accuracy remains below fixed K=3K=3 and K=6K=6. These results support the task-level rule used in the main benchmark and directly characterize the query-wise alternative. We have clarified both rules in Section 4.3 and expanded the comparison in Section 5.4 (Table 5) and Appendix E.3.1 of the revision.

6. "How is the abstention threshold selected without labeled user examples, and why does abstention provide limited benefits on some tasks?"

For a given feature set and preprocessing, the threshold τ\tau specifies the low-support operating point. Without user labels, we inspect the candidate-context support-score distribution, place the cutoff at the lower-tail quantile that leaves the desired retained coverage (e.g., 90%) of scores above it, and fix that threshold before processing user queries. This provides a simple, label-free heuristic for selecting τ\tau.

Support-based abstention targets errors associated with unfamiliar feature combinations. Errors that occur in familiar regions can retain high support and therefore pass the threshold, which makes the benefit task-dependent. This distinction may explain why abstention benefits some tasks more than others, consistent with positive mean AURC gains on 25 of 41 tasks (Section 5.4; Appendix E.3.2). We have clarified the label-free threshold choice in Section 4.4 of the revision.

We hope these clarifications and revisions address your concerns.

References

[1] Kaggle. Predict Future Sales, 2018. URL https://www.kaggle.com/competitions/competitive-data-science-predict-future-sales.

[2] Kaggle. M5 Forecasting - Accuracy, 2020. URL https://www.kaggle.com/competitions/m5-forecasting-accuracy.

[3] Kaggle. Corporación Favorita Grocery Sales Forecasting, 2017. URL https://www.kaggle.com/competitions/favorita-grocery-sales-forecasting.

[4] Zhi-Hao Tan, Jian-Dong Liu, Xiao-Dong Bi, Peng Tan, Qin-Cheng Zheng, Hai-Tian Liu, Yi Xie, Xiao-Chuan Zou, Yang Yu, and Zhi-Hua Zhou. Beimingwu: A learnware dock system. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 5773–5782, 2024.

[5] Jian-Dong Liu, Zhi-Hao Tan, and Zhi-Hua Zhou. Identifying and reusing learnwares across different label spaces. In Proceedings of the 34th International Joint Conference on Artificial Intelligence, pages 5734–5742, 2025.

[6] Lan-Zhe Guo, Zhi Zhou, Yu-Feng Li, and Zhi-Hua Zhou. Identifying useful learnwares for heterogeneous label spaces. In Proceedings of the 40th International Conference on Machine Learning, pages 12122–12131, 2023.

Reviewer_hqEKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies the reuse of pretrained tabular predictors that share a prediction target but require different input features. Each predictor is accompanied by a compact statistical specification summarizing its training-input distribution, while the original training data are unavailable.

Critical strengths
  1. The research direction is interesting.
  2. It can have real world application.
Critical weaknesses

From writing perspective (serious):

  1. The abstract is hard to understand.

  2. The setting is very confusing. why models with different input requirements are provided? And since different models operate on different set of datasets, why should we combine them, and how to select these models? And why different models will have a subset of a total feature list?

  3. Without clear prior knowledge of the background, the latter of introduction is hard to understand, expecially the fifth paragraph of introduction.

  4. And Line 99 to Line 102, can the author further explain it?

From reproduction perspective:

  1. The manuscript provides no link to the proposed method’s implementation or statement of code availability.

From experiment perspective:

  1. The baseline selection is insufficient to support the positioning of this paper. Specialized tabular LLM methods discussed in the paper, such as TabLLM, TabuLa-8B, and JoLT, are ignored, and the reasons for ignoring these methods are not explained.
Questions for authors

Please see weaknesses above.

6 comments
AuthorsOpenReview ↗

Response to Reviewer hqEK (Part 1/3)

Thank you for recognizing the practical potential of our work. We address each concern below, using clinical examples for clarification.

1. "The setting is very confusing."

Tabular models are widely used for specific prediction tasks. They learn relationships between input features and outcomes, often from private data that may be unavailable to an LLM. We study how these specialized prediction capabilities can complement the general knowledge of LLMs when answering prediction queries. These models can serve as prediction tools for LLMs, but each typically requires a particular set of input features. A user's query may not provide all of them.

For example, hospitals may independently train models to predict a patient's risk of death during a hospital stay. Each model learns from its hospital's patient records and may require different types of patient information, such as age, blood pressure, or blood test results. A doctor asking an LLM about a new patient would like to benefit from these existing capabilities. However, the available patient information may not satisfy any model's input requirements. We aim to make these capabilities usable for such queries without requiring the doctor to collect a new training dataset.

"why models with different input requirements are provided?"

Hospitals collect different patient information and make different choices about which information to use. For example, one model may use age, blood pressure, and a blood test result; another may use age, blood pressure, heart rate, and oxygen levels. A tabular model's inputs reflect these choices and need not match what a future user can provide. Such differences are a common practical obstacle to using otherwise useful models for new prediction queries.

"And since different models operate on different set of datasets, why should we combine them"

Each hospital's tabular model has learned relationships between patient characteristics and mortality from its own records. These specialized prediction capabilities can complement an LLM's general medical knowledge. Reusing the models lets users benefit from knowledge learned from different hospitals' data without collecting a new training dataset or building another model. However, a potentially useful model may require a blood test result that the query does not contain. We study how to make these existing capabilities usable with the information supplied in the query.

"And why different models will have a subset of a total feature list?"

Hospitals often record the same basic information, such as age and blood pressure. They may use different additional tests or clinical records when building models. A patient's available age, blood pressure, and heart rate may therefore meet only part of each model's input requirements. Such partial matches arise naturally when models are developed independently and users provide their own information. They can prevent potentially useful models from being used directly.

"and how to select these models?"

Our framework uses statistical specifications, compact summaries of the models' training distributions. They retain relationships among input features, allowing us to use this information without accessing the original training records. For a mortality-risk query missing a required blood test result, these relationships help us estimate that result from the available patient information. We then compare the completed query with the specifications to select models with closer matches. We favor models whose missing inputs can be estimated with less uncertainty and average their predictions. The framework can decline to predict when the specifications provide insufficient statistical support for the query.

AuthorsOpenReview ↗

Response to Reviewer hqEK (Part 2/3)

2. "The abstract is hard to understand."

"Without clear prior knowledge of the background, the latter of introduction is hard to understand, expecially the fifth paragraph of introduction."

Our aim is to complement LLMs' general knowledge with the specialized prediction capabilities of existing tabular models. For example, hospital models learn relationships between patient characteristics and mortality from private patient records. These relationships can provide prediction knowledge beyond an LLM's general medical knowledge. However, models trained at different hospitals may require different types of patient information. A patient's query may not supply all the inputs any model needs. We study how to make these existing capabilities usable for individual queries without requiring users to collect a new training dataset.

The fifth paragraph introduces learnware: a trained model provided together with a statistical specification. This specification is a compact summary of the model's training distribution. For a hospital model, it retains information about relationships among age, blood pressure, and test results. These relationships help us use the model when a new patient's query is incomplete and the original records cannot be shared. We have revised the Abstract and Introduction to clarify the setting and the role of specifications.

3. "And Line 99 to Line 102, can the author further explain it?"

We further clarify how one model's specification can help estimate another model's missing inputs. Models A and C both predict the risk of developing type 2 diabetes within five years. A requires age, a blood-test result (HbA1c), and blood pressure. C requires body mass index (BMI), HbA1c, and blood pressure. The query provides age, BMI, fasting glucose, and HbA1c, but not blood pressure.

A specification summarizes a model's training distribution. It retains information about relationships among its input features. C's specification relates blood pressure to BMI and HbA1c. These relationships can help estimate A's missing blood pressure from the patient's BMI and HbA1c, even though A does not use BMI. A can then use the estimated blood pressure together with age and HbA1c to predict diabetes risk. C's specification therefore contributes even when C's predictor is not used. We have clarified this explanation in the revised Introduction.

4. "The manuscript provides no link to the proposed method's implementation or statement of code availability."

Thank you for highlighting reproducibility. We have revised the Reproducibility Statement to point readers to detailed experimental information. Appendix C explains how the statistical specifications are constructed and how models are selected and used. Appendices D.1–D.3 describe data splits, model training, and missing query inputs. Appendices D.4–D.6 provide model configurations, evaluation procedures, and LLM input/output templates, including details of the added tabular LLM baselines. We will make the implementation publicly available with the final version of the paper.

AuthorsOpenReview ↗

Response to Reviewer hqEK (Part 3/3)

5. "The baseline selection is insufficient to support the positioning of this paper. Specialized tabular LLM methods discussed in the paper, such as TabLLM, TabuLa-8B, and JoLT, are ignored, and the reasons for ignoring these methods are not explained."

Thank you for this suggestion. Our initial comparison examined direct LLM prediction, self-refinement, and LLM completion followed by model reuse. Specialized tabular LLMs provide an important further test of whether existing trained models add value. We have now added TabLLM, TabuLa-8B, and JoLT on all 26 classification tasks, with JoLT also covering all 15 regression tasks. They receive the observed feature–value pairs, a target description, and class choices where applicable. No statistical specifications are provided, and the methods are not fine-tuned on the target tasks. We average results over three seeds for each task, then equally across tasks in each group. NRMSE divides each task's RMSE by the standard deviation of its target values.

Task groupMetricTabLLMTabuLa-8BJoLTOurs
General Cls. (16 datasets)Accuracy (%)62.0768.8956.1681.57
General Reg. (14 datasets)NRMSE——2.7240.616
Financial Cls. (8 datasets)Accuracy (%)16.9481.9681.8183.15
MIMIC-III Cls. (2 tasks)Accuracy (%)13.5063.3364.9290.58
MIMIC-III Reg. (1 task)RMSE——1.0130.849

Ours exceeds the strongest specialized baseline in each classification group by 1.19–25.67 percentage points. It reduces General regression NRMSE by 77.4% relative to JoLT and has lower RMSE on all 15 regression tasks. Our framework outperforms the specialized tabular LLMs evaluated in each task group. These results support the value of reusing existing tabular models for individual prediction queries. We have included these baselines in Table 2, with settings and complete results in Appendices D.4 and E.2.1 (Table 13) of the revision.

We hope these clarifications and revisions address your concerns.

Reviewer_TGqFOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies open-world prediction with heterogeneous tabular learnwares, where a query’s observed features may not satisfy any model’s input requirements and neither labeled user examples nor developer training data are available. The authors align RKME specifications from models with overlapping features into a joint specification context, which TabPFN uses to infer missing inputs. The completed query supports model retrieval, uncertainty-aware aggregation, and specification-based abstention. Experiments across 41 tasks show improvements over direct and refined LLM prediction and LLM-assisted model reuse.

Critical strengths

(1) The paper introduces an interesting use of statistical specifications: a specification can help infer inputs required by another model, even when its associated predictor is not used. This extends heterogeneous learnware reuse to queries without predefined feature blocks or additional user-side examples. (2) The fixed-predictor comparison in Table 1 isolates the contribution of the completion context. The marginal-preserving shuffles and synthetic bridge-chain experiment further support the value of cross-feature relationships, beyond simply providing additional support points.

Critical weaknesses

(1) The incremental benefit of cross-specification information varies across tasks. In Table 1, adding auxiliary specifications improves General classification accuracy from 78.948% to 79.344%, while the improvement is more pronounced in General regression. Furthermore, all developer models are constructed from subsets of the same benchmark training partition, and query missingness follows MCAR or Group-MCAR. The experiments therefore provide limited evidence about how the approach performs when developers have different training distributions or conflicting feature relationships.

(2) The classification evaluation relies on accuracy with a fixed 0.5 threshold, although the datasets include potentially imbalanced clinical and financial prediction tasks. For DECOMP, the method achieves 98.50% accuracy with zero variation across three seeds, while IHM accuracy is 82.67%. Without class prevalence, a majority-class baseline, or class-specific performance measures, it is difficult to assess the reported MIMIC-III improvement of 90.58% versus 71.33%. This matters because clinical prediction is one of the main domains used to demonstrate the framework’s effectiveness.

(3) Stronger and more closely related baselines would help clarify the novelty. In particular, Tan et al. (IJCAI 2023), which the paper cites, already use RKME specifications to learn mappings across feature spaces and reconstruct missing inputs for model reuse. The current paper differs by handling individual queries without additional user examples, but it does not experimentally compare against that approach. Comparing its reconstruction method with TabPFN under the same specification context and fixed downstream predictor, or explaining why such an adaptation is not feasible, would be helpful. Other established completion methods would also provide useful comparisons beyond mean and random filling. These experiments would make the added value of the proposed approach clearer.

Questions for authors

Q1. Could the authors report class prevalence, majority-class accuracy, and balanced accuracy for IHM, DECOMP, and the financial classification tasks? AUROC and AUPRC would also be informative for methods that produce prediction scores. Q2. What explains the differences in the benefits of auxiliary specifications across task groups, particularly the limited improvement in General classification and the lack of improvement in MIMIC-III classification? Q3. How does the joint specification context handle inconsistent feature relationships across developers?

6 comments
AuthorsOpenReview ↗

Response to Reviewer TGqF (Part 1/3)

Thank you for recognizing the value of cross-specification reuse and the controls that isolate the contribution of statistical relationships. We address each concern below.

1. "The incremental benefit of cross-specification information varies across tasks."

"particularly the limited improvement in General classification"

In the fixed-predictor comparison of Table 1, the predictor's own specification already provides useful conditional information: accuracy improves from 76.354% with Mean-F to 78.948% with TabPFN-F-Spec. Adding auxiliary context further improves 11/16 tasks, reaching 79.344% on average. Changes in completed inputs affect accuracy only when they change the predicted class, whereas regression error directly reflects numerical prediction changes, with lower RMSE on 12/14 General regression tasks.

"the lack of improvement in MIMIC-III classification"

Greater overlap between developers' training data may leave less complementary information in the auxiliary specifications. Class imbalance further affects how gains appear: positive cases constitute 1.5% of DECOMP and 11.5% of IHM queries, so overall accuracy is dominated by negative cases. In the same fixed-predictor comparison, DECOMP's AUROC rises from 83.14% to 92.50%, whereas IHM's changes from 75.00% to 75.34%. Clinical mean BA improves from 58.70% to 61.06%, despite nearly unchanged accuracy (89.333% to 89.250%). The complete task-level accuracy/RMSE results appear in Appendix E.1.2 (Table 11), and the corresponding clinical BA/AUROC diagnostics are reported in Table 12.

2. "Without class prevalence, a majority-class baseline, or class-specific performance measures, it is difficult to assess the reported MIMIC-III improvement of 90.58% versus 71.33%."

"Could the authors report class prevalence, majority-class accuracy, and balanced accuracy for IHM, DECOMP, and the financial classification tasks? AUROC and AUPRC would also be informative for methods that produce prediction scores."

Thank you for raising this point. The comparisons below use the main comparison's queries and predictions, retaining the original 0.5 threshold for accuracy and BA. The reported clinical accuracy improvement is driven mainly by IHM, where balanced accuracy also improves over LLM-Completed Models.

Positive-class prevalence is 11.50% for IHM and 1.50% for DECOMP, with majority-class accuracies of 88.50% and 98.50%, respectively. The eight Financial tasks average 16.25% positive prevalence and 83.75% majority-class accuracy. The baseline predicts the training-set majority class, giving 50% BA on these binary tasks. All table entries are percentages. The Financial entries are equal-weight averages over eight tasks, after averaging each task's metrics over three seeds.

TaskMethodAccuracyBAAUROCAUPRC
IHMLLM-Completed Models49.1766.2477.4334.19
IHMOurs82.6770.0377.3230.75
DECOMPLLM-Completed Models93.5058.4082.9715.78
DECOMPOurs98.5050.0091.5414.45
Financial (8 tasks)LLM-Completed Models79.7767.1676.7346.45
Financial (8 tasks)Ours83.1570.6579.2152.36

On IHM, accuracy improves by 33.50 percentage points and BA by 3.80 points. Under the original equal-weight clinical average, IHM contributes 16.75 of the 19.25-point accuracy gain, or approximately 87%. For Financial classification, Ours improves all four group-average metrics over LLM-Completed Models; Ours also exceeds majority-class BA on all eight tasks. Across all 26 classification tasks, Ours has higher balanced accuracy than the majority-class baseline on 25 tasks. For DECOMP, Ours' majority-class decisions explain the unchanged 98.50% accuracy and 50% BA across seeds; its AUROC and AUPRC additionally describe the ranking information in its scores. We have added task-level prevalence and majority-class baselines and expanded the accuracy, BA, AUROC, and AUPRC comparisons in Appendix E.2.1 (Tables 13–15) of the revision.

AuthorsOpenReview ↗

Response to Reviewer TGqF (Part 2/3)

3. "Comparing its reconstruction method with TabPFN under the same specification context and fixed downstream predictor … would be helpful."

Thank you for suggesting this comparison. The original Introduction and Section 4.2 describe query-specific completion for individual inputs without predefined feature blocks or additional user examples. Tan et al. [1] learn mappings across feature spaces for reconstruction. We compare the reconstruction methods on all 41 tasks in a common feature-block setting: both receive the same specifications and incomplete queries, reconstruct the same missing block without additional user examples, and use the same fixed LightGBM predictor. NRMSE normalizes RMSE by the query-target standard deviation.

MetricTan et al.TabPFN completion
Accuracy, 26 classification tasks (%)76.8379.78
BA, 26 classification tasks (%)54.9558.12
NRMSE, 15 regression tasks0.9410.802

TabPFN improves accuracy/RMSE on 34/41 tasks, including 14/15 regression tasks. This matched comparison supports the added value of conditional completion over the reconstruction approach of Tan et al. We have added the comparison settings, complete results, and feature-partition/encoding controls to Appendix E.2.2 (Tables 16–19) of the revision.

"Other established completion methods would also provide useful comparisons beyond mean and random filling."

To extend revised Table 4 (submitted Table 3), we compare TabPFN completion with KNN and Bayesian-ridge iterative imputation. General classification accuracy is 80.948% with TabPFN, versus 80.667% with KNN and 79.062% with Bayesian ridge. TabPFN has higher mean accuracy in all three classification groups and lower clinical regression RMSE than these two methods. We have added these comparisons to Section 5.4 (revised Table 4) and Appendix F.2 (Tables 23–24) of the revision.

4. "The experiments therefore provide limited evidence about how the approach performs when developers have different training distributions or conflicting feature relationships."

"all developer models are constructed from subsets of the same benchmark training partition"

Beyond the original controlled benchmark (Section 5.1), we use three independently collected sales-forecasting datasets, evaluating prediction on Predict Future Sales (PFS) [2] with models trained on M5 [3] and Corporación Favorita [4]. PFS contributes neither training data nor specifications to the model library. The table compares Ours with the strongest classical-completion baseline; RMSE is computed on the common log-sales target. We have added this evaluation to Section 5.3 (revised Table 3), with the protocol and store-level results in Appendix E.2.3 (Table 21).

Observed query inputsBest classical baselineBaseline RMSEOurs RMSE
Sales historyMean imputation0.9780.906
Sales history and relative pricesRandom forest iterative imputation0.9810.923

Ours reduces RMSE by 7.3% and 5.9%, respectively, with improvements at every target store. These results show that the framework can reuse models trained on independent sources despite differing input requirements.

"when developers have different training distributions"

We additionally test whether auxiliary specifications remain useful when developers have different training distributions. We divide the training data into groups above and below the median of one feature. Each auxiliary developer samples from the two groups at a ratio of 50:50, 80:20, or 95:5, while the target predictor and queries remain fixed. With unequal sampling proportions, TabPFN improves over mean filling on three of the four datasets (Miami, Airfoil, and Protein), reducing RMSE by 0.7–10.3%. This isolates the contribution of auxiliary specifications under a change in their training distribution. We have added the results to Appendix F.7 (Fig. 8) of the revision.

"and query missingness follows MCAR or Group-MCAR."

Beyond the original random masking (Appendix D.3), we evaluate grouped MAR: the probability that a feature group is missing depends on an input that remains observed. We test weaker and stronger dependence, keeping expected overall missingness at 30%. Relative to mean filling, TabPFN reduces RMSE by 14.7% and 14.0% on average, respectively, with improvements on 4/4 datasets at both strengths. The completion benefits persist on all four datasets when missingness depends on observed features. We have added these experiments and the paired mask-permutation control to Appendix F.6 (Fig. 7 and Table 29) of the revision.

AuthorsOpenReview ↗

Response to Reviewer TGqF (Part 3/3)

5. "Q3. How does the joint specification context handle inconsistent feature relationships across developers?"

Thank you for raising this point. In Section 4.2 (Eqs. (4)–(5)), the joint feature set JJ provides a common coordinate system for heterogeneous feature subsets, while conflicting support points remain in the joint context. When identical observed features correspond to different values of a missing feature, TabPFN combines these responses with its pretrained prior to estimate a conditional predictive distribution. This distribution can assign probability to alternative missing values.

We use this distribution for completion and its normalized entropy to compute model-level uncertainty (Eq. (6)). If the conflicting evidence produces a more uncertain completion, the aggregation rule favors models requiring less uncertain inputs (Section 4.3). Thus, the method incorporates conflicting evidence through probabilistic completion and uncertainty-aware aggregation; it does not explicitly decide which developer is correct.

We hope these clarifications and additional comparisons address your concerns.

References

[1] Peng Tan, Zhi-Hao Tan, Yuan Jiang, and Zhi-Hua Zhou. Handling learnwares developed from heterogeneous feature spaces without auxiliary data. In Proceedings of the 32nd International Joint Conference on Artificial Intelligence, pages 4235–4243, 2023.

[2] Kaggle. Predict Future Sales, 2018. URL https://www.kaggle.com/competitions/competitive-data-science-predict-future-sales.

[3] Kaggle. M5 Forecasting - Accuracy, 2020. URL https://www.kaggle.com/competitions/m5-forecasting-accuracy.

[4] Kaggle. Corporación Favorita Grocery Sales Forecasting, 2017. URL https://www.kaggle.com/competitions/favorita-grocery-sales-forecasting.

Comments on the paper

1 comment
Area_Chair_s84dOpenReview ↗

Prompting for Discussion

Hi Reviewers,

The authors have responded to your reviews. Can you please engage in discussion with them? Have your concerns been addressed?

Regards, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.