- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes TabACE, a training-free wrapper for frozen tabular foundation models. For each query, it runs the backbone on the standard ("Base") context and on R alternative contexts. These alternatives are built by row filtering or reweighting, feature dropping or clipping, kNN label repair, and source-to-target transport. TabACE then aggregates the alternative predictions using weights based on confidence, agreement, and a learned per-constructor prior, and blends the aggregate with Base through a gate α = α_max·U·E. Fusion parameters are calibrated out-of-fold on other datasets.
Critical strengths
Backbone-agnostic design. TabACE wraps frozen predictors without retraining, demonstrating gains across four TFMs (LimiX-2M, TabICL-v2, TabPFN-v3, TabDPT v1.2) and conventional models (kNN, Logistic/Ridge, CatBoost, MLP), making it easy to adopt.
Broad evaluation. The study covers classification and regression, 12 change families at five strengths, approximately 100 OpenML datasets, synthetic experiments with bootstrap confidence intervals, and natural shifts on Folktables.
Informative ablations. Context-budget, aggregation rule, and component ablations isolate key design choices, with Holm-corrected tests for the latter. The gate analysis further clarifies when predictions change. The author also reports failure cases, including a 43.8% worsening rate in one regression setting.
Critical weaknesses
Unjustified context size. The experiments use only 256 context rows without justification, despite several CC18 datasets being much larger (e.g., CIFAR-10 has 60K rows and Devnagari-Script about 92K). This is particularly concerning given that TabICLv2 and TabPFNv3 support much larger contexts (48K rows or more). Such aggressive subsampling may degrade model performance and increase prediction instability, potentially favoring ensembling. The authors should justify this choice and evaluate larger context sizes.
Inadequate ensemble baselines. TabACE is an ensemble method over TFMs, yet the paper does not compare it against established or even naive ensemble methods. A single forward pass with a randomly sampled context is a potentially unstable and weak baseline, especially with only 256 context rows. Thus, the reported gains may reflect generic ensemble benefits rather than TabACE's specific contributions.
Limited generalizability due to perturbation and constructor matching. The perturbations appear closely matched to their corresponding constructors: Shortcut corrupts the top feature, while its constructor drops the top-correlated feature; Regression Concept introduces smooth drift, while its constructor removes the strongest smooth component. Gains diminish when these matched constructors are absent and are most pronounced when shifts appear in the Shared bank. Without such a match (i.e. when the perturbation/shift is not included in the Shared Constructor bank), performance is nearly identical to Base. The gain is also minimal in Folktables settings when the shift is not by design. These results raise concerns about generalizability beyond the specifically matched perturbations and constructors.
Questions for authors
-
Figure 5 reports latency for random contexts but not their accuracy. Could the authors report accuracy to enable a fair comparison?
-
Why use 256 labeled rows for classification but up to 2,048 for regression? Do the reported gains persist with larger context sizes?
-
Folktables evaluation. Which family was specified for the Specific setting? Could the authors report variance across random seeds and compare against DistPFN and Drift-Resilient TabPFN?
-
Regression gate robustness. Since Ureg increases with Base–aggregate disagreement, updates are largest when the alternatives disagree most with Base. How does the method behave when these alternatives are correlated and jointly incorrect?
-
Failure under confident errors. In classification, a low-entropy but incorrect Base prediction closes the gate, potentially preventing useful corrections. How frequently does this occur, and how does it affect overall performance?
8 comments
Part 2: Ensemble baselines and random-context accuracy
Weakness 2 — inadequate ensemble baselines
TabACE is an ensemble method over TFMs, yet the paper does not compare it against established or even naive ensemble methods. A single forward pass with a randomly sampled context is a potentially unstable and weak baseline, especially with only 256 context rows. Thus, the reported gains may reflect generic ensemble benefits rather than TabACE’s specific contributions.
Question — accuracy of random-context ensembles
Figure 5 reports latency for random contexts but not their accuracy. Could the authors report accuracy to enable a fair comparison?
Thank you for suggesting these comparisons. We now report the predictive performance of random-context ensembles alongside the native ensemble and learned static weighting. Native-6 enables TabICL-v2’s six-estimator native ensemble; Random-6 averages predictions from six independently sampled contexts; Convex Stacking-6 learns fixed nonnegative weights summing to one on calibration data. Shared uses one Base and five alternative-context predictions. All four ensemble methods use the same backbone and six predictions per query. The table reports classification Acc/AUC and regression /nRMSE under clean data, the five corruptions, and the seven shifts.
| Task | Method | Clean | Five corruptions (avg.) | Seven shifts (avg.) |
|---|---|---|---|---|
| Classification | Base | 0.804 / 0.901 | 0.494 / 0.676 | 0.763 / 0.826 |
| Classification | Native-6 | 0.803 / 0.901 | 0.507 / 0.683 | 0.763 / 0.825 |
| Classification | Random-6 | 0.800 / 0.897 | 0.523 / 0.696 | 0.764 / 0.828 |
| Classification | Convex Stacking-6 | 0.800 / 0.898 | 0.559 / 0.704 | 0.766 / 0.827 |
| Classification | Shared | 0.804 / 0.901 | 0.575 / 0.720 | 0.772 / 0.831 |
| Regression | Base | 0.739 / 0.476 | −1.077 / 1.270 | 0.273 / 0.730 |
| Regression | Native-6 | 0.740 / 0.487 | −0.597 / 1.145 | 0.287 / 0.721 |
| Regression | Random-6 | 0.742 / 0.505 | −0.397 / 1.089 | 0.351 / 0.702 |
| Regression | Convex Stacking-6 | 0.733 / 0.499 | −0.446 / 1.104 | 0.401 / 0.682 |
| Regression | Shared | 0.738 / 0.478 | 0.363 / 0.783 | 0.467 / 0.659 |
Shared outperforms native ensembling, random-context averaging, and static learned weighting on both classification metrics and both regression metrics under corruptions and shifts. Random-context ensembling improves over Base in several settings, but does not reproduce Shared’s robustness. This directly addresses the concern that the improvements merely result from repeated backbone predictions with a small context.
We additionally evaluated DENSE [2], GES, robust aggregation, TabPFN PHE [3], RF-PFN, and OligoICP [4]. The complete baseline tables and budget conventions are provided in Appendix E.1, Tables 16–19, pp. 36–37, covering both TabICL and TabPFN classification and regression. The Random-6 row above supplies the predictive-performance counterpart to the random-context latency comparison in Figure 5.
Part 3: Generalization beyond perturbation–constructor matching
Weakness 3 — matched mechanisms and limited generalizability
The perturbations appear closely matched to their corresponding constructors: Shortcut corrupts the top feature, while its constructor drops the top-correlated feature; Regression Concept introduces smooth drift while its constructor removes the strongest smooth component. Gains diminish when these matched constructors are absent and are most pronounced when shifts appear in the Shared bank.
Without such a match (i.e., when the perturbation/shift is not included in the Shared Constructor bank), performance is nearly identical to Base. The gain is also minimal in Folktables settings when the shift is not by design. These results raise concerns about generalizability beyond the specifically matched perturbations and constructors.
Thank you for raising this concern. We directly tested whether TabACE remains effective without a matching constructor or calibration exposure to the evaluated mechanism.
Separating calibration coverage from constructor matching (Appendix E.3, Table 22, p. 39). We conduct a Shortcut ablation, independently varying these two factors, with the same available supervision across configurations. C/D use nonmatching constructions; B/D exclude Shortcut from policy calibration.
| Configuration | Shortcut in calibration | Matched constructor | Acc | AUC |
|---|---|---|---|---|
| Base | — | — | 0.51719 | 0.65690 |
| A | Yes | Yes | 0.74394 | 0.87909 |
| B | No | Yes | 0.70777 | 0.85370 |
| C | Yes | No | 0.71187 | 0.85757 |
| D | No | No | 0.69333 | 0.84662 |
C/D outperform Base without the matched feature-dropping constructor; D retains this advantage even when Shortcut is also absent from calibration. These results show that useful corrections can arise from the remaining contexts rather than requiring a directly matched repair branch.
Problem-aware construction is also a useful option when problem knowledge is available: SPN [5] constructs strategic in-context examples to adapt frozen TFMs to post-manipulation inputs. Specific uses available family knowledge to construct additional candidates, while Shared requires no mechanism identifier and remains effective without dedicated matching constructors, as tested below.
Mechanisms outside the Shared bank (Appendix E.4, Tables 23–24, p. 39). We evaluate three new corruptions and three new shifts, all excluded from policy calibration and without adding dedicated matching constructors. Results average strengths and the listed datasets, with the same available supervision and no additional target labels. The corruptions modify source contexts; rotation and nonlinear sensor transformations modify classification queries or regression source-context features; interaction-dependent sampling reweights complete source rows with their labels.
| Unseen mechanism | Datasets (Cls./Reg.) | Classification Acc: Base / Shared | Regression : Base / Shared |
|---|---|---|---|
| Heavy-tailed noise | 64 / 35 | 0.7875 / 0.8176 | 0.6380 / 0.6757 |
| Column misalignment | 72 / 35 | 0.7824 / 0.8186 | 0.6179 / 0.6751 |
| Feature quantization | 64 / 35 | 0.7070 / 0.7576 | 0.3743 / 0.5023 |
| Feature rotation | 31 / 31 | 0.7609 / 0.7861 | 0.3375 / 0.4663 |
| Nonlinear sensor transformations | 32 / 33 | 0.8002 / 0.8208 | 0.1009 / 0.4413 |
| Interaction-dependent sampling | 35 / 33 | 0.8214 / 0.8288 | 0.6487 / 0.6860 |
Shared improves both classification Acc and regression in all six settings. This directly addresses the statement that performance is nearly identical to Base when the evaluated mechanism is absent from the constructor bank: the additional evaluations demonstrate gains under precisely that condition.
Natural shifts. We agree that some Folktables gains are modest and depend on the task and migration setting. We therefore broadened the evaluation to bidirectional hospital-system transfer on PhysioNet Sepsis 2019 and admission-source transfer on UCI Diabetes 130-US Hospitals (non-emergency-room → emergency-room). Shared and Specific improve OOD Acc in all three transfer settings; for example, readmission Acc is 57.27% / 58.97% / 59.24% for Base / Shared / Specific. Together with the unseen-mechanism tests, these clinical results provide further evidence of generalization beyond the specifically matched synthetic conditions. Complete clinical results for the 1:1 case–control evaluation are provided in Appendix E.5.4, Table 27, p. 42.
Part 1: Context size and larger-context evaluation
Weakness 1 — justification for the context budget
Question — classification/regression context budgets and persistence of gains
Thank you for highlighting the context-budget choice. We used a compact 256-row setting to keep the broad robustness evaluation computationally manageable, informed by the small-context setting examined in TabDPT [1] (its inference ablation explicitly includes 256 rows). Classification and regression used separate sampling protocols, with up to 2,048 source-context rows for regression. We have now extended the evaluation to common context caps of 1,024, 4,096, and 8,192 rows for both tasks. The following TabICL-v2 results cover 72 classification and 35 regression datasets for clean/corrupted data, and 35 classification and 33 regression datasets for shifts.
Classification (Acc / AUC; Appendix E.2, Tables 20–21, p. 38).
Regression (R2 / nRMSE).
Shared and Specific continue to outperform Base under both corruptions and shifts at every evaluated context cap, while maintaining close clean performance. The larger contexts also strengthen Base on clean data, yet the robustness benefits remain. These results directly address the concern that TabACE’s gains arise only from a small, unstable 256-row context.