acceptodds
Under review as a conference paper at ICLR 2027

RelICL: Training-free Relational Learning with Tabular Foundation Models

10 views

Abstract

Tabular foundation models achieve state-of-the-art performance on single-table tasks without any training. Recent work suggests that they are also well-suited for relational learning via deep feature synthesis (DFS), which flattens a relational schema into a single table by adding aggregates of the other tables' columns as features. This approach is appealing because it directly benefits from improvements to or customization of the underlying tabular foundation model. In this paper, we identify two key problems with DFS: feature explosion and interaction blindness. The first problem arises because the number of DFS features grows quickly as the schema becomes more complex, limiting scalability and performance. The second problem arises because column-wise aggregates do not account for feature interactions, limiting performance. We propose and explore an alternative method termed RelICL, which keeps the benefits of DFS but alleviates these two problems. At its heart, RelICL propagates and fuses information step by step through the schema graph, using the same tabular foundation model that is eventually used for prediction to do so. In our experimental study using RelBench tasks, RelICL was on par with the strongest approach based on deep feature synthesis.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
relational learning, relational data, relational database, tabular foundation models, in-context learning, deep learning, machine learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_GpZSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces RELICL, a framework for relational data with tabular foundation models. The paper identifies two challenges within the relational data framework, and proposes RELICL that replaces column-wise aggregation with step-by-step row embedding propagation and fusion using an inference Directed Acyclic Graph (DAG). The claims of the paper shown empirically, and results show competitive performance of RELICL.

Critical strengths

In general, the paper is organized and easy to follow. The problems that the paper identifies (which has been around in the literature, but not well formulated) are key to improving tabular models for relational data. Moreover, the performances of the proposed method is competitive.

Critical weaknesses
  • The scope of the datasets used to prove the claims is limited. Only a few datasets from OpenML, which are probably mostly numerical (ignoring non-numerical values) could weaken the argument. Moreoever, most of the analysis on RelBench, which authors do mention as the limitation, could be critical.
  • It would be good to have some comparisons and analysis in computational perspectives.
Questions for authors
  • Since the evaluation is rather limited on RelBench and the results show mixed results (for instance compared to TabPFN-Rel), what are some characteristics from the data that guarantees strong performances or that might hurt RELICL?
2 comments

Thank you for your detailed, constructive feedback and support of our work.

We share our thoughts on the weak points you raised as well as the questions below. We revised the paper accordingly; changes are highlighted in blue.

We'd like to briefly position our paper first: The goal of RelICL is to explore whether strong tabular foundation models (TFM) can be leveraged directly for zero-shot relational learning, without being limited by feature explosion and interaction blindness and without task-specific training, fine-tuning or tuning. Our results indicate that RelICL, albeit simple, is competitive with SOTA methods (and it was the best-performing method that did not use per-task training/fine-tuning/tuning).

W1: Limited scope of datasets & focus on RelBench

We agree that (i) the experiments exploring Assumption 1 focused only on a few classification datasets from OpenML and one TFM, and (ii) our main empirical evaluation was limited to RelBench.

To address (i), we extended our experiments (Sec. 3.1, App. C.2) to include twice the number of datasets (including regression datasets and with different feature types) and also added a different TFM (TabPFN). In short, we found that Assumption 1 is reasonable in that encoding and dimensionality reduction typically retain TFM performance (small drops, if any, often not statistically significant). There are exceptions, but as RelICL is based on Assumption 1, our main experimental results provide further justification for the assumption. (Also note that the DFS-based approaches inherently assume that feature interactions do not carry predictive signal, an assumption that RelICL does not make.)

(ii) is more difficult to address. The goal of our experimental study was to compare RelICL to prior work, and prior work was predominantly evaluated on RelBench (a diverse benchmark in itself: 21 tasks, small and large, regression and classification). We added results on the RelArena benchmark (Sec. 4, App. E.6) to the paper, but note that RelArena is based on RelBench and that RelArena results for relevant related work (Tab. 4) are often unavailable. We found that RelICL was competitive on RelArena as well: It was within critical distance of the best-performing method, without using per-task training/fine-tuning/tuning (in contrast to TabPFN-Rel or RDBLearn, which use per-task tuning).

W1: Details

The following table summarizes our experimental results around Assumption 1. Performance loss when using embeddings (4th vs. 3rd column) and dimensionality reduction (5th vs. 4th col) was small and often not statistically significant (see standard errors). Dataset statistics and detailed results per dataset are available in App. C.2.

TaskBackboneT(R)T(R)T(Tenc(R))T(T_{enc}(R))T(reduce(Tenc(R)))T(reduce(T_{enc}(R)))
Class. (AUROC)TabICL90.8 +- 2.789.3 +- 2.688.5 +- 2.8
Class. (AUROC)TabPFN91.3 +- 2.789.2 +- 2.788.0 +- 2.8
Regr. (R2R^2)TabICL83.3 +- 6.080.5 +- 7.079.8 +- 7.0
Regr. (R2R^2)TabPFN83.7 +- 6.082.3 +- 6.080.1 +- 6.2

Only a few datasets from OpenML, which are probably mostly numerical (ignoring non-numerical values) could weaken the argument.

Thanks! Half of the now 20 datasets also have categorical features, and two of them are purely categorical (kr-vs-kp and mercedes). We added feature types to the dataset statistics table in the appendix.

W2: Missing comparison and analysis of computational perspectives

We added run time results to App. E.7 (single GPU). RelICL's run time grew sublinearly with respect to the number of validation/test rows but it did not grow with the number of rows in the database (due to sampling). Most of RelICL's run time stemmed from the TFM. We did not optimize our code for faster run time, and a direct run time comparison with other methods would require rerunning all baselines on the same hardware, which is infeasible for us.

Also note that RelICL needs neither training nor per-task tuning (App. D.3 and D.4) and that the number of features added to the task table does not explode with the schema size (unlike DFS-based methods, Sec. 2.3).

W2: Details

It would be good to have some comparisons and analysis in computational perspectives.

Compared to many GNN-based methods, e.g., GraphSAGE or RelGNN, RelICL needs no training time. RelICL uses a common set of hyperparameters for all databases and tasks (App. D.3), whereas other methods tune a configuration per task (e.g., RDBLearn and TabPFN-Rel).

Compared to DFS-based methods, RelICL does not require any per-task tuning. Further, the number of features added to the task table for RelICL depends only on its direct neighbors (Sec. 3), while for DFS-based methods it depends on the number of meta paths, which may grow exponentially with respect to the number of tables in the database (Lam et al., 2017; Sec. 2.2 and 2.3).

The added run time results (App. E.7) were produced on a shared cluster on a single GPU; we did not optimize our code towards better run time (no parallelization, single thread, no caching of embeddings or rewrites, etc.).

In short, RelICL's prediction time is dominated by the TFM used to compute row embeddings and make the final predictions (more than half of the total run time, based on the logs). The run time per task is correlated with the number of validation/test rows per task; more of those lead to more batches, in turn leading to a longer prediction time (App. E.7, Fig. 7, left panel). Run time does not grow with the database size (right panel) because RelICL samples rows per context table (App. D.3, Tab. 9b).

Q1: Data characteristics that influence RelICL's performance

Since the evaluation is rather limited on RelBench and the results show mixed results (for instance compared to TabPFN-Rel), what are some characteristics from the data that guarantees strong performances or that might hurt RELICL?

Thank you for this question. We added an exploration of data characteristics (number of tables, columns, and rows) to App. E.5, Fig. 5. We found no evidence for a relationship between these data characteristics and the performance of RelICL; they do not seem to capture task difficulty. In comparison to TabPFN-Rel, RelICL does not tune per task.

Q1: Details

The panels in Fig. 5 (App. E.5) plot the rank of RelICL against the number of (i) tables, (ii) columns, and (iii) rows of its database. The rank is among the models we compared against in the main section (Tab. 5a and 5b). None of (i)--(iii) showed a visible correlation with the rank, i.e., we found no evidence that these data characteristics influence RelICL's performance.

Reviewer_qDNZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper argues that deep feature synthesis, as used by RDBLearn and TabPFN-Rel, suffers from feature explosion and interaction blindness. It proposes RelICL, which encodes context tables with the TFM's row encoder. It then pools these embeddings along foreign keys toward the task table and runs the TFM on the result. On the 21 RelBench entity tasks it wins 11 of 21 comparisons against TabPFN-Rel. All baseline numbers are copied from earlier papers.

Critical strengths

The method is simple, training-free and works with different backbones. Its input width does not grow with the number of meta-paths. The paper is clearly written and the code is released.

Critical weaknesses
  1. The comparison with TabPFN-Rel is not fair. The released configs encode text columns with sentence embeddings, which the paper never mentions, while the TabPFN-Rel numbers come from its text-free version. The supplement's own ablation shows that without text RelICL would lose on study-adverse and study-outcome. The code also cuts the database at each test row's own timestamp instead of freezing it at the test cutoff, which gives RelICL extra context on the three rel-f1 tasks, all of which it wins. Five of the eleven wins are affected, and on the remaining tasks TabPFN-Rel leads 10 to 6.

  2. The default configuration was tuned on 12 of the 21 test tasks. RelICL beats TabPFN-Rel on 5 of the 6 tuned classification tasks but on only 1 of the 6 held-out ones. It also loses on every task with more than a million training rows, even though scalability is the paper's main motivation.

  3. The paper reports no ablation. The supplement contains one, and it suggests that count features and schema inlining, both close to DFS, account for much of the performance. It also has no random-encoder baseline, so it is unclear whether the TFM embeddings themselves help.

  4. Novelty is limited. Meyer et al. (AAAI Summer Symposium 2026) already pair a random GNN encoder with TabPFN on RelBench without training, and K-Space (DATA-FM at ICLR 2026) already uses TabICL's frozen row encoder. Xu et al. (ICML 2026) also test a random GNN plus TFM baseline and argue against mixing columns. Propositions 1 and 2 hold by construction and say little about real tasks.

Questions for authors
  1. Can you compare RelICL and DFS with the same backbone and text handling, under the standard RelBench time cutoff?
  2. Can you report results on the 9 held-out tasks separately, with several seeds?
  3. Can you include the ablations from the supplement and a random-encoder variant?
4 comments

Thank you for your detailed, constructive feedback and support of our work.

We share our thoughts on the weak points you raised as well as the questions below. We revised the paper accordingly; changes are highlighted in blue.

We'd like to briefly position our paper first: The goal of RelICL is to explore whether strong tabular foundation models (TFMs) can be leveraged directly for zero-shot relational learning, without being limited by feature explosion and interaction blindness and without task-specific training, fine-tuning or tuning. Our results indicate that RelICL, albeit simple, is competitive with SOTA methods (and it was the best-performing method that did not use per-task training/fine-tuning/tuning).

W1: Differences in the evaluation pipeline compared to TabPFN-Rel

Thanks! As you point out, RelICL uses text embeddings and a different cutoff than TabPFN-Rel. To provide more insight, we added two new results to the appendix.

First, we evaluated on RelArena (Hayler et al., 2026), where the cutoff regime is fixed for all models (Sec. 4.2, App. E.6). There, RelICL was the best-ranked method that needs neither training nor per-task tuning, without a significant difference from the best-performing method.

Second, we added an ablation without text embeddings (App. E.4) and found that text mattered only on study-adverse and study-outcome, but not on the other tasks.

Note that RelICL's setup follows common practice: RT, PluRel, and KumoRFM-2 use the same cutoff as we do, and the cutoff only affects one database (rel-f1). Likewise, embedding text columns is standard in relational deep learning (e.g., RelGT, RT). We clarified this in Sec. 4.1.

W1: Details

On RelArena-α\alpha, all models use the same cutoff regime. The critical-difference diagram (Fig. 2) did not separate RelICL from the best-ranked method, TabPFN-Rel (Sec. 4.2), which was released concurrently. All five entries ahead of RelICL train or tune per task. The cutoff regime used by RelICL is also used by RT, PluRel and KumoRFM-2.

You are right that without text, RelICL would perform worse on study-adverse and study-outcome (App. E.4). On the other ten ablated tasks, text barely changed the results. Text embeddings are natural to use, as RelGT, RT and TabPFN-Rel's closed-source API version all do.

W2: Performance on tasks not used for HP selection & scalability

You stated that RelICL seems to perform (i) better on the tasks used to select its default configuration and (ii) worse on tasks with many training rows.

Regarding (i), unlike RDBLearn and TabPFN-Rel, which select their configuration per task, RelICL used one configuration for all 21 tasks, selected on (the validation data of) 12 of them (App. D.3). This was done to keep costs affordable, and it puts RelICL at a disadvantage against models that invested more compute. We have now clarified this in App. D.3. Despite that, RelICL did well on the nine unused tasks: it outperformed the models from the main section on two (item-ltv, post-votes; MAE) and all but TabPFN-Rel on three (user-visits, user-badge, user-engagement).

Regarding (ii), RelICL addresses scalability in terms of the number of columns/features (feature explosion), not rows. However, on the nine tasks with more than a million training rows, RelICL outperformed the models from the main section on three (item-sales, item-ltv, post-votes; MAE) and all but the concurrently released TabPFN-Rel on three (user-churn on rel-hm, user-badge, user-engagement).

W3: Missing ablations

Thanks! You raised the points that (i) ablations were only present in the supplement, (ii) count features and schema inlining are close to DFS, and (iii) a random-encoder baseline is missing.

Regarding (i), we moved the ablations from the supplement into the appendix (App. E.4).

Regarding (ii), no single component explained RelICL's performance, e.g., removing schema rewrites hurt all six regression tasks but at the same time helped on half of the six classification tasks; removing count features was beneficial on some tasks but not on others. Neither component is specific to DFS, i.e., neither aggregates along meta-paths. Inlining joins the entity's own columns to the task table (without any aggregation), which is also done in the preprocessing of RDBLearn and TabPFN-Rel. Count features are added per relationship, similar to degree features used by GNNs, rather than per meta-path.

Regarding (iii), the paper now compares against two new baselines that pass the output of a randomly initialized GNN to a TFM: RandGNN (Xu et al., 2026; Sec. 4, Tab. 5) and Meyer et al. (2026; App. E.1). RelICL outperformed RandGNN on 13 of 15 tasks and Meyer et al.'s approach on all 12 tasks they reported results on.

W4: Limited novelty & missing references

You pointed out (i) Meyer et al. (2026) and K-Space (Azevedo et al., 2026) as prior work, (ii) Xu et al. (2026) as arguing against mixing columns, and (iii) Propositions 1 and 2 being irrelevant for real tasks.

Regarding (i), thanks! We discuss both approaches in the paper now (App. A, App. E.1). Meyer et al. is similar to DFS/RelICL in that it uses a TFM for the final prediction, but it differs in that it uses GNN-based propagation instead of TFM embeddings. RelICL outperformed it on all 12 tasks they report results for (App. E.1). K-Space is not training-free; despite this RelICL outperformed it on 10 of 12 tasks (App. E.1).

Regarding (ii), the Xu et al. reference is the RDBLearn paper, which we already discussed (Sec. 2, Sec. 4). Proposition 2 (and the example in Tab. 2) shows that its argument against mixing columns leads to interaction blindness. In addition to RDBLearn, Xu et al. also report results on RandGNN, which models feature interactions with a randomly initialized GNN and predicts with a TFM. We have now added this model to Sec. 4 and App. E.1. Xu et al.'s empirical evidence against mixing columns is that RDBLearn outperforms their RandGNN baseline. RelICL instead models feature interactions with a pretrained TFM encoder and outperformed both RDBLearn (10 of 15 shared tasks) and RandGNN (13 of 15; Tab. 4). So we disagree with the arguments made by Xu et al.

Regarding (iii), maybe, maybe not. We consider ignoring feature interactions a severe limitation, and our work points it out and provides empirical evidence.

Questions

Can you compare RelICL and DFS with the same backbone and text handling, under the standard RelBench time cutoff?

Please see our response to W1.

Can you report results on the 9 held-out tasks separately, with several seeds?

Please see W2 regarding the discussion on held-out tasks. Tab. 5 contains results on all the tasks, including the ones not used for hyperparameter search. In addition, App. E.3 now reports RelICL without ensembling as the mean and standard error over three seeds. The seed noise was small; on average over the tasks, the standard error was 0.2 AUROC points for classification and less than one percent of the respective MAE for regression.

Can you include the ablations from the supplement and a random-encoder variant?

Please see W3: the ablations are now in App. E.4, and RandGNN (Sec. 4) and Meyer et al. (App. E.1) are random-encoder baselines from prior work you mentioned.

I appreciate the author’s detailed response and the additional experiments. The new RelArena results, the ablation study excluding text, the random encoder baseline, and the runtime analysis have addressed some of my concerns. However, core issues remain:

Fairness of comparison: A controlled comparison against DFS, using the same backbone, identical text processing, and the standard RelBench temporal cutoff, is still missing. Furthermore, on RelArena (with a unified cutoff), RelICL does not outperform TabPFN-Rel.

Validation of primary contributions: While the new ablation study shows that "no single component accounts for the performance," it fails to isolate the specific contribution of row-embedding propagation. External methods like RandGNN and those proposed by Meyer et al. involve different workflows and cannot serve as a substitute for a controlled experiment where the TFM embeddings within RelICL are replaced by a random encoder.

Generalizability: On tasks not used for hyperparameter tuning, as well as on ultra-large-scale tasks, RelICL generally continues to underperform compared to TabPFN-Rel.

Therefore, I maintain my original rating.

Thank you for your answer and appreciation of our work. We have some thoughts on your remaining concerns and would be grateful if you considered them in your evaluation. We discuss your remaining concerns in turn.

Generalizability: On tasks not used for hyperparameter tuning, as well as on ultra-large-scale tasks, RelICL generally continues to underperform compared to TabPFN-Rel.

The goal of the paper is not to outperform TabPFN-Rel or any other DFS method. Our goal is to explore how strong TFMs are on relational data without any task-dependent tuning. All of the DFS methods rely on task-dependent tuning (or, if you will, task-dependent feature engineering).

Our results indicate that TFMs without any task-dependent tuning are competitive with all current models that tune per task. We conclude this because neither TabPFN-Rel (both the open-source and the API version) nor any other current relational model outperforms RelICL on RelArena in a statistically significant fashion (see the critical-difference diagram, Fig. 2).

Fairness of comparison: A controlled comparison against DFS, using the same backbone, identical text processing, and the standard RelBench temporal cutoff, is still missing. Furthermore, on RelArena (with a unified cutoff), RelICL does not outperform TabPFN-Rel.

It is not possible to perform such a comparison. RDBLearn and TabPFN-Rel perform task-specific feature engineering and tuning. In fact, the main difference between RDBLearn and TabPFN-Rel lies in feature engineering (such as introducing cyclic features).

That being said, on RelArena, TabPFN-Rel does use the same backbone and cutoff as RelICL does. In its API version, it also uses text embeddings (and potentially other tricks that we are unaware of). Yet it does not outperform RelICL in a statistically significant fashion. RelICL is much simpler, does not perform any task-specific tuning or sophisticated feature engineering, yet it is competitive with leading commercial models.

Lastly, as per ICLR's reviewer policy, TabPFN-Rel must not be considered (and we do not need to report on it), because it is contemporaneous and not peer-reviewed work. We feel that using the performance of TabPFN-Rel against our work is not fair.

Validation of primary contributions: While the new ablation study shows that "no single component accounts for the performance," it fails to isolate the specific contribution of row-embedding propagation. External methods like RandGNN and those proposed by Meyer et al. involve different workflows and cannot serve as a substitute for a controlled experiment where the TFM embeddings within RelICL are replaced by a random encoder.

To address your concern, we performed such a controlled experiment (App. E.4), comparing RelICL (default configuration) to directly applying the TFM to the prediction table (i.e., the extended task table after rewrites) without row embeddings. We found that row embeddings consistently improved performance, providing evidence that they carry predictive signal.

Details

Summary table:

Task typeNo embeddingsRelICL
Regr. (MAE rel. to no embeddings)1.0000.941
Clf. (AUROC (%))72.575.9

Regression (MAE, lower is better):

TaskNo embeddingsRelICL
ad-ctr0.040.04
user-attendance0.260.24
driver-position3.743.68
item-sales0.060.04
site-success0.420.40
study-adverse33.0732.42

Classification (AUROC in %, higher is better):

TaskNo embeddingsRelICL
user-clicks64.264.3
user-ignore87.688.5
user-repeat76.479.3
driver-top366.181.4
user-churn67.468.4
study-outcome73.273.4
Reviewer_hEo7OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes RelICL, a training-free approach to relational learning using tabular foundation models (TFMs). It addresses two limitations of deep feature synthesis (DFS): feature explosion and interaction blindness. Instead of aggregating individual columns along meta-paths, RelICL extracts row embeddings from context tables and propagates them through the schema graph, using the same TFM for feature extraction, fusion, and prediction. Experiments on 21 RelBench entity tasks show that RelICL is competitive with strong DFS-based methods and outperforms several relationally pretrained models. The paper also demonstrates that the underlying TFM can be swapped without changing the method.

Critical strengths

RelICL addresses feature explosion and interaction blindness through row embeddings and stepwise information propagation. This provides a meaningful alternative to extensive feature engineering while retaining the flexibility of different tabular foundation models. And experiments on RelBench show that RelICL is competitive with strong DFS-based approaches and often outperforms relationally pretrained models. Its compatibility with different TFM backbones is also valuable, as improvements in tabular foundation models can potentially benefit relational learning directly.

Critical weaknesses

RelICL depends on Assumption 1: replacing the original features of a context table with its TFM row embeddings should preserve predictive performance. This is fundamental to the method, because the algorithm repeatedly encodes tables, replaces their features with embeddings, and propagates those embeddings toward the task table. However, the supporting experiment is limited to ten single-table binary classification datasets using TabICL. More importantly, the actual 32-dimensional reduction used by RelICL causes substantial performance drops on some datasets:

  • Philippine: 93.6 → 83.5 AUROC (-10.1 points).
  • Jasmine: 89.8 → 86.5 AUROC (-3.3 points).
  • Credit-g: 82.2 → 79.4 AUROC (-2.8 points).

This is not just a minor implementation detail. If the representation discards task-relevant information at one stage, subsequent pooling and encoding cannot reliably recover it. The paper does not demonstrate that this representation remains adequate across multiple relational hops, different table schemas, regression tasks, or the alternative TFM backbone. Consequently, the central assumption underlying RelICL remains insufficiently supported.

The main results are not obtained from one fixed RelICL configuration. They rely on configuration ensembling:

  • Regression uses two configurations, with the additional member introducing windowed count features over the previous 1, 7, and 28 days.
  • Classification uses four configurations. Besides the default, variants remove reference-count features, disable schema rewrites, or change how embeddings from different backbone estimators are reduced.

Yet the paper does not provide a clear component-wise evaluation showing how much performance comes from the proposed row-embedding propagation itself, as opposed to count-based feature engineering, configuration ensembling, and other implementation choices. This matters because the paper's main contribution is the replacement of DFS aggregation with row-embedding propagation. Without isolating that contribution, it is difficult to determine whether the proposed mechanism is responsible for the reported performance or whether the gains rely substantially on auxiliary feature engineering and ensembling.

Questions for authors

Can the authors provide a systematic ablation and evaluation of the row-embedding representation across multiple relational hops, including comparisons against the original context-table features? In particular, how does information loss introduced by the embedding and dimensionality-reduction stages affect downstream relational prediction?

3 comments

Thank you for your detailed, constructive feedback and support of our work.

We share our thoughts on the weak points you raised as well as the questions below. We revised the paper accordingly; changes are highlighted in blue.

We'd like to briefly position our paper first: The goal of RelICL is to explore whether strong tabular foundation models (TFM) can be leveraged directly for zero-shot relational learning, without being limited by feature explosion and interaction blindness and without task-specific training, fine-tuning or tuning. Our results indicate that RelICL, albeit simple, is competitive with SOTA methods (and it was the best-performing method that did not use per-task training/fine-tuning/tuning).

W1: Insufficient empirical evaluation

Thank you! Your thoughts were: (i) the experiments supporting Assumption 1 appear limited, (ii) dimensionality reduction seems to lead to a further loss in performance, and (iii) the paper does not demonstrate that embeddings remain adequate across multiple relational hops, different schemas, regression tasks, or another TFM backbone.

To address these points, we extended our experiments (Sec. 3.1, App. C.2) to include twice the number of datasets (including regression datasets and with different feature types) and also added a different TFM (TabPFN). In short, we found that Assumption 1 is reasonable in that encoding and dimensionality reduction typically retain TFM performance (small drops, if any, often not statistically significant). There are exceptions, but as RelICL is based on Assumption 1, our main experimental results provide further justification for the assumption. (Also note that the DFS-based approaches inherently assume that feature interactions do not carry predictive signal, an assumption that RelICL does not make.)

W1: Details

The following table summarizes our experimental results around Assumption 1. Performance loss when using embeddings (4th vs. 3rd column) and dimensionality reduction (5th vs. 4th col) was small and often not statistically significant (see standard errors). Dataset statistics and detailed results per dataset are available in App. C.2.

TaskBackboneT(R)T(R)T(Tenc(R))T(T_{enc}(R))T(reduce(Tenc(R)))T(reduce(T_{enc}(R)))
Class. (AUROC)TabICL90.8 +- 2.789.3 +- 2.688.5 +- 2.8
Class. (AUROC)TabPFN91.3 +- 2.789.2 +- 2.788.0 +- 2.8
Regr. (R2R^2)TabICL83.3 +- 6.080.5 +- 7.079.8 +- 7.0
Regr. (R2R^2)TabPFN83.7 +- 6.082.3 +- 6.080.1 +- 6.2

In the majority of the experiments, the loss stayed below 2 AUROC or R2R^2 points. We acknowledged in the paper that the assumption does not always hold (Sec. 3.1; e.g., philippine for both TFMs, wine_quality and mercedes mostly for TabICL, bike_sharing mostly for TabPFN). The assumption is explicitly stated though and can in principle be tested (as we did). Moreover, Tab. 2 suggests that RelICL's assumption is weaker than the one made by DFS-based methods (including RDBLearn and TabPFN-Rel).

(continues from above)

W2: Missing ablations on ensembling and further implementation choices

Thank you! You asked about performance gains from (i) ensembling and (ii) additional implementation choices such as adding count features.

Regarding (i), we have added results that compare the default ensembled version against a single configuration variant (App. E.3). The performance of the single version was slightly worse, yet still competitive (on par with TabPFN-Rel on classification). This suggests that RelICL's performance can only be attributed to ensembling to a small degree. We used ensembling in the paper because it is a well-established technique used by, e.g., TabICL and TabPFN, and therefore implicitly also by TabPFN-Rel, RDBLearn and other methods that rely on TFMs.

Regarding (ii), we have added ablations that each modified a single component of RelICL (App. E.4). In short, we have found that no single component explained RelICL's performance; some settings were more beneficial for some tasks but hurt on others. There is no "best" setting for all tasks, which motivated our use of ensembling.

W2: Details

Regarding (i): The table below compares RelICL's ensembled version against the single configuration variant. These results are averaged across tasks; see App. E.3 for results per task.

VariantClassification AUROCRegression MAE change
Ensemble76.3 +- 2.6reference
Single configuration74.8 +- 2.8+5.4% +- 3.5%

In short, we found that ensembling increased performance slightly on average for both classification and regression tasks. For classification, the gains were below 3 points AUROC on 10 out of 12 tasks; and yielded 5.2 points AUROC on a single task (user-clicks). On study-outcome, ensembling lost 0.9 points. Regression is hard to compare due to different MAE scales; ensembling yielded gains at most 0.3 MAE on 8 out of 9 tasks; the exception was item-ltv (6.36 MAE). Without ensembling, RelICL was still on par on average with TabPFN-Rel on classification (74.8 vs. 75.0 AUROC).

Regarding (ii): Please see App. E.4 for detailed results, which show ablations on the 12 tasks used for determining the initial hyperparameter configuration. The results compare RelICL with default hyperparameters but without ensembling against variants where one setting differed (e.g., no count features, no text embeddings). Some changes yielded a gain on some tasks (e.g., disabling count features on driver-top3, +2.1 points AUROC) but degraded results on others (-8.4 points AUROC on user-clicks). Overall, no single component explained RelICL's performance.

Q1: Further evaluations

You asked for: (i) ablations and evaluations of the row embeddings across multiple relational hops and against the context table features, and (ii) the effect of information loss from embeddings or dimensionality reduction for downstream prediction.

See our answer to W1.

Thanks for rebuttal. I will keep my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.