acceptodds
Under review as a conference paper at ICLR 2027

From Sequences to Tables: Sequential Recommendation with Tabular Foundation Models

Abstract

Sequential recommendation aims to predict users' next interactions from their historical behavior. Recent recommendation-specific prior-fitted networks (PFNs), pretrained on synthetic interaction sequences, have achieved strong empirical performance through in-context learning. Meanwhile, the PFN paradigm has also achieved strong results in tabular prediction, inspiring a range of tabular foundation models (TFMs). We investigate whether the general predictive capabilities of these TFMs can support sequential recommendation without recommendation-specific pretraining. To this end, we propose a reusable interface that reformulates next-item prediction as tabular candidate scoring. Our *content-aware feature construction* represents each history–candidate pair using item semantics and explicit short- and long-term relevance signals. These representations can be supplied to different TFM backbones, which condition on labeled historical examples to score and rank candidates while keeping their pretrained parameters fixed. Experiments on five benchmark datasets demonstrate strong recommendation performance, while ablation studies validate the effectiveness of our content-aware feature construction.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Sequential Recommendation, Tabular Foundation Model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ogY5OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes using general-purpose tabular foundation models (TFMs) such as TabPFN, TabFM, and TabICL for sequential recommendation, bypassing the need for recommendation-specific pretraining. The central contribution is a feature construction pipeline that converts each user history and candidate item pair into a tabular row. These rows encode semantic IDs built via residual K-means on Sentence-T5 embeddings, structured item attributes, cosine similarities to recent items, and an average similarity feature capturing long-term preferences. Labeled rows (positive and negative candidates) serve as in-context examples for a frozen TFM, which then scores test candidates in a single forward pass. Experiments on five benchmarks show that TabPFN-v3.5-SR and TabFM-SR outperform dedicated sequential recommenders on four of the five datasets, and ablations confirm that each feature group contributes meaningfully.

Critical strengths

The paper introduces a genuinely novel framing. The idea that frozen general-purpose tabular models can serve as drop-in backbones for recommendation, with no domain-specific pretraining at all, is clean and surprising. The fact that it actually works competitively on four of five datasets makes this more than a thought experiment. If the interface generalizes, improvements in TFMs would automatically translate to better recommendations, which is an appealing property.

The feature construction is thoughtful and the ablation study is thorough. The authors test single and joint feature removals across all five datasets, revealing interpretable patterns: short-term features matter more on short-sequence datasets while long-term features matter more on LastFM with its longer histories. The comparison against three alternative feature construction pipelines from prior work (Table 3) is a useful addition that goes beyond the usual ablation.

Critical weaknesses

The paper never discusses how the method scales. TFMs ingest labeled context rows in a single forward pass, so there is a hard limit on how much training data can be used. For the five benchmarks studied here, D_train could range from roughly 200K to 600K rows, yet TFMs like TabPFN handle at most a few thousand context rows. The paper is silent on how this is managed, whether rows are subsampled, how many context rows are actually used, and how results change as that number varies. Without this information it is impossible to assess whether the approach extends beyond small academic benchmarks, which undercuts the reusable pitch. This feels like a missing section rather than a minor omission.

The experimental comparison is not fully apples-to-apples. The TFM variants consume rich content features (Sentence-T5 embeddings, text-derived semantic IDs, handcrafted similarity scores) that several baselines never see. GRU4Rec, SASRec, CoSeRec, and SRPFN all operate on item ID sequences without text. The strongest baselines that do use text, UniSRec and RecFormer, come closest to the TFM variants. Meanwhile the tree-based models (XGBoost-SR and LGBM-SR) use exactly the same features and land close to the TFMs on most datasets, which raises the question of how much of the improvement comes from the TFM backbone versus the feature construction itself. The paper acknowledges this partially but does not disentangle the two contributions cleanly enough.

Questions for authors
  1. How many context rows does each TFM backbone actually receive at inference time, and how are they selected? This is a critical missing detail. I would like to see a sensitivity analysis of performance versus context size.

  2. Given that XGBoost-SR and LGBM-SR already perform competitively using the same features, what is the practical argument for preferring a frozen TFM over a fitted tree ensemble? The TFMs avoid training, but the feature construction itself (Sentence-T5 encoding, K-means clustering) already requires nontrivial compute. Could you quantify the end-to-end cost comparison?

  3. On Yelp, removing semantic IDs improves performance (Table 12). Have you investigated this further? Does it suggest the method needs some mechanism to decide when semantic IDs help, and is there a risk of similar degradation on other domains with noisy or generic item text?

Reviewer_ierBOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper explores the application of Tabular Foundation Models (TFMs) to sequential recommendation. It proposes a content-aware feature construction method that incorporates item semantics, structured attributes, and user context, augmented with explicit short- and long-term relevance signals. Each training sample is transformed into a single row that serves as a labeled example, which is fed into the TFM to enable in-context learning. Evaluations across five sequential recommendation benchmarks demonstrate that this approach achieves strong performance compared with existing recommendation systems and tree-based models under a random negative candidate setting.

Critical strengths
  1. This is a timely topic, given the current trend of tabular foundation models demonstrating promising performance across various structured-data tasks (e.g., tabular classification/regression, time series, and graph-based tasks).
  2. When combined with TFMs (e.g., TabFM and TabPFN), the proposed method shows strong performance compared with existing recommendation baselines and tree-based models.
Critical weaknesses
  1. The paper lacks clear insights and sufficient feature-level motivation. From a recommendation-system perspective, it is necessary to analyze which specific users or items benefit from individual features. Currently, the analysis is limited to popularity bias, which makes it difficult to understand the specific role of each feature component. Additionally, while TFMs are known to outperform tree-based models across various tabular tasks, the paper’s novelty, which centers on feature construction for sequential recommendation, remains ambiguous. It is unclear whether the proposed feature engineering is specifically tailored to TFMs or simply constitutes a set of generally effective features. In summary, as tabular foundation models continue to evolve, the paper should provide insights that remain useful and applicable in broader contexts.
  2. All experimental results are evaluated using randomly sampled negative candidates. However, this protocol can inflate metrics and lead to biased results [1, 2].

[1] On Sampled Metrics for Item Recommendation (https://dl.acm.org/doi/10.1145/3535335)

[2] A Case Study on Sampling Strategies for Evaluating Neural Sequential Item Recommendation Models (https://arxiv.org/pdf/2107.13045)

Minor errors

  1. There is an unresolved citation on line 111.
  2. HySAR is mentioned in Section 5.4, but it is not defined or referenced anywhere else in the paper.
Questions for authors
  1. Regarding W1, please clarify the motivation for each feature component from a recommendation perspective, and analyze how each component changes performance beyond overall metrics. For example, it would be helpful to show which users or items benefit from each component, such as users with different history lengths or items with different popularity levels.
  2. Regarding W2, could you report full-catalog ranking results?
  3. For sparse benchmarks with short average sequence lengths, such as the Amazon datasets, setting the recent history window to J=6J=6 may result in the same information being used to construct the long-term and short-term features for most users. Please analyze the impact of JJ on each dataset and examine how performance varies with JJ.
  4. For user features, the raw user ID unu_n and the position of the target interaction pnp_n are concatenated. However, unu_n carries no inherent meaning within the dataset, and pnp_n, assuming there are no duplicate items in the sequence, contains the same information as the term log⁡(1+mn)\log(1 + m_n) already included in the long-term preference features. Please conduct an ablation study on the user features.
  5. Please provide more details on how the tree-based model is implemented.
  6. Please include an analysis of inference cost, as well as an ablation study on the context size. Furthermore, as the dataset grows, the computational cost of constructing the tables for full-catalog ranking and performing inference may increase significantly.
Reviewer_VzmcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper explores whether general-purpose tabular foundation models can effectively perform sequential recommendation via in-context learning without recommendation-specific pretraining or parameter finetuning. To bridge sequential interaction data and tabular predictors, the authors propose a reusable content-aware feature construction interface. Each history–candidate pair becomes one tabular row comprising five feature blocks: content-derived semantic IDs for the candidate and the J most recent items, structured item attributes of the same items, short-term item-candidate cosine similarities, long-term aggregated history similarity, and user/position identifiers.Experiments evaluate three TFM backbones on five datasets, with comparisons against sequential recommenders, tree-based models using identical features, alternative feature pipelines, and feature ablations.

Critical strengths

1.A well-motivated perspective on reusable recommendation models. The paper demonstrates that frozen, general-purpose TFMs can support sequential recommendation through a content-aware tabular interface, without recommendation-specific pretraining or target-domain fine-tuning. This offers a practical way to bring advances in general tabular prediction to recommendation.

2.Modular backbone-agnostic Interface and strong empirical gains on tail items:Evaluating three distinct TFM backbones alongside GBDTs under the exact same tabular representation cleanly demonstrates that stronger tabular foundation models translate into better recommendation accuracy, especially on cold/tail items where collaborative signals are sparse.

3.Useful component and subgroup analyses. The ablations support the value of explicit candidate–history similarities and reveal dataset-dependent contributions of long-term information and structured attributes. The popularity-group analysis also shows improved ranking of tail targets relative to the two reported comparison methods.

Critical weaknesses

W1.Unclear context scaling and inference cost. Under the validation/test split in Section 5.1 with (K=1), constructing all training prefixes yields approximately 379k labeled rows on Sports and 450k on Yelp, with over three million test-query rows each. However, the paper does not specify the actual context sizes, context selection or partitioning strategies, or query batching. These omissions hinder reproducibility and assessment of computational feasibility. Furthermore, no inference latency, throughput, or peak-memory comparisons are provided, leaving the accuracy–cost trade-off against recommendation baselines unclear.

W2.Input asymmetry and potential temporal bias in metadata. The proposed variants and the tree baselines receive Sentence-T5-XXL-derived features plus structured attributes (Table 6), including log sales rank on Amazon (salesRank also appears in the text input) and stars and log review count on Yelp, whereas GRU4Rec/SASRec/CoSeRec (and SRPFN, as described in A.2) use only item IDs, and UniSRec/RecFormer use much smaller text encoders and no popularity fields. Consequently, GBDTs on the same rows already capture much of the gain (Beauty NDCG@10: 0.3778 best baseline → 0.4003 XGBoost-SR → 0.4302 TabPFN-v3.5-SR), so the paper cleanly shows “frozen TFM > GBDT on identical inputs” , not that general TFMs beat dedicated recommenders. Moreover, salesRank and review_count are crawl-time snapshots that include post-test activity, and under random negative sampling popularity alone is a strong discriminator.

W3.Direct contradictions between main text claims and experimental tables, plus missing hyperparameters.

  • In the Introduction, the authors state: "Our TabPFN-v3.5 variant achieves the best results on Beauty, Toys, and Sports, while our TabFM variant performs best on LastFM". However, Table 1 and Section 5.2 clearly show that TabFM-SR achieves the best results on all four metrics on Beauty and three out of four metrics on Toys.
  • In Section 5.4, the authors state: "On LastFM, it achieves the highest HR@10, while HySAR yields a slightly higher NDCG@10". Yet in Table 3, Ours achieves the highest NDCG@10 on LastFM at 0.3368 (bolded), outperforming Bauer & Jannach (2024) (HySAR) at 0.3311.
  • The number of clusters B, the negative-sampling distribution, the TFM configurations, and how the three pipelines in Table 3 were adapted are not given.
Questions for authors

1.Context Subsampling and Inference Efficiency. How are the large training and query tables processed by each TFM? Please specify the exact context window size, subsampling or ensembling strategy, query batch size, and report a wall-clock comparison of training/preprocessing time, per-user inference latency (ms/user), and peak GPU memory usage against SASRec, UniSRec, SRPFN, and LightGBM.

2.Popularity features and input comparability. What is the performance of TabPFN-v3.5-SR and XGBoost-SR on Beauty, Toys, Sports, and Yelp if log_sales_rank, salesRank text, and log_review_count are removed. Conversely, if UniSRec or SASRec is initialized with the same Sentence-T5-XXL embeddings and item attributes, does the performance gap still hold?

3.Contribution of semantic IDs. Given the small SID gains on Beauty, Toys, and LastFM, and improved Yelp results after removing SID, when is this feature block beneficial? Have the authors tested constructing SID or similarity features from collaborative item embeddings to address the performance deficit on Yelp?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.