acceptodds
Withdrawn from ICLR 2027

Expressiveness Limits of Autoregressive Semantic ID Generation in Generative Recommendation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Generative recommendation (GR) models generate items by autoregressively producing a sequence of discrete tokens that jointly index the target item. However, this autoregressive generation process also induces a structured decoding space whose impact on model expressiveness remains underexplored. Specifically, token-by-token generation can be viewed as traversing a decoding tree induced by semantic ID tokens, where leaf nodes correspond to candidate items. We observe that the item probabilities produced by GR models are strongly correlated with this tree structure: items that are close in the tree tend to receive similar probabilities for any given user, making it difficult to distinguish among them based on user-specific preferences. We further show theoretically that such structural correlations prevent GR models from representing even simple patterns that can be well captured by conventional collaborative filtering models. To mitigate this issue, we propose Latte, a simple modification that injects a latent token before each semantic ID, reshaping the decoding space from a single tree into multiple latent-token-conditioned trees. This design creates multiple paths with varying tree distances between items, relaxing tree-induced probability coupling and yielding an average of 3.45% relative improvement on NDCG@10.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Generative Recommendation, Semantic ID

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_esEMOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the expressiveness limitations of autoregressive semantic ID (SID) generation in generative recommendation (GR). The authors observe that items sharing longer SID prefixes tend to exhibit highly correlated generation probabilities across users, which may limit the model's ability to capture personalized preferences and complex item-item relationships. To formalize this issue, the paper analyzes two potential limitations: rank reversal suppression and forced transitivity in item similarities. To alleviate these limitations, the authors propose Latte, a simple approach that prepends an additional latent token to the original SID sequence. During training, the latent token is uniformly sampled at random, while during inference, the model autoregressively generates the latent token followed by the SID. This design introduces multiple latent-conditioned generation paths for each item, aiming to reduce the probability coupling induced by shared SID prefixes.

Critical strengths
  1. Interesting research problem. The paper investigates a potentially important issue in generative recommendation: whether the hierarchical structure of semantic IDs introduces undesirable coupling between item probabilities and limits personalized preference modeling. I find this research direction interesting and worth exploring. However, collaborative information can already be incorporated through various approaches, such as collaboration-aware item embeddings or the autoregressive GR model itself. Therefore, I am not fully convinced that the identified limitation constitutes a sufficiently significant bottleneck in practice.
  2. Clear presentation and well-organized writing. The paper is well written and easy to follow. The overall structure is logical, and the figures effectively illustrate the motivation, theoretical analysis, and proposed method. In particular, the visualizations provide an intuitive explanation of the potential limitations associated with SID-based generation.
  3. Simple and flexible method with consistent empirical improvements. The proposed Latte method is straightforward to implement and does not require substantial architectural modifications. Despite its simplicity, it achieves consistent improvements over the PSID baseline across multiple datasets and different semantic ID tokenization strategies, suggesting that the approach has some degree of generalizability.
Critical weaknesses

W1. Marginal improvements and the lack of compute-matched baselines Although the proposed Latte achieves consistent improvements over PSID, the overall performance gains are relatively marginal, especially considering the additional decoding cost introduced by the method. Specifically, Latte prepends an additional latent token to the original semantic ID, increasing the decoding sequence length from three to four tokens. While the design of this latent token is different from conventional semantic ID tokens and can be interpreted as a form of training-time regularization or stochastic target augmentation, it nevertheless introduces an additional autoregressive decoding step during inference. I am concerned that the comparison against PSID is not entirely fair from a computational budget perspective. The original PSID uses three semantic ID tokens, whereas Latte effectively uses four decoding tokens. Under the same decoding budget, a natural alternative would be to extend the original semantic ID from three to four tokens, potentially allowing the model to learn finer-grained item representations. The current experiments do not establish whether Latte provides better performance than simply allocating the same additional decoding budget to a longer semantic ID. Given the relatively modest improvements reported in the paper, I believe that a comparison against a equal-token-number PSID baseline is essential to justify the effectiveness of the proposed latent-token design.

W2. Limited practical value in industrial recommendation systems I have additional concerns regarding the practical applicability of Latte in large-scale industrial recommendation systems, particularly those operating under strict inference latency and continuous training constraints. First, the additional decoding step introduces non-negligible inference overhead. For the three-token semantic IDs used in the experiments, Latte increases the number of autoregressive decoding steps from three to four, corresponding to a 33% increase in decoding sequence length. Although the actual runtime overhead reported in Appendix E is lower, this additional cost should still be carefully justified in latency-sensitive recommendation systems. More importantly, given the same computational budget, it remains unclear whether introducing a randomly sampled latent token is preferable to increasing the semantic ID length or improving the underlying item tokenization strategy. The paper does not provide sufficient evidence that Latte offers a favorable performance-efficiency trade-off compared with these alternatives. Second, I am concerned about the training efficiency and convergence behavior of Latte. During training, the additional latent token is uniformly sampled at random and is not explicitly associated with any user-specific preference or item-specific semantic information. Consequently, the same training example may be associated with different latent tokens across training iterations. While this design may introduce useful regularization effects, it may also increase optimization difficulty or require additional training steps to achieve stable convergence. The paper reports improvements in final evaluation metrics but does not provide training loss curves, convergence comparisons, or training-time overhead relative to PSID. This omission is particularly concerning for industrial recommendation systems that rely on continuous or streaming training, where convergence speed and rapid adaptation to newly arriving interaction data are often as important as final offline ranking performance. Without a more comprehensive analysis of training efficiency and convergence stability, it is difficult to assess whether the modest offline improvements translate into meaningful practical benefits.

Questions for authors

Q1. Fair comparison under the same decoding budget. Could the authors provide a comparison between Latte (one latent token + three SID tokens) and PSID with four SID tokens? This would help clarify whether the observed improvements come from the proposed latent-token mechanism or simply from allocating an additional decoding step.

Q2. Training efficiency and convergence. How does Latte compare with PSID in terms of training time, convergence speed, and stability across different random seeds? In particular, does the uniformly random assignment of latent tokens require more optimization steps to reach comparable performance?

Q3. Performance-efficiency trade-off. Given that Latte introduces additional inference overhead, could the authors provide a comparison of NDCG@10 versus inference latency or computational cost, including a compute-matched PSID baseline? Q4. Applicability to continuous training. Have the authors investigated how Latte behaves under continuous or incremental training settings, where newly arriving user interactions need to be incorporated within limited training budgets?

Reviewer_xsUQOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates how the decoding tree induced by semantic IDs affects item probability correlations in generative recommendation. It proposes Latte, which prepends a latent token to each target SID and aggregates predictions across latent-conditioned generation paths. Experiments show consistent improvements across datasets and tokenization methods.

Critical strengths
  1. I appreciate the focus on the decoding process itself. The analysis provides an interesting perspective on how SID structure can influence recommendation behavior.

  2. The method is remarkably simple. It adds only one token to the target sequence while retaining standard next-token prediction training, making it straightforward to incorporate into existing generative recommenders.

  3. The small modification produces consistent gains across three Amazon datasets and different tokenizers. The average 3.45% relative improvement in NDCG@10 over the strongest baselines is encouraging. The extension to modality ordering also demonstrates the flexibility of the approach.

Critical weaknesses
  1. High correlation between item probabilities does not by itself imply a lack of personalized ranking. The expressiveness claims should be more carefully qualified by the assumptions used in the theoretical analysis.

  2. Uniformly random latent-token labels do not explicitly encourage different generation paths to specialize. Although the empirical gains are convincing, why these paths develop useful differences remains insufficiently explained.

Questions for authors
  1. How different are the item distributions conditioned on different latent tokens? Can the authors provide direct evidence of useful differences between these paths?

  2. Does Latte better capture user-specific rank reversals among items sharing long SID prefixes? This would more directly connect the empirical improvements to the motivation.

Reviewer_UYPnOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper observes that, in existing SID-based generative recommenders, items that are close in the decoding tree often have highly correlated generation probabilities across users. It interprets this as a limitation on the ability to represent rank reversals and intransitive item similarities. The authors propose Latte, which prepends a latent token to each SID and aggregates multiple generation paths to mitigate this coupling. They report positive results on three Amazon categories and in experiments with multimodal token permutations.

Critical strengths
  • Examining probability coupling through the decoding tree is a worthwhile perspective. Figure 2 provides a useful empirical observation about existing trained models and could motivate further analysis of shared SID structure.
  • Latte is simple and has some empirical support. The paper examines multiple tokenizers, aggregation rules, modality orderings, noise-based controls, and inference costs. I do not view lack of experimental effort as a weakness of this work.
Critical weaknesses
  • The central claim of an inherent expressiveness limit has not been established. An arbitrary leaf distribution can be represented by a chain-rule factorization based on prefix subtree masses, and a shared factor does not preclude rank reversals. The main-text interpretation of Theorem 3.4 omits the condition on mu, while Theorem 3.5 relies on an unverified, strong bidirectional threshold relationship. The theory does not support the title or central conclusion.
  • The explanation of Latte's expressive power is also incomplete. Appendix C.4 linearly mixes correlation coefficients and infers a joint dominant-path matching probability from marginal uniformity; both steps require additional conditions. The current accuracy and correlation experiments do not distinguish a structural remedy from optimization or aggregation benefits introduced by the multiple-path parameterization.
Questions for authors
  • Core theoretical clarification: Can you specify a restricted model class and provide a preference distribution that the original model cannot represent but Latte can? If not, should the claims throughout the paper be reframed in terms of an empirical inductive bias of currently trained models?
  • On controlled data with known rank reversals or intransitive relationships, does Latte actually correct the original model's fitting errors? Such evidence, together with a corrected treatment of the joint distribution and covariance in Appendix C.4, could change my assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.