acceptodds
Under review as a conference paper at ICLR 2027

Scale-aware Identifiers for Generative Recommendation

Abstract

Generative recommendation predicts the next item by autoregressively generating its semantic identifier (SID), making SID construction critical to recommendation performance. Existing hierarchical SID methods commonly use residual quantization (RQ), where each level selects a codeword and directly subtracts it from the residual. This update ignores the projection magnitude along the selected codeword and can produce non-orthogonal, non-minimum-norm residuals, leading to redundant decomposition across levels. Therefore, we propose Scale-aware Residual Quantization (SRQ), which computes projection magnitudes and subtracts the selected codeword with its corresponding scale, yielding an orthogonal, minimum-norm residual. To use the continuous magnitudes for autoregressive generation, we discretize them into quantile-based magnitude tokens. Together with codeword tokens, they form Scale-aware Identifiers for Generative Recommendation (SIGER). Moreover, SIGER can be integrated into two-stage training, unified tokenizer–recommender training, and LLM-based post-training. Analysis shows that SIGER improves residual orthogonality and codebook utilization, reducing SID collision from 5.26% to 2.99% with codeword tokens alone and to 0.04% with 128 magnitude bin size. These identifier-level improvements translate into stronger recommendation performance: SIGER improves two-stage baselines by up to 9.6% Recall@5 and 9.9% NDCG@5, improves over unified baselines by up to 2.4% NDCG@5 on Beauty and 8.0% Recall@10 on Pet, and achieves the best performance on the Office in the LLM-based setting. Further ablations on 2-level and code-only variants show that SIGER encodes semantic information more compactly and demonstrate the effectiveness of scale-aware decomposition.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Generative recommendation, semantic identifier, scale-aware residual quantization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_gqgEOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Scale-aware SIDs. In traditional RQ-VAE methods, after selecting a codeword at each layer, the residual and the codeword are not orthogonal, which leads to residual semantic information. This paper uses a geometric approach to ensure that the residual remains orthogonal to the selected codeword. Meanwhile, the residual achieves a minimum L2 norm, allowing different codebooks to fully explore semantic information, thereby improving codebook utilization. The reviewer speculates that the authors might have been inspired by Gram-Schmidt orthogonalization and transferred this concept to semantic vector orthogonality.

Critical strengths
  1. The mathematical proof is concise and very intuitive from a geometric perspective. The algorithm is also easy to implement and plug-and-play, which is a highly important advantage.

  2. The empirical validation across three recommender training paradigms demonstrates the algorithm's effectiveness. Furthermore, the detailed analysis of codebook utilization and collision rates is another significant advantage.

Critical weaknesses
  • The experimental results deserve further analysis.

  • More diverse backbones and evaluation datasets should be evaluated to further validate the effectiveness.

  • More related work on generative recommendation should be discussed in this work.

Questions for authors

I compared the results on the Beauty dataset in Table 2 of this paper with Table 1 in the original LETTER paper, both of which use LETTER and SASRec as baselines. I reviewed the original LETTER paper and noticed that while your reproduced results for LETTER are comparable to the original paper, there is a significant discrepancy regarding the SASRec baseline. In your paper, SASRec achieves a Recall@5 of only 0.0284, which is even lower than BERT4Rec (0.0322). However, in the original LETTER paper, SASRec achieves a Recall@5 of 0.0380 (significantly outperforming BERT4Rec). Could you please explain the reason for this discrepancy?

Reviewer_wYZ4OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper proposes SRQ. In the tokenization construction process of semantic ID-based generative recommendation methods, people used to subtract item representations with centroid representations. The authors argue that we should not simply choose a centroid by distance, but should find a centroid sharing most component in its direction (calculated in their projection space).

Practically, the authors bucketize the scales as additional tokens in each item's semantic ID. Extensive experiments conducted on public datasets and across three backbone models demonstrate the performance of the proposed method.

Critical strengths
  1. The overall idea is interesting, which is a geometry-perspective method in constructing semantic ID tokenizer.
Critical weaknesses
  1. The prior that leads to the key motivation doesn't hold. In "Limitation from a decomposition perspective", a key motivation is that "At each level, RQ selects a codeword direction but subtracts it with a fixed unit coefficient. This implicitly assumes that the current residual vl contains exactly one unit of the selected codeword". However, in the most widely used tokenization methods RQ-VAE and RQ Kmeans, people don't take the codebook themselves as basis of some space, aka there is no restrictions to make the codebook centroids either having length 1, or to be orthogonal with each other. So if the prior "codebook centroids are basis (length 1, orthogonal)" doesn't hold, the motivation (subtracting codebook centroids is wrong) does not make sense anymore.
  2. In Figure 2, the authors claimed that SRQ results in a much lower SID collision rate. However, SRQ has ~2 times more tokens than the compared methods. It's natively easier for a method with more tokens to have a lower collision rate. As a result, the results don't say the improvements come from the projection idea itself.
  3. The authors argue that doubling the length of SIDs in SRQ doesn't affect efficiency a lot (and report training time in Table 3). However, what really matters is inference latency. SID-based generative recommendation requires autoregressive generation. When we have 2 times of tokens to decode, the inference latency just increases significantly, which is very important when considering the practical value of a recommendation method.
Questions for authors

N/A

Reviewer_BCBeOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how residual quantization handles the magnitude of an item's component along a codeword direction when constructing semantic identifiers for recommendation. SIGER selects a codeword and its projection coefficient at each level, then discretizes the coefficient into a magnitude token that is generated alongside the codeword tokens. The authors evaluate the approach under two-stage training, unified training, and LLM post-training, and use code-only and reduced-depth variants to examine where the gains come from.

Critical strengths
  • I find the same-length code-only ablation particularly persuasive. Table 1 shows that changing the decomposition alone can improve over RQ-VAE, so the gains cannot be attributed entirely to additional magnitude tokens. This gives the method's motivation substantive empirical support.
  • The study is reasonably complete. It examines codebook utilization, collisions, recommendation accuracy, and training cost, and covers multiple training paradigms and a KMeans extension. Together, these results support a well-scoped contribution to item tokenization.
Critical weaknesses
  • The novelty framing and geometric conclusions need tighter qualification. The projection update is closely related to established greedy projection methods, and optimality of a single continuous residual update does not establish that the resulting discrete SIDs are better at a comparable coding budget. The paper should identify more precisely what new knowledge it contributes.
  • The claims about compactness and the benefits of magnitude tokens do not fully reflect the results. Two-level SIGER still uses more total tokens than three-level RQ-VAE, and some of the strongest results come from code-only variants. The small differences under unified training also lack variability estimates. These issues limit my confidence in how broadly the claimed benefits hold.
Questions for authors
  • Relative to matching pursuit and gain–shape coding, what are the new technical elements here, and what challenges are specific to recommendation?
  • Does SIGER retain its advantage at a comparable total token or coding budget? Please also report run-to-run variability for the key paired comparisons and explain when magnitude tokens outperform code-only variants. These questions bear directly on compactness and the need for magnitude tokens; they do not require extending the paper to a new application.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.