acceptodds
Under review as a conference paper at ICLR 2027

Internalizing Dense Recommendation Knowledge for Generative Recommendation via Progressive Transfer

Abstract

Generative recommendation (GR) has emerged as a promising paradigm for end-to-end item retrieval, directly generating semantic identifiers (SIDs) of target items from user interaction histories. Recent studies have highlighted the value of dense recommendation (DR) knowledge for improving performance in GR. However, existing approaches primarily exploit DR knowledge for SID learning or auxiliary dense retrieval, leaving its potential for training the generative model itself underexplored. We propose **DEGET**, a progressive **De**nse-to-**Ge**nerative **T**ransfer framework that integrates DR knowledge of dense matching into GR's native encoder-decoder architecture. DEGET first jointly optimizes the semantic projector and encoder through full-catalog dense matching, enabling them to capture user preference knowledge from DR. The learned projector is then used to quantize items into SIDs. With the entire encoder and dense item representations frozen, which preserve user preference knowledge learned from the full-catalog, DEGET trains the decoder to express this dense knowledge through SID generation, thereby grounding the shared SID embeddings in DR knowledge. These SID embeddings are subsequently combined with the frozen item representations for SID-based history encoding, and joint encoder-decoder adaptation internalizes the transferred knowledge within GR. Extensive experiments on real-world datasets demonstrate substantial improvements over strong recommendation baselines. By preserving baseline SID tokenizers and applying DEGET's progressive transfer strategy, existing GR methods also achieve consistent performance gains. Our code is avaliable at https://anonymous.4open.science/r/DEGET.

Primary area:
generative models
Keywords:
Generative Recommendation, Dense Recommendation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_NFyWOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors first investigate the gap between item representations and user preferences in current generative recommendation systems. To address this, they propose a multi-stage training paradigm designed to progressively align item semantic representations with user preferences. Extensive experiments conducted across multiple recommendation datasets demonstrate the effectiveness of the proposed model. While this work builds upon previous research and offers a degree of innovation—supported by comprehensive experimental results—it has several limitations: some experimental outcomes are not particularly strong; the training pipeline is both lengthy and complex; and, unlike models such as TIGER, it cannot accommodate the continuous influx of new items, making it difficult to deploy in real-world industrial generative recommendation settings.

Critical strengths

This paper begins by addressing a long-standing challenge in the field of generative recommendation. While previous studies have attempted to tackle this issue, none have provided such a systematic analysis and investigation. The three-stage co-training framework proposed in this paper effectively aligns item semantic representations, user preference representations, and discrete SID token embeddings.

Furthermore, the paper presents a comprehensive set of experiments; ablation studies clearly demonstrate the necessity of each of the three proposed training stages, and the main experimental results outperform the vast majority of existing models.

Critical weaknesses
  1. Some of the key experimental results in this paper are not particularly impressive. According to the data in Table 2 (ablation study) and Figure 5, the performance of the full DEGET (DE+GE+T) model is nearly on par with that of the dense recommendation model (DE) trained solely in the first stage; in fact, DEGET even underperforms DE on certain metrics.

  2. The training pipeline proposed in this paper is overly complex, requiring three distinct stages—each with carefully designed training strategies and durations—which creates significant hurdles for future reproduction and practical deployment.

  3. The model struggles to handle scenarios characterized by a continuous influx of new items. The pre-trained encoder and the RQ-based quantization codebook are ill-equipped to accommodate new items. Unlike TIGER, which demonstrates genuine industrial-readiness, this model—while a suitable topic for academic research—faces substantial limitations when it comes to real-world application.

Questions for authors

Could the authors elaborate on how the concepts presented in this paper can be practically applied to industrial-scale generative coarse-ranking or retrieval systems? If this approach could be utilized to model item representations and user behaviors within large-scale, dynamic environments, it would be an outstanding contribution.

4 comments
AuthorsOpenReview ↗

Response to Weakness 1

We sincerely thank you for recognizing our contributions and raising practical concerns! We address these concerns from several practical perspectives below and welcome your further feedback.

 

[W1]

We acknowledge this limitation on the Amazon Reviews 2014 benchmarks. DEGET may slightly underperform DE while remaining close to its performance.

DR and GR differ in both architecture and recommendation paradigm, including decoder use, item representations, training objectives, and inference procedures. Internalizing DR knowledge therefore does not imply that GR must outperform DR. Our transfer strategy enables the final generative model to outperform existing GR baselines such as TIGER and ActionPiece (Table 1), while Figure 5 shows that DEGET internalizes the discriminative capabilities of DR to a substantial extent.

We further note that two main factors may explain why DEGET does not consistently outperform DE in Table 2: dataset scale and training objectives.

(a) Data scale. The Amazon Reviews 2023 benchmarks contain approximately 0.41–0.81 million interactions per dataset, compared with 0.17–0.30 million in the 2014 benchmarks (Appendix D.1, Table 5). On these larger datasets, the additional comparison below shows that DEGET exceeds DE across all metrics.

DatasetMethodsR@5R@10N@5N@10
InstrumentDE0.04110.06620.02650.0346
DEGET0.04310.06720.02750.0352
ScientificDE0.03160.05130.02030.0266
DEGET0.03340.05260.02180.0280
GameDE0.06150.09900.03880.0508
DEGET0.06480.10210.04190.0539

This pattern is consistent with the observations in S²GR (arXiv:2601.18664, KDD 2026): traditional DR methods remain competitive on its small-scale dataset, while GR methods exhibit stronger relative performance on its large-scale industrial dataset.

(b) Training objectives. As stated in Sec. 3.2, token-level likelihood training does not explicitly compare the target item against all competing items in the catalog. Recent works, including TCA (arXiv:2601.18457), GFlowGR (arXiv:2506.16114), and REGER (arXiv:2610.37584), use external DR models to provide distillation targets or reward signals for training GR models. Our staged design allows us to retain a frozen copy of the GE-stage encoder as the teacher during the T-stage, adapting such supervision into self-distillation without introducing an external teacher to bridge the training-objective gap. Specifically, we adopt REGER’s Hierarchical Collaborative Preference Distillation (HCPD) on the 2014 benchmarks.

DatasetMethodsR@5R@10N@5N@10
BeautyDE0.05610.09050.03590.0469
DEGET0.05770.09070.03770.0483
DEGET (+HCPD)0.06020.09420.03910.0500
SportsDE0.03420.05370.02240.0286
DEGET0.03440.05280.02220.0281
DEGET (+HCPD)0.03540.05540.02330.0298
ToysDE0.06020.09500.03740.0486
DEGET0.06010.09370.03780.0486
DEGET (+HCPD)0.06380.09790.04080.0517

For further details, please refer to Appendix F of the revised manuscript.

AuthorsOpenReview ↗

Response to Weaknesses 2 and 3, the Question, and Related Manuscript Revisions

[W2]

(a) We acknowledge that our training pipeline is more complex than single-stage baselines. However, the three progressive stages provide a path for converting existing dense recommenders into generative recommenders, enabling the reuse of established techniques from industrial practice and prior research:

For the DE-stage, an existing, well-trained dense recommender can serve as the foundation, avoiding the need to retrain it from scratch.

For the GE- and T-stages, the main design choices concern the training objectives for SID prediction. DEGET’s flexibility allows different training objectives established in prior work to be incorporated according to practical deployment requirements.

(b) The three proposed training stages are closely connected: DEGET first pretrains the encoder, then pretrains the decoder, and finally jointly fine-tunes the encoder–decoder. Compared with LLM-based GR pipelines such as OneRec-Think, which involve pretraining, supervised fine-tuning, and reinforcement learning, DEGET remains lightweight in model size and straightforward to implement, while achieving comparable recommendation performance on the evaluated benchmarks (Appendix E.3, Table 8).

(c) We directly compare the training efficiency of DEGET with that of TIGER and ActionPiece from Table 1 on Beauty, using the same backbone architecture. All experiments are conducted on a single NVIDIA L40 GPU, with training and evaluation batch sizes of 256 and 128, respectively. Hyperparameter settings follow Appendix D.3, Implementation Details. All models are trained to convergence. Total training time includes offline tokenization and validation during training:

MethodTotal training time (min)
DEGET73.74
TIGER89.32
ActionPiece1,971.64

Benefiting from both shorter offline tokenization enabled by reusing the frozen DE-stage projector as the RQ-VAE encoder and faster T-stage convergence enabled by initializing joint fine-tuning with the encoder pretrained in the DE-stage and the decoder pretrained in the GE-stage, DEGET achieves an end-to-end training time comparable to that of TIGER.

 

[W3]

We acknowledge that handling cold-start items remains a limitation of DEGET. Cold-start recommendation has been studied as a dedicated problem in GR, including Cold-Starts in Generative Recommendation: A Reproducibility Study (arXiv:2603.29845) and GenRecEdit (arXiv:2603.14259, SIGIR 2026). Nevertheless, DEGET offers two potential avenues for mitigating this issue in industrial applications:

(a) Reusing dense recommenders designed for cold-start scenarios. DEGET can use an existing industrial dense recommender with cold-start capabilities as its DE-stage foundation. Such a recommender could provide user query representations that generalize to the semantic features of unseen items, while also producing input representations for new items from their content.

(b) Incorporating advances in SID construction and generative recommendation. DEGET’s transfer strategy can be adapted to incorporate techniques developed for cold-start recommendation. We have already demonstrated its applicability to TIGER with RQ-VAE tokenization and ActionPiece with OPQ-based item quantization, and further extended it to RPG with parallel SID prediction (Appendix I, DEGET-RPG). This flexibility provides a basis for integrating cold-start techniques across different tokenization strategies and generation architectures, although their effectiveness within DEGET requires dedicated cold-start evaluation.

 

[Q]

Building on our responses to [W1]–[W3], we highlight three practical implications:

(a) Reusing existing dense recommenders. DEGET can reuse pretrained item representations and history encoders to build a generative retriever, avoiding dense pretraining from scratch. Appendix G.1 demonstrates this conversion using SASRec.

(b) Integrating into existing recommendation pipelines. DEGET generates candidate-item SIDs for downstream ranking without full-catalog dense scoring. For generative coarse-ranking, a possible extension is to score a supplied candidate set using SID sequence likelihoods.

(c) Scaling and adapting to dynamic catalogs. Sampled objectives provide an alternative to full-catalog dense training. Content-based encoders can represent newly arriving items, while reliable generation of their SIDs still requires incorporating cold-start adaptation techniques.

 

[Related revisions]

All newly added content is highlighted in blue in the revised manuscript.

  1. Additional analyses. We have revised Appendix F (DE vs. DEGET) to include detailed analyses addressing [W1].
  2. Practical applications. Following our discussion with you, we will add a dedicated appendix section outlining the concepts and directions for practical applications of DEGET.
AuthorsOpenReview ↗

A Gentle Follow-up on Our Responses

Dear Reviewer NFyW,

We appreciate your careful assessment of our work, your recognition of its contributions, and your constructive feedback on practical deployment.

We have carefully responded to your concerns by (1) expanding the comparison between DE and DEGET, with additional results on the larger Amazon Reviews 2023 benchmarks and experiments incorporating a self-distillation objective on the 2014 benchmarks to clarify the role of training objectives in the performance differences; (2) clarifying the flexibility of the three training stages, comparing DEGET with LLM-based GR, and reporting end-to-end training costs, including offline tokenization and validation; (3) acknowledging the remaining cold-start limitations and outlining potential adaptation strategies, including reusing dense recommenders with cold-start capabilities and incorporating advances in SID construction; and (4) elaborating on potential industrial applications, including reusing pretrained dense recommenders, integrating generative retrieval into existing pipelines, and extending DEGET to generative coarse-ranking. The additional DE–DEGET analyses have been incorporated into Appendix F, with newly added content highlighted in blue. We also plan to add a dedicated appendix section on practical applications following our discussion with you

We value your perspective on DEGET’s practical applicability and sincerely hope these clarifications and additional results strengthen your confidence in our work. We would welcome your feedback and further discussion.

Best regards,
The Authors of Submission 31765

AuthorsOpenReview ↗

A Gentle Follow-up on Our Responses — Final Day of Discussion

Dear Reviewer NFyW,

We appreciate your thoughtful review and your insights into the practical applicability of our work.

With the discussion period ending today, we would welcome your feedback on our clarifications and additional results. If these have addressed your concerns, we would be grateful if you could consider updating your assessment and score.

Best regards,
The Authors of Submission 31765

Reviewer_pCZ7OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes DEGET, a three-stage framework for transferring knowledge from dense recommendation to semantic-ID-based generative recommendation. The encode stage trains a projector and history encoder using full-catalog denser matching, and the learned item representations are quantized into semantic IDs. The express stage trains a generative decoder to predict the semantic IDs. The internalize stage jointly fine-tunes the encoder and decoder using both SID embeddings and frozen dense item representations.

Critical strengths

The paper attempts to bridge dense and generative recommendations, which is a meaningful topic. The staged framework provides a method for injecting collaborative dense signals into a generative recommender.

Critical weaknesses
  1. The results do not convincingly establish the benefit of progressive knowledge transfer. In Table 2, the full DE+GE+T model does not outperform the standalone DE stage in six of the twelve reported comparisons, yet every DE+GE+T entry is highlighted as the best result. This apparent inconsistency should be corrected or explained. More importantly, the strong performance of DE weakens the premise that transferring its knowledge into a generative model is beneficial. The paper should clarify what practical advantage DEGET offers over simply deploying the dense recommender.
  2. The full-catalog InfoNCE objective raises scalability and attribution concerns. Computing scores over the entire catalog during the DE stage may be prohibitively expensive for catalogs containing a large amount of items, limiting the method’s practical utility.
  3. The three-stage training pipeline is computationally heavyweight and is not compared under a controlled budget. DEGET requires dense pretraining, RQ-VAE training, generative warm-up, and joint fine-tuning, giving it substantially more optimization steps and computation than many baselines. The paper does not report end-to-end training cost or compare against baselines trained for equivalent compute. Consequently, the reported gains cannot be cleanly attributed to the proposed transfer mechanism rather than additional training resources.
Questions for authors
  1. The Introduction contains a grammatical error: “however, recent analyzes further reveal...” should be revised, for example, to “however, recent analyses further reveal...”
  2. What’s the architecture for the semantic projector in the DE stage? Can the author clarify how this projector with pure semantic input is able to adapt it to the semantic-collaborative space? Why not use the last layer representation from the encoder?
6 comments
AuthorsOpenReview ↗

Response to Weaknesses 1 and 2

Thank you for your careful review and insightful comments! We have addressed each concern in detail and would appreciate your time in reviewing our responses and reconsidering our work!

 

[W1]

We acknowledge the highlighting error in our submitted manuscript and will correct it. DEGET may underperform DE while remaining close to it on the Amazon Reviews 2014 benchmarks.

DR and GR differ in architecture and recommendation paradigm, including decoder use, item representations, training objectives, and inference procedures. Internalizing DR knowledge therefore does not imply that GR must outperform DR. Our transfer strategy enables the final generative model to outperform existing GR baselines such as TIGER and ActionPiece (Table 1), while Figure 5 shows that DEGET internalizes the discriminative capabilities of DR to a substantial extent.

We further note that two main factors may explain why DEGET does not consistently outperform DE in Table 2: dataset scale and training objectives.

(a) Data scale. The Amazon Reviews 2023 benchmarks contain approximately 0.41–0.81 million interactions per dataset, compared with 0.17–0.30 million in the 2014 benchmarks (Appendix D.1, Table 5). On these larger datasets, the additional comparison below shows that DEGET exceeds DE across all metrics.

DatasetMethodsR@5R@10N@5N@10
InstrumentDE0.04110.06620.02650.0346
DEGET0.04310.06720.02750.0352
ScientificDE0.03160.05130.02030.0266
DEGET0.03340.05260.02180.0280
GameDE0.06150.09900.03880.0508
DEGET0.06480.10210.04190.0539

This pattern is consistent with the observations in S²GR (arXiv:2601.18664, KDD 2026): traditional dense recommenders remain competitive on its small-scale dataset, whereas generative methods show stronger relative performance on its large-scale industrial dataset.

(b) Training objectives. As stated in Sec. 3.2, token-level likelihood training does not explicitly compare the target item against all competing catalog items. Recent works, including TCA (arXiv:2601.18457), GFlowGR (arXiv:2506.16114), and REGER (arXiv:2610.37584), use external DR models to provide distillation targets or reward signals for training GR models. Our staged design allows us to retain a frozen copy of the GE-stage encoder as the T-stage teacher, adapting such supervision into self-distillation without introducing an external teacher to bridge the training-objective gap. Specifically, we adopt REGER’s Hierarchical Collaborative Preference Distillation (HCPD) on the 2014 benchmarks.:

DatasetMethodsR@5R@10N@5N@10
BeautyDE0.05610.09050.03590.0469
DEGET0.05770.09070.03770.0483
DEGET (+HCPD)0.06020.09420.03910.0500
SportsDE0.03420.05370.02240.0286
DEGET0.03440.05280.02220.0281
DEGET (+HCPD)0.03540.05540.02330.0298
ToysDE0.06020.09500.03740.0486
DEGET0.06010.09370.03780.0486
DEGET (+HCPD)0.06380.09790.04080.0517

 

[W2]

We additionally evaluate DEGET when random negative sampling is used during DE-stage training, with 256, 512, and 1,024 negative items.

DatasetDE negative samplesR@5R@10N@5N@10
Beauty2560.05430.08670.03490.0453
5120.05530.08650.03590.0460
1,0240.05250.08450.03360.0439
Sports2560.03290.05140.02170.0277
5120.03420.05320.02210.0282
1,0240.03530.05490.02280.0292
Toys2560.05350.08710.03380.0446
5120.05480.08820.03520.0460
1,0240.05420.08790.03410.0449

Even with this simple random sampling strategy, DEGET retains its performance advantage over generative and hybrid baselines in most reported comparisons.

For further details on [W1] and [W2], please refer to Appendix F of the revised manuscript.

AuthorsOpenReview ↗

Response to Weakness 3, Questions 1 and 2, and Related Manuscript Revisions

[W3]

(a) We directly compare DEGET with TIGER and ActionPiece from Table 1 on Beauty, using the same backbone architecture. All experiments are conducted on a single NVIDIA L40 GPU, with training and evaluation batch sizes of 256 and 128, respectively. Hyperparameter settings follow Appendix D.3, Implementation Details. All models are trained to convergence, and we report the computational costs of individual pipeline components and the total training time.

The peak GPU memory usage and training costs are reported below. Inference time is measured on the validation set and averaged over epochs with validation.

Method / StagePeak GPU memory (MB)Training time (s/epoch)Inference time (s/epoch)
DEGET: DE-stage8,7388.731.04
DEGET: GE-stage1,10211.916.10
DEGET: T-stage4,48018.4916.60
TIGER4,41018.1516.08
ActionPiece42,36873.01513.04

The offline tokenization times are reported below:

MethodOffline tokenizerTime (min)
TIGERRQ-VAE41.67
ActionPieceOPQ + BPE25.73
DEGETRQ-VAE24.13

We further report the total training time to convergence, including offline tokenization and validation performed during training:

MethodTotal training time (min)
TIGER89.32
ActionPiece1,971.64
DEGET73.74

Benefiting from both shorter offline tokenization enabled by reusing the frozen DE-stage projector as the RQ-VAE encoder and faster T-stage convergence enabled by initializing joint fine-tuning with the encoder pretrained in the DE-stage and the decoder pretrained in the GE-stage, DEGET achieves an end-to-end training time comparable to that of TIGER.

(b) The three proposed training stages are closely connected: DEGET first pretrains the encoder, then pretrains the decoder, and finally jointly fine-tunes the encoder–decoder. Compared with LLM-based GR pipelines such as OneRec-Think, which involve pretraining, supervised fine-tuning, and reinforcement learning, DEGET remains lightweight in model size and straightforward to implement, while achieving comparable recommendation performance on the evaluated benchmarks (Appendix E.3, Table 8).

 

[Q1]

We apologize for the grammatical errors in our submitted manuscript. We have thoroughly proofread the manuscript and corrected these errors throughout.

 

[Q2]

(a) The semantic projector is a five-layer MLP, reused after DE-stage training as the frozen RQ-VAE encoder described in Appendix D.3.

(b) Collaborative knowledge is incorporated into the MLP through backpropagation from the recommendation objective. Although the MLP takes semantic features as input, the full-catalog user–item matching objective in Eq. (5) uses observed interactions to jointly update the MLP and history encoder through backpropagation, aligning item representations with user preferences.

(c) In DR, the encoder’s final-layer hidden state at the prediction position serves as a user preference representation, as in SASRec and BERT4Rec. This representation depends on the user’s interaction history and serves as a personalized user query vector in GR rather than an individual item representation. However, both SID construction and the GR encoder’s inputs require item-level representations.

 

[Related revisions]

All newly added content is highlighted in blue in the revised manuscript.

  1. Additional analyses. We have added detailed analyses addressing [W1] and [W2] to Appendix F (DE vs. DEGET).
  2. Training efficiency. We have added training efficiency comparisons to Appendix H (Efficiency Analysis).
  3. Language corrections. We have corrected grammatical errors throughout the manuscript.
AuthorsOpenReview ↗

Clarifying the Substantive Contributions of DEGET

We have carefully considered your initial ratings, comments, and concerns and would like to further clarify the distinctive contributions of our work. We hope these clarifications will help you reassess its value.

 

DR models have evolved through years of industrial development, establishing mature modeling techniques and deployment practices. As a promising generative paradigm with different architectures, GR can benefit from this accumulated knowledge and experience. As Reviewer NFyW noted, although prior studies have explored this direction, they have not provided the same systematic investigation and analysis as DEGET.

 

We would therefore like to highlight the following contributions:

(a) A transferable framework across GR formulations. DEGET internalizes DR knowledge without an external recommendation teacher or an auxiliary dense retrieval module. This design supports autoregressive GR with both residual-quantization-based SIDs (TIGER) and parallel-quantization-based item codes (ActionPiece, using OPQ), and further extends to decoder-only GR with parallel SID prediction, as demonstrated with RPG (Appendix I, DEGET-RPG). These results demonstrate its applicability across quantization schemes, model architectures, and prediction paradigms.

The prior methods or their individual components do not offer this same breadth of transferability. They typically improve a specific tokenizer or tokenization process, or a particular architecture or inference paradigm, making them difficult to apply directly when these choices change.

 

(b) A pathway from pretrained dense recommenders to generative recommenders. DEGET enables an existing dense recommender to be adapted for SID generation by reusing its learned representations and backbone and introducing a generative decoder. Our SASRec case study in Appendix G illustrates this pathway, suggesting practical value for industrial systems seeking to reuse existing recommendation models when adopting generative retrieval.

 

(c) A pretraining approach for non-LLM-based GR. LLM-based GR aligns SID embeddings with pretrained language representations to access textual knowledge, whereas non-LLM-based GR models are trained from scratch and lack such a pretrained representation space. DEGET instead grounds SID embeddings by learning to express the DR knowledge acquired during dense pretraining, conditioning SID generation on the learned user preference representations.

From the perspective of model learning, DEGET involves pretraining and post-training: we first pretrain the encoder, then train the decoder, and finally jointly train the entire generative model. By connecting the two languages of item IDs and SIDs within a single model, we provide a novel pretraining approach for generative models based purely on the languages of recommendation.

 

More generally, DEGET is not merely a method but an extensible and efficient framework. As demonstrated in our response to [W1], DEGET can adapt training signals from DR to help GR further internalize pretrained recommendation knowledge, without introducing an external model and while preserving our original motivation. Its SID prediction objectives can therefore incorporate techniques from existing and future work. Our responses to [W2] and [W3] further address deployment and computational concerns. In particular, the DE-stage can incorporate established industrial negative sampling techniques to refine its training.

Reviewer_pgeEOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes DEGET to transfer dense recommendation knowledge into generative recommendation. It first trains item representations and an encoder through dense matching, then trains a decoder with the encoder frozen, and finally jointly adapts the model using SID inputs enriched with dense representations. The method is motivated by three representation gaps. Experiments show consistent improvements across six datasets, including gains when keeping existing SID tokenizers fixed.

Critical strengths
  • Reusing the model's own encoder for dense pretraining is practical. The final model generates recommendations without an additional dense scoring branch.
  • The results are fairly strong. Improvements with fixed tokenizers and different backbones support the usefulness of the overall framework beyond better SID construction.
Critical weaknesses
  • The three gaps are not convincingly established as actual bottlenecks. The evidence mainly shows that the proposed components help, rather than showing that existing models suffer from the specific problems claimed.
  • The methodological contribution seems incremental. Adding dense representations to SID inputs and transferring collaborative knowledge into generative models both have close precedents. The additional insight from this particular combination remains unclear.
  • The final transition to SID-based histories is not well motivated. The model can already generate SIDs from dense histories, but this simpler design is not evaluated with joint fine-tuning.
Questions for authors
  1. I'm not convinced that the three gaps have been established independently of the proposed stages. Figure 3 shows that collaborative supervision changes the item representations, but does not directly establish the claimed disconnect with user preferences. Better results after dense pretraining support its usefulness, but do not explain why a separate dense–generative capability gap is needed. The SID grounding argument is particularly weak. Random initialization does not mean the embeddings remain poorly grounded after training. Figure 4 also does not show this, since adding dense vectors naturally changes the input distribution. What evidence shows that these are persistent limitations in trained GR models? As presented, the gaps read more like explanations of the three stages than a diagnosis that calls for them.

  2. The dense addition itself is already quite familiar. "Unifying Generative and Dense Retrieval for Sequential Recommendation" adds the same item's dense text representation to its SID token embeddings. "UniGRec: Unified Generative Recommendation with Soft Identifiers for End-to-End Optimization" transfers collaborative knowledge into both the tokenizer and the generative recommender. The former is compared here, while the latter is cited but not evaluated. DEGET differs in its recommendation-trained representations and staged reuse of the encoder. That can be useful, but what is the substantive advance beyond this training recipe? Calling the steps grounding and internalization does not make that advance clear.

  3. Why switch the history to SID tokens at all? After GE, the model already generates SIDs from dense item inputs. T expands those inputs into SID tokens and then adds the dense vectors back. Why is this preferable to retaining dense histories and jointly fine-tuning the encoder and decoder? DE+GE in Table 2 keeps the encoder frozen, so it does not resolve this design question.

7 comments
AuthorsOpenReview ↗

Response to Weakness 1 and Question 1

Thank you for your careful and thorough review and constructive questions!

We hope you will carefully consider our clarifications!

 

[W1,Q1]

1. Clarifying the existence of the three gaps and the motivation

The concerns underlying the three gaps have been recognized in prior studies independently of our proposed method. Our objective is to connect and bridge these gaps to progressively internalize DR knowledge within the generative model itself. Drawing on DR, we formulate these gaps in GR from the perspective of user and item representations, which are central to recommendation, thereby establishing a representation-level connection between DR and GR. We summarize the relevant prior evidence for each gap below:

(a) Semantic–Collaborative Representation Gap. LETTER [1] directly investigates whether trained semantic tokenizers preserve collaborative relationships, while ETEGRec [2] explicitly studies the alignment between user preference representations and target-item semantics, noting that the decoder may complete the generation task with limited reliance on encoder information, potentially leaving the encoder insufficiently trained.

(b) Dense–Generative Capability Gap. MemGen-GR [3] empirically demonstrates the complementary capabilities of DR and GR, while UniRec [4] identifies an expressiveness gap: conventional SID-based GR decodes over compact semantic tokens without direct access to target-item features, limiting the explicit user–item feature interactions available to discriminative models.

(c) SID Grounding Gap. Related concerns have been recognized in both LLM-based GR [5, 6] and non-LLM-based GR. In non-LLM-based GR, DECOR [7] explicitly identifies the problem of discarded pretrained semantics: although semantic IDs are constructed from pretrained item representations, their downstream token embeddings are separately initialized and learned from interaction data without directly inheriting those representations. Similarly, PRISM [8] identifies a disconnect between lightweight SID-based generation and the continuous item feature space, noting that relying solely on discrete SIDs loses fine-grained semantic information.

DEGET also outperforms the compared methods on the corresponding datasets, based on either our reproductions or the results reported in their original papers.

 

2. Clarifying our experimental analyses

In addition to the evidence from prior studies discussed above, the analyses in our original manuscript provide further empirical support for the existence of these gaps:

(a) For the Semantic–Collaborative Representation Gap, Figure 3 shows that collaborative supervision reshapes item representations, while Figure 6 directly examines user–item matching in the fully trained T-stage encoder, indicating a disconnect between SID-only history representations and the user preferences learned through dense recommendation.

(b) For the Dense–Generative Capability Gap, dense pretraining leaves the decoder and SID token embeddings randomly initialized, so the learned DR knowledge still needs to be transferred to SID generation. Figure 4 shows that the projected SID embedding distributions remain broadly similar from GE to T, consistent with GE establishing a representational basis retained during subsequent adaptation. The DE+T ablation in Table 2 provides complementary functional evidence.

(c) For the SID Grounding Gap, DEGET addresses grounding at both the item level and SID level. Table 3 directly shows the contribution of item representations to grounding. Figure 7 further analyzes SID prediction in the GE- and T-stages. Together with the similar projected distributions of SID embeddings in GE and T (Figure 4), these results suggest that SID-level grounding in GE provides a consistent representational basis for T to refine fine-grained SID predictions.

We emphasize that the primary purpose of Sec. 4.4 and Appendix F in the original manuscript (which contains Figures 6 and 7) is to provide an understanding of our proposed transfer.

References

[1] Learnable Item Tokenization for Generative Recommendation. CIKM 2024.

[2] Generative Recommender with End-to-End Learnable Item Tokenization. SIGIR 2025.

[3] On the Memorization and Generalization of Generative Recommendation. KDD 2026.

[4] UniRec: Bridging the Expressive Gap between Generative and Discriminative Recommendation via Chain-of-Attribute. RecSys 2026, Industrial Paper.

[5] Fine-grained Semantics Integration for Large Language Model-based Recommendation. arXiv:2602.22632.

[6] Implicit Reasoning for Large Language Model-based Generative Recommendation. arXiv:2606.14142.

[7] Learning Decomposed Contextual Token Representations from Pretrained and Collaborative Signals for Generative Recommendation. SIGIR 2026.

[8] PRISM: Purified Representation and Integrated Semantic Modeling for Generative Sequential Recommendation. arXiv:2601.16556.

AuthorsOpenReview ↗

Response to Weakness 2 and Question 2

[W2,Q2]

1. Key distinction from prior work

(a) Compared with LIGER, DEGET transfers dense recommendation knowledge through semantic–collaborative item representations learned via user–item matching and performs purely generative retrieval, whereas LIGER integrates dense semantic retrieval capabilities using fixed text embeddings and dense reranking. Compared with UniGRec, which distills an external SASRec teacher into both the tokenizer and generative recommender, DEGET learns DR knowledge through its own item projector and history encoder. We additionally report UniGRec reproduction results below, obtained under our data settings using its official code with the official hyperparameters and configurations:

DatasetMethodsR@5R@10N@5N@10
BeautyUniGRec0.0493 ± 0.160.0738 ± 0.210.0333 ± 0.150.0412 ± 0.16
DEGET0.0577 ± 0.060.0907 ± 0.130.0377 ± 0.040.0483 ± 0.01
SportsUniGRec0.0302 ± 0.060.0454 ± 0.090.0199 ± 0.030.0247 ± 0.04
DEGET0.0344 ± 0.070.0528 ± 0.110.0222 ± 0.030.0281 ± 0.03
ToysUniGRec0.0450 ± 0.160.0696 ± 0.200.0300 ± 0.110.0379 ± 0.12
DEGET0.0601 ± 0.080.0937 ± 0.020.0378 ± 0.040.0486 ± 0.01

Values are reported as mean ± (100 × sample standard deviation).

(b) These related approaches extend the standard GR framework based on RQ-VAE tokenization and an encoder–decoder architecture (TIGER) by adding retrieval components or introducing a dense recommender. However, DEGET builds on the original components of the encoder–decoder architecture and RQ-VAE to unlock the intrinsic potential of GR by learning and internalizing DR knowledge within the model itself. More importantly, DEGET is not merely a specific method but a transferable training framework.

 

2. Substantive contributions of DEGET

(a) A transferable framework across GR formulations. DEGET internalizes DR knowledge without an external recommendation teacher or an auxiliary dense retrieval module. This design supports autoregressive GR with both residual-quantization-based SIDs (TIGER) and parallel-quantization-based item codes (ActionPiece, using OPQ), and further extends to decoder-only GR with parallel SID prediction, as demonstrated with RPG (Appendix I, DEGET-RPG). These results demonstrate its applicability across quantization schemes, model architectures, and prediction paradigms.

The prior methods or their individual components do not offer this same breadth of transferability. They typically improve a specific tokenizer or tokenization process, or a particular architecture or inference paradigm, making them difficult to apply directly when these choices change.

 

(b) A pathway from pretrained dense recommenders to generative recommenders. DEGET enables an existing dense recommender to be adapted for SID generation by reusing its learned representations and backbone and introducing a generative decoder. Our SASRec case study in Appendix G illustrates this pathway, suggesting practical value for industrial systems seeking to reuse existing recommendation models when adopting generative retrieval.

 

(c) A pretraining approach for non-LLM-based GR. LLM-based GR aligns SID embeddings with pretrained language representations to access textual knowledge, whereas non-LLM-based GR models are trained from scratch and lack such a pretrained representation space. DEGET instead grounds SID embeddings by learning to express the DR knowledge acquired during dense pretraining, conditioning SID generation on the learned user preference representations.

From the perspective of model learning, DEGET involves pretraining and post-training: we first pretrain the encoder, then train the decoder, and finally jointly train the entire generative model. By connecting the two languages of item IDs and SIDs within a single model, we provide a novel pretraining approach for generative models based purely on the languages of recommendation.

 

More generally, DEGET is not merely a method but an extensible and efficient framework. As demonstrated in our response to Reviewer pCZ7, these properties are further reflected in two aspects: First, DEGET can adapt training signals from DR to help GR further internalize pretrained recommendation knowledge without introducing an external model, preserving our original motivation. Its SID prediction objectives can therefore incorporate techniques from existing and future work. Second, our analyses address deployment and computational concerns, demonstrating that the DE-stage can incorporate established industrial negative sampling techniques to refine its training.

AuthorsOpenReview ↗

Response to Weakness 3 and Question 3, and Related Manuscript Revisions

[W3,Q3]

(a) In the T-stage, SID tokens provide shared hierarchical codes for modeling cross-item patterns, enabling the encoder to reuse the SID embeddings pretrained during GE and capture fine-grained relationships across items. Dense representations provide item-level semantic–collaborative information, helping preserve dense recommendation knowledge and the encoder’s ability to model personalized user preferences. We have already explained this complementary design from both methodological and theoretical perspectives in Sec. 3.2 (T-stage: Internalize) and Appendix C.2.

(b) We agree that DE+GE in Table 2 does not resolve this comparison. We therefore conducted two additional ablations, both T-stage variants that modify the encoder inputs:

  • Variant 1 — Dense histories: Retain the GE-stage encoder inputs, with one semantic–collaborative dense embedding per historical item.
  • Variant 2 — Repeated dense histories: Repeat each item’s dense embedding L times, where L is the SID length.
DatasetMethodsR@5R@10N@5N@10
BeautyVariant 10.0541 ± 0.090.0869 ± 0.130.0349 ± 0.030.0454 ± 0.03
Variant 20.0525 ± 0.120.0828 ± 0.070.0335 ± 0.080.0433 ± 0.06
DEGET0.0577 ± 0.060.0907 ± 0.130.0377 ± 0.040.0483 ± 0.01
SportsVariant 10.0336 ± 0.020.0526 ± 0.060.0218 ± 0.040.0281 ± 0.03
Variant 20.0322 ± 0.060.0512 ± 0.090.0209 ± 0.040.0270 ± 0.05
DEGET0.0344 ± 0.070.0528 ± 0.110.0222 ± 0.030.0281 ± 0.03
ToysVariant 10.0551 ± 0.090.0873 ± 0.230.0346 ± 0.050.0450 ± 0.09
Variant 20.0540 ± 0.100.0859 ± 0.150.0337 ± 0.070.0440 ± 0.08
DEGET0.0601 ± 0.080.0937 ± 0.020.0378 ± 0.040.0486 ± 0.01

Values are reported as mean ± (100 × sample standard deviation).

 

[Related revisions]

All newly added content is highlighted in blue in the revised manuscript.

  1. Additional baselines. We have added UniGRec and DECOR as baselines.
  2. Additional ablations. We have incorporated the additional ablations suggested in [Q3, W3] into the main text (Sec. 4.4, Table 4).
  3. Expanded main-text analyses. With the main-text allowance extended to 10 pages, we have moved the analyses from Appendix F of the original manuscript into the main text.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.