- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes RF-SID for generative recommendation. It assigns separate Semantic ID coordinates to content and behavioral representations and uses a linear predictor to remove content-related information from the behavioral component. Experiments on three Amazon datasets report improvements over existing SID construction methods under sampled candidate ranking. The paper also presents a simplified rate-distortion analysis to motivate the proposed factorization.
Critical strengths
The separation of content and behavioral information in the semantic ID construction presented in this paper is intuitively sound, and the proposed method is easy to implement.
Critical weaknesses
W1. Separating semantic and collaborative signals through different codebooks, token groups, or generation streams has already been extensively explored, such as GenSAR , FlexCode, EAGER and DiscRec. The proposed method largely reduces to separately quantizing two representations, applying a standard linear residualization step, and concatenating their codes. The paper neither adequately distinguishes itself from these closely related approaches nor provides direct comparisons with the most relevant alternatives. It is difficult to identify a substantial new technical contribution.
W2: The evaluation protocol does not establish the claimed generative recommendation performance. All main results are based on ranking one positive item against 1000 sampled negatives, despite catalogs containing more than 150,000 items as in Table 1. So the reported superiority over existing methods may not hold under realistic retrieval.
W3: A more fundamental concern is that complete SID uniqueness is neither established nor evaluated. Different items sharing the same complete SID necessarily receive identical scores, yet the paper reports no collision statistics.
W4: Finally, the experimental evidence is limited to one training seed (line 658-667).
Questions for authors
see the weaknesses.
2 comments
Thank you to the authors for their rebuttal. The response has partially alleviated my concerns about W2, W3, and W4, but it has not fully addressed my concerns about the novelty of W1. The distinction between RF-SID and existing methods still requires more thorough justification.
Furthermore, the current revision still uses 1000 negative samples and adds neither new comparisons nor collision results, so I do not consider these issues formally resolved. In addition, the latest full-catalog ranking results in the rebuttal differ substantially from the performance reported in the original submitted paper, especially in the comparisons between methods, making them highly misleading.
Therefore, I still believe maintaining my score is appropriate.
Response to Reviewer Q5cj
We thank the reviewer for the feedback. We conducted additional comparisons, full-catalog evaluations, collision analyses, and multi-seed experiments to address all four concerns.
W1: Novelty and Related Work
RF-SID differs from existing methods in how content and behavioral signals are organized within a unified, fixed-budget SID. Unlike GenSAR's shared quantization, EAGER's two-stream generation, FlexCode's dynamic token allocation, and DiscRec's gated embedding fusion, RF-SID uses content-first coordinates and content-adjusted behavioral residuals within a single autoregressive generation stream.
We additionally evaluated GenSAR, EAGER, and FlexCode on all three datasets (W2). RF-SID consistently outperforms these methods under full-catalog evaluation. Removing content adjustment also reduces NDCG@10 from 0.008998/0.009508/0.008240 to 0.008653/0.009345/0.008132 on Beauty/Sports/Toys.
Notably, FlexCode exhibited severe code collapse in our reproduction, with complete-SID collision rates of 44.15%/44.57%/36.10%. Its token allocation concentrated on zero or one behavioral token, and adjustments to pretraining coverage and learning rate did not resolve the issue. Since FlexCode's code is not publicly available, these findings raise substantial concerns about its reproducibility and robustness.
W2: Full-Catalog Evaluation
We conducted exhaustive full-catalog ranking over 207,649/156,235/162,035 items on Beauty/Sports/Toys, using the same 50,000 test contexts across methods, without sampled negatives or beam-search truncation.
Full-catalog Recall@10 / NDCG@10:
RF-SID achieves the highest NDCG@10 on Sports and Toys and outperforms all compared SID-based methods except CCFRec on Beauty. It consistently outperforms GenSAR, EAGER, and FlexCode across all three datasets, confirming that its advantages persist under exhaustive full-catalog ranking.
W3: Complete-SID Collisions
We measured complete-SID collisions over the entire catalog before collision resolution, using CollisionRate=100(1−U/N)%, where U is the number of unique complete SIDs.
RF-SID achieves approximately 99.85%–99.90% unique complete SIDs without any collision-resolution procedure, with at most three items sharing one SID.
During evaluation, all items remain separate ranking candidates, including those sharing a SID. Tied scores are handled consistently across methods using the average rank r=G+(E+1)/2, where G and E count candidates with greater and equal scores, respectively. Thus, the reported results do not benefit from item deduplication or favorable tie-breaking.
W4: Multi-Seed Stability
We completed three independent runs (seeds 2026, 2027, 2028), each covering tokenizer training, AR training, and full-catalog ranking over 50,000 test contexts.
Full-catalog results (mean ± sample standard deviation):
The NDCG@10 standard deviations are only 3.84%/2.56%/3.32% of the corresponding means, demonstrating stable performance across independent training runs.