- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper asks whether Semantic IDs should serve as both shared item representations and autoregressive prediction targets. It shows that an unrestricted legal SID tree can express any positive catalog distribution and contrasts the supervision applied to branch logits and complete-item scores. Based on this analysis, NRP builds item vectors from shared code embeddings and trains them with full-catalog item cross-entropy.
Critical strengths
The analysis separates distributional expressivity from the organization of supervision, which offers a useful perspective on SID-based recommendation.
Critical weaknesses
W1: The method is close to VQ-Rec and especially DiGR, which already combine SID-based item representations with item-level scoring. The paper discusses this overlap, but its distinction from DiGR is not sufficiently sharp, and no direct comparison is provided.
W2: The gradient analysis does not establish that the supervision structure is the main reason for the performance gap. The output heads and optimization paths still differ. Some main-table baselines also use published results from different pipelines, and the larger-catalog experiment reports only one seed.
Questions for authors
Q1: How does NRP compare with DiGR under matched SID construction, backbone, training budget and exact ranking? Which gains remain attributable to the proposed contribution?
Q2: Please provide more direct evidence to prove that the observed performance improvements are caused by the supervision structure rather than differences in the prediction heads or the optimization process.
4 comments
Dear Reviewer AZT7,
With the discussion period ending in about two days, we wanted to briefly follow up and check whether our response and additional experiments have addressed your questions.
To directly resolve your concerns in W1/Q1 and W2/Q2, we have provided:
Matched 5-seed comparisons with DiGR and DiGR (GPT-2) alongside independent 3x3 learning-rate/temperature tuning across all five composers, isolating the substantial gain of learned nonlinear composition over simple pooling.
A fixed-architecture training-objective intervention (comparing standard NTP with catalog-normalized sequence training under the exact same AR head) and independent 5-point learning-rate sweeps for Table 2(d), directly demonstrating the benefit of catalog-level supervision (+15.6% / +30.6% NDCG@10) independent of prediction-head and optimization differences.
Unified 5-seed local reproductions for all baselines in Table 1 and 3-seed evaluations on the 64K-item CDs and Vinyl catalog.
We would be very grateful to hear if these results address your concerns, and we remain happy to clarify any further questions before the deadline. Thank you for your time and consideration!
Thank you for the authors’ response. The rebuttal addressed some of my concerns, and I will raise my rating.
We sincerely thank you for your time and for raising your rating. We are glad our response was helpful, and we will ensure that the matched DiGR comparisons, the fixed-architecture catalog-normalization intervention, and the multi-seed evaluations are all thoroughly incorporated into the final revision.
Response to Reviewer AZT7
Thank you for the constructive review and for recognizing the value of separating distributional expressivity from supervision organization. Below are our new results and planned revisions.
Q1. Comparison with DiGR and our contribution. We now report 5-seed reproductions of original DiGR and DiGR (GPT-2) (updating Table 1 and Table 6 in Appendix B.3). DiGR (GPT-2) shares NRP's GPT-2 architecture, fixed RQ-KMeans SIDs, exact full-catalog testing, and history filtering. NRP retains its original fixed configuration (20-250 epochs), while DiGR (GPT-2) independently searches 3 learning rates and 2 temperatures, with early-stopping patience 10 and no fixed epoch cap. NRP outperforms both DiGR versions on all reported metrics across all three datasets (e.g., Beauty 5-seed Recall@10 is 0.0928 for NRP vs. 0.0635 for DiGR, 0.0496 for DiGR (GPT-2), and 0.0798 for VQ-Rec (GPT-2)).
To examine the contribution of composition, we independently tuned all five composers (Table 22, Appendix D.4) over the same 3-by-3 learning-rate/temperature grid with catalog centering disabled. The strongest simple aggregator, Sum, achieves Beauty/Toys Recall@10 of 0.0508 / 0.0509, vs. 0.0944 / 0.0970 for the residual MLP (D=6). The residual MLP remains substantially stronger after independent tuning with centering disabled, supporting this learnable composition design.
These results provide both end-to-end comparisons and evidence on the composition module. Relative to existing SID-based item scoring (Section 2 / Appendix B.4), our main contribution is systematic theoretical analysis and progressively controlled experiments supporting SID sharing with direct optimization of complete-item scores. We establish that a legal SID tree with unrestricted node conditionals has no inherent catalog-level expressivity limitation, and show that direct scoring remains stronger when we normalize over legal branches, progressively align history representations and model designs, and rank the full catalog exactly.
Q2. Direct evidence, optimization controls, and result reliability (W2). We added two controls addressing scoring-head differences and optimization, plus multi-seed evaluations for W2:
Our gradient analysis characterizes direct supervision at branch-logit and complete-item-score coordinates, and a single-step diagnostic (Appendix C) shows shared-parameter updates alter relative probabilities within non-target subtrees (0.0645 / 0.0669 on Beauty/Toys) as collateral drift. This fixed-architecture intervention directly links the catalog-level objective to improved ranking.