- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies positional encoding in Semantic ID based generative recommendation. The authors argue that flattening multi-code items into a single sequence mixes interaction positions with residual code positions. They propose SimPos, which models these two positional relations using separate linear attention biases without introducing additional learnable parameters.
Critical strengths
The proposed method is simple and easy to integrate into existing generative recommendation models without introducing additional learnable parameters.
Critical weaknesses
W1: The idea of modeling item structure and separating positional information is not entirely new. DiscRec has already introduced within-item positional embeddings for Semantic ID recommendation, while BARGE also studies the structural issues caused by flattening item tokens. The paper does not sufficiently distinguish its contribution from these related methods.
W2: The motivation and experimental evidence are not fully aligned. Flattening fixed-length Semantic IDs does not inherently lose positional information, but changes how positional relations are represented. Moreover, the proposed interaction bias introduces a strong recency preference. It remains unclear how much of the improvement comes from separating the two positional axes rather than simply imposing a better recency bias. The assumption that nearby residual levels should receive stronger attention also deserves further justification.
Questions for authors
See weakness.
7 comments
Response to Reviewer bZK4 (2/4)
W2a. Structural inductive bias rather than information loss.
The advantage of SimPos lies in its structural inductive bias, rather than in recovering information lost by flattening. For fixed-length SIDs with codes, a flattened absolute position determines both coordinates, (interaction position) and (residual level), so LPE retains full positional information in principle. However, SimPos outperforms LPE across all 32 setting–metric combinations (Table 3), indicating that preserving positional information alone is insufficient to achieve the same performance.
Since flattening changes how positional relationships are represented rather than what information is available, we further examine whether explicitly encoding both positional coordinates is sufficient. We introduce a factorized absolute PE baseline that encodes through two learnable absolute embeddings, , added to the input rather than through relative attention biases. This baseline performs similarly to LPE, improving in only 4 of the 32 setting–metric combinations (median relative change: −1.16%). As shown in Table R1, SimPos significantly outperforms factorized absolute PE in all 32 comparisons, with relative gains of 8.19%–38.4% (median: 16.54%). These results suggest that explicitly representing both positional coordinates alone is insufficient, highlighting the benefit of directly modeling their relative relationships through attention biases.
We have clarified this distinction in §3.2 of the revised PDF and revised the corresponding statements in the Introduction and Conclusion to avoid implying that flattening inherently loses positional information.
Table R1. Comparison of SimPos with LPE, factorized absolute PE, and ALiBi, averaged across four datasets for each backbone–tokenizer setting.
| Backbone / Tokenizer | LPE | Factorized abs. PE | ALiBi | SimPos |
|---|---|---|---|---|
| T5 / RQ-VAE | 0.0217 / 0.0136 | 0.0214 / 0.0134 | 0.0224 / 0.0139 | 0.0257 / 0.0164 |
| T5 / RQ-KMeans | 0.0227 / 0.0142 | 0.0224 / 0.0140 | 0.0237 / 0.0151 | 0.0273 / 0.0180 |
| GPT-2 / RQ-VAE | 0.0217 / 0.0136 | 0.0214 / 0.0134 | 0.0205 / 0.0131 | 0.0242 / 0.0157 |
| GPT-2 / RQ-KMeans | 0.0230 / 0.0149 | 0.0230 / 0.0148 | 0.0221 / 0.0146 | 0.0261 / 0.0169 |
Each cell reports R@5 / N@5, averaged over five seeds and four datasets (Beauty, Toys, Sports, and CDs). SimPos significantly outperforms all three baselines on both metrics across all datasets and settings (one-sided paired t-tests over five seeds, all Holm-adjusted p < 0.005 across 96 comparisons). The full per-dataset results are reported in Appendix C of the revised PDF.
Response to Reviewer bZK4 (3/4)
W2b. Contribution Beyond Recency Bias.
The interaction-level bias in SimPos introduces a recency preference, while its residual-level bias captures a separate structural relationship independent of interaction distance. To examine whether recency alone explains the performance gains, we conduct two complementary comparisons.
-
Recency without the residual axis. We consider an interaction-only variant (), which retains the same interaction-level recency bias as SimPos while removing the residual-level component.
-
Tuned recency without axis separation. We introduce a flattened linear relative bias (FlatBias), , which keeps SimPos's linear relative-bias form and a recency prior but uses a single flattened distance instead of separate positional axes. We sweep its strength λ over seven values (Table R2).
As shown in Figures 3 and 4 (§4.3.1), SimPos consistently outperforms the interaction-only variant, indicating that the residual-level bias provides additional benefits beyond interaction-level recency alone. Furthermore, Table R2 shows that SimPos maintains its advantage across the tested bias strengths. The per-dataset results in Appendix C of the revised PDF confirm that SimPos outperforms FlatBias across all 32 setting–metric comparisons, with relative gains of 2.70%–31.22% in R@5 and 6.97%–29.41% in N@5.
Together, these results suggest that SimPos's improvements cannot be explained solely by recency bias, but also arise from explicitly modeling interaction-level and residual-level positional relationships separately. In the revised manuscript, we have added this comparison as §4.3.4 and Appendix C, and revised the recency discussion in §4.3.2 accordingly.
Table R2. Sweep over the strength λ of the flattened linear relative bias, averaged over five seeds and all 16 settings (4 datasets × 2 backbones × 2 tokenizers).
| λ | 0.2 | 0.4 | 0.6 | 0.8 | 1.0 | 1.5 | 2.0 | SimPos |
|---|---|---|---|---|---|---|---|---|
| R@5 | 0.0218 | 0.0222 | 0.0222 | 0.0222 | 0.0225 | 0.0221 | 0.0210 | 0.0258 |
| N@5 | 0.0140 | 0.0142 | 0.0143 | 0.0142 | 0.0144 | 0.0141 | 0.0134 | 0.0167 |
SimPos significantly outperforms the best configuration of this baseline in all 32 setting–metric comparisons (one-sided paired t-tests over five seeds, all Holm-adjusted ). The sweeps per backbone–tokenizer setting and the per-dataset results are reported in Appendix C of the revised PDF.
Response to Reviewer bZK4 (1/4)
We sincerely thank the reviewer for recognizing the simplicity and ease of integration of SimPos, as well as for the constructive feedback. We carefully address these concerns below. (All new results use five training seeds per configuration.)
While DiscRec [1] and BARGE [2] also consider item-level structure in generative recommendation, they leverage this structure for different purposes.
DiscRec introduces learnable item-level position embeddings to distinguish code positions within each SID, together with localized intra-item attention in a separate collaborative branch to disentangle semantic and collaborative signals. BARGE addresses the item-boundary gap by using Item Context-Aware Attention (ICA) to incorporate item-level context into token representations, enabling the encoder to better preserve item grouping in flattened SID sequences.
SimPos, in contrast, directly models the two distinct positional relationships between tokens. It replaces the backbone's original positional encoding with two parameter-free relative biases that separately capture distances between interactions and between residual levels. Unlike DiscRec's position embeddings or BARGE's item-context aggregation, SimPos explicitly models both positional relationships without additional learnable modules.
We have expanded §2.1 to discuss DiscRec in the revised PDF and clarify its distinction from SimPos. We also note that BARGE was first posted to arXiv in July 2026, shortly before our submission.
[1] Liu, Chang, et al. "DiscRec: Disentangled Semantic–Collaborative Modeling for Generative Recommendation." arXiv:2506.15576, 2025.
[2] Zeng, Junchao, et al. "Bridging the Structural Gap: Adapting Autoregressive Generation for Recommendation." arXiv:2607.21028, 2026.