- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper considers generative recommendation. Motivated by the commonly used memory modules in recent deep learning literature (especially sequence modeling), the authors proposed a new model architecture called Hierarchical Conditional Transition Memory (HCTM), which was designed to enhance the reusable structure to achieve performance gains. They first carefully examined the limitations of existing methods including the dense transformers only and pattern lookups only to show they are not sufficient, then they designed two types of memory banks including the item-internal one and the prefix-conditioned one. Experiments show performance gains over existing methods like TIGER and OneRec on some public datasets.
Critical strengths
-
The motivation is clear and easy to follow -- memory modules have been proven effective in sequence modeling in recent years, and it is natural to ask if SID for generative recommendation can also benefit from this type of model architectures.
-
The authors present some systematic study (Section 4) to show the necessity of using memory banks to avoid redundancy from structural reuse, and they investigate each building block of existing architectural designs including dense transformers alone and pattern lookup alone to showcase that each individual part are insufficient.
-
Code is provided. Experiments show performance gains over existing methods.
Critical weaknesses
-
Memory modules are not new in literature -- they have been introduced in some LLMs from frontier labs such as Meta [1] and DeepSeek [2].
-
Certain notations or figures might be confusing. For example in Figure 5, different items should in general have different codes and deduplication suffixes, and thus it might be better to have some subscripts or superscripts to indicate the codes/suffixes are for item j (1<= j <= i). Also there seems no reference to Figure 5, so might be difficult for readers to find definitions for the superscripts written in the "Prefix-Conditioned Transition Memory" box.
-
Experiments are on some public datasets and might not be applicable to industry settings yet.
References
[1] Memory Layers at Scale
[2] Conditional memory via scalable lookup: A new axis of sparsity for large language models.
Questions for authors
-
Since the newly proposed memory banks could achieve structural reuse, could this also bring efficiency gains as we could avoid the redundancy?
-
There is a DeepSeek-Engram baseline added to the memory augmented baselines. However there seems no too many details disclosed to show how it is implemented in this experiment? Also there are some other memory augmented baseline choices such as memory layers. Wondering if the authors can provide some insights on this.
-
Does the word "Hierarchical" refer to the RQ-VAE in SID? If I understand correctly there is no "Hierarchical" structure in the newly introduced memory banks?
-
Are there any results on industrial-scale datasets?
2 comments
Rebuttal (2/2)
Figure 5 notation (W2)
We sincerely apologize for the notation ambiguity in the initial submission and thank the reviewer for catching this. In the revised manuscript, we have updated Figure 5 and its caption to explicitly define the successor-candidate superscripts, which also distinguishes SID code positions from the deduplication suffix.
Industrial applicability (W3, Q4)
While our current evaluation focuses on standard public benchmarks, HCTM possesses two structural properties that offer promising potential for large-scale industrial deployment:
- Bounded memory footprint: Indexing quantized Semantic-ID prefixes with a fixed budget () scales with the hierarchical codebook space rather than the full item catalog, keeping storage overhead manageable as item catalogs grow.
- Off-accelerator lookup potential: Because the transition store is non-parametric, it could be hosted in distributed host memory (RAM/NVMe) and queried asynchronously alongside standard serving pipelines, avoiding GPU HBM bottlenecks.
We acknowledge that real-world production deployment involves additional system-level trade-offs (e.g., distributed cache consistency and tail latency), and we leave large-scale industrial validation to future work.
Efficiency of structural reuse (Q1)
Yes, while structural retrieval introduces a modest per-step overhead, it yields substantial overall training efficiency by significantly accelerating optimization convergence:
- Minimal parameter and per-step overhead: Adding both memory pathways increases trainable parameters by only 4.8% (13.76M vs. 13.13M for TIGER) and reduces per-step training token throughput by 7.8%.
- Substantial overall speedup (48.2% faster total training): Because explicit memory supplies ready-to-use structural priors that accelerate optimization fitting, HCTM converges 2.1× faster—reaching early stopping in 1,050 optimizer steps compared to 2,200 for TIGER—which yields an 48% reduction in total end-to-end training time.
Meaning of "hierarchical" (Q3)
We use “hierarchical” in a structural-granularity sense: HCTM aligns memory across the two compositional levels inherent to Semantic-ID sequences:
- Lower level (intra-item composition): Item memory models how coarse-to-fine Semantic-ID codes combine within individual items to enrich historical representations on the encoder side.
- Higher level (inter-item transition): Prefix memory models sequential transitions across consecutive items by retrieving candidate successor prefixes on the decoder side.
We have clarified this distinction explicitly in the revised manuscript to prevent ambiguity.
Rebuttal (1/2)
We sincerely thank Reviewer BRcQ for the constructive comments and questions regarding prior memory architectures, implementation details, efficiency, and terminology. Following your suggestions, we have incorporated Memory Layers directly into our main benchmark comparisons, added full baseline formulations, and clarified our architectural definitions.
Relation to prior memory modules and baseline details (W1, Q2)
To clarify HCTM's positioning relative to generic memory modules and address the implementation details (Q2), we highlight our recommendation-specific structural specialization, provide the exact formulations added to the appendix for both memory baselines, and report their controlled comparison on TIGER:
mt=Avg(e(n,q,t))
st=d⟨RMSNorm(ht),RMSNorm(Wkmt)⟩
αt=σ(sign(st)∣st∣)
ht←ht+αtmt
It=TopKj(ht⊤kj,4)
α(j)=softmax(ht⊤k(j)),j∈I
m(t)=∑j∈Iα(j)v(j)
The retrieved output replaces the FFN sub-layer within the standard TIGER Transformer block, preserving normalization, dropout, and residual connections without external index construction.
Memory Layers improves upon TIGER across all benchmarks by expanding latent FFN capacity via continuous hidden-state queries, whereas DeepSeek-Engram underperforms TIGER—likely because assuming flat, language-like token adjacency fails to respect the discrete compositional syntax of SID sequences. In contrast, HCTM achieves the highest NDCG@10 across all datasets by explicitly aligning memory retrieval with SID's dual-scale hierarchy—capturing intra-item token compositions and predictive inter-item transitions.