- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes CAP-Mix, a prototype-mixture item representation framework for flow-based generative recommendation. The method addresses the limitations of hard Semantic IDs, which introduce quantization errors, and flat continuous item embeddings, which lack explicit cross-item parameter sharing. CAP-Mix represents each item as a soft mixture of shared prototypes combined with an item-specific vector. The assignments and prototypes are jointly optimized with the recommendation model, supported by code-matching and prototype-utilization objectives. The framework can also incorporate pretrained SID information or continuous content embeddings. Experiments on six public datasets demonstrate improvements in recommendation accuracy over several generative retrieval baselines, along with higher inference throughput compared with the evaluated discrete generative methods.
Critical strengths
-
The paper addresses an important trade-off between hard semantic quantization and independent continuous item representations. Combining shared prototypes with item-specific vectors provides a reasonable way to introduce cross-item parameter sharing without imposing a discrete representation bottleneck.
-
The experiments cover six public datasets and three pretrained-information regimes. The comparisons with flat embeddings, hard SIDs, and generative retrieval baselines, together with the diagnostic analyses in the appendix, provide useful empirical evidence for the proposed framework.
Critical weaknesses
- The central problem of balancing information preservation and cross-item parameter sharing has been extensively explored in representation learning and generative recommendation. Recent studies, such as UniGRec [1] and Differentiable Semantic ID [2], have already investigated adaptive semantic assignments jointly optimized with recommendation objectives. Although extending these ideas to continuous flow-based recommendation is reasonable, the paper does not clearly identify the new challenges introduced by this setting or explain how its research problem fundamentally differs from existing work. As a result, the novelty of the problem formulation is limited.
- The main representation in Eq. (5), which combines a soft mixture of shared prototypes with an item-specific vector, is conceptually similar to existing prototype-based representation learning and embedding decomposition approaches. The soft assignments, attention-based assignment module, and prototype-utilization regularization also rely largely on established techniques. More importantly, the item-specific vector remains fully trainable and unconstrained, meaning that it can theoretically represent the entire item embedding without relying on the shared prototypes. Although auxiliary losses are introduced to encourage prototype learning, the paper provides insufficient evidence that this design learns more effective cross-item sharing than simpler alternatives, such as linear softmax assignments or parameter-matched embedding decompositions. Therefore, the methodological contribution and the necessity of the proposed components are not sufficiently established.
- Although CAP-Mix achieves improvements on several benchmarks, its advantages over the flat baseline are relatively small or inconsistent when full content embeddings are available. As shown in Table 1, the HR@5 improvements are only 1.49% on Beauty and 2.09% on Steam, while performance decreases by 0.52% on Industrial. More importantly, the paper lacks systematic ablations of the attention-based assignment module, code-matching loss, and prototype-utilization regularization, making it difficult to determine their individual contributions. Direct comparisons with closely related soft semantic ID methods are also missing. Furthermore, the reported inference speedups are primarily measured against discrete generative architectures with different backbones and decoding procedures, making it difficult to separate the benefit of the proposed item representation from that of continuous retrieval itself. These limitations weaken the experimental support for the paper's main conclusions.
[1] Li J, Zhang Y, Bai Y, et al. UniGRec: Unified Generative Recommendation with Soft Identifiers for End-to-End Optimization[J]. arXiv preprint arXiv:2601.17438, 2026.
[2] Fu J, Ge X, Karatzoglou A, et al. Differentiable Semantic ID for Generative Recommendation[C]//Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2026: 369-379.
Questions for authors
- What are the key methodological differences between CAP-Mix and existing approaches such as UniGRec and Differentiable Semantic ID? Could the authors further clarify the necessity of the proposed prototype-mixture design and provide comparisons with simpler alternatives, such as linear softmax assignments or parameter-matched embedding decompositions?
- Could the authors provide ablation studies to quantify the individual contributions of the attention-based assignment module, code-matching loss, and prototype-utilization regularization? Additionally, how much of the reported inference efficiency improvement can be attributed to the proposed item representation rather than differences in backbone architectures and decoding procedures?
4 comments
Q3
Although auxiliary losses are introduced to encourage prototype learning, the paper provides insufficient evidence that this design learns more effective cross-item sharing than simpler alternatives, such as linear softmax assignments or parameter-matched embedding decompositions. Therefore, the methodological contribution and the necessity of the proposed components are not sufficiently established.
Could the authors further clarify the necessity of the proposed prototype-mixture design and provide comparisons with simpler alternatives, such as linear softmax assignments or parameter-matched embedding decompositions?
A: We added both simpler alternatives: independent linear-logit softmax heads and a parameter-matched residual factorization. CAP-Mix has higher mean HR@5 in all four tested Beauty/Industrial scratch/PCA-text settings:
| Regime | Variant | Beauty HR@5 | Industrial HR@5 |
|---|---|---|---|
| Scratch | CAP-Mix (attention) | 5.89 (ref.) | 2.88 (ref.) |
| Scratch | Linear-logit softmax | 5.81 (−1.36%) | 2.82 (−2.08%) |
| Scratch | Residual factorization | 5.76 (−2.21%) | 2.73 (−5.21%) |
| PCA-text | CAP-Mix (attention) | 6.79 (ref.) | 3.79 (ref.) |
| PCA-text | Linear-logit softmax | 6.70 (−1.33%) | 3.67 (−3.17%) |
| PCA-text | Residual factorization | 6.51 (−4.12%) | 3.68 (−2.90%) |
Our design predicts prototype assignments from a compact learned item feature (D_f=32 in these experiments, versus retrieval dimension D=128). The attention head enables assignment representations to interact across codebook tokens, providing an architectural route to coordinating potentially complementary prototype contributions rather than predicting each group solely from the item feature. This is an inductive bias.
The parameter-matched factorization is X = R + UB, with R ∈ ℝ<sup>V×D</sup>, U ∈ ℝ<sup>V×q</sup> and B ∈ ℝ<sup>q×D</sup>. R remains unrestricted; U contains unconstrained item-specific weights over q shared vectors in B. We choose q to approximately match CAP-Mix’s reported active-training parameter count.
Q4
Although CAP-Mix achieves improvements on several benchmarks, its advantages over the flat baseline are relatively small or inconsistent when full content embeddings are available. [...]
A: We agree that the gains over PCA-text flat tables are small relative to the other settings. However, the HR improvements are consistent, and increasingly so at larger cutoffs in Table 1 (with prior) and Table 7 (without prior). Among flat, hard and CAP-Mix, CAP-Mix ranks first in 4/6 dataset–prior settings at HR@5, 5/6 at HR@10 and 6/6 at HR@20. Industrial in Table 1 moves from second to first at HR@20, and Steam in Table 7 does so at HR@10.
| Dataset | Variant | T1 HR@5 | T1 HR@10 | T1 HR@20 | T7 HR@5 | T7 HR@10 | T7 HR@20 |
|---|---|---|---|---|---|---|---|
| Beauty | Flat | 6.69 | 9.66 | 13.47 | 6.27 | 9.52 | 13.52 |
| Beauty | Hard | 6.43 | 9.39 | 13.11 | 6.27 | 9.23 | 12.90 |
| Beauty | CAP-Mix | 6.79 | 9.75 | 13.53 | 6.50 | 9.70 | 13.71 |
| Industrial | Flat | 3.81 | 5.79 | 8.33 | 3.62 | 5.54 | 8.03 |
| Industrial | Hard | 3.52 | 5.41 | 7.89 | 3.51 | 5.42 | 7.93 |
| Industrial | CAP-Mix | 3.79 | 5.77 | 8.36 | 3.68 | 5.67 | 8.25 |
| Steam | Flat | 8.13 | 13.65 | 20.55 | 8.10 | 13.45 | 20.70 |
| Steam | Hard | 8.19 | 13.95 | 20.95 | 8.21 | 13.45 | 20.38 |
| Steam | CAP-Mix | 8.30 | 14.28 | 21.50 | 8.14 | 13.77 | 20.75 |
A content-initialized flat table is already a strong baseline, which may leave less scope for additional gains from statistics pooling.
Q5
More importantly, the paper lacks systematic ablations of the attention-based assignment module, code-matching loss, and prototype-utilization regularization [...].
Could the authors provide ablation studies to quantify the individual contributions of the attention-based assignment module, code-matching loss, and prototype-utilization regularization?
A: We added individual and joint loss removals across all three Beauty regimes, and provided the attention-to-linear-logit head swap in Q3.
We removed each loss and partially retuned the surviving auxiliary coefficients. As shown, removing either loss lowers mean HR@5.
| Variant | Scratch HR@5 | Offline SID HR@5 | PCA-text HR@5 |
|---|---|---|---|
| CAP-Mix | 5.89 (ref.) | 6.32 (ref.) | 6.79 (ref.) |
| No code matching | 5.78 (−1.87%) | 5.76 (−8.86%) | 6.68 (−1.62%) |
| No utilization loss | 5.75 (−2.38%) | 6.08 (−3.80%) | 6.77 (−0.29%) |
| Neither loss | 5.63 (−4.41%) | 5.76 (−8.86%) | 6.60 (−2.80%) |
Q6
Direct comparisons with closely related soft semantic ID methods are also missing.
A: We added DIGER and UniGRec comparisons. For UniGRec, we use the authors’ newer official v2 implementation without post-hoc reranking.
| Method | Beauty HR@5 | Beauty NDCG@5 | Industrial HR@5 | Industrial NDCG@5 | Steam HR@5 | Steam NDCG@5 |
|---|---|---|---|---|---|---|
| DIGER | 4.53 | 3.02 | 3.20 | 2.09 | 7.80 | 4.98 |
| UniGRec | 5.63 | 3.79 | 3.52 | 2.32 | 7.94 | 5.10 |
| CAP-Mix | 6.79 | 4.73 | 3.79 | 2.34 | 8.69 | 5.57 |
CAP-Mix has higher mean HR@5 and NDCG@5 than the listed baselines on all three datasets in these comparisons.
Please see the response to reviewer Suzb's Q14 question for more details.
Q7
the reported inference speedups are primarily measured against discrete generative architectures with different backbones and decoding procedures, making it difficult to separate the benefit of the proposed item representation from that of continuous retrieval itself.
Additionally, how much of the reported inference efficiency improvement can be attributed to the proposed item representation rather than differences in backbone architectures and decoding procedures?
A: Thank you for this question. We do not attribute the speedup over discrete generators to CAP-Mix’s assignment head. The assignment heads are used during training, and the resulting item embeddings are materialized as a table for inference. The heads are not evaluated per query and do not accelerate the flow backbone. Thus, CAP-Mix provides training time item representation learning benefits that do not affect inference performance.
Differences from TIGER/MaskGR reflect complete systems with different backbones and decoding procedures. The prior adds per-query cost and remains an accuracy–cost trade-off; it can be omitted, as in Table 7.
To make the timing comparison more consistent, we retained native generation while replacing the original testing-loop timers with a common prediction-only benchmark using three actual history items, including complete SID tuples:
| Model | Beauty HR@5 | Beauty queries/s | Industrial HR@5 | Industrial queries/s |
|---|---|---|---|---|
| CAP-Mix, with prior | 6.79 | 29,759.90 | 3.79 | 28,221.02 |
| CAP-Mix, no prior (Table 7) | 6.50 | 40,305.38 | 3.68 | 40,619.65 |
| FlowRec, no prior | 6.28 | 41,396.55 | 3.62 | 41,207.25 |
| TIGER | 3.61 | 19,664.22 | 2.13 | 6,065.25 |
| MaskGR | 5.08 | 168.37 | 3.46 | 222.28 |
Accordingly, our efficiency claim is that the prototype-assignment head adds no per-query evaluation after materialization, retaining the inference-time benefits of FlowRec over discrete baselines in our experiments. We will modify our statements to make this clear.
Q8
What are the key methodological differences between CAP-Mix and existing approaches such as UniGRec and Differentiable Semantic ID?
A: UniGRec and DIGER jointly adapt semantic tokenization to recommendation while retaining discrete autoregressive generation. One of their main contributions is how to overcome the non-differentiability of discrete SIDs tokenizers. CAP-Mix instead combines adaptive prototype mixtures with unrestricted per-item vectors in a continuous item table, and thus does not have deal with differentiability complexities. The flow predicts a continuous vector and retrieves against the materialized table without semantic-ID decoding. Semantic information enters separately through PCA initialization or optional tokenizer priors, while parameter sharing and statistics pooling are provided by the CAP-Mix's prototype mixtures, without requiring hard quantization.
Thank you for the detailed responses and additional experiments. The rebuttal addresses several of my concerns, but some key issues remain.
Methodological novelty: The differences from existing discrete Semantic ID methods are clearer, but the novelty over soft prototype mixture and embedding decomposition approaches remains insufficiently established. The additional experiments mainly demonstrate the importance of item-specific vectors rather than the contribution of shared prototypes.
Performance improvements: The new comparisons are helpful, but the gains over Linear Softmax and the Flat Baseline remain modest, especially with full content embeddings. Statistical significance analyses are still missing.
Experimental evidence: The additional ablations, comparisons with UniGRec and DIGER, and efficiency experiments substantially improve the experimental completeness. However, they mainly support empirical effectiveness rather than methodological novelty, while some concerns about comparison fairness remain.
Overall, I appreciate the authors' efforts, but my main concern regarding methodological novelty remains. Therefore, I will maintain my original rating.
Dear Reviewer 3ocm,
Thanks very much for your appreciation of our paper’s motivation for cross-item sharing, prototype-mixture design, and breadth of the experimental evaluation. Regarding all of your raised questions in terms of motivation, methodological distinctions from prior work, component ablations, and experimental evidence, for better illustration, we made the following organization with added empirical experiments and extended analysis. We are more than willing to provide any further explanation, please let us know.
Q1
A: Our challenge is to learn useful cross-item sharing while retaining unrestricted, unquantized item representations, while being able to incorporate SID and/or embedding information. Because per-item vectors can bypass shared prototypes, useful sharing is not automatic, and how to incorporate SID information into continuous recommenders while allowing for adaptation during training is not straightforward either. CAP-Mix jointly learns adaptive prototype mixtures and item-specific vectors to pool recommendation gradients; semantic information enters separately through content initialization or optional tokenizer priors. The contribution is this continuous-table learning design, rather than a new solution to tokenizer non-differentiability.
Q2
A: The unrestricted item-specific vector is intentional. In continuous space, we preserve per-item flexibility, including the capacity to represent item-specific patterns. However, the ability to represent the same item table does not imply identical learning behavior: shared parameters introduce gradient coupling that independent item rows lack.
Please note the flat baselines of Table 1 and Table 7 also have this unrestricted item embedding. For example, the scratch scenario initializes all item embeddings randomly and CAP-Mix outperforms the flat baseline. Since both have the same representation capacity, this performance improvement comes from the learning dynamics provided by CAP-Mix.
Furthermore, we separately tested removing the per-item embeddings, as well as replacing them with a compressed item table. The full model has higher mean HR@5 in every tested setting:
HASH10 maps items randomly to ceil(V/10) trainable residual rows, so approximately ten items share each row. No-residual removes r_i while retaining learned features, assignments and prototypes. These controls show the utility of the residual in the tested recipes.