acceptodds
Under review as a conference paper at ICLR 2027

What Makes Semantic IDs Learnable? A Spectral View of Output Factorization.

Abstract

Generative recommender systems commonly use semantic IDs (SIDs) to represent items as sequences of tokens that are in turn predicted autoregressively. When each item has a unique code, different SID mappings preserve the same item information but decompose item prediction into different sequences of token predictions. We study this output factorization: what properties of a code make its successive predictions easier for a model with restricted output rank? We describe atomic and SID mappings as filtrations and derive spectral approximation bounds for the best predictor in a specified class. These bounds motivate the gain-weighted spectral tail (GWT), a mapping-level proxy for learnability at a given head rank. Controlled synthetic experiments isolate spectral concentration at matched predictive gain and numerical rank, showing its consequences for learning. We then assess whether GWT remains useful after training rank-restricted SASRec on three Amazon datasets. With atomic histories fixed, GWT exhibits strong correlation with test negative log-likelihood (NLL) across seven SID mappings, including RQ-VAE and RQ-Kmeans. We also compare GWT with common SID diagnostics, including codebook utilization and the per-level token count Gini coefficient. These initial results support GWT as a principled proxy metric for SID learnability.

Primary area:
learning theory
Keywords:
Semantic IDs, generative recommendation, output factorization, spectral analysis, low-rank approximation, learnability

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_apHTOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

n/a

Critical strengths

n/a

Critical weaknesses
  1. The title on OpenReview is inconsistent with the title in the PDF

  2. The primary area listed on the OpenReview homepage is “learning theory,” but the main result is a diagnostic correlation analysis. Section 3, “FILTRATIONS AND OUTPUT LEARNABILITY,” does not appear to fall under learning theory. In my view, the paper does not fit the stated area and is therefore out of scope.

  3. Finally, and most importantly, I believe that a substantial portion of the paper was generated by a large language model rather than written by the authors. The authors also acknowledge this in the AI USE STATEMENT. This appears to conflict with the iclr conference requirements.

Questions for authors

n/a

2 comments

Dear Reviewer,

We would like to address the three points raised in your review.

On AI use, we do not see a basis for the claim that our disclosed assistance conflicts with ICLR’s requirements. The conference’s AI policy permits assistance with writing and theoretical work, subject to disclosure and author responsibility. Our manuscript includes an AI-use statement, and we take responsibility for its claims, results, and implementation. Could you identify the specific policy provision that you believe has been violated?

The policy is here: https://iclr.cc/Conferences/2027/AIPolicyForAuthors

On scope, Section 3 studies how an item-to-code mapping changes the approximation error of a capacity-restricted predictor. Theorem 2, on page 5, establishes lower and upper bounds on the best attainable excess log-loss under explicit assumptions. The formal statements and proofs are in Appendix C.3. These results motivate GWT, whose relationship with trained prediction error is then investigated experimentally. This theoretical analysis is why we selected learning theory as the primary area.

You are correct about the title mismatch. This is a mistake on our side, and we will correct it.

We would also appreciate clarification of the specific technical concerns underlying your soundness rating. The review does not identify an error in a mathematical statement, proof, or experiment. Identifying those concerns would allow us to address them directly.

Thank you for the authors' response.

First, consistency in the manuscript's title upon submission is a fundamental requirement that every author must carefully check and verify beforehand; mismatched titles run counter to a rigorous academic attitude. Second, Chapter 1 fails to adequately present the research motivation; Chapter 3 appears to be a mere accumulation of formulas without analyzing their internal connections, and lacks necessary discussion regarding generative recommendation, making it very obscure to read. Furthermore, relying extensively on large language models to write the paper without researchers carefully considering and overseeing its content is not encouraged by a rigorous research community. The declaration in the AI use statement cannot serve as a blanket exemption from responsibility. This is why the scores for Presentation, Informativeness, and Soundness are all very low.

Therefore, I will maintain my original rating.

Reviewer_e9F7OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks which properties of a semantic-ID (SID) mapping make generative-recommendation targets easier to learn when the item information is held fixed.

  • Framework. A mapping is described as a filtration of nested item partitions. Injective mappings only redistribute a fixed total predictive gain I(Y;C) across levels (Prop. 1), so differences in learned performance must come from the induced conditional problems.
  • Theory. Parent-wise spectra of centered logit covariances are defined. Under a probability floor ε and a projection-realizability condition, the best excess item log-loss of a rank-r factorized predictor is bounded below by 2ε·Σᵢ Gᵢτᵢ(r) and above by (1/2ε)·Σᵢ Gᵢτᵢ(r) (Thm 2). This motivates the gain-weighted spectral tail (GWT, Eq. 20) as a mapping-level diagnostic at a given head rank.
  • Controlled synthetic study. With predictive gain and numerical rank matched, concentrated spectra are easier to learn (Fig. 1; Spearman 0.84/0.91/0.91 at ranks 2/4/8; rank-2 KL 0.374 vs. 0.915 between the most and least concentrated profiles). A second experiment fixes the code tree so that all counting statistics are identical (Table 1), and GWT still orders trained KL correctly at ranks 4 and 8 (Fig. 2).
  • Real data. On Amazon Beauty, Sports and Toys, rank-restricted SASRec (r ∈ {2, 4, 8}, three seeds) is trained on seven supplied SID constructions — RecDMoN, RQ-KMeans, RQ-GAE, RQ-VAE, LETTER, S2GR and MMGRec. GWT, estimated from a reference atomic SASRec, is correlated with test NLL and benchmarked against common SID diagnostics (Table 2, Fig. 3). Sec. 6 states the scope and limitations explicitly.
Critical strengths
  1. A solid theoretical analysis of a question the SID literature has only treated heuristically.
  • The filtration view separates output factorization from signal density and history encoding (pp.1–2), and Prop. 1 shows that injective codes conserve total predictive gain. This pins down exactly what a mapping can and cannot change, which the field has lacked.
  • Thm 2 then gives two-sided spectral bounds on the best attainable excess log-loss under a rank-restricted head, and GWT follows from them rather than being postulated. Its level-wise contributions identify where substantial predictive gain coincides with variation a head of given rank cannot represent.
  • The theory also explains a negative result that counting diagnostics cannot: Table 1 constructs mappings with identical token utilization, Gini, entropy, effective vocabulary and prefix statistics, yet their trained KL differs, and GWT orders them (Fig. 2).
  • Why it matters: this is new and transferable knowledge about why one SID is easier to learn than another, independent of any particular construction algorithm.
  1. Validation against the SID construction methods practitioners actually use.
  • Seven real, named constructions are compared on three Amazon domains under one fixed predictor and protocol: RQ-VAE, RQ-KMeans, RQ-GAE, RecDMoN, LETTER, S2GR and MMGRec, with atomic histories and the reference law held fixed so only the output factorization varies (Sec. 5.1, App. E.1).
  • At k = 16, GWT has the largest dataset-averaged correlation magnitude among all recomputed diagnostics at every head rank, beating token utilization, token-count Gini, normalized entropy, effective vocabulary size, decoding-state count, valid-next-token fraction and the semantic/collaborative cosine metrics (Table 2). It selects RecDMoN on all three datasets, and RecDMoN has the lowest mean NLL in eight of nine dataset–rank settings. It also agrees with 147 of 189 pairwise mean-NLL comparisons (App. E.2).
  • The scope section is unusually candid: reference dependence, vanishing tails for narrow trees, and the fact that a log-loss ordering does not imply a retrieval ordering are all stated by the authors (Sec. 6).
  • Why it matters: breadth of comparison against the real alternatives, not just synthetic controls, is what demonstrates that the diagnostic is useful rather than merely well-motivated.
Critical weaknesses
  1. The strongest real-data numbers depend on a diagnostic rank selected on the evaluation data.
  • At the theory-consistent setting k = r, GWT's mean within-dataset Spearman correlation with test NLL is 0.476 (Table 2), which is lower in magnitude than the valid-next-token fraction (−0.532) and the decoding-state count (0.520).
  • The headline 0.706 uses k = 16, which App. E.2 states was chosen because it "maximizes this average within the sweep" over the same nine dataset–rank settings used for evaluation. It also decouples the diagnostic from the head rank that Thm 2 ties it to.
  • Each correlation uses only seven mappings, so the estimates are noisy, and orderings beyond the top-1 are often wrong (on Toys the three added mappings are ordered in reverse, App. E.2). The conclusion also refers to selecting among "four candidates" (p.9) while Sec. 5 compares seven.
  • Why it matters: the abstract's claim of "strong correlation" currently rests on post-hoc selection. A held-out choice of k with confidence intervals is needed, and this is checkable within the rebuttal period.
  1. The practical gap: the bounds are loose and the evaluated regime is far from deployed generative recommenders.
  • The two-sided bounds differ by (1/2ε)/(2ε) = 1/(4ε²) (Eqs. 18–19). Since ε cannot exceed 1/B_{i,p}, that factor is at least B_{i,p}²/4, i.e. orders of magnitude for realistic codebooks, so the theorem constrains mapping orderings only weakly.
  • Experiments use heads of rank 2–8 on a two-layer, width-64 SASRec with embedding width 64 (App. E.1), whereas deployed generative recommenders use full-rank heads, where tails vanish once k ≥ B_{i,p} − 1 (Sec. 6). Matching head rank across mappings also leaves parameter counts unmatched (Sec. 5.1).
  • Evaluation centers on NLL, while SID choices in practice are made on retrieval metrics; Recall@10 appears only in App. Table 19. No experiment shows GWT leading to a better SID choice or construction end to end.
  • Why it matters: this limits how much the diagnostic can be acted upon today. It is a bound on the paper's impact rather than a correctness problem, and the authors acknowledge most of it in Sec. 6.
Questions for authors
  1. Held-out choice of k. Please fix k a priori (k = r, or select k on one dataset and freeze it for the other two) and report how GWT then compares with the valid-next-token fraction and the decoding-state count, with bootstrap confidence intervals over the n = 7 mappings. If GWT is not better than these count statistics under held-out selection, I would lower my score.
  2. Retrieval metrics. Does GWT predict differences in Recall@10 / NDCG@10 across the seven mappings (App. Table 19), not only NLL? A version of Table 2 for retrieval metrics would strengthen the practical claim considerably.
  3. Sensitivity to the reference law. How sensitive are GWT and its mapping orderings to the atomic reference model, e.g. references trained with different seeds or capacities? Sec. 6 raises this; a quantitative check would help.
  4. Text consistency. Please reconcile "four candidates" in the conclusion with the seven-mapping study in Sec. 5, and align the abstract's "strong correlation" with whatever the held-out analysis supports.
  5. An end-to-end demonstration — selecting or constructing a mapping by GWT in a standard full-rank TIGER-style setting, versus a utilization-based choice — would turn the diagnostic into an actionable design tool.
3 comments

Dear Reviewer e9F7,

Thank you for the thorough review and for recognizing the explanatory contribution. Our focus, stated in Section 1, is how SID output factorization interacts with output capacity to affect learnability. The fixed-tree experiment surfaces the mechanism explicitly: mappings with the same predictive information and identical code-counting statistics produce different learning outcomes. We clarify below how the rank sweep should be interpreted, report the additional analyses, and specify the corresponding manuscript changes.

Concern 1: Diagnostic rank and the strength of the Amazon evidence

We explored diagnostic rank k and trained head rank r independently. The setting k = r links the diagnostic to the output-rank constraint in Theorem 2. Varying k independently of r examines its association with trained error across rank combinations.

Using the theory-prescribed cutoff k = r, the comparison is:

DiagnosticMean Spearman correlation with NLL95% bootstrap interval
GWT, k = r0.476[−0.050, 0.858]
Valid-next-token fraction, sign reversed0.532[0.123, 0.852]
Decoding-state count0.520[−0.012, 0.880]

Each result averages nine dataset–rank settings, with seven mappings per setting and performance averaged over three training seeds. We reverse the sign of valid-next-token fraction to align the expected directions. The intervals use 10,000 paired bootstrap resamples of mapping identities, preserving the same resample across datasets, ranks and diagnostics. They condition on the existing reference models and training-seed means.

In the additional cross-dataset check you requested, we selected k to maximize mean NLL rank correlation on one dataset and applied it unchanged to the other two, rotating the calibration dataset. Mean Spearman correlation was 0.444 for GWT, compared with 0.532 and 0.520 for the two count baselines. This is a retrospective cross-dataset check, since all three datasets had already been examined.

Both count baselines have higher average correlations in these comparisons. We will therefore qualify the abstract's “strong correlation” statement. We will report the complete k-by-r sweep alongside the matched-rank results, identifying the 0.706 correlation at k = 16 as an observation within that exploration. The main-text figure will use k = r.

We will replace the final four sentences of the abstract with:

We further examine GWT on three Amazon datasets across seven existing SID mappings with atomic histories held fixed. At the theory-matched rank, its observed association with trained prediction error is positive on average and varies across datasets. Two counting-based diagnostics have larger magnitudes of mean correlation. The theory and experiments explain how SID output factorization interacts with output capacity to affect prediction error.

We will also state explicitly in Section 5.2:

We evaluate diagnostic ranks k ∈ {1, 2, 4, 8, 16, 32} independently of trained head ranks r ∈ {2, 4, 8}. This sweep examines how GWT's association with trained prediction error varies with both quantities. Within the evaluated grid, k = 16 gives the largest mean correlation (0.706). The matched setting k = r gives 0.476. We report the sweep as an empirical analysis alongside the comparison tied to the theoretical output-rank constraint.

Concern 2: What the formalization, theory and experiments establish

The filtration view gives atomic IDs and injective SID mappings a common mathematical description. Each additional code token refines a partition of the item catalogue. Proposition 1 establishes conservation of total predictive information. The mapping determines how that information is allocated across steps and which conditional prediction problems the model must learn.

This formalization is a first step toward a more rigorous understanding of SID learnability across different constructions. The spectral analysis develops one mechanism within it. We intend to extend the framework to also support the study of signal density and input representation in the future.

We agree that the probability-floor constants can be loose. Theorem 2 bounds the best attainable excess log-loss in the specified class under its stated assumptions. Ordering trained models requires additional empirical assessment. The lower bound covers arbitrary feature computation. The upper bound requires one model to jointly realize the optimal spectral projections.

The experiments on synthetic data demonstrate the importance of the structure identified by the theory:

  • Section 4.2 varies spectral concentration at matched predictive gain and numerical rank. At output rank 2, mean trained item KL is 0.374 for the concentrated profile and 0.915 for the flat profile. This isolates the effect of spectral concentration under a fixed output-rank constraint.
  • Section 4.3 fixes the item-prediction task and code-tree structure, changing only item-to-code assignments. The statistics in Table 1, including both baselines highlighted in your review, take the same values for every mapping. The resulting models have different prediction errors. GWT correctly orders the four mapping families at ranks 4 and 8. Its ordering is less accurate at lower ranks.

In the fixed-tree comparison, the count diagnostics assign the same score to every mapping. The spectral analysis describes how their conditional prediction problems differ. It supplies information about the prediction task that complements code-usage statistics.

The Amazon experiments provide complementary evidence on real interactions and existing SID methods. At k = r, GWT's mean Spearman correlations with NLL across ranks are 0.679 on Beauty, 0.667 on Sports and 0.083 on Toys. The strength of the observed association depends on the dataset.

The low ranks are deliberate capacity restrictions for studying this mechanism. Parameter counts vary across the Amazon mappings at each matched rank. When spectral tails vanish, learning can still be limited by finite data, optimization and other model constraints. An end-to-end TIGER-style study would test whether GWT can guide SID selection. Our present goal is to explain the mechanism and assess the diagnostic within the evaluated rank restrictions.

Concern 3: Retrieval metrics

At k = r, the corresponding mean within-setting correlations are:

DiagnosticRecall@10NDCG@10
GWT−0.381−0.389
Valid-next-token fraction0.5160.524
Decoding-state count−0.183−0.198

Lower GWT is associated, on average, with higher retrieval quality, which is the expected direction. Valid-next-token fraction has the stronger average association. We will add the retrieval comparison by head rank alongside Table 2. Complete-item NLL remains the primary outcome because it corresponds to the loss analyzed by the theory. Retrieval quality requires a separate empirical comparison.

Concern 4: Sensitivity to the reference model

The Amazon comparisons use one atomic reference model per dataset. Sensitivity to reference initialization and capacity remains to be measured. The synthetic experiments use known conditional distributions.

We will add the following clarification to Sections 5.1 and 6:

The three reported training seeds vary the SID prediction models and reuse one atomic reference model per dataset. The Amazon diagnostic results are conditional on the chosen reference. Sensitivity to reference initialization and capacity requires a separate experiment.

Conclusion and manuscript revision

We will make the role of the formalization explicit at the start of the conclusion:

The filtration view provides a common mathematical description of how atomic IDs and injective SID mappings factorize item prediction. It identifies the information preserved by a mapping and the conditional predictions it requires, providing a starting point for rigorous analysis across SID constructions.

We will replace the second paragraph of the conclusion, including the incorrect “four candidates” wording, with:

Experiments on synthetic data isolate the effect of spectral concentration at matched predictive gain and numerical rank. They also show differences in trained prediction error across mappings with identical code-counting statistics. The Amazon comparison across seven existing SID constructions provides complementary evidence consistent with this mechanism. Its associations between GWT and trained error vary across datasets. The theory and experiments identify conditional prediction structure relevant to learnability under output-rank constraints. GWT diagnoses this aspect of output factorization under the stated assumptions. Other sources of trained prediction error require separate assessment.

The count baselines have higher point estimates in the requested comparisons. We recognize the importance you placed on this criterion. Our case for acceptance rests on the filtration formalization and its analysis of how output factorization affects prediction difficulty under limited capacity. The synthetic experiments demonstrate consequences after training, including different errors across mappings with identical code-counting statistics. The Amazon study adds evidence from existing SID methods. With the empirical claims revised as specified above, we ask you to retain your support for acceptance on the basis of this scientific contribution.

Reviewer_1BaaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

Different semantic ID (SID) mappings can lead to different performance in generative recommendation. This paper investigates which properties make SID mappings easier for models to learn. The authors formulate SID mappings as filtrations, define context-dependent predictive gains, and analyze the covariance spectra of prediction logits across user contexts. Under explicit assumptions, they derive bounds on the best attainable excess log-loss for predictors with rank-restricted output heads. These bounds motivate the gain-weighted spectral tail (GWT) as a proxy for SID learnability. The authors evaluate this metric through controlled synthetic experiments and diagnostic comparisons on real-world recommendation datasets.

Critical strengths
  • Theoretical analysis. The authors analyze how SID mappings affect the best attainable prediction loss under output-rank constraints. Although the analysis relies on simplifying assumptions, it provides a theoretical perspective on assessing SID learnability before training a downstream recommender.
Critical weaknesses
  • Limited experiments. The experiments use SASRec [1] as the backbone, which is a conventional ID-based recommendation model rather than a generative recommender. Moreover, using atomic item representations for historical interactions while predicting SID targets is misaligned with the common SID-based generative recommendation setting. The authors should use generative recommendation backbones such as TIGER [2] or LC-Rec [3] to compare different SID mappings. A more comprehensive comparison of recommendation performance is also recommended to better evaluate SID learnability.

  • Limited practical relevance. The experiments on Amazon focus on very low output-head ranks (typically from 2 to 8), leaving the applicability of the findings to practical recommenders unclear. The authors should evaluate more representative output ranks, discuss the gap between the experimental settings and practical systems, and clarify how the theoretical results can inform the design or selection of SID mappings in such systems.

[1] Kang W C, McAuley J. Self-attentive sequential recommendation[C]//2018 IEEE international conference on data mining (ICDM). IEEE, 2018: 197-206.

[2] Rajput S, Mehta N, Singh A, et al. Recommender systems with generative retrieval[J]. Advances in Neural Information Processing Systems, 2023, 36: 10299-10315.

[3] Zheng B, Hou Y, Lu H, et al. Adapting large language models by integrating collaborative semantics for recommendation[C]//2024 IEEE 40th International Conference on Data Engineering (ICDE). IEEE, 2024: 1435-1448.

Questions for authors

Please refer to the weaknesses

2 comments

Dear Reviewer 1Baa,

Thank you for recognizing the theoretical contribution. As stated in Section 1, our focus is understanding how SID output factorization interacts with output capacity to affect learnability. We address your concerns below.

Concern 1: Experimental setup and backbone choice

We chose SASRec as an established and competitive backbone for a controlled comparison of SID mappings. Recent work shows that well-tuned SASRec with cross-entropy training remains a strong baseline against SID-based generative recommenders. Lepage et al., 2025 We keep the backbone architecture and training protocol fixed across mappings and vary the output-head rank to study capacity. In our implementation, the prediction task is autoregressive SID generation: each token prediction conditions on the interaction history and preceding predicted tokens, with conditional cross-entropy for training and constrained beam search for retrieval.

We keep histories atomic to hold the input encoding fixed while varying the target mapping. Changing both encodings would introduce another source of learning difficulty. Appendix D.5 examines history encoding separately in a synthetic study and shows that it can affect performance even with matched parameter counts. Different input and output representations are also used in generative recommendation: AsymRec, for example, uses continuous history representations while generating discrete SIDs. Huang et al., 2026

The Amazon experiments provide evidence on three real datasets. Complete-item NLL is the primary outcome because it corresponds to the loss analyzed by the theory and evaluates the complete item prediction across different factorizations. Recall@10 is also reported in Table 19.

Concern 2: Low output ranks and practical relevance

The small output ranks are deliberate experimental controls. They allow us to examine how different SID factorizations respond to the same output-rank restriction. Injective mappings preserve item identity and total predictive information, yet induce conditional prediction problems with different spectral structure. Our purpose is to explain how these differences affect prediction error. Establishing how strongly this mechanism affects particular practical systems is a further empirical question.

Theorem 2 makes this relationship precise under its stated assumptions and is formulated for general rank. Its lower bound holds regardless of how the model computes its features, so this output-rank limitation is not specific to SASRec. The upper bound additionally requires one model to jointly realize the optimal spectral projections. These bounds concern the best predictor in the model class; the experiments examine how the mechanism appears after training. GWT is a diagnostic derived from this analysis.

The controlled experiments demonstrate consequences of this mechanism. In Section 4.2, predictive gain and numerical rank are matched, yet at head rank 2 the concentrated and flat spectral profiles yield mean trained item KL of 0.374 and 0.915, respectively. Section 4.3 fixes the item distribution and code-tree structure: different item-to-code assignments produce different prediction errors despite identical counting statistics.

These comparisons reveal differences in prediction difficulty that numerical rank alone and code-counting statistics cannot capture. Their contribution is to identify and explain those differences under controlled conditions. The analysis suggests favouring mappings whose high-gain prediction steps have conditional logit variation concentrated in directions that the available output rank can represent. Balanced or fully utilized codes alone do not characterize this requirement.

Conclusion

We agree that broader architectural and higher-rank experiments would strengthen the evidence for practical generalization. Our case for acceptance rests on understanding how SID output factorization affects learnability under rank constraints, supported by theoretical bounds and controlled experiments. The Amazon experiments provide additional evidence on real data. We will revise the title, abstract and conclusion to make this scope explicit and limit the practical proxy claims accordingly. We ask you to reconsider your recommendation in light of this contribution and the evidence supporting it.

I appreciate the authors' response.

Generative recommenders that use SIDs as both inputs and outputs have been widely studied and adopted in both academia and industry. Therefore, excluding these models as backbones limits the extent to which the theoretical analysis can be validated. Moreover, the low-rank experimental setting does not adequately reflect real-world scenarios.

Given these concerns, I will maintain my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.