acceptodds
Under review as a conference paper at ICLR 2027

The Update Bottleneck: Re-examining Joint Training for Semantic-ID-Based Generative Recommendation

Abstract

Semantic ID based Generative Retrieval models map items to discrete Semantic IDs (SIDs) and train a sequence model over the codes. Several recent methods train the tokenizer jointly with the recommender and report 9 to 14% gains over two-stage pipelines. We re-examine this comparison under matched conditions. Joint training improves Recall@10 on Beauty without a statistically significant gain against a two-stage method, while Recall@10 falls by approximately 8–16% on the other four datasets. We propose the update bottleneck hypothesis to explain these inconsistent benefits: tokenizer updates may leave discrete IDs unchanged, while frequent ID changes can disrupt the recommender's learning. Our theory formalizes the limits on how tokenizer updates affect a recommender through discrete IDs. Controlled experiments support the update bottleneck and show why gradient flow alone does not ensure useful representation learning in semantic-ID-based generative recommendation.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Generative Retrieval, Generative Recommendation, Semantic IDs, Recommender Systems, Discrete Representation Learning, Joint Training, Vector Quantization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_DJq4OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper revisits the effectiveness of joint training of Semantic ID (SID) tokenizers and generative recommenders, compared with the conventional two-stage training paradigm. Through controlled experiments across five Amazon recommendation datasets, the authors find that joint training does not consistently outperform a carefully tuned two-stage baseline and can even lead to performance degradation.

To explain this phenomenon, the paper proposes the update bottleneck hypothesis, which highlights the tension between discrete SID stability and tokenizer adaptability. Specifically, tokenizer updates may have limited effects on the recommender when discrete assignments remain unchanged, while frequent SID reassignments can disrupt recommender optimization. The authors provide a theoretical analysis of this issue and introduce diagnostic metrics, including exposure-weighted reassignment mass and embedding displacement. Additional experiments investigate gradient connectivity, codebook tying, and different tokenizer update configurations.

The findings provide empirical evidence that joint optimization does not necessarily improve generative recommendation and offer insights into the challenges of learning through discrete representation interfaces.

Critical strengths

The paper revisits an important assumption in Semantic ID-based generative recommendation: that jointly optimizing tokenizers and recommenders should naturally outperform a two-stage training approach. The reported negative results challenge this assumption and provide useful insights into the potential limitations of joint training. In particular, the distinction between gradient connectivity and actual changes in discrete representations offers an interesting perspective on understanding optimization behavior in SID-based recommendation systems.

The paper provides relatively rigorous empirical evaluations, including multiple random seeds, statistical significance analyses, and controlled experiments examining different sources of tokenizer updates. The proposed diagnostic metrics offer a useful way to characterize SID reassignment and representation instability during training. The analyses of codebook tying and delayed tokenizer updates further strengthen the empirical investigation and provide practical insights into the interaction between tokenizer updates and recommender learning.

Critical weaknesses

My primary concern is that the experimental evidence may not be sufficient to support the broader conclusions regarding the limitations of joint training. Although the authors conduct carefully controlled comparisons, the joint-training configurations are not independently optimized for each dataset. In particular, the use of hyperparameters initially tuned on Beauty may disadvantage joint training on other datasets, where the optimal tokenizer learning rate, update frequency, or optimization schedule could differ substantially. Moreover, freezing the tokenizer encoder in the main Beauty joint-training experiments limits the extent to which the framework explores the potential benefits of fully end-to-end optimization. Consequently, the observed performance degradation may partly reflect the specific training configurations rather than an inherent limitation of joint SID learning. Stronger per-dataset tuning and experiments with more flexible joint-training configurations are needed to substantiate the generality of the findings.

Although the update bottleneck hypothesis is intuitively reasonable, its empirical validation remains somewhat limited. The theoretical analysis mainly formalizes properties of discrete interfaces and provides qualitative bounds, but does not establish when the proposed bottleneck necessarily leads to worse recommendation performance. More importantly, the key diagnostic analyses of SID reassignment and embedding displacement are conducted on a limited number of datasets and random seeds. The observed relationship between representation instability and recommendation performance is suggestive, but does not fully establish that the proposed bottleneck is the primary cause of joint-training degradation. Alternative factors, including optimization instability, representation quality, and differences in content-embedding backbones, may also contribute to the results. More systematic experiments that directly manipulate SID stability while controlling other factors would strengthen the proposed explanation.

Questions for authors

Could the authors provide additional joint-training experiments with independently tuned hyperparameters for each dataset, particularly tokenizer learning rates, update schedules, and SID refresh frequencies? Do the observed performance degradations persist after such tuning?

What is the rationale for freezing the tokenizer encoder in the main joint-training experiments on Beauty? How does performance change when the encoder is jointly optimized with the recommender, potentially with appropriate regularization or stabilization techniques?

Could the authors provide more direct evidence that SID reassignment and embedding displacement are responsible for the observed performance degradation? For example, would explicitly controlling the reassignment rate through stability regularization or alternative update schedules improve recommendation performance under otherwise identical conditions?

Since the experiments use different content-embedding backbones across datasets, could the authors evaluate whether the observed differences between Beauty and the other datasets persist when using the same backbone? This would help disentangle dataset-specific effects from differences introduced by the representation models.

Have the authors considered retraining a recommender from scratch using the final SIDs obtained from joint training? Such an experiment could help distinguish whether joint training produces inferior final representations or whether its performance degradation mainly arises from changing SID assignments during recommender optimization.

Could the authors extend the key reassignment and embedding displacement analyses to additional datasets and random seeds? In particular, does the relationship between the proposed diagnostic metrics and downstream recommendation performance remain consistent across different settings?

3 comments

"the joint-training configurations are not independently optimized for each dataset ... freezing the tokenizer encoder ... limits the extent to which the framework explores the potential benefits of fully end-to-end optimization."

We now tune every method separately on each dataset. After tuning, the recommendation gradient into the tokenizer still adds nothing over a tuned two-stage pipeline. We keep the encoder frozen because training it collapses the semantic IDs.

We added untied joint training: the recommender keeps its own embedding tables, while the recommendation gradient still reaches the tokenizer's codebook. We ran it with the recommendation gradient on and off. (With the gradient off, only the recommendation loss is blocked from reaching the tokenizer. The tokenizer still trains on its own losses during recommender training and item codes are still recomputed every epoch.) We used Toys, where joint training had lost in the original setup, and Beauty, where it gained in the original setup.

For these runs, two-stage, tied joint and untied joint training each got their own hyperparameter search (grid in reviewer QQds's last comment). Note that this was also one of the common issues raised by the reviewers, which is that we did not perform hyperparameter search for each individual arms. We do this for values presented in the rebuttal. Each gradient-off arm uses the same setting as its gradient-on counterpart, so we can isolate the effect of recommender gradient reaching the tokenizer alone. For each arm, we run 6 seeds at its best hyperparameter setting. We compute mean differences paired by seed, with 95% paired-t confidence interval.

Toys (differences: arm minus two-stage, paired by seed, [95% CI]; bold: joint training is not significantly better than two-stage, i.e. the interval includes 0 or lies below 0)

ArmRecall@10NDCG@10Recall@10 differenceNDCG@10 difference
Two-stage0.06620.0351——
Untied joint, gradient on0.06720.0353+0.0011 [−0.0009, +0.0030]+0.0001 [−0.0011, +0.0014]
Untied joint, gradient off0.06740.0353+0.0013 [−0.0015, +0.0040]+0.0002 [−0.0011, +0.0015]
Tied joint, gradient on0.06470.0338−0.0015 [−0.0031, +0.0002]−0.0013 [−0.0021, −0.0006]
Tied joint, gradient off0.06230.0323−0.0039 [−0.0050, −0.0027]−0.0029 [−0.0036, −0.0021]

Beauty (same format)

ArmRecall@10NDCG@10Recall@10 differenceNDCG@10 difference
Two-stage0.07010.0387——
Untied joint, gradient on0.07110.0389+0.0010 [−0.0013, +0.0033]+0.0002 [−0.0010, +0.0014]
Untied joint, gradient off0.07090.0389+0.0008 [+0.0002, +0.0013]+0.0002 [−0.0009, +0.0012]
Tied joint, gradient on0.07460.0420+0.0045 [+0.0024, +0.0065]+0.0033 [+0.0023, +0.0042]
Tied joint, gradient off0.07340.0414+0.0033 [+0.0017, +0.0049]+0.0027 [+0.0016, +0.0037]
  • Joint training with untied embeddings equals two-stage training on both datasets: all four NDCG@10 intervals and three of four Recall@10 intervals contain 0. The exception, untied joint training with the gradient off on Beauty, is +0.0008 Recall@10. Please note -- what we see in that cell is that turning the reecommender gradient OFF leads to a small gain over two-stage, so it is not recommender gradient that leads to this grain, but rather the additional training of the tokenizer. Also, there is no significant difference between turning the recommendation gradient on or off.
  • Our runs show that, on Beauty, tying the recommender's embeddings to the codebook is what leads to the gain, with or without the recommendation gradient. So the differences the paper attributed to joint training actually come from tying the recommender's embeddings to the codebook, not from the recommendation gradient reaching the tokenizer. We thank the reviewer for pointing this confound out, because this in itself is an interesting finding that can inform future joint training directions. However, tying the embedding does not help in Toys (and neither tied or untied arms exceed the two-stage arm for Toys). In any case, the recommender gradient passed to the tokenizer is not what leads to the gain, which is what the update bottleneck is about, so our main claim holds, and we will update the paper to say so.
  • Bottomline: in none of our runs, on either dataset, does the recommender gradient passing to the tokenizer, produce a gain over two-stage training.

Why the encoder is frozen. Training the encoder lowers accuracy sharply on Beauty (by about a fifth to a third of NDCG@10), and the drop happens even with the recommendation gradient blocked: the tokenizer's own losses collapse the semantic IDs, so most items stop having a unique code. Stabilizers we tried recovered at most a quarter of the drop. The codebooks still train in every joint run, and DIGER's released scripts also freeze the encoder. We will add these results to the appendix.

"The observed relationship between representation instability and recommendation performance is suggestive, but does not fully establish that the proposed bottleneck is the primary cause of joint-training degradation."

We now measure the disruption directly: when item codes change, the recommender's validation loss jumps at that moment, with nothing else changed. We extended the diagnostics to all seeds and both datasets, directly controlled how often codes change, and tested a shared embedding backbone.

1. Direct evidence: validation loss jumps when codes change. We logged validation loss immediately before and immediately after every code recompute (tuned setting, 6 seeds each). Nothing else changes between the two measurements: the recommender's weights are the same; only the item codes differ.

Toys, tiedToys, untiedBeauty, tiedBeauty, untied
Recomputes that raise validation loss93%93%62%85%
Mean validation-loss rise per recompute0.00260.00640.00070.0023
Items whose final code differs from their initial code10.1%18.3%2.7%7.1%
  • Training-batch loss rises by a similar amount at each recompute (Toys tied 0.0029, untied 0.0072), so the rise is not noise in the validation set.
  • The rise is larger where more codes change: on Toys more than on Beauty, and in untied more than in tied training.
  • The untied runs answer the concern about the tied codebook pathway. Untied training has no continuous pathway from the codebook to the recommender: the recommender reads its own embedding tables. Its jumps can therefore come only from code changes, and they are the largest we observe. This reinforces the paper's claim that changing item codes disrupts the recommender's learning.
  • The recommender then has to recover: in 71–85% of epochs, validation loss falls back below its post-recompute level by the next epoch.

These measurements are the immediate loss change that Equation 5 bounds. Equation 5 does not give the sign of this change, but the measurements show that the loss rises at most recomputes.

2. Repeated changes and reversals. Table 4 compared only each item's first and last code, so it could miss items whose code flips back and forth. We now saved every item's code at every epoch, for all 6 seeds on both datasets. In tied training, 4.8% of Toys items and 0.7% of Beauty items change code two or more times, and 4.1% and 0.7% go back to a code they had before. Most changes happen early in training (Toys: 0.35% of items change code per epoch in epochs 0–9, 0.09% after epoch 75). So code changes are not a one-time switch: some items keep flipping, mostly early, and about seven times more often on Toys than on Beauty. This matches the association in the paper: the dataset with more repeated code changes (Toys) is the one where joint training does worse.

3. Displacement on both datasets, all seeds. We saved every epoch's codebook for all 6 seeds on Toys and Beauty. At the reported checkpoint, codebook vectors have moved, as a share of their initial length (levels 1 / 2 / 3): tied training 3.0% / 9.5% / 17.8% on Toys and 0.8% / 1.7% / 1.8% on Beauty; untied training 6.8% / 10.7% / 13.5% on Toys and 2.1% / 3.7% / 4.3% on Beauty. Displacement is several times larger on Toys, where joint training does worse, matching the paper's Beauty-only observation.

4. Are the final codes inferior, or is adapting to changing codes the problem? We trained a fresh two-stage recommender on each joint run's final codes (Beauty, 6 seeds, paired by seed, paper settings):

RecommenderRecall@10NDCG@10NDCG@10 minus two-stage on original codes
Two-stage, original codes0.07040.0388—
Two-stage, final joint-training codes0.07080.0389+0.0001 (p = 0.83)

On Beauty, the final codes are neither better nor worse than the original codes. So joint training does not result in IDs that are more favorable towards a separate recommender training either.

5. Controlling SID stability. Recomputing codes less often, or never, gives the same final accuracy. The recommender recovers from each setback, but changing codes never bring a lasting gain.

6. Same backbone. With the same Qwen embeddings as the other datasets, Beauty also loses with joint training. The embedding backbone decides whether tying helps; in no case does the recommendation gradient produce the gain.

Reviewer_QQdSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper re-examines whether jointly training a semantic-ID tokenizer and a generative recommender improves recommendation over a tuned two-stage pipeline. Across five Amazon categories, the evaluated joint implementation underperforms two-stage training on four datasets, while improving NDCG@10 on Beauty without a statistically significant Recall@10 improvement. This work proposes the update bottleneck hypothesis to explain the inconsistent benefits of joint-training versus two-stage update: unchanged discrete IDs may limit the influence of tokenizer updates, whereas frequent reassignment may disrupt recommender learning. This work also formalizes interface invariance and a reassignment-induced loss-change bound, and investigates the proposed hypothesis through gradient controls, assignment diagnostics, codebook tying, and delayed tokenizer updates.

Critical strengths
  1. Whether recommendation gradients improve semantic IDs beyond a strong two-stage baseline is an is an important empirical question for generative recommendation. Carefully established negative results would be valuable, and the paper appropriately distinguishes complete-pipeline comparisons from controlled investigations of recommendation gradients. This makes the study relevant even without proposing a new recommendation model.
  2. The main comparisons use six random seeds and paired confidence intervals. The paper reports the positive NDCG result on Beauty alongside the negative results elsewhere, rather than interpreting the absence of a significant Recall improvement as evidence that all joint-training benefits disappear. These choices strengthen the evidence for the narrower conclusion that joint training is not consistently beneficial under the evaluated configurations.
Critical weaknesses

The experiments do not sufficiently separate discrete reassignment from continuous codebook updates, leaving the proposed mechanism underdetermined. Lemma 1 assumes that the recommender accesses the tokenizer only through discrete IDs and uses embedding tables independent of tokenizer parameters. The main joint implementation instead ties input and output embeddings to projected codebook vectors. So, the tokenizer updates can change the recommender’s representations and logits even when IDs remain unchanged. The authors acknowledge this continuous pathway, but its contribution is not sufficiently isolated experimentally.

The separate Beauty comparison in Section 7 makes this distinction consequential: tied joint training reaches NDCG@10 of 0.0409, whereas untied joint training and two-stage training both reach approximately 0.0379. A control that fixes IDs while allowing the tied codebook and projection to receive recommendation updates would help determine how much of the observed behavior requires reassignment. Without this separation, the discrete-interface theory provides useful intuition but does not adequately explain the main tied implementation’s results.

The evidence does not yet distinguish reassignment-induced learning disruption from alternative causes of the negative results. Table 3 associates lower reassignment mass with higher accuracy, but the gradient intervention also changes the optimization trajectory and codebook geometry. Table 4 measures final-versus-post-warmup ID differences, which do not capture repeated changes or reversals central to the moving-target explanation. The more relevant displacement diagnostics cover only Beauty and two diagnostic seeds. Delayed updates on Toys improve joint NDCG@10 from 0.0301 to 0.0309, still below the two-stage result of 0.0339, but also alter tokenizer update counts and potentially the learning-rate phase; paired uncertainty for this improvement is not reported.

In particular, the paper does not establish whether the final joint-training IDs are inferior or whether the recommender struggles to adapt to useful but changing IDs. Retraining a recommender on frozen final IDs would directly help distinguish these explanations. Appendix C.1 also distinguishes training-time quantizer assignments from exported nearest-neighbor IDs, leaving training–evaluation mismatch as another possible contributor that needs clarification. These gaps limit the explanatory contribution beyond the observed performance comparisons.

Questions for authors

Please see critical weaknesses. Here are additional questions:

  1. In each Table 3 condition, which assignment map supplies recommender histories, teacher-forced decoder inputs, and prediction targets? In particular, how are changing online assignments and the static deployed SID table used in frozen/on, and what is their agreement at evaluation?

  2. What tokenizer learning-rate and update-frequency searches were performed for the modified joint implementation, as distinct from released DIGER? Please specify the search ranges, training budgets, and checkpoint-selection rules, since these affect the interpretation of the negative results.

  3. Do you have results for fixed IDs with trainable tied codebooks/projections, or for retraining an independent-embedding recommender on frozen final joint-training IDs? The former would separate continuous updates from reassignment; the latter would distinguish final-ID quality from adaptation difficulty. Results on Beauty and one failure dataset would be particularly informative.

4 comments

"The experiments do not sufficiently separate discrete reassignment from continuous codebook updates, leaving the proposed mechanism underdetermined."

We separated the two pathways with untied training, which is the setting of Lemma 1: the recommender keeps its own embedding tables, so the tokenizer reaches it only through item codes. Through codes alone, the recommendation gradient neither helps nor hurts, and code changes disrupt training. The Beauty gain the reviewer points to comes from the continuous pathway that tying adds.

To separate the two differences, we added untied joint training: the recommender keeps its own embedding tables, while the recommendation gradient still reaches the tokenizer's codebook. We ran it with the recommendation gradient on and off. (With the gradient off, only the recommendation loss is blocked from reaching the tokenizer. The tokenizer still trains on its own losses during recommender training and item codes are still recomputed every epoch.) We used Toys, where joint training had lost in the original setup, and Beauty, where it gained in the original setup.

For these runs, two-stage, tied joint and untied joint training each got their own hyperparameter search (grid below). Note that this was also one of the common issues raised by the reviewers, which is that we did not perform hyperparameter search for each individual arms. We do this for values presented in the rebuttal. Each gradient-off arm uses the same setting as its gradient-on counterpart, so we can isolate the effect of recommender gradient reaching the tokenizer alone. For each arm, we run 6 seeds at its best hyperparameter setting. We compute mean differences paired by seed, with 95% paired-t confidence interval.

Toys (differences: arm minus two-stage, paired by seed, [95% CI]; bold: joint training is not significantly better than two-stage, i.e. the interval includes 0 or lies below 0)

ArmRecall@10NDCG@10Recall@10 differenceNDCG@10 difference
Two-stage0.06620.0351——
Untied joint, gradient on0.06720.0353+0.0011 [−0.0009, +0.0030]+0.0001 [−0.0011, +0.0014]
Untied joint, gradient off0.06740.0353+0.0013 [−0.0015, +0.0040]+0.0002 [−0.0011, +0.0015]
Tied joint, gradient on0.06470.0338−0.0015 [−0.0031, +0.0002]−0.0013 [−0.0021, −0.0006]
Tied joint, gradient off0.06230.0323−0.0039 [−0.0050, −0.0027]−0.0029 [−0.0036, −0.0021]

Beauty (same format)

ArmRecall@10NDCG@10Recall@10 differenceNDCG@10 difference
Two-stage0.07010.0387——
Untied joint, gradient on0.07110.0389+0.0010 [−0.0013, +0.0033]+0.0002 [−0.0010, +0.0014]
Untied joint, gradient off0.07090.0389+0.0008 [+0.0002, +0.0013]+0.0002 [−0.0009, +0.0012]
Tied joint, gradient on0.07460.0420+0.0045 [+0.0024, +0.0065]+0.0033 [+0.0023, +0.0042]
Tied joint, gradient off0.07340.0414+0.0033 [+0.0017, +0.0049]+0.0027 [+0.0016, +0.0037]
  • Joint training with untied embeddings equals two-stage training on both datasets: all four NDCG@10 intervals and three of four Recall@10 intervals contain 0. The exception, untied joint training with the gradient off on Beauty, is +0.0008 Recall@10. Please note -- what we see in that cell is that turning the reecommender gradient OFF leads to a small gain over two-stage, so it is not recommender gradient that leads to this grain, but rather the additional training of the tokenizer. Also, there is no significant difference between turning the recommendation gradient on or off.
  • Our runs show that, on Beauty, tying the recommender's embeddings to the codebook is what leads to the gain, with or without the recommendation gradient. So the differences the paper attributed to joint training actually come from tying the recommender's embeddings to the codebook, not from the recommendation gradient reaching the tokenizer. We thank the reviewer for pointing this confound out, because this in itself is an interesting finding that can inform future joint training directions. However, tying the embedding does not help in Toys (and neither tied or untied arms exceed the two-stage arm for Toys). In any case, the recommender gradient passed to the tokenizer is not what leads to the gain, which is what the update bottleneck is about, so our main claim holds, and we will update the paper to say so.
  • Bottomline: in none of our runs, on either dataset, does the recommender gradient passing to the tokenizer, produce a gain over two-stage training.

Direct evidence: validation loss jumps when codes change. We logged validation loss immediately before and immediately after every code recompute (tuned setting, 6 seeds each). Nothing else changes between the two measurements: the recommender's weights are the same; only the item codes differ.

Toys, tiedToys, untiedBeauty, tiedBeauty, untied
Recomputes that raise validation loss93%93%62%85%
Mean validation-loss rise per recompute0.00260.00640.00070.0023
Items whose final code differs from their initial code10.1%18.3%2.7%7.1%
  • Training-batch loss rises by a similar amount at each recompute (Toys tied 0.0029, untied 0.0072), so the rise is not noise in the validation set.
  • The rise is larger where more codes change: on Toys more than on Beauty, and in untied more than in tied training.
  • The untied runs answer the concern about the tied codebook pathway. Untied training has no continuous pathway from the codebook to the recommender: the recommender reads its own embedding tables. Its jumps can therefore come only from code changes, and they are the largest we observe. This reinforces the paper's claim that changing item codes disrupts the recommender's learning, and we thank the reviewer for pointing to this pathway.
  • The recommender then has to recover: in 71–85% of epochs, validation loss falls back below its post-recompute level by the next epoch.

These measurements are the immediate loss change that Equation 5 bounds. Equation 5 does not give the sign of this change, but the measurements show that the loss rises at most recomputes.

Why fixing the codes does not remove the disruption. We ran the suggested control (codes fixed, tied codebook still training; 6 seeds). Its accuracy equals that of recomputing codes every epoch (NDCG@10: Toys 0.0337 vs 0.0338; Beauty 0.0417 vs 0.0420). But this control does not hold the item representations fixed. The tokenizer's own losses keep pulling each codebook vector toward the items the tokenizer would now assign to it. An item keeps its old code, but the vector behind that code moves toward other items. The disruption changes form rather than disappearing, so equal accuracy in this control does not show that code changes are harmless.

How this answers the reviewer's question.

  • Lemma 1's setting is untied training. Our experiments on untied training shows that the recommendation gradient into the tokenizer does not change performance (tables in previous comment), and the recommendation-loss gradient reaching the codebook is 2,000–5,500 times weaker than with tying, at every stage of training. Tokenizer updates reach the recommender only through code changes, and those raise its loss (table above).
  • The Section 7 Beauty comparison has now been tuned separately for hyperparameters, and repeated on 6 seeds: tied-embedding joint training reaches NDCG@10 0.0420, untied-embedding joint training 0.0389 and two-stage 0.0387. The tied gain remains with the recommendation gradient off (0.0414) and with codes fixed (0.0417), so it comes from the continuous pathway that tying the embeddings provide, not from code changes or the gradient. This in itself is an interesting finding, and we thank the reviewer for raising this point.
  • We will revise the paper to state that Lemma 1 and the update bottleneck describe the untied-training, and that the tied-embedding implementation adds a continuous pathway, which is associated with its Beauty gain. We thank the reviewer for this distinction.

"The evidence does not yet distinguish reassignment-induced learning disruption from alternative causes of the negative results."

We measured the effect of code changes at the moment they happen, when nothing else changes: validation loss rises immediately when item codes are recomputed. This rules out the alternative causes the reviewer lists, because between the two measurements the optimization trajectory, codebook geometry, update counts and learning-rate phase are all identical. Retraining on the final codes shows they are not inferior, and training and evaluation read the same code table.

1. Direct evidence that isolates code changes. Our previous reply shows that validation loss rises immediately at most code recomputes (93% of recomputes on Toys, 62–85% on Beauty), and most in untied training, where code changes are the only way the tokenizer reaches the recommender. Only the code table changes between the two measurements: the recommender's weights, the optimization trajectory, the codebook geometry, the update counts and the learning-rate phase are all the same. So the rise cannot come from the alternative causes the reviewer lists.

2. Repeated changes and reversals. Table 4 compared only each item's first and last code, so it could miss items whose code flips back and forth. We now saved every item's code at every epoch, for all 6 seeds on both datasets. In tied training, 4.8% of Toys items and 0.7% of Beauty items change code two or more times, and 4.1% and 0.7% go back to a code they had before. Most changes happen early in training (Toys: 0.35% of items change code per epoch in epochs 0–9, 0.09% after epoch 75). So code changes are not a one-time switch: some items keep flipping, mostly early, and about seven times more often on Toys than on Beauty. This matches the association in the paper: the dataset with more repeated code changes (Toys) is the one where joint training does worse/

3. Displacement on both datasets, all seeds. We now measure how far the codebook vectors move during training for all 6 seeds on both Toys and Beauty, not only Beauty. On Toys, where joint training does worse, the vectors move several times farther than on Beauty: up to about 18% of their initial length versus under 2% in tied training. This matches the paper's Beauty-only observation: more movement goes with worse joint training. We will include these results in the main paper.

4. Are the final codes inferior, or is adapting to changing codes the problem? We trained a fresh two-stage recommender on each joint run's final codes (Beauty, 6 seeds, paired by seed, paper settings):

RecommenderRecall@10NDCG@10NDCG@10 minus two-stage on original codes
Two-stage, original codes0.07040.0388—
Two-stage, final joint-training codes0.07080.0389+0.0001 (p = 0.83)

On Beauty, the final codes are neither better nor worse than the original codes. So joint training does not result in IDs that are more favorable towards a separate recommender training either.

5. Training and evaluation use the same codes (Appendix C.1). One stored code table supplies the history inputs, the teacher-forced decoder inputs, the prediction targets and the decoding constraints at evaluation. It is recomputed once per epoch with Sinkhorn-balanced assignment (Sinkhorn balancing spreads items evenly over codes; it is the default in the public LETTER and DIGER code). The assignment computed inside each training batch enters only the backward pass, through the straight-through estimator. The table used for training matches the table used for evaluation, both in joint training and in two-stage training. Appendix C.1 wrongly called this table nearest-neighbor; we will correct it.

6. Delayed updates. At the tuned setting, 6 seeds paired by seed on Toys, delaying tokenizer updates by 25 epochs does not change NDCG by a significant amount, and we concede this point: at the tuned setting, delay does not improve joint training, and we will replace the earlier result. This does not change the central message of the paper. The recommendation gradient passed to the tokenizer gives no consistent gain over a tuned two-stage pipeline, and the direct evidence that code changes disrupt training is the validation-loss measurement in the previous comment, which does not depend on delay any more.

Hyperparameter search used for every result. One seed (2020) per hyperparameter setting, trained to convergence with early stopping of 20 epochs, best checkpoint/setting chosen by NDCG@10; the winner then ran 6 seeds (2020–2025). The same grid was used on Toys and Beauty.

StepHyperparameters searchedValuesArms
1Recommender learning rate × warmup steps{5e-4, 1e-3, 2e-3} × {500, 1500} (6 settings)Two-stage, tied joint, untied joint
2Tokenizer learning rate × epochs before tokenizer updates start{1e-6, 1e-5, 1e-4} × {0, 25, 50} (9 settings), at the step-1 winnerTied joint, untied joint
3How often item codes are recomputedevery 1, 5 or 10 epochs, at the step-2 winnerTied joint (both datasets), untied joint (Toys)

Two-stage training has no step 2 or 3: its tokenizer is frozen and its codes never change.

Settings used for the 6-seed runs (similar to the paper's main runs, tokenizer updates start at epoch 0 and codes are recomputed every epoch; the best tokenizer learning rate under that condition was used):

ArmRecommender learning rateWarmup stepsTokenizer learning rate
Two-stage2e-3500— (frozen)
Tied joint (gradient on and off)2e-35001e-6
Untied joint (gradient on and off)1e-315001e-6

The same settings won on Toys and on Beauty.

Reviewer_oybAOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper revisits joint tokenizer-recommender training for semantic-ID-based generative recommendation. It compares a tuned two-stage baseline with a modified, gradient-connected DIGER implementation across five Amazon categories, then studies whether SID reassignment helps explain the inconsistent gains. The proposed update bottleneck combines limited changes through a discrete interface with disruption from changing inputs and prediction targets. The analysis includes interface invariance, a loss-change bound, gradient controls, and delayed tokenizer updates.

Critical strengths

The implementation audit distinguishes recommendation gradients from tokenizer self-loss updates, which is useful for evaluating claims of end-to-end learning. Reporting six-seed results and paired intervals also makes the pipeline comparisons more informative than comparisons against published scores alone.

Critical weaknesses
  1. The negative comparison confounds recommendation feedback with a different embedding interface. The joint arm ties input and output embeddings to projected codebooks, whereas two-stage training learns its own token embeddings. Although Appendix B reports a shared learning-rate grid on the other datasets, the modified joint implementation inherits its initial recipe from released DIGER, whose recommendation gradient the audit finds disconnected, without a separate search for the new implementation. The untied control is useful but only covers Beauty, where joint training helps, not the four datasets driving the negative finding. Likewise, moving/on versus moving/off compares gradients within the modified interface rather than isolating joint training against two-stage training. It remains unclear whether the reported deficits arise from joint tokenizer optimization or this particular interface and training recipe.

  2. The update bottleneck remains an interpretation of correlations rather than a demonstrated explanation of the deficits. The invariance lemma follows directly from fixing everything the recommender observes, and Equation 5 is an unsigned, immediate loss-change bound, not a prediction of training degradation. Table 4 measures endpoint disagreement rather than repeated reassignment, while the displacement measurements in Table 5 cover only Beauty. More importantly, the main tied interface has a continuous codebook-vector pathway that the paper explicitly excludes from the reassignment bound. Delaying tokenizer updates on Toys affects this pathway as well as SID changes, so the partial recovery does not isolate the proposed mechanism. Since instability was already recognized in the cited joint-training work, the present theory and diagnostics offer limited explanatory advance without distinguishing these effects.

Questions for authors
  1. On one dataset where joint training loses, does the deficit persist with untied embeddings and an on/off recommendation-gradient comparison from the same initialization and training budget, after tuning the gradient-connected arm itself?
  2. Can you isolate SID reassignment from continuous embedding drift on Toys, for example by keeping input and target SID assignments fixed while allowing the tied codebook vectors to keep updating? This would help determine whether reassignment is responsible for the reported degradation.
4 comments

W1: "The negative comparison confounds recommendation feedback with a different embedding interface..."

We agree that tying was a confound. Once it is removed, joint training matches two-stage training: the recommendation gradient into the tokenizer neither helps nor hurts, which is consistent with our main claim that joint training gives no consistent gain over a tuned two-stage pipeline.

To separate the two differences, we added untied joint training: the recommender keeps its own embedding tables, while the recommendation gradient still reaches the tokenizer's codebook. We ran it with the recommendation gradient on and off. (With the gradient off, only the recommendation loss is blocked from reaching the tokenizer. The tokenizer still trains on its own losses during recommender training and item codes are still recomputed every epoch.) We used Toys, where joint training had lost in the original setup, and Beauty, where it gained in the original setup.

For these runs, two-stage, tied joint and untied joint training each got their own hyperparameter search (grid below). Note that, to strengthen our claim, we now perform hyperparameter search for each arm. Each gradient-off arm uses the same setting as its gradient-on counterpart, so we can isolate the effect of recommender gradient reaching the tokenizer alone. For each arm, we run 6 seeds at its best hyperparameter setting. We compute mean differences paired by seed, with 95% paired-t confidence interval.

Toys (differences: arm minus two-stage, paired by seed, [95% CI]; bold: joint training is not significantly better than two-stage, i.e. the interval includes 0 or lies below 0)

ArmRecall@10NDCG@10Recall@10 differenceNDCG@10 difference
Two-stage0.06620.0351——
Untied joint, gradient on0.06720.0353+0.0011 [−0.0009, +0.0030]+0.0001 [−0.0011, +0.0014]
Untied joint, gradient off0.06740.0353+0.0013 [−0.0015, +0.0040]+0.0002 [−0.0011, +0.0015]
Tied joint, gradient on0.06470.0338−0.0015 [−0.0031, +0.0002]−0.0013 [−0.0021, −0.0006]
Tied joint, gradient off0.06230.0323−0.0039 [−0.0050, −0.0027]−0.0029 [−0.0036, −0.0021]

Beauty (same format)

ArmRecall@10NDCG@10Recall@10 differenceNDCG@10 difference
Two-stage0.07010.0387——
Untied joint, gradient on0.07110.0389+0.0010 [−0.0013, +0.0033]+0.0002 [−0.0010, +0.0014]
Untied joint, gradient off0.07090.0389+0.0008 [+0.0002, +0.0013]+0.0002 [−0.0009, +0.0012]
Tied joint, gradient on0.07460.0420+0.0045 [+0.0024, +0.0065]+0.0033 [+0.0023, +0.0042]
Tied joint, gradient off0.07340.0414+0.0033 [+0.0017, +0.0049]+0.0027 [+0.0016, +0.0037]
  • Joint training with untied embeddings equals two-stage training on both datasets: all four NDCG@10 intervals and three of four Recall@10 intervals contain 0. The exception, untied joint training with the gradient off on Beauty, is +0.0008 Recall@10. Please note -- what we see in that cell is that turning the reecommender gradient OFF leads to a small gain over two-stage, so it is not recommender gradient that leads to this grain, but rather the additional training of the tokenizer. Also, there is no significant difference between turning the recommendation gradient on or off.
  • Our runs show that, on Beauty, tying the recommender's embeddings to the codebook is what leads to the gain, with or without the recommendation gradient. So the differences the paper attributed to joint training actually come from tying the recommender's embeddings to the codebook, not from the recommendation gradient reaching the tokenizer. We thank the reviewer for pointing this confound out, because this in itself is an interesting finding that can inform future joint training directions. However, tying the embedding does not help in Toys (and neither tied or untied arms exceed the two-stage arm for Toys). In any case, the recommender gradient passed to the tokenizer is not what leads to the gain, which is what the update bottleneck is about, so our main claim holds, and we will update the paper to say so.
  • Bottomline: in none of our runs, on either dataset, does the recommender gradient passing to the tokenizer, produce a gain over two-stage training.
  • We were not able to finish the runs on the other 3 datasets, but will be happy to include it in the final version.

Shared hyperparameter grid

We share the hyperparameter search grid and the selected hyperparameters fo each run in the next comment.

Hyperparameter search used for every result. One seed (2020) per hyperparameter setting, trained to convergence with early stopping of 20 epochs, best checkpoint/setting chosen by NDCG@10; the winner then ran 6 seeds (2020–2025). The same grid was used on Toys and Beauty.

StepHyperparameters searchedValuesArms
1Recommender learning rate × warmup steps{5e-4, 1e-3, 2e-3} × {500, 1500} (6 settings)Two-stage, tied joint, untied joint
2Tokenizer learning rate × epochs before tokenizer updates start{1e-6, 1e-5, 1e-4} × {0, 25, 50} (9 settings), at the step-1 winnerTied joint, untied joint
3How often item codes are recomputedevery 1, 5 or 10 epochs, at the step-2 winnerTied joint (both datasets), untied joint (Toys)

Two-stage training has no step 2 or 3: its tokenizer is frozen and its codes never change.

Settings used for the 6-seed runs (similar to the paper's main runs, tokenizer updates start at epoch 0 and codes are recomputed every epoch; the best tokenizer learning rate under that condition was used):

ArmRecommender learning rateWarmup stepsTokenizer learning rate
Two-stage2e-3500— (frozen)
Tied joint (gradient on and off)2e-35001e-6
Untied joint (gradient on and off)1e-315001e-6

The same settings won on Toys and on Beauty.

"The update bottleneck remains an interpretation of correlations rather than a demonstrated explanation of the deficits."

We agree that the reassignment metric in the paper is correlational. Like we mention in the original paper, the metric was intended to describe an association, not to establish causation, and the association still holds in the new hyperparameter-tuned runs. The direct evidence that says changing item codes disrupts the recommender's training is the validation loss at each code recompute: when item codes change, the recommender's validation loss jumps immediately, and the recommender must then readjust.

1. Direct evidence: validation loss jumps when codes change. We logged validation loss immediately before and immediately after every code recompute (these were run on our new tuned setting, and our observations come from all 6 seeds). Note that nothing else changes between the two measurements: the recommender's weights are the same, and only the item codes differ. We refer to the paper’s original “tied embedding” setting as tied, and the untied embedding setting as untied

Toys, tiedToys, untiedBeauty, tiedBeauty, untied
Recomputes that raise validation loss93%93%62%85%
Mean validation-loss rise per recompute0.00260.00640.00070.0023
Items whose final code differs from their initial code10.1%18.3%2.7%7.1%
  • Training-batch loss rises by a similar amount at each recompute (Toys tied 0.0029, untied 0.0072), so the rise is not noise in the validation set.
  • The rise is larger where more codes change: on Toys more than on Beauty, and in untied more than in tied training.
  • The untied runs answer the concern about the tied codebook pathway. Untied training has no continuous pathway from the codebook to the recommender: the recommender reads its own embedding tables. Its jumps can therefore come only from code changes, and they are the largest that we observe. This reinforces the original paper's claim that changing item codes disrupts the recommender's learning, and we thank the reviewer for pointing to this pathway.
  • The recommender then has to recover. We see that in 71–85% of all epochs, validation loss falls back below its post-recompute level by the next epoch.

These measurements are the immediate loss change that Equation 5 bounds. Equation 5 does not give the sign of this change, but the measurements show that the loss rises at most recomputes.

2. The association in the paper still holds in the tuned runs. In the same setting as Table 3 (tied training, recommendation gradient on vs. off), the run with fewer code changes has the higher accuracy on both datasets:

Tied trainingItems whose final code differs from their initial code, gradient on / offNDCG@10, gradient on / off
Toys10.1% / 34.1%0.0338 / 0.0323
Beauty2.7% / 7.3%0.0420 / 0.0414

We will state more explicitly in the paper that this metric shows association, and point to the validation-loss measurements as the direct evidence.

3. Why fixing the codes does not remove the disruption. We ran the suggested control that reviewers suggested, which is to keep codes fixed, but the tied codebook still keeps training. We perform this on 6 seeds. Its performance equals that of recomputing codes every epoch (NDCG@10: Toys 0.0337 vs 0.0338; Beauty 0.0417 vs 0.0420). But we would like to respectfully point out to the reviewers that this control does not hold the item representations fixed, even if the item codes are held fixed. The tokenizer's own losses keep pulling each codebook vector toward the items the tokenizer would now assign to it. So despite an item keeping the same code, the embedding behind that code moves toward other items. The disruption changes form rather than disappearing, so equal accuracy in this control does not show that code changes are harmless.

4. Other points.

  • "Regarding the concern that Table 4 measures endpoint disagreement rather than repeated reassignment": We agree that Table 4 compared only each item's first and last code, so it could miss items whose code flips back and forth. We now saved every item's code at every epoch. In tied training, 4.8% of Toys items and 0.7% of Beauty items change code two or more times, and 4.1% and 0.7% go back to a code they had before. Most changes happen early in training (Toys: 0.35% of items change code per epoch in epochs 0–9, 0.09% after epoch 75). So code changes are not a one-time switch: some items keep flipping, mostly early, and about seven times more often on Toys than on Beauty. This further matches the association in the paper: the dataset with more repeated code changes (Toys) is the one where joint training does worse.
  • Delayed updates: we agree that delay changes both code changes and codebook vectors. We do not rely on it, since the validation-loss jumps isolate the moment codes change.
  • Prior work on instability: prior work reports that joint training can be unstable. We locate where it happens, at each code recompute, and measure its size. Prior work also reports that passing the recommendation gradient to the tokenizer improves performance on public benchmarks. Under matched conditions (every method tuned on the same grid, 6 seeds, paired confidence intervals), we do not find that gain: every gain we measure over two-stage training also appears with the gradient blocked. Our analysis aims to show why. Our message is that a tuned two-stage pipeline is the baseline that joint-training methods should be compared against under matched tuning.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

General response to all reviewers

Our paper asks a representation-learning question in generative recommendation: can the recommendation loss, passed back into the tokenizer, learn better semantic IDs than a tokenizer trained on its own? Several recent methods train the two jointly, passing the recommendation gradient into the tokenizer, and report gains over two-stage training. We re-examine this reported gain under matched conditions and ask where any difference comes from.

Strengths the reviewers noted:

  • An important empirical question, where carefully established negative results are valuable (QQdS, DJq4).
  • The implementation audit separates recommendation gradients from the tokenizer's own updates (oybA), and the distinction between gradient connectivity and actual changes in discrete representations is an interesting perspective (DJq4).
  • Six seeds, paired confidence intervals and significance tests (oybA, QQdS, DJq4), and the positive Beauty result reported alongside the negative ones (QQdS).
  • Useful diagnostics of reassignment and instability, and controlled analyses of tying and delayed updates (DJq4).

Concerns raised by reviewers, what we did, and our findings:

  1. Joint training was not hyperparameter tuned for itself or per dataset (oybA, QQdS, DJq4). We searched every method separately on our two headline datasets, including tokenizer learning rate, update schedule and code-refresh frequency. After tuning, the recommendation gradient into the tokenizer still adds nothing over a tuned two-stage pipeline, as the paper found.
  2. Tying the recommender's embeddings to the codebook may explain the results (oybA, QQdS). We added joint training without tying, with the recommendation gradient on and off. Without tying, joint training equals two-stage training, and the gradient makes no difference. With tying, results move up on Beauty (which was already reported in the paper), and it moves down on Toys, and they do so even with the gradient blocked. So the gains and losses come from tying, not from the gradient; the paper's central claim about joint gradient holds, and we thank the reviewers for this distinction.
  3. The evidence that code changes disrupt training is only correlational (oybA, QQdS, DJq4). We measured validation loss immediately before and after every code recompute, when nothing else changes. The loss usually rises, and the recommender then has to recover. This is direct evidence for the disruption the update bottleneck describes. The paper's association between more code changes and lower performance still holds in our new runs.
  4. Fixing codes while the tied codebook trains, and controlling how often codes change (oybA, QQdS, DJq4). We ran both experiments. Final performance does not change, because the recommender recovers from each setback while the changing codes bring no lasting gain. We also explain why fixing codes does not remove disruption: the vector behind each fixed code keeps moving toward other items.
  5. Diagnostics covered few datasets and seeds, and only first and last codes (oybA, QQdS, DJq4). We saved every epoch's codes and codebooks for all seeds on both datasets. Items flip back and forth, and both repeated changes and codebook movement are several times larger on Toys, where joint training does worse, matching the paper's association.
  6. Are final joint-training codes worse? (QQdS, DJq4). A fresh recommender trained on the final codes does no better and no worse than one trained on the original codes, so joint training does not result in better codes that help a separately trained recommender.
  7. Different embedding backbones across datasets (DJq4). With the same backbone as the other datasets, Beauty also loses with joint training.
  8. Encoder freezing, code tables, theory, delay (oybA, QQdS, DJq4). Training the encoder collapses the semantic IDs, so it stays frozen. Training and evaluation read the same code table, and we correct the Appendix C.1 wording. The new measurements give the sign of the loss change that was missing in our bound: we see the loss change is positive. At the tuned setting, delaying tokenizer updates gives no gain; we concede this point, which does not affect the central message.

Findings: Across tuned, matched comparisons, passing the recommendation gradient into the tokenizer does not give a consistent gain over a tuned two-stage pipeline, and changing codes disrupt the recommender without lasting benefit. Every rebuttal result uses six seeds, a separate hyperparameter search for every method we report, and paired 95% confidence intervals. Based on these result, we believe that joint tokenizer-recommender methods should report results against a two-stage pipeline tuned with the same care, and separate the effect of the gradient from that of architectural choices such as tying. Only then can a reported gain be attributed to joint training.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.