acceptodds
Desk rejected at ICLR 2027

MLPs are Efficient Distilled Generative Recommenders

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Generative recommendation models employing Semantic IDs (SIDs) exhibit strong potential, yet their practical deployment is bottlenecked by the high inference latency of beam-expanded autoregressive decoding. In this work, we identify that standard attention-heavy Transformer decoders represent a structural overkill for this task: the hierarchical nature of SIDs makes prediction difficulty drop sharply after the first token, rendering repeated attention computations highly redundant. Driven by this insight, we propose SID-MLP, a lightweight MLP-centric distillation framework that fundamentally simplifies the decoding paradigm for GR. Instead of executing complex, step-by-step attention mechanisms, our approach captures the global user context in a single operation, decoupled from sequential token prediction. We then distill the heavy autoregressive teacher into position-specific MLP heads, eliminating the dense attention overhead while preserving prefix and context dependencies. Extensive experiments demonstrate that SID-MLP matches the accuracy of teacher models while accelerating inference by . Crucially, this distillation strategy can serve as a plug-and-play accelerator for different backbones and tokenizer settings. Furthermore, we introduce SID-MLP++, extending our distillation framework to replace the Transformer encoder, unlocking further latency reductions. Ultimately, our work reveals that decoder-side MLPs distillation is an effective acceleration path for structured SID recommendation, while full encoder replacement offers an additional speed–accuracy trade-off. Our code is available at: https://anonymous.4open.science/r/SID-MLP.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Generative Recommendation, Efficient Inference, Distillation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_2dXROpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the inference overhead of autoregressive decoding in SID-based generative recommendation. The authors observe that the number of valid SID continuations decreases rapidly after the first few tokens, suggesting that a full Transformer decoder may be unnecessary. Based on this observation, they propose SID-MLP, which replaces the Transformer decoder with position-specific MLP heads. The model computes user context once and conditions each head on the previously generated SID tokens. The MLP heads are trained through knowledge distillation from a frozen teacher model. Experiments on three Amazon datasets show an 8.74× throughput improvement over TIGER-kv with comparable recommendation accuracy. The authors also introduce SID-MLP++, which further replaces the Transformer encoder.

Critical strengths
  1. Achieves substantial inference speedup (8.74×) and memory reduction (95.7 %) while preserving recommendation quality.
  2. Provides clear motivation through analyses of SID branching factors and decoder redundancy.
  3. Includes comprehensive ablation studies that validate the key architectural choices.
  4. Evaluates different tokenizers, teacher architectures, and SID configurations.
Critical weaknesses
  1. Several acceleration baselines require nontrivial adaptations to TIGER, making their reported performance potentially sensitive to implementation choices.
  2. SID-MLP recovers only 83.3%–97.5% of the teacher's NDCG@10 on LC-Rec 7B, indicating non-negligible performance degradation.
  3. The position-specific MLP heads limit direct model reuse when SID lengths or tokenization schemes change.
Questions for authors
  1. Can SID-MLP adapt to longer or modified SID structures without retraining its position-specific MLP heads?
  2. What causes the accuracy degradation of SID-MLP on LC-Rec 7B?
  3. How sensitive are the baseline results to the architectural adaptations made for TIGER?
2 comments

We thank the reviewer for the careful assessment.

In the responses below, we show that SID-MLP maintains a strong quality–efficiency advantage across teacher architectures and SID structures, requiring only lightweight redistillation when the structure changes.

W1/Q3. Sensitivity of baselines. We agree that architectural adaptations might affect baseline results, so we additionally tested AtSpeed-S/R on native decoder-only 7B LC-Rec. We used its original 68M LLaMA draft and retained strict verification and relaxed rejection sampling. We profiled all methods on the same A6000 with the batch size 32 protocol used throughout our paper.

MethodNDCG@10 recovery (Instr./Games/Arts)End-to-end sanples/sSpeedupOffline A6000 GPU-h
AtSpeed-S, N=40100.00% / 100.00% / 100.00%8.7494.117×24.75
AtSpeed-R, N=2087.63% / 83.78% / 87.29%9.8404.630×26.28
AtSpeed-R, N=4081.07% / 82.38% / 79.73%10.0084.709×26.28
SID-MLP97.44% / 83.18% / 88.74%44.26120.830×3.85

AtSpeed performs better on its original 7B backbone than in the TIGER adaptation as expected. However, SID-MLP achieves higher throughput and much lower offline training cost. Section 4.3 (Table 5) also reports NEZHA's LC-Rec results, with 64.96%–82.67% teacher recovery, below SID-MLP's 83.18%–97.44%; Appendices F.1–F.3 document the original TIGER-side baseline adaptations.

Together, these matched comparisons demonstrate that SID-MLP can be readily adapted to larger LLM-based generative recommenders while retaining its quality–efficiency advantage. We have added the native AtSpeed comparison to Section 4.3 (Table 5) and its N=40N=40 operating point to Appendix D.2.

W2/Q2. The LC-Rec quality gap. As we discussed in Section 4.3, the gap is mainly caused by 159× teacher–student compression. We agree that some gap remains on 7B LC-Rec, and it should be considered together with model size and inference cost. SID-MLP distills the 7B autoregressive teacher into only 44.1M trainable parameters (about 0.6% of the teacher) and removes online 7B decoder execution. Despite this compression, it recovers 83.18%–97.44% of teacher NDCG@10 with substantial speedup. 

AtSpeed-R and NEZHA also show quality gaps on LC-Rec, illustrating the speed–quality trade-off when accelerating large teachers. The key advantage of SID-MLP is that it preserves most of the teacher's performance using only 0.6% of its trainable parameters, while providing substantial verify-free acceleration. This supports our main observation that full 7B Transformer decoding is unnecessary at every SID position. We further added the one-shot diagnostic to Appendix D.2 and the position-wise results to Table 19.

W3/Q1. Low-cost adaptation to a changed teacher. We clarify that changing the SID length or tokenizer first changes the teacher itself: the new structure defines different output positions, label spaces, or token semantics, so the generative teacher must also be trained or adapted accordingly.

SID-MLP then follows this updated teacher through the same lightweight redistillation pipeline, without any architectural redesign. However, this retraining is lightweight, and the performance remains robust across different structures. Appendix D.3 (Tables 20–23) evaluates these adaptations:

ChangeTested structuresNDCG@10 recovery
SID length2×256 / 3×256 / 6×64102.4% / 102.8% / 99.8%
Codebook width128 / 256 / 512101.6% / 102.8% / 102.3%
Heterogeneous width[64,128,256] / [256,128,64]100.3% / 101.4%
TokenizerRQ-VAE / RQ-KMeans / PSID-OPQ100.0% / 102.8% / 101.5%
Multimodal SIDUnified / MACRec dual-branch100.1% / 100.2%

All tested structures recover at least 99.8% of their matched teacher's NDCG@10. Thus, SID-MLP can adapt to teachers with new structures through the same distillation framework.

The cost of redistillation is low. Appendix C.2 reports that training the base SID-MLP student takes ~1 hour per dataset on average. Appendix E.2 (Table 33) further shows that SID-MLP requires only 4.48 PFLOP of additional offline compute, lower than the 6.78–66.17 PFLOP required by the other baselines. It also saves about 4.42 GFLOP per online request, so the offline compute is recovered after ~1M compute-equivalent requests. These results show that the additional cost of training the lightweight student is small, making redistillation practical when the teacher is updated.

Therefore, the plug-and-play advantage of SID-MLP is that it can adapt to an existing teacher with low offline training cost while preserving teacher-level recommendation quality. We have clarified this low-cost adaptation point at the end of Appendix D.3.

We hope the native 7B comparison, LC-Rec diagnosis, and SID-structure results above address your concerns about baseline adaptation, large-teacher fidelity, and adaptation across SID structures. We would be grateful if you would reconsider your assessment. If a specific issue remains, please point us to it and we will address it during the discussion.

Reviewer_yhdnOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces SID-MLP, a lightweight knowledge distillation framework for accelerating semantic-ID-based generative recommendation. Motivated by the shrinking space of valid token continuations along semantic IDs, the authors replace the Transformer decoder with position-specific MLP heads that predict SID tokens using shared user-context representations and previously generated tokens. The framework combines KL-divergence distillation with cross-entropy supervision and is further extended to SID-MLP++, which also replaces the Transformer encoder. Experiments demonstrate comparable ranking performance to the TIGER teacher, with 8.74× higher inference throughput and 95.7% lower peak GPU memory usage. Additional evaluations examine different datasets, SID configurations, teacher sizes, and model architectures.

Overall, this paper presents a practical and effective approach to improving the inference efficiency of generative recommendation. The motivation is intuitive, the architecture is relatively simple, and the substantial efficiency gains are supported by evaluations across multiple settings. Nevertheless, the connection between shrinking token continuation spaces and Transformer redundancy remains insufficiently established, and further analysis of the performance–efficiency trade-off would strengthen the methodological justification.

Critical strengths

1. Decoder simplification yields large efficiency gains.

The motivation study shows that SID positions differ substantially in conditional branching factors, and attention ablations show that preserving user context and prefix dependencies matters. The proposed architecture computes user context once and replaces repeated Transformer decoding with lightweight prefix-conditioned MLP heads. SID-MLP achieves an 8.74× throughput improvement over TIGER-kv while matching or slightly exceeding its NDCG@10 on all three main datasets. Ablations show that prefix conditioning, position-specific heads, and knowledge distillation each contribute to recommendation quality, supporting the decoder replacement in the evaluated TIGER setting.

2. Broad evaluation of applicability and efficiency.

Beyond the main comparison, the authors test different SID tokenizers, codebook widths, SID lengths, teacher sizes, and additional datasets. LC-Rec experiments examine adaptation to a substantially larger decoder-only backbone. Multi-seed results, hardware profiling, offline distillation costs, and valid-prefix index scalability measurements further characterize the method’s advantages and limitations beyond the headline speedup.

Critical weaknesses

1. Methodological Inconsistencies in Baseline Adaptations.

Evaluating baseline models (e.g., AtSpeed, NEZHA) within the modified TIGER framework rather than their native operational regimes obfuscates fair comparison. Specifically, applying LLM-oriented speculative decoding techniques like AtSpeed to a lightweight (~4.59M parameter) encoder–decoder setup limits their inherent performance advantages, leaving it unverified whether SID-MLP maintains superiority under original LLM architectures.

2. Trade-offs in Recommendation Quality and Large-Scale Generalizability.

The claimed lossless quality preservation and scalability are primarily validated on small-scale datasets (~25K items) and compact teacher models. When scaled to a 7B LC-Rec backbone, SID-MLP exhibits non-negligible fidelity degradation (retaining only 83.3%–97.5% NDCG@10). Furthermore, the 5M SID validation relies on synthetic prefix indices rather than end-to-end evaluation on real-world large-scale recommendation catalogs, leaving practical scalability in production settings unproven.

Questions for authors

1. Fairness of the acceleration baseline adaptations.

Comparisons with AtSpeed and NEZHA rely on adaptations to TIGER that may alter their original mechanisms. Could the authors clarify these effects and, if feasible, include an AtSpeed comparison under LC-Rec to test whether SID-MLP’s advantage extends beyond the compact TIGER setting?

2. Accuracy degradation in the LC-Rec experiments.

SID-MLP recovers 83.3%–97.5% of the teacher’s NDCG@10 on LC-Rec, with larger gaps in some categories. Could the authors analyze whether these gaps stem mainly from the one-shot context representation, MLP head capacity, or difficulty approximating the teacher’s conditional distributions, so as to clarify the architecture’s limits in larger settings?

2 comments

We thank the reviewer for raising important questions about the fairness of baseline comparisons and scalability on LC-Rec.

In the responses below, we show that SID-MLP extends naturally to 7B LLM-based generative recommendation, with much higher speedup than native AtSpeed and higher teacher recovery than NEZHA. The diagnostics and scale tests further demonstrate that the lightweight student preserves most teacher behavior and scales effectively to larger deployments.

W1/Q1. AtSpeed comparison on LC-Rec. We agree that evaluating AtSpeed in its original LLM setting is valuable. We therefore reproduced AtSpeed-S/R on native 7B LC-Rec using its 68M LLaMA draft. We profiled all methods on the same A6000 with batch size 32, following the throughput protocol used throughout our paper, while retaining each method's native decoding procedure.

MethodNDCG@10 recovery (Instr./Games/Arts)End-to-end speedupDecode-only speedupOffline GPU-h
AtSpeed-S, N=40100.00% / 100.00% / 100.00%4.12×2.59×24.75
AtSpeed-R, N=2087.63% / 83.78% / 87.29%4.63×3.24×26.28
AtSpeed-R, N=4081.07% / 82.38% / 79.73%4.71×3.19×26.28
SID-MLP97.44% / 83.18% / 88.74%20.83×1,429.69×3.85

Recovery uses the matched LC-Rec teacher; speedups are computed from unrounded throughput, and decode-only speedup uses the LC-Rec measurement after the user-history prefill has been completed. Offline cost includes teacher-output extraction and student/draft training.

AtSpeed-S matches the teacher's performance through strict 7B target verification, but achieves only 4.12× speedup, compared with 20.83× for SID-MLP. AtSpeed-R improves speed by relaxing verification, but its NDCG@10 recovery drops substantially and is generally lower than SID-MLP's. In contrast, SID-MLP directly replaces the autoregressive decoder and removes the need for draft generation and target verification. SID-MLP achieves much higher speedup while preserving most of the teacher's performance, with substantially lower offline training cost.

The advantage is larger in decode-only inference. SID-MLP reaches 1,429.69× speedup versus at most 3.24× for AtSpeed. This is because SID-MLP only needs the history readout and position-specific MLP heads for next-item generation, while AtSpeed still runs draft proposals and 7B target verification. Reusing history states is also consistent with industrial recommendation serving practices [1,2]. 

Section 4.3 (Table 5) also reports NEZHA's LC-Rec results, with 64.96%–82.67% teacher recovery, below SID-MLP's 83.18%–97.44%. Appendices F.1 and F.3 document the TIGER-side AtSpeed and NEZHA adaptations, respectively. Together, these 7B results demonstrate that SID-MLP can be readily adapted to larger LLM-based generative recommenders while retaining its quality–efficiency advantage. We have added the native AtSpeed comparison to Section 4.3 (Table 5), with the N=40N=40 operating point discussed in Appendix D.2 and the decode-only comparison shown in Figure 4.

W2/Q2. The LC-Rec accuracy gap.

As we discussed in Section 4.3, the gap is mainly caused by 159× teacher–student model size compression. SID-MLP distills the 7B LC-Rec teacher into a 44.1M-parameter student (about 0.6% of the teacher). Without online 7B verification, it still recovers 83.18%–97.44% of teacher NDCG@10 with substantial speedup. This is a strong quality–efficiency trade-off at the larger teacher scale.

Our added diagnostics further explain why the gap differs across datasets: its magnitude follows the difficulty of approximating the teacher's earliest prefix-conditioned distributions. We examine three possible factors:

(i) Head capacity. Table 18 in Appendix D.2 varies dhd_h from 512 to 4096 without consistent quality improvement. This suggests that simply increasing the head capacity is insufficient to close the remaining gap.

(ii) One-shot context representation. Appendix D.2 reports that, in actual beam search, the student top-50 covers 79%–88% of the teacher top-10 and contains the teacher top-1 in 90%–92% of cases. This shows that the one-shot context representation already preserves most of the teacher's strong candidates.

(iii) Conditional distribution approximation. Appendix D.2 (Table 19) compares teacher–student top-1 agreement at each SID position, considering only valid next tokens:

DatasetAvg. valid continuations d1→d4d_1\to d_4Valid-token agreement d1→d4d_1\to d_4
Instruments162 → 63.8 → 2.0 → 1.576.5% → 59.7% → 93.1% → 97.4%
Arts242 → 73.9 → 2.5 → 1.477.8% → 52.1% → 88.7% → 96.5%
Games208 → 63.9 → 2.2 → 1.274.1% → 50.4% → 88.5% → 97.2%

Across all three datasets, agreement is lowest at d2d_2. This is the first position where the teacher updates its representation based on a generated SID prefix, while many valid continuations still remain. Also, 91.0%–92.8% of the first beam divergences occur at d1/d2d_1/d_2, while teacher–student ground-truth top-1 accuracy at d2d_2 differs by only 0.19–1.87 percentage points.

These results show that the remaining quality gap is concentrated in fine-grained ranking at early SID positions, while SID-MLP retains most of the teacher's recommendation ability. This supports our main observation: SID positions have different decoding difficulty, and lightweight MLP heads can replace much of the repeated Transformer decoding. We have added the one-shot diagnostic to Appendix D.2 and the position-wise results to Table 19.

W2. Scalability. We agree that real-world catalog size matters. Appendix D.1 (Table 15) evaluates a larger real dataset, Office Products (77.6K items, 1.80M interactions), where SID-MLP recovers 100.3% of teacher NDCG@10. Separately, Appendix E.3 (Table 34) measures exact-prefix index storage and lookup cost up to 5M synthetic unique SIDs. We have clarified the distinction between real-catalog quality evidence and the serving-index stress test in Appendix E.3.

We hope the native 7B comparison, LC-Rec diagnostics, and scalability evidence above address your concerns about baseline fairness, the source of the remaining quality gap, and larger-scale deployment. We would be grateful if you would reconsider your assessment. If a specific issue remains, please point us to it and we will address it during the discussion.

References. 

[1] Meta Engineering, From User Sequences to Scaling Laws: A Multi-Stage Architecture for Meta's Ads Ranking, 2026. 

[2] Qiu et al., Deploying an HSTU Generative Recommender with NVIDIA Dynamo-Triton, 2026.

Reviewer_MLaROpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes an MLP-centric decoding paradigm for Generative recommendation models that reduces the high inference latency of semantic IDs. The authors claim that their small MLP model can achieve an 8.74× inference speedup while maintaining the same accuracy as the teacher model.

Critical strengths
  1. SID-MLP replaces repeated attention-heavy autoregressive decoding with one-shot context readout and prefix-conditioned MLP heads, achieving an 8.74× speedup while matching the TIGER teacher.

  2. The paper provides a clear motivation study and thorough ablations showing that SID branching concentrates in early digits and decoder depth/attention are highly redundant.

  3. The authors tested SID-MLP's performance and compared it with previous methods.

Critical weaknesses
  1. SID-MLP++ achieves speedup to 10.25× but incurs a 1.5–5.1% relative NDCG@10 drop and requires two-stage encoder distillation training.

  2. SID-MLP relies on a frozen teacher and offline distillation, so teacher updates require retraining and the one-time offline cost is amortized only after about 1M examples.

Questions for authors
  1. With a batch size of 32 and a beam size of 50, SID-MLP achieves an 8.74x speedup over TIGER-kv. How stable are the speedup and tail latency performance of SID-MLP when subjected to stricter latency-based SLAs or different beam size configurations?

  2. In scenarios such as cold starts, long-tail user segments, temporal drift or popularity bias, can the model maintain a level of ranking quality and recommendation diversity consistent with that of the teacher model?

  3. How scalable is this design, in terms of storage, training, and inference, as the SID length increases, the codebook size expands, or when handling industrial-scale datasets?

4 comments

We thank the reviewer for raising these important questions about practical deployment.

In the responses below, we show that SID-MLP provides a strong quality-first deployment point, combining stable low-latency acceleration, teacher-level recommendation quality across data shifts, and efficient scaling across SID structures and catalog sizes. SID-MLP++ further offers a speed-first option.

W1. SID-MLP++ offers an optional speed-first variant. SID-MLP is our main method. It distills the decoder and recovers TIGER teacher performance while achieving an 8.74× speedup. SID-MLP++ further distills the encoder, increasing the speedup to 10.25×, with 17.3% additional throughput. We agree that SID-MLP++ introduces a small accuracy cost, with a 1.5%–5.1% relative NDCG@10 drop. This is a trade-off we provide for applications with stricter latency requirements.

W2. Redistillation after teacher updates has a low cost. We agree that updates require redistilling the student. However, this only involves training the lightweight attention readout and MLP heads. As reported in Appendix C.2, student training takes approximately one hour per dataset.

Following lifecycle-compute accounting [1], Appendix E.2 (Table 33) reports 4.48 PFLOP of additional offline compute, versus 6.78–66.17 PFLOP for other baselines. At inference time, it reduces computation from 4.88 to 0.46 GFLOP per request compared with TIGER-kv. As a result, the offline cost is recovered after about 1.01M compute-equivalent requests, versus 1.97M–19.66M for baselines. SID-MLP has the lowest offline compute and earliest break-even among these evaluated baselines.

Q1. Speedup remains stable. Section 4.5, Figure 3(b), already evaluates batch sizes from 1 to 512 (up to 16.6× speedup). Figure 3(c) also shows that when beam size increases from 10 to 50, SID-MLP throughput changes by <3%, versus a 37% drop for TIGER-kv. 

To further address the tail-latency concern, we measured synchronized p99 service time under nine batch/beam configurations. Batch size 1 is per-request service latency, while batch sizes 8 and 32 are batch service times; the measurements exclude queueing, network, and scheduler delays. Each cell reports SID-MLP p99 and its p99 speedup over TIGER-kv:

BatchBeam 10Beam 20Beam 50
16.38 ms / 5.74×6.73 ms / 5.36×6.15 ms / 5.79×
86.62 ms / 5.43×6.41 ms / 5.83×6.41 ms / 6.61×
327.53 ms / 6.21×7.39 ms / 6.59×7.56 ms / 10.05×

Across settings, SID-MLP maintains p99 latency <7.6 ms with >5.3× speedup and p99/p50 ratio <1.19. 

These additional results confirm that SID-MLP maintains its speed advantage for different batches, beam sizes, and tail latency, supporting deployment under tighter latency budgets. We have added the complete p50/p95/p99 results to Appendix E.1 (Table 31).

Q2. SID-MLP preserves teacher-level quality and diversity. We thank the reviewer for highlighting these cases. We examine cold-start and temporal shift, long-tail items and users, and recommendation diversity.

(i) Cold-start and temporal shift. Appendix D.5 (Table 27) evaluates recently introduced items under two timestamp-based splits following SpecGR [2] and ColdGenRec [3]. Across three datasets, SID-MLP closely tracks the teacher and achieves slightly higher average NDCG@10 under both Temporal-80 and Temporal-90. These results show that SID-MLP preserves teacher-consistent ranking quality for recently introduced items and temporal shifts.

(ii) Long-tail items and users. Appendix D.5 (Table 28) divides the complete Instruments test set into groups based on target-item training frequency and user-history length:

Target training frequency# Test casesN@10 Recovery
≤617,98299.2%
7–1112,150102.1%
12–2713,123101.6%
>2714,184104.3%
User-history length# Test casesN@10 Recovery
≤527,347103.1%
6–1020,205103.0%
11–207,466101.4%
>202,421105.3%

All groups recover ≥99.2% of teacher NDCG@10. This confirms that the method also preserves teacher-level quality for long-tail items and users.

(iii) Recommendation diversity. Appendix D.5 (Table 29) directly evaluates the top-10 recommendation lists generated for each user. SID-MLP retains 99.97% of the teacher's pairwise SID diversity, 99.77% of its semantic-cluster diversity, and 99.42% of its recommendation novelty. These results show stable quality and diversity across the tested groups.

We have added the long-tail/user-activity and diversity analyses to Appendix D.5 (Tables 28 and 29).

Q3. Scalability of SID-MLP. We agree that scalability is important for practical deployment. 

(i) Longer SIDs. Our main experiments focus on the four-token SIDs commonly adopted by generative recommendation, as longer SIDs do not necessarily improve quality [4,5]. Appendix D.3 (Table 21) tests different SID lengths, including a setting with seven decoding steps. All tested lengths recover ≥99.8% of matched-teacher NDCG@10. Section 2.1 gives the Transformer prefix-attention complexity, whereas Section 3.3 defines SID-MLP's lightweight per-position heads. SID-MLP therefore avoids repeated attention and handles longer SIDs efficiently.

(ii) Larger codebooks. Appendix D.3 evaluates codebook widths 128/256/512 in Table 20 and heterogeneous structures in Table 22, all with >100% teacher recovery. Increasing the codebook width mainly increases the output dimension of the corresponding MLP head. Since these heads only involve lightweight matrix multiplications, both distillation training and inference remain efficient even with larger codebooks.

(iii) Industrial-scale serving. Model size does not directly grow with catalog item count; the main storage cost is the exact valid-prefix index. Appendix D.1 (Table 15) evaluates the larger real Office Products dataset (77.6K items and 1.80M interactions), where recovery is 100.3%. Appendix E.3 (Table 34) separately stress-tests an index with 5M unique SIDs. Batched lookup remains ~0.22 ms, full beam search ~3.6 ms, and lossless bit-packing reduces index size to 284.5 MiB. Thus, over the tested 25K–5M range, catalog growth primarily increases index memory rather than lookup latency.

We hope the tail-latency audit, robustness analyses, and scalability results above address your deployment questions and clarify the quality-first and speed-first roles of SID-MLP and SID-MLP++. We would be grateful if you would reconsider your assessment. If a specific issue remains, please point us to it and we will address it during the discussion.

References. 

[1] Sardana et al., Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws. 

[2] Ding et al., Inductive Generative Recommendation via Retrieval-based Speculation. 

[3] Zhang et al., Cold-Starts in Generative Recommendation: A Reproducibility Study. 

[4] Liu et al., Understanding Generative Recommendation with Semantic IDs from a Model-Scaling View. 

[5] Nian et al., ChronoID: Infusing Explicit Temporal Signals into Semantic IDs for Generative Recommendation.

Thanks for your experiments and your effort! The authors have addressed most of my concerns, so I have decided to maintain my positive rating.

We sincerely thank the reviewer for their positive assessment and for acknowledging our additional experiments and clarifications. We greatly appreciate your thoughtful feedback and support throughout the discussion!

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal summary

We sincerely thank all reviewers for their constructive feedback and valuable discussions during the rebuttal period. We have addressed the concerns raised by the reviewers, provided additional experiments and analyses, and submitted an updated version of the paper incorporating these changes. As the discussion stage is coming to an end, we summarize our main responses below:

  • Baseline comparison fairness: We added AtSpeed comparisons on the native 7B LC-Rec architecture and further clarified the baseline adaptation details. The new results confirm SID-MLP's quality–efficiency advantage across different model architectures. Please see Reviewer yhdn W1/Q1 and Reviewer 2dXR W1/Q3.
  • Recommendation quality on larger teachers: We further analyzed the remaining accuracy gap on LC-Rec and added position-wise diagnostics. These results provide additional support for SID-MLP's core design and its quality–efficiency trade-off on larger teacher models. Please see Reviewer yhdn W2/Q2 and Reviewer 2dXR W2/Q2.
  • Robustness and scalability: We added experiments on tail latency, long-tail users and items, and recommendation diversity. We also clarified scalability across different SID structures and larger catalogs. These results further confirm the robustness and scalability of SID-MLP. Please see Reviewer MLaR Q1–Q3.
  • Deployment and adaptation costs: We clarified the speed–accuracy trade-off of SID-MLP++ and the low redistillation cost when the teacher or SID structure changes. Please see Reviewer MLaR W1/W2 and Reviewer 2dXR W3/Q1.

We again thank the reviewers and the Area Chair for their time and valuable feedback. We hope these additional results and clarifications will be helpful for the final evaluation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.