acceptodds
Under review as a conference paper at ICLR 2027

Residual Trajectory Distillation for Generative Retrieval

18 views

Abstract

Generative retrieval has emerged as a general retrieval paradigm, representing items with discrete Semantic IDs (SIDs) and retrieving them through autoregressive identifier generation. When SIDs are constructed with residual quantization (RQ), standard retrieval training supervises only the selected codes and discards the residual trajectories that produce them. The same hard code can nevertheless arise from different preferences over competing codewords, while the residual trajectory also contains information about subsequent quantization decisions. As a result, hard SID supervision collapses distinct quantization behaviors into identical targets and leaves information available during indexing unused in retrieval training. We introduce ResTD, a Residual Trajectory Distillation framework that transfers this discarded indexing information into retrieval training. Treating the frozen RQ indexer as a process teacher, it distills residual-induced codeword preferences into SID-decoding states. This supervision recovers distinctions hidden by hard assignments and allows earlier decoder states to capture information about subsequent quantization decisions before the corresponding SID suffix is generated. In this way, richer information from SID construction is incorporated into retrieval learning while preserving the original retrieval index and inference procedure. Experiments on multilingual e-commerce retrieval show consistent improvements over strong baselines and matched training controls. Controlled comparisons show that residual-derived targets outperform the tested codebook-only soft targets. Representation probes further show that future codebook preferences become more recoverable from earlier decoder states. ResTD can also be readily extended beyond retrieval to generative recommendation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Generative Retrieval, Semantic IDs, Residual Quantization, E-Commerce Search

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_g3KiOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes ResTD, which uses residual-derived codeword distributions from a frozen indexer to supervise a generative retriever’s decoder states. It supervises current and future SID levels without changing the index or inference procedure. Experiments show gains in multilingual retrieval, NQ320K document retrieval, and generative recommendation.

Critical strengths
  1. The method uses information from the indexer without adding inference cost. ResTD derives soft targets from residual geometry and supervises future SID levels at earlier decoder states. Reconstructing residuals along stored SID paths and correcting teacher–SID disagreements handles a practical compatibility issue. The auxiliary heads are removed at inference, so the method requires no changes to the deployed index or decoding procedure.
  2. The experiments include thorough controls. The main comparisons use the same initialization, minibatch order, optimization budget, and three paired seeds. ResTD improves over continued fine-tuning, current-level distillation, and matched future hard-target supervision. The code-conditioned, permuted, and shape-matched teacher controls provide further evidence that residual-derived preferences are useful. The NQ320K and recommendation experiments extend the evaluation beyond multilingual product search, and the measured training overhead is modest.
Critical weaknesses
  1. The advantage over simpler representation supervision and related distributional methods remains unclear. Soft codebook supervision has been used in RQ-Transformer and related recommendation methods, including ETEGRec and SODA. ResTD’s main distinction is the combination of a frozen indexing teacher, SID-consistent targets, and multi-horizon decoder supervision.

    The controls support the value of residual-derived targets, but there is no matched comparison with direct residual-vector regression or codebook-expectation matching at the same decoder states and horizons. This would help determine whether the distributional objective adds value beyond simpler continuous supervision using the same indexing information. A more explicit comparison with existing codebook-distribution alignment methods would also clarify the novelty.

  2. The broad baseline comparison is less controlled than the CaLIR experiments. Table 1 combines published baseline scores with reproduced CaLIR and ResTD results. Appendix A.3 evaluates distinct SIDs using binary relevance for Exact, Substitute, and Complement items, taking the maximum relevance across products that share a SID. It is unclear whether all published baselines use the same evaluation unit and relevance definition.

    The matched CaLIR comparisons remain convincing. However, the claim that ResTD outperforms all sparse, dense, and generative baselines needs either qualification or a reevaluation of selected strong baselines under the same protocol.

Questions for authors
  1. How does ResTD compare with direct residual-vector regression or codebook-expectation matching using the same heads, horizons, training budget, and auxiliary-loss schedule? Please also explain how it differs from SODA’s distribution-level supervision.
  2. Do all Table 1 scores use the same SID-level deduplication and binary relevance definitions? If not, could selected strong baselines be reevaluated under the shared protocol, or could the table distinguish directly comparable results from published scores?
  3. In the recommendation experiments, were TIGER and the other baselines trained with matched data, tokenization, and optimization settings? Please report variability across seeds, particularly for the smaller gains on Instruments.
5 comments
AuthorsOpenReview ↗

Official Response to Reviewer g3Ki (1/3)

We sincerely appreciate your careful review and thoughtful comments. We provide detailed responses to your questions below and have highlighted the corresponding changes in the revised manuscript.

Q1: (From Weakness 1 and Question 1) Soft codebook supervision has been used in RQ-Transformer and related recommendation methods, including ETEGRec and SODA. ResTD’s main distinction is the combination of a frozen indexing teacher, SID-consistent targets, and multi-horizon decoder supervision. How does ResTD compare with direct residual-vector regression or codebook-expectation matching at the same decoder states, using the same heads, horizons, training budget, and auxiliary-loss schedule? This would help determine whether the distributional objective adds value beyond simpler continuous supervision using the same indexing information. Please also explain how ResTD differs from SODA’s distribution-level supervision.

A1: Thank you for suggesting these controls. We added direct residual-vector regression and codebook-expectation matching to test whether matching codeword distributions improves retrieval when all three methods use the same indexing information.

(1.1) All three variants use the same supervised decoder positions, projection heads, prediction horizons, training budget, and auxiliary-loss schedule. Direct residual-vector regression trains the projected decoder state to predict the residual vector reconstructed along the final stored SID. Codebook-expectation matching trains the same state to predict the mean codeword vector weighted by the corrected teacher probabilities. ResTD uses the same corrected teacher distribution. It computes a student distribution from the distances between the projected state and the frozen codewords, then matches it to the teacher distribution using KL divergence.

Comparison of distributional supervision and continuous regression objectives on ESCI (%)

ESCI localeTraining objectiveR@5R@10R@100N@10N@100
USDirect residual-vector regression8.9613.9839.0712.5520.36
USCodebook-expectation matching9.0413.9139.2112.6120.42
USResTD9.2814.3439.5812.9920.79
ESDirect residual-vector regression6.5811.3834.9613.7221.12
ESCodebook-expectation matching6.6411.3135.0713.6621.20
ESResTD6.8611.7635.3814.2221.54
JPDirect residual-vector regression7.2511.1732.4913.7120.36
JPCodebook-expectation matching7.3211.2132.4213.7920.31
JPResTD7.6111.6332.8614.1820.79

ResTD scores higher than both baselines on every metric across all three ESCI locales. The comparison with codebook-expectation matching directly tests how to use the same residual teacher. Matching probability distributions over the actual codewords achieves better retrieval than regressing the teacher’s expected codeword vector.

(1.2) ResTD’s contribution is to reconstruct codeword preferences at each level along the final SID stored in a fixed index and use those preferences to supervise earlier decoder states across multiple horizons. We reconstruct the residuals along the collision-resolved SID and adjust the teacher distributions so that each stored codeword has the highest probability while preserving the probability ratios among the alternatives. During training, the same decoder state, conditioned on the query and the current SID prefix, matches teacher distributions from both the current and later quantization levels. This lets the decoder learn preferences among the actual alternative codewords at later levels before generating the corresponding codes.

RQ-Transformer applies residual soft labels to autoregressive next-code prediction at the corresponding level. ETEGRec aligns the codeword distributions of the user history and the target item, and aligns the first decoder state with the reconstructed item representation. SODA constructs distributions from pooled history encoder states and uses their similarity to the item distribution in a contrastive ranking objective.

To evaluate the effect of soft targets at later levels, we compare future hard SID targets with future residual soft targets while keeping current-level supervision and the prediction horizon fixed.

Retrieval performance with matched horizons and different future targets

ESCI localeNDCG@10 with future hard targets (%)NDCG@10 with ResTD future soft targets (%)NDCG@10 gain (pp)
US12.18 ± 0.0812.99 ± 0.07+0.81
ES13.40 ± 0.1014.22 ± 0.07+0.82
JP13.12 ± 0.0714.18 ± 0.08+1.06

With the same prediction horizon, future residual soft targets improve retrieval ranking in all three locales. The comparison shows that relative preferences among alternative codewords at later levels provide a useful training signal for earlier decoder states.

AuthorsOpenReview ↗

Official Response to Reviewer g3Ki (2/3)

Q2: (From Weakness 2 and Question 2) Do all Table 1 scores use the same SID-level deduplication and binary relevance definitions? If not, could selected strong baselines be reevaluated under the shared protocol, or could the table distinguish directly comparable results from published scores?

A2: Thank you for pointing this out. We apologize for the lack of clarity about the sources and evaluation protocols for the scores in Table 1. We will clarify these details in the experimental setup appendix.

(2.1) For the published baselines that also appear in the CaLIR paper’s main table, Table 1 uses the ESCI scores reported in that paper. The scores for FORGE, HGRec, LaSER, and CaLIR in Table 1 come from reproductions conducted for this study. The table below lists the published CaLIR scores, the reproduced baseline scores, and the ResTD results.

Published and locally evaluated results on ESCI (%)

MethodUS R@5US R@10US R@100US N@10US N@100ES R@5ES R@10ES R@100ES N@10ES N@100JP R@5JP R@10JP R@100JP N@10JP N@100
FORGE (reproduced)5.599.2635.118.5016.214.878.2928.429.9916.663.886.5824.068.2714.22
HGRec (reproduced)6.7011.4134.6110.1917.325.769.6031.5611.3718.323.786.1824.378.4013.83
LaSER (reproduced)6.5210.2335.378.1915.745.288.4032.228.6916.165.888.7828.669.2815.53
CaLIR (published)7.2211.7536.1510.7518.146.0610.4334.8312.1520.035.579.6431.7111.7418.85
CaLIR (reproduced)7.5012.6937.4011.1218.796.0310.4834.1512.4319.886.059.7730.8911.7718.42
ResTD9.2814.3439.5812.9920.796.8611.7635.3814.2221.547.6111.6332.8614.1820.79

(2.2) For each query, we treat Exact, Substitute, and Complement labels as relevant. When multiple products share a complete SID, we assign that SID the highest relevance label among the products mapped to it. Before computing the metrics, we deduplicate each ranking by complete SID and retain its highest-ranked occurrence. The published CaLIR scores follow the product-level ranking protocol described in the CaLIR paper and are listed separately in the table.

For a fair comparison, we follow the training protocol described in the CaLIR paper when reproducing CaLIR. We use exactly the same complete-SID deduplication rule and binary relevance definition to evaluate the reproduced FORGE, HGRec, LaSER, and CaLIR models, as well as ResTD.

AuthorsOpenReview ↗

Official Response to Reviewer g3Ki (3/3)

Q3: (From Question 3) In the recommendation experiments, were TIGER and the other baselines trained with matched data, tokenization, and optimization settings? Please report variability across seeds, particularly for the smaller gains on Instruments.

A3: Thank you for asking us to clarify the recommendation setup and report results across seeds.

(3.1) All baselines and ResTD use the same data splits, item tokenizer, and SID mapping. We evaluate them against the same full item catalog. For each random seed, we use the same order of training examples, optimization procedure, and maximum number of training epochs. Each method uses its default hyperparameters.

(3.2) The table reports the results for each of the three seeds, along with the mean and standard deviation. The last row reports ResTD’s relative improvement over the strongest baseline’s three-seed mean for each metric.

Recommendation results across three training seeds (%)

MethodSeedB-Shop R@10B-Shop N@10I-Shop R@10I-Shop N@10Yelp R@10Yelp N@10
SASRec425.883.139.476.902.961.52
SASRec20275.833.059.496.863.041.63
SASRec20286.033.239.366.972.931.60
SASRecMean ± SD5.91 ± 0.103.14 ± 0.099.44 ± 0.076.91 ± 0.062.98 ± 0.061.58 ± 0.06
BigRec422.991.985.764.911.691.42
BigRec20273.131.845.665.001.841.47
BigRec20282.991.855.835.061.661.34
BigRecMean ± SD3.04 ± 0.081.89 ± 0.085.75 ± 0.094.99 ± 0.081.73 ± 0.101.41 ± 0.07
P5-SID425.843.359.647.303.241.70
P5-SID20275.733.209.537.213.341.64
P5-SID20285.943.229.777.383.241.59
P5-SIDMean ± SD5.84 ± 0.113.26 ± 0.089.65 ± 0.127.30 ± 0.093.27 ± 0.061.64 ± 0.06
P5-CID425.973.479.877.513.471.81
P5-CID20275.883.399.847.413.541.72
P5-CID20286.063.569.997.613.461.80
P5-CIDMean ± SD5.97 ± 0.093.47 ± 0.099.90 ± 0.087.51 ± 0.103.49 ± 0.041.78 ± 0.05
LETTER426.723.6411.228.314.262.31
LETTER20276.813.6811.228.424.242.20
LETTER20286.753.5811.298.344.412.40
LETTERMean ± SD6.76 ± 0.053.63 ± 0.0511.24 ± 0.048.36 ± 0.064.30 ± 0.092.30 ± 0.10
TIGER426.103.3110.587.974.072.13
TIGER20275.993.4410.647.894.222.16
TIGER20285.973.4210.438.124.082.02
TIGERMean ± SD6.02 ± 0.073.39 ± 0.0710.55 ± 0.117.99 ± 0.124.12 ± 0.082.10 ± 0.07
ETEGRec426.153.3511.068.104.152.14
ETEGRec20276.263.3410.928.204.242.06
ETEGRec20286.023.2210.948.164.212.21
ETEGRecMean ± SD6.14 ± 0.123.30 ± 0.0710.97 ± 0.088.15 ± 0.054.20 ± 0.052.14 ± 0.08
ResTD427.284.0011.428.454.872.66
ResTD20277.374.0211.538.565.022.76
ResTD20287.354.0811.478.584.882.54
ResTDMean ± SD7.33 ± 0.054.03 ± 0.0411.47 ± 0.068.53 ± 0.074.92 ± 0.082.65 ± 0.11
Improv. (%)Mean+8.48+11.01+2.05+2.07+14.41+15.20

On Instruments, ResTD improves both recall and ranking over TIGER and LETTER, the strongest baseline, at every seed. The paired gains over LETTER are 0.23 ± 0.07 percentage points in R@10 and 0.17 ± 0.06 percentage points in N@10. The smaller gains on Instruments are consistent across all three seeds.

Thank you again for taking the time to review our paper. We hope our responses and additional results address your concerns. We would be happy to answer any further questions.

Reviewer_zZyiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ResTD (Residual Trajectory Distillation), a framework that leverages residual quantization information discarded by conventional hard SID supervision to improve generative retrieval. ResTD constructs soft teacher distributions from a frozen RQ indexer and distills current and future codeword preferences into decoder states through multi-horizon supervision. A collision-aware correction ensures consistency with stored SIDs. Experiments on multilingual retrieval benchmarks and generative recommendation tasks demonstrate consistent performance improvements over baselines.

Critical strengths
  1. Consistent empirical improvements The paper provides empirical evidence that residual-derived multi-horizon supervision improves generative retrieval and generative recommendation, and outperforms baselines.

  2. Practical design without additional inference overhead ResTD uses a frozen RQ indexer as a teacher during training while preserving the existing retrieval index and autoregressive decoding procedure. The collision-aware teacher construction ensures consistency with stored SIDs, and auxiliary components are removed at inference. This design improves retrieval performance without modifying the existing retrieval index or introducing additional inference overhead, making it practical to integrate into existing generative retrieval systems.

Critical weaknesses
  1. The experiments do not sufficiently isolate the unique contribution of codebook-space distillation from general multi-horizon soft-target supervision. Although the paper demonstrates improvements over current-level distillation and future hard-SID supervision, it remains unclear whether these gains specifically require the proposed distance-based alignment in the frozen codebook space. More carefully controlled experiments are therefore needed to disentangle these two effects. Although Table 3 presents several teacher-target ablation studies, the comparison with future hard-SID supervision alone does not sufficiently distinguish the benefits of codebook-space alignment from those of general multi-horizon soft-target supervision. A multi-horizon soft-output distillation baseline using the same residual-derived teacher distributions and training conditions would provide a more controlled comparison.

  2. Limited justification and isolation of the contribution of residual geometry The paper argues that hard SID supervision discards residual-to-codeword distance information, but it remains unclear why preserving this information is particularly beneficial for retrieval. As the authors mentioned, quantization proximity does not necessarily imply query relevance. While residual geometry captures relationships between item embeddings and quantization codebooks, retrieval relevance depends on query-item relationships, which may not align with this geometry. Thus, the extent to which residual preferences provide retrieval-relevant information beyond general soft supervision remains unclear. This concern is also reflected in Table 3, where the shape-matched teacher swap retains approximately 87-89% of ResTD's NDCG@10 improvement over Base-FT, despite replacing almost all item-specific teachers with those from other items. Although this control preserves hard code assignments and approximately matches distributional shapes, the results suggest that a substantial portion of the improvement can be achieved without exact item-teacher correspondence. While ResTD consistently performs better, the incremental contribution of precise item-specific residual preferences warrants further investigation. Furthermore, several prior studies have explored distilling soft distributions derived from RQ codebooks into recommenders [1,2,3]. One of the key differences between these approaches and the proposed method appears to be the use of a frozen codebook rather than a trainable one. However, the advantages of directly leveraging the residual geometry of a frozen codebook remain insufficiently justified, particularly in terms of why such a design would be more effective for learning retrieval relevance. A more detailed analysis comparing the proposed approach with prior methods using trainable codebooks would help clarify the unique benefits and effectiveness of preserving frozen codebook geometry. [1] Generative Recommender with End-to-End Learnable Item Tokenization (SIGIR '25) [2] Distribution-Level Contrastive Supervision for Generative Recommendation (RecSys '26) [3] UniGRec: Unified Generative Recommendation with Soft Identifiers for End-to-End Optimization (arXiv, Jan. 2026)

Questions for authors

Q1. What retrieval-relevant information does residual geometry provide beyond general soft supervision? Could the authors further explain or empirically demonstrate what complementary information residual-derived codeword preferences provide to improve query-conditioned retrieval performance? Q2. How significant is the contribution of exact item-specific residual preferences? In Table 3, the shape-matched teacher swap retains approximately 87–89% of ResTD's NDCG@10 improvement over Base-FT, despite replacing almost all item-specific teachers. Could the authors further analyze the incremental benefit of exact item–teacher correspondence and provide statistical significance tests for the performance differences? Q3. Does codebook-space alignment provide benefits beyond general multi-horizon soft supervision? Could the authors compare ResTD with a multi-horizon soft-output distillation baseline using the same residual-derived teacher distributions, supervision horizons, and training conditions? Q4. Does the proposed method remain effective without CaLIR's latent reasoning? Since ResTD appears to be implemented on top of CaLIR, could the authors investigate whether the proposed method remains effective when CaLIR's latent reasoning component is removed? Q5. Could you clarify the collision resolution algorithm used to construct the final SID index? In particular, how are codeword assignments modified when multiple items share the same SID?

14 comments
AuthorsOpenReview ↗

Official Response to Reviewer zZyi (1/6)

Sincerely thanks for your efforts in reviewing this paper. Below, we respond to your questions in detail. The corresponding changes are highlighted in the revised manuscript.

Q1: (From Weakness 2 and Question 1) What retrieval-relevant information does residual geometry provide beyond general soft supervision? Could the authors further explain or empirically demonstrate what complementary information residual-derived codeword preferences provide to improve query-conditioned retrieval performance?

A1: We appreciate your thoughtful and constructive feedback.

(1.1) The retrieval labels specify which items are relevant to each query. For a relevant item, the residual teacher adds information that the hard SID leaves out. Given an SID prefix, it tells the decoder which alternative codewords can best represent the item and how strongly it favors each one. For example, two queries can have different relevant items that share the same stored codeword at a given level. The hard labels give the same target at that level. The two residuals may be closer to different alternatives, such as codewords A and B, giving the decoder different supervision for the two queries. ResTD also uses preferences from later levels to supervise earlier decoder states, so the model can learn these preferences before generating the corresponding codes.

(1.2) We tested whether these preferences help retrieval by permuting the probabilities assigned to alternative codewords. This control keeps the probability of the stored codeword and the full set of probability values fixed. It changes only which alternative codeword receives each probability. With the probability profile held fixed, preserving the original codeword–probability assignments improves retrieval in all three ESCI locales (Table 3). Averaging the residual teachers of items assigned to the same codeword also yields better retrieval results than constructing soft targets solely from codebook distances. This shows that items in the same codeword group can share useful residual preferences.

We also analyzed queries with multiple relevant items. For each query, we measured disagreement as the average pairwise Jensen–Shannon divergence between the first-level residual teachers of its relevant items. Within each ESCI locale, we grouped the queries into low-, medium-, and high-disagreement groups. Below we show the average gains of ResTD over current-level RBD (H=1) on all five metrics.

Retrieval gains by teacher disagreement (ResTD − RBD, percentage points)

ESCI localeTeacher disagreementQueriesMean JS (nats)ΔR@5ΔR@10ΔR@100ΔN@10ΔN@100
USLow1,5500.236+1.04+0.93+1.17+0.85+1.04
USMedium1,5500.394+1.38+1.20+1.50+1.34+1.54
USHigh1,5490.538+1.70+1.51+1.87+1.82+1.88
ESLow4500.432+0.46+0.59+0.51+0.95+0.97
ESMedium4490.580+0.72+1.08+1.12+1.56+1.43
ESHigh4490.658+0.65+1.13+1.17+1.30+1.41
JPLow5120.409+0.71+0.76+0.83+0.83+1.00
JPMedium5110.570+1.21+1.60+1.59+2.01+2.17
JPHigh5110.652+1.95+1.98+2.08+2.64+2.35

In all three locales, the high-disagreement group achieves larger gains than the low-disagreement group on all five metrics. In US and JP, the gains on all five metrics increase from the low- to the medium- to the high-disagreement group. After controlling for the number of relevant items and the retriever’s score on each query before fine-tuning, greater teacher disagreement remains positively associated with gains in Recall@100 and NDCG@10 in both locales. Combined with the probability permutation control, these results show that assigning probability mass to specific alternative codewords is useful for retrieval and that ResTD tends to yield larger gains when the relevant items for a query have more diverse codeword preferences.

AuthorsOpenReview ↗

Official Response to Reviewer zZyi (2/6)

Q2: (From Weakness 2 and Question 2) How significant is the contribution of exact item-specific residual preferences? In Table 3, the shape-matched teacher swap retains approximately 87–89% of ResTD's NDCG@10 improvement over Base-FT, despite replacing almost all item-specific teachers. Could the authors further analyze the incremental benefit of exact item–teacher correspondence and provide statistical significance tests for the performance differences?

A2: Thank you for raising this question. We quantified the additional gain from using each item’s own residual teacher and tested the difference in a paired query-level analysis.

(2.1) The shape-matched swap exchanges teachers between items that share the same stored codeword at the same quantization level. We match teachers using the probability assigned to the stored codeword and the sorted probabilities of the alternative codewords, then swap the full teacher distributions. The supervision horizons are the same in both conditions. Each swapped teacher comes from the residual of another item in the same codeword group and has a probability profile close to that of the original teacher. The swap retains most of the retrieval gain, showing that items in the same codeword group can share useful residual supervision.

Shape-matched teacher swap (NDCG@10)

ESCI localeBase-FT (%)Shape-matched swap (%)ResTD (%)Gain retained by swapResTD − swap (pp)95% paired bootstrap CI (pp)Raw p-valueHolm-adjusted p-value
US11.2512.7912.9988.5%+0.20[0.07, 0.34]0.00310.0062
ES12.5314.0014.2287.0%+0.22[0.06, 0.39]0.00740.0074
JP11.9013.9014.1887.7%+0.28[0.12, 0.45]0.00090.0027

In the paired query-level comparison, restoring each item’s own residual teacher significantly improves retrieval in all three ESCI locales (Holm-adjusted p < 0.05). This comparison measures the additional benefit of using each item’s own teacher when the swap already provides residual supervision from the same codeword group.

(2.2) Two teacher controls help explain which parts of the residual supervision contribute to retrieval. The first preserves the probability assigned to the stored codeword and all probability values in the distribution, changing only which alternative codeword receives each probability. Preserving the original assignments improves retrieval, showing that it matters which alternative codewords receive higher probabilities. The second averages the residual teachers of items that share a stored codeword at the same quantization level. The code-conditioned mean improves retrieval over soft targets constructed solely from codebook distances, showing that the residual preferences shared by these items provide useful supervision.

Residual teacher controls (mean NDCG@10)

ESCI localeCodebook-only (%)Code-conditioned mean (%)Code-conditioned mean − codebook-only (pp)Non-target permutation (%)ResTD (%)ResTD − permutation (pp)
US12.0512.50+0.4512.3612.99+0.63
ES13.2213.66+0.4413.5414.22+0.68
JP12.9513.47+0.5213.3114.18+0.87
AuthorsOpenReview ↗

Official Response to Reviewer zZyi (3/6)

Q3: (From Weakness 1 and Question 3) Does codebook-space alignment provide benefits beyond general multi-horizon soft supervision? Could the authors compare ResTD with a multi-horizon soft-output distillation baseline using the same residual-derived teacher distributions, supervision horizons, and training conditions?

A3: Thank you for suggesting this matched comparison. We added a multi-horizon soft-output distillation baseline to test whether distance-based scoring in the frozen codebook space improves retrieval when both methods use the same multi-horizon residual targets.

(3.1) Both methods use the same collision-corrected residual teacher distributions, supervised decoder positions, prediction horizon (H=4), and loss weights across horizons. They retain the same hard SID generation objective and share the same backbone initialization, sample order, training budget, and decoding settings. The difference is how the student scores the codewords. Multi-horizon soft-output distillation uses a learned classifier to score the codewords at each target level. ResTD computes the scores from the squared distances between the projected decoder state and the frozen codewords at that level.

Matched comparison with multi-horizon soft-output distillation

Training objectiveUS R@10US R@100US N@10US N@100ES R@10ES R@100ES N@10ES N@100JP R@10JP R@100JP N@10JP N@100
Multi-horizon Soft-output KD (H=4)13.92 ± 0.0939.08 ± 0.1212.56 ± 0.0820.46 ± 0.1111.48 ± 0.0735.09 ± 0.1013.78 ± 0.0921.10 ± 0.1211.19 ± 0.1332.43 ± 0.1113.65 ± 0.1020.34 ± 0.09
ResTD (H=4)14.34 ± 0.0439.58 ± 0.1112.99 ± 0.0720.79 ± 0.0511.76 ± 0.0435.38 ± 0.1314.22 ± 0.0721.54 ± 0.1211.63 ± 0.1732.86 ± 0.0914.18 ± 0.0820.79 ± 0.05
Δ (ResTD − Soft-output KD, pp)+0.42+0.50+0.43+0.33+0.28+0.29+0.44+0.44+0.44+0.43+0.53+0.45

(3.2) Under the same multi-horizon residual supervision, ResTD achieves higher mean scores on all four metrics in each of the three ESCI locales. This matched comparison supports an additional retrieval benefit from distance-based scoring in the frozen codebook space.

Reviewer_hVVsOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper uses the intermediate residuals from RQ-based Semantic ID construction for training a generative retriever. At each SID level, the residual gives a soft distribution over the codebook, instead of only the selected code. The decoder is trained to match the distributions for both the current and future SID levels. The paper also corrects the teacher when it disagrees with the final collision-resolved SID. This is only used during training. The index and inference are unchanged.

Critical strengths

The problem is clear. Hard SID labels discard useful information from the quantization process. The method is simple and has no inference overhead. The ablations are fairly complete. The results consistently support using soft future targets over hard future targets.

Critical weaknesses

Novelty of this work is limited. RQ-Transformer already uses residual distance soft labels. Supervising earlier states with future targets is also not new. The main contribution is combining these ideas for generative retrieval and handling collision-resolved SIDs.

I am not convinced by the main motivation about item-specific residual information. Replacing an item's teacher with a similar teacher from another item still keeps most of the gain. This suggests the soft target and future supervision matter more than the actual residual trajectory of that item.

The paper argues that future supervision helps keep the correct SID path during beam search. This is only supported by selected examples. There is no full analysis of how often the correct prefix is pruned at each decoding step.

The overall comparison mixes published results from different training setups. The controlled experiments are mainly continuation training from CaLIR. The recommendation experiment has no matched ablations. It only shows that the method can be applied there.

Questions for authors

Besides the retrieval setting and the extra projection head, how is current level RBD different from the soft labeling in RQ-Transformer?

Is ResTD significantly better than the shape-matched teacher swap? If the difference is small, what evidence shows that item-specific residual information is important?

Can the authors report correct-prefix survival at each decoding step for the full test set?

Does the method still help when the retriever is trained with ResTD from the beginning, rather than added through CaLIR continuation training?

10 comments
AuthorsOpenReview ↗

Official Response to Reviewer hVVs (1/4)

We sincerely appreciate your constructive and insightful comments. We respond to each point below. The corresponding changes are highlighted in the revised manuscript.

Q1: RQ-Transformer already uses residual-distance soft labels, and future-target supervision has precedents. ResTD combines these ideas for generative retrieval and handles collision-resolved SIDs. Besides the retrieval setting and the extra projection head, how does current-level RBD differ from RQ-Transformer’s soft labeling?

A1: Thank you for your thoughtful feedback.

(1.1) RQ-Transformer uses residual soft labels in its autoregressive next-code loss. Current-level RBD retains the hard SID generation target and uses residual soft labels in a separate auxiliary loss. This auxiliary loss matches the residual teacher distribution to a distribution computed from the distances between the projected decoder state and the frozen codewords at the same level.

We test this distinction using Full-output KD. Both Full-output KD and RBD use the same hard SID targets and the same corrected teacher distribution. Full-output KD applies soft supervision directly to the SID output probabilities. RBD applies it to the auxiliary distribution defined by distances in the frozen codebook. RBD improves retrieval ranking over Full-output KD in every ESCI locale.

(1.2) RQ-Transformer uses the residual soft labels at each quantization level to supervise the code prediction at that level, whereas ResTD starts from the final SID stored in the fixed index and reconstructs the residual teacher distributions along that path. During training, the same decoder state, conditioned on the query and the current SID prefix, receives supervision from teacher distributions at both the current and later quantization levels. This lets the state learn the relative preferences among candidate codewords at later levels before generating the corresponding SID codes. The hard SID remains the generation target.

Collision resolution can change the initial code assignments. At each level, we adjust the reconstructed teacher distribution so that the stored codeword has the highest probability while preserving the probability ratios among the other codewords. The auxiliary heads are used only during training. At inference, retrieval uses the existing index and autoregressive decoding procedure.

We freeze RBD and ResTD and fit separate probes with identical architecture and capacity to predict teacher distributions from their decoder states. We evaluate KL divergence on held-out queries. The table reports relative KL reductions with respect to current-level RBD. Each range gives the minimum and maximum across the corresponding decoder-state and target-level pairs. The retrieval comparison holds current-level supervision and the prediction horizon fixed and replaces the hard SID targets for later levels with residual teacher distributions.

Codeword preferences in decoder states and retrieval performance

ESCI localeCurrent-level relative KL reductionLater-level relative KL reductionN@10 with future hard targetsN@10 with ResTD future soft targetsN@10 gain (pp)
US0.1–0.26.5–11.912.18 ± 0.0812.99 ± 0.07+0.81
ES0.1–0.35.3–11.113.40 ± 0.1014.22 ± 0.07+0.82
JP0.1–0.36.9–13.813.12 ± 0.0714.18 ± 0.08+1.06

In all three locales, the largest relative KL reduction occurs when predicting the fourth-level teacher distribution from the first decoder state.

AuthorsOpenReview ↗

Official Response to Reviewer hVVs (2/4)

Q2: The shape-matched teacher swap retains most of ResTD’s retrieval gain. Does ResTD significantly outperform this control, and what evidence supports an additional benefit from using each item’s own residual teacher?

A2: Thank you for raising this question about teacher swapping and item-specific residual information.

(2.1) The shape-matched swap exchanges teachers between items that share the same stored codeword at the same quantization level. We match teachers by the probability assigned to the stored codeword and the sorted probabilities of the alternative codewords, then swap the full teacher distributions. Each swapped target comes from the residual of another item in the same codeword group and has a probability profile close to that of the original teacher. The swap retains most of ResTD's retrieval gain over Base-FT, showing that items assigned to the same codeword can share useful residual supervision. Using each item's own teacher further improves mean retrieval performance in every locale.

Shape-matched teacher swap (NDCG@10)

ESCI localeBase-FT (%)Shape-matched swap (%)ResTD (%)Gain retained by swapResTD − swap (pp)95% paired bootstrap CI (pp)Raw p-valueHolm-adjusted p-value
US11.2512.7912.9988.5%+0.20[0.07, 0.34]0.00310.0062
ES12.5314.0014.2287.0%+0.22[0.06, 0.39]0.00740.0074
JP11.9013.9014.1887.7%+0.28[0.12, 0.45]0.00090.0027

In the paired query-level analysis, ResTD significantly outperforms the shape-matched teacher swap in all three ESCI locales (Holm-adjusted p < 0.05). The two conditions use the same supervision horizons, and the swapped teachers come from the same codeword group and have similar probability profiles. Under these conditions, the additional gain from using each item's own residual teacher shows that item–teacher correspondence contributes to retrieval performance.

(2.2) We use two controls to examine which aspects of the residual teachers contribute to retrieval. The first preserves the probability assigned to the stored codeword and the full set of probability values, changing only which alternative codeword receives each probability. Preserving the original assignments improves retrieval, showing that it matters which alternative codewords receive higher probabilities. The second control averages the residual teachers of items that share the same stored codeword at the same quantization level. The code-conditioned mean improves retrieval over soft targets derived solely from codebook distances, showing that the residual preferences shared by these items provide useful supervision.

Residual teacher controls (mean NDCG@10)

ESCI localeCodebook-only (%)Code-conditioned mean (%)Code-conditioned mean − codebook-only (pp)Non-target permutation (%)ResTD (%)ResTD − permutation (pp)
US12.0512.50+0.4512.3612.99+0.63
ES13.2213.66+0.4413.5414.22+0.68
JP12.9513.47+0.5213.3114.18+0.87
AuthorsOpenReview ↗

Official Response to Reviewer hVVs (3/4)

Q3: The beam-search claim relies on selected examples. Can the authors report how often correct SID prefixes survive or are pruned at each decoding step across the full test set?

A3: Thank you for your insightful comments. We evaluated cumulative retention and first loss of relevant SID prefixes at every decoding step on the full ESCI test sets for US, ES, and JP.

For each query, we deduplicate relevant items by their complete SIDs. At step tt, we calculate the fraction of all relevant SIDs whose length-tt prefix remains in the active beam, then average these fractions with equal weight across queries to obtain the cumulative retention rate RtR_t. Step 4 measures retention of complete SIDs. Setting R0=100%R_0=100\%, we compute the first-loss rate at step tt as Rt−1−RtR_{t-1}-R_t. This is the fraction of all relevant SIDs whose prefix is first lost at that step.

Table A. Cumulative retention of relevant SID prefixes on the full test sets (%)

ESCI localeMethodStep 1Step 2Step 3Step 4
US (6,014)Base-FT94.1461.4845.9237.60
USRBD94.1261.9446.7038.20
USRBD + Future Hard SID94.1762.2847.3238.68
USResTD94.1762.9248.1339.58
ES (1,656)Base-FT89.4351.8340.2734.22
ESRBD89.4652.1940.7134.46
ESRBD + Future Hard SID89.4452.5741.0834.81
ESResTD89.4853.1341.6535.38
JP (1,883)Base-FT86.2248.6337.2230.98
JPRBD86.1949.1337.7931.48
JPRBD + Future Hard SID86.2549.5838.3231.96
JPResTD86.2550.2739.1432.86

Table B. First-loss rate of relevant SID prefixes at each decoding step (%)

ESCI localeMethodStep 1Step 2Step 3Step 4
USBase-FT5.8632.6515.578.32
USRBD5.8832.1915.238.50
USRBD + Future Hard SID5.8331.8814.978.64
USResTD5.8331.2514.798.55
ESBase-FT10.5737.6111.556.05
ESRBD10.5437.2711.486.25
ESRBD + Future Hard SID10.5636.8711.496.27
ESResTD10.5236.3511.486.27
JPBase-FT13.7837.5811.426.24
JPRBD13.8137.0711.336.31
JPRBD + Future Hard SID13.7536.6611.276.36
JPResTD13.7535.9911.126.28

Step 2 accounts for the largest first loss of relevant prefixes for every method in all three locales. Retention is similar across methods at step 1. From step 2 through step 4, ResTD achieves higher cumulative retention than both current-level RBD and RBD + Future Hard SID in all three locales. With the prediction horizon held fixed, ResTD retains 0.64, 0.56, and 0.69 percentage points more relevant prefixes at step 2 than the future-hard control in US, ES, and JP, respectively. The advantage persists at step 4, where complete-SID retention is 0.90, 0.57, and 0.90 percentage points higher, respectively.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

Dear all reviewers:

We sincerely thank all reviewers for their careful reading of our paper and for their constructive suggestions. We are encouraged by their recognition of the following strengths.

  • Motivation. Reviewer hVVs highlighted the clear problem formulation and the motivation for using quantization information to improve generative retrieval.
  • Methodology. Reviewer hVVs found the method simple, and Reviewer zZyi highlighted its practicality for generative retrieval systems. All three reviewers noted that ResTD incurs no additional inference overhead.
  • Experiments. Reviewer hVVs noted the breadth of the ablations, Reviewer g3Ki emphasized the thorough experimental controls, and Reviewer zZyi highlighted the consistent gains across generative retrieval and recommendation.
  • Presentation. Reviewer g3Ki found the paper clear and easy to follow.

We summarize the main additions in response to each reviewer’s comments.

  • Reviewer hVVs. We have analyzed item-specific teachers and prefix survival on the full ESCI test sets, evaluated ResTD from the start of retriever training, and added matched recommendation ablations. We also clarify the distinction from RQ-Transformer.
  • Reviewer zZyi. We have compared codebook-space and multi-horizon soft-output distillation using the same teacher distributions and evaluated ResTD with CaLIR’s latent reasoning removed. We also explain how residual codeword preferences complement relevance labels and how SID collisions are resolved.
  • Reviewer g3Ki. We have added matched comparisons with residual-vector regression and codebook-expectation matching, and reported recommendation results across seeds. We also clarify the distinction from SODA.

All revisions in the paper are marked in blue font. We sincerely appreciate the reviewers' constructive suggestions and remain committed to further improving our work.

Below, we address each reviewer's specific concerns in detail. We extend our heartfelt thanks to all reviewers for their recognition and valuable feedback. We welcome further discussion and look forward to continued engagement. Thank you!

Sincerely yours,

Authors

AuthorsOpenReview ↗

Summary of Responses and Revisions

Dear Area Chair and all Reviewers,

We sincerely thank the reviewers for their time and constructive feedback. We are encouraged that the reviewers recognized the following strengths:

  • Motivation: Reviewer hVVs highlighted the clear motivation for using residual quantization information to improve generative retrieval.
  • Methodology: Reviewers g3Ki and zZyi noted the practical value of aligning residual teachers with the stored SIDs. All three reviewers noted that ResTD adds no inference overhead.
  • Experiments: Reviewer g3Ki highlighted the matched training comparisons, Reviewer hVVs noted the breadth of the ablations, and Reviewer zZyi recognized the consistent gains across retrieval and recommendation.

In response to the reviewers’ feedback, we made the following revisions:

  • Reviewer g3Ki: We added matched comparisons with residual-vector regression and codebook-expectation matching, clarified the baseline evaluation protocols, and reported recommendation results across three seeds. Reviewer g3Ki indicated that most of their concerns had been addressed and maintained their positive rating.
  • Reviewer hVVs: We evaluated ResTD from the start of retriever training, added matched recommendation ablations, and analyzed SID prefix survival across five beam widths. Reviewer hVVs stated that the new prefix survival results resolved their remaining empirical concern and raised their rating from 2 to 3.
  • Reviewer zZyi: We revised the paper to foreground prefix selection and clarify how ResTD’s cross-horizon supervision differs from prior work. We added a matched soft-output distillation comparison and analyzed ranking among candidates that share an SID prefix, as well as prefix survival across beam widths. We hope these revisions and additional results help clarify ResTD’s contribution and resolve the reviewer’s remaining concerns.

ResTD uses residual distributions from later quantization levels to teach earlier decoder states which codewords relevant items favor when those states see only the query and a shorter SID prefix. We believe ResTD’s cross-horizon supervision takes an important step toward improving prefix selection in generative retrieval by helping beam search retain more paths to relevant items while preserving the existing SID index.

We have incorporated the additional experiments and clarifications into the revised manuscript and highlighted the changes in blue. Thank you again for helping us improve the paper.

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.