- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates how natural-language preferences should condition semantic-ID generation for steerable recommendation. It proposes a decoupled dual-branch interface that connects a frozen language tower to a trainable SID generator through pooled global modulation and token-level cross-attention, using the language tower’s final-layer representations. Under a shared Gemma 3 backbone, the paper compares this interface with end-to-end joint conditioning, cross-attention-only conditioning, and representation-depth controls. Experiments on three Amazon domains evaluate preference recommendation and find/avoid steering, alongside auxiliary steering tasks, meaning-preserving rephrasings, and general-language capability tests. Joint conditioning achieves the highest original-test retrieval accuracy, while the proposed interface trails by 0.34–0.79 percentage points in preference Recall@10 and improves over cross-attention alone. The paper reports smaller degradation under synonym substitution and preservation of the frozen language tower’s pretrained capabilities, whereas joint fine-tuning substantially degrades the tested reasoning capabilities. Component and depth ablations further examine the roles of global modulation, cross-attention, and final-layer conditioning.
Critical strengths
-
The paper evaluates lexical robustness and capability retention in addition to retrieval accuracy, revealing meaningful failure modes of joint conditioning.
-
The dual-branch design is technically coherent, and the ablations support the complementary roles of global modulation and token-level cross-attention.
Critical weaknesses
-
The benefit of using a causal LLM as the frozen text encoder remains unclear. Comparisons with a bidirectional encoder of comparable size and smaller or distilled alternatives would help establish whether the additional model capacity and serving cost are justified.
-
Freezing the language tower preserves its pretrained capabilities but prevents retrieval supervision from adapting its representations. Could this constraint limit the conditioning interface's flexibility when handling domain-specific terminology or new preference types?
-
It is unclear whether the robustness gains come from freezing the language tower or from the proposed dual-branch interface. Please evaluate the frozen cross-attention-only baseline under the same rephrasing tests to isolate the additional benefit of global modulation.
Questions for authors
-
Rephrasing sensitivity and a single attention example do not establish that joint conditioning relies on keyword memorization rather than semantic understanding. Could the authors support this interpretation with controlled keyword interventions and analysis across more examples, or moderate the claims accordingly?
-
Will more comparisons with public datasets be provided to validate the generalizability?
6 comments
CW3. Does rephrasing robustness come from freezing or from the dual-branch interface?
The requested comparison shows that lexical robustness is shared by both frozen interfaces, while the dual-branch design adds retrieval and steering gains. With the same frozen backbone and generator architecture, Dual-Branch improves test Preference R@10 by 0.15–1.24 pp and Sentiment@10 by 0.81–1.71 pp across all four datasets (Tables G1 and G3), and improves attribute selectivity across Amazon (Table G2). These gains go beyond keeping the language tower frozen.
Table G1. Original-request retrieval and paired rephrasing on Amazon (%, three-seed mean sample SD). Test columns use held-out original requests; the four validation columns compare the original wording with lexical, syntactic, and conversational rewrites. Bold marks the highest mean within each domain and metric. Manuscript: §5.1, Table 2 (p. 7, lines 333–348), §5.4, and Appendix E.6, Table 13 (p. 30, lines 1592–1605).
Table G1(a). Beauty
| Interface | Test Preference R@10 | Test Sentiment@10 | Val Original R@10 | Val Lexical R@10 | Val Syntactic R@10 | Val Conversational R@10 |
|---|---|---|---|---|---|---|
| Joint Conditioning (Full-FT) | ||||||
| Joint Conditioning (LoRA) | ||||||
| Frozen Cross-Attention-only | ||||||
| Dual-Branch Injection (ours) |
Table G1(b). Sports
| Interface | Test Preference R@10 | Test Sentiment@10 | Val Original R@10 | Val Lexical R@10 | Val Syntactic R@10 | Val Conversational R@10 |
|---|---|---|---|---|---|---|
| Joint Conditioning (Full-FT) | ||||||
| Joint Conditioning (LoRA) | ||||||
| Frozen Cross-Attention-only | ||||||
| Dual-Branch Injection (ours) |
Table G1(c). Toys
| Interface | Test Preference R@10 | Test Sentiment@10 | Val Original R@10 | Val Lexical R@10 | Val Syntactic R@10 | Val Conversational R@10 |
|---|---|---|---|---|---|---|
| Joint Conditioning (Full-FT) | ||||||
| Joint Conditioning (LoRA) | ||||||
| Frozen Cross-Attention-only | ||||||
| Dual-Branch Injection (ours) |
For lexical rephrasing, cross-attention-only is higher on Beauty, the frozen interfaces are close on Sports and Toys, and Dual-Branch is higher on Steam. Dual-Branch also has higher syntactic and conversational R@10 than cross-attention-only in all four datasets. Configurations are independently validation-selected; each rephrasing pair holds history and target fixed. Figure 3 now includes all four interfaces (§5.4, pp. 9–10, lines 444–496; highlighted; Appendix E.6, Table 13).
The LoRA comparison uses 375 configurations per Amazon domain: ranks {4, 8, 16, 32, 64}, learning rates {0.0005, 0.001, 0.005, 0.01, 0.02}, batch sizes {8, 12, 16, 32, 48}, and training steps {30k, 50k, 70k}, followed by three-seed retraining. Dual-Branch exceeds selected LoRA in original test Preference R@10 and all nine Amazon rephrasing comparisons. Sweep and selection details are in Appendices D.1 (p. 22, lines 1163–1170; highlighted) and D.2 and Table 8 (p. 23).
Q1. Do keyword interventions and additional examples support the semantics interpretation?
We have narrowed the keyword-memorization claim to observed lexical sensitivity and added a 2,612-pair attribute-flip evaluation. This directly tests whether changing the requested attribute changes retrieval while history and most wording remain fixed.
The evaluation contains 2,612 request pairs across Beauty, Sports, and Toys, tested on existing checkpoints without further training. For a concrete example, the same Toys history is paired with “Select a Melissa & Doug sound puzzle that depicts modes of transportation” and “Select a Melissa & Doug sound puzzle that depicts melodic orchestral tools.” Let contain the catalog's vehicle sound puzzles and its musical-instrument sound puzzles. For the first request, is the desired group and the opposing group; for the second, these roles reverse. The groups are disjoint. The attribute phrases avoid the original attribute-value words and word stems of at least four characters, while retaining shared brand and product context.
For the two top-10 prediction sets and , selectivity measures the desired-group hit rate minus the opposing-group hit rate, averaged over the two request directions:
\begin{aligned} &\underbrace{\Pr(T_A \cap M_A \neq \varnothing) - \Pr(T_A \cap M_B \neq \varnothing)}_{\text{Request A}}\\\\[0.5em] &+ \underbrace{\Pr(T_B \cap M_B \neq \varnothing) - \Pr(T_B \cap M_A \neq \varnothing)}_{\text{Request B}} \end{aligned} \right].$$ Pr is the empirical fraction of paired records; selectivity is reported in percentage points. Identical predictions for the two requests give zero selectivity. **Table G2. Attribute selectivity on paired paraphrased-attribute requests (percentage points; higher is better).** Domain scores are three-seed mean $\pm$ sample SD. The final column is the equal-weight mean across domains. Bold marks the highest mean in each column. Manuscript: §5.1 (p. 8, lines 386–389; highlighted) and Appendix E.7 (pp. 31–32, lines 1652–1703; highlighted), Table 16. | Interface | Beauty $100\Delta \uparrow$ | Sports $100\Delta \uparrow$ | Toys $100\Delta \uparrow$ | Mean across domains: $100\Delta$ (pp) $\uparrow$ | | --- | ---: | ---: | ---: | ---: | | Joint Conditioning (Full-FT) | **$\mathbf{9.99 \pm 0.70}$** | **$\mathbf{5.59 \pm 0.38}$** | **$\mathbf{8.00 \pm 2.64}$** | **7.86** | | Joint Conditioning (LoRA) | $4.86 \pm 0.59$ | $1.09 \pm 0.78$ | $3.01 \pm 0.67$ | 2.99 | | Frozen Global-only | $-0.05 \pm 0.27$ | $0.00 \pm 0.28$ | $0.17 \pm 0.42$ | 0.04 | | Frozen Cross-Attention-only | $1.42 \pm 0.19$ | $1.03 \pm 0.53$ | $1.94 \pm 0.50$ | 1.46 | | Dual-Branch Injection (ours) | $3.80 \pm 2.20$ | $2.68 \pm 1.14$ | $7.27 \pm 0.71$ | 4.58 | Dual-Branch exceeds both frozen single-branch interfaces in every domain, and LoRA on Sports, Toys, and the domain average. LoRA is higher on Beauty in selectivity and Sentiment@10. Full-FT has the highest selectivity despite lower lexical robustness, supporting separate evaluation of these behaviors. Selectivity covers the labeled catalog groups. The evaluation is in §5.1 and Appendix E.7; the abstract (p. 1, lines 22–32; highlighted) and §§1, 5.4, and 6 state the narrower conclusions. Appendix E.1 (p. 24, lines 1270–1272; highlighted) and Figure 5 treat the attention example as qualitative, not proof of keyword-only behavior.
Reply to Reviewer ghQj
We thank ghQj for their feedback!
Some naming / terminology clarification in our replies below: Dual-Branch Injection corresponds to Final-State Injection in the initial submission. Manuscript references use revised numbering unless marked “submitted” (which means the original submitted version); page and line references point to the revised, highlighted PDF.
CW1. Why use a causal LLM rather than a bidirectional or smaller encoder?
Our goal is retrieval as an add-on to a resident causal LM: its hidden states condition a trainable SID generator without updating the host. Reusing the host avoids loading separate text-encoder weights into accelerator HBM in this deployment. The SID generator adds its own memory and compute costs; we make no measured latency claim. The deployment motivation is in §1 (p. 1, lines 51–53; highlighted).
Using the same pretrained Gemma-3-1B-IT checkpoint, tokenizer, and hidden dimension controls the language backbone across interfaces. A bidirectional or smaller encoder comparison would address standalone retriever efficiency, which we have not evaluated. The backbone control is described in §§1 and 4, and the encoder-comparison scope in §6.
The revised manuscript clarifies this motivation in §1 and corrects the initialization description in §4.2 (p. 6, lines 279–281 and 302–303; highlighted), Figure 2, and Appendix C.1: only the language tower loads pretrained weights; the SID generator uses the Gemma-3-1B architecture with random initialization.
CW2. Does freezing limit adaptation to domain-specific terminology or new preference types?
Retrieval adaptation occurs in the trainable modulation projections, cross-attention, and SID generator, while the host representations remain fixed. This supports new request formats already representable by the host. The trainable pathways are specified in §4.2 (p. 6, lines 286–304) and Appendix C.1.
The submitted Intent Following task presents up to five candidate preferences with distractors, a format absent from training. Dual-Branch vs. Full-FT test R@10 is 5.55% vs. 6.38% on Beauty, 2.72% vs. 2.84% on Sports, and 4.84% vs. 4.60% on Toys (Train = 0; validation Intent Following is used in model selection; §§3.2 and 5.1, Appendices D.2 and E.4, Table 11). Under syntactic and conversational rewrites, Dual-Branch exceeds LoRA in all domains and Full-FT on Sports and Toys (Table G1).
The new Steam experiments extend the domain evaluation to games (Q2), and attribute flips test responses to changed requests without further training (Q1). Unfamiliar terminology still depends on the host's representations; these experiments do not test learning new concepts or zero-shot domain transfer. This scope is stated in §6 (p. 10, lines 513–523; highlighted).