acceptodds
Under review as a conference paper at ICLR 2027

OxygenFlowRec: Semantic-ID Recommendation via One-Step Continuous Generation

10 views

Abstract

Semantic-ID (SID) recommenders commonly predict discrete code tuples, coupling recommendation to token-level decoding and, in beam-based methods, a search budget. We ask whether SID-based semantic sharing and SID outputs can be retained without discrete code prediction. We propose OxygenFlowRec, a catalogue-grounded SID recommender based on one-step continuous generation. Fixed SID assignments define parameter sharing, while task-trained code embeddings and frequency-gated item residuals learn recommendation-specific geometry. Conditioned on user history, the model generates a continuous query in a single update, ranks catalogue items by similarity, and returns their SIDs through deterministic lookup while preserving exact item identities. Training combines target recovery with supervision of the actual inference endpoint. To accommodate the different gradient aggregation patterns of shared codes and item residuals, we assign separate learning-rate scales to these parameter groups. This design separates the continuous generation process from the discrete SID output interface, avoiding token-level decoding and beam search. Experiments on six Amazon Reviews 2023 categories demonstrate competitive recommendation accuracy, while inference benchmarks show substantially lower retrieval cost relative to our beam-based LLaDA-Rec reproduction.

Primary area:
generative models
Keywords:
Generative Recommendation,Continuous Generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_mCKZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes OxygenFlowRec, a one-step continuous recommender that retains semantic-ID structure without autoregressively or diffusively generating SID tokens. It predicts a continuous query in one flow-style step, retrieves items using full-catalog cosine similarity, and returns their fixed SIDs through a lookup. Experiments on Amazon datasets compare accuracy and efficiency with dense, generative, and diffusion-based recommenders.

Critical strengths
  1. The method avoids autoregressive SID decoding and beam search, providing a potentially useful accuracy-efficiency tradeoff.
  2. The paper reports several ablations, matched controls, collision analyses, and latency measurements.
Critical weaknesses
  1. The presentation is very difficult to follow. The paper lacks a clear narrative and often introduces symbols and design choices without adequate definitions or motivation. This is especially prominent in section 3. These are central parts of the method, making the paper difficult to understand and reproduce.
  2. The contribution is insufficiently distinguished from dense retrieval. Inference ultimately produces one continuous query, performs full-catalog cosine retrieval, and looks up the selected items’ fixed SIDs. The brief related-work discussion and limited comparison with the closest methods make it unclear what is fundamentally new beyond a complicated representation and training recipe for dense retrieval.
Questions for authors
  1. In what precise sense is OxygenFlowRec a generative recommender rather than a dense retriever with SID-structured item embeddings and a fixed SID lookup?
  2. Can the authors provide clearer, matched comparisons with a capacity-matched dense retriever?
  3. Can the authors provide a self-contained definition of every quantity in Section 3?
  4. What is the motivation for frequency-gated residuals, and why should training frequency determine whether an item receives item-specific capacity? What alternatives were considered?
5 comments
AuthorsOpenReview ↗

Response to Reviewer mCKZ

We thank the reviewer for the careful assessment. We particularly appreciate the recognition that avoiding autoregressive SID decoding and beam search provides a potentially useful accuracy–efficiency trade-off, and that the paper includes matched controls, collision analyses, and latency measurements.

Below, we address the two weaknesses and four questions point by point.

1. Continuous prediction versus dense retrieval

Addresses Weakness 2 and Question 1.

At inference, OxygenFlowRec is a dense retriever: it produces one continuous query, ranks catalogue item representations by cosine similarity, and returns the selected items’ fixed SIDs through lookup. The SID lookup does not generate SID tokens, alter the item ranking, or resolve SID collisions.

Our use of “continuous generation” describes how the continuous query is trained and constructed. The network learns to recover target-item states from continuous interpolants. At inference, a single update from a source state produces the query used for catalogue ranking.

The system therefore has three distinguishable stages:

  1. Query prediction: one-step continuous target recovery conditioned on the user history;
  2. Item selection: dense similarity retrieval over the catalogue;
  3. SID readout: deterministic lookup after the item IDs have been selected.

We will use “one-step continuous query prediction with catalogue retrieval” as the precise operational description.

The contribution is not the use of dense retrieval alone. Our modeling question is how to train the predictor and item space when fixed SID assignments determine parameter sharing, item-specific residuals provide additional capacity, and catalogue CE supervises the actual one-step inference endpoint.

Thus, SID assignments fix the parameter-sharing structure, while the item geometry and one-step predictor are jointly trained for the recommendation task. The controlled comparisons below test whether these choices improve ranking beyond a direct dense predictor.

2. Comparison with capacity-matched dense retrievers

Addresses Weakness 2 and Question 2.

We agree that a close dense-retrieval comparison is essential. Table 4 already separates the effects of representation and training recipe:

Representation and trainingScientific R@20Toys R@20
Direct CE, item-only0.06940.0530
Direct CE, SID + residual0.07130.0549
Full recipe, item-only0.07130.0541
Full recipe, SID + residual0.07390.0566
No recovery, SID + residual0.07200.0555

Our closest capacity-matched control is Hybrid Direct CE: it uses the same trainable retrieval parameterization and inference architecture as Full, while differing in the training objectives. Training-only auxiliary components are accounted for separately.

With SID + residual representations, Direct CE obtains R@20 of 0.0713 on Scientific and 0.0549 on Toys, versus 0.0739 and 0.0566 for Full. This comparison changes multiple training objectives, so the entire gap should not be attributed to recovery alone.

The no-recovery control changes only the recovery coefficient within the full recipe. It obtains 0.0720 on Scientific and 0.0555 on Toys, providing a more targeted comparison of recovery in the tested setting.

Near parameter-budget-matched dense alternatives

The existing item-only and hybrid models have similar but not exactly equal total parameter counts. We add two direct dense predictors:

  1. Near parameter-budget-matched dense: Starting from the item-only Direct CE predictor, we increase the hidden width of the four prediction-head MLP blocks from 2048 to 2060 on Scientific and to 2054 on Toys. This adds approximately 0.10M and 0.05M parameters, respectively, while retaining the 512-dimensional output and the same retrieval architecture.

  2. Near parameter-budget-matched text-enhanced dense: We concatenate each item's title, brand, categories, features, and description, encode them with a frozen Sentence-T5-base encoder followed by PCA to 128 dimensions, and add a trainable 128-to-64-to-512 projection of these features to the item-ID vector before normalization. We adjust the prediction-head width so that the total recommender parameter budget remains close to Full.

The trainable text projection is included in the recommender parameter count. The frozen Sentence-T5 encoder is excluded. Storing one 128-dimensional FP32 text vector per item requires 12.6 MiB on Scientific and 79.1 MiB on Toys, reported separately from model parameters and peak inference memory.

ModelScientific total parametersScientific R@20Toys total parametersToys R@20
Direct CE, item-only41.06M0.0694110.79M0.0530
Direct CE, near-budget-matched dense41.16M0.0700110.84M0.0535
Direct CE, text-enhanced dense41.16M0.0718110.84M0.0555
Direct CE, SID + residual41.19M0.0713110.92M0.0549
Full OxygenFlowRec41.19M0.0739110.92M0.0566

The alternatives use the same interaction split, history limit, training budget, exact-item evaluator, and validation selection rule. They are near parameter-budget-matched, not architecturally identical.

Relative to the near-budget-matched dense predictor, SID + residual improves R@20 under Direct CE by 1.9% on Scientific and 2.6% on Toys. Full improves R@20 by 5.6% and 5.8%, respectively. These comparisons directly test whether the proposed model improves over a dense retriever with a similar recommender parameter budget.

The text-enhanced dense model is a stronger comparison than an item-ID-only table. Its Direct CE results exceed Direct CE with SID + residual on both datasets; Full OxygenFlowRec remains higher. We therefore do not claim that SID + residual is uniformly better than every direct dense representation.

The original SID assignments also matter. In Table 4, shuffling complete SID tuples reduces R@20 from 0.0739 to 0.0694 on Scientific and from 0.0566 to 0.0538 on Toys. The shuffle preserves tuple-use counts and collision sizes, so it tests the assignment structure rather than simply the presence of an additional embedding table.

We also compare close continuous methods on Scientific and Instrument:

MethodScientific R@20Instrument R@20
FAVE0.07350.0950
Hybrid Direct CE0.07130.0925
MIRAGE0.07390.0948
Full OxygenFlowRec0.07390.0954

MIRAGE and OxygenFlowRec match at the displayed precision on Scientific. The Instrument comparison favors OxygenFlowRec among these rows. These single-seed point comparisons do not establish universal superiority over continuous retrievers. The original MIRAGE comparisons remain available in Appendix M.3.

3. Self-contained definitions for Section 3

Addresses Weakness 1 and Question 3.

We agree that Section 3 should define each quantity and its role before introducing the complete objective. We will reorganize it around item representation, history encoding, training, optimization, and inference, using the following compact definitions.

3.1 Item representation

Let I\mathcal I be the catalogue and N=∣I∣N=|\mathcal I|. For item ii:

SymbolDefinition and role
ci=(ci1,…,ciM)c_i=(c_{i1},\ldots,c_{iM})Fixed SID tuple assigned to item ii
MMNumber of codebook positions
KcbK_{\mathrm{cb}}Number of codes per codebook
Em∈RKcb×dcE_m\in\mathbb R^{K_{\mathrm{cb}}\times d_c}Trainable embedding table for codebook position mm
dcd_c, D=MdcD=M d_cCode-embedding and item-vector dimensions
ri∈RDr_i\in\mathbb R^DTrainable item-specific residual
fif_iItem frequency in the training interactions
g(fi)g(f_i)Frequency-dependent residual gate
α\alphaResidual scale
s(i)s(i), sˉ(i)\bar{s}(i)Item representation and its normalized version

The item representation is

sbase(i)=1Mconcat⁡(E1[ci1],…,EM[ciM]),s_{\mathrm{base}}(i) = \frac{1}{\sqrt M} \operatorname{concat} \left(E_1[c_{i1}],\ldots,E_M[c_{iM}]\right),

s(i)=sbase(i)+αg(fi)ri,sˉ(i)=s(i)∥s(i)∥.s(i)=s_{\mathrm{base}}(i)+\alpha g(f_i)r_i, \qquad \bar{s}(i)=\frac{s(i)}{\Vert s(i)\Vert }.

The gate is

g(fi)=min⁡(1,max⁡(fi−1,0)8).g(f_i)= \min\left( 1,\sqrt{\frac{\max(f_i-1,0)}{8}} \right).

Thus, an item observed once has zero residual contribution, and the gate reaches one at nine training occurrences.

The main configuration uses M=4M=4, Kcb=256K_{\mathrm{cb}}=256, dc=128d_c=128, D=512D=512, and α=0.2\alpha=0.2.

3.2 User history, state, and history-encoder query token

SymbolDefinition and role
uu, Su=(i1,…,iL)S_u=(i_1,\ldots,i_L)User and their history sequence
LL, y=iL+1y=i_{L+1}History length and next-item target
x1=sˉ(y)x_1=\bar{s}(y)Normalized target-item state
x0x_0Sampled continuous source state
ttTraining interpolation time
xt=(1−t)x0+tx1x_t=(1-t)x_0+t x_1Current interpolated training state
ϵ\epsilonBound used for training time and the inference source
q0q_0Learnable query token appended to the history sequence
h(Su,xt,t)h(S_u,x_t,t)History context conditioned on state and time

The source is sampled as x0∼N(0,σ2I)x_0\sim\mathcal N(0,\sigma^2 I) with σ=0.03125\sigma=0.03125. Training samples t∼U(0,1)t\sim U(0,1) and clips it to [ϵ,1−ϵ][\epsilon,1-\epsilon], where ϵ=10−3\epsilon=10^{-3}.

The history encoder is a six-layer, eight-head Transformer. Its state-dependent update to the appended query token is

Δq=γϕ(xt,t)1+∥ϕ(xt,t)∥/β,\Delta q= \gamma\frac{\phi(x_t,t)} {1+\Vert \phi(x_t,t)\Vert /\beta},

where ϕ\phi combines learned state and time projections, γ=2.0\gamma=2.0, and β=0.5\beta=0.5.

Δq\Delta q updates the learnable history-encoder query token q0q_0. It is distinct from the final continuous retrieval query x^\hat{x}.

3.3 Recovery and inference endpoint

FθF_\theta is the target-recovery network with parameters θ\theta. At an interpolated state, it predicts

x^1=Fθ(xt,t,h(Su,xt,t)).\hat x_1=F_\theta(x_t,t,h(S_u,x_t,t)).

At inference, the context is recomputed at the source state:

h0=h(Su,x0,ϵ).h_0=h(S_u,x_0,\epsilon).

One update produces the retrieval query

x^=x0+(1−ϵ)(Fθ(x0,ϵ,h0)−x0).\hat{x} = x_0+(1-\epsilon) \left(F_\theta(x_0,\epsilon,h_0)-x_0\right).

At inference, x0x_0 is generated deterministically from a history hash; the predictor does not access the target-containing training interpolant.

Reviewer_u1rqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper uses fixed SID assignments to share trainable code embeddings across items, with frequency-gated residuals for item-specific information. A history-conditioned network learns target recovery and receives catalogue-ranking supervision at the one-step inference endpoint. Recommendations are obtained by scoring the catalogue with the predicted continuous query; the selected items’ SIDs are then returned by lookup. The experiments cover six Amazon Reviews 2023 categories, with representation and training controls on Scientific and Toys and retrieval timing against a LLaDA-Rec reproduction. All Beauty is a negative result, and the controlled continuous comparisons show modest accuracy gains. The clearest measured practical benefit is competitive accuracy with lower retrieval cost against the tested discrete beam decoder.

Critical strengths
  1. Coherent representation design. Task-trained code embeddings support semantic sharing, while item residuals provide item-specific capacity. This is a sensible way to retain SID structure while adapting representations to recommendation.

  2. Simple and clearly specified inference. The model produces one continuous query, ranks catalogue items, and returns their SIDs through lookup. Separating item selection from SID readout preserves item identities and ranking even when SID tuples are duplicated.

Critical weaknesses
  1. The paper’s contribution needs to be positioned more clearly against recent work. CCFRec [1] uses shared semantic-code embeddings to construct continuous item representations and performs similarity-based recommendation without SID decoding at inference. FAVE [2] uses flow-based average-velocity modeling to generate an item representation in one step, followed by catalogue ranking. Neither work covers OxygenFlowRec’s full design, but both provide relevant precedents for its representation and prediction mechanisms. The controlled results support the usefulness of SID sharing; what remains less clear is the new modeling insight gained by combining it with the proposed one-step training scheme.
  2. The changes to representation and optimization are relatively straightforward, with limited methodological novelty. Combining shared code embeddings with item residuals is sensible, but the role of frequency gating needs a more precise explanation. For an unconstrained residual, a positive gate can be absorbed into the residual’s scale without changing representational capacity. Only disabling the residual imposes an explicit restriction; otherwise, the gate mainly affects optimization and regularization. The grouped learning rates raise a similar concern: differences in gradient aggregation do not, by themselves, justify the chosen multipliers after AdamW’s adaptive normalization. The authors acknowledge that this is an empirical choice. As presented, this part reads more as an optimization configuration tailored to the model than as a substantial methodological contribution.
  3. The paper uses item frequency both to allocate residual capacity and to rebalance query learning, but the reasoning behind these choices needs to be clearer. When two items share the same SID tuple and both have their residuals disabled, they receive identical scores for every query; keeping separate item IDs does not make them distinguishable. Frequency alone may therefore miss cases where an item needs additional capacity because its semantic codes cannot distinguish it from other items. In the query-balancing correction, the catalogue is detached, but gradients can still reach item parameters through the history embeddings. The paper should explain why frequency is a suitable basis for deciding which items need their own residuals, and what imbalance is corrected by blocking direct catalogue updates while retaining these indirect updates.
  4. The practical value of the added representation and training choices is not yet clear. The clearest gains are in accuracy and retrieval efficiency over the tested LLaDA-Rec beam-decoding reproduction. Against the hybrid direct-CE predictor, the relative Recall@20 gains shrink to about 3.6% on Scientific and 3.1% on Toys. Results are close to MIRAGE, while performance on All Beauty falls below LLaDA-Rec, although the small test set warrants caution. The efficiency study does not include direct continuous retrievers, and the method retains a catalogue-sized residual table whose storage cost should be considered alongside retrieval latency. These comparisons leave unclear when the full design offers a worthwhile advantage over simpler alternatives.

[1] Enze Liu, Bowen Zheng, Wayne Xin Zhao, and Ji-Rong Wen. Bridging Textual-Collaborative Gap through Semantic Codes for Sequential Recommendation. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2025, pp. 1788-1798.

[2] Ke Shi, Yao Zhang, Feng Guo, Jinyuan Zhang, JunShuo Zhang, Shen Gao, and Shuo Shang. FAVE: Flow-based Average Velocity Establishment for Sequential Recommendation. In Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2026, pp. 1633-1643.

Questions for authors
  1. Is SID sharing intended as a general representation prior for continuous retrieval, or is there a specific modeling reason to pair it with one-step recovery?
  2. For positive frequency gates, is the intended benefit primarily optimization or regularization? How does this differ from adjusting the residual learning rate under AdamW?
  3. What is the rationale for allowing recovery gradients to update both the target item representations and the interpolants? Which relationships between items is this joint adjustment intended to preserve or change? Since recovery error can decrease through changes to the target space as well as improved prediction, please explain how catalogue CE constrains this process and why the resulting geometry should improve ranking at inference.
  4. Under the same hardware, precision, catalogue caching and top-20 output, how does OxygenFlowRec compare with direct continuous retrievers in accuracy, retrieval latency and memory use? Separately, what training time and peak memory do Full, hybrid direct CE and the minimal recipe in Table 25 require? Since these variants share the serving architecture, their training costs would help clarify whether the additional objectives are worth retaining.
  5. Why does the query-balancing correction stop direct gradients to the catalogue while retaining gradients through the query? Since query gradients can still update item parameters through history embeddings, what effect is this asymmetric routing intended to have on query learning versus item-representation learning?
5 comments
AuthorsOpenReview ↗

Response to Reviewer u1rq

We thank the reviewer for the careful reading and constructive feedback. We particularly appreciate the recognition of our SID-structured representation, which combines semantic sharing with item-specific capacity, and our clear separation between item selection and SID readout, which preserves item identities and ranking.

Below, we address the four weaknesses and five questions point by point. Our central modeling question is how to train a one-step predictor when SID assignments fix the parameter-sharing structure, while the item geometry remains trainable for recommendation.

1. Contribution relative to CCFRec and FAVE

Addresses Weakness 1 and Question 1.

We thank the reviewer for pointing out CCFRec and FAVE. We agree that semantic-code-based continuous representations and one-step continuous recommendation have relevant precedents. We will cite and discuss both works and do not claim either component as an independent first.

Our modeling contribution is more specific:

SID assignments fix the parameter-sharing structure, while the item geometry remains task-trained. We jointly optimize this space and the one-step predictor using recovery supervision and catalogue CE at the actual inference endpoint. Our matched controls test whether this training improves ranking beyond direct CE with the same hybrid representation.

This also clarifies the relationship to FAVE. FAVE establishes a preference space in its first stage and freezes the embedding layer during second-stage one-step consolidation. In OxygenFlowRec, the SID assignments remain fixed, but the code embeddings and item residuals remain trainable: recovery can update the item states as well as the predictor, while endpoint catalogue CE directly supervises the ranking used at inference.

This is a concrete difference in training design, not a claim that joint optimization is intrinsically superior to freezing. The relevant question is whether joint training and endpoint supervision improve ranking under controlled comparisons.

SID sharing is intended as a general representation prior for continuous retrieval; it does not require recovery. Pairing it with recovery is the specific modeling choice studied here.

Representation and recovery controls

We cross two representations with recovery enabled or disabled while retaining the other components of the full training recipe:

RepresentationRecoveryScientific R@20Toys R@20
Item-onlyOff0.07030.0533
Item-onlyOn0.07130.0541
SID + residualOff0.07200.0555
SID + residualOn0.07390.0566

Recovery improves both representations. Its absolute gain is 0.0010 for item-only and 0.0019 for SID + residual on Scientific; the corresponding gains on Toys are 0.0008 and 0.0011.

The difference in recovery gains is 0.0009 on Scientific and 0.0003 on Toys. These controls show that SID sharing and recovery can each contribute to the final ranking objective. The interaction is small, especially on Toys, and these single-seed point comparisons do not establish a statistically reliable synergy.

The gradient-path controls in Section 3 further examine whether recovery acts only as additional predictor supervision or also benefits from updating item representations.

Comparison with CCFRec

CCFRec is an important baseline. Its paper reports Recall@10 of 0.0555, 0.0682, and 0.1042 on Scientific, Instrument, and Game, respectively. The corresponding OxygenFlowRec values in our Table 1 are 0.0484, 0.0628, and 0.0940.

DatasetCCFRec published R@10OxygenFlowRec Table 1 R@10
Scientific0.05550.0484
Instrument0.06820.0628
Game0.10420.0940

These published CCFRec results are higher. CCFRec uses five textual fields—title, brand, categories, features, and description—in addition to its semantic-code-based fusion. Although the reported dataset statistics match ours, this alone does not establish that the item content, tokenizer, data split, checkpoint selection, and evaluator are identical. We therefore retain the published result as a separate full-input comparison.

Comparison with direct continuous retrieval

For a direct continuous-retrieval comparison under the matched protocol in Appendix M.3, SASRec with full-catalogue CE obtains R@20 of 0.0694 on Scientific and 0.0538 on Toys, versus 0.0739 and 0.0566 for Full OxygenFlowRec. These correspond to relative gains of approximately 6.5% and 5.2%.

MIRAGE obtains 0.0739 R@20 on Scientific, matching OxygenFlowRec at the displayed precision.

2. Frequency gating and grouped learning rates

Addresses Weaknesses 2–3 and Question 2.

We agree with the reviewer’s representational argument. For

s(i)=sbase(i)+αg(fi)ri,s(i)=s_{\mathrm{base}}(i)+\alpha g(f_i)r_i,

a positive gate can be absorbed into an unconstrained residual by rescaling rir_i. Therefore, a positive gate does not by itself reduce the set of representable item vectors. When g(fi)=0g(f_i)=0, the residual contribution is removed and an explicit restriction is imposed.

Why use frequency for the residual gate?

The motivation is to use interaction frequency as a rough proxy for the amount of evidence available to learn an item-specific deviation. Items with limited interaction evidence rely more heavily on the SID-shared representation; items with more evidence can make greater use of their residual contribution.

This is a heuristic rather than a claim that an item's residual only receives gradients when that item is observed. Under full-catalogue CE, negative catalogue items can also receive direct scoring gradients. Frequency nevertheless provides a simple signal about the available interaction evidence.

For a positive gate, we regard the intended benefit as primarily an optimization effect: the gate modulates how the item-specific displacement develops relative to its shared SID representation during training.

At the representation level, a positive gate can be absorbed into an unconstrained residual. During optimization, however, it changes the parameterization and the contribution of the residual to the item representation. Under simplifying assumptions, its effect on the displacement update can resemble frequency-dependent learning-rate scaling; with AdamW, the two training trajectories are not generally identical.

Frequency is also an imperfect proxy for whether an item needs its own capacity. If two items share a complete SID tuple and both residuals are disabled, their representations and scores are identical for every query. Retaining separate item IDs preserves their identities but does not make them distinguishable.

The existing frequency analysis also shows why this allocation involves a trade-off. In Scientific, among targets observed two or three times during training, removing all item residuals increases the number of R@20 hits from 9 to 33; it simultaneously reduces hits substantially in the frequent-target buckets. This comparison removes the residual rather than only changing the gate, so it does not isolate the gate's effect.

Gate versus learning-rate and collision-aware alternatives

We compare the original gate with a model without gating, a binary gate, a frequency-dependent residual learning-rate alternative, and a collision-aware allocation. The collision-aware setting uses a matched residual budget.

Scientific, seed 13R@20N@20
No frequency gate0.07450.0320
Binary gate0.07340.0319
Frequency-dependent residual learning rate0.07360.0320
Collision-aware gate0.07400.0322
Original frequency gate0.07390.0322

At the displayed precision, collision-aware gating and the original gate tie on N@20; collision-aware gating has a slightly higher R@20. The five settings differ by roughly 1.5% or less in R@20 and about 1% in N@20. The comparison does not support claiming that the original gate uniformly improves either metric.

Grouped learning rates

We agree that gradient aggregation does not theoretically determine the learning-rate multipliers after AdamW normalization. We will present the grouped multipliers as an empirically evaluated optimization configuration, rather than as a theoretically derived contribution.

In the existing equal-budget Scientific comparison, the selected shared-code multiplier performs best among the tested settings. At seed 13, the same-seed ablation with equal item-group multipliers lowers R@20 from 0.073865 to 0.060371. This supports the selected training recipe, while not proving a particular gradient-aggregation mechanism.

3. Recovery gradients, moving targets, and ranking geometry

Addresses Question 3.

Let x1=sˉ(y)x_1=\bar{s}(y) be the normalized target-item representation and

xt=(1−t)x0+tx1.x_t=(1-t)x_0+t x_1.

The recorded configuration allows recovery gradients to update the prediction network and reach item-representation parameters through both the target and interpolant paths.

The purpose is joint adaptation: the predictor learns to recover task-trained item states, while the shared SID embeddings and item residuals can adapt to the recommendation objective.

What remains fixed is the SID assignment and its parameter-sharing structure. We do not fix pairwise item distances, and we do not claim that recovery preserves a predetermined semantic geometry.

The reviewer is correct that lower recovery error can result partly from changes to the target space. The target vectors are normalized, so simply shrinking their raw magnitudes cannot by itself shrink the normalized target vectors. However, normalization does not rule out directional collapse or establish a general non-collapse guarantee.

Rollout catalogue CE provides a complementary constraint. For a fixed query, its softmax objective increases when a negative item's logit rises relative to the target item's logit. It therefore directly supervises target–candidate discrimination at the inference endpoint. It does not prove that every change to the item geometry is beneficial; retrieval metrics test whether the learned predictor and item space improve ranking together.

Gradient-path controls

We separately stop recovery gradients through the target and interpolant paths while retaining the other objectives:

Scientific, seed 13Target pathInterpolant pathRecovery errorR@20
Detach target and interpolantOffOff0.630.0715
Detach targetOffOn0.580.0721
Detach interpolantOnOff0.560.0727
FullOnOn0.550.073865

The two-detach setting reaches 0.0715 R@20, below the 0.0720 no-recovery result, while the full two-path setting reaches 0.073865. This pattern is consistent with an important role for recovery's item-representation updates in the tested configuration.

It does not, by itself, prove that all recovery gains come from representation shaping: the additional predictor gradient may interact with other objectives, and the separately trained models can learn different spaces. The single-path controls and the final retrieval metric provide the more informative comparison.

Likewise, the recovery errors are not errors against one fixed target space. Even on the same held-out examples, separately trained models have different target representations. The decrease from 0.63 to 0.55 alone does not prove better prediction on an unchanged geometry or preservation of semantic distances.

The existing rollout-supervision ablation lowers Scientific R@20 from 0.073865 to 0.040679. This ablation also removes the associated query-balancing correction, so it measures the combined rollout branch rather than catalogue CE in isolation.

Reviewer_uDywOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2(+1)
Presentation
1
Soundness
2(+1)
Read review
Paper summary

The paper proposes OxygenFlowRec, aiming to achieve "one-step continuous generation" in generative recommendation.

The proposed method is to:

  1. Instead of representing each item as semantic ID tokens, representing each item as one single representation as input of the sequence model. The single representation comes from the sum of semantic ID token embedding and a (weighted) item embedding.
  2. The generation process first requires a query token as the last position of input, then involves MLP layers.
  3. The final inference procedure is to calculate similarities between the "generated" representation with all item representations.

Experiments are conducted on public datasets.

Critical strengths

None

Critical weaknesses
  1. The proposed method violates the basic motivation of semantic ID-based generative recommendation. A key motivation in developing semantic ID-based generative recommendation is that conventional models suffer from large item embedding table. In real production scenarios, it's easy to get millions or billions of items, and maintaining/optimizing/serving such a large item embedding table is very challenging. This also makes it difficult to scale up models by making it deeper like nowadays large language models. Semantic ID-based generative recommendation replaces this large item embedding table by compact semantic ID token embedding tables shared by all items. However, a large item embedding table becomes a necessity again in the proposed method. For input side, it requires a "residual" per item (aka item embedding) combined with semantic ID token embeddings. For inference side, similarity scores are calculated across all existing item embeddings. This design fundamentally against the motivation we are studying semantic IDs and generative recommendation.
  2. The writing can be improved.
    1. Section 2 is overly compressed.
    2. Some terms (that are not widely used in this community) are introduced without explanation. For example, "flow state" in line 182.
  3. Experimental setting is questionable.
    1. The proposed method has both semantic ID token embeddings and item embeddings. However, there is no baseline having both embeddings. Given that, we don't know whether the gains come from the ensemble of both embeddings, or from the continuous generation process.
    2. The authors claimed that they use six datasets. Three datasets were used for overall comparison of the proposed method against baselines. Two datasets were used for ablation study. There is only one dataset shared between overall comparison and ablation study, so as other analytical experiments. Not really get why we need different datasets everywhere.
Questions for authors

None

5 comments
AuthorsOpenReview ↗

Response to Reviewer uDyw

We thank the reviewer for the detailed feedback. Below, we clarify the motivation for retaining SID structure, distinguish the effects of the hybrid representation and recovery-based training, and address the dataset organization, practical costs, and presentation.

1. SID-indexed sharing remains useful with item-specific parameters

We respectfully disagree with the conclusion that retaining item-specific parameters defeats the motivation of our approach. Our objective is to retain SID-indexed semantic sharing without requiring discrete token prediction, rather than to reproduce the code-only compression of discrete SID models. These are distinct benefits, and our controlled experiments test whether the former remains useful when item-specific capacity is present.

The SID literature motivates semantic IDs through both compact representations and sharing across semantically related items. TIGER, for example, reports improved generalization to items without prior interaction history. OxygenFlowRec retains the sharing aspect while trading away code-only compression.

Dense item representations and catalogue retrieval are also established design choices in recommendation. TIGER describes large-scale retrieval using query and item embeddings followed by approximate nearest-neighbour search. Hybrid methods such as LIGER and COBRA combine semantic IDs with dense representations to address accuracy trade-offs associated with discrete generation. Separately, HSTU-based generative recommenders show that models using high-cardinality recommendation features can be scaled in large production systems.

These examples do not establish that OxygenFlowRec itself has been validated at million- or billion-item scale. They do show that retaining dense or item-specific information is a legitimate design choice rather than, by itself, a contradiction of the goals of recommendation research.

In OxygenFlowRec, fixed SID assignments determine which code parameters are shared across items, whereas item residuals provide additional capacity to distinguish individual items. The relevant empirical question is whether the SID assignment contributes beyond simply adding item-specific parameters.

SID sharing versus item-only representations

Table 4 compares item-only and SID + residual representations under both direct catalogue CE and the full training recipe:

Training and representationScientific R@20Toys R@20
Direct CE, item-only0.06940.0530
Direct CE, SID + residual0.07130.0549
Full recipe, item-only0.07130.0541
Full recipe, SID + residual0.07390.0566

SID sharing improves the reported result under both training settings. The item-only and hybrid models have similar, although not exactly equal, total parameter counts.

Original versus shuffled SID assignments

We also compare the original SID assignments with shuffled assignments. Shuffling complete SID tuples preserves tuple-use counts and collision sizes while changing which tuple is assigned to each item. The model architecture and code-embedding parameterization remain the same.

DatasetShuffled SID R@20Original SID R@20
Scientific0.06940.0739
Toys0.05380.0566
Instrument0.09110.0954
Game0.13620.1422

Because the shuffled model retains the additional code embeddings, this comparison tests more than whether another embedding table is present. The consistent difference indicates that the item-to-SID assignment itself affects performance.

Together, the item-only and shuffled-SID controls support the usefulness of SID-indexed sharing even when item-specific residuals are retained.

2. Recovery-based training adds value beyond the hybrid representation

The paper already includes the baseline requested by the reviewer: Hybrid Direct CE uses both SID embeddings and item residuals.

The Hybrid Direct CE control uses the same representation parameterization, history encoder and prediction-head architecture, inference source convention, and query-balancing correction. The comparison therefore examines the training objectives rather than giving only OxygenFlowRec access to the hybrid representation.

We compare four training configurations:

  1. Hybrid Direct CE: catalogue CE with the query-balancing correction;
  2. Minimal: recovery plus catalogue CE and the query correction;
  3. No recovery: the full recipe with only the recovery coefficient removed;
  4. Full: the complete OxygenFlowRec training recipe.
DatasetHybrid Direct CEMinimal: Recovery + CENo recoveryFull
Scientific0.07130.07240.07200.0739
Toys0.05490.05580.05550.0566
Instrument0.09250.09380.09400.0954
Game0.13850.14010.14040.1422

The controls answer different questions:

  • Minimal versus Direct CE: This tests recovery without retaining the other auxiliary objectives.
  • Full versus No recovery: This tests recovery within the full recipe by removing only its coefficient.
  • Full versus Direct CE: This compares complete training recipes. Since multiple objectives differ, the entire gap should not be attributed to recovery alone.

All eight comparisons that isolate recovery favor it: Minimal exceeds Direct CE by approximately 1.2–1.6%, and Full exceeds No recovery by approximately 1.3–2.6%, across the four datasets. These are single-seed point comparisons and do not by themselves establish statistical significance.

The Scientific and Toys values are reported in Tables 4 and 25. The Instrument and Game controls use seed 13, the 24-epoch budget, and validation item-R@20 checkpoint selection of Table 4. Their Full values correspond to the archived checkpoints in Table 14.

Together, these controls distinguish the benefit of SID-indexed sharing from the additional benefit of recovery-based training. The gains are not fully explained by giving the model both SID and item embeddings.

3. Accuracy, retrieval cost, and storage trade-offs

OxygenFlowRec retains a catalogue-sized residual table and scores the full catalogue, so it does not offer the storage compression of a code-only decoder. The paper states this trade-off in the related-work, efficiency, and limitations sections.

At the same time, the experiments demonstrate an accuracy–retrieval trade-off against the tested beam decoder. OxygenFlowRec has higher R@20 than our LLaDA-Rec reproduction on five of the six categories:

DatasetLLaDA-Rec reproductionOxygenFlowRec
Scientific0.0656270.074074
Instrument0.0859520.095406
Game0.1383470.142198
Toys0.0455160.056634
Digital Music0.0067660.013867

All Beauty has 253 test users and is a failure case: the reproduction achieves 0.075099 R@20 versus 0.051383 for OxygenFlowRec, as reported in Table 2.

On Toys, the largest category in our study, R@20 increases from 0.045516 to 0.056634, a relative gain of 24.4%.

The reproduction uses its original SID-level evaluator, whereas OxygenFlowRec is evaluated at the exact-item level. Scientific is reported as a three-seed mean; the other OxygenFlowRec values above use seed 13. The Scientific, Instrument, and Game R@20 values come from the archived extended-cutoff comparison in Table 14. We keep these evaluation, checkpoint, and reporting differences explicit.

On Toys, OxygenFlowRec has approximately 110.9M total model parameters, compared with 6.85M for our LLaDA-Rec reproduction. Under the reported B200/FP32 cached-retrieval setup, OxygenFlowRec takes 0.133 versus 2.906 ms per user at batch 64, and 7.432 versus 13.945 ms at batch 1. The 21.9× batch-64 figure is an amortized per-user retrieval measurement; the batch-1 speedup is 1.88×.

Inference memory and latency

We compare the methods under the same B200, FP32, cached-catalogue, and top-20 retrieval protocol. Peak memory is measured as peak allocated GPU memory at batch 64, including the resident catalogue cache.

MethodBatch 1, ms/userBatch 64, ms/userPeak allocated memory, batch 64
LLaDA-Rec reproduction13.9452.9063.10 GiB
Hybrid Direct CE7.300.1304.90 GiB
Minimal: Recovery + CE7.430.1324.90 GiB
Full OxygenFlowRec7.4320.1334.90 GiB

On Toys, Full uses 1.80 GiB more peak allocated inference memory than the LLaDA-Rec reproduction at batch 64, in exchange for the reported 21.9× lower amortized per-user retrieval time and 24.4% higher R@20.

The hybrid training variants share the serving architecture and have similar retrieval costs. The additional objectives improve accuracy over Hybrid Direct CE without requiring a different retrieval architecture.

Training cost of the additional objectives

Training configurationToys R@20Training time / seedPeak training memoryRelative training time
Hybrid Direct CE0.05496.2 h18.6 GiB1.00×
Minimal: Recovery + CE0.05588.1 h22.4 GiB1.31×
No recovery0.05558.8 h24.1 GiB1.42×
Full0.056610.4 h27.6 GiB1.68×

Minimal recovers approximately 53% of the R@20 gap between Direct CE and Full on Toys at 1.31×, rather than 1.68×, the Direct-CE training time, and retains 98.6% of Full's R@20.

4. Dataset organization and coverage

All six categories are compared with the LLaDA-Rec reproduction: Table 1 covers Scientific, Instrument, and Game, and Table 2 covers Toys, Digital Music, and All Beauty. The corresponding archived R@20 results for the first three categories are in Table 14.

In the submitted version, two datasets, Scientific and Toys, already appear in both an overall comparison and the controlled analysis. The comparison scopes differ: Scientific appears in the standard three-category baseline table, whereas Toys appears in the additional comparison against our LLaDA-Rec reproduction.

Within the narrower scope of the standard three-category baseline table, Scientific is the dataset shared with the controlled analysis. We agree that these scopes were not sufficiently explicit and will add a dataset-coverage table to Section 4.1.

The additional Instrument and Game experiments extend the shuffled-SID and training controls to the other two standard-benchmark categories.

5. Definition of “flow state” and presentation

We agree that Section 2 and Section 3 should be easier to follow.

During recovery training, the continuous state is

xt=(1−t)x0+tx1,x_t=(1-t)x_0+t x_1,

where x0x_0 is the sampled source and x1x_1 is the normalized target-item representation. The predictor uses the user history and this interpolated state to predict the target representation.

At inference, the predictor uses the user history and the source state, without access to the target-containing training interpolant. One update produces the continuous query, which is used to rank catalogue items. The selected items' SIDs are then returned through deterministic lookup.

We hope these clarifications and controls make the central distinction clear: OxygenFlowRec does not preserve code-only compression, but its SID assignments provide a useful sharing structure even when the model retains item-specific capacity and performs continuous catalogue retrieval.

References:

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.