acceptodds
Under review as a conference paper at ICLR 2027

MOGR: Multi-Objective Generative Recommendation via Objective-Conditioned Distillation and Listwise Generative Policy Optimization

Abstract

Generative recommendation has emerged as a promising paradigm that generates ranked lists in a single autoregressive pass. However, user behaviors such as reads, clicks, and deeper engagement reflect distinct dimensions of user value, and this single-pass design typically forces all objectives into one merged training signal, which exposes inter-objective conflicts: improving one objective often degrades another. To address this, we propose MOGR (Multi-Objective Generative Recommendation), a single-decoder generator that emits, for each list position, a task token declaring which objective the item serves, then generates its Semantic IDs (SIDs). We train MOGR with Objective-Conditioned Distillation (OCD) and Listwise Generative Policy Optimization (LGPO). OCD distills per-objective specialists into one shared decoder under matching objective labels, so each objective keeps its own supervision. LGPO optimizes complete lists under a concave reward, tuning which objective each ranking position serves, a decision OCD alone cannot make. We further introduce OBRQ, an objective-aware codebook variant for industrial logs. On Kwai26, MOGR achieves the best Recall@512 on all three objectives, with 4.3% minimum relative gain (Min-Gain) over MD-CBS and 20.6% over TIGER, and LGPO is compared with a reward-model proxy. Online, OCD+LGPO improves the online Min-Gain over three live objectives relative to frozen OCD by 0.218% under matched exploration (p < 0.05), and by 0.258% under deployed exploration; a separate A/B test gives a 0.524% Min-Gain over the production cascade.

Primary area:
generative models
Keywords:
Multi-objective recommendation, Generative recommendation, Knowledge distillation, Reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Me1zOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MOGR generates a task token before each item's semantic ID, allowing one decoder to serve several objectives. Objective-Conditioned Distillation transfers objective-specific teachers into this decoder; LGPO updates list composition using online feedback. An additional tokenizer, OBRQ, incorporates objective-specific collaborative embeddings. Evaluation covers Kwai26 retrieval, an offline reward-model proxy, industrial codebook comparisons, and live A/B tests.

Critical strengths
  1. The online comparison addresses an important confound. Beyond comparing against greedy OCD, Table 7 includes frozen OCD with identical exploration. LGPO still improves Read/Poi/Int by 0.254%/0.218%/0.269%. This supports a benefit from learning beyond changing the exploration policy.
  2. Distillation improves all three offline objectives. In Table 1, removing OCD reduces Long-View/Like/Forward Recall@512 from 0.193/0.139/0.145 to 0.175/0.127/0.140. Table 6 also compares the student and teachers under forced objective tokens, making the transfer effect more directly observable.
Critical weaknesses
  1. LGPO's update is only partially aligned with its serving process. Advantages compare consecutive requests with changing histories, while the clipped ratio covers only one sampled task token. Other task tokens and SIDs are decoded greedily, and shared parameters continue to change. The paper acknowledges that this is not the full serving-policy ratio. The A/B supports the implemented procedure, but does not establish that the update cleanly attributes reward differences to objective placement rather than changing request states.
  2. The tokenizer results do not isolate exact-item retrieval. The evaluation accepts a gold-item or SID match, and Table 8 reports collisions under every industrial codebook. Because changing the codebook also changes item equivalence under this hit rule, Table 5 alone cannot determine how much of OBRQ's improvement survives exact-item scoring.
Questions for authors
  1. Clarification: What are the request-time spans and history changes within the online groups? Is there evidence that the reported gain is robust to the resulting within-user state variation, beyond changing group size in Figure 5?
  2. Evaluation clarification: For Table 5, how many hits match only the target SID rather than the target item, and do the codebook gains persist when these are separated?
Reviewer_PxmzOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper proposes MOGR, a multi-objective generative recommendation framework combining objective-conditioned distillation (OCD), listwise policy optimization (LGPO), and objective-aware SID quantization (OBRQ). The authors evaluate OCD on Kwai26 and provide additional industrial offline experiments and online A/B tests for the other components.

Critical strengths

1.The paper addresses a practical problem in multi-objective generative recommendation. The objective-conditioned decoder provides a reasonable way to handle multiple objectives within a single model.

2.The large-scale online A/B experiments are a strength of the paper. The matched-exploration comparison provides useful evidence for the effectiveness of LGPO.

Critical weaknesses
  1. Insufficient experimental validation given the complexity of the proposed method. The paper spends nearly five pages introducing three components, yet the main results consist of a single table on one public dataset, evaluating only SFT+OCD. LGPO and OBRQ are evaluated separately, without a comprehensive ablation showing their individual and combined contributions.

  2. Unclear necessity of the proposed components. The connections among OCD, LGPO, and OBRQ are not sufficiently justified, making the framework appear more like a combination of separate techniques than a unified method. In particular, LGPO lacks a simple fixed-allocation baseline with matched objective proportions, making it difficult to assess whether the listwise RL formulation is necessary.

Questions for authors

1.Can the authors provide a joint ablation of OCD, LGPO, and OBRQ under the same experimental setting? The current experiments evaluate these components separately, making it difficult to assess their individual necessity and combined effectiveness.

2.How does LGPO compare against a simpler fixed-allocation strategy with matched objective proportions? The current results mainly show changes in overall task-token distributions, rather than demonstrating the benefit of learned position-level allocation.

3.Why is the main evaluation limited to a single public dataset and Recall@512? Given the extensive methodology, more comprehensive experiments, including ranking-sensitive metrics and stronger baseline comparisons, would be helpful to support the claims.

Reviewer_Cc5FOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The study explores multi-objective optimization for generative recommendations. Each time the recommendation list has a slot available, MOGR creates a token representing the objective, then adds the item's SID. The same student generator represents the objectives and items selected for each position in the list using an auto-regressive sequence.

After the SFT warm-start, training consists of two major parts: Objective-Conditioned Distillation (OCD) and Listwise Generative Policy Optimization (LGPO). In OCD, the SID generation distributions of per-objective specialist decoders that share a single encoder are transferred to the student model based upon the objective conditions. Then, in LGPO, feedback from the served lists is used to improve objective selection. This is achieved by allowing the student to explore one position in each list and updating using a PPO-type algorithm. Additionally, the authors present OBRQ, which utilizes objective-specific collaborative representations and residual energy weighting to create the SIDs.

The evaluation of various components of the proposed approach was done in several ways. For example, Recall@512 on Kwai26 was used to assess SFT+OCD. LGPO was evaluated using a reward-model proxy on public data and through industrial online A/B testing. The authors also performed ablation studies on the reward form, λsid , and the window size. Finally, the authors evaluated OBRQ by replacing codebooks on their proprietary industrial data. In each case, the evaluations demonstrated improved performance across all three objectives in their corresponding contexts.

Critical strengths

The use of conditional distillation and objective selection in conjunction with each other provides the opportunity to build a single student generator from multiple expert distributions, each of these expert distributions having been transferred under the corresponding objective conditions, and this approach does not average the results from the various experts. LGPO also utilizes the same objective tokens in order to make adjustments to list generation. Only one student generator is utilized, allowing the student generator to support multiple objectives in a single model and provide simplicity for deployment. This is why I am giving a positive review of this work.

The major improvement is verified through complementary experiments. For Kwai26, MOGR (SFT + OCD) has higher Recall@512 than MD-CBS for each of the three objectives, and the smallest improvement was approximately 4.3%. Removing OCD decreases the performance (Table 1). The matched-exploration online experiment demonstrates a positive impact on all three objectives compared to the frozen OCD, with a minimum relative improvement of 0.218% (Table 7). The codebook experiment evaluated the differences between content-only quantization, concatenated collaborative representation and OBRQ to help differentiate between the benefits derived from the inclusion of the new representations and those related to energy gating. Additionally, the authors clearly identified the key approximations in their Limitations section.

Critical weaknesses

One major thing missing from this paper is a determination of the independent contribution of position-level objective allocation. Rather than learning a single objective for all positions from both user and list (which is the main contribution of this paper), the online results for the proxy (in Figures 4 and 6) report only the overall task-token share. Therefore, it is not clear whether the gain due to improved overall mix of objectives between two different settings (as shown in the online results) and the gain due to better allocation of objectives to positions (and their corresponding contexts) are indeed two separate things. It may be possible to add a per-position analysis to the current log files to determine this.

The extent to which the behavior of LGPO transfers to different settings has not been thoroughly investigated to date. In addition, a number of the results presented online lack supporting statistics. The single-position exploration process, within-group advantages over consecutive requests, and localized probability ratio adjustments of LGPO are all left as is. The results presented online investigated the effect of changing the window size (Figure 5), changing the type of reward used (Table 7), and the effect of increasing the value of λsid (Table 7). Note, however, that the number of positions that the user explores of a list (i.e. learning what the objective of every position is from both the user and the previous list items) has not been investigated. The online results presented above show small improvements in performance. Importantly, the standard errors and statistics for the results in Table 7 are not provided. However, the p-value for the gain of 0.218% reported in the abstract for the matched-exploration experiment (which used approximately 120,000 users per arm, see Appendix A for further details of the matched-exploration experiment) is less than 0.05. Note that the public proxy used for the online experiment had feedback of a different type, groups were constructed differently, and the lists used were of different lengths (N=8 as opposed to N=20).

Questions for authors

Additional evidence that position-level objective allocation (of user and list previous items) has an independent effect vs. just changing the overall distribution of objectives for the user, is LGPO still better than a simple quota-based approach using the same distribution of objectives that LGPO learned to use (Read / Poi / Int = 73.35 / 22.26 / 4.39 online)?

Online statistics and exploration: The p-values and supporting statistics for user level results are provided in Table 2. However, Table 7 does not contain any p-values or even 95% confidence intervals for the comparison among methods. Could the author please provide more information about the tests used for the comparisons in Table 7, and corresponding confidence intervals? Also, please state the sample size for the matched-exploration test case. Is the gain of the author additionally dependent on the number of exploration positions in the list for each of the previous list items? If not possible to compare, please clearly state which of the previous conclusions only hold for the reported serving configuration.

OCD training context (clarification): I assume that Equation 5 does not include prior list content L but Equation 6 does include BOS as a constraint that is applied to each logged slot individually. How does this individual slot-based application of a constraint on BOS (start of document) relate to generator’s conditioning on list prefix when it is serving?

Deployment and Training Costs: Please include #parameters, training budget, serving latency, and throughput when compared to MD-CBS under matching conditions. It would be helpful to separate the additional cost of training a specialist for each of the K + 1 decoders from the savings that can be achieved during serving.

I will raise the score of this very competitive submission if the rebuttal is able to address questions 1 and 2.

Comments on the paper

1 comment
Area_Chair_wv2EOpenReview ↗

Discussion encouraged during this period!

Hi all reviewers,

We’re about halfway through the discussion period. This is a friendly reminder to check the authors' replies. If the authors have answered questions or responded to critiques in the reviews, please leave a comment for the authors!

Thank you so much!

Best, Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.