acceptodds
Under review as a conference paper at ICLR 2027

RIGC: A Recommendation-Informed Foundation Model for Personalized Multimodal Content Generation

Abstract

In real-world recommender systems, generating personalized multimodal content remains largely unexplored, and the readily available recipe is a cascaded pipeline: a recommendation model compresses user behavior into an intermediate representation, on which a separate frozen generator then conditions. Such a bottleneck interface is fundamentally lossy, as the user is invisible to the generator once compressed, and personalization reduces to rendering a static description. Closing this gap calls for an end-to-end structure where recommendation and content generation share one model, one representation. We therefore present RIGC (Recommendation-Informed Generated Content), a foundation model realizing this design within a single mixture-of-transformers backbone, where user information persists in the shared attention context and flows losslessly into generation. Such unification is non-trivial, as the backbone must reconcile heterogeneous objectives spanning item retrieval, text generation, and image synthesis. To this end, we construct a large-scale industrial multimodal dataset pairing user behavior with item content, and design a multi-stage alignment curriculum that progressively instills recommendation capability into a native multimodal generative model. Aligning with real user preferences further poses a cross-modal credit assignment problem, as a single feedback signal entangles all modalities, for which we introduce Nested-GRPO to sample hierarchical rollouts per modality and credit each by its marginal advantage. Experiments on public and industrial benchmarks show that RIGC surpasses dedicated recommenders in retrieval accuracy and consistently outperforms cascaded and unified baselines in generation fidelity, quality, and personalization, with Nested-GRPO yielding further gains in conversion and user specificity. These results indicate that recommendation and generation are mutually reinforcing over a shared representation, positioning unified generative recommendation as a foundation for personalized content delivery.

Primary area:
foundation or frontier models, including LLMs
Keywords:
personalized content generation, generative recommendation, multimodal foundation model

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_JQLFOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper studies personalized generation for recommender systems. To reduce the information loss caused by cascaded pipelines, the paper proposes RIGC, an end-to-end unified framework for recommendation and generation. It first predicts the next item and then produces personalized marketing text and image. RIGC is trained through a five-stage SFT procedure and a newly designed Nested-GRPO method. The authors also construct IndusAds-MM from large-scale industrial advertising logs. Experiments show that RIGC outperforms several recommendation and generation baselines across all reported metrics.

Critical strengths

The paper studies personalized multimodal generation for recommender systems, which is relevant to real-world content-delivery and advertising applications. It also provides a valuable attempt to integrate item recommendation, personalized text generation, and visual content generation into a single end-to-end framework.

Critical weaknesses
  1. Insufficient ablation studies. The five-stage training pipeline and Nested-GRPO contain many intertwined design choices, but the paper provides few controlled ablations. See Question 3 for details.

  2. Unclear experimental setup and baseline comparison. The paper does not clearly specify what information each baseline receives, making it difficult to assess whether the experiments fully support the claimed advantage of the unified framework. See Question 7 for details.

Questions for authors
  1. According to the introduction and the bottom-right example in Figure 1, RIGC can restyle a generated keyframe into different visual styles. It would strengthen the paper if the authors could evaluate RIGC on general image-generation and editing benchmarks. This would help determine whether the RIGC training comes at the cost of its general multimodal capabilities.

  2. The relationship between the multi-level SID representation and the expanded vocabulary is not sufficiently clear. The paper reports that 32,770 rows are appended to the vocabulary, but does not explain how the SID tokens are constructed or mapped to these new entries. It would be helpful if the authors could clarify the complete SID tokenization scheme and report relevant statistics, such as the number of active codes at each level.

  3. The five-stage training pipeline contains many carefully designed components, but most of them lack ablation studies. Such experiments are needed to better identify the contribution of each component. In particular, the authors should investigate: (1) the size of the expanded SID vocabulary; (2) the data-mixture ratios in Stages 2 and 4; (3) the parameter-freezing strategy in each stage; and (4) the relative weights of the autoregressive and flow-matching losses in Stage 5. It would also be useful to examine whether all five stages and their current ordering are necessary.

  4. According to the appendix, Nested-GRPO uses K=M=4K=M=4 and L=8L=8. This results in K×M×L=128K\times M\times L=128 rollouts. Since every rollout includes image generation, the computational cost may be substantial. The paper would be more convincing if the authors reported the configuration details such as rollout parallelism, training time, and GPU usage for both the SFT and Nested-GRPO stages.

  5. During Nested-GRPO training, the reward is provided by a user-feedback simulator trained by the authors. It would be better if the authors could provide additional evidence to rule out reward hacking. The authors should also provide more information about the simulator's training data such as the number and distribution of low-, medium-, and high-interest samples.

  6. The paper uses a very large industrial dataset, but the reported statistics are mainly limited to aggregate counts. It would improve the transparency of the dataset if the authors could show more detailed statistics and visualizations.

  7. One of the paper’s central claims is about end-to-end frameworks versus cascaded systems. Therefore, the evaluation settings for the baselines are crucial to judge this comparison. It would be better if the authors could clarify what information each baseline receives (such as the ground-truth item, the complete user history, or a reference image). A controlled comparison between these two methods using the same experimental setup is necessary. Besides, the relationship between recommendation accuracy and keyframe quality also requires further explanation. As RIGC achieves only 0.1300 Recall@10 on IndusAds-MM, if a single predicted SID is used for subsequent generation, an incorrect recommendation will influence the marketing text and keyframe. An oracle experiment that provides the ground-truth SID would help distinguish recommendation errors from text- and image-generation errors.

Reviewer_1QjoOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes RIGC, a unified foundation model designed for personalized multimodal content generation. Existing recommender systems typically rely on cascaded architectures, compressing user behavior into text prompts before passing them to an independent generator, which causes a loss of personalized features during the generation phase. To overcome this limitation, RIGC unifies item recommendation, marketing text generation, and native visual image generation within a single mixture-of-transformers backbone, allowing recommendation and generation to share the same representation.

Critical strengths
  1. RIGC discards the traditional cascaded pipeline and achieves deep unification of user modeling, item recommendation, and native multimodal generation within a single foundation model. This shared representation design effectively solves the information loss caused by compressing user behavior into static prompts, allowing the generator to directly observe the complete user history.
  2. The authors accurately identify the "cross-modal credit assignment" problem faced when applying standard GRPO to unified multimodal models (e.g., a strong item recommendation might be incorrectly penalized because of a poorly generated accompanying image). The proposed Nested-GRPO algorithm calculates marginal advantages separately for the item (SID), text, and image through hierarchical sampling. Ablation studies confirm that this method significantly improves high-depth conversion rates (DeepConv.) and user-specificity of the content.
  3. The paper evaluates not only on public benchmarks but also constructs a large-scale industrial multimodal dataset (IndusAds-MM) containing over four million advertisement records. Experiments demonstrate that even when integrated with complex generation tasks, RIGC surpasses dedicated recommendation models like SASRec and OneRec in Recall@10 and NDCG@10 metrics.
Critical weaknesses
  1. The reward signal relied upon by Nested-GRPO comes from an offline user-feedback simulator trained on historical logs. Although the simulator achieves reasonable AUC in predicting three interest levels, offline predictors often struggle to perfectly capture the dynamic preferences of users when faced with "entirely new generated content" in a real online environment. The paper lacks online A/B testing data, which limits the evaluation of how this offline reward model ultimately impacts conversion rates in a live commercial setting.
  2. The training dataset, IndusAds-MM, originates from real advertising logs with abundant interaction records. The model's high degree of personalization relies on rich historical sequences (retaining up to 256 historical interactions). It remains insufficiently discussed and evaluated whether this massive generative recommendation system can still stably produce high-quality, hallucination-free text and images for long-tail items with sparse interaction data or for entirely new users (cold start).
Questions for authors

see weakness

Reviewer_fi6uOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies the recommend and personalized advertising image generation setting. In which, given the users' profile and interaction, model try to predict the next shopping item together with the personalized advertising image. To that end, the paper collect data and train the GRPO on this data to verify the idea.

Critical strengths
  • I really like the proposed direction and I think the research insights presented, together with the problem setting (personalized ad generation) is very helpful and practical.
  • I appreciate the dataset, which contains 200k users and 115.6M users's interactions with system.
  • Generally the proposed variant of GRPO makes sense and the results looks promising.
Critical weaknesses

The current experiement setting are not extensive enough. I appreciate authors' efforts, yet the current evaluation is missing comparison with:

  • memory-augmented methods (e.g., mem0, memOS)
  • frontier models (e.g., Gemini, GPT) For example, simple baseline can be: run memory-layer to create some memories/ extract facts about this users. Then in test time, retrieve from memory database to augment the answer of Gemini/ GPT models. These comparisons dont need to be at large scale (e.g., can be for 20-50 users), but can help clarify the proposed unified training provides clear benefits; or just more contextualized generations over stored/ retrieved users' preferences is enough.
Questions for authors

Do you have plan to release the dataset? If not, how do you think the following-up works can build upon this work? Or study this setting?

Comments on the paper

6 comments

Thank you for the responses and additional experiments. These have addressed some of my concerns. I have two remaining questions:

  1. For BAGEL and SenseNova-U1 in Table 2, are text and images generated directly from the user profile and history, without selecting a specific item from the catalog?

  2. How are Recall/NDCG computed for IndusAds-MM in Table 3? If they measure product-level recommendation accuracy, how is a predicted SID mapped back to a unique product, given that one SID may correspond to multiple advertisements?

AuthorsOpenReview ↗

BAGEL and SenseNova-U1 generation with and without catalog retrieval

Thank you for reviewing our additional experiments and for letting us know that our first response addressed some of your concerns. Your insightful question regarding Table 2 prompted us to add catalog retrieval to BAGEL and SenseNova-U1. In all four comparisons, retrieval lowers baseline FID and the judge still prefers RIGC’s outputs. Furthermore, when counting only SID hits that identify a single promoted entity, RIGC remains ahead on both metrics in Table 3.

We try to address both of your remaining questions in detail below, and if these clarifications and results resolve your concerns, we kindly hope you will consider raising your overall rating.

BAGEL and SenseNova-U1 generation with and without catalog retrieval

In the original Table 2 setting (A), BAGEL and SenseNova-U1 infer what to depict from user context without a catalog lookup. Each model uses the interaction history and user profile to write a creation instruction and an image description. Its native image generator then renders the image from that description.

We evaluated explicit catalog retrieval (Setting B) on both full test sets. We use Qwen3-Embedding-0.6B to retrieve the catalog entry closest to the model’s predicted name and category. We fix the retrieved name and category as an output prefix. Given that prefix and the same user context, the model writes the instruction and image description, then renders the description with its native image generator. The Office Products catalog draws on Amazon product metadata. The IndusAds-MM catalog pairs product and other promoted-entity names from interaction histories with their industry labels.

Both settings follow the Table 2 evaluation protocol. Pref. is the baseline’s pairwise win rate minus RIGC’s, so negative values favor RIGC.

DatasetModelSettingBLEU-4 ↑ROUGE-L ↑FID ↓CTS ↑CIS ↑DIS ↑Pref. ↑Attr. ↑
Office ProductsBAGELA0.1012.6148.660.13560.56410.1144−0.170.750
Office ProductsBAGELB0.1813.2747.630.13810.56990.1113−0.150.740
Office ProductsSenseNova-U1A0.4014.3870.980.13330.55140.1015−0.060.780
Office ProductsSenseNova-U1B0.3514.2767.110.13500.55180.1106−0.060.785
IndusAds-MMBAGELA0.337.74120.930.15040.47690.0755−0.100.730
IndusAds-MMBAGELB0.848.45117.310.15820.47540.0778−0.120.715
IndusAds-MMSenseNova-U1A0.537.4974.560.15530.47970.0822−0.190.640
IndusAds-MMSenseNova-U1B1.007.7372.670.17940.49150.0846−0.180.630

Adding catalog retrieval lowers FID by 2.12%–5.45% and raises CTS in all four model–dataset comparisons. CIS and DIS each improve in three of the four comparisons. Even with these retrieval gains, RIGC exceeds both retrieval variants on both datasets in instruction overlap (BLEU-4 and ROUGE-L) and target-image alignment (CTS, CIS, and DIS). On IndusAds-MM, RIGC’s FID of 32.52 is 55.25% lower than the best FID achieved by either baseline with retrieval (72.67, SenseNova-U1). The judge also favors RIGC’s text–image pairs in all four comparisons with retrieval, by 6–18 percentage points in pairwise win rate.

AuthorsOpenReview ↗

Recall/NDCG Computation and SID-to-Entity Mapping

Recall/NDCG Computation and SID-to-Entity Mapping

Table 3 scores exact matches to the target evaluation SID on IndusAds-MM. All three methods use the same SID representation for scoring. We keep the first occurrence of each distinct predicted SID in rank order. A shared SID therefore occupies one position, and scoring requires no choice among the advertisements it represents. With one target per query, a hit at rank $r≤10r\leq 10 $ contributes 1 to Recall@10 and �1��1� to NDCG@10, and a miss contributes zero to both. We average these contributions over all evaluation queries.

When does a hit identify the promoted item? Each photo_id identifies a video creative, and several creatives can promote the same product, shop, or app. As reported in our first response, 12.03% of advertisements in the full catalog share their complete SID with another ad. For each shared evaluation SID represented among the test targets, we checked the logged product, shop, or app name for every creative in its catalog group. In 26.94% of these shared groups, all creatives have the same name for the promoted entity.

We then counted hits only for targets whose evaluation SID maps to a single creative or to a group whose creatives all share the same entity name. We applied this rule to all three methods and kept their original rankings. Every row below uses the full set of evaluation queries as its denominator. The last three rows give the contributions from hits that identify one promoted entity.

MethodTargets contributing to the scoreRecall@10 ↑NDCG@10 ↑
NaviGenAll evaluation targets0.04300.0312
OneRecAll evaluation targets0.10100.0655
RIGCAll evaluation targets0.13000.0972
NaviGenTarget SID identifies one promoted entity0.02300.0163
OneRecTarget SID identifies one promoted entity0.07300.0458
RIGCTarget SID identifies one promoted entity0.10600.0849

RIGC’s hits linked to one promoted entity contribute 0.1060 to Recall@10 and 0.0849 to NDCG@10, compared with 0.0730 and 0.0458 for the strongest baseline on the same targets. Notably, RIGC’s score from these single-entity hits alone already exceeds that baseline’s overall performance (0.1010 and 0.0655). This provides direct evidence that RIGC's recommendation advantage holds at the product/item level when the SID-to-entity mapping is unambiguous.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.