- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Retrieval-to-Generation Mapping (R2G-Map), a framework integrating generative retrieval and personalized visual generation for fashion recommendation. It extracts continuous hidden states from frozen recommenders to preserve fine-grained preferences. A Factorized Feature Predictor (FFP) maps these into content, description, and collaborative features, while a Dual-Branch Visual Generator (DVG) uses joint attention for masked visual-token generation. This supports both catalog ranking and personalized image synthesis, demonstrating substantial improvements over existing baselines in ranking and generation metrics.
Critical strengths
- Meaningful integration: Effectively bridges collaborative preference modeling and visual synthesis by using continuous recommender hidden states rather than discrete IDs to preserve fine-grained information.
- Well-structured framework: Combines factorized feature prediction and dual-branch visual-token generation coherently. Keeping retrieval backbones frozen allows integration with various recommenders without expensive retraining.
- Strong experiments and analyses: Consistently improves ranking across two recommendation tasks, two backbones, and different ranking protocols. Informative ablations and additional analyses on preference representations and generation efficiency provide empirical support for the design.
Critical weaknesses
- Limited personalization and generalization: Experiments use POG-derived dataset lacking explicit user histories, demonstrating context-conditioned outfit completion rather than genuine user-level personalization. The sequential setting's relevance to real temporal behavior is unclear, and single-dataset evaluation limits generalizability.
- Unfair baseline comparisons: User-preference conditioning is removed from visual baselines (e.g., DiFashion, FashionDPO) due to missing user histories, while R2G-Map leverages learned retrieval representations. This discrepancy makes it difficult to attribute improvements solely to the proposed architecture.
- Insufficient visual generation validation: Lacks human evaluation and explicit category/personalization metrics. IS is calculated using a fixed uniform category distribution, potentially hiding mode collapse. Generated items sometimes mismatch ground-truth categories, and limited qualitative/novelty evaluations weaken claims of generating useful new items.
- Incomplete architectural isolation and efficiency: Ablations do not independently prove the necessity of dual-branch joint attention or auxiliary feature predictions. Direct ranking with predicted features already improves retrieval, necessitating stronger capacity-matched reranking baselines. Efficiency comparisons focus only on generation backbones, leaving end-to-end recommendation latency and practical scalability insufficiently demonstrated.
Questions for authors
- Personalization & datasets: How does R2G-Map capture genuine user preferences without explicit user histories? How are sequences ordered, and how is train/test data leakage prevented within the same original outfits?
- Baseline fairness: Can you compare R2G-Map against visual baselines equipped with comparable context/preference representations? How much improvement stems from stronger retrieval conditioning versus the mapping architecture?
- Visual evaluation: Why is IS calculated against a fixed uniform category distribution instead of the marginal one? Can you provide standard metrics, category-consistency measurements, human evaluations, and verification of item novelty versus catalog reproduction?
- Architecture: Can you provide ablations separately removing the recommendation latent branch, joint attention, and auxiliary feature supervision? Would a parameter-matched reranker on predicted features or pretrained embeddings achieve comparable improvements?
- Ranking & scalability: At which denoising step are ranking probabilities obtained, and how sensitive are they to the stochastic trajectory? What are the end-to-end inference latency and computational costs for full-catalog ranking?
2 comments
Q5. Ranking and scalability.
Thank you for requesting these details. R2G-Map obtains ranking probabilities from a first prediction step. Each catalog item is scored by the mean log probability of its 128 precomputed visual tokens. This first timestep scoring enables efficient catalog recommendation without requiring the full 16-step refinement process, which is performed when image generation is needed.
Regarding stochastic sensitivity, ranking probabilities are independent of subsequent visual-token sampling trajectories when the recommender hidden states and conditioning are fixed. In contrast, final-step scoring depends on previously sampled tokens, which may favor a particular visual mode and reduce the ranking accuracy of other plausible candidates. Thus, stochastic visual-token sampling affects image generation but does not introduce trajectory-dependent variation into our catalog ranking.
Scalability. We evaluate on the complete POG bundle recommendation test set, comprising 2,521 bundles and 5,042 target items, using a single NVIDIA A40. Each bundle contains two observed items and two prediction positions, and both DDBC and R2G-Map use batches of 32 bundles. R2G-Map uses the 9k checkpoint and CFG = 6. Full-catalog ranking uses a single fully masked DVG prediction to score all 31,217 items, while image generation uses 16 refinement steps. We measure synchronized wall-clock times for each stage over the entire test set and report amortized time per bundle.
| Stage | Total test-set time (s) | Amortized time per bundle (ms) |
|---|---|---|
| DDBC prediction | 91.090 | 36.132 |
| R2G-Map full-catalog ranking | 20.452 | 8.113 |
| R2G-Map image generation | 353.918 | 140.388 |
Runtime breakdown. The full-catalog scoring introduce limited additional runtime, requiring 8.113 ms per bundle. Image generation accounts for most of the additional computation, requiring 140.388 ms per bundle. Nevertheless, our generation pipeline remains substantially more efficient than the evaluated SD3-based baseline. As shown in Table 4, DVG achieves 17.348 images/s, compared with 2.349 images/s for the SD3-based framework, representing a 7.39× improvement in generation throughput.
Thank you for your thoughtful questions about the scope, evaluation, and practical performance of R2G-Map.
Response overview. Below, we clarify the preference signals and dataset construction, discuss the baseline comparison and visual evaluation, and present additional component ablations. We also explain the ranking procedure and its dependence on sampling trajectories, and report full-catalog inference costs and generation throughput.
Q1. Personalization and datasets.
Thank you for asking us to clarify the personalization scope and dataset construction.
Preference signals in bundle completion. R2G-Map addresses context-conditioned bundle recommendation. It does not use individual user interaction histories to construct personalized conditions or claim to recover individual users’ preferences. Our objective is to bridge generative retrieval and content generation by mapping frozen recommendation representations to visual-token prediction and image synthesis. We have revised the relevant descriptions to clarify this scope and remove claims of user-specific personalized content generation.
Scope of the sequential setting. Our sequential setting focuses on outfit-derived sequence continuation rather than user-specific personalization. Our sequential records serialize POG outfits containing four or five items. We use the first three items as context, and retain the remaining one or two as targets. We verify train–test separation using the original source bundle IDs and confirm that no bundle ID appears in both splits.
Q2. Baseline fairness.
Thank you for raising this question. In our experiments, both R2G-Map and the visual baselines operate without explicit preference signals derived from individual users’ interaction histories. R2G-Map bridges generative retrieval and content generation by mapping frozen recommendation states inferred from the observed partial bundle into visual representations that support both catalog retrieval and image synthesis. This setup is appropriate for evaluating context-conditioned bundle recommendation and content generation, and we have revised the manuscript’s descriptions of personalization to accurately reflect this setting.
Table 4 compares our R2G-Map with the SD-v3 visual-generation baseline under the reported bundle recommendation setting. The DVG module outperforms SD-v3 in LPIPS (0.601 vs. 0.614) and compatibility (6.050 vs. 5.910), while achieving 7.39× higher generation throughput (17.348 vs. 2.349 images/s), demonstrating the effectiveness and generation efficiency of our implementation.
Q3. Visual evaluation.
Thank you for asking us to clarify the visual evaluation protocol. Our IS calculation follows the modified evaluation protocol of DiFashion [1], adopting the same formulation for consistency with prior work.
In the revised manuscript, we will supplement this score with additional visual metrics to assess category consistency and item novelty relative to the existing catalog.
Q4. Architecture.
Thank you for asking us to clarify the contribution of the architectural components and the role of additional retrieval controls.
Component ablations. We provide the following component ablations for DDBC-based R2G-Map on POG bundle recommendation under Negative Sampling Ranking:
Removing each component reduces all three ranking metrics. F1 decreases by 0.023, 0.027, and 0.029, respectively. These results support the usefulness of the direct recommendation latent branch, joint attention, and continued feature alignment within this setting. We will include these results and clarify the ablation definitions in the revised manuscript.
Comparison with discriminative rerankers. R2G-Map already uses explicit ranking supervision. During Stage 1, the Factorized Feature Predictor (FFP) uses a feature-level ranking loss to encourage the target item to score above competing candidates (Sec. 3.3.1, Eqs. 4–5). This loss remains part of the auxiliary feature objective during Stage 2 joint training of the FFP and visual generator (Sec. 3.3.2, Eq. 11). Table 3 also reports a separate ranking-loss ablation. Nevertheless, we recognize that a parameter-matched reranker would provide a useful control for understanding the retrieval gains. In future work, we will add this comparison using the same predicted features or pretrained item embeddings, with matched trainable capacity, supervision, and evaluation protocols.
[1] Xu, Yiyan, et al. "Diffusion models for generative outfit recommendation." Proceedings of the 47th international ACM SIGIR conference on research and development in information retrieval. 2024.