- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
NaviGen turns a user's interaction history into a textual AIGC instruction, which a frozen text-to-image/video model renders. Training has three stages: cold-start initialization, full finetuning, and GRPO. Inference is a single policy pass. Evaluation covers three domains on image and video with four automatic metrics, ablations, human evaluation, and a cost study.
Critical strengths
As a researcher in AIGC, I don't think this paper has any strengths in visual generation. I am not recommendation system experts, so I would like to see the reviewers' comments regarding the recommendation system for further evaluation.
Critical weaknesses
-
Generation is never shown to be better than retrieval. Nothing compares generating a product image against simply retrieving the product's real image — and generation is slower and typically lower-quality than real images, so the burden of proof is on the paper and is not met.
-
The interface to the visual generator is text-only. It should be multimodal (text prompt + visual reference image/video).
-
Relevance leaks the target. The anchor is the mean of history and target caption embeddings, so "the instruction matches the next item" and "the instruction matches this user's taste" score identically — and the model is explicitly trained to predict the target TID. There's no target-free variant to separate the two.
-
"Evolutionary search" oversells what it is. R=1, population of 3: three founders, one crossover/mutation round, keep the best. There's no comparison to cheaper ways of manufacturing the same supervision (best-of-N, self-refine, DPO over candidate pairs), so I can't tell whether evolution specifically is doing anything.
-
The two stages push in opposite directions on the same thing. The search prompt penalizes any candidate that "explicitly mentions, copies, or quotes the target item text or tokens", while the RL reward contains R_ins↔t̃ requiring the instruction to anchor to the target TID. Depending on how "anchor" is computed, this either contradicts SFT outright or is badly underspecified.
-
Split is by instance, not by user or time. An 8:1:1 split of (history, target) pairs is described; if a user contributes multiple pairs, later pairs can land in train while earlier ones land in test. The paper doesn't say how many instances each user contributes.
Questions for authors
Q1. Please explain the advantage of attaching a visual generator over the existing retrieval-based approach.
Q2. Please separately state what your contribution and your difference is, with respect to (i) recommender systems, (ii) preference learning, and (iii) multimodal visual generation.
3 comments
I thank the authors for the detailed reply.
W6 is resolved. One instance per user with disjoint user sets rules out the leakage I raised, and I withdraw that point.
The three central concerns remain.
-
W1 (generation vs. retrieval). The reply offers a qualitative argument for why retrieval may have nothing to return, but runs no retrieval baseline and reports no count of how often a target lacks a usable asset. The owner scores offered as validation are absolute ratings with no retrieval comparison, and the video means (2.487 / 2.386 / 2.154) all fall below the scale midpoint, which the reply does not discuss. The burden of proof I raised is therefore still unmet.
-
W2 (text-only interface). The reply states that a reference frame is "often unavailable", while W1 states that a catalog packshot exists. It defends the 0.51s policy pass — a figure that excludes the generator dominating end-to-end latency — and concedes that a reference image is "a natural extension". This reads as accepting the limitation rather than answering it.
-
W3 (Relevance). My concern was that the metric mixes the target caption into its anchor, not that the target is an input at test time; no target-free Relevance variant is given. The reported Jaccard of 0.023 (59% of cases sharing no term) shows history and target are semantically distant, which supports rather than refutes the reading that the model predicts a next item instead of translating a preference.
On Q2. The reply confirms that the contribution is not in visual generation (the image/video model is not trained, "the contribution stops at the instruction") and not in recommender architecture (next-CID is "only a diagnostic"). If so, the work has to be judged on whether producing an instruction beats retrieving an asset, and that comparison is still missing. The rebuttal also introduces "plain-text CoRRe" without definition, numbers, or a table; W5 attributes triangular alignment to the judge whereas Appendix A.3/A.5 restricts the judge to the instruction-side quality term; and describing the Relevance drop under w/o GRPO as "holding the instruction on the user" conflates target alignment with user alignment — the same conflation W3 concerns. Since the authors themselves frame the recommendation side as a diagnostic, I no longer consider a recommender-system expert's opinion necessary for evaluating the core claim.
I therefore maintain my rating of weak rejection, and my Soundness score.
Thank you for the follow-up and for confirming that the split concern is resolved. We would like to clarify the remaining points more directly.
Generation and retrieval. The practical distinction is between finding an existing asset and creating a new visual concept. A product packshot may exist without depicting the scene or composition a user would enjoy. NaviGen learns to express such interests as executable instructions. Our experiments test this capability against other personalization methods with the generator fixed; they do not establish superiority over retrieval. For example, Games image Novelty and Aesthetic improve from PROSE's 0.5796/0.7804 to 0.6479/0.8423, with complementary support from the 46-participant comparison. We agree that the owner video means of 2.487/2.386/2.154 are below the scale midpoint. These indicate that the current video experience needs improvement. We will discuss them explicitly and distinguish this preliminary evidence from the stronger image results.
Text-only interface. Our setting studies personalization from behavioral and textual evidence without requiring visual references. Reference images remain valuable for preserving product identity and appearance, and our experiments do not establish that they are unnecessary. We will state this input assumption clearly. The 0.51-second figure measures instruction-generation overhead only, excluding rendering, and does not justify ruling out visual conditioning.
Relevance. You are right that the concern is about the metric's anchor. Mixing history and target embeddings does not separate historical preference from next-item agreement. We will describe the metric as target-conditioned Relevance and interpret the GRPO ablation accordingly. The lexical Jaccard statistic does not resolve this issue. Target-blind Engagement and owner ratings provide complementary human evidence, but a separate target-free automatic evaluation has not been reported.
Reward computation and contribution. We confirm that the LLM judge scores both instruction quality and all three triangular alignment terms. We will correct the inconsistent appendix wording. Alignment measures semantic agreement, whereas the search penalty discourages copying the target label; both aim to preserve the intended concept. The reward is a weighted sum, however, so it encourages agreement rather than enforcing a hard condition, and the shared judge leaves room for correlated biases.
The methodological contribution is learning the behavior-to-instruction mapping through synthesized supervision, distilled reasoning, and joint reward optimization. Keeping the renderer fixed lets us evaluate this learned control without changing the visual backbone. We will also make the evidence boundaries clear: the evolutionary-round study measures offline teacher scores, not superiority over compute-matched best-of-N or self-refinement, and the reasoning ablation does not isolate evolutionary operators. The preliminary CoRRe comparison uses a different split assignment and will not be used to substantiate the main claim.
Thank you for helping us sharpen both the contribution and the interpretation of the results.
Dear Reviewer Ga9T,
Thank you very much for your time and for the thoughtful feedback. We have carefully addressed all comments in our response and revised our explanations accordingly.
Should there be any further questions or concerns, we would be very happy to provide additional clarification or address them during the discussion period.
Thank you again for your consideration.
Best regards,
The Authors