- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper introduces ProAgent, a training-free multi-agent framework for high-fidelity product advertising video generation. The system consists of context grounding through web/image retrieval, structured shot planning and prompt rewriting, iterative visual critique and targeted recovery, and an experience-learning module that reuses lessons from previous generation trajectories. The paper also introduces PV-Bench, containing 120 real e-commerce products from 12 categories and evaluating generated videos along six product-centric dimensions. Experiments on three video generators show that ProAgent improves over Direct Prompting and Single Rewriting on PV-Bench, general video-quality metrics, semantic/reference consistency metrics, and human evaluation.
Critical strengths
-
Product advertising generation places stricter requirements on product identity, visual details, logos/text, hallucination avoidance, and commercial presentation than generic video generation. Explicitly formulating these requirements is useful and practically relevant.
-
The combination of external product grounding, structured shot planning, prompt rewriting, visual critique, targeted recovery, and experience reuse forms a reasonably complete end-to-end workflow. Its training-free nature and compatibility with black-box generators also make the framework practically deployable.
-
PV-Bench contains 120 products from 12 e-commerce categories, and the main experiments cover three different video generators. The reported improvements are not restricted to a single generator, which provides some evidence of cross-model generality.
-
In addition to PV-Bench, the authors report VBench-Quality, VisionReward, VQAScore, DINO similarity, human evaluation, component ablations, iteration analysis, and backbone ablations. This provides a broader picture than relying on a single automatic metric.
Critical weaknesses
- ProAgent can repeatedly generate and inspect candidates until a stopping criterion or maximum number of attempts is reached, while DP and SR are single-pass baselines. Therefore, the current results cannot cleanly establish that critique/recovery is responsible for the improvement rather than simply having multiple opportunities to sample a better video. A crucial baseline would be DP and SR with the same number of generation attempts, followed by best-of-K selection or an equivalent fixed-budget protocol.
- ProAgent constructs a knowledge base from web search and a relatively large pool of retrieved images, including textual specifications, selling points, visual attributes, and multiple views. In contrast, DP and SR are given only the top-1 retrieved reference image, even though the paper describes this as “equal grounding.” This makes it difficult to isolate the contribution of multi-agent planning and recovery from the benefit of simply providing substantially richer product evidence.
- The six-dimensional checklist is evaluated by Gemini-3.1-Pro, and PV-Score uses manually chosen weights of 0.25/0.25/0.20/0.10/0.10/0.10. Although a human evaluation of generated outputs is provided, I do not see a direct validation of the benchmark judge itself, such as agreement between human annotations and MLLM checklist decisions, per-dimension accuracy, or sensitivity analysis for the PV-Score weights. This is particularly important because PV-Bench is presented as an independent contribution.
- There is no matched-budget repeated-sampling baseline, generic self-refinement baseline, or competitive iterative/multi-agent video-generation baseline. As a result, the experiments establish that ProAgent outperforms two relatively simple pipelines, but do not convincingly demonstrate that its specific multi-agent structure is necessary or superior to simpler test-time scaling strategies.
- The paper states that experience distilled from previous trajectories is stored and reused in later generations, such that even the initial prompt can benefit from prior experience. However, it is unclear whether this experience pool is reset between benchmark examples, generators, or categories, and whether benchmark products can influence later benchmark products. Because removing Experience Learning measurably reduces performance, this detail could materially affect the reported results and their reproducibility.
Questions for authors
What is the exact maximum and average number of video-generation calls used by ProAgent in the main results, and can the authors provide results against DP/SR with exactly the same generation budget (e.g., best-of-K)? Can the authors also evaluate a baseline that receives the same full retrieved knowledge base—including multiple images and web-derived product information—as ProAgent, so that the contribution of planning/critique/recovery can be isolated? How was the Gemini-3.1-Pro judge validated against human annotations for the six PV-Bench dimensions, and how sensitive are the conclusions to the chosen PV-Score weights? Finally, is the Experience Learning pool reset across products, categories, and generators, or can experience accumulated from earlier benchmark examples influence later ones?