acceptodds
Withdrawn from ICLR 2027

AdSpark: A Large-Scale Dataset and Benchmark for Product-Centric Advertisement Video Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Product-centric advertisement video generation aims to create promotional videos that preserve fine-grained product identity while presenting selling points through coherent multi-shot narratives. However, this emerging task remains underexplored due to the lack of large-scale advertisement-specific datasets and comprehensive evaluation frameworks. To address this gap, we introduce AdSpark, a large-scale dataset and benchmark for product-centric advertisement video generation, based on data from a major e-commerce platform. AdSpark-300K contains approximately 300K reference image–prompt–video triplets, comprising a real-world subset and a synthetic subset. Each sample provides structured advertisement metadata, including product identity annotations, selling-point descriptions, shot-level creative plans, and aligned audio scripts, enabling models to learn product preservation and advertisement-oriented visual storytelling. We further propose AdSpark-Bench, a diagnostic benchmark that evaluates generated advertisements across six dimensions, including visual quality, product fidelity, instruction adherence, temporal coherence, audio alignment, and advertisement effectiveness. Based on AdSpark-Bench, we evaluate representative models, revealing key challenges in product preservation, multi-shot storytelling, and selling-point visualization. Experiments with AdSpark-300K-finetuned models further validate the effectiveness of our dataset. AdSpark provides a unified dataset and benchmark for future research, and we will release the dataset upon acceptance.

Primary area:
datasets and benchmarks
Keywords:
Product-centric Advertisement Generation, Video Generation Benchmark

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_5shBOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces AdSpark-300K and AdSpark-Bench, addressing the under-researched task of product center advertising video generation. The dataset contains a huge number of samples with structured annotations. The benchmark evaluates beyond visual quality through metrics such as product fidelity, shot structure and execution, temporal coherence, audio alignment, and advertising effectiveness. The authors compare numerous open-source and closed-source models, demonstrating that fine-tuning LTX-2 on this dataset significantly enhances cross-shot consistency, shot fidelity, and narrative coherence. As a dataset/benchmark paper, this work exhibits notable practical value and potential community impact.

Critical strengths

The proposed task is pratically meaningful and important, while the proposed dataset is large-scale with complete annotation and cleaning process, which sounds like high quality. The hierarchical metrics and detailed experiment can provide important reference for this area.

Critical weaknesses
  1. the data generation and annotation process are high dependent on VLM, it will be more acceptable with a sampling manual inspection.
  2. Data deduplication and leakage analysis are insufficient. Table 3 combines indicators with different scopes and directions, lacking definitions for confidence intervals and significance tests.
Questions for authors
  1. How to ensure the generation and annotation quality of the automatic process?
  2. Is there any data deduplication analysis or significance analysis?
Reviewer_YbydOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces a product conditioned advertisement video generation dataset with 300k data points (video, prompt, and product image) including about 100k real videos annotated by models and validated by human. In their synthetic data generation pipeline they utilize commercial model (GPT-5.5) to write an ad plan which is next reviewed by another commercial model (Gemini) and then the videos are generated by video generation models. They further introduce an evaluation framework for generated videos on different aspects such as effectiveness, temporal coherence, and other video qualities utilizing LLMs as a judge. They finally fine-tune a video generation model to show the impact of their dataset.

Critical strengths
  • The task is under explored. The use of product image as condition for video generation is very interesting and under-explored.
  • They have comprehensive ad video generation benchmark on 18 models which can be helpful in future research.
Critical weaknesses
  • Part of advertisement video impact is the result of audio; however, in the current evaluation framework GPT5.5 only sees the frame which can cause errors in the ad effectiveness evaluation.
  • Human validation of the metrics:
    • 20 annotators evaluate and correct 300k advertisement videos and no human agreement is reported. Given the subjectiveness of advertisement impact, this can cause bias in the data.
    • Annotators are just text-to-video generation researchers and advertisement researchers. This is not necessarily evaluating the audience behavior making the result validation challenging.
    • The human annotators only validate videos based on GPT-5.5 generated plans. Human generated plans might be more implicit and complex preventing the current human validation results from generalizing to different inputs.
  • Missing citations and overstated claim: - The paper’s claim on lack of advertisement video text pairs are not completely accurate. For example, PittAd dataset [1] (which is missing citation in the paper) has 3477 ad videos along with message annotations.

[1] Hussain, Zaeem, et al. "Automatic understanding of image and video advertisements." 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017.

Questions for authors
Reviewer_TD5BOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces AdSpark, a large-scale dataset and benchmark for product-centric advertisement video generation. AdSpark-300K contains approximately 300k advertisement videos, together with structured annotations covering product identity, selling points, creative plans, shot designs, and audio scripts. The paper also proposes AdSpark-Bench, which evaluates generated advertisements across six dimensions: visual quality, product fidelity, instruction adherence, temporal coherence, audio alignment, and advertisement effectiveness. Experiments involving video generation models and fine-tuning on AdSpark-300K provide evidence of the dataset’s utility for advertisement-oriented video generation.

Critical strengths
  1. Large scale and broad product coverage. The dataset combines 100k real-world advertisements with 200k synthetic advertisements and covers 40 major product categories and more than 3k subcategories. This scale and diversity make it a potentially useful resource for training.
  2. Rich structured annotations and a multi-stage quality-control pipeline. AdSpark provides structured information about product identity, selling points, creative plans, shot-level design, and audio scripts. And the construction pipeline also incorporates automatic filtering, annotation review, and human inspection, demonstrating considerable attention to data quality.
  3. Empirical evidence of training utility. The fine-tuning experiments demonstrate improvements over the base model across several advertisement-relevant metrics. Moreover, the ablation experiments provide initial evidence that the two data sources offer different benefits. These results support the practical value of the dataset.
Critical weaknesses
  1. Multi-shot metrics do not sufficiently disentangle different sources of failure. The implications of several multi-shot-related metrics are not entirely clear. Low scores may stem from drastically different failure modes: the model failing to generate the requested number of shots, incorrect timing of shot transitions, poor generation quality of individual shots, or even when individual shots are generated as required, product inconsistencies and visual artifacts across shots. These failures reflect different capability limitations and require distinct measures for improvement.
  2. The baseline coverage could be updated or its evaluation cutoff clarified. The paper evaluates a considerable number of models, but incorporating more recently released models equipped with relevant reference-condition controls and multi-shot generation capabilities, such as Seedance 2.5 and MiniMax H3, would make the conclusions regarding the limitations of existing systems more compelling.
  3. The credibility of subjective VLM-based evaluation. Several evaluation dimensions, particularly advertisement attractiveness, creative quality, and narrative coherence, involve inherently subjective judgments. Their scores can be influenced by factors such as the evaluation model's intrinsic biases, prompt formulation, and assumptions about the target audience. Consequently, these metrics cannot reliably reflect actual target consumer preferences or real-world advertising effectiveness.
  4. The ablations do not fully isolate the contribution of structured annotations. Table 6 provides valuable comparisons, including those between real and synthetic data at a 100K data scale, as well as comparisons across synthetic data subsets of varying scales. However, these experiments do not fully elucidate why fine-tuning on AdSpark leads to performance improvements. Specifically, the performance gains could arise from multiple factors: adaptation to ad-specific content, richer structured annotations, complementary supervision provided by real and synthetic data, or the sheer increase in data diversity and scale. The comparison between Synthetic-200K and AdSpark-300K simultaneously alters both the scale and the composition of the datasets, making it difficult to definitively attribute the additional performance improvements to the mixing of real and synthetic data.
Questions for authors
  1. The synthetic data pipeline utilizes multiple generative models, but what about the relative contribution of each model to the final dataset, as well as the cost required to yield the final filtered dataset?
  2. Could the authors separately demonstrate the multi-shot completion rate. This would help distinguish failures to follow the planned shot structure from poor product consistency or transition quality.
  3. Under the same condition, what are the independent gains from using structured annotations and mixing real and synthetic data?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.