- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
COMIC is a fully automated multi-agent system that turns character and background references into 1–2 minute sketch comedy videos. It aligns LLM writing critics with YouTube engagement data, evolves scripts through island-based pairwise tournaments, and renders videos with script-conditioned critics and shot- and video-level tournaments.
Critical strengths
-
An interesting and very open-ended task. Generating genuinely funny long-form video is hard, and the system covers the whole production pipeline from concept and script to storyboard, audio-video synthesis and continuity.
-
Grounding critics in real audience data. Selecting LLM critics by how well they predict real viewer engagement, rather than hand-writing judge prompts, is a reasonable idea.
Critical weaknesses
-
The aligned critics are barely above chance. On held-out data, the selected critics reach only 0.578 on Top vs. Middle, where chance is 0.5, and 0.716 on Top vs. Bottom (Table 1). So they can hardly tell a hit sketch from an average one, yet the whole script evolution relies on them as judges. The selection is also noisy, with only about 10 scripts per tier per channel in each split.
-
The effect of alignment is never shown end to end. The engagement-aligned critic is the key contribution, but there is no ablation that swaps aligned critics for unaligned ones (e.g., zero-shot or random personas) in the full pipeline. The "No Critics" study (Fig. 9) removes critics entirely, which is a different question.
-
The automated evaluation is close to circular. The benchmark video critics use the same model (Gemini 3 Flash), the same meta-critic procedure and the same engagement data as the rendering critics that COMIC optimizes against. Being "independently sampled" does not remove shared biases.
-
The "diversity" metrics do not measure diversity. Both metrics are variances of win probabilities (lines 348–349), and Intra-Diversity is essentially disagreement among evaluators on the same video. No semantic or stylistic diversity of scripts is measured, even though diverse comedic styles are a main selling point of the island design.
-
The human study is too small. Only 22 raters and 4 videos per method, which is not enough to support the conclusions.
Questions for authors
-
What happens if the aligned critics are replaced with unaligned ones (e.g., zero-shot personas) in the full pipeline?
-
Can you evaluate with judges from a different model family, or with a larger human study?
-
Can you add Seedance 2.5 as a frontier baseline?