- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates whether visual failures in text-to-video diffusion models can be predicted before generation is complete. The authors find that intermediate video-to-text attention patterns contain early signals of whether individual prompt constraints will be satisfied, with successful constraints generally exhibiting more concentrated spatial attention. Based on this insight, they introduce VideoProbe, a lightweight spatiotemporal model that predicts token-level failures from intermediate attention and denoising timesteps, alongside AttnPulse-10K, a benchmark of 10,000 generation trajectories. Experiments across Wan2.2, Wan2.1, and HunyuanVideo-1.5 demonstrate consistent early predictability, achieving 0.340 AUPRC on Wan2.2 and maintaining performance at unseen denoising steps, including the very first step.
Critical strengths
- This paper propose a novel setting: to identify whether a generation would fail during the diffusion process;
- The whole story is complete: from proposing a new problem, prior-analysis, to formulate a method to solve this, and experiments to prove the method can work;
Critical weaknesses
- The biggest weakness is the paper only identify the failure, but fail to provide a method to rectify this;
Questions for authors
- Can authors clarify why identify the failure during the diffusion is improtant? (to stop early, have another try?)
- The methods need to re-train for different models? Otherwise, switching different diffusion models, need to retrain this prob-network would not be efficient.