- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper proposes a pipeline that leverage VLMs, depth estimation models, semantic segmentation models, V2A models, acoustic rendering models to turn a silent video in to panoramic video with spatial video.
Critical strengths
- The direction is interesting and promising, and this paper is the first attempt on this problem. The examples in the huggingface link is impressive
- The object-centric approach is intuitive and sensible. It also provide meaningful control.
Critical weaknesses
- The proposed approach is a pipeline, which degrades the novelty and excitement
- Due to the nature of the approach being a pipeline, the comparison with other approach is not fair, for example, MMAudio is a component of the proposed pipeline, and therefore a fair comparison would be an ablation study that gradually strip down other added components and see which techniques proposed by this paper bring benefits
- the evaluation is only done on a narrower scope. ImserseSet contains only a smaller amount of cliips and explicitly excludes sever camera motion, jitter, etc. Not sure how this approach would work in in-the-wild video. Object centric approach is also posit a challenge as in the wild video might have too many moving piece for this approach to work well
Questions for authors
In each stage of the pipeline, a model is selected, for example, gemini for VLM, SAM2 for segmentation, AcoustiX for acoustic effect simulation, have you tried alternative models for different stages and how did they perform?