- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes ALIVE, a comprehensive framework for hand-interactive video editing. The authors first construct an interaction-heavy dataset that comes from 3D rendering or is curated with off-the-shelf segmentation and removal models. The dataset contains 35k video editing pairs and is then used to train a video diffusion model for editing. The video diffusion model specifically uses a VLM to convert a general prompt into an interaction prompt that is temporally grounded in the video diffusion model's cross-attention to achieve interactivity. The authors also proposed a benchmark and achieved competitive performance on the benchmark, surpassing all open-sourced baselines.
Critical strengths
- The paper proposed an interaction-heavy dataset that is elegantly curated, and I believe it would be beneficial for the community.
- The ALIVE video editing model has very good quantitative performance on the benchmark and showcases impressive performance in the supplementary materials.
- I enjoy reading the paper because it is easy to follow, well-structured, and the figure is quite intuitive
Critical weaknesses
- To my understanding, the main value of this paper is the data curation pipeline and the dataset. However, the authors have not promised to release the dataset, and the data curation pipeline is described vaguely. For example, SpacetimePilot assets are not accessible to the public, and we do not know which scenes are chosen. The lack of dataset access and the ambiguity in the data-curation pipeline description limit this work's value as an academic paper, though it is an interesting application.
- The paper lacks an ablation comparison to previous datasets. For example, EgoPlay and EgoEdit also have interaction-heavy datasets. The uniqueness and advantage of the ALIVE datasets need to be proven.
Questions for authors
- Is the dataset promised to be released? If not, can the community reproduce a similar dataset with accessible assets? Please provide more detailed instructions
- You need to train the same model with previous similar datasets (EgoEdit, EgoPlay) to prove its uniqueness and advantage.
I am willing to raise my score if the questions are addressed.
3 comments
Thank you for your rebuttal
The authors have addressed my concerns; therefore, I decided to raise the score. I encourage the AC to check that the authors actually will do what they promised. Otherwise, this is still a clear rejection if they do not release any data/data processing pipeline
Thank you!
Thank you for your suggestions and for helping us strengthen the paper! We’re glad we addressed your concerns and agree that sharing these resources will benefit the community. As promised, we will share our data processing resources and release the data wherever source permissions allow
Response to Reviewer aUa8 (Part 1 / 1)
We sincerely thank Reviewer aUa8 for the encouraging feedback and for recognizing the value of the dataset.
W1 & Q1: Is the dataset promised to be released? If not, can the community reproduce a similar dataset with accessible assets? Please provide more detailed instructions.
A1: Thank you for this question. We will release as much of the data as we can.
Release. We will do our best to release the editing pairs with their P1–P4 annotations, after checking the license of each source. For example, for the 3D-rendered pairs we are asking the asset owners for permission to redistribute.
Reproduction. For parts we cannot redistribute, we will release the processing pipeline and scripts. Appendix B now details clip selection, pair construction, models, annotation, and filtering, so the community can rebuild similar data.
W2 & Q2: You need to train the same model with previous similar datasets (EgoEdit, EgoPlay) to prove its uniqueness and advantage.
A2: Thank you for this suggestion. We checked the released EgoEdit data: its pairs are mostly replacement edits, with no insertion or removal pairs we can train on. EgoPlay has not released its training set. We therefore compare with ReCo, ReCo combined with EgoEdit, and the Señorita data. The same LTX-2.5 editor is trained on each for 5k steps (main Table 3), using released instructions for external datasets (Appendix A.1).
R3-Table1: Same model, different training data, on ALIVE-interaction.
ALIVE data leads by 7 to 21 points in Overall and by 13 to 32 points in Motion, which measures the object's response to source actions. Adding EgoEdit to ReCo helps but remains well below ALIVE.
Thank you again for your constructive suggestions. We hope this addresses your concerns; if so, we would be grateful if you would consider updating your assessment.