acceptodds
Under review as a conference paper at ICLR 2027

ALIVE: Interaction-Aligned Object Insertion for First-Frame-Guided Video Editing

Abstract

Current video editors can insert objects but often struggle to make them participate in interactions such as being picked up or manipulated. We introduce ALIVE, a framework that makes inserted objects “alive” through coherent interactions with the source video's contents, using an edited first frame and an instruction naming only the added object. We curate 35,800 editing pairs combining 3D-rendered, model-generated, and real-world videos with general editing pairs from ROSE. Each pair differs in the target object's presence while preserving the surrounding action, teaching editors coordinated object behavior and source preservation. We further train a vision-language model (VLM) to predict interaction guidance from the same inputs. We introduce the ALIVE-interaction benchmark to assess interaction fidelity, source preservation, and visual coherence using a unified VLM-based protocol, and evaluate on the general video object insertion benchmark. Without VLM guidance, ALIVE improves Overall over the strongest evaluated baseline by 43.9% and 4.4% on the two benchmarks, respectively. VLM-predicted guidance further improves the ALIVE-interaction score by 0.95 points without additional user inputs.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video editing, Object insertion

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_aUa8OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ALIVE, a comprehensive framework for hand-interactive video editing. The authors first construct an interaction-heavy dataset that comes from 3D rendering or is curated with off-the-shelf segmentation and removal models. The dataset contains 35k video editing pairs and is then used to train a video diffusion model for editing. The video diffusion model specifically uses a VLM to convert a general prompt into an interaction prompt that is temporally grounded in the video diffusion model's cross-attention to achieve interactivity. The authors also proposed a benchmark and achieved competitive performance on the benchmark, surpassing all open-sourced baselines.

Critical strengths
  1. The paper proposed an interaction-heavy dataset that is elegantly curated, and I believe it would be beneficial for the community.
  2. The ALIVE video editing model has very good quantitative performance on the benchmark and showcases impressive performance in the supplementary materials.
  3. I enjoy reading the paper because it is easy to follow, well-structured, and the figure is quite intuitive
Critical weaknesses
  1. To my understanding, the main value of this paper is the data curation pipeline and the dataset. However, the authors have not promised to release the dataset, and the data curation pipeline is described vaguely. For example, SpacetimePilot assets are not accessible to the public, and we do not know which scenes are chosen. The lack of dataset access and the ambiguity in the data-curation pipeline description limit this work's value as an academic paper, though it is an interesting application.
  2. The paper lacks an ablation comparison to previous datasets. For example, EgoPlay and EgoEdit also have interaction-heavy datasets. The uniqueness and advantage of the ALIVE datasets need to be proven.
Questions for authors
  1. Is the dataset promised to be released? If not, can the community reproduce a similar dataset with accessible assets? Please provide more detailed instructions
  2. You need to train the same model with previous similar datasets (EgoEdit, EgoPlay) to prove its uniqueness and advantage.

I am willing to raise my score if the questions are addressed.

3 comments
AuthorsOpenReview ↗

Response to Reviewer aUa8 (Part 1 / 1)

We sincerely thank Reviewer aUa8 for the encouraging feedback and for recognizing the value of the dataset.

W1 & Q1: Is the dataset promised to be released? If not, can the community reproduce a similar dataset with accessible assets? Please provide more detailed instructions.

A1: Thank you for this question. We will release as much of the data as we can.

  1. Release. We will do our best to release the editing pairs with their P1–P4 annotations, after checking the license of each source. For example, for the 3D-rendered pairs we are asking the asset owners for permission to redistribute.

  2. Reproduction. For parts we cannot redistribute, we will release the processing pipeline and scripts. Appendix B now details clip selection, pair construction, models, annotation, and filtering, so the community can rebuild similar data.

W2 & Q2: You need to train the same model with previous similar datasets (EgoEdit, EgoPlay) to prove its uniqueness and advantage.

A2: Thank you for this suggestion. We checked the released EgoEdit data: its pairs are mostly replacement edits, with no insertion or removal pairs we can train on. EgoPlay has not released its training set. We therefore compare with ReCo, ReCo combined with EgoEdit, and the Señorita data. The same LTX-2.5 editor is trained on each for 5k steps (main Table 3), using released instructions for external datasets (Appendix A.1).

R3-Table1: Same model, different training data, on ALIVE-interaction.

Training dataOverallTaskMotion
ALIVE (ours)82.3589.6582.03
ReCo72.2675.7862.50
ReCo + EgoEdit (1:1)75.0578.5269.14
Señorita data61.6966.0250.20

ALIVE data leads by 7 to 21 points in Overall and by 13 to 32 points in Motion, which measures the object's response to source actions. Adding EgoEdit to ReCo helps but remains well below ALIVE.

Thank you again for your constructive suggestions. We hope this addresses your concerns; if so, we would be grateful if you would consider updating your assessment.

Reviewer_aUa8OpenReview ↗

Thank you for your rebuttal

The authors have addressed my concerns; therefore, I decided to raise the score. I encourage the AC to check that the authors actually will do what they promised. Otherwise, this is still a clear rejection if they do not release any data/data processing pipeline

AuthorsOpenReview ↗

Thank you!

Thank you for your suggestions and for helping us strengthen the paper! We’re glad we addressed your concerns and agree that sharing these resources will benefit the community. As promised, we will share our data processing resources and release the data wherever source permissions allow

Reviewer_mEjXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

ALIVE addresses first-frame-guided object insertion with coherent responses to source actions. It combines interaction-focused training data, a diffusion editor with temporal text routing, and a VLM that predicts interaction guidance from the original editing inputs. The paper introduces an interaction benchmark and reports improvements over existing editors.

Critical strengths

1.The task is meaningful, and the method is clearly formulated, including the temporal routing mechanism.

2.The dataset combines complementary sources. Experiments cover data composition, prompt guidance, training budget, backbone choice, and computational cost, with strong reported interaction results.

Critical weaknesses
  1. Limited independent validation of the evaluation. The headline results depend on a single GPT-based judge, while GPT also participates in filtering and annotation, raising a concern about correlated biases. Additional automatic metrics are reported, but no human evaluation or cross-family validation of interaction judgments is provided. The 0.95-point gain from predicted P4 lacks uncertainty estimates or repeated judge evaluations, leaving its robustness unclear.

  2. Unclear trade-off between convenience and user control. The paper does not establish whether users prefer inferred interactions or how much effort automatic guidance saves. Table 3 shows essentially unchanged Overall scores for annotated P2 versus P1, but does not measure user effort or controllability. Although the editor supports P2–P4, no workflow is demonstrated for inspecting, rejecting, or revising predicted guidance and reliably controlling the output.

  3. Unsupported interaction predictions are not quantified. P4 supervision may use the target video, whereas prediction uses only the source, edited first frame, and object instruction. This is valid supervision, but some target-specific details may not be inferable from those inputs. The limitations acknowledge multiple plausible object responses and note that VLM-predicted guidance errors can cause incorrect behavior, but do not quantify the frequency of such errors. Although the judge allows plausible alternatives to the reference, its ability to distinguish such alternatives from unsupported interactions is not independently validated.

  4. Weak methodological novelty relative to prior reasoning-guided editing. Place-it-R1 [1] already combines MLLM scene and interaction reasoning, automatic insertion trajectories, and reasoning-guided diffusion for video object insertion, yet it is neither cited nor compared in the provided manuscript. VOID [2], which is cited, likewise translates VLM-based causal reasoning about interaction-affected regions into mask guidance for video diffusion, albeit for removal. These works are relevant precedents for using multimodal understanding to guide diffusion editing. While ALIVE differs in its first-frame conditioning and temporally localized text guidance, the paper does not sufficiently establish the additional methodological contribution of its supervised interaction predictor and temporal routing relative to these approaches.

  5. The contribution of the proposed mechanisms remains insufficiently isolated. The interaction benchmark includes held-out cases from the same data sources used for training and two explicitly designated OOD subsets (§3.3), but their results are not reported separately, making out-of-domain generalization difficult to assess. Baselines are not reported as adapted to the ALIVE data. Wan outperforms LTX in the P1-only backbone comparison; the efficiency rationale for LTX is reasonable, but the main mixed-prompt setting is not compared. Temporal routing also lacks an isolated ablation: prompt-level comparisons change both text content and organization.

[1] Place-it-R1: Unlocking Environment-aware Reasoning Potential of MLLM for Video Object Insertion. arXiv:2603.06140. [2] VOID: Video Object and Interaction Deletion. ECCV 2026. arXiv:2604.02296.

Questions for authors
  1. Does the predicted-guidance gain persist under human evaluation and another judge family? Please report uncertainty estimates.
  2. Can users revise predicted guidance, and do those revisions reliably control behavior? How does this compare with user-written instructions?
  3. How frequently does predicted guidance introduce details unsupported by the inputs, and how reliably can the judge identify them?
  4. How does ALIVE advance beyond Place-it-R1 and other reasoning-guided editing methods? Please discuss the differences in conditioning and guidance, provide a compatible comparison.
  5. Please separate same-source and OOD results, adapt a baseline to the ALIVE data, compare backbones under the main setting, and ablate temporal routing while keeping P4 descriptions fixed.
4 comments
AuthorsOpenReview ↗

Response to Reviewer mEjX (Part 1 / 2)

We sincerely thank Reviewer mEjX for the thorough and constructive review. Brackets are 95% intervals over cases.

In short: (i) the average gain from predicted P4 (0.95 in the paper) is modest and not significant; (ii) ALIVE's advantage holds with P1 alone, under a second judge, human raters and OOD sources; (iii) accurate P4 gives a consistent gain, which users can reach by revising the predicted P4; (iv) temporal routing is effective and makes P4 editable in time.

W1 & Q1: Does the predicted-guidance gain persist under human evaluation and another judge family? Please report uncertainty estimates.

A1: These checks are now in Appendix C.1–C.2. Claude Opus 5.5 took no part in data filtering or annotation, and human raters compare full videos blind (prefer first / "About the same" / prefer second).

R2-Table1: ALIVE-interaction (128 cases).

EvaluationPredicted P4 − P1ALIVE (P1) − Señorita
GPT-5.6 (paper)+0.95 [−1.17, 3.06]+26.46 [22.57, 30.26]
Claude Opus 5.5 (new)+1.06 [−0.75, 2.89]+29.30 [25.51, 32.97]
Human study20 / 60 / 1944 / 2 / 2

R2-Table2: Predicted P4 − P1 by difficulty (main Table 5). Interaction cases are split by the P1 score of a separately trained editor.

HalfP1GPTOpus
Easier (64)93.9−0.82 [−3.77, 1.72]+0.37 [−2.27, 2.64]
Harder (64)79.5+2.71 [−0.25, 5.86]+1.76 [−0.90, 4.47]
  1. Predicted P4. The second judge gives +1.06, close to the 0.95 in the paper, and raters mostly chose "About the same": the gain is modest and not significant. It comes from the harder half, where P1 leaves room; the easier half already scores 93.9.

  2. Main comparison. With P1 alone, ALIVE leads Señorita by more than 26 points under both judges and in 44 of 48 human ratings, on OOD sources as much as on same-source ones (A5).

  3. Accurate P4 gives consistent gains. Annotated P4 improves on P1 under both judges (+1.98 GPT, +1.38 Opus; mean +1.68 [0.15, 3.25]). A user can reach this level by revising the predicted P4 (A2).

W2 & Q2: Can users revise predicted guidance, and do those revisions reliably control behavior? How does this compare with user-written instructions?

A2: Yes (Appendix C.4). Predicted P4 is plain text with frame intervals: a user can read it, edit a sentence or a boundary, and rerun the editor. For 8 of the lowest-scoring predicted-P4 cases, we compared four sources of P4 with the same editor and seeds. GPT-5.6, acting as a user, revised and wrote P4 from the source video and edited first frame only; a human checked the prompts.

R2-Table3: Correcting predicted P4.

P4 sourceOverallvs. predicted P4
Predicted P473.91—
Revised predicted P480.94+7.03 [−0.16, 13.59]
P4 written from scratch79.53+5.63 [0.94, 10.00]
Annotated P479.38+5.47 [−2.81, 14.84]

Revision raises Overall from 73.91 to 80.94, on par with P4 written from scratch and annotated P4: editing the predicted P4 gives the quality of writing the full description. Each sentence is routed to its own frame interval, so an edit acts on its own frames (A5).

W3 & Q3: How frequently does predicted guidance introduce details unsupported by the inputs, and how reliably can the judge identify them?

A3: Appendix C.5 reports our audit of all 128 predicted P4 descriptions: an auditor from a different model family (Claude Opus 5.5) checked each one against the source video, the edited first frame and P1, and logged every correction.

  1. Frequency. From the audit, we identify 22 inaccurate predicted P4 among the 128: the motion contradicts the source in 14, a phase is missing or extra in 4, and the wrong hand or body part is named in 4.

R2-Table4: Overall gain by accuracy of the predicted P4.

Predicted P4CasesPredicted P4 − P1Annotated P4 − P1Annotated P4 − Predicted P4
Inaccurate22−0.80+4.20+5.00
Others106+1.31+1.52+0.21
  1. Effect. On the other 106 cases, predicted P4 gains +1.31 over P1 and matches annotated P4. It falls behind only on the 22 inaccurate cases, where annotated P4 is 5.00 higher. The average gain is thus limited by a small set of inaccurate predictions rather than by the guidance mechanism, and annotated P4 shows the headroom.

  2. Judge. The judge sees only videos, never the P4 text. Its scores still separate the two groups (+5.00 vs. +0.21 in the last column), in line with the audit by Claude Opus 5.5, which never saw the edited videos.

AuthorsOpenReview ↗

Response to Reviewer mEjX (Part 2 / 2)

W4 & Q4: How does ALIVE advance beyond Place-it-R1 and other reasoning-guided editing methods? Please discuss the differences in conditioning and guidance, provide a compatible comparison.

A4: Thank you for pointing us to Place-it-R1; we now cite and discuss it with VOID in Sec. 2. The VLM plays a different role in ALIVE. In Place-it-R1 the MLLM is the planner that drives generation. In ALIVE, interaction quality comes first from the interaction-aligned training data (+26 over Señorita with P1 alone; the same model trained on ReCo+EgoEdit is 7 points lower, A5), and the VLM part studies which description improves interaction further and how users can supply and control it.

Place-it-R1VOIDALIVE
TaskInsertion: where to place the objectRemovalInsertion: how the object responds to source actions
VLM outputBoxes / trajectory, hidden statesAffected-region masksText per action stage, with frame intervals
User conditionText and a reference object imagePoints on the object, background promptEdited first frame
User control of guidanceNot describedRepaint the maskEdit a sentence or a frame interval (A2)
  1. What is reasoned about. Place-it-R1 plans a location and a trajectory. In ALIVE the first frame already fixes appearance and placement, and the VLM infers how the object responds over time to actions in the source (grasped, cut, worn), which spatial planning does not answer.

  2. How the guidance is learned and used. Our predictor is supervised on stage-wise annotations of real interaction videos and outputs readable text bound to frame intervals. This format gives an upper bound (annotated P4), lets us audit every prediction (A3), and lets users inspect, edit or drop it.

  3. Compatible comparison. The Place-it-R1 project page links no code or weights, and it takes no first-frame condition while ALIVE takes no trajectory, so a like-for-like run is not possible; we compare with five editors under a shared protocol.

W5 & Q5: Please separate same-source and OOD results, adapt a baseline to the ALIVE data, compare backbones under the main setting, and ablate temporal routing while keeping P4 descriptions fixed.

A5: Thank you for these concrete requests. We address the four in turn.

R2-Table5: Requested controls on ALIVE-interaction (Overall).

RequestComparisonResult
Same-source vs. OODALIVE (P1) − Señorita, 71 same-source cases+26.0 [20.8, 31.3]
ALIVE (P1) − Señorita, 57 OOD cases+27.1 [21.3, 32.6]
Data, same editorReCo+EgoEdit → ALIVE data75.05 → 82.35
Backbone, same dataLTX-2.5 → Wan 2.282.35 → 89.82
Routing, same annotated P4Text-only editor → routed editor83.19 → 83.38
Routing, same predicted P4Text-only editor → routed editor80.13 → 82.37
  1. Same-source vs. OOD (Appendix C.3). With P1, ALIVE leads Señorita by the same margin on same-source and OOD cases. The gain of predicted P4 over P1 is also similar: +1.00 on same-source and +0.88 on OOD cases.

  2. Adapting a baseline. The baselines cannot be retrained on pairs: AnyV2V is training-free, I2VEdit fits a LoRA per video, NOVA trains pair-free from keyframes and masks, and Señorita and PropFly release inference code and weights only. We therefore ran the two closest controls: the same editor on other data (75.05 with ReCo+EgoEdit vs. 82.35 with ALIVE data; main Table 3), and another backbone on our data (below).

  3. Backbones (Appendix A.3.2). With the same data, schedule and P1, Wan 2.2, the backbone family of NOVA, reaches 89.82 against 82.35 for LTX-2.5, which we use for efficiency. The data transfers across architectures, and a stronger backbone benefits from it.

  4. Routing with P4 fixed (main Table 6). We trained a routed and a text-only editor with the same data, seed and steps (5k); the text-only editor receives the same P4 sentences as one prompt. With predicted P4, routing improves Overall from 80.13 to 82.37. Routing is also what lets the prompt switch between segments of the generated video, which gives users a time axis to edit (A2).

Thank you again for the detailed review. We hope these results address your concerns; if so, we would be grateful if you would consider updating your assessment.

Thank you for the detailed rebuttal and additional experiments. The following comments focus on the remaining issues. Given the limited discussion time, the authors may respond directly to the following questions.

1. Inter-rater reliability of the human study. The predicted-P4 vs. P1 human result (20 / 60 / 19, "mostly About the same") is now the only non-VLM signal for the paper's central +0.95 claim and anchors the "modest and not significant" framing in the response. However, the rebuttal reports only aggregate preference counts over "147 ratings on 93 video pairs from 13 questionnaires", which implies roughly 1.6 raters per pair on average and gives no view of rater overlap or agreement. Could the authors report (i) how many raters independently evaluated each pair, (ii) the per-pair agreement rate (e.g., fraction of pairs with unanimous decisions), and (iii) a chance-corrected measure such as Cohen's κ or Krippendorff's α on the three-way (prefer first / about the same / prefer second) labels? One or two split-vote examples would also help clarify whether raters disagreed about the interaction timing itself, or were trading off better motion against lower visual quality. This distinction matters for how the +0.95 gain should be interpreted and for whether "About the same" reflects equivalence or ambiguity.

2. OOD video results on user-recorded sources. R2-Table5 and Appendix C.3 report that ALIVE's margin over Señorita is similar on same-source (+26.0) and OOD (+27.1) cases, which addresses the numerical half of W5. The remaining question is qualitative: the current supplementary material shows results drawn from the benchmark's own sources (HOIGen-1M and the newly recorded videos), both of which share collection pipelines with the training data. Could the authors add, in an updated supplementary package, full edited videos on user-captured real-world sources that are clearly outside the training distribution, for example a handheld clip of a user picking up a pineapple, an everyday first-person capture, or any in-the-wild interaction video not sourced from the benchmark families? Since "making inserted objects alive" is the paper's framing, a small number of such demonstrations (even 3 to 5 clips, including successes and failures) would be the most convincing evidence that the mechanism generalizes beyond the curated data regime, and would meaningfully strengthen the OOD claim in §3.3.

Based on the information currently available and any further clarification from the authors, I will finalize my recommendation during the reviewer–AC discussion phase.

AuthorsOpenReview ↗

Follow-up Response to Reviewer mEjX

Thank you for your follow-up questions! We respond to both points below.

Q1. Inter-rater reliability of the human study

(i) How many ratings did each pair receive?

Twelve participants completed 13 questionnaires, yielding 147 ratings on 93 video pairs. Each pair received one to three ratings, and 44 pairs received multiple ratings. One Motion response was marked “cannot judge."

(ii) How often were viewers unanimous? and (iii) What was the chance-corrected agreement?

The table reports unanimity and Krippendorff’s α for the three-way preference judgments.

ComparisonOverall unanimousOverall αMotion unanimousMotion α
Full study27/44 (61%)0.4332/43 (74%)0.62
Predicted P4 vs. P114/29 (48%)0.2418/28 (64%)0.43
ALIVE (P1) vs. Señorita13/15 (87%)0.3014/15 (93%)0.49

For predicted P4 vs. P1, viewers chose “About the same” in 60 of 99 Overall ratings. The remaining votes split almost evenly. On 11 of the 15 pairs with mixed judgments, viewers differed only on whether one version was preferable or the two were about the same. This pattern is consistent with our conclusion that the average gain from predicted P4 was modest and not statistically significant.

Where the performance gap was larger, the same blind study showed both a clear preference and greater agreement. Viewers preferred ALIVE over Señorita in 44 of 48 Overall ratings and 45 of 48 Motion ratings.

Split-vote example. Viewers rated Overall editing quality and Motion separately. Of the 15 pairs with mixed Overall judgments, 10 also had mixed Motion judgments. Across 98 valid P4 vs. P1 ratings, viewers gave different answers to the two questions 12 times.

On the game-controller clip, the two viewers split on Overall, with one preferring P1 and the other predicted P4. In our frame-by-frame review, P1 better preserved the controller’s shape during pickup and lowering, while predicted P4 more closely matched how the controller rotated with the hand in the reference.

Q2: OOD video results on user-recorded sources.

A2: Thank you for this suggestion. Following your request, we recorded new handheld clips with an iPhone and added four of them to the updated supplementary, in a new section "OOD video results on user-recorded sources", as a complement to our benchmark.

  1. Apple (two clips). We used an apple for illustration: in one clip a hand moves it on a table, in the other it is picked up.

  2. Saucepan and bowl (first-person). A hand moves a saucepan between burners while the camera moves, and lifts a bowl from a counter.

Each case shows the source video, the edited first frame and the full ALIVE + VLM result, produced with the same model and settings as in the paper. These clips are not part of the training data or of any benchmark source.

Thank you again for your time and for the constructive discussion.

Reviewer_PxAPOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The proposed method, ALIVE, studies adding an object to a video and making it respond to the actions in that video. The inputs are the source video, an edited first frame, and an instruction naming the object. The authors build 35,800 editing pairs from rendered, generated, and real videos, plus ROSE data. The authors use their proposed data to train an LTX-2.5 editor. A separate VLM predicts descriptions of the interactions and their time intervals. Cross-attention then uses each description at the right time. On the 128-case interaction benchmark, the P1 editor scores 86.71, compared with 60.24 for the SOTA baseline. On the 103-case general-insertion benchmark, the scores are 92.28, compared with 88.39.

Critical strengths
  1. The added object must respond to actions such as grasping, cutting, or wearing. This is harder than just keeping its appearance consistent. The dataset includes both interaction and non-interaction examples. With the same backbone and 5k training steps, ROSE-only training scores 60.12 on the interaction benchmark. The mixed data reach 82.35 and also give the best general-insertion score in Table 2. This shows that the interaction data help.

  2. The model already gets a large improvement when the text only names the object. The paper separately tests annotated interaction descriptions and VLM predictions. This makes it clear which results use extra interaction information. The benchmark also includes HOIGen-1M and newly recorded videos outside the training sources. Appendix A explains the scoring rules, preprocessing, and VLM prediction cost.

Critical weaknesses
  1. The main interaction scores come from one VLM. It sees reference frames and judges 16 output frames, or 14 for I2VEdit. No comparison is reported with people watching the full videos. A few frames may miss broken contact, fast shape changes, or timing errors. CLIP/ViCLIP and feature consistency do not directly test these problems either. Human ratings would help check whether the scores are reliable, especially for the small 0.95 gain from VLM guidance.

  2. P2, P3, and P4 contain different amounts and types of information. Table 3 does not test the same P4 text with and without time-based routing. So it is hard to tell how much Equation 3 helps. Predicted P4 only raises the motion score from 86.13 to 86.91, while general-insertion Overall falls by 0.57 points.

Questions for authors
  1. Do people watching the full videos align with the VLM scores for contact, timing, and shape changes? This comparison requires a blind user study.

  2. If the predicted P4 text stays the same, how do the results change when all descriptions are used throughout the video instead of only during their assigned intervals?

3 comments
AuthorsOpenReview ↗

Response to Reviewer PxAP (Part 1 / 1)

We sincerely thank Reviewer PxAP for the careful and encouraging feedback. Below we respond to both points.

W1 & Q1: Do people watching the full videos align with the VLM scores for contact, timing, and shape changes? This comparison requires a blind user study.

A1: The blind study is now in Appendix C.1. People agree with the judge where methods differ clearly (ALIVE vs. Señorita). Where the judge's gap is small (predicted P4 vs. P1, 0.95 points), most raters chose "About the same".

Raters watch the full videos next to the source, the edited first frame and the reference, with method names hidden and the left/right order balanced. They answer which video is better overall, and which has better motion (contact, response timing, and shape change that follows the action). We collected 147 ratings on 93 video pairs from 13 questionnaires.

R1-Table1: Blind pairwise human study.

ComparisonQuestionPrefer first"About the same"Prefer second
ALIVE (P1) vs. SeñoritaOverall4422
ALIVE (P1) vs. SeñoritaMotion4521
ALIVE (predicted P4) vs. ALIVE (P1)Overall206019
ALIVE (predicted P4) vs. ALIVE (P1)Motion166517
  1. ALIVE (P1) vs. Señorita. Raters prefer ALIVE in over 90% of ratings, and when a rater has a preference, the judge picks the same video in 42 of 46 cases (91%). The study confirms that ALIVE with P1 alone already significantly outperforms the prior method Señorita.

  2. Predicted P4 vs. P1. The judge's average gain is 0.95 points, as reported in the paper, and a second judge (Claude Opus 5.5) gives a similar +1.06. A gap of this size may be hard to see side by side, consistent with raters choosing "About the same" in 61% (Overall) and 66% (Motion) of ratings.

    R1-Table2: Predicted P4 − P1 by subset (Overall). Interaction cases are split into halves by the P1 score of a separately trained editor.

    SubsetP1Predicted P4 − P1
    Interaction, easier half (64)93.9−0.82
    Interaction, harder half (64)79.5+2.71
    General insertion (103)92.28−0.57
  3. Where P4 helps. Predicted P4 helps where P1 leaves room (R1-Table2; main Table 5): it adds 2.71 on the harder half of the interaction cases. On the easier half and on general insertion, P1 already scores above 92, so P4 has little to add.

W2 & Q2: If the predicted P4 text stays the same, how do the results change when all descriptions are used throughout the video instead of only during their assigned intervals?

A2: Thank you for this question. We ran this ablation in two ways: removing routing at inference from the paper's editor, and training two editors with the same data, seed and steps, one with routed P4 and one with the same sentences concatenated into a single prompt.

R1-Table3: Routing ablation on ALIVE-interaction (Overall).

EditorP1Predicted P4, routedPredicted P4, all textAnnotated P4, routedAnnotated P4, all text
Paper editor, routing removed at inference86.7187.6683.8288.6985.55
Trained per format (5k steps)82.3582.3780.1383.3883.19
  1. Routing is higher or on par in all four comparisons, and matters most for predicted P4. With the paper's editor, using all descriptions throughout lowers Overall by 3.84 (predicted P4) and 3.14 (annotated P4). With editors trained per format, routing is on par for annotated P4 and 2.25 higher for predicted P4. A text-only editor has to infer from the wording when each sentence applies, while routing gives the timing directly.

  2. Role of Eq. 3. Routing binds each sentence to its frame interval, so the prompt can switch between segments of the generated video, which a single global prompt cannot do. This gives P4 an explicit time axis: a user can edit one stage or one boundary. R1-Table3 is now included as Table 6 in the revised manuscript.

Thank you again for your time and constructive comments. We hope these results address your concerns, and we are happy to answer any further questions.

Thanks to the authors for their clarifications in the rebuttal. They addressed most of my concerns. I will maintain my original positive score.

AuthorsOpenReview ↗

Thank you!

Thank you for your thoughtful review and continued support! Your feedback has been really helpful for both our evaluation and our revisions to the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.