- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents a training-free framework for motion-centric video editing. It optimizes audio latents and residual text conditioning at test time while keeping the audio-video generator frozen. A frozen VLM evaluates the generated video and provides gradients to guide optimization, together with latent, perceptual, and temporal regularization. Experiments on LTX-2 Retake and MiniMax-H3 demonstrate improvements over their prompt-only outputs.
Critical strengths
- The observation that the audio-conditioning can help control visual motion is interesting and motivates exploring multimodal conditioning for video editing.
- The method is evaluated on two different audio-video backbones, providing evidence of applicability beyond a single architecture.
Critical weaknesses
-
The technical contribution is limited, and the audio-specific benefit is not sufficiently established. Differentiable feedback from frozen VLMs and truncated backpropagation have been explored in Diffusion-DRF. The main distinction is therefore the choice of audio and text conditioning as optimization variables. However, Supplementary Fig. 15 shows that joint tuning mainly improves naturalness and preservation rather than motion strength, leaving the stronger interpretation of audio as a motion-specific control space insufficiently supported.
-
The experimental setting does not fully support the claimed scope of video motion editing. The LTX experiments preserve only 1–25 frames of an 89-frame video and regenerate the remaining suffix. This primarily demonstrates prefix-conditioned motion generation, while preservation of unedited dynamics, camera motion, and content appearing later in the source video remains insufficiently tested. The paper also does not clearly establish whether baselines use comparable source-conditioning, making the results difficult to interpret.
-
The evaluation is too limited to establish broad effectiveness. The main comparison uses only 12 examples. The full LTX set contains 25 examples, including six transfer targets, while H3 is evaluated on a different set. These results provide promising examples but limited evidence of robustness across scenes. Evaluation on a fixed public motion-editing subset, such as that of IVEBench [1], would strengthen the conclusions.
-
The computational cost limits practical use. Each optimization iteration requires a complete video generation followed by backpropagation. Editing a short clip takes approximately 10 minutes on one H100 for LTX and 20–30 minutes on two H100s for H3, making the method costly and limiting its practical use.
[1] IVEBench: Modern Benchmark Suite for Instruction-Guided Video Editing Assessment. ICLR 2026.
Questions for authors
- Under the same cost, how does the method compare with best-of-N, i.e., generating multiple candidates and selecting the best using the same VLM critic?
- How sensitive is the editing performance to the location and length of the backpropagation window, particularly the use of only the final two denoising steps for H3?
4 comments
W3: Evaluation scope Thank you for the IVEBench pointer. We now report results on all 64 subject-motion items in the general response. The comparison in the paper is backed by human raters and GPT-5.5 Thinking as a VLM judge. For the rebuttal we also added Two more independent judges (mean yes-probability):
| Method | Qwen3-VL-8B | InternVL3-8B |
|---|---|---|
| Ours | 0.73 | 0.73 |
| Runway-Aleph | 0.62 | 0.64 |
| LoRA-Edit | 0.56 | 0.51 |
| LTX-2 Retake | 0.48 | 0.56 |
Also we used Claude Opus 5.5 as a judge, with same prompt as GPT-5.5:
| Method | Task success | Overall score (0-10) | Avg. rank (1 = best) |
|---|---|---|---|
| Ours | 75% | 6.03 | 2.08 |
| Runway-Aleph | 58% | 5.40 | 2.79 |
| LoRA-Edit | 46% | 5.30 | 2.96 |
| LTX-2 Retake | 46% | 5.01 | 3.12 |
Gemini-3.1 also ranks ours first in 11 of 12 scenarios.
W4: Computational cost Each iteration costs 1.44 times a forward Retake pass, about ten minutes per edit on LTX-2 and 20 to 30 minutes on H3 across two GPUs. We see the method as test-time control and a probe of the conditioning space, not an interactive editor. However, learned latents can be reused on new videos without re-tuning (Sec. 4.4), which spreads the cost. We will move the runtime table to the main text if needed.
We hope these answers are helpful, and we would be glad to hear your thoughts. Thank you again for your careful review.
Dear Reviewer s2LM,
Thank you again for your concrete suggestions. As the discussion period nears its end, we wanted to highlight the new experiments prompted by your review: the same-critic best-of-N comparison, the backpropagation-window study, and evaluation on all 64 IVEBench subject–motion items. We also clarified what audio conditioning contributes, the source conditioning used by the baselines, and the runtime limitation. If our responses have resolved your concerns, we would be grateful if you could reconsider your score before the discussion period closes; we would also be glad to clarify any remaining points.
Thank you for the responses and additional experiments, which strengthen the evidence for the method’s effectiveness. However, I still find the technical novelty limited beyond existing test-time optimization methods. My concerns about the experimental setting and computational cost are also not fully addressed, leaving doubts about the method’s applicability to practical video editing. I will therefore maintain my original rating.
Thank you again for your thoughtful responses.
Thank you very much for your precise review and concrete suggestions; both of your questions led to new experiments which we will discuss below.
Q1: Best-of-N at the same cost The controls in the general response already spend our budget on other variables and showed that our method works best. For best-of-N, we rendered prompt-only Retake with N seeds per scenario and kept the sample that the same VLM critic scores highest. Mean yes-probability of held-out VLMs over the 12 scenarios:
For some motions, no value of N helps: when the prior rarely or never samples the action, more seeds do not help, and no sample makes the rose bloom or the headlights flash periodically.
Best-of-N works when the backbone already produces the action for some seed. Where a seed does produce the action, the critic picks the one with the most motion, which is often abrupt or exaggerated: the turtle's neck collapses, a fruit appears in the monkey's hand, and a fish appears in the tank instead of jumping out.
The two approaches are also complementary rather than competing. Best-of-N selects among samples, while our method changes the conditioning and yields a reusable latent, so they can be combined: applying best-of-N on top of our tuned latents beats all of the other approaches, and at N = 8 both Qwen3-VL and InternVL3 (held-out judges) judge the edit successful in all 12 scenarios. Sample videos are on the supplementary website (https://sound-sparks-motion.supplementary-site.workers.dev/#rb-bon8).
Q2: Sensitivity to the backpropagation window LTX-2 backpropagates through the last 8 of 30 denoising steps. In a sweep over K in {2,4,8,12} on five scenarios, easier edits (a dog yawning, a bird opening its wings) work with any window, while harder ones benefit from a longer window, such as the turtle extending its neck. Memory grows with K, and K=12 already fills an 80 GB H100, so we use K=8. On H3, memory limits us to the last 2 of 16 steps (a 33B model across two GPUs), so we cannot run more than 2 steps with our limited compute, even on a two-H100 setup. However, in K=2 the method still raises task achievement from 46.7% to 68.3% and improves H3 meaningfully.
W1: Technical contribution and the audio-specific benefit We agree with your interpretation of Supp. Fig. 15. The text residual controls what happens (objects and actions), while the audio latent controls how it happens (motion trajectory). Joint optimization therefore improves naturalness, source preservation, and locality rather than simply increasing motion strength. These are important for motion editing, where prompt-only methods often produce abrupt changes, unwanted objects, or drift. The benefit of audio conditioning is further supported by our control experiments (46.3% first-place votes for ours versus at most 15.4% for the controls; see general response) and H3 results (28.9% for audio-only versus 12.2% for text-only).
W2: Prefix conditioning and baseline inputs Retake is the available LTX-2 pipeline for text-driven editing of existing videos. LTX-2 does not provide a general video-to-video editing pipeline. Its IC-LoRA adapters target specific appearance edits and preserve the source structure, making them unsuitable for motion editing. We therefore use Retake for LTX-2. In contrast, MiniMax-H3 supports full-video editing without preserving a prefix. Despite these differences in architecture and editing pipelines, our method improves motion-editing success on both models: from 50% to 83.3% on LTX-2 and from 46.7% to 68.3% on H3. To further support this result, we extended our VLM evaluation using Claude Opus 5.5 to compare Ours+H3 against H3: