- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper studies professional motion editing, where video editing models given only the name of a professional motion often produce the wrong motion. It proposes MotionCoach, a Qwen3-VL-8B agent that uses search, inspect, and clip tools with self-verification to produce a short reference clip and a detailed grounded prompt, which are fed to a frozen video editor. The agent is trained by SFT on teacher-generated trajectories, then GRPO with prompt, video-outcome, and format rewards. The paper also introduces VPMEBench, 314 human-verified tasks over 25 categories scored by a VLM judge, and reports overall gains from 55.1 to 65.6 on Seedance 2.0 and 51.2 to 64.5 on MiniMax-H3.
Critical strengths
- The paper proposes MotionCoach, an agent trained with SFT and GRPO (prompt, video-outcome, and format rewards) that retrieves and verifies a reference clip and writes a grounded prompt, so frozen video editors can execute named professional motions.
- The paper introduces MotionCoach-SFT-10k and RL-2k, teacher-generated trajectories with VLM checks and human spot-checks, and VPMEBench, 314 human-verified tasks over 25 categories with reference clips, expert keyframes, and technique annotations, whose VLM judge is validated against domain experts.
- The paper achieves the best results on VPMEBench, raising the overall score from 55.1 to 65.6 on Seedance 2.0 and from 51.2 to 64.5 on MiniMax-H3, supported by ablations on training stages, inputs, agent components, and reward weights.
Critical weaknesses
-
Claim–method mismatch. The introduction defines the gap as acquiring and verifying external motion knowledge. The method that fills this gap is a fixed workflow made of a system prompt, three tools over a pre-built index, and a self-check. This workflow exists before any training, because the teacher, Gemini 3.5 Flash, already runs it end to end to produce the training data. The paper’s contribution, however, is to train an 8B policy to imitate those teacher trajectories, which answers a different question, whether a small model can reproduce the teacher, that the introduction never raises. The two sides are left unreconciled by the experiments. On retrieval, no baseline lets an LLM write the same detailed prompt from its own knowledge without search or clip, although on MiniMax-H3 removing the reference clip costs only 0.8 points and on IVEBench the score improves even though the professional-motion corpus cannot match its generic instructions. On training, the untrained agents compared are only Qwen3.5-9B and 27B, while the teacher and the other flagship models already used in the pipeline, GPT-5.5 and Claude Opus 4.7, are not run as agents. If Gemini 3.5 Flash solves the task, the trained policy adds at most an unmeasured cost advantage. If it does not, the SFT trajectories and the teacher prompts used as RL reward targets are unreliable.
-
Human-only scope. The paper claims that video editing models fail at professional motion editing in general, and the title, abstract and contributions keep this general framing. Every piece of evidence, however, is a person performing a dance, sports or martial-arts move. The data preparation keeps only human-centered videos, the taxonomy has only these three domains, and the judge prompt is written for human motion. The restriction is never stated, and no limitation is given. Within this human scope, pose-driven human animation and reference-based video motion editing take exactly a source video plus a reference motion and directly target the motion correctness this paper measures, yet this line of work is neither cited nor compared.
-
Results rest on a self-built benchmark. All main results and ablations are reported on VPMEBench, which the authors construct, score with a judge of their choice, and aggregate with weights of their choice. The only human evaluation checks agreement between this judge and annotators on 100 pairs and does not compare methods. The single external result, on the subject-motion subset of IVEBench, does not add support. It uses one editor, compares only the base model with and without MotionCoach, gives neither the subset size nor the definitions of its ten metrics, and its total gain comes mainly from one metric while two others decrease. Its instructions are generic motions that the professional-motion index cannot match, so the improvement cannot come from retrieved professional knowledge and does not test the paper’s claim.
-
Limited novelty. Retrieval-augmented generation and visual-outcome-trained agents exist and are cited, and the new ingredient, the video reward, adds 0.7 points. The methods used for positioning, RS-Gen, VACE and Kiwi-Edit, are not compared. There is also no lookup baseline without an agent, i.e. exact-match retrieval, fixed clipping and an LLM-written prompt, which would show whether the agent’s decisions add anything beyond the pre-built index.
Questions for authors
-
How does search work? What is the source, size and labeling of the reference index, does it contain the tutorials behind the VPMEBench reference clips, and will it be released?
-
How were the training data collected and filtered? How many trajectories were human spot-checked, by whom, and with what acceptance rate? Who wrote the technique and negative-constraint annotations in VPMEBench?
-
What are the details of the 100-pair human study? Which pairs were sampled, what were the human preference rates, and why are the Table 1 baselines not included?
4 comments
To W4 (methodological contribution and novelty):
Our core contribution is a policy that learns temporal grounding for professional motion guidance, with feedback from a frozen video editor shaping how it selects and organizes temporal evidence. MotionCoach also enables Editor-in-the-loop RL with cross editor transfer: the policy is trained with video rewards from MiniMax-H3, but its guidance raises Seedance 2.0 MP from 29.6 to 53.0 (Table 1).
The lookup baseline uses Exact-Match Retrieval → Fixed Clipping → LLM-Written Prompt, with code-controlled execution and the same candidate references, video observations, and inference budget, but no adaptive reference selection or temporal refinement.
| Configuration | SD2 | H3 | ||||||
|---|---|---|---|---|---|---|---|---|
| MP | PP | VQ | Overall | MP | PP | VQ | Overall | |
| Base editor | 29.6 | 85.8 | 77.1 | 55.1 | 16.2 | 84.7 | 87.3 | 51.2 |
| Fixed retrieval–clipping–captioning | 31.06 | 77.78 | 54.55 | 47.45 | 31.47 | 85.94 | 36.90 | 44.00 |
| MotionCoach (Ours) | 53.03 | 87.10 | 72.24 | 65.61 | 33.1 | 92.2 | 98.2 | 64.46 |
MotionCoach gains 18.16 / 20.46 Overall points on SD2 / H3 over this fixed pipeline. This supports learned temporal decisions beyond reference access and reliable tool execution.
Video reward. With matched training budgets, the table reports means and 95% CIs across three independent training seeds.
| Configuration | Overall [95% CI] | MP [95% CI] | PP [95% CI] | VQ [95% CI] |
|---|---|---|---|---|
| RL: prompt reward only | 64.70 [64.03, 65.37] | 53.56 [51.53, 55.59] | 84.45 [79.95, 88.94] | 70.11 [68.78, 71.45] |
| RL: prompt + video rewards | 65.28 [64.48, 66.07] | 53.98 [51.61, 56.36] | 85.72 [82.16, 89.28] | 70.47 [66.24, 74.70] |
All metric means improve with mixed rewards. The mean Overall gain is +0.58 [95% CI: −0.10, 1.26].
To Q1 (search implementation and index overlap):
MotionCoach searches Bilibili online using the editing instruction, rather than a pre-built index (Introduction, p. 2, L072–075; Ethics Statement, p. 11, L556–557). SFT, RL, and benchmark splits have disjoint target motion names, source videos, and edit prompts (Section 3.1, L195–196).
To Q2 (training-data filtering and benchmark annotations):
Teacher trajectories undergo programmatic validation of formats and tool arguments, VLM checks, and human spot-checks (Section 3.1). VLM checks cover visible and complete motion phases/mechanics and prompt–clip consistency.
Of 14,621 candidates, 12,686 (86.77%) passed programmatic and VLM checks. Five experts in sports, dance, and martial arts spot-checked 500 randomly selected cases, with a 94% pass rate.
For VPMEBench, Gemini 3.5 Flash drafted motion-technique and negative-constraint annotations; five experts reviewed and corrected every item.
To Q3 (human-study design and method comparisons):
Category-stratified sampling. Appendix A.1 (p. 14, L725–734) uses stratified random sampling of 100 Base vs. Base+RL pairs: 50 per editor, with 46 dance, 34 sports, and 20 martial-arts pairs. Each pair shares a source video and instruction and is independently rated by three of eight experienced annotators, with randomized, anonymized A/B order.
Human preferences. Annotators compare Base+RL with the same frozen editor's Base output and assign Good / Same / Bad (G/S/B) judgments for MP, PP, VQ, and Overall. We aggregate the three judgments per pair by their ordinal median (Bad < Same < Good). For each editor and dimension, we count G, S, and B across the 50 pairs and compute the GSB ratio R = (G + S) / (S + B). R > 1 indicates more improvements than regressions relative to Base. For MiniMax-H3, MotionCoach achieves an Overall GSB ratio of 2.10 over Base.
Choice of comparisons. Comparing the same editor with and without MotionCoach measures the incremental value of agent-constructed guidance while controlling for the generation model. Following the reviewer's suggestion, we additionally compare MotionCoach + MiniMax-H3 against Kiwi-Edit and Ditto from Table 1 using 50 video pairs per comparison. MotionCoach achieves Overall GSB ratios of 6.25 and 5.00, respectively, indicating more favorable than unfavorable judgments against both baselines.
We hope the clarification and additional experiments have sufficiently addressed your concerns. If you are satisfied, we kindly invite you to consider adjusting the score accordingly. We will also remain responsive to any further issues during the discussion phase.
Thank you for the rebuttal. It resolves part of my concerns, but several important issues remain.
-
The paper demonstrates improvements in editing quality and tool-execution success. However, it remains unclear whether training also improves verification, reference selection or temporal localization.
-
There are still no comparisons with related specialized methods for human motion editing.
-
I spent considerable time examining the implementation details described in the paper but the search procedure remains insufficiently unclear. In particular, the original paper does not clearly specify the online Bilibili search implementation described in the rebuttal. The clarification leaves important questions about candidate retrieval and processing unanswered. What's more, the authors have not provided any code to help verify these details.
Therefore, I maintain my original score.
For Reviewer FLE1 (R3)
To W1 (claim-method mismatch):
Adaptive policy and outcome-based learning. MotionCoach learns temporal grounding through frozen-editor feedback, rather than following a fixed workflow: it adaptively selects queries, references, intervals, and refinement actions. Teacher trajectories initialize tool use through SFT; RL optimizes guidance with prompt and edited-video rewards (Sections 3.1–3.3). Table 4 shows further gains from RL beyond SFT.
Added comparisons. The no-retrieval LLM writes a prompt from its own knowledge, without search or clips. Teacher and flagship agents share MotionCoach's tools, 65,536-token context budget, and 10-round limit.
On SD2 / H3, MotionCoach gains 14.61 / 9.79 Overall and 30.42 / 19.25 MP over no retrieval, and 4.08 / 3.71, 6.69 / 5.65, and 1.22 / 0.22 Overall over Gemini, GPT, and Claude. These gains show that editor-feedback training can surpass the teacher's aggregate performance.
Clip ablation. Table 5 removes the editor's reference clip but retains the prompt grounded in the searched video. Most external motion evidence is already encoded in the prompt; the clip supplies supplementary visual guidance.
To W2 (scope and comparisons with motion-control methods):
Why professional motion editing and human-motion scope. IVEBench identifies “more challenging editing types, such as subject motion editing” and notes that “the editing capability of current models is particularly limited” (Sec. 6.2). MiniMax-H3 scores only 48.0/100 Total on this subset (Table 9). On VPMEBench, Seedance 2.0 scores 55.1/100 Overall and 29.6/100 MP (Table 1). We therefore focus on human professional motions in dance, sports, and martial arts, and will clarify this scope and limited non-human coverage.
Motion-control comparisons. Pose or reference-conditioned methods “assume that suitable reference is already available” (Introduction, p. 1, L049–052). MotionCoach takes only a source video and short instruction, then searches for demonstrations, localizes relevant intervals, and verifies motion correctness, visibility, and completeness. It addresses guidance acquisition and verification that those input settings leave to users.
To W3 (benchmark credibility and evaluation robustness):
Evaluation protocol. Sections 3.1 and A.3.2–A.3.4 disclose the rubric, scoring levels, and aggregation weights. Human preferences agree with the automatic judge, with Overall QWK = 0.846 (Table 10); Q3 details the study design. Reward and evaluation use different judge models: Qwen3-VL-30B-A3B computes RL rewards, while Gemini 3.5 Flash performs evaluation and also serves as the teacher.
IVEBench. Both editors are evaluated on the same 64 subject-motion tasks from its short subset. We retain its video-quality (SC, BC, TF, MS, VTSS), instruction-compliance (OSC, PSC, IS), and video-fidelity (SF, CF) metrics. Total averages the three dimension scores, each a weighted mean of normalized applicable metrics (Sec. 4 and Appendix C); we report it on a 100-point scale.
Online search uses each editing instruction, so generic motions are not excluded by a fixed professional-motion index (Q1).
MotionCoach raises Total by 5.8 on MiniMax-H3 and 7.48 on Seedance 2.0. H3 gains mainly in SF and OSC; Seedance also improves IS and CF. These aggregate gains support transfer to generic motion instructions across both frozen editors.