- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors construct a trailer-editing benchmark from 49 films and their released trailers. Shots are aligned to source footage and grouped into chunks within story beats. Each chunk gives rise to four task types: Content Selection, evaluated by shot F1; Narrative Sequencing, evaluated by a permutation metric; Rough Cut, evaluated using timing and both reference-conditioned and reference-free judge scores; and Fine Cut, covering dialogue, production sound, music and transitions on a fixed picture, evaluated entirely by judges.
After Gemini-based auditing, the dataset contains 716 training tasks from 43 films and 81 validation tasks from six held-out films, for 797 tasks in total.
A Qwen3.5-9B agent is trained for 50 updates. The GRPO runs use 16 tasks with eight rollouts each per update. The paper compares outcome-only GRPO, GRPO with a fixed turn-level process reward from Gemini 3.8 Flash at lambda=0.5, and the same reward with lambda decayed to zero over updates 20-30. An on-policy self-distillation (OPSD) arm instead uses one rollout per task and a frozen copy of the initial policy as teacher, conditioned on either reference answers or Gemini-generated turn suggestions.
Outcome-only GRPO raises the overall score from 0.284 to 0.413. Decayed PRM reaches 0.447, with the largest gain on Fine Cut, 0.293 versus 0.201. Fixed PRM reaches only 0.334. OPSD scores 0.277 with references and 0.044 with suggestions. The paper also reports turn-quality analyses, tool-use and reasoning statistics, and diagnostics of the OPSD gradients and KL signal.
Critical strengths
(1) The testbed gives multimodal agents a substantial editing task with an inspectable final product. Film-disjoint splits and hidden references reduce obvious shortcuts. Validating and rendering deliverables, and instructing judges to score the result rather than the agent's declarations, are sensible choices. Episodes of 60-90 turns with delayed and partly subjective outcomes are a useful setting for studying credit assignment. The benchmark and planned release of its construction workflow are the strongest contribution in my view.
(2) The fixed-PRM failure is documented in enough detail to be useful. Figure 1(c) and Fig. 3 show process reward increasing while outcome reward falls after about update 30. Table 5 reports 165.8 tool calls versus 52.2 for GRPO, more frames viewed, and 44.1% truncation. Decaying the process weight reduces truncation to 6.2% and produces the best overall score. Even if this behavior is specific to the reward design, the measurements give practitioners a concrete failure mode to look for.
Critical weaknesses
(1) The evidence does not isolate a benefit from longer horizons, and its reliability across training runs is unknown.
All methods use a single training seed, 42. The overall difference between 0.413 and 0.447, and the Fine Cut difference between 0.201 and 0.293 on 26 validation tasks, need uncertainty estimates and repeated runs before they can support the central conclusion.
There is also no controlled variation of horizon. Fine Cut is the only task type scored entirely by an LLM judge, with channel weights 0/0.4/0.6. It depends on audio the policy cannot hear and on which the process judge must abstain. It has the lowest base score, 0.152, and requires rendering. Any of these differences could affect the value of the process reward.
Both the process judge and the outcome grader, Gemini 3.1 Pro, are Gemini models. Their agreement could contribute to the gain on the task scored entirely by a judge, independently of horizon. Rough Cut, which is 60% judge-scored, shows no PRM benefit, so judge alignment alone is not a complete explanation either. The current experiments cannot distinguish these possibilities.
The mixing analysis in Table 4 uses the same process judge on the untrained policy and is correlational. To support the horizon claim, I would like at least three seeds for the key arms and a comparison within one task type, such as Fine Cut tasks with different numbers of joins or Rough Cut tasks with different numbers of clips.
(2) The three training schemes change more than the granularity of credit assignment.
Adding a turn-level process reward also adds information from a strong external Gemini judge. Moving to OPSD then removes outcome reward, introduces distillation from a frozen 9B teacher with privileged information, and reduces rollouts per task from eight to one. These comparisons are informative about the chosen recipes but do not place trajectory-, turn- and token-level credit on a common footing.
The OPSD diagnostics reinforce this concern. With reference answers, per-token KL is only 0.0019, suggesting that the answer card barely changes the teacher distribution in a long multimodal context. The median trajectory contains 23 turns and 13.7k generated tokens, with a 131k context limit. There may simply be too little distillation signal in this setup. With suggestions, training collapses and 22 of 50 updates exceed the clipping threshold. Neither result establishes that token-level credit itself is ineffective. A comparison retaining outcome reward, for example OPSD combined with GRPO or a token-level value-based advantage, would be more informative.
The fixed-PRM conclusion also depends on one choice, lambda=0.5. Turns that produce new information earn +1, process advantages are normalized across rollouts at each turn index, and there is no step cost. This can reward continued inspection over finishing the edit. Because the shaping is not potential-based in the sense of Ng et al. (1999), it does not preserve the optimal policy by construction. Smaller fixed weights, such as 0.1 and 0.25, would help establish whether decay is necessary or whether the reported collapse follows mainly from an overly strong process reward.
Questions for authors
- How much of the fixed-PRM score loss comes from failure to deliver, and how much from poorer completed edits? Scores conditional on delivery would help distinguish these effects.
- Can you report uncertainty over validation tasks and, if feasible, repeat the outcome-only and decayed-PRM training runs? Since tasks share source films, please account for that dependence when interpreting intervals.
- Were smaller fixed process weights tested, and was the 20–30 decay window selected solely from the fixed run's training curves?
- Is there any human agreement check for the Fine Cut outcome grader or the process judge? Even a modest, clearly described sample would help interpret the judge-based improvements.
3 comments
As the discussion period ends soon, we hope that our rebuttal has addressed your concerns. If any questions remain, we would be happy to provide additional information or clarification. Thank you for your time.
Dear Reviewer,
Thank you for taking the time to review our paper. As the discussion period draws to a close, we wanted to briefly follow up on our rebuttal. If you have a chance to read our response before the discussion closes, we’d appreciate hearing whether any of your concerns remain. We’d be happy to clarify any points you’d like to discuss further.
Thank you again for your time. The authors
Response to Reviewer vg2q
Dear Reviewer vg2q,
We greatly appreciate your thoughtful feedback and constructive suggestions. Below, we provide detailed responses and clarifications. If our responses address your concerns, we would be grateful if you would consider updating your score.
W1. The evidence does not isolate a benefit from longer horizons, and reliability across training runs is unknown
W2. The three training schemes change more than the granularity of credit assignment
Q1. How much of the fixed-PRM loss comes from failure to deliver?
Almost all of it comes from failure to deliver. Over the 324 validation episodes (81 tasks, four repetitions):
When the fixed-λ model delivers, its edits score the same as those of outcome-only GRPO. It delivers less often.
Q2. Uncertainty over validation tasks, accounting for shared films
GRPO+PRM (decayed λ) against outcome-only GRPO, paired per task, with intervals from resampling whole films:
The Fine Cut difference is clear. The overall difference is positive in five of six films, but its interval touches zero. By-film numbers for Fine Cut are in the response to Reviewer yMuD, Q2. Repeated training runs: see the general response (part 1/3).
Q3. Smaller fixed weights, and how the decay window was selected
Q4. Human agreement check for the Fine Cut grader
See general response 1/3.