- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
CutMaster takes hour-long footage and edit it down into a short montage, based on a user request, a piece of background music, and target duration. Its workflow have six roles and bring together a reusable memory of the source, slots aligned to the music, dialogue anchors, candidate clips that is checked visually, and a sequence selection step with constraints. The paper also introduce MontageX-Bench, which has 40 task built from ten source video, and compares the system against five other editing systems.
Critical strengths
1.The workflow turn editing decisions into concrete steps you can actually inspect. Handling dialogue anchors, music-aligned slots, and candidate selection separately seem like a good way to deal with requirements that pulls against each other. The worked example also show visual verification catching a bad retrieval before anything get rendered. 2.The improvement is backed up by several separate check. CutMaster comes first in main evaluation, stay ahead when a second VLM judge is used, and get higher human ratings than the strongest baseline, 4.47 compared with 4.13. That's more convincing than a gain that only show up with single automated evaluator.
Critical weaknesses
1.Taking out the Timeline Scout and the Edit Composer together do show that these parts helps, but it doesn't isolate verification, sequence selection, or shared-state coordination on their own. The comparisons backs up an end-to-end advantage. To really support the central design claim, though, I'd want to see simplified workflow that use the same models and similar inference budget. 2.The variation in Table 1 come from three evaluation run, so it doesn't tell us how much the results varies across independently generated edits. The human study is useful, but the paper don't say how the 100 ratings are spread across systems and tasks, and it doesn't give the sample count or uncertainty behind pairwise preferences. Since everything comes from just ten source video, these details matters for judging how far the results generalize.
Questions for authors
1.If the models and inference budget are kept about the same, how much does verification and replanning actually adds compared with simplified workflow? It would help to also report how many model call were made, or what the cost was. 2.How many distinct task and ratings goes into each human score and each pairwise preference? Was the comparisons matched by task? And once you account for the same sources and participants showing up more than once, how much uncertainty are left in the differences?
4 comments
Response to Reviewer B1Gv (Part 2 of 2)
Q1: How much do verification and replanning improve over a simplified workflow?
We additionally evaluate a simplified CutMaster using the same models in a single-pass pipeline, without VLM-based verification or scoring, retries, or replanning. Here we report the API calls, API cost and runtime per task.
| Configuration | Quality | API calls / task | API cost / task | Runtime / task |
|---|---|---|---|---|
| CutMaster | 81.40 ± 0.10 | 63.78 | $0.525 | 734.4 s |
| Simplified | 76.50 ± 0.42 | 11.47 | $0.300 | 223.0 s |
The full workflow obtains a Quality score of 81.40, compared with 76.50 for the simplified pipeline, at an additional API cost of only $0.225 per task. Verification, scoring, and iterative repair increase inference expenditure while supporting higher-quality outputs. Moreover, the simplified workflow successfully produces final videos for only 27 of the 40 tasks, highlighting the importance of these mechanisms for reliable execution.
Dear Reviewer B1Gv,
As the discussion period draws to a close, we would appreciate your feedback on whether our rebuttal has addressed your concerns. We would be happy to clarify any remaining issues.
Best regards,
The Authors
Thanks you for the detailed response. I decided to keep my positive score.
Response to Reviewer B1Gv (Part 1 of 2)
We are grateful for your constructive feedback and your positive recognition of our work. We address your concerns point by point below, and update the manuscript accordingly.
Thanks for your detailed suggestion. Timeline Scout (T) retrieves candidate clips from the source video, defining the candidate space on which Edit Composer (E) operates; removing T alone would therefore leave E without its required input. We therefore introduce w/o E, which replaces E's global sequence selection with local clip selection.
We also introduce Simplified CutMaster workflow, which uses the same models in a single-pass workflow without VLM-based verification, scoring, retries, or replanning.
Contribution of E. Removing E reduces Quality by 1.10 points, demonstrating the benefit of sequence-level optimization over local selection using the same candidates.
Contribution of T. Further removing T reduces Quality by 0.93 points, indicating the combined contribution of candidate retrieval and visual verification.
Simplified workflow. Compared with the full workflow (81.40), the simplified version achieves 76.50 Quality, with lower scores across all four VLM-judged metrics. This supports the benefit of verification and iterative repair over single-pass execution.
To assess system stability, we rerun the benchmark twice with CutMaster, obtaining Quality scores of 80.69 and 80.67. Across all three generation runs, Quality is 80.96 ± 0.48, indicating consistent performance.
We have included these results in Appendix H.
Thank you for the detailed suggestion. We clarify the human evaluation setup and statistical uncertainty below.
Human Likert scores. We randomly sampled 100 video outputs for Likert evaluation, covering 37 tasks and 10 source videos, with ratings from 10 participants. The resulting ratings comprise 17 for CutClaw, 16 for DIRECT, 16 for NarratoAI, 19 for OpenMontage, 17 for VideoAgent, and 15 for CutMaster.
Pairwise comparisons and uncertainty. All comparisons are task-matched. During rebuttal, we expanded the evaluation to 20 comparisons between CutMaster and each baseline. We report each baseline's preference rate as (wins + 0.5 × ties) / n, with approximate 95% Wilson confidence intervals. All baseline preference rates are below 50%, indicating a consistent preference for CutMaster.
We have updated Figure 2 and included these analyses in Appendix I.