- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies message-driven video editing, where a model is given a long source video and a natural-language message describing the intended narrative, and must select and order temporal segments to communicate that message.
To support this task, the authors introduce MEDit-Bench, which contains 60 long-form videos collected from LongVideoBench, VideoMME, and Blender Studio. Each video is paired with three author-created messages, resulting in 180 video-message pairs. Three professional editors independently edit each pair, producing 540 visual-only, cut-based reference edits. Editors are instructed to create 1–3 minute summaries, freely reorder shots, and ignore audio. Each message is also annotated for ambiguity and contextfulness.
The paper evaluates frozen commercial and open-source MLLMs by prompting them to directly predict a sequence of temporal cuts. It also introduces MEDitor, a GRPO-post-trained model initialized from Qwen3-VL. The post-training data consist of 2,931 GPT-5-generated message-cut pairs from 977 MLVU videos. The reward combines union-based temporal coverage with cut-level matching, encouraging both reference coverage and a reasonable number and placement of cuts. Performance is evaluated using temporal alignment metrics, LLM-based preference, and a human perceptual study.
Critical strengths
-
MEDit-Bench provides multiple professionally produced edits for each video-message pair. This is a potentially useful resource for studying instruction- or message-conditioned video editing, particularly because multiple human references better reflect the subjectivity of editing than a single reference.
-
The authors conduct a comprehensive human study covering several perceptual dimensions, including message transmission, cut order, shot continuity, contextual depth, and narrative pacing. This study helps reveal aspects of editing quality that cannot be captured by temporal overlap alone.
-
The post-training objective is carefully designed. The reward ablation demonstrates that the coverage and cut-level matching terms are complementary, and that combining them improves temporal alignment while producing a more reasonable cut structure.
-
The analysis of LLM-as-a-judge is transparent and informative. In particular, identifying severe position bias and order inconsistency is valuable because it cautions against using LLM preference as a reliable substitute for human evaluation in this task.
Critical weaknesses
-
The writing and task positioning need to be improved. I found it difficult to follow the distinction among video editing, video summarization, and recap generation. The paper moves between these concepts without providing a sufficiently precise boundary, even though the proposed task operationally consists of selecting and reordering temporal segments to create a shortened video.
-
The opening claim that “video editing is fundamentally message-driven” appears overly broad relative to the actual scope of the benchmark. MEDit-Bench is entirely visual-only: editors are explicitly instructed to ignore audio, and the models do not use dialogue, narration, music, sound effects, or audiovisual synchronization. These elements are often central to communicating a narrative in realistic video editing. The task would therefore be more accurately described as visual-only, cut-based, message-conditioned editing.
-
It remains unclear whether the proposed task is fundamentally different from existing language-conditioned video summarization or recap generation. Several provided “messages,” such as “Breathtaking Swiss landscapes,” “History of the World Cup,” and “Transformation of ingredients,” resemble topic or theme conditions rather than communicative messages that express a specific proposition. The paper needs a clearer definition of what constitutes a communicative message and a stronger explanation of the additional capability required beyond existing language-conditioned summarization.
-
The main MEDit-Bench experiments primarily compare general-purpose MLLMs and simple embedding-based retrieval baselines. Although SD-VSum is evaluated through cross-dataset transfer, this is not a controlled comparison because the methods use different training data, input semantics, and output formats. It would strengthen the paper to adapt or retrain representative language-conditioned summarization, recap-generation, or instruction-driven compilation methods using the same training data and evaluation protocol.
-
More objective and scalable evaluation metrics are needed. The current automatic evaluation is almost entirely based on temporal overlap through R@θ, F1@θ, and mIoU. These metrics cannot directly determine whether an edit communicates the intended message, provides sufficient narrative context, or forms a coherent and well-paced story. This is especially problematic because the paper emphasizes that multiple temporally different edits may be valid for the same message.
Questions for authors
- Can you formally distinguish a “communicative message” from a topic, theme, query, or language instruction?
- What additional capability does MEDit-Bench evaluate beyond existing language-conditioned summarization benchmarks?
- How should the benchmark score an edit that communicates the intended message effectively but selects temporally different shots from all three reference edits?
- A comparison among the base model, SFT, GRPO, and SFT followed by GRPO would clarify whether the improvement comes specifically from reinforcement learning or more generally from task-specific pseudo-supervision.
- Could you include stronger task-specific baselines, particularly language-conditioned summarization, recap-generation, and instruction-driven compilation methods, under a controlled evaluation setting?