acceptodds
Under review as a conference paper at ICLR 2027

Text2Cut: Controlling Non-Linear Video Editing with Language

Abstract

Non-linear video editing manipulates clips, tracks, transitions, and effects over structured timelines, yet existing editing systems often rely on software-specific representations, making it challenging to translate natural-language instructions into portable and executable edits. OpenTimelineIO (OTIO) provides a software-agnostic abstraction for timeline-level editing, but does not uniformly represent many effect-level transformations required in practical workflows. To this end, we introduce Text2Cut, a language-driven non-linear video editing framework that extends OTIO with a unified space of executable operations spanning both timeline-level manipulations and effect-level transformations. Rather than directly generating a complete timeline, Text2Cut predicts atomic editing operations that are executed and validated over structured timeline states, with the resulting state compiled into a portable OTIO-based representation. Furthermore, we train Text2Cut on a hybrid corpus of synthetic trajectories and real editing projects, and evaluate it on controlled compositional benchmarks and held-out real editing workflows. Experiments show that Text2Cut achieves highest final-state correctness while maintaining strong execution validity.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Text2Cut

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_imEcOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

Text2Cut translates language instructions into executable operations for non-linear video editing. It executes and validates these operations before compiling an OTIO-based timeline. The planner is trained on synthetic and human editing trajectories.

Critical strengths
  • The explicit operation representation provides a useful interface for inspecting and revising edits, including effects not directly supported by standard OTIO.
Critical weaknesses
  • The novelty is limited. The framework primarily combines a predefined editing interface, deterministic execution, OTIO export, and supervised fine-tuning. While this integration requires engineering effort, the paper does not clearly establish a substantive learning or planning innovation beyond this task-specific system design.

  • The experiments mainly demonstrate that task-specific fine-tuning improves performance over zero/few-shot prompting. Text2Cut is fine-tuned on the proposed instruction–operation corpus, whereas the primary baselines are evaluated in zero/few-shot settings. Moreover, these baselines already predict operations that are evaluated using the same executor, so this comparison does not establish the advantage of the proposed intermediate representation over alternative representations. There are no controlled comparisons against comparably trained direct timeline or code-generation baselines, nor are there ablations isolating the contributions of execution feedback and the hybrid training data. Without matched-training comparisons or component ablations, the paper does not demonstrate which design choices are responsible for the improvements or offer clear lessons beyond learning the proposed editing interface.

Questions for authors
  • Does the planner alternate between operation prediction and execution and receive the resulting state as input, or does it generate the entire trajectory before execution? Which procedure was used to obtain the results in Tables 1–2, and what happens when an intermediate operation is invalid?

  • With the same training data and backbone, does the operation representation outperform direct timeline or executable-code generation?

Reviewer_g8nYOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Text2Cut, an agentic framework that uses an extended OTIO operation space with a planner, executor, validator, and finally OTIO compilation to convert natural language video editing instructions into a format that can be directly imported into popular editing tools such as Adobe PR or Final Cut Pro. The paper also introduces a dataset that contains 100 real clip editing traces from real human editors and 1000 synthetic traces.

Critical strengths
  1. The motivation of this paper seems quite strong to me; we indeed need an agentic framework for non-linear editing.
  2. The author smartly uses an existing interface, OTIO, that is compatible with industry software and aligns with the industry production pipeline. The agentic workflow is also well designed with execution and validation loops.
Critical weaknesses

I have little/almost no expertise in this area, so the weakness below should be examined by the AC, and the authors should correct me if I am wrong.

1: In the supplementary, the natural language instructions seem to all be in the same language style. Normally human will not write in the same format and language style; I wonder how robust the model is at handling more diverse language inputs. 2: Similarly, can the model handle more ambiguous requirements in the natural language input? How severe will the performance drop if the input is ambiguous? 3: The numbers in the table seem to be inconsistent with the text. In line 332 it says "For example, few-shot Qwen3.7-Plus achieves MV=1.000M_V=1.000 and MS=0.960M_S=0.960 on Cold-easy..." but in table 1, it reports MV=0.960M_V=0.960 and MS=1.000M_S=1.000. Also table 3 in the main text (Lines 394–420) and table 5 in the appendix (Lines 918–945) seem to be duplicates

Questions for authors

See weakness

Reviewer_WBrGOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This work proposed a framework, Text2Cut, to perform nonlinear video editing (NLE) using textual instructions. The authors proposed a unified, software-agnostic description of NLE operations based on an extended OTIO representation, and constructed a hybrid dataset consisting both human-authored editing trajectories and synthetic ones. The authors evaluated the effectiveness of Text2Cut through experiments.

Critical strengths
  1. The proposed framework does not rely on existing, specific video editing software.
  2. The proposed representation is formal, structured and verifiable. It allows the output to be automatically verified, thus ensuring it is executable.
Critical weaknesses
  1. The claim that the method is software-agnostic is not fully supported. Although the authors list software-agnostic representation as a core contribution, much of this property can be directly attributed to OTIO itself rather than to the proposed design. The paper does not clearly separate what OTIO already provides from what the proposed representation adds on top of it, so the source of this contribution remains unclear.
  2. The extension to OTIO is not strictly software-agnostic. For effects that are not captured by vanilla OTIO, the method simply transforms the video assets and then references the already-processed assets with OTIO (line 210). This transformation, i.e., the application of effects, must rely on a specific video processing software or toolkit, as it is an integral part of the generation pipeline. As a result, the proposed method is deeply coupled to a specific video editing software, which contradicts the software-agnostic claim.
  3. The NLE problem is modeled as a state (timeline state and asset pool) plus actions that transition the state, but the paper does not explain what is special about this problem and does not propose any design that targets the specific properties of NLE. The formulation appears to be a general state-transition formulation that could be applied to many sequential decision tasks. As a result, the work mainly applies an existing LLM pipeline to a new dataset, and the problem-specific design is limited. This makes the significance of its contribution questionable.
  4. The significance of this contribution is limited. As shown in, e.g., Fig. 2,5,6; Table 3,5-7, the natural language inputs already contain very explicit and specific editing operations and parameter information. The system mainly converts these explicit descriptions into a specific formal language. Combined with the limited software-agnostic property discussed above (W1), it is unclear how much new capability the proposed method provides beyond a format conversion.
  5. The formulation used in the trajectory synthesis process (Appendix A.1) lacks justification. During operation sampling, only feasibility is considered, and the sampling probability does not depend on the current timeline state or on the operations previously executed in the trajectory. However, human editing often uses combinations of operations to jointly achieve a specific goal, and the operations within a single trajectory are often correlated. This suggests a Markovian or higher-order formulation, where P(o)P(o) depends on the current timeline state and previous operations in the trajectory. The paper does not explain why the independent sampling formulation is sufficient or how it affects the realism of the synthesized trajectories.
Questions for authors
  1. (W1) The software-agnostic claim needs clarification. Please address the following questions: Which parts of the proposed representation are inherited from vanilla OTIO, and which parts are newly introduced by this work? What functionality would be missing if the authors used vanilla OTIO directly? How does the paper isolate the benefit of the proposed extension from the benefit of OTIO itself?
  2. (W2) The software-agnostic property is questionable for effects not covered by vanilla OTIO. Please address the following questions: How is the transformation of the video assets at line 210 actually implemented? If the software / backend supporting this transformation is replaced by another one, do the transformed assets and the resulting OTIO representation still hold? If this is a misunderstanding, please clearly explain how these effects and the "materialization" process in line 210 are handled.
  3. (W3) The problem-specific design is unclear. Please address the following questions: What are the unique properties of the NLE problem compared to general state-transition tasks? Which design choices in the method are made specifically for these properties, rather than being generic to any state-action formulation?
  4. (W4) The significance of the proposed representation is unclear. Please address the following questions: Do the natural language instructions already specify all the editing operations and parameters explicitly and precisely? What fraction of the inputs require reasoning beyond mapping explicit descriptions to the formal representation? What does the proposed representation enable that a direct conversion to a formal language would not? Can the authors provide examples where the input is ambiguous or underspecified and show how the method resolves them?
  5. (W5) The justification for the operation sampling formulation in Appendix A.1 is unclear. Please address the following questions: Is the operation sampling probability independent of the current timeline state and the previously executed operations? If so, what is the justification for this independence assumption? Did the authors consider a Markovian or higher-order formulation where P(o)P(o) depends on the current timeline state and previous operations? Can the authors provide experiments or analysis showing how such a formulation affects trajectory realism and downstream performance?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.