acceptodds
Under review as a conference paper at ICLR 2027

OmniXRef: A Unified Framework for Multimodal Multi-Reference Video Editing

Abstract

Creative intent in video editing can be expressed through text instructions (e.g., "remove the people"), image references (e.g., subject appearance), video references (e.g., camera movement), or a combination of the above. We introduce OmniXRef, a unified framework for multimodal multi-reference video editing that allows users to combine variable numbers of text, image, and video references to control diverse attributes of a source video. We first devise a unified input design with a reference-bank interface and introduce R-RoPE to support generalization to more references. Then our model, OmniXRef-Model, uses a single-stream diffusion transformer to jointly process the source video and multimodal context from an MLLM in an interleaved sequence. To address the scarcity of training data for video-reference editing, we construct OmniXRef-Data using synthesis pipelines for different editing tasks and combinations of references. We use curriculum learning to progressively increase task complexity and resolution. To support systematic evaluation of video as a control modality, we introduce OmniXRef-Bench, a benchmark for video-reference attribute transfer across Camera, Motion, Content, Style, and Visual Effects (VFX). Comprehensive evaluations show that our OmniXRef-Model outperforms task-specific baselines and commercial models.

Primary area:
generative models
Keywords:
Video Editing

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_sef3OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a unified multimodal video editing paradigm, a pipeline for constructing training data, and a benchmark. The paper is easy to follow, and the results appear good.

Critical strengths

A viable, unified multimodal framework is proposed for the popular field of multimodal video editing. Judging by the results on display, the model is good. The paper is easy to understand.

Critical weaknesses

The proposed module lacks in-depth research; it looks like just make the system work but not satisfactory.

Some critical issues are not solved, for example:

  1. Facial appearance changes when the motion transfer.

These issues will detract from practical utility.

Questions for authors
  1. It seems that R2-Rope offers only limited improvement to model performance; why was the number of reserved slots set to 8?
  2. Is there any open-source code or benchmark available?
2 comments
AuthorsOpenReview ↗

Response to Reviewer sef3 (1/2)

We thank the reviewer for finding the framework viable and unified and the paper easy to follow. We address each point below. Tables R0–R5 are in the General Response, and Table and section numbers refer to the revised PDF.

W. Depth of the study of the proposed modules

We respectfully disagree that the modules lack in-depth study. Each design choice is ablated with per-category results under three judges (Table 4 and Appendix Table 14), and each one proves necessary. R²-RoPE replaces sequential asset indexing with reserved reference slots, resolving the conflict in which the same positional index denotes the output in some samples and a reference in others; a sweep from one to five references shows that its benefit grows with the number of references (Table 5 and Appendix Table 15; see Q1). Mix-Patchify uses task-dependent patch sizes as a spatial prior; removing it lowers the Overall score from 2.95 to 1.99 (Table 4) and raises the per-step latency with four references by 3.7× (Appendix Figure 8). The three-stage curriculum, from single-reference adaptation to two-reference composition to high-resolution fine-tuning, lowers the score to 1.83 when replaced by training on the full data at 480P. Finally, our dropout strategy drops only the instruction for classifier-free guidance, which keeps the guidance focused on the requested attribute; dropping the whole reference bank instead lowers the score and raises the leakage rate from 5–8% to 19–23% (Sec. 5.4).

Beyond these ablations, the revision comprehensively extends the evaluation (G2, Table R1): a new multi-reference split with 35 samples at K = 2–5, three frontier VLM judges with three passes each, geometric camera metrics, confidence intervals and significance tests on the average and per category, sensitivity to 8 score aggregation strategies, agreement with human raters, and our recipe applied to the MiniMax-H3 backbone. It also analyzes responsiveness to references, robustness to reference order, and inference cost (Sec. 5.5 and Appendix E).

W. Facial appearance changes in motion transfer

This occurs only in very few cases, which we already reported in the Limitations for transparency (Appendix F, Appendix Figure 9). In these cases, the model transfers the reference's facial expression together with its body pose, since both belong to the motion that the reference demonstrates. Overall, OmniXRef handles Motion robustly: it is the category where its advantage is largest, with an Overall score of 3.68 / 3.86 / 3.60 against 0.92 / 1.33 / 0.90 for Seedance 2.5 under Gemini / Claude / GPT (Appendix Table 12). Separating finer factors within an attribute, such as facial expression from body pose, is a direction we plan to pursue in the future.

AuthorsOpenReview ↗

Response to Reviewer sef3 (2/2)

Q1. Does R²-RoPE offer only limited improvement, and why eight slots?

The improvement is not limited, and it grows with the number of references:

Table R11. Sequential indexing vs. R²-RoPE: Overall (↑) and leakage rate (↓) at K = 1, 3, and 5, mean of the three judges (Table 5; per judge in Appendix Table 15).

K=1K=3K=5
IndexingOverall ↑Leak ↓Overall ↑Leak ↓Overall ↑Leak ↓
Sequential1.740.072.140.480.990.84
R²-RoPE2.950.062.720.231.920.44

R²-RoPE outperforms sequential indexing in both RA and Overall at every K from 1 to 5 (Table 5). Its advantage is clearest in leakage: beyond the two references seen in training, sequential indexing leaks in 48% of the judgments at K = 3 and 84% at K = 5, against 23% and 44% for R²-RoPE. Even at K = 5, the Overall score of R²-RoPE (1.92) remains above that of sequential indexing with a single reference (1.74).

Eight slots is a practical choice rather than a limit of the design (Appendix B.4). It is already above the practical ceiling of current research: reference-guided video editing in the community involves at most a few references per edit (Table R0), and no currently available data provide eight references for one editing sample. R²-RoPE itself places no constraint on the number, so more slots can simply be reserved once such data become available.

Q2. Open-source code and benchmark

We will release the full OmniXRef-Bench and the VLM evaluation protocol, including the judge prompts and scoring code, upon company approval. In the meantime, the updated supplementary material includes visualizations of a subset of 10 benchmark cases, two for each number of references from one to five, and their results.

We hope these revisions address the reviewer's concerns.

Reviewer_jM7KOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents OmniXRef, a unified framework for multimodal multi-reference video editing. The main idea is that OmniXRef can support flexible combinations of text, image, and video references, and can be used for applications including camera motion, object motion, content, style, and visual effects. OmniXRef-Model combines a frozen Qwen3-VL-8B-Instruct encoder with a single-stream video diffusion transformer to jointly process semantic context, source videos, visual references, and noisy target tokens.

Critical strengths
  1. The editing datasets, if open sourced, can be a solid contribution to the video editing community.
  2. The idea of training a generalist multi-reference video editing model is important and timely.
Critical weaknesses
  1. Overall, I think this is a quite solid work but without too much novelty. I think the designs on patch embeddings and RoPE are quite incremental. The frozen Qwen3-VL + diffusion transformer type of model design has also already been adopted by most of the mainstream image or video generation models (e.g. Qwen-Image, MiniMax-H3)
  2. Data collection is a substantial effort in this paper but a large portion of the data (7M text-only editing data) is in-house data and the details of how these samples are constructed is unknown. Other data collection methods are not too different from previous approach like OmniEdit [1] where you collect high-quality data from specialist editing models and use the combined dataset to train the generalist model.
  3. The motivation behind R^2-RoPE is reasonable: randomly assigning reference banks to reserved positional slots can make the model adapt to a wider range of positional indices during training. However, this primarily addresses the distribution shift of positional embedding during training and inference but does not necessarily guarantee generalization to unseen combinations of reference signals.
  4. OmniXRef-Bench contains only 125 evaluation samples across five different editing categories and the results rely on Gemini 3.8 Flash as a VLM judge. Therefore the results may not be very reliable.

[1] Wei, Cong, et al. "Omniedit: Building image editing generalist models through specialist supervision." International Conference on Learning Representations. Vol. 2025. 2025.

Questions for authors

See weaknesses

3 comments
AuthorsOpenReview ↗

Response to Reviewer jM7K (1/3)

We thank the reviewer for finding our work "quite solid" and the idea of a generalist multi-reference video editor "important and timely". Reviewer B9xE likewise found video as a reference modality for editing "genuinely under-formulated". We address each concern below. Tables R0–R5 are in the General Response, and table and section numbers refer to the revised PDF.

W1. Novelty

We agree that a frozen MLLM with a diffusion transformer is a common backbone design, and we do not claim it as a contribution. Our contribution is the systematic formulation of a novel task, multimodal multi-reference video editing, together with the unified reference-bank input interface, two architectural components (R²-RoPE and Mix-Patchify), the data engines, and the benchmark that make it work (G1). Among the methods in Table R0, only OmniXRef edits a source video with any number of text, image, and video references; the commercial systems accept multiple references mainly for generation.

The two architectural components are deliberately lightweight, and the experiments show that each one is necessary. With sequential indexing, the same fourth-axis index denotes the output in some training samples and a reference in others. R²-RoPE removes this conflict; without it, the Overall score drops at every K, and beyond the two references seen in training, the model breaks down, with leakage rising sharply (Table 5). Removing Mix-Patchify lowers the Overall score (Table 4), and Mix-Patchify reduces the per-step latency with four references by 3.7× (Appendix Figure 8). Finally, MiniMax-H3, which the reviewer cites as an example of this backbone design, nearly doubles its Overall score with our recipe compared to its original weights (Table 3), so the gains come from our design rather than from the backbone.

AuthorsOpenReview ↗

Response to Reviewer jM7K (2/3)

W2. Data

Difference from OmniEdit. We respectfully disagree that our data collection is "not too different" from OmniEdit; the two approaches are fundamentally different. OmniEdit distills specialists that already perform each target task well: every training pair is the direct output of one existing image editor, so its data is a union of what existing models can already produce. This route is not available for video-reference editing, because capable specialists do not exist: even on their own categories, the existing experts fall far behind OmniXRef, so their outputs cannot serve as training data.

Table R10. Existing experts vs. OmniXRef on their own categories, Overall score (Gemini; Appendix Table 12).

CategoryExpert modelExpertOmniXRef
MotionMotionFollower0.753.68
Content / Style / VFXRefVFX0.00 / 0.00 / 0.624.10 / 2.59 / 3.83
Camera (CamMC ↓)CamCloneMaster10.654.56

Instead, our core data contribution is a set of data engines, one per category, that compose multiple existing tools, none of which performs reference-guided editing itself, to create source–reference–target triplets that did not exist before.

For example, starting from a target video from a T2V pool, the VFX engine chains:

  1. an MLLM that mines and captions target videos with visual effects;
  2. an image editing model that removes the effect from the target's first frame;
  3. a video propagation model that removes it throughout the video, yielding the source;
  4. an MLLM that describes a different scene with the same effect, rendered into the reference by image and video generation models;
  5. MLLM quality filtering of the resulting source-reference-target triplets.

Likewise, the Camera engine samples a camera path from primitives (orbits, dollies, cranes, zooms, pans, tilts, and freeform moves of varied scale, speed, framing, and rhythm), lifts two visually distinct videos into 4D point clouds, re-renders both along the same path while a video diffusion model completes the regions unseen in each source, and pairs the results crosswise. Similarly, each of the other engines is carefully designed around the attribute it teaches, which is fundamentally different from collecting the outputs of existing editors as in OmniEdit.

The text-only data. The ~7M text-only pairs are standard instruction-based editing data (source video, instruction, target video), which are used to maintain general editing ability, including instruction following and source preservation; they contain no references and are not the source of our reference-editing ability. That ability comes from the ~3M reference examples, which our data engines construct (Sec. 4) and are fully described in the paper.

AuthorsOpenReview ↗

Response to Reviewer jM7K (3/3)

W3. Does R²-RoPE generalize to unseen combinations of references?

R²-RoPE removes the positional mismatch that prevents generalization, and our new multi-reference benchmark explicitly tests the generalization ability of the model: 33 of its 35 samples combine categories never composed in training, and every sample with K ≥ 3 has more references than any training sample. On this multi-reference benchmark, R²-RoPE outperforms sequential indexing in both RA and Overall at every K from 1 to 5 (Table R3). Beyond the two references seen in training, sequential indexing leaks in 48% of the judgments at K = 3 and 84% at K = 5, against 23% and 44% for R²-RoPE (Table 5). Finally, swapping which slot each bank occupies yields consistent edits (Figure 6), meaning that the model binds each reference to its edit rather than to a specific slot.

W4. Benchmark size and a single judge

We comprehensively enhanced the benchmark (G2, Table R1): 160 cases including 35 multi-reference ones, three judges with three passes each, and geometric camera metrics. We also report confidence intervals and significance tests on the average and per category, sensitivity to 8 score aggregation strategies, and a user study with 20 domain experts:

  • All three judges rank the methods identically.
  • Our models' advantage over both commercial models is significant under every judge (Appendix Table 9).
  • In the user study, the experts prefer OmniXRef overall over both Seedance 2.5 (58% vs. 34%) and MiniMax-H3 (73% vs. 19%), consistent with the VLM evaluation (Sec. 5.6, Figure 7).
  • The judges agree with the human raters as often as two raters agree with each other (Table R5).

We will release the full OmniXRef-Bench and the VLM evaluation protocol upon company approval, and the updated supplementary material already includes visualizations and results on 10 benchmark cases (G4).

We hope these revisions address the reviewer's concerns.

Reviewer_B9xEOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper formulates multimodal multi-reference video editing: given a source video, the user supplies K "reference banks", each consisting of a task definition, a free-form edit instruction, and an optional image or video reference; the model must transfer the requested attribute from each reference while leaving everything else in the source untouched. The claimed formulation spans four dimensions — reference number, modality, attribute (Camera / Motion / Content / Style / VFX), and composition.

The paper contributes four things. (i) A unified reference-bank interface that converts text, image, and video references into a common interleaved sequence. (ii) OmniXRef-Model, which encodes that sequence with a frozen Qwen3-VL-8B MLLM and denoises with a single-stream video DiT (described as EditVerse-like), plus two mechanisms: Mix-Patchify, which uses task-dependent patch granularity for reference tokens ((1,1,1) for Content, (1,2,2) for Camera/Motion/Style/VFX) to cut conditioning cost while injecting a spatial prior; and R²-RoPE, which replaces sequential asset indexing on the 4th RoPE axis with a pool of reserved reference slots (8 in practice) randomly assigned during training, so that more references at inference remain in-distribution. Training uses a three-stage curriculum (272P single-reference → 272P two-reference composition → 480P). (iii) OmniXRef-Data, ~10M examples built from synthesis pipelines for the five attribute categories plus compositional pipelines (Content+Camera, Motion+Camera), using Vista4D reshooting, GPT Image 2 first-frame editing, video propagation, Wan2.2-TI2V-5B, and MLLM filtering. (iv) OmniXRef-Bench, 125 single-video-reference cases across five categories, scored by Gemini 3.8 Flash with closed-set labels mapped deterministically to RA / VQ / SP and a composite score 0.45·RA + 0.25·min(VQ,RA) + 0.30·SP, complemented by VGGT-Ω-based camera trajectory errors.

Critical strengths
  1. Timely and well-scoped problem. Video-as-a-control-modality for editing is genuinely under-formulated, and the four-dimensional decomposition (number × modality × attribute × composition) is a clean way to organize a messy space. The reference-bank abstraction is simple enough to be adopted by others and is plausibly the right level of interface for scale.
  2. R²-RoPE is an elegant, well-motivated fix. The diagnosis of the training–inference mismatch in sequential 4D RoPE indexing (the same 4th-axis index meaning "output" with one reference but "reference" with several) is concrete and convincing, and the remedy — reserve slots, assign randomly, so every slot is exercised — is minimal and cheap. The order-robustness analysis (Fig. 5) and the 3-reference composition (Fig. 6) are the right sanity checks for it.
  3. Mix-Patchify is a practical win with measured evidence. Appendix E quantifies a ~3.8× latency / FLOPs reduction at 3 references, which matters a lot in a setting where cost scales linearly with the number of reference videos. The initialization trick (averaging the pretrained fine-grained patchify MLP) is sensible.
Critical weaknesses
  1. W1. The central "multi-reference" claim is only qualitatively validated. OmniXRef-Bench is explicitly a single-video-reference benchmark (Sec. 5.1, App. C.1), so all of Table 2 measures V′V2V with one reference. The ability to compose references — the paper's headline novelty — is supported only by Figure 5 (one example, two banks) and Figure 6 (one example, three video references). There is no quantitative multi-reference evaluation anywhere, despite the fact that the framework is motivated by and named for composition. Compositional training data is also thin: 100K of 10M (1%), covering only Content+Camera and Motion+Camera, with no Style/VFX compositions and no 3-reference training examples. As it stands, two of the four claimed dimensions (number, composition) are exercised only anecdotally.

  2. W2. The VLM-judge protocol is not validated, and the sample is small. All main conclusions rest on Gemini 3.8 Flash over 125 cases (~25 per category), with no human evaluation, no inter-rater agreement, no confidence intervals, and no significance tests. The mapping from labels to scores (5 / 2.5 / 0), the 0.45 / 0.25 / 0.30 weights, and the leakage caps are hand-designed; a sensitivity analysis over alternative weights or a second judge (Qwen3-VL, GPT-4o-class) is absent. This matters because the gaps in Table 2 are large and largely driven by the SP/leakage caps: MiniMax-H3 gets VQ 3.84 yet Score 1.45. Reporting the pre-cap RA/VQ/SP and the Score under two or three alternative weightings would let readers judge how much of the conclusion is protocol versus method. Relatedly, the absolute numbers deserve framing: the best model reaches RA 3.48 (Camera) and 2.86 (Motion) out of 5, i.e. the task is far from solved; the paper should say so more explicitly rather than leading with "outperforms".

  3. W3. The commercial-model comparison is under-specified and confounded. The paper never states how a reference bank is translated into the input format of Seedance 2.0/2.5 and MiniMax-H3: do they receive the structured task definition and the "take nothing else from any reference" system prompt, or a flattened free-form prompt? What resolution, duration, number of samples, and retry policy? Access dates? Since SP and leakage are precisely what the composite penalizes most, and since commercial systems were not given the same anti-leakage instruction that OmniXRef-Model is trained on, a nontrivial part of the reported gap may be a prompting artifact. This is the single most important thing to fix: it is the difference between "our method is better" and "our interface is better specified". Also note that OmniXRef-Model's VQ is lower than all three commercial systems (3.67 vs 3.83 / 3.70 / 3.84), so the abstract's "outperforms … commercial models" should be qualified to "on reference adherence and source preservation".

  4. W4. No open-source multi-reference baselines. ReBind (multi-reference video editing with structured reference relationships), OmniTransfer, LoomVideo, UniVideo, and VINO are cited in related work but never compared against, even on the subset of settings they support. Only three single-task experts (each scored solely in its own category, hence no cross-category aggregate) and three black-box commercial APIs are used. Without at least one open multi-reference or unified baseline, it is hard to attribute the gains to the reference-bank interface and R²-RoPE rather than to backbone capacity and the 10M-example data scale.

Questions for authors
  1. How exactly were reference banks rendered into prompts/inputs for Seedance 2.0/2.5 and MiniMax-H3 (system prompt included? resolution/duration? samples per case? access date)? Please also report pre-cap RA/VQ/SP and the composite under at least two alternative weightings.
  2. Can you add a quantitative multi-reference evaluation (e.g., ~50 cases at K=2 and K=3) and a K-sweep (K=1…5) comparing R²-RoPE with sequential indexing? What is the largest K that works, and how fast does quality decay?
  3. Can you report agreement between the Gemini 3.8 Flash judge and human raters on a subset (say 30 cases), and per-category confidence intervals?
3 comments
AuthorsOpenReview ↗

Response to Reviewer B9xE (1/3)

We thank the reviewer for the detailed and constructive review, and for finding the problem "timely and well-scoped" and both R²-RoPE and Mix-Patchify convincing. Below, we address each concern in detail. The General Response summarizes the revision and contains Tables R0–R5; Table and section numbers refer to the revised PDF.

Q1. How the reference banks are rendered for the commercial models

Appendix C.7 now specifies the full setup. Every method receives the same source and reference clips and the same benchmark instruction, adapted only to its native prompt format.

  • Inputs and prompt. Seedance 2.0/2.5 is queried through its API, with the instruction given verbatim and "the source video" and "the reference video" bound to the two input clips. MiniMax-H3 is run from the released Base Ref2VA weights, with each instruction rewritten into its official six-section prompt format (official GitHub repository).
  • System prompt. Both commercial models receive the same instruction as OmniXRef's system prompt (Appendix A.1), with "take nothing else from any reference" appended to their prompts (Appendix C.7).
  • Resolution and duration. Seedance 2.5: 854×480, 24 fps, 5 s. MiniMax-H3: 864×480, 24 fps, 5 s. Seed 1 for both.
  • Samples per case. One, with no cherry-picking, for every method.
  • Retry policy. A request is retried only when the API rejects it; no output is regenerated or selected based on its quality.
  • Access date. All commercial outputs were regenerated on Nov 6, 2026.
  • Pre-cap RA / VQ / SP and alternative weightings. Leakage caps are removed; RA, VQ, and SP are reported per category and judge (Appendix Table 12), and the Overall score is recomputed under 8 score aggregation strategies (Appendix Table 16). Our models rank first under all of them (Table R7, W2 / Q1 below).

W3. Whether the gap is a prompting artifact

We also give both commercial models the same instruction as OmniXRef, appending "take nothing else from any reference" to their prompts (Q1). The gap remains:

Table R6. OmniXRef vs. Seedance 2.5 on single-reference editing (Gemini / Claude / GPT; Table 3 and Appendix Tables 13, 16).

MetricOmniXRefSeedance 2.5
Leakage rate (↓)0.05 / 0.07 / 0.060.09 / 0.13 / 0.08
RA (↑)3.26 / 3.19 / 3.262.73 / 2.69 / 2.62
Overall, default (↑)3.04 / 2.84 / 2.962.44 / 2.41 / 2.31
Overall, every leaked case scored 0 (↑)2.85 / 2.61 / 2.752.21 / 2.05 / 2.05

The gap is mainly in RA. The appended instruction targets what is taken from the reference, not whether the edit is performed. Even when every leaked case of both methods scores zero, OmniXRef still leads by 0.56–0.70 under every judge.

AuthorsOpenReview ↗

Response to Reviewer B9xE (2/3)

W1 / Q2. Quantitative multi-reference evaluation and K-sweep

The reviewer noted that two of the four dimensions, number and composition, were "exercised only anecdotally". Both are now evaluated quantitatively.

  • Multi-reference split in our benchmark. 35 samples with K = 2, 3, 4, and 5 references (15 / 10 / 5 / 5), where each reference is from a different category. Each edit is judged separately for RA, and SP and leakage are judged jointly across all references (Appendix C.4). Both of our models outperform both commercial models in multi-reference Overall under every judge (Table R2).
  • K-sweep (Q2). Table R3 compares R²-RoPE with sequential indexing from K = 1 to 5. R²-RoPE is better at every K in both RA and Overall, as well as in per-judge comparisons (Appendix Table 15). Beyond the two references seen in training, sequential indexing leaks in 48–84% of the judgments, against 23–44% for R²-RoPE. On the camera edits of this split, R²-RoPE also lowers CamMC from 14.86 to 11.90 (Appendix Table 17).
  • Largest K and decay (Q2). OmniXRef keeps its single-reference RA up to K = 3 (3.30 vs. 3.24), already beyond training. RA then drops at K = 4 and 5 but stays 1.6–1.9× that of sequential indexing. The revised Conclusion and the Limitations (Appendix F) state this degradation explicitly.

W1. Compositional training data

Our data strategy prioritizes quality over quantity (Table 2). The 3M single-reference examples teach diverse reference-based editing across all five categories, and about 7M text-only examples maintain general video editing ability. On this foundation, 100K high-quality compositional examples, covering two compositions with at most two references and taking 40% of the sampling weight in Stages 2 and 3 of the curriculum (Appendix Table 7), suffice for the model to generalize to multi-reference editing. The results on the multi-reference benchmark split confirm this: 33 of the 35 samples combine categories never composed in training, including Style and VFX, every sample with K ≥ 3 has more references than any training sample, and both of our models still outperform both commercial models (Table R2).

W4. Open-source multi-reference baselines

We searched for an open baseline, but no method is able to accept the same input as ours (Table R0). Most listed methods accept only text and image references or do not edit a source video, so they cannot run on OmniXRef-Bench. Also OmniTransfer and ReBind are not open-sourced.

Instead, we show that the gains come from our design and training recipe rather than from our backbone. Applied to a different backbone, MiniMax-H3, the same recipe roughly doubles its Overall score under every judge compared to its original weights and brings it close to OmniXRef (Table 3 and Appendix Table 9, see also G3, finding 2). At a fixed backbone and data, removing R²-RoPE, Mix-Patchify, or curriculum learning each lowers the Overall score substantially (Table 4).

AuthorsOpenReview ↗

Response to Reviewer B9xE (3/3)

W2 / Q1. The scoring protocol

The protocol was rebuilt. All labels are now binary, and fixed, deterministic rules turn them into scores (Appendix C.2). Leakage is no longer a cap and is reported as a separate rate. The gaps are not driven by the protocol. Pre-cap RA, VQ, and SP are reported per category and judge in Appendix Table 12, and Appendix Table 16 recomputes the Overall score under 8 score aggregation strategies:

Table R7. Single-reference Overall score (Gemini / Claude / GPT) under alternative score aggregation strategies (Appendix Table 16).

Aggregation strategyOmniXRefSeedance 2.5MiniMax-H3
Default (0.7 / 0.1 / 0.2)3.04 / 2.84 / 2.962.44 / 2.41 / 2.311.46 / 1.31 / 1.31
Equal weights2.64 / 2.34 / 2.682.10 / 2.10 / 2.081.28 / 1.07 / 1.19
RA only3.26 / 3.19 / 3.262.73 / 2.69 / 2.621.58 / 1.51 / 1.45
No cap at RA3.45 / 3.21 / 3.402.95 / 2.92 / 2.822.35 / 2.21 / 2.29

OmniXRef outperforms both commercial models in every row, including RA alone and without any cap. Across all 18 capped settings, our models rank first in every one. The case the reviewer raised, MiniMax-H3 with high VQ but a low score, is because it mostly returns the source almost unedited, which keeps VQ and SP high, and it stays last even without the cap.

W2 / Q3. Judge validation, human agreement, per-category CIs

As suggested, we added two more frontier judges, Claude Opus 5.5 and GPT-6.1-sol. All three rank the methods identically, and any two assign the same label in 84–86% of the judgments (Appendix C.6).

Agreement with human raters (Q3). We use all trials of our user study: 600 pairwise trials, more than the 30 cases suggested.

Table R8. Agreement with the human raters (%), among trials without ties (Appendix Table 11).

CriterionHuman–humanHuman-GeminiHuman-ClaudeHuman-GPT
Overall preference78787882
RA (primary)79898385
SP71625251
VQ77555176

On overall preference and RA, the judges agree with the raters as often as, or more often than, two raters agree with each other. Agreement is lower on SP and VQ: their binary labels often give two outputs the same score, whereas the raters also state preferences between two acceptable outputs. Both criteria carry low weight and count only up to RA.

Per-category confidence intervals (Q3). Appendix Table 10 reports every category's Overall score with 95% bootstrap CIs, and paired per-category differences with permutation tests. Against MiniMax-H3, both of our models are significantly better on Camera, Motion, and VFX under every judge. Against Seedance 2.5:

Table R9. OmniXRef − Seedance 2.5: Overall difference on Camera and Motion (p-value; full results in Appendix Table 10).

CategoryGeminiClaudeGPT
Camera+0.28 (0.305)+0.69 (0.015)+1.83 (<0.001)
Motion+2.76 (<0.001)+2.53 (<0.001)+2.68 (<0.001)

The advantage over Seedance 2.5 is significant on the dynamic attributes, Camera and Motion, which are precisely what video references enable.

Rather than relying on point estimates as in the original submission, we now quantify their uncertainty: averaged over all categories, OmniXRef’s lead over both commercial models is significant under every judge (Appendix Table 9).

W2. Absolute numbers

We agree that the task is far from solved, and the revised Conclusion now states this explicitly. The clearest evidence is the trend with the number of references: as it grows beyond three, reference adherence declines and leakage increases (Table R3). This leaves ample room for future work, and OmniXRef-Bench allows systematically measuring progress in this domain.

W3. Visual quality and the claim in the abstract

We agree that OmniXRef's VQ is below the commercial models, and we revised the claims in the abstract, introduction, and conclusion to "superior overall performance, especially in reference adherence". OmniXRef (MiniMax-H3), trained with the same recipe on the MiniMax-H3 backbone, closes most of this gap. Visual quality is therefore largely bounded by the backbone rather than by our recipe.

We hope these revisions address the reviewer's concerns.

Comments on the paper

4 comments
AuthorsOpenReview ↗

General Response (1/3): Summary of Revisions

We thank all reviewers for their insightful reviews. The reviewers found the problem "timely and well-scoped" and "genuinely under-formulated" (B9xE), and the idea of a generalist multi-reference video editor "important and timely" (jM7K). They described the framework as "viable" and "unified" and the paper as "easy to follow" (sef3), R²-RoPE as "an elegant, well-motivated fix" and Mix-Patchify as "a practical win with measured evidence" (B9xE), and the data collection as "a substantial effort" (jM7K).

Shared concerns. This general response addresses the shared concerns raised by several reviewers:

  • (i) Novelty of the contributions (jM7K W1; sef3 W). Addressed in G1.
  • (ii) Multi-reference evaluation. The multi-reference claim was supported only qualitatively (B9xE W1; jM7K W3). Addressed in G2 and G3.
  • (iii) Reliability of the evaluation. The evaluation relied on 125 cases and a single VLM judge, without human evaluation or confidence intervals (B9xE W2, Q3; jM7K W4). Addressed in G2 and G3.

We have uploaded a revised PDF in which every change is shown in blue. The revision adds a multi-reference benchmark to OmniXRef-Bench, rebuilds the evaluation protocol from the ground up, and re-evaluates every result under it. The summary is:

  • A new multi-reference benchmark. 35 samples with two to five references, a per-reference evaluation protocol, and a sweep from one to five references comparing R²-RoPE with sequential indexing (Sec. 5.1, 5.4; Appendix C.4; Tables 3, 5 and Appendix Tables 13, 15; Video results in supplementary material).
  • A rebuilt evaluation protocol. Three VLM judges with three passes each, describe-then-label binary scoring with fixed rules, and all judge prompts listed verbatim (Appendix C.1–C.4, G; Appendix Table 8). Every result is re-evaluated under it and reported per judge, including the main results, per-category results, and all ablations, for every method: ours, the latest Seedance 2.5 and MiniMax-H3 with regenerated outputs and documented inference settings, and the task-specific experts (Appendix C.7, D.1.1; Tables 3, 4 and Appendix Tables 12, 14).
  • A reliability analysis of the protocol. Stability across passes, agreement between judges, bootstrap CIs and permutation tests on the average and per category, sensitivity to 8 score aggregation strategies, and agreement with human raters (Appendix C.6, D.1.4; Appendix Tables 9, 10, 11, 16).
  • A user study with 20 domain experts comparing OmniXRef with both commercial models (Sec. 5.6; Figure 7).
  • Geometric camera evaluation extended to the multi-reference camera edits and validated under three pose estimators (Appendix C.5, D.2; Appendix Tables 17, 18).
  • Our recipe on a new backbone. We apply our recipe to the MiniMax-H3 backbone to obtain OmniXRef (MiniMax-H3). It roughly doubles the single-reference Overall score of the original MiniMax-H3 and achieves the best multi-reference results, demonstrating the effectiveness of our recipe beyond a single backbone (Sec. 5.1, 5.2; Table 3 and Appendix Table 9).
  • Sharpened positioning and claims. A capability comparison with prior methods (Table 1), revised claims in the abstract, introduction, related work, and conclusion, and the rationale for the number of R²-RoPE slots (Appendix B.4).
AuthorsOpenReview ↗

General Response (2/3): Novelty and Benchmark (G1, G2)

G1. Novelty: a unified framework that systematically formulates multimodal multi-reference video editing (jM7K W1; B9xE W4)

The gap. Reference-guided video editing has mostly been studied in settings of a text instruction with an optional image reference, or multiple image references. Frontier systems increasingly support broader combinations of multimodal references, but many support them only for generation, not editing. As Table R0 shows, a systematic formulation of multimodal multi-reference video editing remains largely underdeveloped.

Table R0. Input-interface capabilities. Editing: edits an input video rather than generating a new one. Text-/Image-/Video-Ref.: accepts references of that modality. Multi-Ref.: accepts more than one reference. ◐: multiple references are accepted, but designed and evaluated mainly for generation rather than editing.

MethodEditingText-Ref.Image-Ref.Video-Ref.Multi-Ref.
EditVerse✓✓✓✗✗
LoomVideo✓✓✓✗◐
VINO✓✓✓✗◐
UniVideo✓✓✓✗◐
OmniTransfer✗✓✓✓✗
ReBind✓✓✓✗✓
Seedance 2.5✓✓✓✓◐
Gemini Omni Flash✓✓✓✓◐
Kling 4.0✓✓✓✓◐
MiniMax-H3✓✓✓✓◐
OmniXRef (ours)✓✓✓✓✓

Only OmniXRef supports every column. Most editing methods accept only text and image references. OmniTransfer accepts a video reference, but it generates new videos rather than editing a source video, and it takes a single reference. The commercial systems Seedance 2.5 and MiniMax-H3 accept video references, but their multi-reference input is designed mainly for generation.

Our contribution. To our knowledge, OmniXRef is the first work to systematically establish video as a reference modality for universal video editing, and to make video editing flexibly conditioned on multiple multimodal references, transferring any attributes from a text, image, or video reference. We study this novel task end to end, covering formulation, model design, data, benchmark, and analysis.

G2. A comprehensively enhanced benchmark (B9xE W1, W2, Q1, Q3; jM7K W4)

We rebuilt our benchmark around the two questions raised by the reviewers: can multi-reference editing ability be validated quantitatively, and is the evaluation reliable?

Table R1. Original submission vs. revision.

AspectOriginal submissionRevised (where in the PDF)
Cases125 single-reference160: 125 single-reference and 35 multi-reference with K = 2/3/4/5 (15/10/5/5). 33 of the 35 combine categories never composed in training (Sec. 5.1, Appendix C.4)
JudgesGemini 3.8 Flash, single runThree judges (Gemini 3.8 Flash, Claude Opus 5.5, GPT-6.1-sol), three runs each (Appendix C.1)
ScoringGraded labels (5/2.5/0), hand-set weights and leakage capsDescribe-then-label: binary labels. VQ and SP capped at RA; leakage reported separately (Appendix C.2, G)
CameraVGGT-ΩThree pose estimators (Appendix Tables 17, 18)
ReliabilityNonePass stability, agreement between judges, bootstrap CIs (average and per category), permutation tests, 8 score aggregation strategies (Appendix Tables 9, 10, 16)
HumansNoneUser study with 20 experts (Sec. 5.6) and VLM judge–human agreement reported (Appendix Table 11)

Table R2. Main results: Overall score (0–5) under Gemini / Claude / GPT (Table 3).

ModelSingle-reference (K=1)Multi-reference (K≥2)
OmniXRef3.04 / 2.84 / 2.962.49 / 2.39 / 2.65
OmniXRef (MiniMax-H3)2.74 / 2.64 / 2.903.24 / 3.29 / 3.84
Seedance 2.52.44 / 2.41 / 2.312.41 / 2.37 / 2.52
MiniMax-H31.46 / 1.31 / 1.311.75 / 1.77 / 1.96

Both of our models outperform both commercial models in every column.

AuthorsOpenReview ↗

General Response (3/3): New Analyses and Open Source (G3, G4)

G3. New analyses and findings

1. Generalization beyond the training distribution, enabled by R²-RoPE (B9xE W1, Q2; jM7K W1, W3; sef3 Q1). Every training sample has at most two visual references, yet OmniXRef handles up to five, and 33 of the 35 multi-reference samples combine categories never composed in training. With sequential indexing instead of R²-RoPE, the reference adherence drops and leakage increases significantly beyond two references (Table 5 and Appendix Table 15):

Table R3. RA (↑) and leakage rate (Leak, ↓), averaged over the three judges.

K=1K=2K=3K=4K=5
RA, sequential1.971.962.701.111.27
RA, R²-RoPE3.243.383.301.752.44
Leak, sequential0.070.100.480.600.84
Leak, R²-RoPE0.060.110.230.270.44

Video results are in the updated supplementary material.

2. Our recipe is effective and general (B9xE W4; jM7K W1). To separate our recipe from the backbone, we also apply our recipe to MiniMax-H3. On both backbones, the resulting models outperform both commercial models in every column of Table R2. On MiniMax-H3, the recipe roughly doubles the single-reference Overall score compared to the original MiniMax-H3 weights.

3. Camera reference evaluation is complemented by geometric accuracy metrics of the camera trajectory (jM7K W4; B9xE W2). The submission already reported VGGT-Ω trajectory errors. The revision extends them to multi-reference edits and checks them with three pose estimators, which rank the methods identically (Appendix Tables 17, 18):

Table R4. Camera trajectory errors (↓), poses from VGGT-Ω (Appendix Table 17).

ModelRotErrTransErrCamMCCamMC, multi-ref.
OmniXRef0.924.054.5611.90
OmniXRef (MiniMax-H3)1.576.257.119.23
CamCloneMaster2.119.6610.65–
Seedance 2.54.1421.0722.4022.60
MiniMax-H34.2926.5827.8520.91

4. The conclusions still hold on a more comprehensive evaluation protocol (B9xE W2, Q1; jM7K W4). The quantitative results hold across three judges (identical ranking), across 8 score aggregation strategies (Appendix Table 16), and against human raters (Table R5).

Table R5. Agreement with the human raters (%), among trials without ties (Appendix Table 11).

CriterionHuman–HumanHuman-GeminiHuman-ClaudeHuman-GPT
Overall preference78787882
RA79898385

5. Leakage increases significantly without our reference bank design. Dropping the whole reference bank for classifier-free guidance raises the leakage rate from 5–8% to 19–23% (Sec. 5.4).

G4. Open source (sef3 Q2)

We will release the full OmniXRef-Bench and the VLM evaluation protocol, including the judge prompts and scoring code, upon company approval. The updated supplementary material includes visualizations of a subset of 10 benchmark samples, two for each number of references from one to five, and their results.

Dear Reviewers B9xE, jM7K, and sef3,

Thank you again for your time and constructive feedback. As the discussion period is progressing, we would like to gently remind you that we have posted our responses to each review, along with a General Response, a revised PDF (revisions highlighted in blue), and updated supplementary material.

We would greatly appreciate it if you could take a look and let us know whether our responses address your concerns. If you have any additional questions, please let us know, and we are happy to provide further clarification during the remaining discussion period.

Best regards,

Submission2098 Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.