acceptodds
Under review as a conference paper at ICLR 2027

SEGUE: Visual Signal as Exclusive Guidance for One-Shot Transition Effect Generation

Abstract

One-Shot Transition Effect Generation (TEG) aims to bridge two video clips using a transition effect provided in a reference clip. However, existing models learn a text shortcut: they rely heavily on prompt descriptions, causing the visual transition signal to vanish. We present SEGUE, a framework designed to read the transition strictly from the visual demonstration via two key mechanisms: i) Visual reference isolation during training, pairing inputs with effect-free neutral prompts and applying a one-way attention mask that keeps the reference independent of the target; and ii) Null-reference guidance at inference, which defines the unconditional path as a dissolve of the reference endpoints—adding no motion or material of its own—to isolate and amplify the visual transition effect. SEGUE flexibly supports endpoints that are video clips, static images, or absent, unifying two-sided TEG and single-sided visual effect transfer within a single model. We also introduce the SEGUE-Benchmark, a cross-paired dataset of 1,692 transition classes and 56,368 training pairs spanning creative presets, parametric procedural transitions, and visual effects. On held-out effects, SEGUE outperforms baselines adapted to TEG and prior visual effect transfer methods even without the transition prompt, serving as the first open-source framework for one-shot TEG.

Primary area:
generative models
Keywords:
visual effect transfer, video generation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_icXVOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces SEGUE for one-shot transition effect generation. Given a reference video showing a transition, the model transfers that effect to a new pair of video clips or images without requiring a text description of the effect. The method uses neutral text prompts and one-way attention during training, followed by null-reference guidance at inference. The authors also build a dataset with 1,692 transition classes and evaluate the method on held-out effects and endpoints. SEGUE clearly outperforms the adapted baselines in both automatic metrics and human evaluation, especially when the effect description is removed from the prompt.

Critical strengths
  1. The task is practical and clearly motivated. Existing methods often depend on a text description of the effect, which cannot fully describe its motion or timing. SEGUE instead learns from the reference video itself. The idea of using a dissolve as the negative reference is simple but appears effective.

  2. The experiments are fairly comprehensive. The paper includes held-out effects and endpoints, comparisons with several related methods, automatic metrics, and a human study. The performance gap is large, particularly with neutral prompts, which supports the claim that SEGUE makes better use of the visual reference.

Critical weaknesses
  1. I found that it is quite hard to tell which parts of SEGUE are actually responsible for its improvement. The method includes neutral-prompt training, one-way attention, and null-reference guidance, but the main ablation mostly focuses on removing the guidance. The “Plain LoRA” baseline in the appendix is not explained clearly enough, and the paper does not separately remove the neutral prompt or the attention mask. The comparison between the dissolve null and an empty null is also only qualitative.

  2. The evaluation of the main two-sided transition task is quite small. It contains only 38 test samples, while most of the evaluation focuses on the single-sided visual-effect-transfer setting. I am also unsure how the proposed transition-fidelity metric is computed for EffectData classes, since the dataset description appears to keep only one reference video per effect while the metric excludes the reference itself. The paper does not report how well this new metric correlates with human judgments at the sample level.

Questions for authors
  1. Could the authors separately ablate neutral-prompt training and the one-way attention mask? At the moment, it is hard to know which training choice contributes most to the result.
  2. What exactly is included in the “Plain LoRA” baseline in Table 6? Does it use effect prompts, full attention, or null-reference guidance?
  3. How is transition fidelity computed for EffectData effects when there is only one reference video for each effect and the reference itself is excluded from the comparison set?
  4. Does the transition-fidelity metric correlate with human preferences on individual samples? This would help establish that the large automatic improvement reflects a real perceptual improvement rather than a property of the metric.
  5. Why is the two-sided TEG evaluation limited to 38 samples? Since this is the main new task introduced by the paper, results on a larger two-sided set would make the overall claim more convincing.
6 comments
AuthorsOpenReview ↗

Response to Reviewer icXV (1/3)

Videos for W2.1 are in the updated supplementary material (open index.html → Rebuttal material, Part 3).

We thank the reviewer for recognizing the practical motivation, the effectiveness of the dissolve negative, and the strong results across automatic metrics and human evaluation, particularly without effect descriptions. We appreciate the detailed suggestions, which have helped us strengthen the ablations, expand the two-sided evaluation and clarify the baselines and metric.

We have provided the full ablation study (W1.1, Q1); clarified plain LoRA (W1.2, Q2); compared the dissolve and empty nulls in numbers (W1.3); expanded our evaluation set and compared its size with related work (W2.1, Q5); EffectData fidelity uses the effect's other clips (W2.2, Q3); and transition fidelity agrees with the raters on individual samples (W2.3, Q4).

Evaluation setting. The new experiments use a class-balanced subset of our evaluation set: 134 samples (all 26 seen, 40 held-out instance, 68 held-out class; tiers as in App. G). Each table states which tier it uses.

Metrics. Transition fidelity (TF) measures whether the output reproduces the reference’s transition effect, scaled so that real videos score at the top. Motion fidelity, reference similarity, and action KL compare the output with the reference. Seam-free rate is the percentage of clips that have smooth connections at the boundaries between the generated transition and the given endpoints (Table 2 and App. C). Reference-endpoint similarity measures copying of scenes and objects from the reference endpoints into the generated output (Table R10).

W1.1 and Q1: Separate ablations of the neutral prompt, one-way attention and NRG

Could the authors separately ablate neutral-prompt training and the one-way attention mask? …

We trained all four combinations on the same data for half the full training schedule:

  • i) Full attention – effect prompt
  • ii) One-way attention – effect prompt
  • iii) Full attention – neutral prompt
  • iv) One-way attention – neutral prompt

We evaluated each on held-out effect classes with both neutral (N) and effect (E) prompts. Table R8 reports Transition Fidelity and the gap between the two prompts (|Δ|), which measures text dependence.

  • Neutral training prompts. These have the largest effect, reducing |Δ| from 37.9 to 16.7 with full attention and from 29.3 to 14.5 with one-way attention.

  • One-way attention. It further reduces text dependence under both training prompts: from 37.9 to 29.3 with effect prompts and from 16.7 to 14.5 with neutral prompts.

  • Combined. The SEGUE recipe achieves the highest neutral-prompt TF (73.9) and the smallest gap (14.5) among the four ablation models. Full training improves TF to 86.9 and reduces the gap to 3.6. NRG further improves these to 91.7 and 1.7, respectively.

This table is also in the revised paper (Table 8, Appendix E).

Table R8. Prompt and attention ablation on held-out class samples (N, E: TF with the neutral / effect prompt; |Δ|: their gap)

AttentionTraining promptStepsN ↑E ↑Δ ↓
Ablation models
FullEffect5k52.190.037.9
FullNeutral5k72.689.316.7
One-wayEffect5k58.988.229.3
One-wayNeutral (SEGUE recipe)5k73.988.414.5
Final models on the same samples
One-wayNeutral (SEGUE w/o NRG)10k86.990.53.6
One-way + NRGNeutral (SEGUE)10k91.793.41.7
AuthorsOpenReview ↗

Response to Reviewer icXV (2/3)

W1.2 and Q2: What Plain LoRA is

What exactly is included in the "Plain LoRA" baseline in Table 6? Does it use effect prompts, full attention, or null-reference guidance?

Plain LoRA (now Table 12) was our first experiment, where we tested and verified text dependence. It fine-tunes the same LTX-2 backbone with LoRA (rank 32, learning rate 2e-4, 5,000 steps) on 139 Higgsfield clips covering 26 effects. It is trained with neutral prompts, and it uses full attention, with no null-reference guidance. We have added this baseline definition to Appendix E (paragraph Plain LoRA, Table 12).

With neutral-prompt training, Plain LoRA becomes less dependent on text, with a gap of only 7.6 on seen samples (Table R9). However, on held-out effect classes, TF drops to 61.9, and the gap grows to 19.6, showing poor generalization and greater reliance on text. This motivated our cross-paired dataset and the other components of SEGUE, which achieve 93.4 TF with a gap of just 1.2 on the same classes.

Table R9 reports Transition Fidelity with neutral (N) and effect (E) prompts, along with the gap between them (|Δ|), across the three tiers of our full evaluation set, both tasks pooled.

Table R9. Plain LoRA vs. SEGUE by tier (N, E: TF with the neutral / effect prompt; |Δ|: their gap)

TierPlain LoRA NEΔSEGUE NEΔ
Seen effects85.292.87.696.196.70.6
Held-out instances72.684.311.787.989.92.0
Held-out effect classes61.981.519.693.494.61.2

W1.3: Dissolve null vs. empty null, in numbers

The comparison between the dissolve null and an empty null is also only qualitative.

We compared both nulls on 68 held-out class samples (Table R10; added as Table 11, App. E). They achieve similar TF, but the dissolve produces smoother transitions near the target endpoints and less copying of scenes and objects from the reference endpoints. This is consistent with the reference-copying failure shown for the empty null in Fig. 13.

Table R10. Negative references for NRG

Negative referenceTF ↑Motion fid. ×100 ↑Ref. sim. ×100 ↑Seam-free % ↑Ref.-endpoint sim. ×100 ↓
NRG with each negative (w = 6)
Dissolve (ours)91.722.096.495.629.1
Empty (reference dropped, Fig. 13)91.622.796.591.231.2

W2.1 and Q5: Size of the two-sided evaluation

Why is the two-sided TEG evaluation limited to 38 samples? …

Two-sided creative transitions with several real videos per class came only from Higgsfield, and most of its held-out classes are single-sided, as are the EffectData effects. We further expanded the two-sided evaluation to all tiers of our full evaluation set (148 creative transition videos over 12 classes) and to 100 videos from 10 held-out procedural shader families (248 two-sided videos in total), and provided the results in Table R11. Supplementary Part 3 shows SEGUE outputs on six of the held-out shader families (3.5–3.16). We also want to clarify that related works use test sets of 48 samples (Video-As-Prompt) and 56 reference videos (refVFX), making our previous evaluation set in a comparable range (Table R12).

Table R11. Expanded two-sided evaluation

SetVideosClassesBase LTX-2VACEWan2.1 FLF2VrefVFXSEGUE
Held-out classes (two-sided set of Table 3 in the paper)76549.8 ± 2.751.9 ± 2.556.6 ± 2.349.8 ± 2.897.0 ± 0.7
Held-out-instance classes64759.7 ± 3.057.2 ± 2.862.6 ± 3.054.2 ± 2.597.4 ± 0.5
Seen classes8258.1 ± 5.950.6 ± 9.466.4 ± 7.438.7 ± 4.697.8 ± 1.0
All two-sided samples1481254.5 ± 2.054.1 ± 1.859.8 ± 1.851.1 ± 1.897.3 ± 0.4
Held-out shader families1001061.3 ± 2.078.7 ± 1.480.0 ± 1.278.0 ± 1.297.8 ± 0.5

Table R12. Evaluation set sizes of related work

MethodTaskTest setHuman study
SEGUE (ours)Reference + both endpoints, or start only38 two-sided + 183 single-sided held-out samples, 2 seeds (442 videos); 384 samples over all tiers29 raters, 197 pairs on 76 samples; VLM judge on 625 pairs
Video-As-PromptReference-based effect48 (24 conditions × 2)20 researchers
refVFXReference-based effect56 reference videos, 28 unseen effectsover 500 annotators
VFXMasterReference-based effectnot stated (15 test + 20 out-of-domain effects)50 participants
AuthorsOpenReview ↗

Response to Reviewer icXV (3/3)

W2.2 and Q3: TF for EffectData effects

How is transition fidelity computed for EffectData effects when there is only one reference video for each effect …?

As the reviewer points out, TF requires comparison with other clips of the same effect. To make this evaluation reliable, we selected held-out EffectData effects from the largest classes, each containing at least 45 clips. The model receives one clip as its reference, while TF compares the output against a fixed set of eight other clips from the same class, excluding the input reference (now stated in App. C). “One reference video per effect” in App. A refers only to the model’s input.

W2.3 and Q4: Sample-level agreement of TF with humans

Does the transition-fidelity metric correlate with human preferences on individual samples? …

Yes. TF achieves the highest agreement with both human preference (86.7%) and VLM judgments (87.3%) among all evaluated metrics (Table R13). This holds for both two-sided and single-sided transitions. For comparison, the VLM judge agrees with human raters 89.9% of the time. These results support TF’s ability to reflect human preferences on individual sample pairs (Table R13; added as Table 5, App. C). Examples where TF and the raters disagree are in Supplementary Part 1 (1.1–1.5).

Table R13. Agreement of each metric with the human preference and the VLM judge

MetricAgreement with human preferenceAgreement with VLM judge
Transition fidelity86.7% (two-sided 89.5%, single-sided 85.3%)87.3% (two-sided 92.7%, single-sided 84.7%)
Reference similarity81.9% (two-sided 80.7%, single-sided 82.5%)85.1% (two-sided 87.3%, single-sided 84.0%)
Motion fidelity70.4% (two-sided 66.7%, single-sided 72.4%)71.3% (two-sided 67.9%, single-sided 73.1%)
Action KL59.3% (two-sided 59.6%, single-sided 59.2%)60.9% (two-sided 65.5%, single-sided 58.8%)

To further evaluate how well TF captures known transition effects, we run retrieval and separation tests on 222 Higgsfield videos covering 39 effect classes. All metrics are used without training or tuning for these tests, and no SEGUE-generated videos are involved. Retrieval accuracy measures how often a video’s closest match shows the same effect. Separation measures how clearly each metric distinguishes same-effect pairs from different-effect pairs, using Cohen’s d. TF achieves the highest retrieval accuracy (73%) and the second-highest separation (1.39), supporting its ability to recognize and distinguish effects across different videos (Table R14; added as Table 6, App. C).

Table R14. Effect retrieval and separation on real Higgsfield videos

MetricRetrieval accuracy ↑Separation (Cohen’s d) ↑
Transition Fidelity73%1.39
Reference similarity64%1.54
Action KL32%0.60
Motion fidelity8%0.43
Reviewer_PNmcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces SEGUE for one-shot transition effect generation. It fine-tunes LTX-2 with cross-paired transition videos, using neutral prompts and one-way reference attention to encourage learning from visual references. At inference, null-reference guidance uses a dissolve transition as the negative reference to enhance the transferred effect. The paper also introduces SEGUE-Benchmark.

Critical strengths
  1. The paper addresses a useful task by transferring a reference transition effect between two given video clips. The unified support for different endpoint types is practical.
  2. The cross-paired dataset is a useful contribution.
Critical weaknesses
  1. Limited novelty. The method mainly fine-tunes a pretrained video model on a transition effect dataset. The in-context conditioning and one-way reference attention follow existing approaches (e.g. VFXMaster). Overall, the methodological novelty is limited.
  2. The paper lacks separate ablations for neutral prompting and one-way attention. It is unclear whether the model can distinguish the intended transition effect from other changes in the reference video, such as subject motion, camera movement, or scene changes.
Questions for authors
  1. Could the authors compare full attention and one-way attention, each trained with neutral prompts and detailed effect prompts? This would help clarify the contribution of each component.

  2. How does the model distinguish transition effects from unrelated subject motion or camera movement? For example, can model transfer the same effect from references with different subject or camera movements without copying those unrelated dynamics?

4 comments
AuthorsOpenReview ↗

Response to Reviewer PNmc (1/2)

Videos for W3/Q2 are in the updated supplementary material (open index.html → Rebuttal material, Part 2, examples 2.1–2.10).

We thank the reviewer for recognizing the usefulness of reference-based transition generation, the practical support for different endpoint types, and the value of our cross-paired dataset. We appreciate the suggestions to clarify our contributions and separately evaluate the training components and the model’s ability to separate effects from unrelated motion.

In short, our contribution is that we are the first to address the task of connecting two video clips by one-shot transition generation, and achieved it through neutral-prompt training and our null-reference guidance formulation (W1). We performed ablation studies and showed the contribution of the components (W2, Q1), and outputs follow the reference's effect without copying its subject or camera motion (W3, Q2).

W1: Novelty

… The in-context conditioning and one-way reference attention follow existing approaches (e.g. VFXMaster). …

We agree that the in-context conditioning layout and one-way attention follow prior work, and we credit VFXMaster for the latter (App. B). However, the contribution of our work is broader: we introduce a new task and develop the data, training approach, guidance and evaluation needed to address it.

  • New task: We transfer a transition from a single visual reference between two given video endpoints. This lets users apply an example transition to their own clips without describing its motion and timing in words. To our knowledge, prior work has not addressed this setting.

  • Text dependence and training: Although the visual reference supplies the intended effect, existing methods rely heavily on text descriptions (Table 3). We identify this shortcut and address it through neutral-prompt training and one-way attention. Controlled ablations show that both help, with the strongest results when combined (Table R7).

  • Task-specific guidance: NRG uses a dissolve of the reference’s own endpoints as the negative, preserving endpoint content while replacing the intended transition. This helps emphasize the effect without requiring condition dropout during training.

  • Dataset: We construct a cross-paired dataset where effects appear across different endpoints and endpoints appear with different effects, encouraging separation of the effect from scene content. Plain LoRA trained on the limited collected examples generalizes poorly to held-out effects, motivating this broader dataset (App. E, Table 12).

  • Benchmark and metric: We evaluate held-out effects and endpoints using automatic metrics, human ratings and a VLM judge. We introduce Transition Fidelity to measure effect-transfer success. Additional validation shows the highest human agreement and effect-retrieval accuracy among the evaluated metrics (App. C, Tables 5–6).

AuthorsOpenReview ↗

Response to Reviewer PNmc (2/2)

W2 and Q1: Separate ablations of the neutral prompt and one-way attention

… Could the authors compare full attention and one-way attention, each trained with neutral prompts and detailed effect prompts?

We trained all four combinations on the same data for half the final training schedule:

  • i) Full attention – effect prompt
  • ii) One-way attention – effect prompt
  • iii) Full attention – neutral prompt
  • iv) One-way attention – neutral prompt

We evaluated each on a class-balanced subset of 68 held-out class samples (App. G) with both neutral (N) and effect (E) prompts. Table R7 reports Transition Fidelity and the gap between the two prompts (|Δ|), which measures text dependence.

  • Neutral training prompts. These have the largest effect, reducing |Δ| from 37.9 to 16.7 with full attention and from 29.3 to 14.5 with one-way attention.

  • One-way attention. It further reduces text dependence under both training prompts: from 37.9 to 29.3 with effect prompts and from 16.7 to 14.5 with neutral prompts.

  • Combined. The SEGUE recipe achieves the highest neutral-prompt TF (73.9) and the smallest gap (14.5) among the four ablation models. Full training improves TF to 86.9 and reduces the gap to 3.6. NRG further improves these to 91.7 and 1.7, respectively.

This table is also in the revised paper (Table 8, Appendix E).

Table R7. Prompt and attention ablation on held-out class samples (N, E: TF with the neutral / effect prompt; |Δ|: their gap)

AttentionTraining promptStepsN ↑E ↑Δ ↓
Ablation models
FullEffect5k52.190.037.9
FullNeutral5k72.689.316.7
One-wayEffect5k58.988.229.3
One-wayNeutral (SEGUE recipe)5k73.988.414.5
Final models on the same samples
One-wayNeutral (SEGUE w/o NRG)10k86.990.53.6
One-way + NRGNeutral (SEGUE)10k91.793.41.7

W3 and Q2: Separating the effect from subject motion, camera motion and scene changes

… can model transfer the same effect from references with different subject or camera movements without copying those unrelated dynamics?

Yes, three parts of our design encourage the model to separate transition effects from unrelated content and motion:

  • Training. Each target effect generation task is paired with up to three references from the same transition class (Table 1). These references share the transition effect but differ in endpoints, objects, and unrelated motion, encouraging the model to learn the shared effect.

  • Cross-paired dataset. Each transition type appears across multiple endpoint pairs, and each endpoint pair appears with multiple transitions (Sec. 3.4, Table 1). This variation makes the separation learnable.

  • NRG. The dissolve null retains the reference’s endpoint subjects and motions. Guiding away from this null helps emphasize the transition effect over those shared dynamics.

Since it is difficult to determine precisely which motion belongs to the transition effect and which is unrelated, we provide qualitative examples in Supplementary Part 2:

  • The same effect transferred from different references of one class (2.1–2.4)

  • Unrelated camera motion in the reference is not transferred (2.5)

  • the same effect transferred differently to match the target endpoint motion (2.6–2.8)

  • the same effect transferred differently to match the target endpoint environment (2.9–2.10).

These examples illustrate transfer of the effect without copying unrelated reference dynamics.

AuthorsOpenReview ↗

We would be happy to clarify any remaining concerns.

Dear Reviewer, we would like to kindly remind you that you can find our explanations regarding novelty, ablation study, and qualitative results in supplementary as detailed in the response above. We would be happy to clarify any remaining concerns.

Thank authors for the detailed responses to my questions. The added ablation experiments clarify the respective contributions of the prompt design and one-way attention. I still have reservations about the technical novelty of this work: while the task setting is different, the method is essentially fine-tuning a pre-trained video model on a transition-effect dataset. Moreover, neutral-prompt training and one-way attention have also been proposed in prior work. Therefore, I keep my score unchanged. But I would not object if other reviewers recommend acceptance.

Reviewer_p5WCOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This work reframes VFX transition generation from "text-described visual effects" to "extracting transition dynamics from a single visual exemplar and reproducing it between two specified endpoints." The problem formulation is crisp. By employing neutral prompts and unidirectional attention to suppress text shortcuts, and constructing precise negative conditioning via NRG (reference endpoint dissolution), the method achieves 97.0 dual-endpoint transition fidelity under neutral prompts (vs. LTX-2 baseline at 49.8). The sensitivity gap between with/without VFX text drops to 0.9 (baseline: 34.9), directly validating the core hypothesis. Strong accept overall, though concerns about synthetic data bias and metric independence remain.

Critical strengths
  1. Testable "Text Shortcut" Hypothesis: Directly comparing VFX prompts vs. neutral prompts under identical references provides solid empirical grounding.
  2. Unified Task Formulation: Elegantly merges dual-endpoint transitions and single-endpoint VFX transfer, treating reference scene leakage and endpoint stitching as independent constraints.
  3. Valuable Dataset & Open-Source Contribution: The released dataset with transition categories, cross-endpoint pairings, and interactive supplementary videos greatly facilitates dynamic quality inspection by the community.
Critical weaknesses
  1. Specificity of Negative Reference: "Endpoint dissolution" inherently involves temporal changes, which may overlap with genuine fade/dissolve transitions. Please break down NRG gains by transition type (wipe, warp, smoke, dissolve, etc.) and compare against static-frame concatenation, temporally shuffled references, no-reference, and alternative negative designs to verify that NRG truly isolates VFX removal.
  2. Domain Shift from Synthetic Data: Of 1,692 effect classes, 1,590 are procedurally generated, potentially biasing the distribution toward parameterizable motions. Despite hold-out tests, please add grouped results for fully real-captured / unknown-tool transitions, diverse semantic endpoint combinations, and large style discrepancies.
  3. Metric Credibility and Independence: "Transition Fidelity" combines specific video features with same-category real samples, which may favor texture/motion similarity over narrative naturalness in real editing. Report human preference per VFX category, VFX transfer success rate, and robust statistical intervals for frame-level endpoint constraints. Crucially, include examples where high scores contradict subjective judgment.
Questions for authors

For fade/dissolve-type effects, does the null reference inadvertently cancel out the target signal?

5 comments
AuthorsOpenReview ↗

Response to Reviewer p5WC (1/4)

Videos for W1.1 and W3.3 are in the updated supplementary material (open index.html → Rebuttal material, Parts 3 and 1).

We thank the reviewer for the positive assessment and for recognizing the empirical support for our text-shortcut hypothesis, the unified task formulation, and the value of our cross-paired dataset. We appreciate the suggestions for further testing NRG, generalization and metric reliability, which have helped strengthen our evaluation.

In short, NRG improves creative and EffectData effects while performance remains high on dissolve-like targets (W1.1, Q1). Among the tested negatives, the dissolve gives the smoothest endpoint boundaries and least copying (W1.2). SEGUE maintains strong performance across sources, endpoint types and style differences (W2). Additional tests support TF’s agreement with human judgments and ability to represent and distinguish effects, while we clarify its limitations and provide disagreement examples and category-level human results (W3).

Metrics. Transition fidelity (TF) measures whether the output reproduces the reference’s transition effect. Motion fidelity, reference similarity, and action KL compare the output with the reference. Seam-free rate is the percentage of clips that have smooth connections at the boundaries between the generated transition and the given endpoints (Table 2 and App. C). Reference-endpoint similarity measures copying of scenes and objects from the reference endpoints into the generated output (Table R2).

W1: Specificity of the negative reference

W1.1: NRG gain by transition type

… Please break down NRG gains by transition type (wipe, warp, smoke, dissolve, etc.)

We compared SEGUE with and without NRG across transition types (Table R1; added as Table 10, App. E). NRG gains are larger for effects that differ more from a dissolve, such as creative Higgsfield and EffectData effects, and smaller for dissolve-like transitions. However, unguided performance (w = 1) is already high for dissolve-like transitions because the base model produces them by default (App. H), so overall performance remains high for these targets. Supplementary Part 3 shows SEGUE outputs on procedural shader transitions: the dissolve and fade (3.1–3.4) and six held-out families (3.5–3.16).

Table R1. NRG gain by transition type

Transition typeSamplesTF, w = 1 (w/o NRG)TF, w = 6 (w/ NRG)NRG gain
Creative (Higgsfield), all tiers
Smoke / gas / fire / explosion5490.695.5+4.9 ± 1.3
Particles and elements (rain, petals, water, earth, air)4884.090.5+6.5 ± 1.4
Geometry / wireframe / point-cloud restyle3876.886.2+9.5 ± 2.4
Camera / motion / portal6285.693.8+8.2 ± 1.8
Appearance and character restyle8083.285.0+1.8 ± 1.3
All creative28284.490.0+5.6 ± 0.7
Out-of-distribution (EffectData), held-out class tier
All EffectData effects10290.495.9+5.4 ± 0.8
Procedural, 10 held-out shader families (added check, not in the main evaluation)
Wipe / warp / zoom5097.196.6−0.5 ± 0.4
Blur dissolve / grayscale fade2098.898.7+0.0 ± 0.4
Glitch / mosaic / burn3098.299.1+0.9 ± 0.6
All procedural10097.897.8+0.0 ± 0.3
AuthorsOpenReview ↗

Response to Reviewer p5WC (2/4)

W1.2: Alternative negative references

… compare against static-frame concatenation, temporally shuffled references, no-reference, and alternative negative designs …

We compared our dissolve negative with an empty reference, shuffled reference frames, and another effect’s reference (Table R2; added as Table 11, App. E). All negatives improve TF over the unguided model (w = 1), indicating stronger presence of the target visual effect. However, TF does not account for temporal smoothness or copying of reference content. Among the guided variants, the dissolve achieves the highest seam-free rate and the lowest reference-endpoint similarity. Seam-free rate measures smoothness near the boundaries with the input clips, capturing flicker and other boundary discontinuities. Reference-endpoint similarity is the DINOv2 similarity between the generated transition frames and the reference's own endpoint frames, averaged over all frame pairs in both directions; lower means less copying. These results support the dissolve as the negative reference that best preserves boundary smoothness and limits copying while strengthening the target effect.

Table R2. Negative references for NRG

Negative referenceTF ↑Motion fid. ×100 ↑Ref. sim. ×100 ↑Seam-free % ↑Ref.-endpoint sim. ×100 ↓
Without NRG
SEGUE without guidance (w = 1)86.919.096.198.524.4
NRG with each negative (w = 6)
Dissolve (ours)91.722.096.495.629.1
Empty (reference dropped, Fig. 13)91.622.796.591.231.2
Shuffled reference frames92.121.296.491.229.4
Another effect's reference90.520.996.488.229.7

W2: Synthetic data and domain shift

… grouped results for fully real-captured / unknown-tool transitions, diverse semantic endpoint combinations, and large style discrepancies.

In the paper, procedural transitions are used only in training; the paper's main evaluation and human study contain none of them (App. A and G). We group the held-out evaluation by source, endpoint kind, and style gap (Table R3; App. G). EffectData is out of distribution for all earlier methods, so we hold it out of training entirely; it is the closest to unknown-tool transitions. TF drops only moderately for real endpoints and large style gaps, and every group scores higher than the best prior method's overall result in the paper. Endpoint kind is where the given clips come from; style gap is the DINOv2 distance between the endpoint clips and the reference clip; the samples are sorted by this distance and split into three equal groups (low, mid, high).

Table R3. SEGUE by source, endpoint kind and style gap

GroupVideosTF (± s.e.)Motion fid. ×100Ref. sim. ×100Seam-free %
Reference Source
Higgsfield (17 effects)23891.2 ± 0.917.196.091.2
EffectData (34 effects)20495.9 ± 0.625.596.997.5
Endpoint Source
Same-class clip13096.1 ± 0.621.696.993.8
Other-class clip15693.7 ± 0.920.696.596.8
Real DAVIS / HumanVid clip15690.8 ± 1.220.995.991.7
Style gap between Reference and Endpoints
Low14896.3 ± 0.722.797.295.9
Mid14693.0 ± 0.821.496.490.4
High14890.7 ± 1.218.995.695.9
All (Table 9, held-out class, w = 6; Table 7 in the submission)44293.4 ± 0.621.096.494.1
AuthorsOpenReview ↗

Response to Reviewer p5WC (3/4)

W3: Metric credibility and independence

W3.1: Does TF reflect what people see?

"Transition Fidelity" … may favor texture/motion similarity over narrative naturalness in real editing.

Transition Fidelity (TF) measures the presence of the target visual effect by matching pixel-level and semantic trajectories. Among the evaluated metrics, it achieves the highest agreement with both human preference (86.7%) and VLM judgments (87.3%) (Table R4). For comparison, the VLM judge agrees with human raters 89.9% of the time. These results support TF’s ability to capture the intended effect, but it does not measure reference copying, smoothness at endpoint boundaries, or overall editing naturalness. It should therefore be considered alongside complementary metrics and human evaluations.

Table R4. Agreement of each metric with the human preference and the VLM judge

MetricAgreement with human preferenceAgreement with VLM judge
Transition fidelity86.7% (two-sided 89.5%, single-sided 85.3%)87.3% (two-sided 92.7%, single-sided 84.7%)
Reference similarity81.9% (two-sided 80.7%, single-sided 82.5%)85.1% (two-sided 87.3%, single-sided 84.0%)
Motion fidelity70.4% (two-sided 66.7%, single-sided 72.4%)71.3% (two-sided 67.9%, single-sided 73.1%)
Action KL59.3% (two-sided 59.6%, single-sided 59.2%)60.9% (two-sided 65.5%, single-sided 58.8%)

To further evaluate how well TF captures known transition effects, we run retrieval and separation tests on 222 Higgsfield videos covering 39 effect classes. All metrics are used without training or tuning for these tests, and no SEGUE-generated videos are involved. Retrieval accuracy measures how often a video’s closest match shows the same effect. Separation measures how clearly each metric distinguishes same-effect pairs from different-effect pairs, using Cohen’s d. TF achieves the highest retrieval accuracy (73%) and the second-highest separation (1.39), supporting its ability to recognize and distinguish effects across different videos (Table R5; added as Table 6, App. C).

Table R5. Effect retrieval and separation on real Higgsfield videos

MetricRetrieval accuracy ↑Separation (Cohen’s d) ↑
Transition Fidelity73%1.39
Reference similarity64%1.54
Action KL32%0.60
Motion fidelity8%0.43

W3.2: Human preference per category, success rate and intervals

Report human preference per VFX category, VFX transfer success rate, and robust statistical intervals …

Human raters prefer SEGUE in every effect category, both for transition fidelity and overall quality (Table R6). A majority of raters identifies the intended effect in every evaluated SEGUE clip (Table 4). We report category-level results with standard errors below, followed by endpoint-preservation and smoothness rates with confidence intervals.

Table R6. Human preference for SEGUE and checks of its clips per category (mean ± s.e.; a tie counts as half a win)

Effect categoryPairsTransition win %Overall win %Seam-free %Endpoint miss %
Smoke, fire and particles2590.0 ± 5.090.0 ± 5.0100 ± 0.00.0 ± 0.0
Geometry / wireframe / point cloud1485.7 ± 9.789.3 ± 7.766.7 ± 21.10.0 ± 0.0
Camera / motion / portal2871.4 ± 7.969.6 ± 7.992.3 ± 7.70.0 ± 0.0
Appearance and character restyle31100 ± 0.096.8 ± 2.2100 ± 0.00.0 ± 0.0
EffectData effects9986.4 ± 2.985.9 ± 3.0100 ± 0.00.0 ± 0.0
Two-sided pairs6286.3 ± 4.084.7 ± 4.196.8 ± 3.20.0 ± 0.0
Single-sided pairs13587.0 ± 2.586.7 ± 2.595.6 ± 3.10.0 ± 0.0
All pairs19786.8 ± 2.186.0 ± 2.196.1 ± 2.20.0 ± 0.0

Frame-level endpoint constraint: SEGUE's clip starts on the given frame, and ends on the given one when two are given, in 100% of the study clips by the raters (95% Wilson interval 95.2 to 100) and is free of a visible cut at the handoff in 96.1% by the seam metric (89.0 to 98.6).

W3.3: Examples where high scores contradict subjective judgment

TF agrees with human preference 86.7% of the time. Supplementary Part 1 provides examples of disagreements, including near-ties and cases involving reference copying or differences in naturalness.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Common Response to AC and Reviewers

We thank the AC and reviewers for their constructive feedback. We summarize the shared concerns first, followed by reviewer-specific clarifications. Detailed results are provided in the individual responses. We have revised the paper with changes marked in blue, and updated the supplementary material with a new “Rebuttal material” tab (Parts 1–3). Each response points to the relevant sections in the revised paper and supplementary material.

Shared concerns

1. Separate contributions of neutral prompts and one-way attention (R2, R3)

We trained all four combinations on the same data. Neutral prompts reduce text dependence most, and one-way attention helps further. Together they achieve the highest neutral-prompt fidelity and smallest gap between the two prompts among the ablation models. Full training and null-reference guidance (NRG) further reduce the gap to 1.7 (R2, Table R7; R3, Table R8).

2. Agreement of transition fidelity (TF) with human judgment (R1, R3)

TF achieves the highest agreement with human preference (86.7%) and VLM judgments (87.3%) among the evaluated metrics. It also achieves the highest effect-retrieval accuracy (73%) on Higgsfield videos, without training or tuning for this test. We clarify its limitations in measuring copying, boundary smoothness and naturalness, and provide disagreement examples in Supplementary Part 1 (R1, Tables R4–R5; R3, Tables R13–R14; App. C, Tables 5–6).

3. Choice of negative reference (R1, R3)

All tested negatives improve effect fidelity. Among guided variants, the dissolve gives the smoothest connections at endpoint boundaries and the least copying of scenes and objects from the reference endpoints (R1, Table R2; R3, Table R10).

Reviewer-specific points

Reviewer 1 (p5WC)

Q1. NRG gains across transition types and dissolve-like targets

NRG gives larger gains on creative and EffectData effects and smaller gains on dissolve-like transitions. The base model already produces dissolve-like transitions by default, so unguided performance is high on them, and overall performance remains high (Table R1; examples in Supplementary Part 3).

Q2. Generalization across sources, endpoints and styles

SEGUE maintains strong performance across sources, endpoint types and style differences, including held-out EffectData effects and real video endpoints (Table R3). An additional evaluation on 220 samples from a fully external dataset with real footage shows strong performance across semantic gaps, style gaps and amounts of endpoint motion, with low text dependence (follow-up, Table R15).

Q3. Human preference by effect category

Human raters prefer SEGUE in every evaluated category, and a majority identifies the intended effect in every evaluated SEGUE clip. We report category-level standard errors and confidence intervals for endpoint preservation and boundary smoothness (Table R6; paper Table 4).

Reviewer 2 (PNmc)

Q1. Novelty and relationship to prior work

We clarify that our contribution spans a new task and the dataset, training approach, guidance and evaluation developed to address it. We acknowledge the established conditioning layout and credit prior work for one-way attention.

Q2. Separating transition effects from unrelated reference motion

We explain how training pairs, the cross-paired dataset and NRG encourage this separation. Supplementary Part 2 provides examples of effects transferred across different subjects, camera motions and endpoint scenes without copying unrelated reference dynamics.

Reviewer 3 (icXV)

Q1. Plain LoRA definition and generalization

We added its definition: the same backbone trained on 139 clips with neutral prompts, full attention and no NRG. It reduces text dependence on seen effects but generalizes poorly to held-out effects, motivating the full SEGUE design (Table R9).

Q2. Expanded two-sided evaluation

We expanded the two-sided evaluation to 248 videos: 148 creative transitions and 100 held-out procedural transitions (Table R11). Our follow-up adds 220 samples from a fully external dataset with real footage, bringing the total to 468 videos. SEGUE leads in every evaluated group while maintaining low text dependence (Table R16).

Q3. TF evaluation on EffectData

Each selected EffectData class contains at least 45 clips. The model receives one reference, while TF compares the output against eight other clips of the same effect, excluding the input reference. We added this clarification to App. C.

Area_Chair_jVv5OpenReview ↗

Please engage into the discussion with authors and fellow reviewers

Dear reviewers,
The authors have already provided their responses. Please check whether the authors' rebuttal has addressed your concerns.
Please engage into the discussion with authors and fellow reviewers.
Thanks!
Best,
AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.