acceptodds
Under review as a conference paper at ICLR 2027

TunerDiT: Training-free Progressive Steering of Diffusion Transformer for Multi-Event Video Generation

Abstract

Text-to-video (T2V) generation faces challenging questions when generating videos with long horizons containing multiple events. Inspired by the intrinsics of the diffusion process, we probe video diffusion transformers (DiTs) and uncover intrinsic turning points in the DiT denoising trajectory where conditioning text affects generation from global layout to fine-grained details. Building on this finding, we present TunerDiT, a simple yet effective progressive steering method that requires no additional training for multi-event generation. TunerDiT comprises two steering handles: (1) Event-Partitioned Masking that enforces event boundaries while allowing cross-event transition bands; (2) Cross-Event Prompt Fusion that injects neighboring event semantics for late-stage refinement. We contribute a self-curated prompt suite for benchmarking multi-event generation, i.e., MEve. TunerDiT achieves the highest text alignment while maintaining visual consistency, and offers a tunable trade-off between video consistency and event separation, compared with other training-free methods. Besides, TunerDiT is model-agnostic, and the performance is consistent across different model architectures.

Primary area:
generative models
Keywords:
Multi-Event Video Generation, Diffusion Transformer, Controllable Video Generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ga7bOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper targets at training-free multi-event text-to-video generation. To achieve this, it designs the following two interventions on the top of pretrained video DiT:

  • Cross-event prompt fusion: it conditions every temporal segment on the first event's prompt for the first couple of steps. After that, each segment gets its own prompt.
  • Event-partitioned mask: it restricts video self-attn and video-text cross-attn to event-matched blocks. At event boundaries it adds "transition bands" whose width shrinks linearly to zero.

These two gates and the band ratio are per-backbone hyperparameters.

To verify, the authors implement TunerDiT on Open-Sora 1.2/2.0 and Wan2.1/2.2 and compared with MEVG, DiTCtrl, FreeNoise and SwitchCraft. The authors show that TunerDiT adds 11.5% relative text-alignment over the base models on average.

Critical strengths

[S1]

It is a cheap & training-free method and the authors show the gains can be transferred into different backbones. TunerDiT is purely an inference-time schedule: a prompt swap plus attention masks. If the evidence holds up, this is a drop-in improvement for current open models.

[S2]

I like the design of the gates (i.e., two tau hyperparameters) and the band ratio. It can serve as explicit and characterized control knobs. The authors provide several supporting analysis (e.g., sensitivity curves, component ablations) and also discuss failure modes at both extremes. This makes the trade-off between text alignment and consistency something a user can tune, rather than an accident of the backbone.

Critical weaknesses

[W1]

My major concern is that two of the three listed contributions are mostly from the cited prior work [1]. Specifically:

  • A prompt suite called MEve, with the same five categories, the same prompt counts (two events per prompt), and the same three sources.
  • The same P1 => P2 switching sweep over x in [0, 0.1, …, 1.0] with the finding that the turning point falls within roughly the first 30% of steps, followed by a plateau.

The probing paragraph (Sec 3.2) and the benchmark description (App. A) closely mirror that paper's text. The real additions seem to be four-event prompts, Wan backbones, and TunerDiT itself.

Moreover, the coarse-to-fine premise is established in T2I, and the mechanisms resemble existing event-wise masking (Mask2DiT) and attention control (DiTCtrl, SwitchCraft). So the novelty rests on the specific PF+EM schedule, which raises the bar for the empirical evidence.

[1] Liao et al., "When and Where do Events Switch in Multi-Event Video Generation?", ICCV 2025

[W2]

Several empirical evidences are not reliable as reported. For example:

  • In Algorithm 1, you pick the gates by maximizing TA, the turning point is located on MEVE, and no held-out split is described. The per-backbone settings in Table 3 therefore appear tuned on the evaluation benchmark, while baselines run at default settings.
  • Some numbers also look contradictory. E.g., TVA is defined as a Sentence-BERT cosine similarity, yet values range from 1.0 to 2.25. The human study has 18 participants in Section 5.1 but 14 in Appendix.
  • Some comparisons are confounded. Specifically, methods differ in backbone, resolution, length and frame rate. MEVG and FreeNoise run at 256x256 with visible stock-footage watermarks; TunerDiT-Wan2.1 runs at 832×480 with 81 frames. So "highest TA" and the human-preference wins largely reflect backbone quality.
Questions for authors

[Q1]

I am wondering what exactly is new in MEVE and section 3 relative to [1]? Please revise the contribution list and discuss that work explicitly.

[Q2]

On which prompts were two tau and r selected? You should report results with settings chosen on a disjoint split, or with one shared setting across backbones. Did any baseline receive comparable tuning?

[Q3]

I am also not sure how is EM implemented on Wan and on Open-Sora 1.2? Is video self-attention fully block-diagonal from tau onward? If so, different events never attend to each other for 70–90% of the steps. Given that prompts are zero-padded, which text tokens fall inside a band, and could a band contain only padding?

[Q4]

I notice that PF conditions every segment on P1 early, and section 3.2 shows that even 10% of steps on P1 imprints its layout. Does this suppress later events? Could you report TA by event index, and separately for scene-change prompts like the bird example in Fig. 2.

[Q5]

Also, I found TA rises with event count for nearly every method, including the base models. But, TA does not grow (Wan2.2: +0.016/+0.007/+0.010). What supports the "boundaries as anchors" explanation in section 5.5?

1 comment
AuthorsOpenReview ↗

Response to Reviewer ga7b (1/2)

We thank the reviewer for the careful reading and for appreciating the gates as explicit control knobs.

W1 / Q1. [1] is a non-archival (No Proceedings track) ICCV 2025 workshop paper, presented for work-in-progress discussion as stated in the preprint. It is an early exploration of diagnosing multi-event generation and an early attempt to identify potentially influential topics, including both the position (“where”) and the timing (“when”) of the video diffusion model with a P1→P2 prompt-switch sweep.

Relative to [1], this submission adds:

  1. Extending multi-event evaluation to three- and four-event MEve settings
  2. Systematic probing that determines that timing in the video diffusion process matters more than the models’ internal blocks.
  3. The simple but effective method, TunerDiT, a progressive inference-time schedule (PF + independently gated EM + shrinking transition bands).
  4. Further discussion of the natural trade-off among metrics in multi-event settings, to raise attention to it in the field.

We will revise the contribution list to attribute the benchmark and the probe to [1] and revise §3.2 and App. A. What distinguishes TunerDiT is the determination of the working factor proposed by [1], the fact that timing matters more than position, and a simple and effective scheduling steering method for longer event settings. Fig. 6 shows that each piece matters: EM drives alignment, and adding PF recovers part of the consistency that EM costs, and discusses its trade-offs.

W2 / Q2: hyperparameter selection. The gates were selected by the TA on MEve in general, and no held-out split was described; we agree this is a weakness of the protocol. Although in our inference-time method the turning point is a tunable value that applies to the same base model, three facts limit the risk of overfitting:

  1. the PF gate is 0.1 on all four backbones; only the EM gate and the band ratio vary per backbone;
  2. selection uses aggregate TA over a coarse grid with 0.1*N interval (N is diffusion steps), not per-prompt tuning. This further shows that the turning point generalizes: one value per model is effective for individual prompts.
  3. In Fig. 6b, TA stays within about 0.002 of its peak for EM gates from 0 to 0.6, so the result does not hinge on the exact tuned value.

W2: TVA. We sincerely apologize: the Sentence-BERT definition is an unfixed leftover from an earlier, failed version that could not distinguish among complex event descriptions. The final TVA uses a VLM as a judge (Gemini): the generated video's frames are first described chronologically, and the judge then compares this description with the input prompt and outputs an integer score from 1 (differs a lot) to 5 (same meaning). TVA is the mean score, hence the 1.0–2.25 range. We will correct §5.1 and give both judge prompts in the appendix.

W2: human study. Sorry, it is a typo; we recruited 18 people in total, with 14 valid responses, and will make §5.1, Table 1b, and App. C consistent.

Reviewer_TEVwOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

TunderDiT proposes a method for training-free multi-event video generation. To enable this, the paper proposes Event-Partitioned Masking and Cross-Event Prompt Fusion which manually adjust the attention and the prompt usage in the test time.

Critical strengths
  1. The method requires no additional training and improves alignment across 4 backbones.
  2. It achieves large gains in human preference study.
Critical weaknesses
  1. The proposed inference technique shows limited usage of multi-event generation, especially within the original clip duration. The easiest, scalable way to make this happen might be fine-tuning or post-training the model on multi-event data. The real problem we have to solve may be long-horizon video generation with multiple events, especially when it generates a way longer video sequence than the training. If every frames are able to attend to each other, it is very easy to fine-tune the model with good data rather than proposing complex training-free methods. I believe the proposed method introduces too much complexity considering its gain.
  2. The inspiration—diffusion models generate an object in a coarse-to-fine manner—is the inherent, well-known dynamics of the diffusion models (not only applied on T2I, but also T2V, T2A, I2V, …). Applying this to T2V inspired from T2I is not new. I felt like the validation in section 3 provides a good empirical explanation for an well-known knowledge.
  3. From my understanding, the main results shown in Table 1 (a) shows that it achieves marginal improvements on TA and TIS (compared to the training-free baselines like MEVG or DiTCtrl) but occurs some degradation on consistency. For example, Wan2.2 + TunerDiT shows almost similar scores compared to MEVG. Furthermore, I believe it would be great to report the scores of training-based methods, to see how training-free methods stand compared to training-based ones.
  4. According to App B, each event is assigned the same number of latent frames, which doesn’t make a lot of sense. We could easily think the events that should have diverse duration. For example, event A for the first 1 seconds and B for the last 15 seconds. I think it could be adjustable in a manual way but it introduces a new hyperparameter.
Questions for authors
  1. I think the intrinsic turning point should be determined sample-by-sample, not by model-wise. For example, the events with negligible movement (singing and talking) has much higher turning points than the ones with the large movements. Otherwise, I think the authors could report that the standard deviation of the points was sufficiently small within a single model so we don’t have to care about it.
  2. Could you please report the latency breakdown, especially compared against the base model and the baseline, before and after applying this method? I think we have to see if the proposed method prevent us to use flash attention so that make the model too slow.
2 comments
AuthorsOpenReview ↗

Response to Reviewer TEVw (1/2)

We thank the reviewer for the detailed and constructive review.

W1. We agree that (1) post-training on multi-event data is a strong approach, and that (2) generating far beyond native durations is an important problem. However, it’s completely outside the domain of improving training-free video generation. TunerDiT focuses on identifying basic findings that hold even in training-free settings under limited resources. We regret to disagree with the reviewer’s criticism that the proposed technique shows limited use of multi-event generation:

  • TunerDiT needs no training and runs on a single H100 GPU, whereas training-based alternatives are costly: Mind the Time [2] fine-tunes the full model at batch size 512 for 15k steps, and LongLive [3] needs 32 GPU-days of fine-tuning.
  • The base models like Wan have already shown limited multi-event capabilities, as shown in the failure cases in Fig. 2. But event merging, wrong order, multi-event expressiveness are still limited even though there are complex prompt settings in the pretraining data of Wan.
  • TunerDiT shows that considering the early dominance nature and adopting purely scheduling could potentially inspire future works, even massive training methods, to train models more efficiently by focusing on the early steps, transition bands in attention, etc. Besides, multi-event data is rare, private, and expensive.

W2. Agreed. §3.2 applies an existing prompt-switch probe [1] to new backbones (Wan) to select the steering region, and we could present this as an empirical analysis. However, this diffusion dynamic is intuitive. Our method digs much deeper to analyze the dynamics of video generation and leverages this insight to the best through the proposed schedule, comprising simple yet effective prompt fusion and event masking in attention. TunerDiT provides a thorough analysis of the findings and makes the best use of this insight.

References [1] Liao et al., "When and Where do Events Switch in Multi-Event Video Generation?", ICCV 2025 Workshop (non-archival), arXiv:2510.03049. [2] Wu et al., "Mind the Time: Temporally-Controlled Multi-Event Video Generation", CVPR 2025. [3] Yang et al., "LongLive: Real-time Interactive Long Video Generation", ICLR 2026.

AuthorsOpenReview ↗

Response to Reviewer TEVw (2/2)

W3. We agree that the claims should reflect effect sizes and the consistency trade-off, and we will revise them accordingly.

Within-backbone evidence (the controlled comparison). As discussed in Sec. 5.3 and Appendix D, event isolation and consistency form a metric paradox: by nature, they are opposing metrics in multi-event settings and require a trade-off. Considering computation error, the reviewer compared MEVG and DiTCtrl only on TA and TIS, while we provide complementary analysis with 8 metrics across (1) automatic embedding-based evaluation, (2) VLM-as-judge evaluation, and (3) human studies, where TunerDiT surpasses MEVG and DiTCtrl by a large margin, especially in human review. This is because, while developing the method, we noticed in real inference results and quality checks that DiTCtrl targets background changes rather than event changes with subject motion, and MEVG is over-optimized for event isolation: within each event its videos are nearly static, hence the high TA and TIS scores. Both methods show reward hacking in actual inference. Hence, we report results across various types of evaluation, both subjective and objective. The videos in the supplementary results make this easier to see.

Compared to training-based methods. We already compare with a training-based method, Mask²DiT (CVPR'25), in App. G. It shows the same reward hacking: videos within an event are nearly static, which inflates the metrics. Few prior training-based methods target multi-event generation, and most cannot be run because they are closed-source, release no weights, or demand heavy GPU resources. We therefore extend the benchmark MEve for training-free evaluation targeting real visualization.

  • Mind the Time [2] (CVPR'25), the most direct trained method, releases no weights, base model, or training data, so a matched quantitative comparison is not possible.
  • Mask²DiT (CVPR'25), already in App. G, is trained for multi-scene generation; we will label it as trained.

The revised paper will discuss training-free and training-based methods in separate blocks.

W4. The reported experiments use equal latent-frame spans, a common default in training-free multi-event methods: DiTCtrl assigns each prompt a fixed-length clip, and SwitchCraft defaults to an equal split. The authors agree with the reviewer that unequal durations require no new mechanism or tuning. As with SwitchCraft's duration weights or Mind the Time's timestamps, the user specifies them; they map to contiguous spans rounded to the latent time grid, which is just another tunable hyperparameter. TunerDiT’s PF and EM can easily operate on those manual spans unchanged. We kept equal spans with the deliberate intent that the methods isolate the temporal schedule, our main focus and contribution, without introducing a side knob. Hence, we do not think this traditional, simple setting for separating events is a major weakness of our proposed method.

Q1. The intrinsic turning point per sample is highly subject to video quality preferences. Ideally, it can be tuned per sample, depending on quality preferences and how massive a change you want the video content to shift across events. Hence, this method is called TunerDiT with “Knobs” as a metaphor. The method's contribution is mainly to bring a temporal view to multi-event generation and its inspiration. In Fig. 6b, TA stays within about 0.002 of its peak for EM gates from 0 to 0.6, so moderate per-sample shifts of the turning point change the result only slightly.

Q2. Latency and GPU memory on Wan2.1, before and after applying each method:

Wan2.1BaseSwitchCraftTunerDiT
Latency (mm:ss)15:5416:3428:34
GPU memory (MiB)71,88174,09578,025

All measured backbones, base → TunerDiT:

BackboneLatency (mm:ss)GPU memory (MiB)
Wan2.204:19 → 06:4533,413 → 33,838
Wan2.115:54 → 28:3471,881 → 78,025
Open-Sora 2.004:50 → 09:1763,648 → 63,854

On FlashAttention: the slowdown stems from EM's block-diagonal mask, which, in our current implementation, prevents the attention call from using the FlashAttention kernel. This is an implementation limit rather than an intrinsic cost: EM's mask is block-structured (event blocks plus narrow transition bands), so it can be computed with block-sparse attention kernels instead of a dense mask. The revision will report this latency comparison and list it as a limitation.

[2] Wu et al., "Mind the Time: Temporally-Controlled Multi-Event Video Generation", CVPR 2025.

Reviewer_V7ycOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper addresses multi-event video generation. By switching event prompts at different denoising-step fractions, it identifies an intrinsic turning point in the denoising trajectory of video DiTs: early steps determine the global layout, while late steps refine appearance and motion. It then proposes TunerDiT, a training-free progressive steering method with two handles: Cross-Event Prompt Fusion and Event-Partitioned Masking. The authors also introduce MEve, a multi-event prompt benchmark, and validate the method on Wan2.2, Open-Sora 1.2, and Open-Sora 2.0.

Critical strengths

The MEve benchmark fills a clear gap in current video-generation evaluation: the VBench family covers neither multi-event nor multi-scene settings, whereas MEve provides a systematic multi-event testbed with 300 prompts spanning up to four events and five categories. The evaluation is comprehensive and the results are solid: five automatic metrics, a VLM judge, and an 18-person user study corroborate each other, showing SOTA across eight metrics on three backbones, with gains that increase with event count; the ablations cleanly isolate the roles of PF and EM and characterize the alignment–consistency trade-off.

Critical weaknesses
  1. The turning point is not a new finding. Coarse-to-fine generation is a widely recognized property in T2I, and SwitchCraft already explicitly exploited the early-layout/late-detail principle for multi-event video and designed its algorithm around it. This contribution should be downgraded from a "discovery" to a systematic re-verification.

  2. The conceptual framing has a clear flaw: describing event ordering as "fine-grained detail" is semantically incorrect (swapping event order changes the entire video) and contradicts the paper's own τ≈0.3 result, which shows the layout is settled within the first 30% of steps.

  3. The term "shared layout" is vague. What is actually shared early is the scene/camera layout, while event ordering is structurally enforced by temporal segmentation and the mask; the paper never states the mapping between denoising phases and the goals of ordering, transition, and consistency.

  4. The paper reports no overhead of its own method (latency, memory), whereas its most direct competitor SwitchCraft provides a per-module latency table.

Questions for authors
  1. How do you reconcile the claim that event ordering is fine-grained detail with the τ≈0.3 result showing the layout is settled early?

  2. Please define "shared layout" rigorously and state which module, at which phase, guarantees ordering, transitions, and consistency, respectively.

  3. Please report the latency and memory overhead of PF/EM relative to the base backbones.

1 comment
AuthorsOpenReview ↗

Response to Reviewer V7yc

We thank the reviewer for recognizing MEve and the breadth of our evaluation.

W1. We agree that prior work established the value of early intervention. Liao et al. [1] shared the original two-event MEve benchmark and prompt-switch probe as a short work-in-progress through the No Proceedings track to discuss at the ICCV 2025 LongVid-Foundations workshop and on arXiv. Our submission credits that earlier study, introduces TunerDiT’s progressive inference-time steering schedule, and extends the evaluation to three- and four-event prompts. SwitchCraft was submitted to CVPR 2026 in the concurrent review cycle following Liao et al. (2025). Its method mainly focuses on the query edit and the adaptive strength solver. Early steering in SwitchCraft is a rough implementation choice rather than the main focus; for 20 of 50 steps, the implementation rationale for their steering and the rough value choice lacks empirical analysis. Our study provides a systematic, in-depth diagnosis of how to utilize it as a general video diffusion model-based finding, and asks how to achieve multi-event generation simply by scheduling different steering operations across denoising. We build on the earlier probe in §3.2 and use Fig. 6 to vary the prompt-fusion gate, the event-partitioned masking gate, and the transition-band width, and to ablate the two steering components. The resulting schedule operates directly on the segmented event prompts. In our Wan2.1 evaluation, it achieves higher event alignment than SwitchCraft (Table 1a). We will present it as a systematic re-verification of coarse-to-fine denoising for multi-event video generation and highlight the simple scheduling steering as the method.

W2 / Q1. We apologize for the problematic terminology. Event order is a property of the clip's temporal structure, and calling it "fine-grained detail" was inaccurate. TunerDiT fixes the event order by assigning the ordered prompts to consecutive latent segments. Later denoising steps then refine the action and appearance within each assigned segment. We will revise §3.2 accordingly.

W3 / Q2. We will explicitly clarify the following in the revision. "Shared layout" denotes coarse scene composition and camera framing, which are denoised in early phases before any gating (PF/EM) differentiation takes effect. During this period, the general layout of all segments is conditioned on the first event prompt, which is issued early in the denoising process. Along the schedule, PF and EM start to differentiate events by isolating attention masks and textual prompts. The mapping between modules and the target is listed as follows. We will revise the paper for a clearer definition, too.

ModuleActive phaseRole
Shared Layoutsinitial phases, before the PF and EM gatesshared scene and prompt → consistency
PFbefore the PF gate (first 10% of steps)all segments share the first event's prompt; after the gate, each segment gets its own prompt → event order
EMafter the EM gateevent-local attention → event order
Transition bands within EMafter the EM gate, gradually shrinking to 0neighbor context at boundaries → smooth transitions

W4 / Q3. End-to-end measurements, base → +TunerDiT (PF+EM):

BackboneLatency (mm:ss)GPU memory (MiB)
Wan2.204:19 → 06:45 (+56.4%)33,413 → 33,838 (+1.3%)
Wan2.115:54 → 28:34 (+79.7%)71,881 → 78,025 (+8.5%)
Open-Sora 2.004:50 → 09:17 (+92.1%)63,648 → 63,854 (+0.3%)

For reference, SwitchCraft on Wan2.1 takes 16:34 and 74,095 MiB. The memory overhead is small; the latency overhead is substantial, and we will state it in the paper. It comes from EM's block-diagonal mask, which currently prevents the use of FlashAttention; this is an implementation limit rather than an intrinsic cost. EM's mask is block-structured (event blocks plus narrow transition bands), so it can be computed with block-sparse attention kernels instead of a dense mask to reduce latency.

[1] Liao et al., "When and Where do Events Switch in Multi-Event Video Generation?", ICCV 2025 Workshop (non-archival), arXiv:2510.03049.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

We thank all reviewers for their careful reading. Below we summarize our conclusions on the shared concerns and the revisions; the individual replies give full breakdowns.

Summary of conclusions

  • The contribution is the inference-time progressive steering schedule (V7yc, TEVw, ga7b). We agree coarse-to-fine denoising is well established. The two-event MEve suite and the prompt-switch probe are from a non-archival ICCV 2025 workshop work-in-progress paper [1], which we now explicitly credit. This submission contributes TunerDiT (early cross-event prompt fusion, PF; independently gated event-partitioned masking, EM; shrinking transition bands), the three- and four-event extension of MEve, and probing on the Wan family.
  • Event order is fixed by construction, and later events are not suppressed (V7yc, ga7b). Order is set in video time by assigning prompts to consecutive latent segments; the ~30% turning point concerns denoising time. On all three-event MEve prompts (Wan2.1), TunerDiT improves event-level TA at every index: 0.214 / 0.211 / 0.211, vs. 0.192 / 0.180 / 0.179 (base) and 0.197 / 0.193 / 0.192 (SwitchCraft).
  • Gains are consistent within every backbone (TEVw, ga7b). TA and TIS improve over the unmodified backbone on all four backbones at every event count (Table 1a).
  • Training-based methods are separated from training-free ones (TEVw). The most direct trained method, Mind the Time, releases no weights or data, so it cannot be run on MEve. We have already discussed Mask²DiT as a trained model comparison in the Appendix and discuss training-based methods in a separate block.
  • Latency Cost is reported transparently (V7yc, TEVw). On Wan2.1, base / SwitchCraft / TunerDiT take 15:54 / 16:34 / 28:34 and 71,881 / 74,095 / 78,025 MiB; other backbones are in the replies to V7yc and TEVw. The latency gap comes from EM's block-diagonal mask, which currently prevents the use of FlashAttention; block-sparse attention kernels can avoid this.
  • Reporting issues are corrected (ga7b). We fix the TVA definition and the rater count. The PF gate is already shared across all backbones, and TA changes little around the selected EM gate (Fig. 6b).

[1] Liao et al., “When and Where do Events Switch in Multi-Event Video Generation?”, ICCV 2025 LongVid-Foundations Workshop (non-archival), arXiv:2510.03049.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.