acceptodds
Under review as a conference paper at ICLR 2027

TempAct: Advancing Temporal Plausibility in Autoregressive Video Generation via Planner-Executor Reinforcement Learning

Abstract

Autoregressive (AR) video diffusion models enable low-latency streaming generation by synthesizing videos chunk by chunk with cached visual context, but this chunk-wise formulation makes temporal instruction following ambiguous. A single global prompt does not specify which sub-event should be realized in each chunk, while naively switching to step-wise prompts often leads to delayed reactions, blended step semantics, and error propagation across prompt transitions. These failures are difficult to address with supervised fine-tuning (SFT) or distillation alone: SFT suffers from exposure bias, while rollout-based distillation still optimizes low-level denoising or teacher-distribution matching rather than directly enforcing action ordering and prompt-transition correctness. We propose **TempAct**, **a planner–executor reinforcement learning** framework that jointly optimizes temporal decomposition and step-conditioned execution for temporally plausible AR video generation. TempAct uses an LLM planner to explore span-aware step prompts that are executable by the video model, and trains an AR diffusion executor to follow these prompts under its own generated histories. Its key mechanism is hierarchical group exploration: candidate plans form planning groups, and each plan induces an execution group of multiple continuations from a shared visual context, enabling plan-level credit assignment for long-horizon temporal outcomes and executor-level credit assignment for prompt-switch behavior. We further design hierarchical rewards that combine plan-quality and full-video temporal feedback for the planner with local transition-level step-following rewards, aesthetic regularization, and KL constraints for the executor. Experiments on Self-Forcing and LongLive show that TempAct improves temporal consistency while preserving overall visual quality. Detailed comparisons of video generation results are provided in a **local HTML page** included in the supplementary materials.

Primary area:
generative models
Keywords:
Autoregressive Video Generation, Reinforcement Learning, Temporal Plausibility

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_weptOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes TempAct, an RL based framework to jointly learn how to decompose a long global instruction into chunk-level input prompts for executable temporal steps, and how to make the modern ar distilled video diffusion (e.g. longlive, sf) act correctly with those temporal instructions. The authors design careful ablation study to verify the effectiveness of each components in the joint RL framework. Finally, quantitative and qualitative experiments across self forcing and longlive codebase demonstrate the effectiveness of the approach.

Critical strengths
  1. The problem and motivation is good and reasonable. Most modern bidirectional to ar model either use global prompts or directly combining chunk-wise prompt without tailored design improvement, making one of the important purpose of ar-diffusion (interactive rolling) less explored.
  2. The design is neat and the validation is clean. The planner and executor design is a natural and comfortable extension of the motivation. And the effectiveness of each component is validated by ablation by the authors.
Critical weaknesses
  1. "Distinction between TempAct and previous work might need further discussion". KVPO [1] shares similar idea on the executor side whereas VideoDirectorGPT [2] shares the idea of planner. Longlive also create a baseline for the current setting, which makes the distinction of TempAct to previous work relatively weak.

[1] KVPO: ODE-Native GRPO for Autoregressive Video Alignment via KV Semantic Exploration

[2] VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided Planning

Questions for authors
  1. Is the planner reward limited effectiveness (seen through Table 2 and figs on the right) caused by the small planner model (1.3B qwen) or the method itself, since I observe that most of the gain is from the executor.

  2. Could the author test on the setting with extended rollout duration, since current 141 frames is not deviated much from the training horizon, and might be far from what a user want during inference, that could also test the framework's generalization capacity to OOD domain.

  3. Could the author provide more discussion regarding previous work like KVPO [1] and VideoDirectorGPT [2] ?

[1] KVPO: ODE-Native GRPO for Autoregressive Video Alignment via KV Semantic Exploration

[2] VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided Planning

3 comments
AuthorsOpenReview ↗

Response to Reviewer wept

Thank you for the constructive feedback. We address each of your suggestions below.


W1 / Q3. Distinction from previous work.

A1: We are sorry that the introduction did not end with a summary of our contributions, which made the novelty unclear. Our method is fundamentally different from VideoDirectorGPT and KVPO. The contributions are:

  1. Cooperative RL of the planner and the executor. The planner learns a temporal decomposition that the video model can execute, and the executor learns to follow the prompt switch under its own history.
  2. Hierarchical group sampling, with the reward and the credit assigned separately at the plan level and at the prompt-switch level.
  3. Fine-grained reward design. We perform precise credit assignment for the rewards induced by policy decisions: full-video temporal-following and plan-consistency rewards are assigned to the planner, while local execution-consistency and aesthetic rewards are assigned to the executor.
  4. A benchmark and a metric for temporal following: a Simple Set (1–2 steps) and a Hard Set (3–4 steps), scored by the Temporal-Following Score with both Qwen3-VL and Gemini.

KVPO extends the policy decision to the KV-cache router and proposes a reward-weighted contrastive flow-matching objective, using the VideoAlign reward to improve general video-generation properties such as motion quality, visual quality, and text alignment. Both its planning and execution rely on the diffusion model; it does not explore hierarchical sampling or multi-granularity reward credit assignment, nor does it focus on temporal consistency.

We add a comparison with KVPO. Since it uses VideoAlign's general video-generation reward, for a fair comparison on temporal generation, we instead use Qwen3-VL-8B for the Local Step-Following Score and the Temporal-Following Score, together with the same aesthetic reward and weights as TempAct. The planner is kept at initialization and frozen. The experiment is conducted on LongLive, and only the first chunk is trained. The results are shown in the table below:

MethodSimple QwenSimple GeminiHard QwenHard GeminiAvg.TotalQualitySemanticPickScore
LongLive0.4110.5210.3140.4810.43279.5582.1069.3520.8
+ KVPO0.451 (+9.7%)0.542 (+4.0%)0.331 (+5.4%)0.493 (+2.5%)0.454 (+5.1%)79.8282.3769.4520.8
+ TempAct0.508 (+23.6%)0.579 (+11.1%)0.352 (+12.1%)0.512 (+6.4%)0.488 (+13.0%)79.9782.6169.4020.8

As expected, KVPO improves temporal plausibility by enhancing the executor and the cache-router scheduling. However, the gain is limited. On one hand, the router's language scheduling ability is very weak and cannot match planner-level semantic scheduling, resulting in a lack of collaboration between the planner and the executor. On the other hand, both the global reward (semantic planning consistency) and the local reward (action execution consistency) are optimized through diffusion, mixing the two credits and reducing optimization efficiency. We will add this result to the final version.

VideoDirectorGPT only switches the prompt at scene timestamps, with a frozen GPT-4 writing the scene text and layout for a fixed generator. T2I-R1 is joint RL for one autoregressive image. We introduce time into streaming video, so planning and execution are coupled differently. LongLive's KV-recache is an inference method for smoother prompt switches. It is orthogonal to our RL. Table 1 shows that TempAct still improves LongLive.


Q1. Is the limited planner gain caused by the small planner or by the method?

A2: Our planner is Qwen3-1.7B. Replacing it with a frozen GPT planner, while keeping Self-Forcing as the executor, only moves the Qwen scores from 0.414 / 0.269 / 0.342 to 0.421 / 0.286 / 0.354:

MethodSimple SetHard SetAvg.
Self-Forcing0.4140.2690.342
+ GPT Planner0.4210.2860.354

The gain is limited. A strong planner can produce a good plan, but the bottleneck is the diffusion model, which cannot effectively execute that plan. We will add this result to Table 2.


Q2. Extended rollout duration.

A3: Following your suggestion, we provide a 30s evaluation on LongLive, using the 100 Hard-Set samples and Gemini. Baseline: 0.529. TempAct: 0.562. As the video gets longer, the VLM's frame sampling rate and accuracy drop, and real-time models are still weak at complex temporal logic. The main evaluation in this paper is therefore at 15s.

Thank you for response, since llm model scale does not influence much, the bottleneck is the DiT. I learn something useful and will raise the score.

Dear Reviewer,

Thank you again for your timely and constructive review. Following your recommendation, we have incorporated the explanations from our rebuttal into the main text of the revised manuscript. We are grateful for the time and care you devoted to our paper.

Reviewer_6eGuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces TempAct, a planner-executor RL framework for temporally consistent AR video generation. It uses an LLM planner to decompose the textual prompt into per-step prompts and trains an AR diffusion model to follow these prompts and generate the video step by step. It's core contribution is hierarchical group exploration: the LLM generates planning groups, and each plan results in an execution group of multiple continuations from a shared visual context. Experiments on Self-Forcing and LongLive demonstrate the effectiveness of the proposed framework.

Critical strengths
  1. The paper addresses a meaningful problem: temporal transition in autoregressive video generation. Such transitions are common in streaming videos but remain underexplored in current work.

  2. Substantial implementation details are disclosed in the paper and appendix, which enhance the reproducibility of this paper.

Critical weaknesses
  1. Limited novelty: 1) Each component of TempAct closely follows existing work (LLM-based planning [VideoDirectorGPT], training the planner with the executor's feedback [Prompt-A-Video], jointly training the planner and the executor within one RL framework [T2I-R1]). The authors should further clarify what is fundamentally beyond the simple combination of existing methods. 2) The KV-recaching mechanism of LongLive also tries to address the prompt transition problem. However, the discussion between the proposed method and the KV-recaching design is missing.

  2. Evaluation reliability: 1) Long videos are a primary application scenario for AR video models. However, evaluations of long-video generation (e.g., 30s and 60s) are not provided in the paper. 2) The Qwen3-VL-8B is used as both the training reward and the evaluator, which increases the risk of reward hacking. 3) The following baselines should be presented: a strong planner (like GPT or a large open-source model like Qwen) with a frozen executor, and existing AR Video RL methods (like Astrolabe) with the same rewards. Such baselines can better isolate the effectiveness of each component.

Questions for authors

Please refer to weaknesses.

8 comments
AuthorsOpenReview ↗

Response to Reviewer 6eGu (1 / 2)

Thank you for the constructive feedback. We address each of your suggestions below.


W1. Novelty.

A1.1: We are sorry that the introduction did not end with a summary of our contributions, which made the novelty unclear. Our method is fundamentally different from VideoDirectorGPT and T2I-R1. The contributions are:

  1. Cooperative RL of the planner and the executor. The planner learns a temporal decomposition that the video model can execute, and the executor learns to follow the prompt switch under its own history.
  2. Hierarchical group sampling, with the reward and the credit assigned separately at the plan level and at the prompt-switch level.
  3. Fine-grained reward design. We perform precise credit assignment for the rewards induced by policy decisions: full-video temporal-following and plan-consistency rewards are assigned to the planner, while local execution-consistency and aesthetic rewards are assigned to the executor.
  4. A benchmark and a metric for temporal following: a Simple Set (1–2 steps) and a Hard Set (3–4 steps), scored by the Temporal-Following Score with both Qwen3-VL and Gemini.

VideoDirectorGPT only switches the prompt at scene timestamps. A frozen GPT-4 writes the scene text and layout, and a fixed generator renders each scene. It does not use RL to explore a better-aligned planner or generator, nor does it have a hierarchical sampling strategy or fine-grained reward design and credit assignment.

T2I-R1 studies RL on a unified autoregressive model for image understanding and generation. In contrast, we first introduce time into streaming video generation. Planning and execution are coupled differently: the plan assigns events over time, and the executor must realize each event under its own causal history. Moreover, T2I-R1 does not explore or construct hierarchical sampling or more accurate credit assignment between planning and execution.

We will add our contributions to the Introduction and the discussion of related methods to the Related Work.


W1. KV-recache.

A1.2: We cite and discuss LongLive's KV-recache in the related work. It is an inference method for smoother prompt switching and stronger instruction following. Our RL does not conflict with it. The two are orthogonal and can be stacked. Table 1 shows that TempAct also improves LongLive, which supports that they do not conflict.


W2.1. Evaluation video length.

A2.1: Following your suggestion, we provide a 30s evaluation on LongLive, using the 100 Hard-Set samples and Gemini. Baseline: 0.529. TempAct: 0.562. As the video gets longer, the VLM's frame sampling rate and accuracy drop, and real-time models are still weak at longer and more complex temporal logic. The main evaluation in this paper is therefore at 15s.


W2.2. Reward hacking.

A2.2: To prevent reward hacking, we add an aesthetic reward during training. At evaluation we use Gemini, a closed model different from the training reward. This controls reward hacking in both training and evaluation.


AuthorsOpenReview ↗

Response to Reviewer 6eGu (2 / 2)

W2.3. A stronger RL baseline and a stronger planner.

A2.3: Thank you for the suggestion. We added KVPO to the main comparison in Table 1.

KVPO extends the policy decision to the KV-cache router. Since it uses VideoAlign's general video-generation reward, for a fair comparison on temporal generation, we instead use Qwen3-VL-8B for the Local Step-Following Score and the Temporal-Following Score, together with the same aesthetic reward and weights as TempAct. The planner is kept at initialization and frozen. The experiment is conducted on LongLive, and only the first chunk is trained. The results are shown in the table below:

MethodSimple QwenSimple GeminiHard QwenHard GeminiAvg.TotalQualitySemanticPickScore
LongLive0.4110.5210.3140.4810.43279.5582.1069.3520.8
+ KVPO0.451 (+9.7%)0.542 (+4.0%)0.331 (+5.4%)0.493 (+2.5%)0.454 (+5.1%)79.8282.3769.4520.8
+ TempAct0.508 (+23.6%)0.579 (+11.1%)0.352 (+12.1%)0.512 (+6.4%)0.488 (+13.0%)79.9782.6169.4020.8

As expected, KVPO improves temporal plausibility by enhancing the executor and the cache-router scheduling. However, the gain is limited. On one hand, the router's language scheduling ability is very weak and cannot match planner-level semantic scheduling, resulting in a lack of collaboration between the planner and the executor. On the other hand, both the global reward (semantic planning consistency) and the local reward (action execution consistency) are optimized through diffusion, mixing the two credits and reducing optimization efficiency. We will add this result to the final version.

We also test GPT as the planner, with Self-Forcing as the frozen executor. On the Qwen3-VL scores of the Simple Set, the Hard Set, and their average:

MethodSimple SetHard SetAvg.
Self-Forcing0.4140.2690.342
+ GPT Planner0.4210.2860.354

The gain is limited. A strong planner can produce a good plan, but the bottleneck is the diffusion model, which cannot effectively execute that plan. We will add this result to Table 2.

Dear Reviewer 6eGu,

Thank you again for your valuable comments. As the discussion period is nearing its end, we wanted to kindly follow up on our rebuttal—where we clarified the distinctions from prior work (e.g., VideoDirectorGPT, T2I-R1, and KV-recache) and added the 30s evaluation, KVPO, and GPT-planner baselines—to see whether these address your concerns.

We would greatly appreciate any further discussion and are very happy to provide additional clarification if you still have questions.

Reviewer_TndgOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TempAct, which jointly trains an LLM planner and an autoregressive video executor to improve the execution order of multi-step instructions through hierarchical sampling, shared visual histories, and local transition rewards. Across both Self-Forcing and LongLive, the composite temporal-following score improves by 15.5% and 13.0% relative to step-prompt baselines, supported by ablations, an independent VLM evaluator, and human evaluation.

Critical strengths

This paper's hierarchical planner-executor looping framework directly addresses prompt-transition failtures. Further, joint training outperforms training either the planner or executor alone. Improvements hold consistent across both VLM and human preferences, contributing to reliability of the results.

Critical weaknesses

The paper does not directly compare against existing autoregressive video RL methods such as AR-CoPO, Astrolabe, or KVPO.

Questions for authors
  1. Do the gains persist against existing video RL methods under matched data, rewards, prompt processing, and training budgets?
  2. Could you clarify the number of human annotators and how their judgments were aggregated?
5 comments
AuthorsOpenReview ↗

Response to Reviewer Tndg

Thank you for the constructive feedback. We address each of your suggestions below.


W1 / Q1. Comparison with other RL methods.

A1: Thank you for the suggestion. We added KVPO to the main comparison in Table 1.

KVPO extends the policy decision to the KV-cache router. Since it uses VideoAlign's general video-generation reward, for a fair comparison on temporal generation, we instead use Qwen3-VL-8B for the Local Step-Following Score and the Temporal-Following Score, together with the same aesthetic reward and weights as TempAct. The planner is kept at initialization and frozen. The experiment is conducted on LongLive, and only the first chunk is trained. The results are shown in the table below:

MethodSimple QwenSimple GeminiHard QwenHard GeminiAvg.TotalQualitySemanticPickScore
LongLive0.4110.5210.3140.4810.43279.5582.1069.3520.8
+ KVPO0.451 (+9.7%)0.542 (+4.0%)0.331 (+5.4%)0.493 (+2.5%)0.454 (+5.1%)79.8282.3769.4520.8
+ TempAct0.508 (+23.6%)0.579 (+11.1%)0.352 (+12.1%)0.512 (+6.4%)0.488 (+13.0%)79.9782.6169.4020.8

As expected, KVPO improves temporal plausibility by enhancing the executor and the cache-router scheduling. However, the gain is limited. On one hand, the router's language scheduling ability is very weak and cannot match planner-level semantic scheduling, resulting in a lack of collaboration between the planner and the executor. On the other hand, both the global reward (semantic planning consistency) and the local reward (action execution consistency) are optimized through diffusion, mixing the two credits and reducing optimization efficiency.

We will add this result to the final version.


Q2. Details of the human annotation.

A2: Thank you for the suggestion. We recruited 50 annotators. Each pair was labeled independently in two rounds, in random order and without model names. If the two labels agree, we keep that label. If they disagree, the pair is counted as a tie. We will add these details to the main text.

Thank you for your response. This feedback basically addresses my concerns. Could you mention why you did not compare against other methods that I mentioned?

Dear Reviewer Tndg

Thank you for your prompt response and constructive feedback, which have significantly improved our paper.

Video RL methods are computationally expensive due to the large number of rollouts and VLM verifier calls required. Given the tight rebuttal timeline and limited compute budget, we prioritized KVPO as it is the most recent approach and reports state-of-the-art performance among these methods in the original paper (albeit by a relatively modest margin).

That said, we will add comparisons against AR-CoPO and Astrolabe in the final version to provide a more comprehensive evaluation of existing methods in terms of temporal logic.

Thank you again for your helpful comments and suggestions!

Comments on the paper

3 comments
AuthorsOpenReview ↗

Overall Response

We sincerely thank all three reviewers for their constructive feedback. We are encouraged that they recognized the underexplored temporal-transition problem (6eGu, wept), our neat planner–executor design (Tndg, wept), the consistent gains across VLM/human evaluations and ablations (Tndg, wept), and our reproducibility (6eGu).

Below, we briefly overview and respond to the reviewers' comments on the paper.

1. Comparison with KVPO, and a stronger planner (Tndg W1/Q1; 6eGu W2.3; wept Q1). We add these two comparisons: (1) Under the same rewards and task setup, KVPO reaches 0.454 (+5.1%) on LongLive, while TempAct reaches 0.488 (+13.0%). (2) Likewise, limited by the executor's diffusion capability, a strong planner (GPT-6-sol) on Self-Forcing only moves the Qwen reward average from 0.342 to 0.354. These results show that TempAct outperforms existing RL methods and that joint RL of the planner and executor is clearly effective.

2. Difference from VideoDirectorGPT, T2I-R1(6eGu W1; wept W1/Q3), and LongLive KV-recache (6eGu W1). (1) Our method is fundamentally different from existing methods: TempAct is a joint RL algorithm that decomposes and optimizes planning and execution in streaming video generation to directly address temporal plausibility. In contrast, VideoDirectorGPT only switches the prompt at scene boundaries. T2I-R1 is joint RL for one autoregressive image, whereas we introduce an RL optimization mechanism for temporal planning and execution in streaming video. (2) LongLive KV-recache is an inference method for smoother prompt switches. It is orthogonal to our RL and can be stacked with it. TempAct still improves LongLive in Table 1.

3. Human annotation (Tndg Q2). We used 50 annotators and two independent rounds. Agreed labels are kept; disagreements are counted as ties. Details will be added to the main text.

4. Video length (6eGu W2.1; wept Q2). We add longer-duration test results: on the 100 Hard-Set prompts at 30s, Gemini scores are 0.529 (LongLive) and 0.562 (TempAct). The main evaluation stays at 15s, because longer videos lower VLM frame sampling and accuracy, and real-time models are weak on longer and more complex temporal logic.

5. Reward hacking (6eGu W2.2). We add an aesthetic reward in training, and evaluate with Gemini, a closed model that is not the training reward.

We will reply to each reviewer's comments one by one.

Dear Reviewers and AC,

We have uploaded the revised manuscript incorporating the clarifications and additional experimental results discussed in our rebuttal. All newly added and modified contents are highlighted in red in the updated PDF for ease of inspection, including:

  1. Section 1 (Introduction): Added an explicit four-point summary of our core contributions at the end of the Introduction.
  2. Section 2 (Related Work): Added discussions comparing TempAct with VideoDirectorGPT and T2I-R1.
  3. Section 4.2 & Table 1 (Quantitative Results): Added the quantitative comparison and analysis with KVPO on LongLive under a unified reward protocol.
  4. Section 4.3 & Table 2 (Ablation Study): Added the + GPT Planner baseline result with a frozen Self-Forcing executor.
  5. Section 4.4 (User Study): Clarified the human evaluation protocol (50 recruited annotators, anonymized two-round independent labeling, and tie-handling criteria).

We sincerely thank the reviewers again for their constructive feedback, which has helped us significantly improve the clarity and completeness of the paper.

Dear Reviewers,

As the initial review scores for this submission are somewhat divergent, it is important that we thoroughly discuss our perspectives before reaching a final recommendation.

Now that the authors' rebuttal is available, I kindly ask each of you to respond in the discussion and clarify whether your original concerns have been addressed or still remain.

Thank you in advance for your timely engagement before the discussion period ends.

Best, Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.