acceptodds
Under review as a conference paper at ICLR 2027

FLARE: Improving Long-Horizon Fidelity in Autoregressive Video Generation via Factorized Latent Reward Feedback

Abstract

Autoregressive video diffusion with streaming training enables real-time long-form generation, but error accumulation across chunks makes long-horizon consistency a fundamental challenge. Existing streaming distillation objectives keep generated chunks on the natural-video manifold, but provide limited guidance for resolving quality and alignment ambiguities during autoregressive rollout. As a result, long rollouts often suffer from semantic drift, visual quality decay, and motion degradation. We introduce FLARE, which closes this gap through a latent preference reward that scores video latents directly via a DiT backbone, equipped with three learnable reward queries that attend to backbone features and route to independent heads, allowing a single video pair to express conflicting preferences across aspects. The reward is trained in two stages: a generic Bradley-Terry preference SFT, followed by a consensus-guided latent preference adaptation stage in which multiple pixel-space evaluators serve as weak head-wise annotators and only cross-evaluator-agreed comparisons are retained. The trained reward is then integrated as an online, differentiable per-chunk signal alongside DMD distillation, providing aspect-specific preference guidance during streaming optimization while retaining distribution-level supervision from DMD. Experiments on long-video generation show that FLARE improves generation fidelity over strong autoregressive baselines, and ablations verify the effectiveness of the latent reward design.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Autoregressive Video Generation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_2T39OpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes FLARE, a factorized latent reward model for autoregressive video generation, quality signals are directly predicted in the VAE latent space by sharing the DiT backbone with three learnable query headers (semantic alignment/visual quality/dynamic quality). The reward model is trained in two stages: first doing generic preference SFT, then doing latent adaptation with majority voting labels from multiple pixel spatial evaluators, and then jointly optimizing the generator with DMD stream distillation as a frozen differentiable critic. The paper claims consistent improvement in different settings, and ablation experiments verify the effectiveness of the decomposition design.

Critical strengths
  1. Problem setting is important and practical. Semantic drift, visual quality attenuation and motion degradation in long video generation are the core bottlenecks of autoregressive video generation. This paper attempts to provide fine-grained quality reward along with DMD to alleviate consistency and motion quality degradation caused by error accumulation in long video inference.

  2. The factorized rewards are reasonable and well designed. Instead of compressing all preferences into a single scalar, the paper split reward into three heads: semantic alignment, visual quality, and dynamic quality, and helped the model improve on their corresponding dimensions.

  3. The two-stage reward training strategy is more complete. Firstly, Bradley-Terry SFT is performed using the general preference data, and then the target domain adaptation is performed using the consistency signals of multiple pixel-space evaluators, which helps to obtain the general preference ability and long video target domain adaptation at the same time.

  4. The experimental coverage is more comprehensive. The paper evaluates not only interactive 60-second generation, but also single-prompt long video generation and regular 5-second short video generation, indicating that the method is not only effective in specific interaction scenarios.

Critical weaknesses
  1. The training details of the reward model in the latent preference adaptation stage are missing, such as the magnitude of the training data set and the performance of the corresponding test set.

  2. The paper emphasizes the advantage of reward in latent space, but does not give the corresponding experimental data, such as performance or efficiency in the latent space compared with the pixel space.

Questions for authors
  1. Besides the reported gains in subject consistency, background consistency, and motion smoothness, how does latent reward optimization affect other evaluation dimensions? Are there any dimensions that improve less, remain unchanged, or degrade after applying FLARE?

  2. Could the authors provide a duration-wise or segment-wise analysis of subject consistency, background consistency, motion smoothness, and imaging quality across different generation lengths, e.g., 10s, 20s, 30s, 40s, 50s, and 60s, to verify the effectiveness of the improvement for chunk error accumulation.

  3. Could the authors provide more implementation details of the single-head baseline in Tab. 7? Specifically, does it use a single reward query and a single scalar head, or does it aggregate the three aspect labels/scores into one overall preference objective?

  4. Since one key motivation of latent-space reward is to avoid expensive pixel-space reward inference, could the authors provide a quantitative efficiency comparison against pixel-space reward computation, such as GPU memory usage and inference time cost?

  5. In CLPA, UnifiedReward-Flex is used for all three aspects, while VideoAlign and UniPercept are used selectively. What is the rationale behind choosing UnifiedReward-Flex as the common evaluator across all dimensions? Could the authors provide an ablation or analysis comparing different evaluator combinations for CLPA.

  6. Could the authors report the agreement between each weak evaluator (VideoAlign, UnifiedReward-Flex, and UniPercept) with human preferences on the internal long-horizon autoregressive test set?

Reviewer_wP97OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FLARE, which integrates a factorized latent reward model into the streaming DMD process as a differentiable loss. The reward model predicts semantic alignment, visual quality, and dynamic quality from video latents and is trained with a consensus-guided, head-wise distillation scheme to further integrate preference signals from multiple pixel-space evaluators. Experiments show improvements on long interactive videos, long single-prompt videos, and short videos.

Critical strengths
  1. This paper addresses a meaningful and unresolved problem: long-horizon fidelity in autoregressive video generation.

  2. The reward design is generally practical for streaming AR training without the need for VAE decoding.

Critical weaknesses
  1. Motivation mismatch: The reward signal does not match the long-horizon motivation. The reward model only sees one overlapping frame plus the current chunk. It has no access to earlier content, so it cannot directly penalize drift relative to the beginning of the rollout. In addition, there are prior explorations on combining DMD with a differentiable reward like Reward Forcing. The authors should provide more analysis on how the reward model addresses the long-term drift and further discuss the novelty.

  2. Lacking video results: No video comparisons with other baseline methods are provided in the supplementary material. Without such results, it is hard to assess the performance and improvement of the proposed method.

Questions for authors

What is the training overhead of the proposed method compared to other baseline methods?

Reviewer_wMnoOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

FLARE improves autoregressive long-video generation using a differentiable latent reward model with separate heads for semantic alignment, visual quality, and motion quality. The reward model is trained on generic preferences, adapted using multi-evaluator consensus, and integrated into streaming DMD training to provide chunk-level feedback.

Critical strengths
  1. The factorized reward design preserves aspect-specific preferences, while consensus-based adaptation offers a practical way to incorporate supervision from multiple evaluators.
  2. Experiments cover both interactive and single-prompt generation at multiple durations. The reported improvements and reward-model evaluations provide encouraging evidence for the approach.
  3. The presentation is clear and the writing is good.
Critical weaknesses
  1. The human-annotated evaluation measures reward-model accuracy, not preference for FLARE’s final outputs. Human comparisons and motion evaluations beyond smoothness would strengthen the conclusions.

  2. The efficiency comparison in Table A.1 lacks window units, hardware details, and other details. Also methods like Reward Forcing which do not require backpropagation through the reward model, should also be considered in this comparison.

  3. The paper insufficiently distinguishes FLARE from closely related latent reward methods, like DOLLAR [1], PRFL [2], and VELR [3]. A clearer comparison and a controlled baseline within the same streaming DMD framework are needed to establish its incremental contribution. [1] Ding et al. DOLLAR: Few-Step Video Generation via Distillation and Latent Reward Optimization. [2] Mi et al. Video Generation Models Are Good Latent Reward Models. [3] VELR: Efficient Video Reward Feedback via Ensemble Latent Reward Models.

Questions for authors

N/A

Comments on the paper

1 comment
Reviewer_2T39OpenReview ↗

Further concerns regarding the response to Q1 and Q2

  1. It is suggested to supple the comparison of baselines (Wan2.1 and methods in Tab.1), DMD-only, and the proposed method across all VBench dimensions, so as to better reflect how the quality varies across different evaluation metrics. In particular, given that the Temporal flickering scores are close, it would be useful to examine whether applying the proposed post-training to the baseline introduces any impact on other dimensions.

  2. It is suggested to supplement the Wan2.1 model (baseline) with dmd-only and the full comparison of FLARE.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.