acceptodds
Under review as a conference paper at ICLR 2027

Golden Anchor for Video Generation

Abstract

Video generation supports digital content creation, with applications in virtual avatars, AI-driven filmmaking, interactive game engines, and world models. State-of-the-art systems achieve controllability primarily through image-to-video (I2V) generation, in which an injected anchor image determines the identity, composition, and style of the resulting video. In modern pipelines, this anchor is synthesized by a text-to-image (T2I) generator. This raises a largely overlooked question: Are images obtained directly from the T2I model truly suitable for current video models? Our analysis shows that anchor aesthetic and quality scores are weakly correlated with downstream video quality. The structure-and-texture score is negatively correlated with I2V video quality in both the visual and motion dimensions. Fine-tuning the T2I model on an image reward accordingly yields anchors with visibly denser structural and character detail: they score higher on image metrics, yet lower the video reward on every I2V model we test and change downstream motion unpredictably. We therefore introduce Golden Anchor Learning (GAL), to our knowledge the first framework for learning golden anchors by optimizing a T2I generator with downstream video feedback. GAL evaluates candidate anchors through the videos they induce under a frozen I2V model and converts video rewards into image-level preferences for DPO fine-tuning of the T2I generator. On all four I2V models, GAL raises the VBench average by 2.97%–8.46% over the untuned generator. The rendered clips also score higher under VideoReward-Overall, and their extracted frames under HPSv3 and UniPercept. GAL yields consistent motion gains across all four I2V models, as measured by a 5.62%–38.24% increase in VBench dynamic degree. Although GAL is trained only on HunyuanVideo-1.5 renders, these gains extend to the other three I2V models.

Primary area:
generative models
Keywords:
video generation, image-to-video generation, text-to-image generation, preference optimization, direct preference optimization, diffusion models, flow matching

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Cs5nOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper explores the preference issue in text-to-image models from the perspective of the entire t2i-i2v video generation pipeline, optimizing the image generation output based on whether it is a suitable input for a video generation model. Overall, this is an interesting attempt. The authors propose Golden Anchor Learning, which treats a frozen I2V video model and a video evaluation scheme as a generalized "image reward model," and builds a preference dataset on this basis. Following this idea, the paper fine-tunes the T2I generator with DPO, ultimately improving the video score on the t2i-i2v pipeline.

Critical strengths
  1. Optimizing the image model's performance from the perspective of the overall t2i-i2v pipeline is an interesting idea, and this paper is the first to attempt the full pipeline and make it work.

  2. The paper reasonably adopts an offline learning strategy, simplifying the complex pipeline into preference alignment of the image model, using the I2V pipeline directly as a component of the reward construction pipeline, and applying DPO for preference optimization. The experiments show that this approach indeed achieves better scores, and the overall architecture and experiments are solid.

Critical weaknesses
  1. Theoretical novelty is relatively limited. Although the problem is new, the paper as a whole remains within the simple pipeline of preference dataset construction and DPO fine-tuning, so its novelty as academic research warrants further consideration.
Questions for authors
  1. Why were the experiments conducted on LTX-Video and CogVideoX rather than newer models such as LTX-2? Does this suggest that part of the improvement in Table 1 stems from insufficient I2V capability of the video models themselves?

  2. Regarding the approach itself, have you explored whether directly optimizing text-to-image prompts for I2V would be simpler than fine-tuning the T2I model, and potentially even more effective? How much gain does GAL offer over such an approach?

4 comments
AuthorsOpenReview ↗

Looking Forward to Your Feedback:

Dear Reviewer Cs5n,

Thank you again for your valuable comments. As the discussion period closes in two days, we would like to kindly check whether our response and revision have addressed your concerns. We would be grateful for any feedback and are happy to clarify anything further.

Best regards, The Authors

AuthorsOpenReview ↗

A Gentle Reminder on Our Response

Dear Reviewer Cs5n,

We apologize for following up once more. As the discussion period ends in less than 11 hours, we would be grateful if you could take a brief look at our response when convenient. Any feedback, even brief, would be very helpful, and we are happy to clarify anything promptly.

Thank you again for your time and consideration.

Best regards, The Authors

Thank you for the detailed rebuttal and clarifications. I will maintain my original rating.

AuthorsOpenReview ↗

Thank You for Your Positive Assessment

Dear Reviewer Cs5n,

Thank you for taking the time to read our rebuttal and for maintaining your positive score. We sincerely appreciate your support and constructive comments, and we will carefully incorporate your suggestions into the revised version.

Best regards, The Authors

Reviewer_zB9HOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper considers a two-stage T2I→I2V video generation pipeline. It freezes the I2V model and applies preference-based RL (Diffusion-DPO) only to the T2I model. The reward is defined on the final video: each candidate anchor image is rendered by the frozen I2V model and scored with VideoReward. The goal is to improve the video quality of the whole system by modifying only the T2I weights.

Critical strengths
  • The problem is interesting. The paper explores whether optimizing a lighter, and usually more accessible, T2I model can improve the complete T2I→I2V pipeline.
  • The results support the feasibility of this direction. The gains appear across four I2V models, including three that were never used to construct preferences.
Critical weaknesses
  1. The VideoReward signal is underspecified and never decomposed.

    • VideoReward has three dimensions: visual quality (VQ), motion quality (MQ), and text alignment (TA). The paper does not explain how these are normalized and combined into the "Normalized Final" / Overall score used to build preference pairs.
    • The paper reports the training with only VideoReward-Overall, with no per-dimension breakdown, which makes the results hard to interpret.
    • The rewarded aesthetic and motion aspects do show up on VBench. Imaging quality rises by 8.0–10.3%, aesthetic quality by 2.5–6.2%, and dynamic degree by 5.6–38.2% (Table 1). However, the text-alignment aspect is hard to say. The only reported indicator is Overall Consistency, which changes by only +0.07% to +1.45%. All remaining semantic dimensions of VBench are abscent (See next weakness)
  2. Only 6 of the 16 VBench dimensions are reported.

    • VBench defines 16 dimensions, but the paper reports only 6, averages them without weighting, and gives no justification for the selection.
    • The remaining 10 dimensions are all computable in this setting.
    • In particular, all semantic dimensions are omitted: object class, multiple objects, human action, color, spatial relationship, scene, appearance style, and temporal style.
    • Without them, it is impossible to judge whether the model gains motion and machine-judged aesthetic quality at the cost of semantic alignment with the prompt.
Questions for authors

See weaknesses

3 comments
AuthorsOpenReview ↗

Looking Forward to Your Feedback:

Dear Reviewer zB9H,

Thank you again for your valuable comments. As the discussion period closes in two days, we would like to kindly check whether our response and revision have addressed your concerns. We would be grateful for any feedback and are happy to clarify anything further.

Best regards, The Authors

Thanks to the authors for the detailed response. My concerns have been addressed, and I have therefore raised my score to 3.

AuthorsOpenReview ↗

Thank You for Your Positive Feedback

Dear Reviewer zB9H,

Thank you very much for your positive feedback and for raising your score. We sincerely appreciate your time and constructive comments, which have helped improve our paper. We will incorporate the corresponding revisions into the final version.

Best regards, The Authors

Reviewer_uvotOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates a problem in text-2-image-2-video pipeline: generated images are not always suitable for later video generation. To address the challenge, the paper proposes Golden Anchor Learning (GAL), which uses a frozen image-to-video model and a video reward model to construct preferences over candidate anchor images, then fine-tunes the upstream text-to-image generator with Diffusion-DPO. Experiments show clear improvements on different benchmark metrics. And the method can be transfered to three models not used for preference construction.

Critical strengths
  1. The challenge the paper introduces exists and very practical. Its empirical analysis motivates optimizing anchor images according to the videos they induce, offering a useful perspective on improving T2I-to-I2V pipelines.
  2. The pipeline is efficient. The only finetuned module is the t2i model, with downstream video feedback from t2v model. And offline preference construction is preferable compared to online method.
  3. The results are convincing and three transfer experiments on different video models show the capability of the method.
Critical weaknesses
  1. One of the biggest concern is about lack of methodological novelty. GAL primarily applies existing diffusion preference optimization to a new supervision target. VPO[1] use downstream generated video to optimize a prompt rewriter, which GAL optimizes t2i task. ALG[2] shares a similar problem: "Text-to-video (T2V) models excel at producing high-quality, dynamic videos, and recent works have adapted these pre-trained T2V models for image-to-video (I2V) generation to enhance visual controllability. However, this adaptation often suppresses motion dynamics, yielding more static videos than their T2V counterparts." Although training the T2I generator is a meaningful distinction, it seems an incremental modification to me.
  2. The same VideoReward model provides training labels and evaluates video rewards. Human preference evaluation would strengthen the evidence beyond automated metrics.
  3. There is no visualization results. It's not straightforward for me to judge the method's effect directly.

[1] VPO: Aligning Text-to-Video Generation Models with Prompt Optimization. Cheng et al. [2] Improving Motion in Image-to-Video Models via Adaptive Low-Pass Guidance. Choi et al.

Questions for authors
  1. Please answer the Weakness 1 I proposed.
  2. Could the authors provide some visualization videos/image sets for direct comparison?
  3. A human preference evaluation is prefered.

I would raise my score if my concerns are addressed well.

4 comments
AuthorsOpenReview ↗

Looking Forward to Your Feedback

Dear Reviewer uvot,

Thank you again for your valuable comments. As the discussion period closes in two days, we would like to kindly check whether our response and revision have addressed your concerns. We would be grateful for any feedback and are happy to clarify anything further.

Best regards, The Authors

AuthorsOpenReview ↗

A Gentle Reminder on Our Response

Dear Reviewer uvot,

We apologize for following up once more. As the discussion period ends in less than 11 hours, we would be grateful if you could take a brief look at our response when convenient. Any feedback, even brief, would be very helpful, and we are happy to clarify anything promptly.

Thank you again for your time and consideration.

Best regards, The Authors

Reviewer_uvotOpenReview ↗

Not fully convincing rebuttal

Dear authors,

Thank you for the detailed response and additional experiments. The human evaluation and qualitative comparisons substantially address my concerns in W2 and W3. However, my main concern about methodological novelty remains.

I understand the biggest difference between GAL and VPO is the modeling, yet the idea inside is the same: both VPO and GAL use downstream video feedback to optimize an upstream conditioning module. GAL shifts the optimization target from a prompt rewriter to a T2I generator and applies existing diffusion preference optimization. While this is a meaningful distinction, it still appears incremental to me.

The additional results strengthen the empirical evidence, but do not sufficiently clarify the methodological novelty beyond this change. I therefore maintain my original rating.

We thank Reviewer uvot for considering our response and additional experiments. As we understand it, the reviewer recognizes that GAL and VPO make distinct modeling choices. The remaining question is whether this distinction constitutes a substantive methodological contribution.

GAL optimizes the T2I stage of the T2I → TI2V pipeline using downstream video feedback. VPO optimizes the prompt rewriter for T2V. These choices are complementary. A pipeline can learn both what to say in the prompt and what to show in the first frame.

CaseWhat changedIs it a modeling change?Parallel with GALImpact
Unified video generation and editingTasks such as T2V, I2V, and V2V can share a backbone by treating text, images, and source videos as context for denoising target video tokens.Yes. A shared conditioning formulation brings several tasks into one model.Both treat how visual conditions are modeled as part of the video generation problem.Generation and editing can share infrastructure and be offered within one system, as illustrated by Seedance 2.0 and HappyHorse 1.0.
REPA and RAE [1,2]REPA aligns DiT features while keeping the VAE fixed. RAE replaces the conventional VAE with a frozen pretrained encoder and a learned decoder.Yes. RAE changes the latent space in which the DiT learns.Both revisit an upstream component that determines what the downstream generator receives.The DiT learns in a semantically richer latent space.
ChatGPT Voice Mode and GPT‑4o [3]Earlier Voice Mode passed a transcript to the LLM. GPT‑4o integrates speech understanding and generation into one end-to-end trained model.Yes. The model learns directly from speech as well as text.Both reconsider an upstream stage in light of what the full pipeline needs to accomplish.The model can use cues such as tone that transcription can discard.

We view GAL as a substantive modeling contribution because it defines what makes a good first frame/image through the video that frame/image produces . The same idea is promising beyond GAL. Consider LLM → text prompt → prompt refinement → image → video → LLM review. Could feedback from that final review be used to train the full pipeline jointly through reinforcement learning? We think this is an exciting direction.

[1]. Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think, 2025 ICLR

[2]. Diffusion Transformers with Representation Autoencoders, 2026 ICLR

[3]. https://openai.com/index/hello-gpt-4o/

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.