acceptodds
Under review as a conference paper at ICLR 2027

Towards Recursive Self-Improvement: A Post-Training Paradigm for Video Generation

Abstract

Post-training has advanced large language models by converting feedback into effective supervision. Extending this process to video generation remains challenging: quality is multidimensional, automated evaluators imperfectly reflect human preferences, and aggregate scores provide limited guidance on what to improve next. We introduce a closed-loop, backbone-agnostic post-training paradigm that connects capability-level evaluation to targeted interventions in data collection and training. The paradigm couples data and training loops through human-calibrated evaluation, combining task-specific metrics, order-controlled pairwise comparisons, and statistically grounded checkpoint promotion criteria. Experimental outcomes guide revisions to training recipes, data mixtures, and collection priorities, while validated improvements are carried forward into subsequent iterations. This feedback-to-intervention mechanism enables adaptation of both the generator and the process used to produce its successors, offering a practical path toward system-level recursive self-improvement. We instantiate the paradigm on several open-weight and licensed video foundation models, including LTX-2.3/2.5, Wan-2.2, and MiniMax-H3. Post-trained LTX checkpoints win 88.2% and 80.6% of decisive human comparisons; Wan-2.2 reduces audio–visual offset from 6.25 to 1.69 frames. H3 achieves an 85.3% advertising reference-fidelity pass rate, the highest among four models and six frontier video-generation APIs, while remaining competitive on general video quality. These results demonstrate the applicability of the paradigm across model families and provide a foundation for studying recursive self-improvement in video generation.

Primary area:
generative models
Keywords:
video generation, post-training, recursive self-improvement, human-calibrated evaluation, MLLM evaluators, checkpoint selection, data curation, video foundation models, preference evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_jy4WOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a closed-loop post-training framework designed to tackle the multidimensional evaluation challenges inherent in video generation. By explicitly connecting capability-level feedback to targeted interventions across data collection, training, and inference, the authors propose a structured pathway toward recursive self-improvement (RSI).

Critical strengths
  • The paper addresses an important problem: visual quality, reference fidelity, action adherence, and audio synchronization can move in different directions.
  • The empirical results are quite interesting. The post-trained MiniMax-H3 model achieved an 85.3% advertising reference-fidelity pass rate, outperforming six frontier APIs. The Wan-2.2 post-trained system reduced absolute audio-visual offset from 6.25 to 1.69 frames.
Critical weaknesses
  • Benchmark coverage is too narrow. 17 advertising briefs whose prompts meet the strictest API character limit are too narrow.

  • Experiments and discussions do not show that the benchmarked prompts span the failures motivating the method, such as varied action sequences, long-range identity retention, speech timing, or audio–visual interactions.

  • The main evaluation is llm-judged, with no matching human study or human audits.

  • I find the novel contribution hard to identify. Video-T1 already uses verifier feedback to guide video-generation decisions, while prior post-training work combines data, rewards, prompting, and inference optimization. This paper’s possible distinction is using feedback to choose subsequent training interventions, but it neither specifies that policy clearly nor tests it against a comparable baseline.

  • Ablation studies or budget-comparable baselines are also missing.

  • While without clear evidence, many parts of the paper appear to be AI-generated, e.g., the Introduction. The reading flow is not very good.

Questions for authors
  • At a fixed total training and evaluation budget, how does this policy compare with a fixed multi-stage recipe, random valid proposals, and an expert using the same diagnostic reports?
  • Can the authors evaluate the final selected systems on fresh briefs using a blinded human panel, including the frontier API comparisons and the reference-fidelity threshold?
  • How often did a change that improved its diagnosed capability pass the other capability guardrails and result in an actual promotion?
1 comment
AuthorsOpenReview ↗

Training baselines, blinded human evaluation and candidate promotion

Q1. How does adaptive training compare with fixed, random and expert choices? Adaptive training led both fixed recipes on 24 held-out LTX prompts at the same training and evaluation budget. In paired video comparisons judged by Gemini 3.8 Flash, its preference score was 54.2% against the fixed multi-stage schedule and 52.1% against the constant recipe, with half credit for ties.

Comparison with adaptive trainingSeed 1Seed 2Pooled preference for adaptive [95% CI]
Fixed multi-stage schedule54.2%54.2%54.2% [43.8, 65.6]
Constant recipe56.2%47.9%52.1% [40.6, 63.5]
Random valid proposals58.3%50.0%54.2% [41.7, 66.7]

Each seed contributes 24 comparisons, so the two scores against random proposals average to the same value as the fixed schedule.

All methods started from the same LTX-2.3 weights and trained on the same 64 clips. We used two training seeds. Each run used 1,024 updates, 32 development videos and 64 development ratings. For each seed the judge saw all 24 test video pairs in both orders. Both 95% intervals against fixed recipes include 50%, so the lead on new prompts remains uncertain (Appendix R).

For the expert comparison, an author chose to lower the learning rate. We tested that choice for one 64-update round on each seed and retained the existing model both times. A full-budget expert comparison remains open (Appendix P).

Relation to Video-T1. Video-T1 uses verifier feedback to choose a generation path for the current video. We use feedback from one training run to decide what to change in the next. For H3, we found that the same video received inconsistent scores. We changed the training reward to use repeated rankings of videos for the same prompt. The post-trained model then scored 5.40 points above base on the original Gemini 3.5 Flash checklist across 20 briefs (Appendix F). We have revised the introduction to make this distinction clear and added an AI-use disclosure. Video-T1

Q2. What did the blinded comparisons show on fresh briefs? After selecting the post-trained H3 model and fixing the evaluator, we collected new human judgments. H3 won 5 of six comparisons against Seedance 2.5 (one tie) and 4 of six against H3 Max. Each API had its own six briefs. The full set of 24 new briefs includes objects disappearing and reappearing and speech timed to an action.

The paper’s 85.3% fidelity pass rate comes from a model evaluator giving at least 7/10 on 29 of 34 ratings. People judged overall preference in the blinded study. Human calibration of the 7/10 cutoff remains open. Video comparisons

How does H3 fare beyond advertising? On 238 general image-to-video prompts with audio, people chose post-trained H3 in 57.7% of 3,149 votes across 25 comparison models. In this blinded human review, 113 raters saw two videos made from the same reference and prompt with the model names hidden. Against Seedance 2.5, the vote was nearly even. People favored H3 over Veo 3.1 and Kling 3.0 Pro. Some votes from this larger study later informed evaluator design. Appendix Q reports all 25 comparisons with uncertainty by prompt.

Comparison on general promptsVotes for H3 / totalHuman preference [95% CI]
All 25 comparison models1,817 / 3,14957.7% [55.2, 60.2]
Six frontier models from the advertising study336 / 73445.8% [41.8, 49.8]
Seedance 2.556 / 11548.7% [39.8, 57.5]
Veo 3.186 / 14559.3% [50.4, 68.2]
Kling 3.0 Pro (1080p)83 / 12268.0% [59.1, 76.3]

Both post-trained H3 models also scored above 65% against their base in a separate blinded comparison on the same 29 briefs. Ties count half.

Model compared with the H3 baseBriefsHuman preference [95% CI]
Post-trained H3 (model 1)2967.2% [56.9, 77.6]
Post-trained H3 (model 2)2965.5% [55.2, 75.9]

Q3. How often did an improvement lead to promotion? 1 of 22 candidates (4.5%) passed the other checks and replaced the incumbent after an initial gain on the LTX product-video test. The LTX study targeted product-video quality. We counted a gain when a candidate won more comparisons than it lost against the incumbent on its first test. Of those 22 candidates, 17 had complete checks for visual quality, prompt following and lip sync. 15 of 17 (88.2%) passed all three on every fully scored test, including retests. Each candidate counts once (Appendix I).

Reviewer_ZmXeOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes a backbone-agnostic post-training paradigm that connects capability-level evaluation to targeted interventions in data collection and training, which may cultivate a practical path toward system-level recursive self-improvement.

Critical strengths

This paper propose an important practical problem in video post-training. Treating quality, reference fidelity, instruction adherence, synchronization, and serving constraints as separate diagnostic axes is sensible. Video generation lacks a general-purpose verifier, and a single scalar score provides little guidance on what to improve next. Treating evaluator reliability itself as a subject of diagnostic analysis is a sound perspective.

Critical weaknesses

1、Figures 1 and 2 appear to be AI-generated raster images, and their presentation lacks professionally diagrams expected of a formal research paper. Although this paper focuses on video generation, it does not provide any video examples. This makes it difficult to assess the method’s effectiveness and leaves the empirical claims insufficiently substantiated.

2、To demonstrate the effectiveness of the proposed post-training paradigm, the paper should evaluate the resulting models using standard video generation metrics and established general-purpose benchmarks. What conclusions can be drawn from evaluations restricted to a few narrow tasks? In particular, the relevance of “decisive human comparisons,” “audio–visual offset,” and “advertising reference-fidelity pass rate,” mentioned in lines 25–26 of the abstract, needs clarification. It seems like narrow direction assessment. How do these measures support the broader claims about the proposed paradigm?

Questions for authors

The metrics reported in Tables 1 and 2 do not appear to be adequately explained in the main text. What does these metrics mean? Clear definitions of the metrics, stronger methodological explanations, and more careful writing would improve its clarity and credibility.

1 comment
AuthorsOpenReview ↗

Video comparisons, standard benchmarks and the metrics in Tables 1 and 2

Video examples and figures. You can now compare 49 video pairs side by side. Each pair includes the original brief, reference and model names. We also redrew Figures 1 and 2 as vector diagrams. Figure 1 shows who makes each decision, and Figure 2 follows the decisions through actual experiments. Watch the selected video pairs

What do the 17 advertising briefs tell us? They test whether H3 preserves the person or product in a supplied image. This matters for an ad even when the video looks convincing overall. To see how post-trained H3 fares on everyday scenes, we also report a blinded human review of 238 general image-to-video prompts with audio. Each pair used the same reference and prompt. Across the 25 comparison models, raters chose H3 in 57.7% of 3,149 votes. Against Seedance 2.5, 56 votes went to H3 and 59 to Seedance (Appendix Q).

Comparison on general promptsVotes for H3 / totalHuman preference [95% CI]
All 25 comparison models1,817 / 3,14957.7% [55.2, 60.2]
Six frontier models from the advertising study336 / 73445.8% [41.8, 49.8]
Seedance 2.556 / 11548.7% [39.8, 57.5]

Standard benchmarks. On 32 official VBench prompts from eight categories, people preferred post-trained LTX to base. VBench also recorded more motion in the post-trained videos and the same motion smoothness. Appendix N gives the full set of dimensions we measured.

MeasurePost-trained LTX compared with base
Blinded human preference on the 32 VBench prompts79.7% (95% CI 67.2–90.6%). Ties count half.
VBench dynamic degree0.875 vs. 0.813
VBench motion smoothness0.992 for both
MUSIQ image quality on 40 matched advertising inputs+3.13 points [1.01, 5.10]

How to read Tables 1 and 2. We added definitions alongside the results. An overall preference tells us which video a viewer would choose. A capability measure identifies something we can change in training. For Wan, that was the timing of speech and mouth movement. After synchronization training, the mean absolute error fell from 6.25 to 1.69 frames on the 16 pairs we could score.

MeasureWhat it measures
Decisive preferenceWins divided by wins plus losses. The original four-way human test counted how often a viewer chose one of three post-trained variants over base among non-tied choices. In the paired model test, a win requires agreement in both viewing orders.
Audiovisual offset and LSE-COffset is the absolute gap in frames between speech and mouth movement. LSE-C is SyncNet’s confidence in their alignment.
Reference-fidelity pass rateThe share of model ratings of at least 7/10 for preserving identity, shape and markings. H3 passed 29 of 34 ratings on the 17 briefs (85.3%).
Brief adherence and defectsThe share of outputs meeting the brief’s requirements, and the number of visible defects per clip.
Quality and motionMUSIQ, CLIP-IQA+ and Q-Align estimate perceptual quality. LAION aesthetics measures visual appeal. Warping measures deformation, and the optical-flow score measures steadiness.
Time and costGeneration time and cost per second of output video. The cost index expresses that cost relative to the median API cost.
Reviewer_t2XpOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents a closed-loop post-training framework for video generation. Capability diagnostics and experiment history guide data acquisition, evaluator revisions, training recipes, and inference optimization, while comparisons against the incumbent determine which candidates are retained. Experiments on LTX-2.3/2.5, Wan-2.2, and MiniMax-H3 document reward revision, synchronization training, candidate confirmation, and serving optimization. The strongest local results include 25 wins, 10 ties, and 5 losses for a fixed LTX-2.3 checkpoint under model-based evaluation; a reduction in mean audiovisual offset from 6.25 to 1.69 frames on 16 scoreable Wan pairs; and a 5.40-point H3 acceptability gain after reward revision under the same enhanced prompts. I view the contribution as a human-in-the-loop adaptive post-training system, with recursion primarily implemented through feedback-dependent intervention choices.

Critical strengths

Feedback is connected to inspectable subsequent interventions. The paper goes beyond before-and-after scores and explains why a subsequent intervention was chosen. For H3, repeated ratings of the same clip have variance 0.0178, comparable to the 0.014–0.016 variation within rollout groups. This motivates replacing a scalar reward with repeated, order-shuffled listwise judgments. With enhanced prompts held fixed, the step-40 candidate gains 5.40 points, with a paired interval of [1.33, 9.93]. This is a useful, concrete example of diagnosing supervision before modifying training. The LTX candidate that passes an initial screen but fails on a second seed also makes the selection account more credible.

Critical weaknesses

The central adaptive policy lacks an attributable overall evaluation. The paper establishes that interventions follow feedback and that several interventions help, but this does not establish that the history-dependent policy Π is more effective than a fixed workflow, history-free proposals, or a human workflow with the same budget. The 84 training runs and replay of a retention decision do not supply this counterfactual; increased data supply is also not paired with subsequent model gains. Appendix E acknowledges that curriculum comparisons change initialization, learning rate, and budget, while Appendix I assigns objectives, evaluator revisions, and release decisions to researchers. I therefore accept that the system is adaptive, but the additional benefit of the adaptive policy over a fixed workflow has not been isolated.

Questions for authors

Q1: Can you compare the adaptive policy with a fixed workflow or a policy without experiment history, using the same data sources, total training/generation budget, and final test set? Please identify which interventions were chosen by the proposer and which by researchers; this directly affects my assessment of the recursive mechanism.

Q2: Are results available on new briefs and blinded human judgments unused for evaluator selection, reward revision, or checkpoint screening? Please distinguish complete-system gains from weight-only gains with prompts and serving implementation fixed, and report paired uncertainty over tasks.

1 comment
AuthorsOpenReview ↗

Matched-budget training comparison and blinded human evaluations

Q1. Does adapting the training recipe help at the same budget? Adaptive training led both fixed recipes on 24 held-out LTX prompts at the same training and evaluation budget. In paired video comparisons judged by Gemini 3.8 Flash, its preference score was 54.2% against the fixed multi-stage schedule and 52.1% against the constant recipe, with half credit for ties.

Comparison with adaptive trainingPreference for adaptive [95% CI]
Fixed multi-stage schedule54.2% [43.8, 65.6]
Constant recipe52.1% [40.6, 63.5]
Same proposer using only the latest development report49.0% [38.5, 59.4]

All methods started from the same LTX-2.3 weights and trained on the same 64 clips. We used two training seeds. Each run used 1,024 updates, 32 development videos and 64 development ratings. For each seed the judge saw all 24 test video pairs in both orders. Both 95% intervals against fixed recipes include 50%, so the lead on new prompts remains uncertain (Appendix R).

The adaptive proposer could change the learning rate or training captions. In both runs, adaptive and constant training selected models from different training stages, even though all four selected models had used the starting recipe. We held data collection, reward design and inference fixed in this comparison.

Who made the decisions? In the broader framework, the automated proposer suggested changes to the training recipe or data mix. The supervisor tested candidates against the current model. For example, one LTX candidate won eight comparisons and lost two on its first training seed. On a second seed it won four and lost seven, so the supervisor kept the current model. Researchers set the objectives, revised the evaluators and approved replacements. For data collection, we measured the number of relevant clips recovered. We have yet to measure the effect of those clips on the next model.

For H3, repeated scores for a single video varied almost as much as scores across different videos for the same prompt. We therefore asked the evaluator to rank eight videos together, repeating the ranking four times with their order shuffled. On the same 20 advertising briefs, the resulting post-trained H3 scored 5.40 points above base on the original Gemini 3.5 Flash acceptability checklist (Appendix F).

H3 training methodAcceptability (out of 100)Gain over base [95% CI]
Post-trained H3, repeated rankings70.33+5.40 [1.33, 9.93]
Post-trained H3, one score per video65.13+0.20 [-9.10, 9.50]
H3 base, same prompts64.93Reference

Q2. What did people prefer on briefs held out from model and evaluator selection? On 32 new VBench prompts, post-trained LTX earned 79.7% preference over base in a blinded human review (95% CI 67.2–90.6%, ties count half). Only the model weights changed. We used the same prompts, seeds and inference settings for both models.

We also compared the complete post-trained H3 system with two APIs on new briefs. Human preference for H3 was 91.7% against Seedance 2.5 (95% CI 75.0–100.0%) and 66.7% against H3 Max (33.3–100.0%), with six different briefs per API. We collected these judgments after fixing the models and evaluation rules. Appendix Q gives all blinded comparisons, including H3 against base.

To separate the contribution of the weights from the complete system, we compared each with base on the same 24 new briefs. The table gives the paired change in the model evaluator’s 0–10 score for preserving the reference image (Appendix O).

H3 comparison on 24 new briefsPaired change in reference fidelity [95% CI]
Complete post-trained system vs. base+0.25 [+0.00, +0.60]
Weights only, with prompts and inference settings fixed+0.10 [-0.02, +0.21]

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.