acceptodds
Under review as a conference paper at ICLR 2027

TTPE: Test-Time Policy Evolution for Agentic Text-to-Video Refinement

Abstract

A single text-to-video (T2V) generation rarely satisfies every semantic requirement of a challenging prompt, so recent work spends additional video generations at test time on iterative generate–evaluate–refine loops. Existing methods refine each prompt in isolation under fixed strategies, repeatedly paying for video synthesis while discarding the experience accumulated from previous refinement episodes. We instead cast refinement as a budget-constrained sequential decision problem: each action shapes both the next video and the budget left for later correction. Conventional reinforcement learning is ill-suited here, as it requires costly video-generation rollouts and yields policies that must be retrained for each new generator. We introduce **TTPE (Test-Time Policy Evolution)**, a training-free framework that accumulates completed refinement trajectories in an episodic memory and reuses them to decide whether another generation is worth spending and which refinement action to take. From similar past trajectories, TTPE estimates two decoupled long-horizon advantages per action family: a quality advantage capturing expected future quality gain, and a cost advantage capturing expected future generation cost. Rather than collapsing them into a single objective, TTPE resolves their trade-off only at decision time, pruning Pareto-dominated actions in the quality–cost space and selecting among the survivors under the remaining budget. Across VBench-2.0 and StoryEval with three video generators, TTPE consistently improves the quality–cost frontier over prior iterative refinement, achieving the best aggregate quality in every evaluated generator–benchmark setting, with gains of up to 5.38 points and generation-cost reductions of up to 65.5% relative to fixed-round refinement. Code and implementation details are available at: https://anonymous.4open.science/r/ttpe-C245/

Primary area:
generative models
Keywords:
Text-to-Video Generation, Agentic Generation, Test-Time Compute, Long-Horizon Optimization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_d8gUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TTPE, a training-free method for iterative text-to-video refinement. TTPE stores previous refinement trajectories, retrieves similar cases, estimates the expected quality improvement and generation cost of each action, and uses these estimates to decide whether and how to refine the current video.

The problem is important, and reusing previous test-time experience is a sensible idea. However, the mathematical formulation has several fundamental problems. In particular, the reward does not match the final objective, and the quantities called “advantages” are not valid policy advantages. Therefore, the MDP and reinforcement-learning interpretation is not well supported.

Critical strengths
  • The paper addresses the practical problem of allocating test-time generation compute.
  • Reusing completed refinement trajectories is intuitive and potentially useful.
  • The method does not require training the video generator.
  • The overall pipeline is easy to implement.
Critical weaknesses
  • The system ultimately returns the video with the highest quality observed during the episode. However, the reward is defined as rt=qt+1−qtmax⁡r_t=q_{t+1}-q_t^{\max}. This penalizes temporary regressions even though such regressions do not affect the final delivered video. For example, suppose the initial quality is 5. A refinement first produces a video with quality 3 and then one with quality 6. The final output improves from 5 to 6. Nevertheless, the two rewards are 3−5=−2,6−5=13-5=-2,\qquad 6-5=1, so the total return is negative, even before discounting. TTPE therefore considers a successful trajectory worse than immediately stopping.
  • The paper estimates (Q(s,a)) by averaging the returns of retrieved historical states where action (a) was selected. But actions in the memory were not randomly assigned. They were selected by the base agent and earlier versions of TTPE. Therefore, different actions are observed in different types of states. For example, “reword” may appear successful simply because the base agent selects it for easier failures, while “resample” may appear unsuccessful because it is used in difficult cases. Subtracting the average return of all neighbors does not remove this selection bias.
  • Equation (3) defines the state using the current prompt, scalar quality, best quality, and remaining budget. It does not include the current video, the individual failed requirements, or the refinement history. However, the effect of an action clearly depends on these missing variables. Two videos can have the same prompt and quality score while failing completely different requirements, requiring different refinement actions.
  • Every non-stop action has exactly the same immediate cost: one video generation. The “cost return” instead measures how many future generations were used after that action in historical trajectories. This future cost depends mainly on the historical policy’s later stopping decisions. Since the policy evolves as memory grows, these historical costs do not estimate the future cost under the current policy.
Questions for authors

Equation (11) combines quality and cost using the remaining-budget fraction:

ρtAQ+(1−ρt)AC\rho_t A^Q+(1-\rho_t)A^C.

No optimization argument shows that this weighting solves the stated budget-constrained problem. The hard budget is already enforced by the episode horizon, so the connection between this rule and optimal budget allocation is unclear.

5 comments
AuthorsOpenReview ↗

Response to Reviewer d8gU (1/3)

Thank you for recognizing that the problem is important, that reusing previous test-time experience is a sensible idea, and that our method is easy to implement. Your review raises five issues: the optimization justification of the budget-conditioned weighting (Eq. 11), negative returns and premature stopping, selection bias in the retrieval-based Q estimate, the state representation in Eq. 3, and the role of the cost return. We address them in that order.

Q. Optimization Justification of the Budget-Conditioned Weighting (Eq. 11)

We answer this question from two aspects: why a selection rule is needed for stopping even though the horizon enforces the hard budget, and an analysis of the weight ρ\rho.

1. The horizon enforces feasibility, not allocation. The horizon only guarantees that no prompt exceeds BB generations; any policy that runs until the horizon is feasible. Our objective is to reach high quality with as few generations as possible, so the key decision is when to stop, which the horizon does not determine. Without the cost term, the agent keeps refining and consumes most of the budget (Table 3). Table A reports how many episodes end by the stop action and how many end by the budget. On all three video models, about 70% of episodes end by the stop action, using only a fraction of the budget (e.g., 5.8 of 20 generations for Wan-2.1), and almost all of them stop after all prompt elements are satisfied; only 1.7%–4.6% stop with unmet elements.

Table A. Episodes ending by stop action vs. budget on StoryEval (423 prompts).

Video ModelBudgetEnded by Stop (all met)Ended by Stop (unmet)Ended by BudgetAvg. Gen. (Ended by Stop)
Wan-2.12065.9%4.6%29.5%5.8
HunyuanVideo1065.6%2.3%32.1%3.7
MiniMax-H3470.4%1.7%27.9%1.7

2. Parameter analysis: a budget-dependent ρt\rho_t is necessary. Table B compares fixed and dynamic weights. ρ=1\rho = 1 (quality only) reaches similar completion but costs 6.30 generations per prompt; ρ=0\rho = 0 (cost only) reduces generations to 3.97 but drops to 41.17%. A fixed ρ=0.5\rho = 0.5 is worse on both axes (45.36%, 5.75 generations). Only the budget-conditioned ρt\rho_t achieves both high completion (47.42%) and low cost (4.47).

Table B. Effect of the weight ρ\rho in Eq. 11 (StoryEval).

ρ\rhoCompletion ↑Avg. Generations ↓
ρ=0\rho = 0 (ACA_C only)41.173.97
ρ=1\rho = 1 (AQA_Q only)47.586.30
ρ=0.5\rho = 0.545.365.75
ρt\rho_t (Ours)47.424.47

We have added these results to the revision.

AuthorsOpenReview ↗

Response to Reviewer d8gU (2/3)

W1. Negative Returns and Premature Stopping

Reward design motivation. In test-time generation, only improvements beyond the current best video yield a better output for the user, whereas recovering from regressions consumes budget without improving the delivered result. Our reward therefore credits actions that establish a new best quality. Since the delivered output is always the best video observed, temporary regressions never degrade the final result.

Credit is assigned per action, not per trajectory. The advantage of each action is computed from its return-to-go, i.e., the rewards from that action onward, so earlier regressions never penalize later improving actions. In the reviewer's example 5→3→6, the rewards are −2 and +1, and the returns-to-go are −1 and +1. The action that reaches 6 receives +1, exactly as if it had been reached directly from 5. Only the action that caused the regression receives negative credit, reflecting the extra generation spent on the detour. In practice such detours are usually far longer, making recovery expensive (Table C).

Table C. Regression-and-recovery statistics on StoryEval.

Video ModelR&R Ratio (%)Avg. Gen. to Recover 1 Point
Wan2.116.356.36
HunyuanVideo10.874.89
MiniMax-H34.022.63

Negative returns do not induce stopping.

(i) No within-episode effect. Transitions and their return-to-go values are committed to memory only after the episode terminates, and the retrieval snapshot excludes the current episode and prompt (Alg. 1, lines 4 and 15).

(ii) Memory only re-ranks the agent's candidates. All candidate actions, including stop, are proposed by the frozen agent πbase\pi_{\text{base}} based on its critique of the current video (Alg. 1, line 5). Retrieved trajectories only contribute normalized, clipped, and evidence-gated advantages (Eqs. 7–9); without supporting neighbors the advantages are zero and the agent's choice stands. Actions are compared by relative merit (Eqs. 10–11), not by the sign of any individual return.

(iii) Stopping before the goal under a limited budget. Stop is only selected when the frozen agent proposes it as a candidate. As the remaining budget decreases, Eq. 11 assigns greater weight to cost advantages, making early stopping more competitive when further refinement offers insufficient quality gains relative to its cost.

Result. Only 1.7%–4.6% of episodes stop with unmet elements (Table A), so premature stopping is rare in practice.

W2. Selection Bias in the Retrieval-Based Q Estimate

We agree that the actions in memory are not randomly assigned, and we do not claim that Eqs. 7–8 are unbiased causal estimates: as stated in Sec. 2.4, they are empirical predictions rather than values under a fixed policy. The failure mode you describe matters only if returns earned in easy states are credited to the same action in hard states; retrieval limits this transfer, as verified below.

Action values are estimated from the KK most similar states, not from all history. For the current failure state, TTPE first retrieves the most similar historical states. The retrieval document dtd_t (Alg. 1, line 3) encodes the current score qtq_t, the remaining budget ρt\rho_t, and the element-level feedback on the specific unmet elements (see the revised Eq. 3 in W3). It then estimates Q(s,a)Q(s,a) only from the neighbors whose action family matches aa (Eq. 7). Easy Reword cases that are dissimilar to the current failure therefore do not enter the Reword estimate for a hard state merely because they once achieved positive returns.

Empirical check on the two most frequent actions. We compare Reword and Switch_repr, the two most frequent actions (Fig. 5), in severe-failure states (qt=0q_t = 0) where both were valid candidates (N = 482). The states in which each action was selected are comparable in prompt complexity, remaining budget, and step index, so neither action is reserved for easier cases. Their realized outcomes also show no significant difference on either the quality or the cost axis (Table D).

Table D. Reword vs. Switch_repr in zero-score decisions where both were valid (N = 482).

RewordSwitch_reprDifference [95% CI]
State
Number of elements mm2.732.75−0.03
Remaining budget ρt\rho_t0.670.68−0.01
Step index4.003.89+0.11
Realized outcome
Quality return GQG^Q−1.34−1.32−0.02 [−0.55, 0.46]
Generation cost GCG^C4.634.68−0.04 [−0.55, 0.43]
Immediate improvement rate31.4%31.8%−0.4

The remaining bias has bounded influence. As noted in W1 (ii), the advantages are clipped and evidence-gated (Eq. 9) and only correct the base-policy prior (Eqs. 10–11), so when the retrieved evidence is weak, the frozen agent's own choice dominates.

AuthorsOpenReview ↗

Response to Reviewer d8gU (3/3)

W3. State Representation in Eq. 3

Eq. 3 is a compact notation, not a complete list of what the policy observes. As reflected in our Fig. 2 and code, the policy does receive the current video, the textual feedback on each element (which elements are unmet and why), and the textual history of previous refinements. We have revised Eq. 3 to

st=(xt, vt, et, ht, qt, qtmax⁡, ρt),s_t = \big(x_t,\ v_t,\ e_t,\ h_t,\ q_t,\ q^{\max}_t,\ \rho_t\big),

where vtv_t is the current video, ete_t is the element-level textual feedback on vtv_t, and hth_t is the textual history of previous refinement steps.

W4. Role of the Cost Return

The cost return is not intended as an unbiased estimate of future generation cost under the current policy; it is a relative signal for comparing candidate actions.

All non-stop actions share the same immediate cost of one generation, so they can only be distinguished by how many further generations they lead to. The cost return captures this from historical trajectories. Although it is affected by the later stopping decisions of the historical policy, this effect is shared by all candidates: their estimates are drawn from the same retrieved neighborhood, and the advantage AC=VC−QCA_C = V_C - Q_C (Eq. 8) subtracts the common state baseline. The comparison is therefore consistent across actions, even though the absolute values are not predictions under the evolving policy.

Empirically, this relative signal is informative. Fig. 3 shows that decisions with higher predicted cost incur larger realized generation costs on all three video models, and the cost advantage reduces generations from 6.30 to 4.47 at comparable completion (Table 3).

Summary

The new evidence addresses the concerns at the center of your review: the budget-conditioned weighting in Eq. 11 is supported by its experiments against fixed weights, the reward assigns credit per action and does not induce premature stopping, action values are estimated from the KK most similar states and show no selection bias between the most frequent actions, the state definition in Eq. 3 has been updated, and the cost return provides a consistent relative signal that is reflected in realized generation costs. The revised version includes these experiments and the clarifications above, with all changes highlighted in blue. Thank you again for the constructive assessment and specific requests. We hope the new results and clarifications support a stronger acceptance rating.

Reviewer_wUcwOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper approaches refinement task as a budget constrained sequential decision problem where each action helps to shape both the next video and budget left for correction later as opposed to fixed strategies. They propose TTPE as a training free framework that collects completed refinement trajectories and resumes them to decide whether another generation is worth spedning and which actions to take.

Critical strengths

Method is well motivated and paper is well written.

Method is training free and has negligible overhead, presenting a practical solution.

Experimental comparisons are detailed and supporting the method’s insights.

Code is provided for exploration.

Critical weaknesses

How likely results depend on prompt order is not clear, and while authors stated they reset memory per vbench dimension, this seem’s to favor within-dimension similarity.

Ablations seems to be limited and key parameters seems to be not ablated (e.g. K=10)

Several details from ablations seems to be omitted e.g. what is “Effective Exploration” in Table 4?

Reward hacking risk is not discussed.

Questions for authors

L429 references to Fig 7 for action distributions, but I think it is supposed to refer to Fig 5?

What is “Nreward” in Fig 4?

5 comments
AuthorsOpenReview ↗

Response to Reviewer wUcw (1/2)

Thank you for recognizing that TTPE is well motivated, practical as a training-free method with negligible overhead, and supported by detailed experimental comparisons. We first answer your questions about the figures and then address the four weaknesses: prompt order sensitivity, ablations of key parameters, details of the ablation metrics, and reward hacking risk.

Q. Figure Reference and Method Label Typos

Figure reference at L429. Thank you for catching this typo. The reference should be Fig. 5, and we have corrected it in the revision.

"NReward" in Fig. 4. This is also a typo: "NReward" refers to our method, TTPE. We have corrected the label in Fig. 4 of the revision.

W1. Prompt Order and Per-Dimension Memory Reset

We applied no ordering or selection to the prompts: all reported results use the original order released by each benchmark. To measure order sensitivity, we further test random, long-to-short, and short-to-long orderings (by prompt length) with Wan-2.1 on three streams of different sizes: Composition, the VBench-2.0 dimension with the fewest prompts (53); Camera Motion, the dimension with the most prompts (108); and the full StoryEval benchmark (423) (Table A).

Table A. Effect of prompt order (Wan-2.1).

OrderComposition (53)Camera Motion (108)StoryEval (423)
Original (reported)67.1746.3037.47
Random64.4548.1537.30
Short → Long65.8046.7137.93
Long → Short71.2651.3238.70
Mean ± std (3 alternative orders)67.17 ± 3.6148.73 ± 2.3637.98 ± 0.70

Order sensitivity decreases with benchmark size. The reported original order is not the best on any benchmark, and no alternative order lowers TTPE's ranking among the compared methods. Long-to-short consistently achieves the best results across all three benchmarks, suggesting that experience from longer, more detailed prompts may transfer effectively to shorter prompts. We have included this analysis in the revision.

Memory reset. On VBench-2.0, each dimension is scored by a different official evaluator, so we reset memory per dimension to keep their evaluations independent. This is not required by TTPE: on StoryEval, memory is never reset and all mixed prompts run as a single stream. We have clarified both settings in the revision.

W2. Limited Ablations and Key Parameters (e.g., K=10K=10)

Beyond the ablations on the dual advantages and Pareto pruning (Table 3) and memory utilization (Table 4) in the main text, the appendix also studies the language-model backbone and the size of the retrievable memory under cross-benchmark transfer. We have further added ablations of two key hyperparameters: the number of retrieved neighbors KK (Eq. 7) and the quality-cost weight ρ\rho in the selection score (Eq. 11).

1. Number of retrieved neighbors KK. K=1K=1 draws on too little history to be a reliable reference and easily biases the estimate, while K=20K=20 introduces irrelevant states. The best KK may vary across video models within this range (e.g., K=5K=5 for Wan-2.1), but we did not tune KK per model and use the same K=10K=10 for all benchmarks and video models (Table B).

Table B. Effect of the number of retrieved neighbors KK (VBench-2.0 Composition).

Video ModelK=1K=1K=5K=5K=10K=10 (Ours)K=20K=20
Wan-2.155.8271.5467.1759.57
HunyuanVideo52.7759.0662.9956.63
MiniMax-H358.8263.0871.0462.18
Average55.8064.5667.0759.46

2. Quality-cost weight ρ\rho. Compared with fixed weights, our dynamic ρt\rho_t achieves the best trade-off between completion and generation cost (Table C).

Table C. Effect of the weight ρ\rho in Eq. 11 (StoryEval).

ρ\rhoCompletion ↑Avg. Generations ↓
ρ=0\rho = 0 (ACA_C only)41.173.97
ρ=1\rho = 1 (AQA_Q only)47.586.30
ρ=0.5\rho = 0.545.365.75
ρt\rho_t dynamic (Ours)47.424.47

We have added both ablations to the revision.

W3. Details of the Ablation Metrics

We have added these definitions to the captions of Tables 3 and 4.

Table 4.

  • Effective Exploration: a step whose score exceeds the best score of all previous steps, i.e., rt>0r_t > 0 in Eq. 5.
  • Repeated Failure: a refinement step after which a previously unmet element remains unmet.
  • Reward-based Memory: ranks candidates by the immediate rewards rtr_t of retrieved neighbors instead of the advantages A^Q\hat A_Q and A^C\hat A_C.

Table 3.

  • Conflict: a candidate whose A^Q\hat A_Q and A^C\hat A_C have opposite signs, i.e., better on one axis but worse on the other. Both advantages are always estimated; the ablated variants differ only in which advantage is used for selection.
  • Candidate Conflict / Selected Conflict: the fraction of conflicting candidates among all candidates / among selected actions.
AuthorsOpenReview ↗

Response to Reviewer wUcw (2/2)

W4. Reward Hacking Risk

Analysis following standard practice. To detect reward hacking, a standard practice is to optimize with one evaluator and assess the results with another evaluator that is not involved in the optimization [1]. We therefore score every TTPE video (HunyuanVideo, StoryEval) under the official StoryEval protocol with Qwen3.8-27B, a VLM that was never involved in optimization, so the two evaluators share neither the model nor its scoring tendencies. The StoryEval score rises together with the internal score, without the late decline that indicates over-optimization (Table D).

Table D. Internal and StoryEval scores of the video selected by the internal score as refinement proceeds.

Refinement iterations so far013579
Internal score40.458.974.882.385.988.2
StoryEval score37.848.457.461.563.064.5

Judge design grounded in prior evidence. Our internal judge (Gemini-3.1 Pro) follows the widely used decomposed binary VQA paradigm [2]: each prompt is decomposed once, before any generation, into frozen atomic requirements, and each requirement receives an independent binary verdict on a concrete, checkable question rather than a holistic score. Because the requirements are fixed before refinement, the evaluation target cannot shift across iterations, which reduces opportunities to exploit the judge. PRIS [3] evaluates verifiers against human-verified selections and finds that this paradigm is at least as accurate as reward models trained on human preference data (Table E, reproduced from PRIS Table 5), without requiring preference-based training.

Table E. Verifier accuracy in selecting ground-truth videos (PRIS, Table 5).

VerifierAccuracy ↑
VisionReward0.571
UnifiedReward0.498
VideoAlign0.693
Decomposed binary VQA (used in TTPE)0.700

Human evaluation against baselines. To check whether satisfying the atomic requirements comes at the expense of overall video quality, we randomly sample 50 prompts from our experiments and ask three practitioners in video generation to compare the final videos of TTPE and each baseline generated by the same video model. Given the prompt, raters judge which video is better overall; method names are hidden and the video order is randomized. TTPE is preferred over every baseline (Table F). We have added this evaluation to the revision (Appendix I) and will release all generated videos of TTPE and the baselines for public inspection.

Table F. Human pairwise preference (50 prompts, 3 raters).

ComparisonTTPE wins ↑TieBaseline wins
TTPE vs. PRIS58%19%23%
TTPE vs. VQQA61%18%21%
TTPE vs. VISTA56%20%24%
TTPE vs. VPO63%17%20%

References

[1] Ma, N., Tong, S., Jia, H., Hu, H., Su, Y.-C., Zhang, M., Yang, X., Li, Y., Jaakkola, T., Jia, X., and Xie, S. Scaling Inference Time Compute for Diffusion Models. In CVPR, 2025.

[2] Singh, J. and Zheng, L. Divide, evaluate, and refine: Evaluating and improving text-to-image alignment with iterative vqa feedback. In NeurIPS, 2023.

[3] Kim, Subin, et al. Rethinking prompt design for inference-time scaling in text-to-visual generation. In CVPR, 2026.

Summary

The clarifications and additional analyses address the concerns raised in your review: results are reported under the original benchmark order, with order sensitivity analyzed and per-dimension memory reset clarified; the added ablations of KK and ρ\rho show that a single untuned K=10K=10 works across models and that the budget-conditioned ρt\rho_t achieves the best quality-cost trade-off; the metrics in Tables 3 and 4 are now defined; optimizing the internal reward also improves an independent evaluator, and human raters prefer TTPE over all baselines, with no clear sign of reward hacking observed; and the figure reference and method label have been corrected. The revised version includes these experiments and clarifications, with all changes highlighted in blue. We sincerely appreciate your positive assessment of our presentation and soundness. We hope that the additional experiments, analyses, and clarifications provided in our response adequately address the concerns raised, and we respectfully ask you to reconsider the rating in light of these new results.

AuthorsOpenReview ↗

Looking forward to your feedback

Dear Reviewer wUcw,

Thank you sincerely for your thoughtful and constructive feedback. We truly appreciate your recognition of the motivation, practicality, and experimental evaluation of our work. We would be very grateful if you could spare some time to revisit our responses.

We fully understand that you have many responsibilities, and we sincerely appreciate your time and consideration. To facilitate your evaluation, we have summarized the main revisions and additional analyses as follows:

  • We have conducted additional experiments with different prompt orderings to investigate order sensitivity and clarified the rationale for resetting memory across VBench dimensions (Table A).

  • We have added ablation studies on the number of retrieved neighbors and the quality-cost weighting, further validating the robustness of our method across different settings (Tables B and C).

  • We have clarified the definitions of the ablation metrics, including Effective Exploration, Repeated Failure, and Candidate Conflict, to improve the reproducibility and interpretability of our results (Tables 3 and 4).

  • We have conducted additional analyses to investigate potential reward hacking, including evaluation with an independent VLM and human preference comparisons against all baselines (Tables D, E, and F).

  • We have also corrected the figure reference and method label typos identified in your review.

These additional experiments, analyses, and clarifications have been incorporated into our revised manuscript.

We sincerely appreciate the time and effort you have devoted to reviewing our work. We are eager to ensure that we have adequately addressed your concerns. Please feel free to let us know if you have any additional questions or concerns, and we would be more than happy to discuss them.

Should you find that our responses have satisfactorily addressed your main concerns, we would sincerely appreciate your reconsideration of our paper and its overall rating in light of our responses.

Thank you again for your time and consideration.

Best regards,

The Authors

Reviewer_HKi1OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper makes text-to-video result better many times by training-free method, TTPE. It splits prompt to small condition one time, and judge counts how many condition is satisfied. When one try is finished, it saves the try to memory with quality and number of generation after that. So when new situation comes, it finds similar past situation and checks which action is good for quality and cost. And it removes the action that is worse in both quality and cost, and it choose action by confidence and left budget. It doesn't change model weight. It achieved positive score on VBench-2.0 and StoryEval with three generator. Also number of generation is much more lower than VISTA.

Critical strengths
  1. I think the biggest advantage is that the score is more higher with less video generation. For example, in HunyuanVideo StoryEval, generation number drops from 4,210 to 2,410 compared to VISTA. I think this is very practical contribution.
  2. The authors marked memory part very detail. Memory got much better than no memory and reward-based memory, and it also helps in other benchmark.
  3. The analysis is also easy to understand. The authors seem to be very careful for showing how the action is chosen.
Critical weaknesses
  1. I'm curious about the prompt order. Memory is saved prompt by prompt, and each prompt only uses memory from before. I couldn't find how the order is decided, and how the performance is at the beginning.
  2. It's also unclear that what estimate is used for the reported result. The code mixes the estimate of each candidate with action type estimate, but the paper only explains the estimate for each action type. Also Section 2.2 says five caption style but Appendix A says six.
  3. Table 3 is useful, but some difference is very small and I couldn't find repeated run. Using only AQ is 47.58 and full model is 47.42. Therefore, only AQ is more higher in completion. It seems like the gain of full model is mostly from less generation.
Questions for authors
  1. Which prompt order is used? Is there result when the order is changed? Also could you report the result by position in the stream please? Then it will be clear that how memory is built at the beginning.
  2. Is the code with candidate search and mixing used for the reported result? Could you report the settings like alpha and minimum similarity please, and how they are matched with Eq. 7 to 10?
  3. Which model is used for strict judge, and is it related to evaluator of VBench-2.0 and StoryEval? Final video is chosen by internal score, so I'm curious about how often internal score and benchmark score go up together.
  4. How many episode ends by stop action, and how many ends by budget? Section 3.1 shows average generation per prompt, but stop step distribution will show stop decision more directly.
5 comments
AuthorsOpenReview ↗

Response to Reviewer HKi1(1/2)

Thank you for highlighting the practical value of achieving higher quality with fewer video generations, the effectiveness of our memory design, and the clarity of our action-selection analysis. We address the three weaknesses and four questions below.

W1 & Q1. Prompt Order and Early-Stream Performance

We applied no ordering or selection to the prompts: all reported results use the original order released by each benchmark. To measure order sensitivity, we further test random, long-to-short, and short-to-long orderings (by prompt length) with Wan-2.1 on three streams of different sizes: Composition, the VBench-2.0 dimension with the fewest prompts (53); Camera Motion, the dimension with the most prompts (108); and the full StoryEval benchmark (423) (Table A).

Table A. Effect of prompt order (Wan-2.1).

OrderComposition (53)Camera Motion (108)StoryEval (423)
Original (reported)67.1746.3037.47
Random64.4548.1537.30
Short → Long65.8046.7137.93
Long → Short71.2651.3238.70
Mean ± std (3 alternative orders)67.17 ± 3.6148.73 ± 2.3637.98 ± 0.70

Order sensitivity decreases with benchmark size. The reported original order is not the best on any benchmark, and no alternative order lowers TTPE's ranking among the compared methods. Long-to-short consistently achieves the best results across all three benchmarks, suggesting that experience from more detailed prompts may transfer effectively to shorter prompts. We have included this analysis in the revision.

How memory is built at the beginning of the stream. We sort the 423 StoryEval prompts (HunyuanVideo) by the order in which they start, split them into quarters, and score both the initial and the delivered video under the StoryEval protocol with Qwen3.8-27B, the independent evaluator in Q3 (Table B; absolute scores are not directly comparable to our reported Gemini-3.1 Pro results). When memory is nearly empty, the gain is limited; it increases once about 100 episodes are accumulated and then remains stable (the last quarter contains more hard prompts). We have added this analysis to the revision (Appendix J).

Table B. Gain by position in the stream (HunyuanVideo, StoryEval, scored by Qwen3.8-27B).

Stream position1–106107–212213–318319–423
Avg. episodes accumulated in memory21121231351
Easy / Medium / Hard prompts26 / 60 / 2025 / 55 / 2623 / 61 / 2226 / 47 / 32
Initial video42.835.536.636.6
Delivered video62.067.168.060.9
Gain+19.3+31.7+31.4+24.3

W2 & Q2. Estimate, Implementation Settings, and Caption Styles

Estimate. The candidate-level search and blending in the released code are not used for the reported results. For all reported results, the minimum similarity threshold for candidate-level matching is set to 1, so no candidate-level neighbor is retrieved; the blending weight α\alpha (0.5) then has no effect, and the blended estimate reduces exactly to the action-family estimate. Thus, all reported results are obtained solely with the action-family estimate of Eqs. 7–9. We also explored candidate-level blending, but found no clear benefit, as sparse candidate-level neighbors make the blended estimate high-variance. To keep the released code consistent with the paper, we will remove the candidate-level search and blending, so that the code implements exactly the action-family estimate of Eqs. 7–9.

Caption styles. This is a typo in Section 2.2: switch_repr uses six caption styles, as listed in Appendix A. We have corrected it in the revision.

W3. Repeated Runs and the Gain of the Full Model in Table 3

Repeated runs. All results of TTPE and the reproduced baselines are averaged over three runs; the std of completion is ±1.77 for AQA_Q alone and ±0.72 for the full model.

Gain of the full model. Your reading is correct, and it is the intended outcome. TTPE formulates refinement as a budget-constrained decision problem, so its contribution is the quality–cost trade-off rather than quality alone, which is the practical value you highlighted as the main strength. The two advantages serve different roles: AQA_Q steers refinement toward higher quality, while ACA_C constrains the generation budget by favoring actions that need fewer further generations. In Table 3, the full model matches the completion of AQA_Q alone (47.42 vs. 47.58) while using 29% fewer generations (6.30 → 4.47 per prompt). Conversely, ACA_C alone saves generations but drops completion to 41.17. Only the full model achieves both.

AuthorsOpenReview ↗

Response to Reviewer HKi1(2/2)

Q3. Strict Judge and Agreement with Benchmark Scores

1. Strict judge and its relation to the benchmark evaluators. The strict judge is Gemini-3.1 Pro with element-level binary VQA. All reported scores follow each benchmark's own protocol rather than the strict judge's: VBench-2.0 uses dimension-specific open-source models such as LLaVA-Video, Qwen2.5, and CoTracker, and StoryEval uses its own event lists, scoring prompts, and unanimous voting over three runs, which can be run with any video-language model; we run it with both Gemini-3.1 Pro (reported results) and Qwen3.8-27B (point 2).

2. How often the internal and benchmark scores go up together. We score every TTPE video (HunyuanVideo, StoryEval) under the official StoryEval protocol with Qwen3.8-27B, a VLM never involved in optimization, so the two scores share neither the model nor its scoring tendencies. Within the same prompt, among the 3,056 video pairs that differ in both scores, the video with the higher internal score also has the higher StoryEval score in 80.6% of cases (95% CI [77.6, 83.3]). For the delivered video, both scores rise together as refinement proceeds (Table C).

Table C. Internal and StoryEval scores of the video selected by the internal score as refinement proceeds (HunyuanVideo).

Refinement iterations so far013579
Internal score40.458.974.882.385.988.2
StoryEval score37.848.457.461.563.064.5

Q4. Episodes Ending by Stop vs. Budget

About 70% of episodes end by the stop action, well before the budget (Table D).

Table D. Episodes ending by stop action vs. budget on StoryEval (423 prompts).

Video ModelBudgetEnded by Stop (all met)Ended by Stop (unmet)Ended by BudgetAvg. Gen. (Ended by Stop)
Wan-2.12065.9%4.6%29.5%5.8
HunyuanVideo1065.6%2.3%32.1%3.7
MiniMax-H3470.4%1.7%27.9%1.7

Summary

The new evidence addresses each question on memory, estimation, and evaluation: TTPE's ranking is robust to prompt reordering, its gain grows as memory accumulates over the stream, the reported results use the action-family estimate of Eqs. 7–9, all results are averaged over three runs, the full model reaches comparable completion with fewer generations, the internal and StoryEval scores rise together, and most episodes end by the stop action before exhausting the budget. All corresponding changes are highlighted in blue in the revised version. Thank you for the careful and detailed questions, which led us to make the implementation settings and evaluation protocol explicit. We hope the new results and clarifications support a stronger acceptance rating.

AuthorsOpenReview ↗

Looking forward to your feedback

Dear Reviewer HKi1,

Thank you sincerely for your thoughtful and constructive feedback. We truly appreciate your recognition of the practical value of our method, particularly its ability to improve video quality while reducing generation costs. We would be very grateful if you could spare some time to revisit our responses.

We fully understand that you have many responsibilities, and we sincerely appreciate your time and consideration. To facilitate your evaluation, we have summarized the main revisions and additional analyses as follows:

  • We have conducted additional experiments with different prompt orderings and analyzed performance across different positions in the prompt stream, providing further insights into the impact of memory accumulation (Tables A and B).

  • We have clarified the estimation procedure and implementation settings used in our reported results, explained the role of candidate-level search and blending, and corrected the inconsistency in the number of caption styles.

  • We have further clarified the quality-cost trade-off achieved by the full model, which maintains comparable completion performance while substantially reducing generation costs.

  • We have provided additional evaluations using an independent VLM to examine the consistency between internal scores and benchmark scores, showing that higher internal scores correspond to higher StoryEval scores in 80.6% of comparable video pairs (Table C).

  • We have added statistics on episode termination, showing that approximately 70% of episodes terminate through the stop action rather than exhausting the generation budget (Table D).

These additional experiments, analyses, and clarifications have been incorporated into our revised manuscript.

We sincerely appreciate the time and effort you have devoted to reviewing our work. We are eager to ensure that we have adequately addressed your concerns. Please feel free to let us know if you have any additional questions or concerns, and we would be more than happy to discuss them.

Should you find that our responses have satisfactorily addressed your main concerns, we would sincerely appreciate your continued support and reconsideration of our paper in light of the additional results and clarifications.

Thank you again for your time and consideration.

Best regards,

The Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

Request for Assistance with Reviewer Discussion

Dear Area Chair,

As the author-reviewer discussion period is approaching its deadline, we would like to briefly update you regarding the discussion of our submission.

We have provided comprehensive point-by-point responses to all reviewer concerns and made substantial revisions to the manuscript, including additional controlled experiments, robustness and transfer evaluations, and theoretical clarifications. We also posted a concise follow-up message inviting the reviewers to comment on whether these additions addressed their concerns.

We are grateful that one of the reviewers has already responded and indicated that our responses and revisions have fully addressed their concerns. However, the other two reviewers have not yet participated in the discussion or provided further comments on our rebuttal.

We fully understand that the reviewers may be handling many submissions and may not have had sufficient time to revisit our responses. Nevertheless, without further interaction, the discussion period may conclude without the new evidence and clarifications being considered, particularly where the original assessments were based on issues that have now been directly addressed.

If appropriate, we would sincerely appreciate your help in encouraging the remaining reviewers to consider our responses before the discussion deadline. At a minimum, we hope that the final assessment can take into account our complete rebuttal, the additional experiments and analyses, and the revised manuscript.

Thank you very much for your time and consideration.

Best regards,

Authors of Submission 44854

AuthorsOpenReview ↗

Summary of Rebuttal and Revisions for Final Consideration

Dear Program Chairs, Senior Area Chairs, Area Chairs, and Reviewers,

We sincerely thank all reviewers for their time and constructive feedback. We are greatly encouraged that the reviewers recognized the significance and practical value of our work. Reviewer HKi1 acknowledged TTPE's ability to improve video quality while substantially reducing the number of generations, and recognized the effectiveness of its memory design. Reviewer wUcw considered our approach well motivated and practical, with comprehensive experimental support. Reviewer d8gU also recognized the importance of the problem and the intuitive idea of reusing previous test-time experience. These assessments affirm the contributions of our work: we propose a training-free framework that leverages accumulated refinement experience to achieve a better balance between video quality and generation cost.

In response to the reviewers' concerns, we conducted additional experiments and analyses covering prompt-order sensitivity, memory accumulation, key hyperparameters, and evaluation procedures. The results further validate the robustness of TTPE and clarify how its memory mechanism contributes to performance improvements. We also further clarified the quality-cost trade-off, demonstrating that the full model maintains comparable video generation quality while reducing generation costs by approximately 29%.

Furthermore, we strengthened our evaluation framework through independent vision-language model (VLM) evaluations and human preference studies, providing further evidence that improvements in internal scores translate into improvements in overall video quality. We also clarified the reward design, state representation, and cost estimation, and conducted additional analyses of action selection across different task difficulty levels to address concerns regarding potential selection bias. All relevant clarifications and additional results have been incorporated into the revised manuscript.

Following our responses, Reviewer wUcw expressed satisfaction with our rebuttal and increased the rating. Reviewer HKi1 confirmed that the additional experiments and clarifications had addressed the questions raised and maintained the original rating. Reviewer d8gU acknowledged our responses but retained some remaining concerns. In response to the reviewer's latest feedback, we further provided detailed explanations and additional empirical evidence.

All revisions and additional results have been incorporated into the revised manuscript and highlighted in blue. We sincerely appreciate the valuable feedback from all reviewers, which has helped us further strengthen the empirical validation and presentation of our work. We hope that the reviewers' recognition of the practical contributions of our work, together with the additional evidence and clarifications we have provided, will support a favorable final evaluation.

Best regards,
Authors of Submission 44854

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.