acceptodds
Under review as a conference paper at ICLR 2027

FlowCred: Hierarchical Credit Assignment and Functional Coordination for Joint Text-to-Video Post-Training

Abstract

Prompt optimization and generator post-training are two complementary ways to improve text-to-video (T2V) generation, and recent systems increasingly train both modules jointly. However, optimizing a prompt policy and a video generator from the same terminal reward creates two ambiguities: prompt quality is confounded with stochastic generation, and the two modules may realize overlapping local improvements. We introduce FlowCred, a two-level framework for joint T2V post-training. First, hierarchical credit assignment uses matched prompt–seed rollout grids to separate prompt-level credit from prompt-conditioned generator residuals under shared randomness. Second, functional coordination probes alternative current-policy prompts at states from actual generation trajectories, providing generator-aware evidence for prompt optimization while selectively attenuating generator updates already supported by conditioning changes. Across multiple T2V benchmarks, FlowCred consistently improves matched joint-training baselines. On LTX-2.5, FlowCred improves a strictly matched joint-training control from 61.47 to 64.46 on T2V-CompBench and from 55.58 to 58.08 on StoryEval, without increasing decoded-video or reward-model-call budgets. Factorial and realized-update analyses show complementary contributions from prompt shaping and generator routing, while human evaluation and transfer across backbones and updater families provide further supporting evidence.

Primary area:
generative models
Keywords:
Text-to-video generation, Reinforcement learning, Prompt optimization, Credit assignment

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_BdmxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces FlowCred, a joint post-training framework for improving text-to-video generation by coordinating a trainable prompt optimizer and video generator. The authors identify two challenges in conventional joint optimization: ambiguous credit assignment when both modules share a terminal reward, and overlapping improvements between prompt conditioning and generator updates. To address these, FlowCred introduces hierarchical credit assignment, which separates prompt-level rewards from generator-specific stochastic variation using matched prompt–seed rollouts, and functional coordination, which probes alternative prompts at intermediate generation states to guide prompt optimization and reduce redundant generator updates.

Critical strengths
  1. The paper addresses an important but relatively underexplored issue: how to optimize two interacting modules without relying on an undifferentiated shared terminal reward.
Critical weaknesses
  1. The previous work [1] explicitly addresses separating action quality from environmental randomness through counterfactual baselines and low-variance policy-gradient estimation, which makes this work incremental.

[1] Counterfactual Credit Assignment in Model-Free Reinforcement Learning (ICML 2021)

Questions for authors

Please answer the major weaknesses.

1 comment

Dear reviewers,

Will you please check the authors' response and reply it?

Best, AC

Reviewer_7TZiOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents FlowCred, a framework for jointly post-training a prompt policy and a text-to-video generator. Hierarchical credit assignment uses matched-seed sampling to distinguish prompt-level mean performance from trajectory variation under a fixed prompt, providing module-specific training signals. Functional coordination uses local output responses to prompt substitutions at the same generator state to shape prompt-policy learning and selectively attenuate generator reward updates along overlapping response directions. The framework is evaluated on LTX-2.5 across four T2V benchmarks through controlled comparisons, component ablations, mechanism analyses, and human evaluation. Additional experiments assess transfer to HunyuanVideo-1.5 and alternative generator updaters. The results show improvements over joint-training baselines implemented in a common training framework under matched decoded-video and reward-model-call budgets.

Critical strengths
  1. Functional coordination based on local response geometry provides a concrete technical contribution. FlowCred uses output responses to prompt substitutions at the same generator state for both generator-aware prompt shaping and selective generator routing. By relating conditioning responses to generator reward updates in the generator’s output space, it introduces an explicit coordination mechanism into shared-reward joint training.

  2. Controlled experiments provide substantial support for the performance gains and component contributions. Under a common training framework and matched decoded-video and reward-model-call budgets, the method consistently improves over joint-training baselines across multiple T2V benchmarks, with positive gains also observed on another backbone and alternative generator updaters. Component ablations and realized-update analyses support the individual contributions of prompt shaping and generator routing, while human evaluation and an independent evaluator provide further evidence of improved generation quality.

Critical weaknesses
  1. Statistical dependence in the key control limits attribution of the hierarchical credit assignment gains. The Matched Global-LOO baseline includes rewards from other prompts sharing the current seed, which may correlate with the current trajectory’s score-function term SS. Baseline subtraction preserves the expected score-function gradient when E[bS]=0E[bS]=0, but this condition is not guaranteed under shared randomness. Consequently, HC’s improvement over this control may also reflect changes in the gradient estimator’s expectation, leaving the contribution of the prompt-conditioned baseline insufficiently isolated.

  2. The benefit of selective routing over general update shrinkage is not sufficiently isolated. The ablations and realized-update analyses support the design’s effectiveness, but the comparability of the uniform-shrink control and Full remains unclear regarding functional prompt shaping, treatment of the reward and reference branches, and realized update magnitude. Consequently, the comparisons do not sufficiently distinguish gains from selectively attenuating prompt-supported directions from those due to auxiliary training signals and general update shrinkage, limiting attribution to the proposed cross-module coordination mechanism.

Questions for authors
  1. In Matched Global-LOO, the baseline includes rewards from other prompts sharing the current seed. How is their potential dependence on the generator trajectory’s score-function term handled? Could a comparison with a global baseline excluding the entire current seed column, while keeping the prompt-policy update rule identical to HC, help isolate the contribution of prompt conditioning? This comparison could be limited to the main setting.

  2. Does the uniform gradient shrink control in Table 11 retain the same functional prompt shaping as Full and apply shrinkage only to the reward branch at the same selected transitions, leaving the reference/KL branch unchanged?

  3. Could the realized-update audit in Appendix F.1 include uniform gradient shrink under the same branching protocol, reporting its realized output displacement norm relative to Full? If the existing control satisfies the conditions above, clarification and this diagnostic would suffice. Otherwise, could one matched training comparison in the main setting help distinguish selective routing from general update shrinkage?

1 comment

Dear reviewers,

Will you please check the authors' response and reply it?

Best, AC

Reviewer_hRxzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies joint post-training of a prompt-optimizer LLM and a text-to-video generator from a single terminal reward. It argues that this setting raises two distinct problems. The first is statistical credit: the terminal reward mixes prompt quality with generator stochasticity. The second is functional overlap: both modules may realize the same local improvement. FlowCred addresses these with two mechanisms: Hierarchical credit assignment and Functional coordination. The full method on LTX2.5 improves T2V-CompBench from 61.47 to 64.46 and StoryEval from 55.58 to 58.08. Gains on VBench and VBench-2.0 are smaller. Decoded-video and reward-call budgets are the same as the baseline.

Critical strengths
  1. A well-posed problem with a carefully controlled experimental design. Separating "who should receive credit" from "which module should realize an overlapping improvement" is a useful lens for cascaded generative systems.

  2. The rollout-budget result suggests practical sample efficiency. Full at M = 4 (16 decoded videos per request) reaches 64.46 on T2V-CompBench. HC at M = 8 (32 videos per request) reaches only 63.34 (Table 14). The increment also carries over, at smaller magnitude, to a second backbone and two different generator updaters. If this holds, the method improves sample efficiency in a setting where decoded rollouts dominate the cost.

Critical weaknesses
  1. The central mechanism (functional coordination) is under-justified. The paper never explains why overlap between the two modules is harmful. When both modules ascend a shared objective, correlated local effects are not a problem in themselves. Proposition 3 only shows that the routed update still ascends; it does not show that routing improves the objective. The size of the effect is also hard to reconcile with the mechanism. Routing touches only one transition per trajectory and attenuates at most a 3-dimensional subspace, yet routing alone adds +1.04 on T2V-CompBench. The controls do not settle this. The random span is matched on rank rather than on removed gradient energy, so its null effect is expected. In addition, response norm alone already reaches ρ = 0.313 in the ranking study, which suggests the functional score may partly track prompt magnitude or detail. As a result, the main conceptual claim, that overlap exists and must be routed, is not established.

  2. Contribution (1) is largely standard, and the protocol overstates the evidence. Hierarchical credit amounts to grouping generator samples by prompt, as GRPO already does, plus common random numbers. Appendix A.2 concedes that the prompt-side ranking is identical to Global-LOO. All baselines are re-implemented by the authors with only 2 runs, and Table 1 reports no variance. A sequential or alternating training baseline is missing. Compute overhead is understated: per request, Full adds 144 forwards of a 22B DiT and 96 extra LLM-generated prompts, and no wall-clock numbers are reported.

Questions for authors
  • Can you give direct evidence that overlap hurts under HC, for example overshoot along the supported directions, or a weaker generator when it is given raw prompts?
  • What fraction of the total generator reward gradient does routing actually change, given that only one transition per trajectory is routed?
  • Can you run random and wrong-query span controls that remove the same gradient energy as the true span?
  • Please report wall-clock time or GPU-hours for HC and Full, and compare against HC given the same compute.
  • "No reliability gate" (Table 11) and "routing only" (Table 3) both report 63.67 on T2V-CompBench. Are these separate runs?
1 comment

Dear reviewers,

Will you please check the authors' response and reply it?

Best, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.