acceptodds
Under review as a conference paper at ICLR 2027

Beyond the Final Reward: Depth-aware Post-Training for Instructional Video Editing

Abstract

Post-training with reinforcement learning is the standard route to aligning instruction-guided video editing models with human preferences. Current pipelines realize it by rolling out a full denoising trajectory and scoring only the final decoded video, typically with several specialized reward models whose scores are combined by hand. Such outcome-only supervision only observes the final output of each diffusion-transformer evaluation, treating the evaluation as a single atomic decision. In reality, nevertheless, the prediction is progressively assembled across dozens of instruction-conditioned blocks. Thus, this supervision cannot reveal at which depth editing quality is formed, nor which part of the instruction is responsible for it. To address this issue, we introduce a post-training method for **V**ideo editing through **I**ntermediate **S**elf-evaluation and **T**oken Attribution (**VISTA**). At selected transformer layers, VISTA first decodes groups of intermediate edit candidates, and evaluates them with a single reward model. A return-to-go objective then assigns each layer credit using its own reward and those observed at subsequent layers. The winning candidate is distilled into a token-importance memory that conditions later layers and re-weights a layer-wise GRPO update. By doing this, VISTA converts outcome-only GRPO into depth-aware self-evaluation without composing multiple specialized reward models. We further bound the bias caused by reusing the token-importance memory within a layer update, and give a sufficient condition under which this bias is dominated by genuine descent on the training objective. Experiments on 5B and 14B video-editing backbones demonstrate consistent improvements on OpenVE-Bench and FiVE-Bench, while ablations verify the complementary effects of intermediate self-evaluation and token attribution.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video Editing, Reinforce Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_vMyaOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

Existing RL post-training for video editing scores only the final generated video, so it cannot tell at which depth of the network editing quality is formed, or which instruction words are responsible for it. The authors observe that hidden states decoded from intermediate DiT layers are already recognizable edits, and that their rewards increase with depth. Based on this, they propose VISTA: at selected intermediate layers, the model decodes a group of candidate edits and scores them with a single reward model, and each layer is credited with its own reward plus the rewards of subsequent layers. The best candidate at each layer is then converted into a token-importance weighting over the instruction, which strengthens the conditioning of later layers on the key words and reweights the GRPO update at that layer. Experiments start from Kiwi-Edit (5B) and SAMA (14B) and evaluate on OpenVE-Bench and FiVE-Bench, together with ablations, hyperparameter analysis, and a human preference study.

Critical strengths
  1. Densifying the reward along network depth rather than along denoising steps is a novel perspective, and it requires only a single reward model.
  2. The evaluation is comprehensive: four VLM judges, FiVE-Bench, and a human preference study with 24 participants.
  3. The paper is well written with a clear motivation, and the framework and qualitative comparison figures are intuitive and well designed.
Critical weaknesses
  1. The 14B model is initialized from SAMA, yet its OpenVE-Bench average is essentially unchanged under all four judges (3.27→3.22, 3.99→3.99, 3.31→3.33, 4.00→4.06; Fig. 15), which contradicts the conclusion that VISTA improves over its initialization at both scales.
  2. The claim in Sec. 4.2 that the 5B model surpasses SAMA holds only under the two non-official judges (3.31 vs 3.27, 4.05 vs 3.99); under the official judges in App. G.4 it is lower or tied (3.09 vs 3.31, 4.00 vs 4.00). How do the authors interpret this inconsistency?
  3. In Table 2, removing either component drops the average to 2.47 / 1.43, far below both the base model (3.16) and outcome-only GRPO (2.98). Why does removing a single component make the model much worse than not training at all?
  4. Minor: The two ablation rows in Table 2 (2.47 / 1.43) are swapped relative to the text in Sec. 4.3.
Questions for authors
  1. Is the decoding head gϕ in Eq. (1) the backbone's final output projection applied to H(ℓk), or a separately trained module?
  2. See weakness.
5 comments

We sincerely thank the reviewer for the positive assessment, in particular for recognizing depth-wise reward densification as a novel perspective and for finding the paper clear and well motivated. We address each point below.

W1: 14B against SAMA.

We would like to offer a fuller reading of the 14B results. The 14B model is LoRA-adapted from SAMA, which is already the strongest baseline in our comparison, so the headroom is considerably smaller than at 5B. Even so, it obtains the highest average among all methods under both judges (3.33 vs. 3.31 under gemini-3.1-pro and 4.06 vs. 4.00 under qwen3.5-35b-a3b), and it is on par with SAMA under the two additional judges. On FiVE-Bench, it improves 28 of 36 task–metric entries over SAMA, including Struct, PSNR, and LPIPS on all six edit types and MF on five. The 14B model therefore does improve over its initialization, mainly in source preservation, while maintaining or slightly improving overall editing quality. We agree that the OpenVE margins at 14B are small, and we have revised Sec. 4.2, the abstract, and the conclusion to state that the improvement at 14B lies mainly in source preservation.

W2: 5B against SAMA.

We thank the reviewer for this careful cross-check. Across the four judges, the 5B model scores higher than SAMA under two judges (3.31 vs. 3.27 and 4.05 vs. 3.99), ties under qwen3.5-35b-a3b (4.00 vs. 4.00), and is lower under gemini-3.1-pro (3.09 vs. 3.31). We attribute this to differences between the judges, which score on noticeably different scales and weight the evaluation axes differently; this is also why Appendix G.4 compares methods by their rank within each judge. The more robust observation is the change from initialization: Kiwi-Edit is below SAMA under all four judges (e.g., 2.94 vs. 3.31 under gemini-3.1-pro), and RL post-training brings the 5B model to match or exceed SAMA under three of them, with about a third of SAMA's parameters. We have revised Sec. 4.2 to report all four comparisons and to state that the 5B model matches or exceeds SAMA under three of four judges, while SAMA remains ahead under gemini-3.1-pro.

W3: Why the ablations fall below the base model.

We thank the reviewer for raising this point; the answer also explains why the two components are designed together. Regarding the minor comment, the correct mapping is w/o Self-eval: 2.47 and w/o TI memory: 1.43. The Sec. 4.3 explanations describe the correct mechanism for each number, and only the component names were inconsistent; we have aligned them in the revision. The complete ablation, with a new random-memory control (5B, gemini-2.5-pro), is:

Training strategyAvg.ICCFVQ
Base model3.163.412.993.08
+ outcome-only GRPO2.983.242.832.87
w/o Self-eval2.472.892.242.29
w/o TI memory1.431.551.331.40
Random TI memory (new)2.492.942.232.31
Full method3.313.563.163.22

Both ablations still take K=4K=4 gradient steps per rollout, but removing either component makes these steps largely redundant. Without self-evaluation, every step uses the same terminal-reward advantage; without the memory, every step uses the same conditioning. The model therefore repeats nearly the same update four times on one rollout and over-fits it, which is why both ablations fall below the base model and even below outcome-only GRPO (2.98), which takes a single step. Random memory, which carries no attribution, reaches only 2.49, so the memory must reflect the tokens that drive reward. With both components, each step receives a distinct signal, and the full method reaches 3.31. We have added this explanation to Sec. 4.3 and will include a four-step outcome-only control.

Q1: Decoding head.

We thank the reviewer for the question. The reviewer's first reading is correct, and no separately trained module is used. Each H(ℓk)\mathcal H^{(\ell_k)} passes through the backbone's own final output projection and is then decoded by gϕg_\phi, the VAE decoder of the base model, kept frozen throughout training. The decoding path adds no parameters, so the intermediate outputs are decoded exactly as the final output is. The revised Eq. (1) shows both steps, and Sec. 3 and Appendix E.1 state this explicitly.

We hope these clarifications address the reviewer's questions, and we would be happy to provide any additional details.

I would like to flag a formatting issue in the revised PDF. The text block appears to be wider than the ICLR template allows. It looks like the page geometry was modified on top of the template. Could you please restore the official margins in the next revision? I wanted to raise this early so it does not cause problems later.

Thank you so much for bringing this to our attention! We truly appreciate your careful reading and helpful feedback. This was something we had overlooked during the revision, and your comment helped us identify the issue. We have now uploaded a newly revised PDF to fix this issue.

We are genuinely grateful for your time and attention to these details. We would also really appreciate it if you could let us know whether our responses have adequately addressed your concerns. We remain happy to clarify any further questions and are open to continued discussion.

Reviewer_51LCOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+2)
Read review
Paper summary

The paper proposes VISTA, an RL post-training method for instruction-guided video editing that evaluates intermediate predictions along transformer depth. At selected DiT blocks, it decodes candidate edits, scores them with one vision-language reward model, and forms group-relative advantages from a depth-wise return-to-go. A winning candidate provides gradient-times-input token attribution, which is normalized into a memory and added to instruction embeddings for subsequent computation and policy updates. Stored rollouts are reused across checkpoints, with a two-evaluation surrogate adapted from LeapAlign to reduce optimization cost. The method post-trains Kiwi-Edit at 5B and SAMA at 14B. Evaluations include OpenVE-Bench with four VLM judges, FiVE-Bench automatic metrics, and a human preference study for the 5B model. The final 5B model has encouraging results, but inconsistencies in the objective, configurations, and ablation descriptions limit the conclusions about the proposed mechanism.

Critical strengths
  1. Architectural-depth supervision is an interesting direction. VISTA turns intermediate representations into instruction-conditioned supervision, complementing denoising-time rewards. Reusing a rollout across depth checkpoints and carrying a compact token memory could extract more learning signals from expensive video samples. This conceptual contribution supports acceptance if the implementation is clarified.

  2. Complementary evaluations support the final 5B model. Against Kiwi-Edit, average gains across four OpenVE judges are 0.15, 0.18, 0.15, and 0.07 points (Tables 9-12), with improvements in 32/36 FiVE entries (Tables 7-8). A blinded, randomized study with 24 participants on 100 cases raises average ratings from 2.99 to 3.25 (Appendix G.5). This human evidence complements VLM-based evaluation and supports the final system, though it does not isolate the mechanisms.

Critical weaknesses
  1. The core objective and training configuration are inconsistent. Eqs. (2)/(4) multiply all downstream rewards by the same gamma: zero eliminates every reward rather than retaining the local reward, while positive gamma cancels in Eq. (9)'s normalized advantage, assuming nonzero variance, and leaves the winner unchanged. Thus the equations do not explain the discount sweep; a corrected formula must be connected to the actual reported runs. A missing exponent could explain this, but the implementation needs clarification. Appendix E.1 assigns gamma = 0 to 5B, whereas Sec. 4.4/Appendix G.2 select 0.5; gains from local rewards would not establish a downstream return-to-go benefit. The decoding head's architecture, initialization/training, and noisy-feature-to-video conversion are also insufficiently specified. These gaps affect the central mechanism.

  2. The ablations do not isolate the mechanisms. Table 2 labels 2.47 as w/o Self-eval and 1.43 as w/o TI memory, but pp. 8-9 assign them to the opposite removals. Optimizer-step counts and resource matching need clarification because VISTA performs K updates per stored rollout. The outcome-only decline from 3.16 to 2.98 does not establish that terminal rewards are intrinsically harmful. Table 6 also gives local rewards and gamma = 0.5 the same rounded mean, 4.20, with one seed per configuration and spread across quality dimensions rather than training runs. Against SAMA, 14B OpenVE gains are mixed: -0.05, 0.00, +0.02, and +0.06 across judges. The human study compares final systems, not RL ablations, leaving mechanism-level evidence weaker than the evidence for the final 5B model.

Questions for authors
  1. What return and configuration actually produced the reported results? Please give the implemented return-to-go, including the gamma = 0 case, and identify K, gamma, and the checkpoint used for each main comparison and ablation. Is the discrepancy between Eqs. (2)/(4) and the text a missing exponent? Does the 5B result use local rewards or discounted downstream rewards? A consistent mapping from implementation to tables would materially affect my assessment.

  2. How are intermediate predictions and token memory computed in practice? Please specify the decoding heads, their initialization and training, selected block indices, and the denoising states used to obtain the videos being scored. A pseudocode trace should show which incoming memory is used to compute the attribution loss, when the new memory is detached, which parameters receive each update, and how stored downstream rewards are used. Please also confirm the inference procedure and how training-time augmented conditioning relates to evaluation-time conditioning. This would resolve ambiguities in the claimed refinement loop.

  3. Can the component evidence be reconciled under controlled optimization budgets? Please correct the Table 2/paragraph mapping and report the precise removals, total gradient steps, rollout counts, and wall-clock cost for each variant. A stable outcome-only control, local depth rewards, discounted return-to-go, and a simple uniform or shuffled-token memory control would help distinguish the claimed mechanisms. Paired case-level uncertainty and repeated training for the key comparison would make the small differences interpretable. These checks target the present causal claims.

  4. What data and evaluation protocol underlie the gains? Please describe the size, provenance, and sampling of the self-collected RL corpus, its separation from OpenVE/FiVE evaluation and hyperparameter selection, and the exact reward-model checkpoint. For the human study, please provide the raw win/tie/loss counts, aggregation rule, and uncertainty accounting for repeated judgments of the same clips and participants. Some percentages in Table 14 need clarification under the stated denominator of 300 comparisons per baseline. This is a request for reproducible reporting; no evaluation overlap or data-integrity violation was established here.

5 comments

We sincerely thank the reviewer for the thorough and thoughtful review, and for the many detailed questions that helped us improve the clarity of the paper. We address each point below.

W1/Q1: Implementation and configuration.

Return-to-go: The implemented return is Rg(ℓk)=∑j=kKγ j−kR(V^g(ℓj))\mathcal{R}^{(\ell_k)}_g=\sum_{j=k}^{K}\gamma^{\,j-k}R(\hat{\mathbf V}^{(\ell_j)}_g), i.e., it includes the exponent j−kj-k that the reviewer anticipated. With γ=0\gamma=0, only the local reward at ℓk\ell_k remains (γ0=1\gamma^0=1). With γ>0\gamma>0, the reward at ℓj\ell_j receives weight γ j−k\gamma^{\,j-k}, so γ\gamma changes how much later checkpoints count relative to the local one. It therefore does not cancel in the normalized advantage of Eq. (9), and it can change the winner when candidates rank differently across checkpoints. The revised Eqs. (2) and (4) display the exponent explicitly.

Configuration: Every reported experiment uses γ=0.5\gamma=0.5 and 300 training steps, except that the γ\gamma sweep varies γ\gamma. The value γ=0\gamma=0 for the 5B model in Appendix E.1 is a typo, which we have corrected in the revision, so the main 5B results do use discounted downstream rewards. The 14B run additionally uses the layer-wise advantage correction of Appendix E.1.

Role of the discount: We would like to clarify the scope of our claim. The central claim is that depth-wise intermediate supervision improves over outcome-only training, which Table 2 supports (outcome-only GRPO 2.98, without intermediate self-evaluation 2.47, full method 3.31). The return-to-go contains local supervision as the special case γ=0\gamma=0, and we should note that γ>0\gamma>0 is not the main source of the gain. We adopt γ=0.5\gamma=0.5 because it ties for the best average (4.20) and gives the highest IC and late-layer reward. The revised Sec. 4.4 describes the method as robust for γ∈[0,0.5]\gamma\in[0,0.5] and clarifies the figure caption accordingly.

W2: Ablations and mechanism-level evidence.

Table 2 mapping: The correct mapping is w/o Self-eval: 2.47 and w/o TI memory: 1.43. The explanations in Sec. 4.3 describe the correct mechanism for each number; only the component names were inconsistent, and we have aligned the table and text in the revision.

Isolating the memory: Following the reviewer's suggestion, we trained a random TI memory control that replaces the Grad$\times$Input attribution with random token weights, keeping all else fixed. The complete ablation (5B, gemini-2.5-pro, G=4G=4, γ=0.5\gamma=0.5, 300 steps) is:

Training strategyAvg.ICCFVQ
Base model3.163.412.993.08
+ outcome-only GRPO2.983.242.832.87
w/o Self-eval2.472.892.242.29
w/o TI memory1.431.551.331.40
Random TI memory (new)2.492.942.232.31
Full method3.313.563.163.22

Because the memory weights sum to one, a uniform memory would scale every instruction token by the same factor (1+1/N)(1+1/N), so w/o TI memory already serves as a uniform-weighting control. Uniform (1.43) and random (2.49) weighting both fall below the base model, whereas gradient-based attribution reaches 3.31, so the gain depends on what the attribution selects and not merely on reweighting the instruction.

Optimizer steps: We respectfully note that extra steps alone do not explain the gain: both ablations take the same four steps per rollout as the full method, yet fall below outcome-only GRPO (2.47 and 1.43 vs. 2.98). The benefit comes from what makes the four steps differ, namely checkpoint-specific advantages and an attribution-informed memory; random memory also varies across checkpoints, yet reaches only 2.49. Budget details are in our Q3 response.

Outcome-only result: We do not claim that terminal rewards are intrinsically harmful. Our observation is narrower: with the same reward, data, and rollout budget, outcome-only GRPO does not improve over its initialization, whereas the depth-wise method does (2.98 vs. 3.31). Sec. 4.3 now states this scope.

Table 6: see Role of the discount in our Q1/W1 response.

14B results: Starting from SAMA, our strongest baseline, the 14B model has less headroom, yet it obtains the highest average among all methods under both official OpenVE judges (3.33 and 4.06), is on par with SAMA under the other two (−0.05-0.05, 0.000.00), and improves 28 of 36 FiVE entries, including Struct, PSNR, and LPIPS on all six edit types, so its gains lie mainly in source preservation.

Human study: The study now covers both scales (Appendix G.5; 100 clips, 24 participants, 1–5):

MethodICCFVQAvg.
Omni-Video (1.3B)1.611.391.441.48
Lucy-Edit (5B)2.181.962.072.07
VINO (13B)2.512.222.292.34
Kiwi-Edit (5B)3.222.832.922.99
SAMA (14B)3.392.943.003.11
Ours (5B)3.443.143.173.25
Ours (14B)3.532.943.133.20

Raters score our 5B model above Kiwi-Edit and our 14B model above SAMA. Mechanism-level evidence comes from the controlled ablations, where the full method leads every trained variant by at least 0.33 points (3.31 vs. 2.98, 2.49, 2.47, and 1.43).

Q2: Intermediate predictions and memory.

No separate decoding head is used, and gϕg_\phi denotes the frozen VAE decoder of the base model. Each intermediate hidden state H(ℓk)\mathcal H^{(\ell_k)} passes through the backbone's own final output projection, the same layer that produces the final output, and is then decoded by gϕg_\phi. The decoding path therefore introduces no new parameters, and gϕg_\phi is neither initialized separately nor trained or modified during RL post-training (stated in Sec. 3 and Appendix E.1). The KK checkpoints are spaced uniformly over the DiT blocks, and each is decoded at the final denoising step, so every scored video comes from the end of the denoising trajectory, not from a noisy intermediate timestep. Per rollout:

  1. Sample GG candidates from πθold\pi_{\theta_{\mathrm{old}}}, decode the intermediate outputs at ℓ1,…,ℓK\ell_1,\dots,\ell_K, and score each once with RR.
  2. For k=1,…,Kk=1,\dots,K: compute returns and advantages from the stored rewards and select the winner; compute its Grad$\times$Input attribution under the incoming memory mˉk−1\bar{\mathbf m}_{k-1}, normalize it, and detach it as mk\mathbf m_k; take one gradient step on L(ℓk)\mathcal L^{(\ell_k)} under mˉk−1\bar{\mathbf m}_{k-1}, updating the DiT (all weights at 5B, LoRA adapters at 14B); pass mk\mathbf m_k to ℓk+1\ell_{k+1}.

Inference procedure: At inference, we sample the post-trained DiT with the base model's standard sampler, conditioned on the original instruction. Intermediate decoding, reward evaluation, and token attribution are used only during training, so inference adds no time cost over the base model.

Q3: Controlled budgets and additional controls.

Removals and budget: w/o Self-eval gives every checkpoint the terminal reward; w/o TI memory conditions every checkpoint on the original c\mathbf c. All Table 2 variants share the data, seeds, G=4G=4, and 300 rollout iterations, so every variant has the same rollout budget. The full method and the three component ablations take K=4K=4 gradient steps per rollout, while outcome-only GRPO takes one. At K=4K=4, one iteration takes 80 reward evaluations, versus 16 evaluations for outcome-only GRPO. The revision reports steps, rollouts, and wall-clock time for every variant.

Requested controls: The local-reward (γ=0\gamma=0) and discounted-return (γ=0.5\gamma=0.5) controls are already in Table 6 (both 4.20), and the uniform and random memory controls are under W2. We will add a four-step outcome-only control and a shuffled-memory control.

Uncertainty: Since VISTA is a post-training method, all variants start from the same pretrained weights and use fixed inference seeds, so each model's outputs are fixed and the remaining variation in its scores comes from the evaluator. We therefore use four judges: both official OpenVE judges and two others, none of which is the training reward, and two of which lie outside the reward's Qwen family. Because the judges score on different scales, we compare methods within each judge rather than across judges.

Q4: Data and evaluation protocol.

RL corpus and reward model: Both models and all Table 2 variants are post-trained on the same self-collected editing corpus of (source video, instruction) pairs (Sec. 4.1), so differences among the variants do not come from the data. In each iteration, the policy samples G=4G=4 candidate edits per pair with the rollout settings of Appendix E.1: a 30-step trajectory at 5B, and a 20-step trajectory with 33 frames at 480×832 at 14B. The corpus has no overlap with OpenVE-Bench or FiVE-Bench, so every evaluation case is unseen during post-training. The reward model is Qwen3.6-35B-A3B (Sec. 4.4, Appendix E.2). It is not one of the four evaluation judges, so no main comparison or ablation is scored by the training reward.

Hyperparameter selection: KK and γ\gamma were selected on OpenVE-Bench with the reward model as judge, and all KK settings and all γ∈[0,0.5]\gamma\in[0,0.5] lie within 0.03 of one another (4.17–4.20), so this selection has little effect. FiVE-Bench was not used for any selection, and every reported model uses the step-300 checkpoint.

Denominators in Table 14: The 300 comparisons per baseline are the collected comparisons, not the analyzed ones. As described in Appendix G.5, we excluded every comparison in which either output was an obvious generation failure, i.e., a video that is unusable as an edit, since its outcome is trivial and says nothing about the preference between valid edits. The rule depends only on whether each output is usable, so it treats both methods in a pair alike, and it applies only to the pairwise task. All percentages and tests in Table 14 are computed over the retained comparisons, which is why they do not correspond to whole counts out of 300.

Aggregation and uncertainty: In the rating task, a method's score on each axis is the mean of its 300 ratings, and Avg. is the mean of the three axes. In the pairwise task, each judgment counts as one comparison, without majority voting, and the pp-values come from a two-sided binomial test on the non-tied comparisons. Because each clip and each participant contribute several judgments, the revision computes the 95% intervals with a bootstrap that resamples both clips and participants, and adds a clip-level analysis based on the majority of the three judgments per clip.

We hope these clarifications address the reviewer's concerns and would be glad to provide further details.

Reviewer_c8DQOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper proposes VISTA, a depth-aware RL framework for video editing. It evaluates intermediate DiT outputs, uses return-to-go (RTG) for cross-layer credit assignment, and introduces token-importance memory to guide layer-wise GRPO. Experiments on 5B and 14B backbones report improvements on OpenVE-Bench and FiVE-Bench, supported by ablations and human evaluation.

Critical strengths

S1. It introduces interesting depth-aware RL formulation. Applying intermediate reward supervision along transformer depth, rather than denoising steps, offers an interesting perspective on RL post-training for video editing.

S2. The results on the 5B model are promising. The 5B model improves over Kiwi-Edit on multiple benchmarks, with additional support from human preference evaluation.

Critical weaknesses

W1. The validation of the proposed RTG and token-attribution mechanisms is insufficient. The main 5B configuration uses γ = 0 (Appendix E.1), eliminating downstream credit propagation, while Table 6 shows no average-score improvement from γ = 0 to γ = 0.5. Thus, the added benefit of RTG over local intermediate supervision remains unestablished. Moreover, the Grad × Input memory lacks comparisons with simpler token-weighting alternatives, making it unclear whether the gains stem from meaningful attribution or generic changes to text conditioning. These gaps weaken the evidence for the paper's central methodological contributions.

W2. The improvements at the 14B scale are inconsistent. Relative to SAMA, the 14B model achieves average-score changes of −0.05, 0.00, +0.02, and +0.06 across four VLM judges (Tables 9–12), with regressions in several editing categories (Figure 16). Although FiVE-Bench shows improvements in preservation-oriented metrics, the evidence for consistent gains in overall editing quality remains limited. The human evaluation also covers only the 5B variant.

Questions for authors

Q1. Appendix E.1 specifies γ = 0 for the main 5B configuration, whereas Section 4.4 selects γ = 0.5. Could the authors clarify the configurations used in the main results and the hyperparameter study? Given the identical average scores for γ = 0 and γ = 0.5 in Table 6, what evidence demonstrates the additional benefit of downstream reward propagation over local intermediate supervision? Please also clarify the discrepancy between Eq. (4) and the stated interpretation of γ.

Q2. Could the authors compare gradient-based token attribution with simpler alternatives, such as uniform or random token weighting, under matched settings? This would clarify whether the gains arise from meaningful token attribution or generic changes to text conditioning. Please also clarify the apparent mismatch between the ablation labels in Table 2 and their explanations in Section 4.3.

4 comments

We sincerely thank the reviewer for the careful and insightful review, and for the valuable suggestions on validating the return-to-go and the token attribution, which have helped us strengthen the paper. We address each point below.

W1/Q1: Configuration and the role of γ\gamma.

Configuration: The main 5B model uses γ=0.5\gamma=0.5. In fact, every reported experiment uses γ=0.5\gamma=0.5 except the γ\gamma sweep in Table 6, including the main 5B and 14B models, the Table 2 ablations, and the KK sweep. The value γ=0\gamma=0 in Appendix E.1 is a typo, so the main 5B results do include downstream credit propagation. We have corrected Appendix E.1 in the revision.

Role of γ\gamma: The return-to-go is Rg(ℓk)=∑j=kKγ j−kR(V^g(ℓj))\mathcal{R}^{(\ell_k)}_g=\sum_{j=k}^{K}\gamma^{\,j-k}R(\hat{\mathbf V}^{(\ell_j)}_g). Because of the exponent j−kj-k, γ=0\gamma=0 retains the local reward at ℓk\ell_k (γ0=1\gamma^0=1), and γ>0\gamma>0 adds discounted rewards from later checkpoints, as described in the text. The revised Eqs. (2) and (4) display this exponent explicitly.

Benefit of downstream propagation: We would like to clarify the scope of our claim. Our central claim is that depth-wise intermediate supervision improves over outcome-only training, which Table 2 supports: outcome-only GRPO reaches 2.98, removing intermediate self-evaluation gives 2.47, and the full method reaches 3.31. The return-to-go is the credit-assignment form of this supervision and contains local supervision as the special case γ=0\gamma=0. Please note that γ>0\gamma>0 is not the main source of the gain. In Table 6, γ=0.5\gamma=0.5 matches the best average (4.20) while giving the highest IC (3.86 vs. 3.81 for γ=0\gamma=0), the highest late-layer reward, and a more balanced profile across the three quality dimensions. As such, we adopt it as the default setting. The revised Sec. 4.4 now states explicitly that the method is robust for γ∈[0,0.5]\gamma\in[0,0.5], while larger discounts lower the average under the same budget.

W2: Results at the 14B scale.

We would like to offer a fuller reading of the 14B results. Since the 14B model is LoRA-adapted from SAMA, the strongest >13B baseline, its headroom is smaller than at 5B. Under the two official OpenVE-Bench judges, the 14B model obtains the highest average among all methods (3.33 and 4.06), and under the two additional judges it is on par with SAMA. The category-level differences in Figure 16 change sign across judges, which points to evaluator variance rather than a systematic regression. On FiVE-Bench, the 14B model improves 28 of 36 task–metric entries, including Struct, PSNR, and LPIPS on all six edit types.

Human study. We have added the 14B model to the human study (Appendix G.5):

MethodICCFVQAvg.
Omni-Video (1.3B)1.611.391.441.48
Lucy-Edit (5B)2.181.962.072.07
VINO (13B)2.512.222.292.34
Kiwi-Edit (5B)3.222.832.922.99
SAMA (14B)3.392.943.003.11
Ours (5B)3.443.143.173.25
Ours (14B)3.532.943.133.20

Human raters score the 14B model above SAMA (3.20 vs. 3.11), with gains on IC and VQ and a tie on CF, and rate both of our models above every baseline.

Q2: Token attribution versus simpler weighting.

As suggested, we trained a random TI memory control that replaces the Grad$\times$Input attribution with random token weights, keeping all else fixed. The complete ablation (5B, gemini-2.5-pro, G=4G=4, γ=0.5\gamma=0.5, 300 steps) is:

Training strategyAvg.ICCFVQ
Base model3.163.412.993.08
+ outcome-only GRPO2.983.242.832.87
w/o Self-eval2.472.892.242.29
w/o TI memory1.431.551.331.40
Random TI memory (new)2.492.942.232.31
Full method3.313.563.163.22

Because the memory sums to one, a uniform memory mn=1/Nm_n=1/N only scales every instruction token embedding by the same factor (1+1/N)(1+1/N), so w/o TI memory is a close proxy for uniform weighting. Neither simpler weighting reproduces the gain: uniform weighting reaches 1.43 and random weighting 2.49, both below the base model (3.16), whereas gradient-based attribution reaches 3.31 and improves all three dimensions. This indicates that the gain arises from meaningful token attribution rather than a generic change to text conditioning, consistent with the 5B gains concentrating on localized edits, where attribution falls on a few tokens.

Table 2 labels: The table above uses the correct mapping, w/o Self-eval: 2.47 and w/o TI memory: 1.43. The Sec. 4.3 explanations already follow this mapping, and only the component names were swapped; we have made the table and text consistent in the revision.

We hope these clarifications address the reviewer's concerns and would be glad to provide any further details.

Thanks for the new experiments and results, and I have raised the soundness score to 3.

Thank you for your consideration and for raising the soundness score to 3! We appreciate your positive feedback on our additional experiments and results. Given the clarifications and new results we have provided, we would also appreciate it if you could reconsider the overall rating. Thank you again for your time and consideration.

Dear reviewer c8DQ:

As the rebuttal period is coming to a close, we would like to follow up on our previous response and ask whether you have any remaining concerns that we could address. We would appreciate hearing your feedback before the discussion concludes. Thank you again for your time and consideration.

Best regards,

The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.