acceptodds
Under review as a conference paper at ICLR 2027

On-Policy Adversarial Fine-Tuning for Long-Horizon Autoregressive Video Generation

Abstract

Autoregressive video generation enables efficient long-horizon synthesis by repeatedly conditioning on its own outputs. However, this feedback allows small perturbations to a conditioning image to influence generation far beyond the initial frames. We investigate this vulnerability in distilled autoregressive video models. Our analysis links long-horizon sensitivity to local prediction discrepancies and their propagation, motivating corrective supervision during generation. We propose teacher-guided on-policy adversarial fine-tuning to improve distilled models with input robustness while retaining their few-step, closed-loop long-video generation capabilities. A frozen self-teacher provides corrective supervision along the model's own adversarial rollouts, with regularization preserving its original generative behavior. We further design an efficient attack surrogate to sustain frequent adversarial exposure during adaptation. Across long-video evaluations, our approach reduces mean latent perturbation response by 22.3–31.4% relative to the pretrained baseline, with a 14.5% reduction in later frames under attacks specifically targeting late-video generation. VBench-I2V further shows improved quality under attack with near-unchanged clean scores. These results demonstrate that distilled autoregressive generators can learn greater input robustness without additional inference cost.

Primary area:
generative models
Keywords:
Autoregressive video generation, Adversarial robustness, On-policy fine-tuning, Teacher-guided learning, Video diffusion, Long-horizon generation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_V5irOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies how small perturbations of the conditioning image affect long rollouts of distilled autoregressive image-to-video models. Native Self-Forcing shows growing sensitivity and HiAR is less sensitive but not robust. The authors fine-tune HiAR for 200 updates with "teacher-guided on-policy adversarial fine-tuning": the student rolls out on an adversarially perturbed image, and a frozen copy of the original model, given the same query and raw history but the clean image, provides a per-node prediction target. A cheap 7-block single-step surrogate generates attacks, refreshed every six updates, and one in three updates uses clean inputs to anchor normal behavior. Against model-specific white-box PGD, mean latent response falls 22–31%; VBench-I2V scores under attack improve. A control that instead uses the teacher's independent clean-trajectory prediction lowers response further but blurs outputs badly.

Critical strengths
  • The context-matched vs clean-trajectory target control is the most valuable finding. It shows that a lower sensitivity metric can hide a large quality loss, a useful caution for robustness work on generators.
  • Evaluation pipeline is well designed: own-model attacks, PGD40 and late-target PGD10 in addition to PGD20, a held-out confirmation set disjoint by parent and frame hash, paired whole-source bootstrap intervals, and explicit reporting of a source that gets worse.
Critical weaknesses
  • The threat model is not motivated. Who perturbs the conditioning image, and why? In practice, imperceptible perturbations of I2V inputs are mostly used defensively to stop misuse of personal photos. Robustness to them weakens those protections; this is a dual-use issue the ethics statement only touches.
  • The fine-tuned model drifts substantially on clean inputs. Its clean drift N = 0.49 exceeds the original model's response to random 8/255 noise, and random-input Q rises 39%. Lower sensitivity may partly come from relying less on fine image detail rather than from true robustness.
  • Scope is narrow: one base model , one data domain, one threat model, one training seed, and attacks that maximize latent MSE only. Attacks are not converged. Fig. 6 shows the PGD objective still rising at 40 steps, so the reported reductions are finite-budget numbers.
  • Simple defenses are missing as baselines: input purification (JPEG, blur, diffusion purification) or standard adversarial training with a clean-output target.
  • The paper is very hard to read. Dense, nonstandard terms ("source eviction", "parent-disjoint", "raw-state construction", "causal replay metadata", "ancestors of inherited cache entries") appear without definitions, and Sec. 2 needs several passes.
Questions for authors
  1. Threat model and dual use. Please describe a concrete scenario in which an adversary perturbs the user's own conditioning image. How does your fine-tuned model change the effectiveness of protective perturbations such as I2VGuard or Anti-I2V? If it neutralizes them, this belongs in the main paper, not a single ethics sentence.
  2. Robustness vs ignoring the image. Does the adapted model still respond correctly to meaningful image edits? A test of sensitivity to semantic edits would separate invariance to imperceptible noise from reduced use of the conditioning image.
  3. VBench-I2V numbers. The clean scores of original HiAR in Table 2 are exactly round while every other cell has finer precision. Were these computed in your setup with the same seeds as "Ours", or copied from elsewhere?
  4. Attack strength. Since PGD has not converged at 40 steps, please report results with many more steps, multiple restarts, and EOT over generation noise. Also try attacks with other objectives, e.g., targeted changes or VBench-quality degradation.
2 comments

W1/Q1. Motivation and dual use

We consider video generation from authorized product or character images. A third party handling these images could deliberately make small pixel changes that alter the generated subject, requiring regeneration or manual review. OAT ('Ours' in the paper) aims to keep the video faithful to the reference image. It may also weaken perturbations added by image owners to prevent unauthorized animation. We will clarify this scenario and discuss dual use in the revision.

W2/Q2. Meaningful image edits

Yes. OAT still follows edits to the input image. We tested 16 images (eight color/shape edits and eight object additions/removals), two seeds per model/image, and ~20-second videos. Original/edited inputs shared prompts and noise, and prompts did not reveal edits. For color/shape edits, DINOv2 similarity of tracked subjects to the edited reference rose from 0.746 to 0.814 (+0.068) for HiAR and 0.738 to 0.803 (+0.065) for OAT. Clothing colors and shapes followed edits throughout the videos. Removing a green bowl produced four bowls in place of five, with the bowl staying absent and other colors/layout largely preserved.

W3. Seeds and perturbation budgets

We reproduced the improvement in all three training runs. We repeated training with two new seeds, using the same HiAR weights and fixed 200-update recipe. On 160 confirmation sources, under each checkpoint's own prefix PGD20 (20-step projected-gradient descent), OAT reduced full/late latent response S by 22.11% ± 2.11 / 20.16% ± 2.25 percentage points (mean ± sample standard deviation across three training seeds).

OAT also reduced attack-induced changes at smaller and larger perturbation budgets. On 80 development sources, the fixed OAT checkpoint trained at 8/255 reduced full/late S by 33.17%/28.79% at 4/255, 30.20%/26.47% at 8/255, and 19.30%/15.17% at 16/255. Model-specific prefix PGD20 used steps of 1/255, 1/255, and 2/255, with paired prompts/noise. Prefix attacks target blocks 1–7, with full/late evaluation on blocks 1–27/21–27.

W4. Purification and target choice

All five preprocessing methods helped, and OAT did better at preserving the subject under attack. We re-optimized PGD20 through each full pipeline, targeting blocks 21–27 at ℓ∞ = 8/255. JPEG70 and diffusion used true forward passes and identity BPDA. Resizing, blur and noise+upscaling used direct gradients. Noise+upscaling and diffusion used four-draw EOT with fixed purifier noise. Late-segment DINOv2 distances to the input subject were 0.247 for JPEG, 0.261 for resizing, 0.270 for blur, 0.266 for noise plus upscaling, and 0.235 for diffusion purification, versus 0.294 for HiAR and 0.218 for OAT. JPEG+OAT further reduced it to 0.209, with clean distance changing from 0.192 to 0.195.

Matching teacher targets to the student's current context helped preserve detail. With an independent clean-trajectory target, S fell from 1.3539 to 0.9500, but clean/attacked MUSIQ image-quality scores also fell from 75.69/76.26 to 55.81/48.59, with blur or ghosting in later frames.

Q3. Table 2 provenance and precision

All four rows were computed in our own evaluation under the official VBench-I2V protocol, with paired seeds across HiAR/OAT and clean/attacked conditions. We generated five ~20-second videos per image–prompt pair, using the official image subsets for each metric. Each model received its own PGD20 under the same budget and procedure.

The scoring procedure can produce round numbers. We display two decimal places. For example, Dynamic Degree is the fraction of videos classified as having motion. With 246 inputs × five videos, 50.00%, 49.19%, 40.98%, and 48.05% are arithmetically consistent with 615, 605, 504, and 591 out of 1230 after rounding. Trailing zeros also occur in continuous-metric entries in Ours clean and HiAR attacked rows.

W3/Q4. Stronger attacks and generation randomness

OAT still produced smaller output changes when we used more attack steps and random restarts. On 16 fixed development sources, both fixed checkpoints received their own prefix PGD200 plus two independent PGD100 restarts at 8/255 with step size 1/255. For each source and model, we selected the quantized candidate with the highest prefix objective across all three trajectories and evaluated all 27 blocks. OAT's S reduction on these sources was 29.30% with PGD20, 26.89% with PGD200, and 25.77% including restarts (HiAR/OAT S = 3.4827/2.5851).

OAT also reduced output changes when we evaluated with fresh generation noise. EOT (Expectation over Transformation)-PGD20 averaged each model's prefix objective over four generation-noise draws and evaluated four separate held-out draws. We compared each attacked video with a clean video from the same model and noise draw. Full-video S was 0.9981 for HiAR and 0.7129 for OAT (28.57% lower).

W5. Clarity

We apologize for the unclear terminology and thank you for pointing it out.

Thank you again for your suggestions. We have further evaluated our on-policy adversarial fine-tuning (OAT) on Self-Forcing, adding evidence beyond HiAR and measuring subject fidelity alongside latent response.

We initialized the student from the official Self-Forcing DMD checkpoint (self_forcing_dmd.pt, EMA weights) and used an independent frozen copy as the teacher. We used Self-Forcing's blockwise autoregressive generation while retaining OAT's objective and fixed 200-update budget. A matched random-perturbation control used the same initialization, training data, update count, teacher-target construction, and optimizer, replacing only the adversarial training perturbations with random perturbations at the same budget.

We evaluated 16 image sources with two generation seeds each (32 videos per model and input condition), using paired prompts/noise and approximately 20-second videos. We averaged the two seeds within each source, then averaged across sources. Each model received its own prefix PGD20 attack at an ℓ∞ budget of 8/255 and step size 1/255, maximizing latent MSE on blocks 1–7 relative to its own clean output with the same prompt and noise. Full/late S used the same clean reference over blocks 1–27/21–27; the late result therefore measures the long-term effect of the prefix attack. Subject similarity is the DINOv2 cosine similarity between the tracked video subject and the input subject.

ModelFull S ↓Late S ↓Clean late subject similarity ↑Attacked late subject similarity ↑
Self-Forcing2.1482.7820.6590.542
+ random-perturbation training2.0192.6510.6500.547
+ OAT1.6972.3270.6380.558

OAT reduced full/late S by 21.0%/16.4% relative to Self-Forcing, and by 15.9%/12.2% relative to matched random-perturbation training. The attack-induced drop in late subject similarity was 0.117 for Self-Forcing, 0.103 for random training, and 0.080 for OAT. Thus, OAT achieved both a smaller attack-induced drop and higher subject similarity under attack, beyond the gains from random perturbation augmentation.

OAT's clean late subject similarity was 0.021 lower than Self-Forcing's, while its attacked similarity was 0.016 higher. The lower latent response therefore accompanies better subject fidelity under attack, with a measurable clean-fidelity cost. Visual comparisons showed reduced road-texture distortion and better retention of some clothing attributes, although long-video content errors remained.

Our earlier response also reported semantic-edit tests, purification comparisons, and stronger attacks on HiAR: OAT retained a 25.77% reduction in S with PGD200 plus two PGD100 restarts, and a 28.57% reduction with generation-noise EOT evaluated on held-out noise draws. The new Self-Forcing experiment further broadens the model coverage.

Due to the time required for a full VBench-I2V evaluation, complete scores for these Self-Forcing experiments are not yet available in this rebuttal; we will report them in the revised manuscript.

We will include the additional results and observed trade-offs in the revision. We would appreciate your updated assessment in light of these results and our earlier response.

Reviewer_LX19OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the input robustness of distilled autoregressive image-to-video generators such as Self-Forcing and HiAR. Because these models condition on their own outputs, a small perturbation to the conditioning image can keep altering the video long after the image leaves the attention window. The authors attribute this to local prediction residuals that propagate through generated history and propose teacher-guided on-policy adversarial fine-tuning. Along the student's own rollouts on perturbed images, a frozen copy of the original model provides targets computed with the clean image at the same generated state. Interleaved clean-input updates preserve normal behavior, and a cheap surrogate attack keeps adversarial exposure affordable. On HiAR, the method reduces latent perturbation response by 22–31% under adaptive PGD and by about 14% under late-targeted attacks. It also improves VBench-I2V scores under attack with nearly unchanged clean scores. A control experiment shows that supervising toward an independent clean trajectory lowers response further but blurs the generated videos.

Critical strengths
  1. The evaluation protocol is careful for a generative-model robustness paper. Attacks are adaptive and model-specific, where each model is attacked with white-box PGD that differentiates through the native four-stage generator, the encoder, and the full causal history. The results also hold under a stronger budget (PGD40) and under attacks aimed directly at late blocks through the full history. The paper reports a frozen confirmation set of 160 sources disjoint from training and development, paired source-level bootstrap intervals, and per-source counts. It also reports negative results, such as rising random-input QQ, worse late response after 400 updates, and controls whose intervals include zero. This makes the headline reductions credible within the tested setting and sets a good standard for future work on this problem.

  2. The target-control experiment yields a useful finding. Supervising robust updates toward an independently generated clean trajectory lowers the latent response metrics by another 25–30%. Yet it produces blur and ghosting on every source, and MUSIQ drops by about 18 even on clean inputs. Context-matched supervision avoids this, with its clean MUSIQ statistically indistinguishable from the original model. This shows that latent-distance robustness metrics alone can be misleading. The insight that teacher targets must respect the student's own generated context is likely relevant beyond robustness, like to on-policy distillation of autoregressive video models.

Critical weaknesses
  1. The threat model and the practical significance of the robustness gains are not well established. The attacker is assumed to perturb the conditioning image of an image-to-video generation, but the paper does not describe a concrete deployment scenario in which this causes harm. In practice, the most common source of such perturbations is protective tools such as I2VGuard and Anti-I2V, which add perturbations so that personal photos cannot be animated. Making generators robust to perturbations therefore mainly weakens these protections, and the paper does not discuss this. The measured gains are also moderate and mostly in latent space. Response falls by 22–31% under prefix attacks and by about 14% under late-targeted attacks. Under the late-targeted attack, attacked-output deviation from the original clean behavior (QQ) is essentially unchanged (+0.1%). The fine-tuned model's clean drift from the original (N≈0.49N \approx 0.49) is larger than its response to random perturbations. As a result, it is hard to judge whether the model has become more faithful under attack or has mainly become less sensitive to its input.

  2. The empirical scope is narrow, and there are no external baselines. All results come from a single model (HiAR), a single ℓ∞\ell_\infty budget of 8/255, and a single data domain (OpenVid). Self-Forcing is used to motivate the problem but is never adapted. The comparisons are all internal ablations. The paper does not compare with simple input-purification defenses (JPEG compression, noise and upscaling, or diffusion-based purification), which Hönig et al. [1] showed reliably neutralize protective perturbations for image models. It also does not compare with standard adversarial fine-tuning using the original distillation objective, or with a clean-only fine-tuning control. The frozen-state controls in Table 6, which are meant to support the on-policy design, have paired intervals that include zero. Longer training improves early response but worsens late response by 12.5%, which suggests the result is sensitive to the stopping point. Without these comparisons, it is difficult to tell whether the proposed on-policy teacher guidance is necessary, or whether a cheaper inference-time defense would achieve similar robustness.

References

[1] Robert Hönig, Javier Rando, Nicholas Carlini, and Florian Tramèr. Adversarial Perturbations Cannot Reliably Protect Artists From Generative AI. In International Conference on Learning Representations (ICLR), 2025.

Questions for authors
  1. What concrete deployment scenario motivates this threat model, and how does the method interact with protective perturbations such as I2VGuard and Anti-I2V? Would the fine-tuned model also neutralize those protections?

  2. How does the method compare with inference-time input purification (JPEG, Gaussian noise plus upscaling, or diffusion purification) and with off-policy adversarial fine-tuning under the same attack protocol? A clean-only fine-tuning control with the same 200 updates would also help.

  3. Do the gains hold on Self-Forcing or another distilled autoregressive video model and under larger or different perturbation budgets? How should the number of training updates be chosen without access to long-horizon evaluation, given that 400 updates hurt late response?

  4. Under the late-targeted attack, QQ is unchanged, and clean drift NN is substantial. Can the authors show that robustness improves semantic fidelity to the input, for example with identity or subject-consistency metrics computed against the clean output, or a small human study?

5 comments

Thank you for the feedback.

W1/Q1. Deployment and dual use

We consider video generation from authorized product photos or licensed character artwork. A third party handling the images before generation may deliberately introduce small pixel changes that alter the generated product or character, leading to inaccurate displays and extra review or regeneration work. OAT ('Ours' in the paper) aims to keep the video faithful to the reference image under such changes. Reduced sensitivity may also weaken perturbations that image owners add to discourage unauthorized animation. We will clarify this scenario and discuss dual use in the revised manuscript.

W2/Q2. Purification and training controls

All five preprocessing methods helped, and OAT did better at preserving the subject under attack. We re-optimized PGD20 (20-step projected-gradient descent) through each full pipeline, targeting blocks 21–27 at ℓ∞ = 8/255. JPEG70 and diffusion used true forward passes and identity BPDA. Resizing, blur and noise+upscaling used direct gradients. Noise+upscaling and diffusion used four-draw EOT with fixed purifier noise. DINOv2 cosine distances between tracked subjects and the input subject (lower is better) in the late segment were 0.247 for JPEG, 0.261 for resizing, 0.270 for mild blur, 0.266 for noise plus upscaling, and 0.235 for lightweight diffusion purification, versus 0.294 for HiAR and 0.218 for OAT. JPEG+OAT further reduced it to 0.209 (4.1%), with clean distance changing from 0.192 to 0.195.

Using the current student's states gave a lower average attack response than using states from frozen HiAR (S = 1.3539 versus 1.4184 and 1.4709 under each model's own PGD20). Two off-policy controls replace current-student states with frozen-original-HiAR states in the clean or robust branch.

Matching teacher targets to the student's current context helped preserve detail. With an independent clean-trajectory target, S fell to 0.9500, but clean/attacked MUSIQ image-quality scores also fell from 75.69/76.26 to 55.81/48.59, with blur or ghosting in later frames.

W2/Q3. Budgets, training length, and replication

OAT reduced the effect of attacks at both smaller and larger budgets, with a smaller gain at twice the training budget. On 80 development sources, we evaluated the fixed OAT checkpoint trained at 8/255 at three budgets. Each model received its own prefix PGD20, with paired prompts and generation noise. Attacks optimized blocks 1–7, with evaluation over all 27 blocks and late blocks 21–27.

BudgetStep sizeFull-video S reductionLate S reduction
4/2551/25533.17%28.79%
8/2551/25530.20%26.47%
16/2552/25519.30%15.17%

We fixed 200 updates before launching the three method-comparison groups, independently of development and confirmation results. This choice followed short post-training regimes in SF (Self Forcing), CF (Causal Forcing), and OPD (on-policy distillation). After ODE (ordinary differential equation) initialization, SF uses 600 DMD (distribution-matching distillation) training iterations, and CF describes approximately 100. The 400-update continuation was a subsequent control. We attribute its late deterioration primarily to continued optimization of the seven-block training rollout while evaluation spans 27 blocks. Short rollouts make on-policy training affordable, while longer training further optimizes this limited horizon. We will clarify this rationale in the revision.

We reproduced the improvement in all three training runs. We repeated the fixed recipe with two new seeds from the same HiAR weights. On the same 160 confirmation sources, each checkpoint's own prefix PGD20 yielded full-video S of 1.2512 ± 0.0340 and late-segment S of 1.2964 ± 0.0365, corresponding to reductions of 22.11% ± 2.11 and 20.16% ± 2.25 percentage points (mean ± sample standard deviation across three training seeds).

W1/Q4. Semantic fidelity under direct late-target attacks

OAT preserved the input subject better when we attacked the late blocks directly. On 80 development sources, we measured DINOv2 cosine similarity of tracked, background-removed subjects to the input subject, using paired prompts/noise and each model's own PGD20 targeting blocks 21–27. Clean HiAR/OAT similarities were 0.858/0.846 over the full video and 0.812/0.808 in the late segment, versus 0.751/0.812 and 0.706/0.782 under attack. OAT reduced attack-induced similarity loss by 68.2% and 75.5%. Paired examples also showed that OAT followed clothing-color edits and removal of a green bowl, retaining responsiveness to these image changes.

Thank authors for their response. I appreciate the clarification on deployment and on-policy comparisons as well as additional experiments on purification baselines, perturbation budgets, semantic fidelity. These results address most of my concerns. However, one major concern remains that the method is still evaluated only on HiAR. Results on another base model would help establish whether the benefits generalize beyond this particular backbone. If the authors can provide results supporting this generality, I would consider raising my recommendation to acceptance. For now, I maintain my current score.

Thank you for clarifying the remaining concern. We have now evaluated our on-policy adversarial fine-tuning (OAT) on Self-Forcing, the additional model you suggested. It improves robustness over both the original model and matched random-perturbation training.

We initialized the student from the official Self-Forcing DMD checkpoint (self_forcing_dmd.pt, EMA weights) and used an independent frozen copy as the teacher. We used Self-Forcing's blockwise autoregressive generation while retaining OAT's objective and fixed 200-update budget. A matched random-perturbation control used the same initialization, training data, update count, teacher-target construction, and optimizer, replacing only the adversarial training perturbations with random perturbations at the same budget.

We evaluated 16 image sources with two generation seeds each (32 videos per model and input condition), using paired prompts/noise and approximately 20-second videos. We averaged the two seeds within each source, then averaged across sources. Each model received its own prefix PGD20 attack at an ℓ∞ budget of 8/255 and step size 1/255, maximizing latent MSE on blocks 1–7 relative to its own clean output with the same prompt and noise. Full/late S used the same clean reference over blocks 1–27/21–27; the late result therefore measures the long-term effect of the prefix attack. Subject similarity is the DINOv2 cosine similarity between the tracked video subject and the input subject.

ModelFull S ↓Late S ↓Clean late subject similarity ↑Attacked late subject similarity ↑
Self-Forcing2.1482.7820.6590.542
+ random-perturbation training2.0192.6510.6500.547
+ OAT1.6972.3270.6380.558

OAT reduced full/late S by 21.0%/16.4% relative to Self-Forcing, and by 15.9%/12.2% relative to matched random-perturbation training. The attack-induced drop in late subject similarity was 0.117 for Self-Forcing, 0.103 for random training, and 0.080 for OAT. Thus, OAT achieved both a smaller attack-induced drop and higher subject similarity under attack, beyond the gains from random perturbation augmentation.

OAT's clean late subject similarity was 0.021 lower than Self-Forcing's, but its attacked similarity was 0.016 higher. Qualitative comparisons also showed reduced road-texture distortion, better retention of some clothing attributes, and improved kitten–bottle contact. Long-video content errors remain, but the improvements in both attack response and attacked subject fidelity support the method's effectiveness on this second autoregressive generator.

Due to the time required for a full VBench-I2V evaluation, complete scores for these Self-Forcing experiments are not yet available in this rebuttal; we will report them in the revised manuscript.

We will incorporate these results and the clean-fidelity trade-off into the revision. We hope this directly addresses your remaining concern and would appreciate your reconsideration.

Reviewer_D8xTOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the robustness of distilled autoregressive image-to-video generators to small perturbations of the conditioning image, whose effects propagate through the autoregressive feedback into late frames. The remedy is a short fine-tuning of HiAR: a frozen copy of the original model acts as a self-teacher that sees the clean image while the student sees the perturbed one, both placed at the same node of the student’s own rollout, and the student’s prediction is regressed onto the teacher’s; analogous clean-input updates serve as a regularizer, adversarial inputs come from a cheap surrogate attack, gradients are local to the sampled node, and inference is unchanged. Under white-box PGD, latent response falls across the whole video, VBench clean scores are roughly unchanged and attacked scores improve; a control that instead regresses onto the teacher’s independent clean trajectory lowers response further but produces progressive blur, which the authors use to argue that context-matched targets are necessary.

Critical strengths
  1. Clean formulation and a simple method: a frozen self-teacher with privileged access to the clean image, queried at the student’s own states, with no change to the deployed model or its inference cost.
  2. The evaluation is more careful than usual for this area: the attacks are properly white-box, the conclusion survives a larger budget, and random perturbations give a gradient-free cross-check; the confirmation set was fixed in advance, with per-source counts and intervals throughout. Notably, the unfavorable results are stated in the main text rather than buried.
Critical weaknesses
  1. The practical value is not shown. The only concrete source of such perturbations the paper names is protective perturbation of photos, while the natural settings where this robustness would plausibly matter are never tested. Meanwhile input purification, the standard cheap defense that equally leaves the generator untouched, is absent, so the “no inference cost” advantage cannot be weighed.
  2. Both headline properties are weaker than framed. Training covers only a short prefix of the deployed horizon, late gains look like propagation of reduced early injection, and attacking the late blocks directly yields a clearly smaller reduction; the adapted model’s clean outputs deviate from the original by more than a random perturbation does, and the drift grows with training. Having shown that latent distance can be lowered by blur, the paper still evaluates every control other than clean-trajectory with latent distance alone.
Questions for authors
  1. Does the baseline actually change generated content under natural perturbations such as re-encoding, resizing or watermarking? This decides whether the work has value outside adversarial settings.
  2. How was the training length chosen, and was it selected on the development set? The shrinkage from development to confirmation and the late degradation under longer training suggest it may have been tuned on evaluation data.

I would raise my score to clear acceptance if the authors show that the baseline actually changes content under natural perturbations (such as JPEG compression, resizing, or watermarking) and that the proposed method outperforms simple input purification in that setting, and if the main result is reproduced over several seeds or on another base model with a training-length selection protocol that does not depend on evaluation data.

11 comments

W1/Q1. Practical motivation and ordinary processing

We consider video generation from authorized product photos or licensed character artwork. A third party handling the images before generation may deliberately introduce small pixel changes that alter the generated product or character, leading to inaccurate displays and extra review or regeneration work. OAT aims to keep the video faithful to the reference image under such changes.

Yes. JPEG compression and resizing changed the subject's appearance, and OAT reduced the changes caused by JPEG. We measured DINOv2 cosine distance between tracked subjects and the original input subject (lower is better). HiAR's late-segment distance rose from 0.188 on clean inputs to 0.207 after JPEG70, 0.220 after half-size downsampling and upsampling, and 0.193 with a mild watermark. Examples showed gradual changes in clothing color and object appearance in the middle and late segments, milder than under adversarial attack. On the same images and protocol, JPEG70 increased OAT's distance from 0.192 to 0.195. OAT reduced JPEG-induced distance growth by about 84% (0.003 versus 0.019), keeping the subject closer to the input after compression.

W1/Q1. Input purification

All five preprocessing methods helped, and OAT did better at preserving the subject under attack. We re-optimized PGD20 (20-step projected-gradient descent) through each full pipeline, targeting blocks 21–27 at ℓ∞ = 8/255. JPEG70 and diffusion used true forward passes and identity BPDA. Resizing, blur and noise+upscaling used direct gradients. Noise+upscaling and diffusion used four-draw EOT with fixed purifier noise. Late-segment object distances were 0.247 for JPEG, 0.261 for resizing, 0.270 for mild blur, 0.266 for noise plus upscaling, and 0.235 for lightweight diffusion purification, versus 0.294 for HiAR and 0.218 for OAT. JPEG+OAT further reduced it to 0.209 (4.1%), with clean distance changing from 0.192 to 0.195.

W2. Long-horizon behavior and input fidelity

When a perturbation causes less error in the early frames, there is less error to carry into later frames. Each new prediction depends on earlier outputs, even after the reference image leaves the attention window. This helps explain why training on seven blocks can improve a 27-block video. Appendix B.1 explains this under linearized propagation with fixed operator Φ, where scaling an injected discrepancy by α scales downstream squared response by α².

When we attacked the late blocks directly, OAT better preserved the subject, reducing the similarity drop by 68.2% overall and 75.5% late. Each model's own PGD20 targeted blocks 21–27 on 80 development sources, with paired prompts and generation noise. DINOv2 cosine similarity of tracked, background-removed subjects to the input subject was as follows (higher is better).

ConditionHiAR full/lateOAT full/late
Clean0.858 / 0.8120.846 / 0.808
Direct late-target attack0.751 / 0.7060.812 / 0.782

OAT still follows edits to the input image. We tested meaningful edits on 16 images (eight color/shape edits and eight object additions/removals), with two seeds per model and image and approximately 20-second videos. Original/edited inputs shared prompts and noise, and prompts did not reveal edits. For color/shape edits, similarity to the edited reference rose from 0.746 to 0.814 (+0.068) for HiAR and 0.738 to 0.803 (+0.065) for OAT. OAT followed red-to-blue clothing changes and removal of a green bowl, retaining four bowls in place of five while largely preserving the remaining colors and layout.

Q2. Training length and replication

We fixed 200 updates before launching the three method-comparison groups, independently of development and confirmation results. This choice followed short post-training regimes in SF (Self Forcing), CF (Causal Forcing), and OPD (on-policy distillation). After ODE (ordinary differential equation) initialization, SF uses 600 DMD (distribution-matching distillation) training iterations, and CF describes approximately 100. The 400-update continuation was a subsequent control. We attribute its late deterioration primarily to continued optimization of the seven-block training rollout while evaluation spans 27 blocks. Seven-block rollouts make on-policy training affordable.

We reproduced the improvement in all three training runs. We repeated the fixed recipe with two new seeds from the same HiAR weights, varying image order, generation noise, attack initialization, and stage/block sampling. On the same 160 confirmation sources, each checkpoint received its own prefix PGD20 and 27-block evaluation. Full-video S was 1.2512 ± 0.0340 and late-segment S was 1.2964 ± 0.0365, reducing HiAR's latent response by 22.11% ± 2.11 and 20.16% ± 2.25 percentage points (mean ± sample standard deviation across three training seeds). We will clarify the motivation and training-budget rationale in the revision.

I thank the author's response. Two more questions:

  1. What is OAT method? Is OAT the proposed method in paper?
  2. And if OAT is the proposed method, the results in the new report include OAT+jpeg70, but OAT+resize and OAT+watermark are missing; could you please provide the results for these two as well? These results can also verfiy the performance of proposed method.

Thank you for the follow-up.

1. Meaning of OAT. Yes. OAT refers to our proposed teacher-guided on-policy adversarial fine-tuning method. The results labeled “OAT” use the fine-tuned HiAR model labeled “Ours” in the paper. We apologize for introducing this abbreviation without defining it.

2. Results for resizing and watermarking. We provide both results below, alongside the clean-input and JPEG70 results. We measure the late-segment DINOv2 cosine distance between tracked subjects and the original input subject, where lower is better. “Increase” denotes the distance after processing minus the same model’s clean-input distance.

Input processingHiAR distanceOurs (OAT) distanceHiAR increaseOurs increase
Clean0.1880.192——
JPEG700.2070.195+0.019+0.003
Resize to half size and back0.2200.198+0.032+0.006
Mild watermark0.1930.194+0.005+0.002

Our method reduces the additional appearance drift caused by all three tested transformations. Resizing causes the largest increase for HiAR, and our method reduces this increase from 0.032 to 0.006, an 81.25% reduction, while also achieving a lower final distance (0.198 versus 0.220). The mild watermark has little effect on either model. Our method shows a smaller increase from its own clean level (+0.002 versus +0.005), with similar final distances (0.194 versus 0.193).

Together, these results show greater stability across the tested image-processing operations, with better absolute subject fidelity under JPEG compression and resizing.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.