acceptodds
Under review as a conference paper at ICLR 2027

VPG: Visual Prefix Guidance for Next-Scale Autoregressive Image and Video Generation

Abstract

Autoregressive image and video generators are trained with teacher-forced histories but must sample from their own generated prefixes at inference time, creating exposure bias: early prediction errors can become part of the conditioning context and influence subsequent predictions. Existing remedies mainly modify the training or fine-tuning procedure to expose the model to perturbed or model-generated prefixes, while sampling-time guidance primarily strengthens external semantic conditions such as class labels or text prompts. We propose Visual Prefix Guidance (VPG), a training-free inference-time guidance method for visual autoregressive generation. VPG contrasts predictions under the generated and corrupted prefixes and favors candidates that are both sensitive to and positively supported by the generated prefix, encouraging continuations more consistent with the realized visual history. Across class-conditional image generation with VAR, text-to-image generation with Infinity, and text-to-video generation with InfinityStar, VPG improves generation quality without retraining or fine-tuning, reducing FID on VAR by 0.40 on average and improving benchmark performance on both image and video generation.

Primary area:
generative models
Keywords:
Generative Models, Autoregressive Visual Generation, Inference-Time Guidance

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_NyFAOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes VPG, a training-free inference-time guidance for next-scale visual autoregressive models. Where CFG contrasts along the condition axis, VPG contrasts the prediction under the generated prefix with the prediction under its corrupted copy (same-scale full-embedding replacement), extrapolates the logits, and interprets this as prefix-axis log-ratio guidance. It needs no extra model or retraining, composes with CFG, and reports gains on VAR, Infinity, and InfinityStar. The method seems promising, yet two major concerns remain. First, the comparison with existing training-free guidance uses quoted numbers from a different environment, so whether the prefix axis actually adds something beyond existing guidance is not shown. Second, exposure bias is the stated motivation but it is never measured, and by its mechanism VPG may amplify errors in the prefix rather than prevent them.

Critical strengths
  1. The practicality of VPG is strong, as it is training-free, needs no auxiliary model, applies to three public checkpoints directly, and may work harmoniously with CFG.

  2. The VAR results are robust. Tab. 9 reports five-seed statistics, with λ\lambda selected on seed 42 (as it seems only seeds 43-46 are used to claim that the chosen λ\lambda is robust) and the gain holding on the four held-out seeds. Tab. 5 rules out alternative references, and Fig. 11 shows this is not just added diversity.

Critical weaknesses
  1. The comparison with existing training-free guidance is concerning because of a possibly unfair comparison. The paper picks SSG as the only training-free comparison in Tab. 2, but the SSG numbers are values quoted from the SSG paper. The official VAR checkpoint is known to give worse FID than the VAR paper reports, so SSG measured its own VAR baseline, and this paper also runs its own CFG baseline, which differs again (for d24, 2.39 for SSG against 2.16 here). So the two methods sit on different baselines, and absolute FID comparison is meaningless. At d24 the paper's own CFG baseline (2.16) is already below the SSG result (2.20), so "outperforms SSG on four of five depths" is just a pipeline difference. SSG adds no forward pass, while VPG in Tab. 2 costs 2×2\times Transformer FLOPs, and the table has no computation cost column. The code of SSG is public, so a same-pipeline, compute-matched comparison seems feasible. IGG [1], a token-adaptive guidance for VAR, belongs in the same discussion as well. Since the novelty claim is the prefix axis, a fair comparison is required to judge whether this method actually outperforms the existing ones.

  2. The motivation and the mechanism do not match. The paper motivates VPG with exposure bias, but there are two problems. First, no experiment measures exposure bias itself. There are no error propagation measurements or related experiments. The evidence is only FID, DPG, and VBench, so "exposure bias is reduced" is not something the paper shows. Instead, it reads more like an unsubstantiated explanation put on top of a better score. Second, the mechanism may go opposite to the intention of the authors, as Eq. (8) favors candidates that are more faithful to the generated prefix. If the prefix is already wrong (or contaminated), VPG pushes later scales to be more consistent with that wrong prefix. That is the error propagation of exposure bias itself, not its prevention. No experiment separates the two. The same-seed comparison in Fig. 12 has VPG on from the start, so it cannot separate them either. From my perspective, what the paper actually shows is that conditioning more sharply on the prefix improves FID.

Questions for authors
  1. Can you reproduce SSG in the same pipeline, with the same seeds and matched compute? Could you also compare with IGG [1] under the same setting?

  2. Given that the T2V result is single-seed with no variance, can you add a second T2I benchmark such as GenEval, and multi-seed VBench? Also, given that Tab. 13 shows all-scale VPG gives almost nothing (83.91 vs. 83.86) and only the coarse-scale schedule reaches 84.22, how was this setting chosen, and was it selected on the same VBench run reported in Tab. 4?

  3. Given that the gain shrinks from 0.622 at d16 to 0.121 at d30, and VPG on d30 ends up the same as VPG on d24, is there any result on a larger or stronger backbone that addresses the scalability of the method?

  4. Can you inject controlled errors at coarse scales and measure the downstream effect with and without VPG, so as to separate whether VPG prevents error propagation or amplifies it?

[1] Rethinking Visual Autoregressive Sampling with Information-Grounding Guidance (ICML 2026)

7 comments
AuthorsOpenReview ↗

Response to Reviewer NyFA (1/3): W1 and Q1

Thank you for the technically substantive feedback. Our response is split into three comments: W1 and Q1 here, W2 and Q4 in (2/3), Q2 and Q3 in (3/3).

W1 / Q1 (same-pipeline comparison with SSG and IGG; compute).

Thank you for the constructive feedback. The concern is that our advantage over SSG could reflect different pipelines and baselines rather than the method. In a same-pipeline comparison, which replaces the quoted SSG numbers, the advantage persists on four of the five models, and SSG is ahead on d30. The coarse-scale schedule costs 22% more Transformer FLOPs than SSG, and the revised Tab. 2 now reports this cost. SSG and IGG [1] were re-run in our pipeline with the same official checkpoints of VAR and 50k-sample ADM protocol, the same seeds (42–46), and the official VAR sampler (CFG 1.5 linear, top-k 900, top-p 0.96). SSG uses the official update rule from the released code. IGG follows the released code and Eq. 6 of its paper.

Each baseline was tuned on seed 42. VPG keeps the paper's settings: all-scale VPG uses the λ of Tab. 7, and the coarse-scale schedule the λ of Tab. 12. Tab. 12 has no d36 entry, so that λ (1.8) was chosen on seed 42, like the baselines. Tuning grids:

  • SSG: scale {0.25, 0.5, 1.0} × temperature {1.0, 1.15, 1.3} on d16, plus scales 0.1 and 2.0 at T = 1.15 to close the grid edges, and the neighbors of the optimum on d20–d36. Our SSG is below the FID reported in the SSG paper on d16–d24 (e.g. d16 3.05 vs 3.27, d24 1.92 vs 2.20) and within 0.05 of it on d30–d36. SSG needs a higher temperature: at T = 1 every SSG scale is worse than CFG on d16 (4.02–6.58 vs 3.40), while at T = 1.15 it reaches 3.05.
  • IGG: w ∈ {1.0, 1.5, 2.0, 2.5} on d16, {1.0, 1.5, 2.0} on d20–d30 and {1.5, 2.0, 2.5, 3.0} on d36. w = 1.5 is best on d16–d30 and w = 2.0 on d36 (the IGG paper uses 2.1 there).
  • CFG temperature: T ∈ {0.9, 0.95, 1.0, 1.02, 1.05, 1.1, 1.15}; the best T is 1.05, 1.1 and 1.05 on d16–d24, the official T = 1 on d30, and 0.95 on d36.

No selected setting lies at the edge of its grid.

Five-seed results (FID ↓, mean ± s.d.; Transformer FLOPs relative to CFG; the coarse-scale schedule costs 1.13× on d36, whose finest maps are larger). CFG and all-scale VPG are the five-seed means already in the paper (Tab. 9).

ModelCFG (1.00×)CFG, tuned T (1.00×)SSG (1.00×)IGG (1.00×)VPG, coarse (1.22×)VPG, all scales (2.00×)
VAR-d163.37 ± 0.033.16 ± 0.023.05 ± 0.023.42 ± 0.022.74 ± 0.012.74 ± 0.03
VAR-d202.71 ± 0.022.47 ± 0.012.35 ± 0.032.70 ± 0.032.25 ± 0.022.26 ± 0.02
VAR-d242.16 ± 0.032.03 ± 0.021.92 ± 0.022.16 ± 0.021.76 ± 0.021.84 ± 0.02
VAR-d301.97 ± 0.021.97 (T = 1)1.70 ± 0.011.95 ± 0.021.83 ± 0.011.84 ± 0.02
VAR-d36 (512²)2.65 ± 0.012.60 ± 0.032.44 ± 0.022.59 ± 0.022.33 ± 0.022.17 ± 0.02

With every method tuned and run on the same seeds, VPG with the coarse-scale schedule lowers FID by 0.40–0.63 on d16–d24, where SSG lowers it by 0.24–0.36 and CFG with a tuned temperature by at most 0.24. VPG is ahead of these two baselines in each of the five seeds at each of these depths. On d36 VPG is best at both costs, and on d30 SSG is best. IGG stays within 0.06 of CFG, in line with the IGG paper's own VAR results (d30: 1.93 → 1.92; d36: 2.61 → 2.56).

Revised Tab. 2 replaces the quoted SSG column with these rows and adds the FLOPs row; App. C.3 gives the protocol and grids. Sec. 2 now discusses IGG.

References (for parts 1/3 and 2/3)

[1] Nguyen et al. Rethinking Visual Autoregressive Sampling with Information-Grounding Guidance. ICML 2026.

[2] Stein et al. Exposing Flaws of Generative Model Evaluation Metrics and Their Unfair Treatment of Diffusion Models. NeurIPS 2023.

AuthorsOpenReview ↗

Response to Reviewer NyFA (2/3): W2 and Q4

W2 / Q4 (motivation vs. mechanism; controlled errors).

Thank you for this point and for the proposed test. The concern is that the paper never measures exposure bias, and that VPG may amplify errors in the prefix rather than prevent them. Your test confirms the second point for an error that is already committed: VPG follows a planted error more closely than CFG. What we can now measure in support of the motivation is: the prefixes that VPG generates are closer to those of real images than CFG's. This is one aspect of dealing with exposure bias and does not show reduced error propagation.

Exposure bias is the mismatch between the ground-truth (teacher-forced) histories seen in training and the generated histories seen at inference. The two experiments below measure different parts of it: the distributional analysis measures how far the prefixes VPG generates drift from teacher-forced ones, and the controlled-error test measures how VPG responds to an error that is already in the prefix.

(i) Distributional analysis. We decode the latent after each scale and compare the generated partial images with the same partial decodes of real ImageNet images encoded by the VAR tokenizer, i.e., with teacher-forced prefixes (FD-DINOv2 [2] (lower is closer), mean ± s.d. over seeds 42–44 with 10k samples each, per token map):

FD ↓ per token map2×23×34×45×56×610×1016×16 (final)
CFG32.3 ± 0.633.2 ± 0.138.1 ± 0.545.4 ± 0.157.9 ± 0.9186.9 ± 0.7474.5 ± 1.0
CFG, T = 1.134.9 ± 1.035.0 ± 0.239.7 ± 0.446.3 ± 0.655.8 ± 1.1182.0 ± 1.3466.6 ± 1.7
CFG, scale 1.2533.0 ± 1.133.4 ± 0.338.4 ± 0.746.0 ± 1.057.5 ± 0.6180.8 ± 1.2461.1 ± 1.9
SSG (T = 1.15)34.0 ± 0.532.3 ± 0.336.5 ± 0.441.8 ± 0.352.5 ± 0.9180.6 ± 1.0469.7 ± 2.4
VPG, all scales26.3 ± 0.426.3 ± 0.431.7 ± 0.641.2 ± 0.157.5 ± 0.4183.3 ± 1.2447.4 ± 2.6
VPG, coarse-scale schedule25.5 ± 0.425.3 ± 0.431.1 ± 0.940.3 ± 0.455.5 ± 0.3178.4 ± 0.3442.1 ± 2.0

On the 2×2–4×4 maps, only VPG moves the prefixes clearly toward teacher-forced ones. Both VPG settings lower FD there by 6–8, and the coarse-scale schedule (with larger λ) also gives the closest final images. The three controls also add diversity and lower the final FID, yet on these maps they stay within 2.6 of CFG. A higher temperature on the coarse maps moves them further from teacher-forced ones (4×4: 39.7 ± 0.4 at T = 1.1 against 38.1 for CFG).

Most of the FID gain comes from the coarse scales (VPG switched on at chosen scales, paired random numbers, CFG reference 3.40):

VPG applied on mapsall2×2–8×82×2–4×410×10–16×16
FID ↓2.732.883.103.29

The 2×2–8×8 maps hold 23% of the tokens but give 77% of the gain, and the 2×2–4×4 maps give 45% of the gain. Across depths, the reduction in coarse-map drift is largest on d16 and absent on d30 (mean over the 2×2–4×4 maps, CFG − VPG, three seeds):

d16d20d24d30
Drift reduction6.42.72.9−0.6
FID gain (Tab. 9)0.620.450.320.12

Larger models already produce coarse prefixes closer to teacher-forced ones (CFG FD on the 2×2 map: 32 on d16, 28 on d30). On d30, VPG still lowers FID by 0.12 without reducing coarse drift, so drift does not account for the whole gain.

(ii) The controlled test. It isolates an error that is already committed: VPG is switched on only afterwards. We sample with CFG up to the 4×4 (or 8×8) scale, replace 30% of its codes with random codes, and continue the same prefix with CFG or with VPG.

Injected atContinuationFID ↓ clean → injectedDistance to clean twin
4×4CFG3.40 → 3.260.281
4×4VPG (scales 5–10)3.17 → 3.600.311
8×8CFG3.40 → 3.430.182
8×8VPG (scales 8–10)3.29 → 3.660.209

Distance: DINOv2 cosine distance (10k pairs). Paired difference VPG − CFG: +0.030 [0.027, 0.033] and +0.027 [0.025, 0.029]. We use the paired distance, which compares each sample with its uninjected twin. VPG follows the planted error more closely than CFG, and its FID ends up above CFG's. Eq. 8 implies this: it favors what the prefix supports and cannot tell that a planted code is wrong.

So the answer to Q4 is that VPG does not prevent the propagation of an error already in the prefix. It follows it more closely, and correcting such errors is what training-time remedies (scheduled sampling, noisy-context training) address. What the measurements support is: in normal sampling, VPG's coarse prefixes are closer to those of real images; i.e., on these maps VPG narrows a mismatch between generated and teacher-forced prefixes that relates to exposure bias.

The Introduction and Limitations now state this scope: VPG reduces prefix drift during sampling and does not reduce an error already present in the prefix. App. D.5 reports both experiments.

AuthorsOpenReview ↗

Response to Reviewer NyFA (3/3): Q2 and Q3

Q2 (multi-seed VBench; schedule selection; GenEval).

The questions are whether the video gain holds beyond a single seed, how the schedule was chosen and whether it was chosen on the reported run, and whether the text-to-image gain carries over to GenEval [3]. The outcome is mixed: the video gain is positive in all four seeds (+0.18 ± 0.13 on the three seeds not used to choose the schedule). On the three held-out seeds, the chosen schedule has the highest mean gain. GenEval does not improve, while the DPG-Bench gain holds in every seed and per-prompt diversity rises on every measure we used.

(i) Multi-seed VBench. On the three seeds not used to select the schedule, VPG raises VBench Overall over InfinityStar's original sampler without VPG by +0.18 ± 0.13, and the gain is positive in all four seeds (+0.22 ± 0.14 including the selection seed; Tab. 10, bottom). Sec. 5.2 now reports the four-seed result.

InfinityStar-8B, four seedsQuality ↑Semantic ↑Overall ↑ (mean ± s.d.)
Original sampler84.2382.5083.88 ± 0.10
+ VPG84.4782.6584.10 ± 0.22

(ii) Schedule selection. Yes, the schedule was selected on the seed-0 run of Tab. 4. On the three seeds not used for that choice, we ran all four combinations of guided scales and guided repeats. The chosen schedule has the highest mean gain (+0.18 ± 0.13, against +0.07 to +0.09) and is the cheapest (1.07× FLOPs), but the differences between the means (at most 0.11) are smaller than each schedule's s.d. (0.13 to 0.24), so three seeds do not show that one schedule is consistently best. Its gain comes mainly from dynamic degree and scene content, while temporal-consistency metrics change little (Sec. 5.2). On these seeds, the gain lies in the Quality score, and the seed-0 Semantic gain of Tab. 4 does not recur.

Guided scalesGuided repeatsTransformer PFLOPs per clip ↓ΔOverall, mean ± s.d. over 3 seeds ↑
first 12 of 15 (ours)first21.76 (1.07×)+0.18 ± 0.13
all 15first35.66 (1.76×)+0.09 ± 0.24
first 12 of 15all24.66 (1.21×)+0.09 ± 0.17
all 15all40.61 (2.00×)+0.07 ± 0.20

(iii) GenEval. VPG does not improve GenEval: Overall is 0.707 ± 0.007 with VPG, against 0.707 ± 0.006 without it (mean ± s.d. over five seeds; Infinity-2B with the λ and n_p of DPG-Bench and Infinity's prompt rewriter, 553 prompts, four images each). What VPG does change in text-to-image is sample diversity, consistent with its higher recall on ImageNet: images for the same prompt are more varied on every measure we used (CLIP distance [4] +6% on DPG-Bench prompts; DreamSim [5] +6% on OneIG-Bench [6]; LPIPS [7] +2.6%, higher on 98% of prompts). App. D.6 reports the GenEval categories (Tab. 20) and these diversity measures (Tab. 21).

Q3 (scalability).

Thank you for raising this point. The question is whether VPG's gain survives at larger scale, given that it shrinks from 0.62 FID on d16 to 0.12 on d30. At 256² the gain does shrink with depth, but it stays positive at every depth (no paired 95% interval includes zero; Tab. 9). As you note, VPG on d30 reaches the same FID as VPG on d24 (1.84 for both with all-scale VPG, and 1.83 on d30 against 1.76 on d24 with the coarse-scale schedule). VPG thus narrows the gap between the two depths (CFG: 2.16 on d24, 1.97 on d30) rather than preserving it, since its gain is smaller where the CFG baseline is already stronger. On the largest checkpoints we could test, the gain is clear on VAR-d36 (0.48 FID, 5/5 seeds) and positive in all four seeds on InfinityStar-8B.

On the larger backbones, VAR-d36 (2.3B) at 512² improves from 2.65 to 2.17 FID (Tab. 9) and is ahead of SSG (2.33 with the coarse-scale schedule and 2.17 on all scales, against 2.44). On InfinityStar-8B, the Overall gain is +0.18 ± 0.13 on the three held-out seeds.

At 256², with CFG alone, the coarse prefixes of larger models are already closer to teacher-forced ones (FD-DINOv2 [2] averaged over the 2×2–4×4 maps: 34.5 on d16, 30.5 on d30). VPG's reduction of this drift shrinks as well, from 6.4 on d16 to about 3 on d20–d24 and none on d30 (reply to W2/Q4). The correspondence is not exact. The drift reduction matches the FID gain at the two ends of the range but not between d20 and d24 (reductions 2.7 and 2.9, FID gains 0.45 and 0.32). On d30 VPG still lowers FID by 0.12 without reducing coarse drift, so drift does not account for the whole gain. References ([1, 2] in part 1/3)

[3] Ghosh et al. GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment. NeurIPS 2023.

[4] Radford et al. Learning Transferable Visual Models From Natural Language Supervision. ICML 2021.

[5] Fu et al. DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Data. NeurIPS 2023.

[6] Chang et al. OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generation. arXiv:2506.07977, 2025.

[7] Zhang et al. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. CVPR 2018.

Reviewer_JCyBOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Visual Prefix Guidance, a training-free sampling method for next-scale autoregressive image and video generation. VPG corrupts a prefix by replacing selected embeddings with full code-position embeddings from other positions at the same scale. It then contrasts predictions from the original and corrupted prefixes to favor continuations consistent with the generated history. The method can be combined with existing guidance and is tested on VAR, Infinity, and InfinityStar.

Critical strengths
  • The method improves released models without retraining. FID decreases for all five tested VAR sizes, and Infinity’s DPG-Bench score rises from 83.03 to 83.66. I lean toward acceptance mainly because the gains hold across different model sizes and both class-conditional and text-conditioned generation.
  • I find the ablations useful because they test both the corruption design and simpler sampling adjustments. On VAR-d16, full-embedding replacement works, while token-only replacement, position-only replacement, and the other weak references tested do not. The CFG-scale and temperature comparisons are useful for checking whether VPG’s gains can be obtained by tuning standard sampling parameters alone.
Critical weaknesses
  • My main concern is the link to exposure bias. A continuation can agree with its prefix and still be wrong if an early scale already contains an error. The paper distinguishes the corrupted reference from the prefix marginal in the Bayes argument. Its ablations support the reference choice, but I am not sure they show reduced error propagation. Could the authors introduce a controlled error into an early generated scale and test whether VPG reduces or amplifies its effect on later predictions?
  • The video results are harder to interpret. All-scale VPG changes VBench Overall from 83.86 to 83.91, while the coarse-scale schedule reaches 84.22. Table 13 changes both the guided scales and whether later repeats receive guidance, so their individual effects are unclear. Could the authors test these two choices separately? Motion and temporal-consistency scores would also help clarify whether the gain reflects better temporal behavior or mainly improved semantics.
Questions for authors

Please see "Critical Weaknesses" part for reference.

4 comments
AuthorsOpenReview ↗

Response to Reviewer JCyB (1/2): W1

Thank you for the thoughtful and technically precise feedback. W1 is answered below and W2 in a second comment.

W1 (link to exposure bias; controlled early error).

Thank you for raising this point and for proposing the test. The question is whether reinforcing the generated prefix reduces error propagation or reinforces mistakes. For an error that is already in the prefix, VPG reinforces it: in your test it follows the planted error more closely than CFG. Separately, the coarse prefixes that VPG generates itself are closer to those of real images.

Exposure bias is the mismatch between the ground-truth (teacher-forced) histories seen in training and the generated histories seen at inference. The two experiments below measure different parts of it: the distributional analysis measures how far the prefixes VPG generates drift from teacher-forced ones, and the controlled-error test measures how VPG responds to an error that is already in the prefix.

(i) Distributional analysis. We compare partial decodes after each scale with the same partial decodes of real ImageNet images, i.e. teacher-forced prefixes (FD-DINOv2 [1], VAR-d16, three seeds; lower is closer):

FD ↓ per token map (mean ± s.d.)2×23×34×4
CFG32.3 ± 0.633.2 ± 0.138.1 ± 0.5
VPG26.3 ± 0.426.3 ± 0.431.7 ± 0.6

On these maps VPG's prefixes are closer to the teacher-forced ones than CFG's. A higher temperature, a lower CFG scale, and SSG also lower the final FID, but on these maps they stay within 2.6 of CFG over the same three seeds, while VPG is 6–7 lower (all token maps: reply to Reviewer NyFA, W2). This is a distributional measurement on the coarse maps between self-generated prefixes and teacher-forced prefixes.

(ii) Controlled test. On VAR-d16, 30% of the 4×4 (or 8×8) codes are replaced with random codes, and the same prefix is continued with CFG or with VPG, which is switched on only after the injected scale (distance: DINOv2 cosine distance between each injected sample and its clean twin).

Injected atContinuationFID ↓ clean → injectedDistance to clean twin
4×4CFG3.40 → 3.260.281
4×4VPG (scales 5–10)3.17 → 3.600.311
8×8CFG3.40 → 3.430.182
8×8VPG (scales 8–10)3.29 → 3.660.209

VPG follows the planted error more closely than CFG (paired distance +0.03, 95% CI excludes zero), and its FID ends above CFG's (3.60 vs 3.26, and 3.66 vs 3.43). The rule implies this: it favors what the prefix supports and cannot tell that a planted code is wrong. We use the paired distance, which compares each sample with its own uninjected version.

So VPG does not reduce the propagation of an error that is already committed, and correcting such errors is what training-time remedies (scheduled sampling, noisy-context training) address. What the measurements support is: in normal sampling, VPG's coarse prefixes are closer to those of real images; i.e., on these maps VPG narrows a mismatch between generated and teacher-forced prefixes that relates to exposure bias.

The revised Introduction and Limitations state this scope, and App. D.5 reports both experiments.

References

[1] Stein et al. Exposing Flaws of Generative Model Evaluation Metrics and Their Unfair Treatment of Diffusion Models. NeurIPS 2023.

AuthorsOpenReview ↗

Response to Reviewer JCyB (2/2): W2

W2 (video: scales vs. repeats; temporal metrics).

Thank you for asking to separate the two choices. The questions are whether guided scales or guided repeats drive the video gain, and whether the gain reflects temporal behavior or mainly semantics. On the three seeds not used to select the schedule, the four combinations differ by less than their seed-to-seed standard deviation. The chosen schedule has the highest mean gain and is the cheapest, and its gain comes from more motion and scene content rather than better temporal consistency. We ran all four combinations on each of these seeds (each scale is predicted in up to three repeated passes; "first" guides only the first pass). ΔOverall against InfinityStar's original sampler without VPG (20.31 PFLOPs per clip), full VBench:

Guided scalesGuided repeatsTransformer PFLOPs per clip ↓Mean ± s.d. over 3 seeds ↑
first 12 of 15 (ours)first21.76 (1.07×)+0.18 ± 0.13
all 15first35.66 (1.76×)+0.09 ± 0.24
first 12 of 15all24.66 (1.21×)+0.09 ± 0.17
all 15all40.61 (2.00×)+0.07 ± 0.20

The differences between the means (at most 0.11) are smaller than each schedule's seed-to-seed s.d. (0.13 to 0.24), so three seeds do not show that one schedule is consistently best. We keep the chosen schedule because it has the highest mean gain (+0.18, against +0.07 to +0.09) and is the cheapest (1.07× FLOPs).

The gain reflects more motion and scene content, not better temporal consistency. All four schedules raise dynamic degree (+1.9 to +3.7) and scene (+1.2 to +2.9), and the chosen schedule has the largest rise in both. Motion smoothness, temporal flickering, and subject and background consistency change by less than 0.2 under every schedule. For the chosen schedule, the gain lies in the Quality score (+0.25), while the Semantic score is about unchanged.

Thanks for the authors' response. After carefully reviewing the response, I think the error-injection test and video ablations address my main concerns. Thus, I will keep my score of 3.

We sincerely thank you for the follow-up and for the continued positive assessment of our work. We are glad our response addressed your concerns.

Reviewer_ewLfOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes Visual Prefix Guidance (VPG), a training-free inference-time guidance rule for next-scale visual autoregressive models (VAR, Infinity, InfinityStar). CFG contrasts predictions with and without the condition at a fixed prefix; VPG instead fixes the condition and contrasts predictions under the generated prefix against those under a corrupted copy, where a fraction n_p of prefix sites has its code + position embedding swapped with another site at the same scale. The guided logits are (1+λ)·ℓgen − λ·ℓcorr, composed with CFG by applying CFG to each branch first. A Bayes identity motivates the direction of the contrast, with the corrupted-prefix prediction standing in for the intractable prefix marginal. VPG lowers VAR FID by 0.12–0.62 across depths (5 seeds), raises DPG-Bench on Infinity-2B by 0.63 (5 seeds), and VBench Overall on InfinityStar-8B by 0.36 (1 seed). Ablations cover corruption rules, non-prefix weak references, λ / n_p sweeps, precision/recall, and a coarse-scale schedule that keeps most of the gain at 1.22× FLOPs.

Critical strengths

1: Clear and simple idea. Reframing guidance as “which input is contrasted” (Table 1) is a clean way to position VPG relative to CFG and self-guidance methods, and the resulting logit rule is a two-line change to any existing sampler.

2: The ablations are informative. Only the full code+position replacement works; token-only, position-only, random-codebook, dropped sites, PAG/SEG-style attention perturbation, entropy-matched heating, and norm-matched random directions all fail (Tables 5, 15). The per-scale analysis (cosine 0.17–0.23 with the CFG direction, entropy change of only 0.05–0.27 nats) convincingly shows VPG is not a rescaled CFG or a temperature effect.

3: The diversity analysis (Fig. 11, Table 16) is a nice touch: VPG reaches a better FID–recall trade-off than lowering CFG scale or raising temperature, and the extra diversity enters in the first four scales.

Critical weaknesses

1: Video evidence is thin. The VBench result is a single seed, the gain (+0.36 Overall) is of the order of typical seed variance on VBench, and the specific schedule (“first 12 of 15 scales, first repeat only”) appears to have been chosen on the evaluation benchmark itself. Multi-seed results, or at least a held-out prompt split for choosing the schedule, are needed before “improves video generation” is supportable.

2: Matched-compute comparison. All headline VAR numbers (Table 2) use all-scale VPG at 2× Transformer FLOPs. Fig. 7 suggests that at matched compute, simply using the next-larger VAR with CFG is competitive (e.g. d16+VPG ≈ 1.2 TFLOPs, FID 2.74 vs. d20 CFG at ≈1.0 TFLOPs, FID 2.71). The main text should state the compute-matched picture explicitly rather than only in an appendix figure, and Table 2 should flag that it is the 2× setting.

Questions for authors

1: Why does small λ hurt FID (Table 14)? Is this precision-driven sharpening, and does it occur at other depths?

4 comments
AuthorsOpenReview ↗

Response to Reviewer ewLf (1/2): W1

Thank you for the careful and constructive feedback. W1 is answered below; W2 and Q1 follow in a second comment.

W1 (video evidence).

Thank you for this suggestion. The concern is whether the video gain holds beyond a single seed and the run on which the schedule was selected. For the multi-seed evidence, on the three seeds not used to select the schedule (10000, 20000, 30000), VPG raises VBench Overall by +0.18 ± 0.13, and the gain is positive in all four seeds (+0.22 on average, including seed 0, the selection seed). The baseline is InfinityStar's original sampler without VPG.

InfinityStar-8BQuality ↑Semantic ↑Overall ↑ (mean ± s.d. over 4 seeds)
Original sampler84.2382.5083.88 ± 0.10
+ VPG84.4782.6584.10 ± 0.22

The ± values in the table are the seed-to-seed spread of each sampler's absolute score. The paired gain, computed seed by seed, is +0.22 ± 0.14 and positive in each seed (+0.36, +0.03, +0.24, +0.26). Seed 0 was used to select the schedule, and the other three seeds were not used for any choice. On these three seeds, the gain lies in the Quality score (+0.25, mainly dynamic degree), while the seed-0 Semantic gain in Tab. 4 does not recur. Sec. 5.2 now reports the four-seed result.

For the schedule selection, we compared the four candidate schedules on the three held-out seeds (10000, 20000, 30000), and the chosen schedule has the highest mean gain. On each of the three held-out seeds, we ran all four combinations of guided scales and guided repeats (each scale is predicted in up to three repeated passes; "first" guides only the first pass). ΔOverall against the original sampler (20.31 PFLOPs per clip) on full VBench:

Guided scalesGuided repeatsTransformer PFLOPs per clip ↓Mean ± s.d. over 3 seeds ↑
first 12 of 15 (ours)first21.76 (1.07×)+0.18 ± 0.13
all 15first35.66 (1.76×)+0.09 ± 0.24
first 12 of 15all24.66 (1.21×)+0.09 ± 0.17
all 15all40.61 (2.00×)+0.07 ± 0.20

The differences between the means (at most 0.11) are smaller than each schedule's seed-to-seed s.d. (0.13 to 0.24), so three seeds do not show that one schedule is consistently best. The chosen schedule has the highest mean gain (+0.18, against +0.07 to +0.09) and is the cheapest (1.07× FLOPs, against 1.21× to 2.00×). The four schedules agree on where the gain comes from: each raises dynamic degree (+1.9 to +3.7) and scene (+1.2 to +2.9) and lowers aesthetic quality slightly (−0.14 to −0.26). Tab. 10 (bottom) lists every seed, and Tab. 13 now includes the factorial on the three held-out seeds.

AuthorsOpenReview ↗

Response to Reviewer ewLf (2/2): W2 and Q1

W2 (compute-matched comparison).

Thank you for this constructive suggestion. The question is whether VPG's extra compute would be better spent on a larger VAR model with standard CFG. With the coarse-scale schedule, VPG on d24 reaches a lower FID than standard CFG on d30 (1.76 vs. 1.97) with 37% fewer Transformer FLOPs (TFLOPs). Revised Tab. 2 adds the results of the coarse-scale schedule and marks the all-scale setting as 2.00× TFLOPs. Five-seed means (seeds 42–46), 256², FID @ Transformer TFLOPs per image:

ModelCFGVPG, coarse (1.22×)VPG, all scales (2.00×)
d163.37 @ 0.602.74 @ 0.732.74 @ 1.20
d202.71 @ 1.142.25 @ 1.392.26 @ 2.29
d242.16 @ 1.951.76 @ 2.381.84 @ 3.90
d301.97 @ 3.761.83 @ 4.601.84 @ 7.53

As you note, all-scale d16 + VPG (1.20 TFLOPs, 2.74) is only on par with d20 + CFG (1.14 TFLOPs, 2.71). With the coarse-scale schedule, VPG lies below the FID–compute curve of standard CFG on d16–d24:

  • d16 + VPG nearly matches d20 with standard CFG at 64% of its FLOPs (2.74 vs 2.71).
  • d24 + VPG beats d30 with standard CFG at 63% of its FLOPs (1.76 vs 1.97).

This comparison is against standard CFG at its default sampling temperature; with a tuned temperature, CFG on d20 reaches 2.47. SSG, which adds no compute, is a stronger alternative: VPG is ahead of it on d16–d24 (e.g. 1.76 vs. 1.92 on d24) and behind it on d30 (1.83 vs. 1.70). Sec. 5.2 now states this comparison, and Fig. 7 (App. C.2) plots it for all 256² depths.

Q1 (why small λ hurts FID).

Thank you for this observation. The question is whether the FID rise at small λ comes from precision-driven sharpening, and whether it occurs at other depths. The precision–recall results are consistent with your sharpening explanation on the smaller models, and the effect weakens with depth. Where FID rises, precision is up and recall is down. On d16, standard CFG has precision 0.849 and recall 0.500. Along λ the following two effects appear in turn (Tab. 14):

  • Sharpening appears first. For λ ∈ [0.4, 1.2], precision rises to 0.862–0.867, recall falls to 0.48–0.49, and FID rises slightly (3.45–3.54 vs 3.40).
  • The diversity gain appears at larger λ. Recall exceeds the CFG value from λ = 2.0 and reaches 0.553 at λ = 3.2, where FID is 2.72.

The dip occurs where precision rises and recall falls. By λ = 1.6 FID (3.29) is below CFG's (3.40), although recall is still slightly lower (0.492), and from λ = 2.0 recall exceeds CFG's. At other depths, it is clear on d20, within seed noise on d24 and absent on d30 (seed 42, FID / precision / recall):

Depth (selected λ)CFGλ = 0.4λ = 0.8
d20 (2.3)2.71 / 0.837 / 0.5542.82 / 0.848 / 0.5442.83 / 0.848 / 0.540
d24 (1.8)2.20 / 0.826 / 0.5812.22 / 0.835 / 0.5822.16 / 0.831 / 0.582
d30 (1.3)1.96 / 0.817 / 0.5931.90 / 0.824 / 0.5961.81 / 0.818 / 0.600

On d30, λ = 0.4 does not lower recall and already improves FID, so the dip weakens as the model grows. The revised Tab. 14 includes these rows.

Thank the authors for the thorough response. My concerns are addressed.

I also appreciate the candid answers to the other reviewers, in particular the controlled-error test and the narrowed exposure-bias claim. The remaining limitations (diminishing gains on larger models, SSG being stronger on d30, no GenEval gain) should be clearly stated in the paper. I keep my positive score of 3.

Thank you very much for the follow-up and for maintaining your positive score! We are glad that our response addressed your concerns, and we sincerely appreciate your careful reading of our discussions with the other reviewers. We will clearly state the remaining limitations in the revised paper.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Revisions

All changes are in blue in the revised manuscript.

  • Tab. 2 and App. C.3: SSG and IGG re-run in our pipeline with their own tuning grids; CFG with a tuned sampling temperature; VPG with the coarse-scale schedule; a FLOPs row (NyFA, ewLf).
  • Sec. 2: IGG is discussed (NyFA).
  • Sec. 5.1–5.2: compute-matched comparison against standard CFG; reference to the tuned baselines; four-seed video result (ewLf, NyFA).
  • Abstract, Introduction, Limitations: exposure bias stated as the drift of generated prefixes; the division between reducing drift and repairing committed errors (NyFA, JCyB).
  • App. D.5: controlled-injection test, prefix-drift measurement with three control samplers, cross-depth table, scale-restricted ablation (NyFA, JCyB).
  • Tab. 10 (bottom): VBench Overall per seed (ewLf, NyFA).
  • Tab. 13: an added schedule row, the four schedule combinations on the three held-out seeds, and the Transformer FLOPs of every schedule (ewLf, JCyB, NyFA).
  • Tab. 14 (bottom): small-λ behavior on d20–d30 (ewLf).
  • App. D.6: GenEval; per-prompt diversity (NyFA).
  • Tab. 7: the video seeds.
AuthorsOpenReview ↗

Summary of the Discussion

Dear AC and Reviewers,

Thank you for your time and careful reading. After the discussion, all three reviewers recommend weak acceptance. Reviewers ewLf and JCyB keep their score of 3 and write that their main concerns are addressed, and Reviewer NyFA raised their score from 2 to 3.

The reviewers highlight that VPG guides on the generated prefix instead of the condition used by CFG. They describe it as simple and clear (ewLf), training-free and applicable to released checkpoints (JCyB, NyFA), with gains across model sizes and tasks (JCyB) and robust five-seed results (NyFA). They also value the analyses: VPG is neither a rescaled CFG nor a temperature effect (ewLf), and it reaches a better FID–recall trade-off than a lower CFG scale or a higher temperature (ewLf, NyFA). Below we summarize each concern and how the reviewers responded. The posted summary of revisions lists every change to the revised paper.

1. Comparison with existing guidance (NyFA W1, Q1). The concern was that VPG's lead over SSG might reflect SSG numbers quoted from its paper rather than the method. In a same-pipeline comparison, VPG still gives the largest FID gain on four of the five VAR models, and SSG is better on d30. We re-ran SSG and IGG [1] with our checkpoints, seeds and evaluator, each tuned on its own grid. Relative to the same checkpoint sampled with CFG alone, VPG with the coarse-scale schedule (1.22× FLOPs) lowers FID by 0.40–0.63 on d16–d24, versus 0.24–0.36 for SSG, and leads in each of the five seeds. Reviewer NyFA: "It is fully resolved, and I appreciate the transparency in reporting the d30 result, where SSG outperforms VPG."

2. Exposure bias and how VPG improves FID (JCyB W1, NyFA W2, Q4). The reviewers asked whether VPG reduces exposure bias or amplifies an error already in the prefix, and why it lowers FID. VPG addresses one part of exposure bias, the drift of the prefixes it generates, and leaves the propagation of an error already in the prefix to training-time remedies. Its FID gain goes with its coarse prefixes covering more of those of real images. In the controlled-error test the reviewers proposed, VPG follows a planted error more closely than CFG. Against teacher-forced prefixes, VPG's coarse prefixes drift less on d16–d24, and we split this measure into k-NN precision and recall. VPG's gain in prefix recall (+0.037, +0.027, +0.016, +0.008 on d16–d30) is ordered like its all-scale FID gain (0.62, 0.45, 0.32, 0.12), and on d16 a higher temperature, a lower CFG scale and SSG do not raise prefix recall. Reviewer NyFA writes that the revised statement on errors already in the prefix "addresses this part of my concern" and that "The recall/precision explanation seems plausible to me in comprehending the role of VPG in improving FID."

3. Video evidence (ewLf W1, JCyB W2, NyFA Q2). The reviewers asked whether the InfinityStar gain holds beyond the run used to choose the schedule, which part of the schedule matters, and whether it reflects temporal behavior. The gain holds on three seeds not used for any choice, and it comes from more motion and scene content rather than from better temporal consistency. Relative to InfinityStar's original sampling procedure without VPG, VBench Overall rises by +0.18 ± 0.13 on these seeds and is positive in all four seeds (+0.22 on average). On the held-out seeds, all four combinations of guided scales and guided repeats differ by less than the seed-to-seed spread, and the chosen schedule has the highest mean gain at the lowest cost (1.07× FLOPs). Every schedule raises dynamic degree and scene, while the temporal-consistency metrics change by less than 0.2. Reviewer JCyB: "the error-injection test and video ablations address my main concerns."

4. Compute (ewLf W2). Reviewer ewLf asked whether VPG's extra compute would be better spent on a larger model with standard CFG. With the coarse-scale schedule, a smaller model with VPG nearly matches or beats the next larger model with standard CFG at lower compute. On VAR, d16 + VPG nearly matches d20 at 64% of its FLOPs (2.74 versus 2.71), and d24 + VPG beats d30 at 63% of its FLOPs (1.76 versus 1.97). It is now in Sec. 5.2. Reviewer ewLf: "My concerns are addressed."

Commitments for the final version. The limitations ewLf lists (smaller gains on larger 256² models, SSG ahead on d30, no GenEval gain) are reported in the revised paper, and we will gather them in the Limitations. Following NyFA (Q3), we will discuss scalability in Sec. 5.2, including that d36 at 512² still gains 0.48 FID and leads SSG. We will also add the prefix-coverage result to L83–86, the per-map precision and recall to App. D.5, and all other points from the discussion.

Thank you again for the detailed feedback, which has helped us improve our work.

References

[1] Nguyen et al. Rethinking Visual Autoregressive Sampling with Information-Grounding Guidance. ICML 2026.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.