acceptodds
Withdrawn from ICLR 2027

NextGAN: Towards Simple and Scalable GAN Training for Image and Video Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

25 views

Abstract

Generative adversarial networks (GANs) enable one-pass sampling, but their critics are trained to separate real and generated samples rather than to provide a useful correction field. We present NextGAN, which directly supervises the critic's input gradient. Independent real–generated pairs are interpolated in frozen feature spaces, and the gradient is regressed onto their displacement. Random-pair regression aggregates into a distribution-dependent field that vanishes when the conditional feature distributions match; for a fixed generator, its population objective has an exact adversarial dual with a gradient-energy penalty. Without auxiliary losses, online teacher scores, or trajectories, the same objective trains pixel- and latent-space image generators and Wan2.1 video generators up to 14B parameters, remaining stable across one- and few-step sampling budgets. On ImageNet 256*256, NextGAN achieves FID=2.87 with pixel-space JiT-B and FID=1.54 with latent-space DiT-XL/1 at one evaluation, while the adversarial form reaches FID=1.36. On text-to-video generation, the 1.3B model achieves the highest VBench total score among the compared 1.3B methods at one- and two-step budgets.

Primary area:
generative models
Keywords:
GAN, Diffusion,video generation, a few step distillation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_k1rxOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

NextGAN trains a scalar critic's feature-input gradient on displacements between independently paired reference and generated samples. The generator then ascends the potential at generated endpoints. The paper characterizes the population field and conditional feature-distribution matching, and evaluates regression and adversarial implementations of the critic objective. Experiments cover controlled mode coverage, one-step image generation, and adaptation of pretrained video generators using offline reference samples without online teacher score or trajectory queries.

Critical strengths
  1. Equations (2)–(6), Proposition 1, and Appendix C clearly explain why independent-pair supervision need not pull each generated sample directly toward an arbitrary reference sample. The paper distinguishes feature-distribution matching from full image matching and states the limits of its particle analysis and moving-corridor interpretation.
  2. The results show that the approach works across image and video settings. Latent one-step generation reaches 1.54 FID with regression and 1.36 with the adversarial form, and one 1.3B video model supports multiple sampling budgets. The objective and representer comparisons provide evidence beyond the main benchmark results.
Critical weaknesses
  1. I am not yet convinced that the paper establishes a substantial advance over its closest construction. TBSM v1 §2.5, Eq. (13), and Appendix E.5.3 already contain the conservative displacement construction and projection interpretation (TBSM). Its illustrative displacement algorithm queries the corridor; its deployed unit-bearing algorithm differs. NextGAN acknowledges the overlap (pp.2–3), but Figure 7/Appendix B does not describe the TBSM comparator well enough to identify which changes explain the improvement. Please separate inherited results from the added analysis and practical contribution, and specify the comparator's critic, target, estimator, and update. If endpoint querying is the main methodological advance, holding the critic construction fixed and changing only the query would be informative. Proposition 1 characterizes the field under the corridor measure; Appendix H.3 and Appendix I do not establish descent for the practical endpoint update. A matched comparison or restricted analysis could establish the value of this change; a well-documented system contribution is another route. I am not asking for general GAN convergence or for regression to outperform its equivalent adversarial estimator.
  2. I could not reconstruct the video training and evaluation setup from Sections 3.6/4.2 and Appendix A. They give backbone, LoRA, batch-size, feature-layer, and pooling details, but omit parts of the optimization schedule, rollout-length and noise-level sampling, potential-head configurations, and VBench version and prompt/seed protocol. Since video adaptation is a principal contribution, please provide the configurations for each model size and sampling budget. Please also specify normalization, penalty coefficients, and training budgets for the objective comparisons. As the paper notes on p.14, rescaling the objective does not ensure identical optimization under spectral normalization.
  3. The stability and ranking conclusions need more evidence. Table 2's 1.3B total-score margins are small: 80.80 versus 80.68 at one step and 83.13 versus 82.78 at two steps, without uncertainty estimates. The toy illustration uses seed 0 (Appendix E). The authors disclose per-budget best-completed-run selection at 14B and acknowledge its weaker total scores, but do not give the number of runs considered. Paired evaluation intervals, selection counts, and repeated lower-cost experiments would help distinguish stable gains from variation across runs. The stability claim could instead be limited to the observed runs; these questions do not require repeated full 14B training.
Questions for authors
  1. What are the new methodological, theoretical, and implementation contributions relative to the closest construction? Which TBSM variant does Figure 7 use, and what differs in its critic, target, estimator, regularization, and generator query? Can existing controlled evidence resolve these differences, or is a query-only comparison needed?
  2. Can you provide the complete video optimization, rollout/noise, potential-head, and VBench configurations, along with the settings used in the objective comparisons?
  3. How many runs/configurations were attempted and considered for each selected 14B result? Do paired evaluation intervals or repeated lower-cost experiments support the small 1.3B margins and the stability claim?
  4. Figure 7b shows gains from combining representers, but each also adds a potential head. What evidence separates complementary feature information from added head capacity? Section 3.6 notes that the video representers tended to favor static appearance over motion. Beyond this observation, what guidance do the existing comparisons offer for choosing representers, and what errors do the chosen features still miss?
  5. For the offline Wan2.1-14B reference data, what are the costs of data creation and storage, and of feature extraction during adaptation? Please distinguish costs incurred in this study from upstream costs of creating the public dataset. Do the available observations suggest which reference-data properties matter most? An accounting or discussion would suffice; a new real-video study is not needed.
6 comments
AuthorsOpenReview ↗

K1 Contributions relative to TBSM

Thank you for the precise distinction between TBSM variants and for identifying the missing implementation and evaluation details. We address each point below.

Reviewer comment W1

I am not yet convinced that the paper establishes a substantial advance over its closest construction. TBSM v1 §2.5, Eq. (13), and Appendix E.5.3 already contain the conservative displacement construction and projection interpretation (TBSM). Its illustrative displacement algorithm queries the corridor; its deployed unit-bearing algorithm differs. NextGAN acknowledges the overlap (pp.2–3), but Figure 7/Appendix B does not describe the TBSM comparator well enough to identify which changes explain the improvement. Please separate inherited results from the added analysis and practical contribution, and specify the comparator's critic, target, estimator, and update. If endpoint querying is the main methodological advance, holding the critic construction fixed and changing only the query would be informative. Proposition 1 characterizes the field under the corridor measure; Appendix H.3 and Appendix I do not establish descent for the practical endpoint update. A matched comparison or restricted analysis could establish the value of this change; a well-documented system contribution is another route. I am not asking for general GAN convergence or for regression to outperform its equivalent adversarial estimator.

Reviewer comment Q1

What are the new methodological, theoretical, and implementation contributions relative to the closest construction? Which TBSM variant does Figure 7 use, and what differs in its critic, target, estimator, regularization, and generator query? Can existing controlled evidence resolve these differences, or is a query-only comparison needed?

Response. Please see Common Response C2 and the revised Appendix B.1 (Algorithm 3, Figure 8). We credit TBSM for the conservative displacement construction, adversarial identity, and projection interpretation. TBSM does not report any experiments demonstrating the effectiveness of the GAN-corner displacement formulations considered here. Its paper identifies Algorithm 3 as an expository displacement analogue and reports experiments using unit-bearing supervision (TBSM v1, Secs. 2.4–2.5).

Our methodological development is endpoint potential training in frozen feature spaces; the implementation contribution is its image/video adaptation system, and the additional analysis characterizes conditional feature distributions under explicit assumptions. Neither Proposition 1 nor the appendix analyses establish descent of the practical endpoint update.

The completed comparison includes both TBSM GAN-corner displacement variants, with direct vv and conservative ∇u\nabla u, using the shared-field, generator-first, one-to-one update rule. Their 50K FIDs are 10.794 and 350.367, compared with NextGAN's 1.335; all three curves contain 20 evaluated checkpoints. NextGAN retains its established two field updates per generator update. These results provide a documented comparison of the configured systems and leave the isolated effect of query location unresolved. We do not reinterpret Figure 7 or these new curves as query-only evidence, or as results for TBSM's deployed unit-bearing algorithm.

AuthorsOpenReview ↗

K2 Complete optimization and evaluation configurations

Reviewer comment W2

I could not reconstruct the video training and evaluation setup from Sections 3.6/4.2 and Appendix A. They give backbone, LoRA, batch-size, feature-layer, and pooling details, but omit parts of the optimization schedule, rollout-length and noise-level sampling, potential-head configurations, and VBench version and prompt/seed protocol. Since video adaptation is a principal contribution, please provide the configurations for each model size and sampling budget. Please also specify normalization, penalty coefficients, and training budgets for the objective comparisons. As the paper notes on p.14, rescaling the objective does not ensure identical optimization under spectral normalization.

Reviewer comment Q2

Can you provide the complete video optimization, rollout/noise, potential-head, and VBench configurations, along with the settings used in the objective comparisons?

Response. We have added the video settings in Appendix A.1 (Table 3). Both model sizes use 8,000 generator updates, AdamW with G/D learning rates 2 × 10⁻⁶/10⁻⁵, a 100-update warmup followed by constant rates, and D:G = 5:1. Global batch sizes are 32 and 16 for 1.3B and 14B. Potential heads have width 768, four blocks, and 12 attention heads. Rollout lengths cycle through 1–4 steps, with log-normal intermediate-noise sampling; only the final generator call is differentiated.

The same weights support 1/2/4 NFE with CFG = 1 and σ_max = 1600. VBench uses the original 16-dimension protocol, 944 fixed augmented prompts × five samples, 832 × 480 × 81 resolution, and 16 fps. We also specify the angle schedules, feature normalization, token/head reductions, spectral normalization, and the absence of video EMA. The ImageNet objective-comparison settings are listed in Response B3. We agree that an algebraic rescaling does not ensure identical optimization under spectral normalization.

AuthorsOpenReview ↗

K3 Uncertainty and run selection

Reviewer comment W3

The stability and ranking conclusions need more evidence. Table 2's 1.3B total-score margins are small: 80.80 versus 80.68 at one step and 83.13 versus 82.78 at two steps, without uncertainty estimates. The toy illustration uses seed 0 (Appendix E). The authors disclose per-budget best-completed-run selection at 14B and acknowledge its weaker total scores, but do not give the number of runs considered. Paired evaluation intervals, selection counts, and repeated lower-cost experiments would help distinguish stable gains from variation across runs. The stability claim could instead be limited to the observed runs; these questions do not require repeated full 14B training.

Reviewer comment Q3

How many runs/configurations were attempted and considered for each selected 14B result? Do paired evaluation intervals or repeated lower-cost experiments support the small 1.3B margins and the stability claim?

Response. We additionally analyze the new trained NextGAN 1.3B checkpoint (8K updates, EMA), distinct from the original Table 2 runs. We use 20,000 paired bootstrap resamples of the same 944 prompts, retaining five videos per prompt and recomputing the 16 VBench dimensions. Brackets give marginal 95% confidence intervals (CIs).

NFEMethodTotal [95% CI]Quality [95% CI]Semantic [95% CI]
1NextGAN81.579 [80.874, 82.284]82.508 [81.770, 83.240]77.863 [76.029, 79.679]
1rCM80.678 [80.106, 81.246]81.762 [81.206, 82.299]76.343 [74.590, 78.098]
1AnyFlow78.117 [77.451, 78.790]78.739 [78.099, 79.386]75.631 [73.626, 77.618]
1Wan2.1 + DMD2 (bidirectional)78.612 [77.925, 79.311]80.301 [79.613, 80.992]71.855 [69.889, 73.794]
2NextGAN82.769 [82.112, 83.427]83.571 [82.887, 84.243]79.564 [77.812, 81.297]
2rCM81.884 [81.336, 82.426]82.607 [82.074, 83.118]78.993 [77.320, 80.649]
2AnyFlow82.656 [81.995, 83.321]83.083 [82.385, 83.775]80.948 [79.229, 82.665]
2Wan2.1 + DMD2 (bidirectional)82.778 [82.138, 83.405]84.083 [83.421, 84.712]77.560 [75.749, 79.387]

Paired score differences (new NextGAN checkpoint − baseline):

NFEBaselineTotal difference [95% CI]Quality difference [95% CI]Semantic difference [95% CI]
1rCM+0.901 [0.376, 1.432]+0.746 [0.156, 1.328]+1.521 [0.350, 2.692]
1AnyFlow+3.462 [2.846, 4.073]+3.770 [3.063, 4.463]+2.232 [0.958, 3.509]
1Wan2.1 + DMD2 (bidirectional)+2.968 [2.335, 3.591]+2.208 [1.510, 2.905]+6.009 [4.615, 7.409]
2rCM+0.885 [0.343, 1.445]+0.963 [0.334, 1.617]+0.571 [−0.502, 1.638]
2AnyFlow+0.113 [−0.403, 0.656]+0.488 [−0.109, 1.126]−1.384 [−2.438, −0.369]
2Wan2.1 + DMD2 (bidirectional)−0.009 [−0.582, 0.583]−0.512 [−1.164, 0.165]+2.004 [0.844, 3.180]

At one step, total, quality, and semantic scores exceed all three baselines with CIs excluding zero. At two steps, total and quality exceed rCM; semantic is lower than AnyFlow and higher than DMD2. Other two-step differences remain unresolved. These intervals are conditional on the archived evaluations and do not cover training-run variability or model selection; no multiplicity correction is applied. Full historical baseline runtime and evaluator-cache identity remain unverified.

Reviewer_BTmYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

NextGAN trains a scalar GAN critic by regressing its input gradient onto displacements between independently sampled reference and generated examples, evaluated along interpolation paths in frozen feature spaces. The generator is then updated by ascending the critic potential at generated endpoints. The paper analyzes the resulting distribution-dependent field, establishes a population-level equivalence to an adversarial objective with a gradient-energy penalty, and gives conditions for detecting conditional feature-distribution mismatch. It acknowledges that the critic objective coincides with TBSM's conservative GAN-corner configuration, while distinguishing its endpoint generator update. Validation includes a synthetic mode-coverage study, objective and feature-space ablations, one-step ImageNet generation, and few-step adaptation of pretrained Wan2.1 video models. Reported ImageNet FIDs are 1.54 for regression and 1.36 for the adversarial form. A single 1.3B video model achieves the highest reported VBench totals among the compared 1.3B methods at one and two steps, whereas the 14B results trail the strongest baselines. Training avoids online teacher queries but uses teacher-generated reference videos.

Critical strengths
  1. The paper provides a useful and clearly qualified interpretation of critic-gradient supervision. It explains why independent real–generated pairings produce a distribution-dependent correction field rather than fixed samplewise targets, and characterizes when the induced discrepancy detects conditional feature-distribution mismatch. The distinction between this population-level analysis and practical neural-network optimization is stated explicitly. This explanatory value is a substantive strength, even though the underlying objective overlaps with prior work.

  2. The experiments demonstrate practical utility across pretrained image and video generators. The framework produces competitive one-step ImageNet results in both pixel and latent spaces, and the same 1.3B video model supports one-, two-, and four-step sampling. The feature-space ablation also supports combining complementary frozen representations. These results establish a credible application of the formulation beyond synthetic examples and constitute the strongest empirical argument for acceptance.

Critical weaknesses
  1. A sufficiently distinct methodological contribution over TBSM has not been established. Section 2 explicitly acknowledges that TBSM's conservative GAN-corner configuration yields the same critic objective. The endpoint generator update may be consequential, but the descriptions of the TBSM baseline in Figures 4 and 7 do not fully specify the generator loss, gradient-detachment rules, and matched training settings needed to attribute the performance gap to that change. The paper should also distinguish more precisely which analytical results extend TBSM rather than reinterpret the shared objective. This unresolved attribution of novelty is the principal reason for my weak-rejection recommendation.

  2. The empirical evidence demonstrates effectiveness more convincingly than a distinct or consistently superior training approach. The strongest ImageNet result and the main objective ablation use the equivalent adversarial formulation, so they do not by themselves establish a specific practical benefit of direct displacement regression. The reported comparison between the two implementations also leaves their relative stability and computational cost unresolved. Meanwhile, the 1.3B VBench leads at one and two steps are only 0.12 and 0.35 points without uncertainty estimates, and the 14B totals trail rCM and AnyFlow at every tested budget despite selecting the best run separately for each budget. These limitations weaken the empirical case for a consequential advance over the shared prior formulation.

Questions for authors
  1. Given their equivalent critic objectives, what exact generator-loss, query-location, and gradient-detachment differences distinguish NextGAN from the TBSM GAN-corner baseline in Figures 4 and 7, and were architecture, normalization, regularization, and optimization otherwise matched?

  2. Were the regression and adversarial ImageNet results (FID 1.54 versus 1.36) obtained with matched initialization, training budgets, feature normalization, effective gradient-penalty weights, and checkpoint selection, and do existing experiments demonstrate any consistent stability or computational-cost advantage for either implementation?

3 comments
AuthorsOpenReview ↗

B1 TBSM attribution and exact generator update

Thank you for the detailed assessment and for recognizing the distributional interpretation and the image/video results. We have clarified the attribution to TBSM and separated the evidence for the objective from claims about optimization or benchmark rankings.

Reviewer comment W1

A sufficiently distinct methodological contribution over TBSM has not been established. Section 2 explicitly acknowledges that TBSM's conservative GAN-corner configuration yields the same critic objective. The endpoint generator update may be consequential, but the descriptions of the TBSM baseline in Figures 4 and 7 do not fully specify the generator loss, gradient-detachment rules, and matched training settings needed to attribute the performance gap to that change. The paper should also distinguish more precisely which analytical results extend TBSM rather than reinterpret the shared objective. This unresolved attribution of novelty is the principal reason for my weak-rejection recommendation.

Reviewer comment Q1

Given their equivalent critic objectives, what exact generator-loss, query-location, and gradient-detachment differences distinguish NextGAN from the TBSM GAN-corner baseline in Figures 4 and 7, and were architecture, normalization, regularization, and optimization otherwise matched?

Response. TBSM does not report any experiments demonstrating the effectiveness of the GAN-corner displacement formulations considered here. Its paper identifies Algorithm 3 as an expository displacement analogue and reports experiments using unit-bearing supervision (TBSM v1, Secs. 2.4–2.5).

We have completed the three-way latent-DiT comparison and rewritten Appendix B.1 to place the exact algorithms, all measured results, and their analysis together (Algorithm 3, Figure 8, pp. 18–19). Common Response C2 specifies the field parameterizations, losses, query locations, and stop-gradient rules.

At 50K generator updates, FID (ADM) is 1.335 for NextGAN, 10.794 for TBSM (vv), and 350.367 for TBSM (∇u\nabla u), using the same 50K-image evaluation protocol. The direct-vector variant is slightly better at 5K (4.004 versus 4.156), so our analysis also discusses its subsequent deterioration. We explicitly credit TBSM's shared conservative construction and projection interpretation. Our added conditional feature-distribution analysis states its assumptions and does not imply descent of the practical endpoint update.

AuthorsOpenReview ↗

B2 Empirical benefit and strength of the evidence

Reviewer comment W2

The empirical evidence demonstrates effectiveness more convincingly than a distinct or consistently superior training approach. The strongest ImageNet result and the main objective ablation use the equivalent adversarial formulation, so they do not by themselves establish a specific practical benefit of direct displacement regression. The reported comparison between the two implementations also leaves their relative stability and computational cost unresolved. Meanwhile, the 1.3B VBench leads at one and two steps are only 0.12 and 0.35 points without uncertainty estimates, and the 14B totals trail rCM and AnyFlow at every tested budget despite selecting the best run separately for each budget. These limitations weaken the empirical case for a consequential advance over the shared prior formulation.

Response. The regression formulation offers concrete implementation benefits: each critic update requires only one critic forward evaluation, compared with three in the explicit adversarial form, reducing critic computation. Our implementation also uses a single regression loss with fixed weighting, simplifying loss-weight selection and reducing the tuning burden. Moreover, our released regression implementation achieves FID 1.27 on ImageNet, demonstrating that strong results are achievable with regression itself. Code and results. For video, the revised 14B evaluation uses the same iteration-8,000 checkpoint across all budgets, with a one-step total of 79.77 versus rCM’s 78.90. AnyFlow retains higher totals. We report the small 1.3B leads as point estimates without claiming statistical significance. The practical benefit is competitive few-step generation up to 14B without online teacher queries, alongside a simpler regression implementation.

AuthorsOpenReview ↗

B3 Regression and adversarial implementation settings

Reviewer comment Q2

Were the regression and adversarial ImageNet results (FID 1.54 versus 1.36) obtained with matched initialization, training budgets, feature normalization, effective gradient-penalty weights, and checkpoint selection, and do existing experiments demonstrate any consistent stability or computational-cost advantage for either implementation?

Response. Yes—the FID 1.54 and 1.36 results were obtained with matched initialization, training budgets, feature normalization, effective gradient-energy weights, and checkpoint-selection policies. The shared settings are:

SettingShared configuration
InitializationSame UCGM DiT-XL/1 checkpoint, VA-VAE decoder, and critic initialization
Training budget100K generator updates; two critic updates per generator update
Global batch size256
OptimizerAdamW, β₁ = 0.9, β₂ = 0.999
Learning ratesGenerator 10⁻⁵; critic 5 × 10⁻⁵
Generator EMA0.999
Feature normalizationEMA centering and standardization, then per-token normalization to norm √d
Evaluation and selection50K generated images, EMA weights, common evaluator and reference statistics, same checkpoint-selection policy

The regression implementation directly minimizes one half of the expected squared error between the critic gradient and the displacement, without a separately tuned λ. Its expansion gives a unit-weight endpoint potential gap and a gradient-energy coefficient of 1/2, matching the adversarial form. Population equivalence does not imply identical finite-minibatch trajectories, particularly with a parameterized, spectrally normalized critic.

Regression requires one potential evaluation per critic update, at the interpolation point. The explicit adversarial implementation requires three, including the real and generated endpoints. Regression thus avoids two endpoint evaluations and their associated parameter-backpropagation computations. This is a lower operation count for the critic update.

Reviewer_s7PuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes NextGAN, which fine-tunes pretrained generators using a GAN inspired objective for image and video generation in one or a few steps. The main idea is to train the critic’s input gradient to match the difference between real and generated features, giving the generator a more useful direction for improving its outputs. Feature extractors and critics are used during training and discarded at inference. Experiments on ImageNet and Wan2.1 show competitive results on image and video generation.

Critical strengths
  1. Improving generation quality while reducing sampling steps is a relevant problem for efficient generation, and the paper proposes an approach applicable to image and video generation.

  2. The paper provides ablations on the training objective and feature extractors, which help analyze the proposed method.

Critical weaknesses

It would be helpful for readers if the following points were clarified or discussed:


  1. The evaluation demonstrates adaptation of pre-trained models. It would be interesting to know how much performance depends on the pretrained initialization? Maybe one small experiment of training from scratch could help the reader understand the efficacy of the new loss objective.

  2. Table 1 mentions that the reported results may use different evaluators. It would be helpful to clarify which evaluation settings differ and whether this affects the comparison ie are the numbers directly comparable ? Evaluating the closest baselines using the same setup would help readers understand the significance of the FID differences.

  3. In Table 2, the 14B model has lower total VBench scores than the 1.3B model, and increasing NextGAN’s sampling steps from two to four does not improve the scores, any discussion around this would be helpful for the readers. AnyFlow performs better at four steps and across the evaluated 14B settings. Could the authors discuss these results with the focus how the benefits of the proposed approach hold up with model scaling.

  4. It would be helpful to clarify the practical benefit over existing approaches. For videos, AnyFlow already supports few-step generation. Comparisons of training cost and inference time, alongside quality under matched settings, could help readers understand the practical benefits for the proposed approach.

  5. It would be helpful if the paper provides larger size image samples, as the images in Figure 6 are a little small to see the differences clearly. Maybe some larger sized examples in the appendix could help readers better compare the quality and diversity of the generated results.
Questions for authors

Please refer to Weakness section for questions.

5 comments
AuthorsOpenReview ↗

S1 Dependence on pretrained initialization

Thank you for recognizing the relevance of efficient image and video generation and the value of the ablations. We address your comments below.

Reviewer comment W1

The evaluation demonstrates adaptation of pre-trained models. It would be interesting to know how much performance depends on the pretrained initialization? Maybe one small experiment of training from scratch could help the reader understand the efficacy of the new loss objective.

Response. Thank you for this suggestion. We tried a randomly initialized generator on ImageNet-64×64, retaining the frozen pretrained VAE and feature encoders, but did not obtain satisfactory results under our current training configuration. Since this setup was developed for pretrained adaptation, training from scratch may require further optimization.

AuthorsOpenReview ↗

S2 ImageNet evaluation and baseline provenance

Reviewer comment W2

Table 1 mentions that the reported results may use different evaluators. It would be helpful to clarify which evaluation settings differ and whether this affects the comparison ie are the numbers directly comparable ? Evaluating the closest baselines using the same setup would help readers understand the significance of the FID differences.

Response. Thank you for pointing this out. The note refers to the JiT and ADM evaluation settings. We apologize for the unclear wording and have clarified this in the revised manuscript.

AuthorsOpenReview ↗

S3 Updated 14B results and sampling steps

Reviewer comment W3

In Table 2, the 14B model has lower total VBench scores than the 1.3B model, and increasing NextGAN’s sampling steps from two to four does not improve the scores, any discussion around this would be helpful for the readers. AnyFlow performs better at four steps and across the evaluated 14B settings. Could the authors discuss these results with the focus how the benefits of the proposed approach hold up with model scaling.

Response. Please see C1 for the updated 14B results. AnyFlow retains higher total scores. Increasing sampling from 2 to 4 NFE lowers our total from 82.73 to 82.02, although the matched examples in Appendix L show visual improvements despite lower imaging-quality scores.

Comments on the paper

4 comments
~Shanchuan_Lin1OpenReview ↗

Comparison baselines

Table 1 reports class-conditional ImageNet-256 results but omits comparisons with recent state-of-the-art GAN-based methods. In particular, the following two papers are highly relevant, as they employ adversarial training and evaluate their methods under the same experimental setting:

AuthorsOpenReview ↗

C1 Updated 14B VBench results

Thank you for the comments on the 14B results. We resumed training from the checkpoint at iteration 6,000 and continued for 2,000 additional generator updates, reaching iteration 8,000. We have updated Table 2 to evaluate this same iteration-8,000 checkpoint at all three sampling budgets (1, 2, and 4 NFE). The updated results are:

NFENextGAN qualityNextGAN semanticNextGAN totalrCM totalAnyFlow total
179.7879.7179.7778.9081.23
282.8282.4182.7383.0983.89
482.2481.1482.0283.1784.10

The updated one-step total exceeds rCM by 0.87 points. AnyFlow retains the highest total at all three budgets, and rCM scores higher at two and four steps. Thus, these results demonstrate few-step adaptation at 14B without online teacher-score or trajectory queries, rather than uniform superiority over the compared methods. Evaluation uses the original 16-dimension VBench protocol, 944 fixed augmented prompts × five samples, 832 × 480 × 81 videos, CFG = 1, and matched prompts and seeds across NextGAN sampling budgets.

AuthorsOpenReview ↗

C2 Relationship to TBSM and new comparative results

TBSM does not report any experiments demonstrating the effectiveness of the GAN-corner displacement formulations considered here. Section 2.4 explicitly describes Algorithm 3 as an “expository displacement analogue only”; its reported experiments use unit-bearing supervision (TBSM v1, Secs. 2.4–2.5).

We credit TBSM for the conservative displacement objective, its adversarial identity, and projection interpretation. Our contribution develops endpoint potential training in frozen feature spaces for image/video adaptation and conditional feature-distribution analysis under explicit assumptions. Appendix B.1 presents the algorithms and complete three-way results together (Algorithm 3, Figure 8, pp. 18–19).

Losses and generator gradients. Let rr and pp be reference and generated features from frozen differentiable representers, and let sg⁡\operatorname{sg} denote stop-gradient. Define

\Delta=\operatorname{sg}[r-p],\qquad \alpha\sim\mathcal U(0,1).$$ For TBSM, write $w_\phi=v_\phi$ for the direct-vector variant and $w_\phi=\nabla u_\phi$ for the conservative variant. The losses are $$\mathcal{L}_D^{\mathrm{NextGAN}} = \frac{1}{2}\mathbb{E}[\\|\nabla_q u_\phi(q,c)-\Delta\\|_2^2].$$ $$\mathcal{L}_G^{\mathrm{NextGAN}} = -\mathbb{E}[u_\phi(p,c)].$$ $$\mathcal{L}_D^{\mathrm{TBSM}} = \mathbb{E}[\\|w_\phi(q,c)-\Delta\\|_2^2].$$ $$\mathcal{L}_G^{\mathrm{TBSM}} = \mathbb{E}[\\|p-\operatorname{sg}[p+w_\phi(q,c)]\\|_2^2].$$ With $J_\theta=\partial p/\partial\theta$, the per-sample generator gradients at fixed field parameters are $$g_{\mathrm{NextGAN}}=-J_\theta^\top\nabla_xu_\phi(p,c),\qquad g_{\mathrm{TBSM}}=-2J_\theta^\top w_\phi(q,c).$$ NextGAN queries the generated endpoint; TBSM queries the corridor. The factor two follows TBSM's unhalved squared loss. For the conservative variant, setting $q=p$ and halving its generator loss gives exactly NextGAN's generator gradient at common potential weights. **New experimental results.** On ImageNet-256 with latent DiT-XL/1, the methods share pretrained generator initialization, VA-VAE, and frozen representers. Evaluation uses 50,000 images, EMA, seed 1, one NFE, and CFG 1: | Method | FID (ADM) at 5K ↓ | FID (ADM) at 50K ↓ | | --- | --- | --- | | NextGAN | 4.156 | **1.335** | | TBSM GAN corner ($v$) | **4.004** | 10.794 | | TBSM GAN corner ($\nabla u$) | 246.367 | 350.367 | TBSM ($v$) is slightly better at 5K but deteriorates thereafter; NextGAN achieves the best final FID. Both TBSM curves are continuous 0–50K runs. Appendix B.1 documents all 60 measurements and NextGAN's distinct early-run/main-run provenance.

Dear Reviewers,

Thank you for your thoughtful reviews. The authors have provided detailed responses to the concerns raised.

I would appreciate it if you could revisit the author responses and clarify whether your main concerns have been adequately addressed, particularly which issues remain critical to your overall assessment.

Please feel free to update your ratings if your assessments have changed following the discussion.

Thank you for your continued contributions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.