acceptodds
Under review as a conference paper at ICLR 2027

Generative Latent Zoning Networks

Abstract

Recent advances in AI have been driven largely by generative models such as diffusion models (DMs), autoregressive (AR) models, and generative adversarial networks (GANs). However, each paradigm has inherent limitations in generation efficiency, training stability, latent-space flexibility, or other desirable properties. We propose a new class of generative models called Generative Latent Zoning Networks (G-LZNs). G-LZNs map each sample to a latent zone with learnable boundaries and decode latents within the zone back to the sample. By design, G-LZNs simultaneously provide six desirable properties: (1) One-step generation, unlike DMs and AR; (2) Stable training with only a reconstruction loss, avoiding competing objectives such as those in GANs; (3) Flexible latent dimensionality and latent-to-data mapping, unlike AR, which lacks an explicit latent space, and DMs, whose latent space is constrained by their formulation; (4) Native inversion from samples to latent space, unlike GANs and AR; (5) Likelihood estimation, unlike GANs; and (6) Flexible architecture, unlike normalizing flows, which require invertible architectures. Their compact latent space and native inversion further enable image editing without task-specific training, including class changes, class and latent interpolation, fine-grained attribute editing, super-resolution, and inpainting. Crucially, G-LZNs achieve these properties without sacrificing generation quality. Among one-step generative models trained from scratch, G-LZNs set new state-of-the-art image generation results, achieving FID 1.88 on CIFAR-10 and FDr6 2.49 on ImageNet 256×256. Code and models will be released upon acceptance.

Primary area:
generative models
Keywords:
generative models, one step generation, likelihood, stable training, ImageNet, CIFAR10, one-ste

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_XJmzOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes G-LZN, in which an encoder predicts anchors and analytic inverse flow matching (IFM) constructs latent regions. Reconstruction training connects the encoder and decoder, while generation requires a single decoder evaluation. The experiments demonstrate strong generation results and several image-editing capabilities.

Critical strengths
  1. Self, recursive, and weighted zoning adapt the original latent zoning approach to standalone generative modelling. This is an interesting stochastic posterior construction worth exploring.

  2. The generation results are strong, and the paper includes multiple training seeds and component ablations, supporting effectiveness and training stability in the tested configurations.

Critical weaknesses
  1. The value of the added complexity is insufficiently established. Compared with a conventional VAE, the method adds an IFM flow module requiring numerical integration and backpropagation, while removing an explicit KL term that is computationally inexpensive. Without comparisons against strong baselines at the same computational budget, it is unclear whether the performance gains justify the additional computational and methodological complexity. This concerns the necessity of the added machinery, rather than merely missing cost reporting.

  2. A validation gap remains between the theory and the implementation. The theory assumes ideal latent regions, whereas the implementation uses finite smoothing and Euler integration. Region quality, the constant-KL property, and reversibility have not been sufficiently validated under these approximations. FID results and overall gradient curves do not directly answer these questions. The applicability of the guarantees to the implementation therefore remains unclear; this does not establish that numerical failures actually occur.

Overall assessment: The method has potential, but the necessity of its added complexity and the practical scope of its theoretical guarantees are not sufficiently supported. I lean toward weak rejection; clarification and targeted validation could change my assessment.

Final assessment after the author discussion

Thank you for the additional experiments, the compute analysis, and the explicit discussion of the reconstruction limitation. The compute analysis addresses my concern about the additional training FLOPs of increasing the latent dimension. Taking these clarifications and the supporting experiments into account, I have increased my rating from 2 to 3 (weak acceptance). Reconstruction fidelity on ImageNet with larger latent dimensions remains experimentally unverified. If the paper is accepted, the authors should complete this evaluation for the final version, reporting PSNR and LPIPS together with the computational cost at different latent dimensions.

Questions for authors
  1. Relative to LZN, DDVI, and related methods, what are the most important new technical elements and practical benefits?

  2. At the same computational budget, does the added IFM module still yield clear benefits over a conventional VAE and stronger variational posterior baselines?

  3. How do the smoothing parameter g, Euler step count, and recursive depth affect actual distributional error, reversibility, and gradient stability? How should the constant-KL result be interpreted for the implementation?

2 comments
AuthorsOpenReview ↗

Rebuttal (1/2)

We thank the reviewer for recognizing self, recursive, and weighted zoning as an interesting stochastic-posterior construction for standalone generation, as well as the strong generation results and multi-seed and component-ablation evidence supporting effectiveness and stability.

We have revised the manuscript in response to the reviewer’s comments, with all changes highlighted in red.

We welcome further discussion and are available to address any remaining questions.

W1/Q2Benefit over VAEs relative to the added complexity
  • The submission already includes a controlled VAE comparison. Appendix F.8 compares G-LZN and VAE under identical CIFAR-10 settings, differing only in the use of KL loss (VAE) versus the latent zoning mechanism (G-LZN). To ensure a fair comparison, we tune the VAE's KL weight over 12 values ranging from 10−610^{-6} to 5×10−35\times10^{-3}.

  • Latent zoning is more stable and achieves better FID in this setting. The VAEs show frequent loss and gradient-norm spikes (Fig. 22), while G-LZN remains smooth (Fig. 23); the best VAE obtains FID 9.23 versus 6.80 for G-LZN (Table 7). We have summarized this conclusion in Secs. 4.3--4.4.

  • G-LZN performs better than VAEs under the same training compute. Following the reviewer's suggestion, we additionally compare the FID scores of G-LZN and VAEs across all KL weights at equal training times in Fig. 25 of the revision. G-LZN consistently achieves lower FID than all VAEs at any given training time. Moreover, latent zoning is computationally lightweight, requiring only element-wise arithmetic on 50-dimensional latents without neural network evaluations inside the integration loop. Consequently, G-LZN incurs only a 16.5% increase in per-iteration training time compared with VAEs.

  • G-LZN also achieves a better compute-quality trade-off than a stronger variational-posterior baseline. The reviewer also requests a comparison with "stronger variational posterior baselines" in Q2. LSGM in Table 2a is one such baseline, employing a hierarchical variational posterior and a diffusion-based prior, both more expressive than those in conventional VAEs. We follow the FLOP-accounting convention in Appendix F.12: for G-LZN, we use the first checkpoint that reaches FID 2.10 and multiply its iteration count by the measured per-step FLOPs; for LSGM, we multiply the official iteration count by the per-step FLOPs measured from its official configuration. At the same FID of 2.10, G-LZN requires only 80% of LSGM's training compute (5.26×10195.26\times10^{19} vs. 6.61×10196.61\times10^{19} FLOPs; Appendix F.8).

Q1Relation to LZN and DDVI
  • G-LZN turns LZN into a standalone generator. Self-latent zoning removes minibatch dependence, while recursive and weighted zoning make very small zones practical. Consequently, whereas LZN can only improve an existing diffusion model, G-LZN samples directly from a Gaussian prior as a standalone generative model.

  • G-LZN and DDVI differ in both their approach and practical benefits.

    • Approach. DDVI parameterizes qϕ(z∣x)q_\phi(z\mid x) with a learned reverse-diffusion model and trains it using a regularized ELBO inspired by wake-sleep. G-LZN instead constructs its posterior using an analytic zoning velocity field determined by encoder-produced anchors; the field is numerically integrated but is not itself a learned diffusion model.
    • Practical benefits. First, DDVI optimizes a regularized ELBO for a learned diffusion posterior, whereas G-LZN uses reconstruction-based losses without an explicit latent-regularization objective. This is an objective-level distinction, not a claim that DDVI is empirically unstable. Second, G-LZN does not train an additional diffusion model: after one encoder pass, latent sampling uses only the inexpensive zoning computation quantified in W1/Q2 above, rather than repeatedly evaluating a diffusion model.

    We thank the reviewer for pointing us to this important related work and have expanded Appendix A (Related Work) to discuss this method explicitly.

AuthorsOpenReview ↗

Rebuttal (2/2)

W2/Q3Finite smoothing and Euler integration

We thank the reviewer for this insightful comment. We have added a new appendix section (Appendix F.13, "Approximation Error of the Implementation") that quantifies the gap between the theory and the implementation on the released CIFAR-10 model, with no retraining. We sweep each of the three parameters the reviewer names, the smoothing parameter gg, the number of Euler steps, and the number of latent zoning layers ll, one at a time around the training configuration, and we measure a superset of the quantities the reviewer asks about: distributional error, zone quality, the KL bound, reversibility, gradient stability, and end-to-end reconstruction quality. All error measurements are made against float64 reference solutions that are themselves cross-validated. The main findings:

  • Distributional error (Figure 35). At the training configuration, the latents deviate from the exact map by a median relative ℓ2\ell_2 error of 2.7%, and the error decreases linearly with the Euler step size. The error accumulates only mildly across layers (1.0% at l=1l=1 to 2.7% at l=15l=15, far below worst-case compounding), and gg has almost no effect (changing gg by two orders of magnitude changes the exact map itself by only 2×10−42\times10^{-4}).

  • Zone quality (Figure 36). At the training configuration, a layer's output lands in the correct zone of that layer's own anchors 91.6% of the time at the final layer, and all 15 layers are simultaneously correct for 24.8% of the samples; both improve monotonically with more steps (100% and 92.1% at 1,000 steps), so the leakage is mainly a discretization effect. Importantly, training does not rely on this within-sample zone separation; it only requires that the zones of different images do not overlap, which the reconstruction loss directly pushes for. The decoder is trained against the same 50-step map, so it fits the 50-step zones, and Figure 40 shows this carries no measurable end-to-end cost.

  • KL bound (Figure 37). This addresses the question of how the constant-KL result should be interpreted for the implementation. We prove in Appendix F.13 that finite smoothing preserves the bound KL≤−log⁡S\mathrm{KL}\le -\log S for any gg (zone overlap can only make the actual KL\mathrm{KL} smaller, so the bound in Corollary 1 remains valid and conservative). Euler discretization is the remaining approximation. As a diagnostic of its transport error, the moment-based Gaussian KL\mathrm{KL} between the numerically transported mixture and the prior is 0.27 nats at the training configuration, versus a 0.08-nat sampling-noise floor, and converges to that floor with more steps. This diagnostic does not directly estimate the conditional KL\mathrm{KL}; the conditional guarantee under exact finite-smoothing integration instead follows from the analytical bound above.

  • Reversibility (Figure 38). The smoothed ODE flow remains mathematically reversible under exact integration; Fig. 38 measures the additional numerical round-trip error introduced by finite-step Euler integration. A single layer inverts cleanly: the round-trip error is 0.044 at 50 steps and decreases linearly with the step size. Inverting the full 15-layer chain is ill-conditioned, for a structural rather than numerical reason: each layer stores the incoming sample at scale gg, so the inverse divides by gg and amplifies any integration error (the measured error scales exactly as 1/g1/g). This reverse integration is not used by the model: native image inversion and editing use the encoder-produced latents directly, while generation is a single decoder evaluation of a prior sample.

  • Gradient stability (Figure 39). The largest singular value of the Jacobian of the zoning map is flat at about 2.2 across all three parameter axes (worst sample 3.1), showing that backpropagation through 15×5015\times50 Euler steps does not exhibit exploding sensitivity. Together with the stable empirical gradient curves reported in the paper, this supports stable optimization of the zoning module.

  • End-to-end effect (Figure 39). Reconstruction FID, computed with the same FID implementation as all FID numbers in the paper, is flat for 25 or more Euler steps (1.73 to 1.80, with the training configuration in the flat regime), and gg is irrelevant to four decimal places across three orders of magnitude. The deviations measured above therefore carry no measurable cost on the inference path that uses the zoning integration.

In summary, the approximations affect the implementation in small, converging, and well-understood ways. Within-sample zone leakage incurs no measurable reconstruction cost, while numerical full-chain reverse integration is not used by either training or generation. We have added all of these results to the revised paper in Appendix F.13.

Reviewer_P8vvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes G-LZN, an encoder–decoder one-step generative model whose core objective is sample-wise reconstruction. Its key component replaces the standard encoder with a latent computation process built on a latent zoning network [1]. The encoder outputs a set of anchor points per sample, and inverse flow matching maps Gaussian noise into a part of the latent space (a zone) whose is determined by the anchors, so that each sample’s latent is the Gaussian prior restricted to its zone. To make this scale, zones are computed per sample rather than jointly (self-latent zoning), and the operation is chained across layers with non-uniform anchor weights (recursive and weighted zoning). Further, their best models add two losses computed with frozen pretrained embedding networks: a perceptual reconstruction loss, and a Fréchet distribution-matching loss on fresh prior samples, which the paper counts as group-level reconstruction. G-LZNs outperform many larger multi-step diffusion and autoregressive models.

[1] A unified principle for generative modeling, representation learning, and classification.

Critical strengths

Strong one-step results. Trained from scratch, G-LZNs reach FID 1.88 on CIFAR10 and FD$_{r}$ 2.49 on ImageNet 256×256, beating many larger multi-step models. Variance across seeds is low.

Novel methods progress prior work. Latent zoning comes from prior work (LZN), but on its own it doesn’t scale and couldn’t serve as a standalone generator. The techniques this paper proposes to close that gap are interesting.

Critical weaknesses

Competitive quality depends on pretrained-embedding losses. With only pixel-space l1 or l2 reconstruction, G-LZN reaches FID 44.87–53.65 on CIFAR10, producing blurry samples. Diffusion models trained with plain MSE, such as DDPM at FID 3.17, perform far better without any pretrained networks. The good results require perceptual losses from pretrained networks plus a Frechet loss that uses the same embedding networks as the evaluation metrics.

No compute-matched comparison with baselines. The paper compares methods by number of sampling steps and by parameter count, but it never compares them at matched training compute, and it reports no training FLOPs or GPU-hours. This makes the results hard to assess, because each G-LZN training step involves considerable computation:

  • an encoder pass followed by 15 sequential zoning layers of 50 Euler steps each, all differentiated through.
  • a decoder pass on the reconstruction, plus a second decoder pass on freshly sampled prior latents for the Frechet loss.
  • forward and backward passes through two frozen perceptual networks and six frozen Fréchet embedding networks.
Questions for authors

Clarifications

  1. What were the total training FLOPs and GPU-hours for G-LZN, and how do they compare with the main baselines in Table 2?
  2. The FD-loss-tuned one-step baselines in Table 10 (e.g., pMF-H + FD-loss) are built on generators trained from scratch and use the same Fréchet loss, yet they outperform G-LZN on FID, recall, and mostly FD$_{r}$, so why are they treated as not directly comparable, and what does latent zoning add over these pipelines?
5 comments
AuthorsOpenReview ↗

Rebuttal (1/2)

We thank the reviewer for recognizing G-LZN's strong one-step results, stable runs, and novel technical advances that turn LZN into a scalable standalone generator.

We have revised the manuscript in response to the reviewer’s comments, with all changes highlighted in red.

We welcome further discussion and are available to address any remaining questions.

W2/Q1Training compute

We thank the reviewer for this suggestion. We measured cumulative training FLOPs for every one-step method listed in Table 2b. The results are reported in Table 15, and the common accounting convention is described in Appendix F.12.

MethodSpace#ParamsFDr6↓\mathrm{FD}_r^6\downarrowFID$\downarrow$BatchIterationsper-step FLOPsCumulative FLOPs×\times ours (lat)
pMF-L/16pix410M9.092.721024400,3202.096×10152.096\times10^{15}8.392×10208.392\times10^{20}0.27
Drifting, L/16pix465M10.511.438192100,0001.422×10161.422\times10^{16}1.422×10211.422\times10^{21}0.46
pMF-H/16pix935M6.872.291024450,3604.435×10154.435\times10^{15}1.997×10211.997\times10^{21}0.64
W-Flow, B/2lat133M+49M11.621.524096200,0001.020×10161.020\times10^{16}2.039×10212.039\times10^{21}0.66
Drifting, L/2lat463M+49M10.921.538192200,0001.101×10161.101\times10^{16}2.201×10212.201\times10^{21}0.71
iMF-XL/2lat610M+49M8.391.8210241,000,8002.735×10152.735\times10^{15}2.737×10212.737\times10^{21}0.88
W-Flow, L/2lat463M+49M10.551.354096200,0001.425×10161.425\times10^{16}2.850×10212.850\times10^{21}0.92
G-LZN (ours)lat230M+49M3.621.202048270,0001.151×10161.151\times10^{16}3.108×10213.108\times10^{21}1.00
W-Flow, XL/2lat679M+49M9.511.294096180,0001.878×10161.878\times10^{16}3.381×10213.381\times10^{21}1.09
G-LZN (ours)pix182M2.491.74204865,0007.046×10167.046\times10^{16}4.580×10214.580\times10^{21}1.47
  • G-LZN's training compute falls within the range of the one-step baselines in Table 2b. Every one-step baseline trains with 0.270.27--1.09×1.09\times the FLOPs of our latent model (last column), i.e., our training cost is within the overall range, although toward its higher end. In particular, our latent model costs about the same as W-Flow XL/2 (3.108×10213.108\times10^{21} vs. 3.381×10213.381\times10^{21}) while substantially improving quality (FDr6\mathrm{FD}_r^6: 3.62 vs. 9.51; FID: 1.20 vs. 1.29). Under the common convention, Table 15 excludes the pretraining cost of public checkpoints for every method, including G-LZN's SD-VAE and embedding models. It therefore also excludes the separately trained 2.69B-parameter MAE feature extractor introduced by Drifting (2.179×10212.179\times10^{21} FLOPs for the latent variant) and reused by W-Flow. If this method-specific training stage is included, Drifting L/2 totals 4.380×10214.380\times10^{21} (1.41×1.41\times ours), while W-Flow B/2–XL/2 totals 4.218×10214.218\times10^{21}--5.560×10215.560\times10^{21} (1.361.36--1.79×1.79\times ours).

  • Measured shares of the three costs the reviewer lists. Of one latent-run training step: the 15×5015\times50 Euler zoning is 0.00009%; the two decoder passes are 7.8% + 7.8%; the frozen perceptual + Fréchet embedding networks are 30.4%. The frozen-network share is indeed substantial. As the next point shows, substantial frozen-network cost is also common among the one-step baselines, and the table above already charges this cost to every method.

  • Frozen-network cost is the norm across one-step methods, not a peculiarity of G-LZN. W-Flow spends 55.6–83.7% of each step (XL/2–B/2) in its frozen MAE extractor; Drifting spends 60.3–69.0% (latent–pixel variants) in its frozen feature extractors; pMF spends 12.8–27.0% (H–L) in its frozen VGG16 + ConvNeXt-V2 perceptual networks.

  • Our GPU-hours, and why we compare in FLOPs. Training one G-LZN model in Table 2 takes ≈43,000 A100-hours (A100-40GB). We report FLOPs as the comparison metric because: (i) none of the one-step baselines reports wall-clock or accelerator-hours, so there is no reported number to compare against; (ii) the alternative would be to time their training ourselves, but wall-clock depends on hardware: several baselines trained on TPUs (pMF on TPU v5p-64, iMF on TPUs, Drifting on 128× TPU v6e; W-Flow does not disclose its hardware) while we use GPUs, and porting them to PyTorch/GPU could be poorly optimized and overestimate their cost; (iii) FLOPs are hardware-independent and verifiable from the released configs, so we consider them the most reasonable metric.

  • Deployment cost is also favorable. The pixel-space G-LZN is tied for the smallest total model in the table, while the latent-space G-LZN is smaller than all listed methods except W-Flow B/2. Both generate with a single NFE. Training is a one-time cost, whereas deployment cost recurs with every generated sample and often dominates in today's large-scale use of generative models.

AuthorsOpenReview ↗

Rebuttal (2/2)

W1Pretrained feature encoders
  • Pretrained representations are a common but real dependency. We agree that this is a limitation, as also acknowledged in the Limitations section. Recent methods such as Drifting Models and REPA likewise use pretrained representations to improve generative training (Deng et al., 2026; Yu et al., 2025). This context does not remove the limitation; it shows that G-LZN follows an increasingly common training strategy whose contribution should be assessed separately from the representation resource.

  • Table 8 quantifies this dependence. Without pretrained embeddings or group loss, pixel ℓ1/ℓ2\ell_1/\ell_2 give FIDs 44.87/53.65; LPIPS gives 10.39; and DINO and ConvNeXtV2 give 5.47 and 6.96 (Table 8; Appendix F.9.1). Thus G-LZN can learn with pixel losses alone, but we agree that this pixel-only result is not competitive with DDPM's reported FID of 3.17; G-LZN's best perceptual quality currently benefits substantially from pretrained representations.

  • The overlap between training and evaluation embeddings requires caution. We agree that FDr6\mathrm{FD}_r^6 is not fully independent evidence when the same embeddings appear in the loss. We have stated this caveat explicitly in Appendix C.1.2 and compare with FD-loss-tuned baselines under the same convention rather than attributing the gains solely to G-LZN.

  • G-LZN's own encoder could remove the external representation. As already discussed in the Limitations section, a potential solution is to derive the reconstruction representation from G-LZN's own encoder, allowing it to be learned with the generator in the same framework.

Q2What latent zoning adds over FD-loss-tuned pipelines
  • The FD-loss-tuned pipelines are directly relevant comparisons, and several obtain better metrics. We agree with the reviewer: Table 12 includes these methods, and pMF-H + FD-loss is better than G-LZN on FID, recall, and most FDr\mathrm{FD}_r variants. Our categorization is intended to distinguish training pipelines, not to exclude these methods from the comparison or discount their results.

  • We distinguish single-stage training from pipelines that use or initialize from an already trained generator checkpoint. The first group in Table 12 includes both distillation and post-training methods. The reviewer-highlighted pMF/JiT/iMF + FD-loss variants specifically are post-trained from already trained base generators, although those base generators were originally trained from scratch. G-LZN instead trains one generator from random initialization in a single stage. Prior one-step work uses the same distinction: Consistency Models separates pretrained-model distillation from standalone training (Song et al., 2023), while iCT specifically studies the latter, training directly from data without distillation (Song & Dhariwal, 2023).

  • Compared with FD-loss-tuned pipelines, latent zoning provides a simpler single-stage training pipeline and additional built-in capabilities. It trains the generator directly from random initialization, without separate generator-pretraining and FD-post-training stages. It also provides native inversion, an explicit flexible latent, and likelihood bounds for compatible decoder/loss choices.

AuthorsOpenReview ↗

Follow-up with Reviewer P8vv

Dear Reviewer P8vv,

We submitted our rebuttal 4.5 days ago and would appreciate knowing whether it has addressed your concerns. We would be happy to discuss any remaining questions.

Thank you again for your time and thoughtful review, which has substantially improved our paper.

The Authors

Reviewer_BdMSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

Generative Latent Zoning Networks (G-LZNs) use self-latent zoning, recursive latent zoning, and weighted latent zoning to learn sample-specific latent zones. A decoder reconstructs samples from these zones and generates images from Gaussian prior samples in one step. Experiments show strong image-generation results on CIFAR10 and ImageNet 256×256256 \times 256, alongside image editing without task-specific training.

Critical strengths
  • Self-latent zoning removes dependence on other samples in a minibatch, while recursive and weighted latent zoning reduce the anchor requirement. The framework combines one-step generation with native inversion and flexible latent dimensionality.

  • The VAE comparison removes Fréchet loss from both models and tunes the KL regularization weight. G-LZNs achieve FID 6.80 versus 9.23 for the best tested VAE, supporting a contribution from latent zoning beyond the added Fréchet loss.

  • Five CIFAR10 runs achieve FIDs between 1.83 and 1.90, with consistent loss and gradient trajectories, supporting stable training in this setting.

Critical weaknesses
  • With S≪1/nS \ll 1/n, the training zones occupy at most nSnS of the Gaussian prior under the idealized construction, leaving almost all prior mass outside the regions directly constrained by reconstruction. The pooled marginal histogram and mean/standard-deviation curves do not establish joint prior matching. The implemented Fréchet loss adds supervision on prior samples, so the prior-coverage argument at S=1/nS = 1/n does not explain the practical model.

  • Theorem 2 requires a normalized, evaluable pD(x∣z)p_D(x \mid z), and Corollary 1 requires reconstruction to correspond to its log-likelihood. This connection is not established for the sample-wise improved perceptual loss and group-wise Fréchet loss used in the main experiments. The bound therefore does not justify interpreting the implemented training objective as likelihood optimization.

  • On ImageNet 256×256256 \times 256, G-LZNs report recall of 0.52 with a pretrained autoencoder and 0.57 without one, versus 0.67 for W-Flow XL/2. These values indicate lower measured distribution coverage, despite strong FID and FDr6\mathrm{FDr}^{6}. The Fréchet loss also uses all six FDr6\mathrm{FDr}^{6} embedding models, limiting the independence of that evaluation. The results therefore retain a quality–coverage trade-off.

  • The shared within-group covariance approximation in the group-wise Fréchet loss discards class-specific covariance differences. Even with correct class means, incorrect class covariances can cancel in the pooled statistic. The group-wise Fréchet term therefore cannot distinguish some mismatched conditional distributions, limiting how fully it addresses multimodal data.

Questions for authors

Please see my "weaknesses" comments.

4 comments
AuthorsOpenReview ↗

Rebuttal (1/2)

We thank the reviewer for recognizing the technical advances that make latent zoning scalable, its combination of one-step generation, native inversion, and a flexible latent space, and the VAE and multi-seed evidence supporting its effectiveness and stability.

We have revised the manuscript in response to the reviewer’s comments, with all changes highlighted in red.

We welcome further discussion and are available to address any remaining questions.

W1S≪1/nS\ll 1/n, Corollary 1, and prior coverage
  • Theorem 1's scope is already clearly separated from the practical S≪1/nS\ll1/n setting. Immediately after Theorem 1, we state that G-LZNs seek generalization beyond the training set; Sec. 3.4 explains our use of S≪1/nS\ll1/n; and the Limitations section states that a full characterization remains open. We have added a paragraph at the beginning of Sec. 3.3 to make this distinction clearer.

  • Corollary 1 explains why reconstruction training can still work when S≪1/nS\ll1/n. Unlike Theorem 1, Corollary 1 does not require a finite dataset or S=1/nS=1/n. For a compatible decoder/loss pair, Elog⁡pD(X)≥−βLrec+E[c(X)]+log⁡S\mathbb E\log p_D(X)\ge-\beta\mathcal L_{\rm rec}+\mathbb E[c(X)]+\log S. For fixed SS, log⁡S\log S is parameter-independent, so minimizing reconstruction loss improves this bound even when S≪1/nS\ll1/n. Corollary 1 therefore explains the optimization principle, not coverage of the prior by the nn training zones.

  • The reported marginal statistics do not establish joint prior matching, but exact matching is not required for the practical model. We agree that the pooled histogram and mean/standard-deviation curves alone do not establish joint matching. In the full model, the group-level loss directly supervises fresh prior samples and improves decoder behavior under z∼PZz\sim P_Z, as the reviewer notes. Moreover, the sample-wise-only results below show that meaningful prior generation already emerges without the group-level loss.

  • Sample-wise training already produces meaningful prior samples. Without any group-level loss, DINO gives FID 5.47, ConvNeXtV2+DINO 6.80, and LPIPS 10.39 on CIFAR-10; group-level loss alone gives FID 33.18 and recall 0.090. Pixel ℓ1\ell_1 alone also produces recognizable samples (Table 8; Appendix F.9.1). Thus sample-wise reconstruction is the primary learning mechanism rather than merely an auxiliary component of Fréchet-loss training. We have revised Sec. 4.4 to highlight it better.

W2Likelihood interpretation of the practical losses

Thm. 2 and Corollary 1 rely on idealized assumptions about the loss functions and decoder implementation. Here, we clarify how the practical implementations affect the interpretation of these theoretical results.

  • The deterministic decoder with pixel ℓ2\ell_2 loss is covered by the theory. This setting is used in Fig. 3 and the ℓ2\ell_2 row of Tab. 8. As discussed in Sec. 3.3, our theory covers this setting by defining pD(⋅∣z)=N(D(z),σ2I)p_D(\cdot\mid z)=\mathcal{N}(D(z),\sigma^2 I), where DD is the deterministic decoder and σ\sigma is a fixed constant.

  • The sample-wise feature loss is covered when likelihood is defined in feature space. Our best results are achieved by applying sample-wise losses in feature space. Let T(x)T(x) concatenate our frozen features and statistics, and treat Y=T(X)Y=T(X) as the observation. A fixed-variance Gaussian pD(⋅∣z)p_D(\cdot\mid z) centered at T(D(z))T(D(z)) yields the p=2p=2 loss, while a fixed-scale Laplace distribution yields the p=1p=1 loss. Thus, Thm. 2 and Corollary 1 apply to the feature-space likelihood, but not necessarily to the normalized raw-image likelihood when TT is non-invertible.

  • The Fréchet loss preserves the exact-recovery solution but does not correspond to a likelihood. Under the assumptions of Thm. 1, exact recovery matches all corresponding feature moments, making the nonnegative Fréchet loss zero without altering the optimum. The practical normalization divides each Fréchet loss by the larger of its stop-gradient value and the clipping constant ϵ\epsilon, and therefore also preserves this zero-valued optimum (Appendix C.1.2). However, the Fréchet loss itself should not be interpreted as a likelihood.

We have added Appendix C.1.3 to include these discussions.

AuthorsOpenReview ↗

Rebuttal (2/2)

W3Quality-coverage trade-off

We agree that image quality and coverage should be considered together. In the revision, we have added Fig. 34, which compares the precision and recall of all methods with available results in Tab. 12. We make two key observations:

  • Most methods exhibit a precision-recall trade-off. Most methods fall within a band extending from the top-left to the bottom-right, suggesting that improvements in precision often come at the cost of recall, and vice versa.

  • G-LZN achieves a strong precision-recall balance given its model size and one-step sampling. The only methods that match or exceed G-LZN (pixel version) in both reported precision and recall are DiT-XL/2, VAR-d24, VAR-d30, PixNerd-XL, Drifting L/16, SiT-XL/2, and MAR-L+DeTok. All these methods use larger models, and most also require substantially more sampling steps.

  • The embedding overlap limits the independence of FDr6\mathrm{FD}_r^6 evaluation. We agree with the reviewer and do not treat FDr6\mathrm{FD}_r^6 as fully independent evidence because its embedding models also appear in the training loss. We have made this caveat explicit in Appendix C.1.2 and interpret FDr6\mathrm{FD}_r^6 together with the precision-recall results and comparisons against FD-loss-tuned baselines evaluated under the same convention.

We have added these results and discussions to Appendix F.11.

W4Shared covariance approximation
  • We agree that shared covariance cannot capture every class-specific mismatch. In particular, Eq. (27) can miss differences among class-conditional covariances.

  • Our narrower claim is that group-wise loss improves several metrics over the original non-group-wise Fréchet loss, not that it resolves every distributional mismatch or improves every metric. We use shared covariance to capture class-mean mismatches that global FD can hide, while retaining only one covariance square root per iteration and thus essentially the same computational cost as the original loss. With all embeddings, this change improves FDr6\mathrm{FD}_r^6 from 3.62 to 2.53, recall from 0.524 to 0.560, precision from 0.792 to 0.820, and sFID from 60.35 to 55.23, while FID changes from 1.20 to 1.50 (Table 9; Appendix F.9.2). Therefore, it is an efficient extension of the original Fréchet loss with clear gains on several complementary metrics, although it neither improves every metric nor addresses all forms of distributional mismatch.

We have revised Appendix C.1.2 to state this scope explicitly and discuss low-rank class-specific covariance as a possible extension.

AuthorsOpenReview ↗

Follow-up with Reviewer BdMS

Dear Reviewer BdMS,

We submitted our rebuttal 4.5 days ago and would appreciate knowing whether it has addressed your concerns. We would be happy to discuss any remaining questions.

Thank you again for your time and thoughtful review, which has substantially improved our paper.

The Authors

AuthorsOpenReview ↗

Follow-up with Reviewer BdMS before the discussion period ends

Dear Reviewer BdMS,

As the discussion period is ending soon, we would like to kindly check once more whether our rebuttal has addressed your concerns. For your convenience, here is a brief summary of our responses and the corresponding revisions (highlighted in red in the paper):

  • W1 (S≪1/nS \ll 1/n and prior coverage). Sec. 3.3 now separates the results that apply to S=1/nS = 1/n from those that hold for general SS. Corollary 1 does not require S=1/nS = 1/n. Sample-wise training alone, without any group-level loss, already produces meaningful prior samples (Table 8).
  • W2 (likelihood interpretation of the practical losses). The new Appendix C.1.3 clarifies which practical losses are covered by the theory, and states that the Fréchet loss should not be interpreted as a likelihood.
  • W3 (quality-coverage trade-off). The new Appendix F.11 compares precision and recall against all methods with available results (Fig. 34), and we state explicitly that FD$_r^6$ is not fully independent evidence.
  • W4 (shared covariance approximation). We agree that the shared covariance cannot capture every class-specific mismatch, and Appendix C.1.2 now states the scope of our claim.

If any concern remains, please let us know, and we will do our best to address it in the time left. If our responses have resolved your concerns, we would be grateful if you could consider updating your evaluation.

Thank you again for your time and thoughtful review.

The Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

Response to All Reviewers

We thank all reviewers for their careful reading and constructive feedback. We have revised the manuscript in response to the reviewers' comments, with all changes highlighted in red. The main revisions are:

  • Training compute. Appendix F.12 reports training FLOPs and shows that G-LZN's training compute falls within the range of the existing one-step methods listed in Table 2b. Latent zoning consists of inexpensive element-wise operations on 50-dimensional latents, requires no neural network evaluations, and accounts for only 0.00009% of the total training FLOPs. Fig. 25 further shows that G-LZN consistently achieves lower FID than the VAEs at the same training time.
  • Quality-coverage trade-off. Appendix F.11 shows that G-LZN achieves a strong precision-recall balance relative to the baselines, given its model size and one-step sampling.
  • Impact of implementation approximations. Appendix F.13 analyzes how finite smoothing and Euler discretization affect distributional error, zone quality, the KL bound, reversibility, gradient stability, and end-to-end reconstruction quality. The measured deviations are either small or affect properties not relied upon by the actual training or generation procedures, and they incur no measurable end-to-end reconstruction cost.
  • Theory clarifications. We added an overview at the beginning of Sec. 3.3 to clarify which results apply to S=1/nS=1/n and which hold for general SS. Appendix C.1.3 further clarifies how the practical loss implementations affect the interpretation of the theoretical results.
AuthorsOpenReview ↗

Response to reconstruction fidelity

We thank the reviewer for the follow-up and the concrete experimental suggestions. We ran both requested evaluations and added the results to the revised paper (Appendix F.3, with all changes highlighted in red). Below, we report the results and answer each question.

Q1IFM versus explicit KL regularization
  • Held-out PSNR and LPIPS using the same checkpoints as Appendix F.8. We use the existing G-LZN checkpoint and the VAE checkpoint with the best generation FID (KL weight 10−310^{-3}, selected from all 12 KL weights) to reconstruct the entire CIFAR-10 test set: the same 10,000 held-out images for both models.
MethodPSNR (dB, higher is better)LPIPS (lower is better)
VAE (KL = 10−310^{-3})15.810.058
G-LZN16.220.053

G-LZN reconstructs held-out images more faithfully than the best VAE on both metrics. These results are now reported in Appendix F.3 (Table 5).

  • We did not run a matched VAE comparison on ImageNet. The matched comparison in Appendix F.8 is limited to CIFAR-10; we do not have a corresponding ImageNet model. Training a comparable VAE on ImageNet 256×256256\times256, particularly with the KL-weight sweep used in Appendix F.8, is beyond what we can complete during the discussion period. We therefore do not claim that the CIFAR-10 benefit carries over to ImageNet, and the paper explicitly scopes this comparison to CIFAR-10. For the ImageNet-specific question of what the 50-dimensional latent preserves, the new experiment below measures it directly against the same SD-VAE used by G-LZN.
Q2The 50-dimensional bottleneck on ImageNet 256×256256\times256
  • The 50-dimensional bottleneck loses a substantial amount of per-image detail. We ran exactly the comparison suggested by the reviewer on the full validation set (50,000 images) rather than a subset. Both paths use the same SD-VAE checkpoint, the same deterministic encoding procedure, the same preprocessing used during training, and the same input images; they differ only in whether G-LZN is inserted.
Reconstruction pathPSNR (dB, higher is better)LPIPS (lower is better)
SD-VAE reconstruction25.690.068
SD-VAE + G-LZN13.520.425

The table, added as Table 6 in the revision, shows that the 50-dimensional bottleneck indeed produces less faithful image reconstructions. We also added Fig. 12 in Appendix F.3, showing 88 classes with 22 validation images per class and three columns: the original image, the SD-VAE reconstruction, and the SD-VAE reconstruction with G-LZN inserted. G-LZN's reconstructions preserve the class, pose, and global scene layout, while local appearance details, including textures, exact object arrangements, and text, are re-synthesized rather than copied. This is consistent with the reviewer's observation about Fig. 6. The SD-VAE alone preserves more detail.

  • How we interpret this result. Information loss is expected when the 32×32×432\times32\times4 spatial latent of the SD-VAE (4,096 dimensions) is compressed into a 50-dimensional global latent, an approximately 82-fold reduction. The G-LZN latent is optimized as a compact generative representation for one-step generation, not as a compression codec; its dimensionality is a tunable design choice.

  • How images are sampled. For the quantitative results above, we use the whole validation set, so no sampling is needed. For the visualization in Fig. 12, the class labels are sampled uniformly without replacement using a fixed seed, and each class shows its first two validation images in the standard filename-sorted order. The full selection can be reproduced with the following code (NumPy 2.4.6, torchvision 0.29.0):

    import numpy as np
    from torchvision.datasets import ImageNet
    
    classes = np.random.default_rng(0).choice(1000, size=8, replace=False)
    # [844, 75, 16, 508, 306, 268, 40, 633]
    ds = ImageNet(root=..., split="val")
    examples = {c: [i for i, t in enumerate(ds.targets) if t == c][:2]
                for c in classes}
    

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.