acceptodds
Under review as a conference paper at ICLR 2027

Tiled Residual Flow: Memory-Bounded Exact-Transform Generation

Abstract

High-resolution image-space generative models are limited by activation memory that grows with image area, while latent-space models reduce memory only by relying on a learned decoder. We propose Tiled Residual Flow, a VAE-free architecture that decomposes an image into a deterministic low-resolution base and a high-resolution residual with exact additive reconstruction. A global flow models the base; a shared tile-local flow models residual tiles conditioned on crops of the upsampled base. Peak residual-stage memory is set by tile geometry rather than image size: the measured per-step training floor is 10.7x below a full-frame counterpart at 512x512, growing to 167x at 2048x2048, where full-frame training exceeds an 80 GB accelerator at batch size 2. Ablations at base resolution 128 show that removing local base conditioning nearly doubles FID, while the stitching rule, halo width, tile size and model width have no detectable effect on FID; halving the base resolution to 64 roughly halves FID at the same parameter count. On FFHQ at 512x512, with 7.4M- and 7.6M-parameter models trained on the same data, this memory bound costs no FID we can detect: the tiled model at base resolution 64 matches the best checkpoints of a full-frame flow-matching baseline whose receptive field is wider than the frame, after the same 350k full-resolution training steps (FID 56.8 against 57.8) and remains level at equal training compute, under a shared evaluation protocol and with no state-of-the-art claim.

Primary area:
generative models
Keywords:
flow matching, generative models, high-resolution image generation, memory-efficient training, pixel-space generation, VAE-free, tiled generation, multiscale decomposition, image synthesis

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_yFHuOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents Tiled Residual Flow, which generates an image in two stages. A global flow model first produces a low-resolution image. A shared local flow model then generates residual patches to fill in what the upsampled image is missing. These patches are blended and added to the image, with no learned decoder needed. Processing patches in small batches keeps the residual network’s activation memory bounded as resolution increases.

Critical strengths
  1. the proposed methods can save memory, which is useful for practical usage.
Critical weaknesses
  1. Both the qualitative results in appendix and the quantitative results in table 2 are not attractive. Therefore, I am still concerned about the effectiveness of the propose methods.
  2. I think authors should provide more analysis why we need another residual flow to learn details. Currently, diffusion model is actually a model to iteratively refine a figure and therefore it is strange to introduce an additional flow to do the same thing.
Questions for authors

see weaknesses.

2 comments
AuthorsOpenReview ↗

Response to Reviewer yFHu: results and the role of the second flow

Thank you for the review.

W1 (results). At 512 the tiled model's FID-50k is now 18.99, against 49.7 for the submitted checkpoints scored with FID-50k (Appendix A.2). It is 10% lower than the full-frame model's 21.00 at about the same training GPU-hours (ratio 0.904 [0.879, 0.930]; general response). The full-frame model keeps better fine detail: native patch FID is 20.72 against 17.30 at 512, and at 1024 the tiled model's FID-50k and native patch FID are 1.068 and 1.195 times the full-frame model's. We withdraw the parity claim. For the qualitative results, Appendix P shows uncurated samples; Figures 4 and 5 show the models of Table 2 at 512 and 1024.

W2 (why a second flow). The second flow is a spatial split that bounds model-step memory, not a second refinement of the same image. The first flow generates a small image that carries the global layout. The second generates only what the upsampled base lacks, one tile at a time, and the two add exactly, with no learned decoder. No network runs on the full frame at full resolution, so this memory, excluding full-resolution data buffers, is set by the base and tile sizes and not by the image size: at 1024 the floor is 0.81 GiB against 11.96 GiB for a full-frame step (general response). It is not universally necessary: a single full-resolution flow keeps better fine detail (W1). Figure 8 shows the same generated bases without and with the second stage, and the reply to bUoa (W1 and Q1) gives the numbers; Section 5.3 and Appendix G report how far an exact base alone is from the data. The precedent is cascaded diffusion (SR3 and Cascaded Diffusion Models; Section 2), whose second stage also conditions on an upsampled low-resolution image; ours generates the exact additive residual, tile by tile. A quality advantage from predicting residual tiles over image tiles is not established (reply to bUoa, W2 and Q2).

Thank authors for the explanation.

For weakness 2, I am still not convinced. I expect authors to provide some theoretical analysis on this method to support the effectiveness and generalization. Besides, the qualitative results in the appendix don't make this method acceptable. This means there still exist some problems on effectiveness of the proposed method.

I highly recommend authors to provide theoretical analysis to show the effectiveness and generalization of the proposed method.

Currently, I will maintain my score.

Reviewer_bUoaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a decoder-free generator that combines a low-resolution global flow with a shared tile-local residual flow. An exact additive decomposition enables deterministic reconstruction, while local processing bounds the residual model’s activation-memory requirement. Experiments characterize the minimum training footprint and report comparable FID to a full-frame baseline on FFHQ at 512×512.

Critical strengths
  1. By fixing the patch size and the number of patches processed per call, the activation memory usage of the residual network becomes independent of the total image area. The authors provide measurements of memory consumption and clearly distinguish between the minimum memory required for a training step, the workload associated with a full image, and the total computational cost—thereby clarifying the benefits and limitations of this design.
  2. The authors also provide a five-level baseline—featuring a receptive field that covers the entire image and a comparable number of parameters—along with comparisons across multiple training seeds and varying training durations. On the FFHQ-512 dataset, the patch-based model achieves an FID of 56.8±2.2, which is close to the 57.8±9.9 FID recorded for the best checkpoints selected separately for the two baseline training seeds.
Critical weaknesses
  1. The paper demonstrates a per-step memory advantage and competitive quality at 512×512. However, generation quality remains insufficiently validated at higher resolutions, where this advantage matters most: the limited 1024×1024 pilot does not improve over the base-only result, and no quality results are reported at 2048×2048.
  2. The quality gains from residual prediction—compared to directly generating RGB image patches—remain unclear. While Appendix H notes that memory savings primarily stem from localized computation, existing ablation studies on quality lack a direct comparison between these two prediction approaches under identical conditions (i.e., the same base image, network architecture, and training budget). Consequently, it is difficult to determine how much additional benefit is derived from the residual representation itself.
Questions for authors
  1. Do you have higher-resolution results reporting generation quality, peak memory, and the training cost of both stages under a fixed memory budget, particularly where full-frame training is infeasible?
  2. Holding other conditions constant, does generating residual patches—as opposed to directly generating RGB patches—consistently yield improvements in quality or training efficiency?
3 comments
AuthorsOpenReview ↗

Response to Reviewer bUoa: higher resolution, cost under a memory budget, and residual against RGB tiles

Thank you for the two precise questions.

W1 and Q1 (higher resolution). At 1024 we now report quality, peak memory and training cost under a memory cap. At 4096 we report memory and a feasibility demonstration, with no quality claim. We have no quality results at 2048.

Quality at 1024 (general response; ten runs against ten). FD-DINOv2 and KD-DINOv2 are 0.831 and 0.775 times the full-frame model's, while FID-50k and native patch FID are 1.068 [1.021, 1.117] and 1.195 [1.096, 1.303] times (95% intervals).

The residual against the base alone. In 1024 pipelines with a 100,000-step base and a residual network without the unshuffle (each stage trained on 25.6M samples; not the 1024 model of Table 2), the generated base alone, upsampled, scores 4.41 and 2.62 times the full-frame FID-50k with 128- and 256-pixel bases, against 1.27 and 1.35 for the full pipelines. This replaces the submission's 50k-step run at 1024, which the revision removes. Figure 8 shows the same generated bases without and with the second stage.

Peak memory at 1024. The model-step floor is 0.33 GiB (128-pixel base) or 0.81 GiB (256-pixel base) against 11.96 GiB for a one-image full-frame step. With the 256-pixel base, the base stage sets the peak reserved training memory, 102.13 GiB against 105.14 GiB for the full-frame model.

Training cost under a memory cap, in projected GPU-hours:

ResolutionTiled (base + residual)Full-frame, no capFull-frame, capped at the tiled model's peak
51244.0 (24.5 + 19.5)42.755.8 (26.75 GiB)
1024234.8 (169.6 + 65.2)231.1230.3 (102.13 GiB)

Under a 40 GiB cap at 1024 the figures are 238.0 tiled against 231.5 full-frame. Capped at the tiled model's peak, the full-frame model takes 1.27 times as long as the tiled model at 512 and slightly less time at 1024. These are projections from throughput benchmarks on one H200 with synthetic data and allocator caps, not measurements on a smaller device, and quality under the caps was not tested. The tiled advantage is mainly the lower floor, which matters where an uncheckpointed full-frame step does not fit.

Where full-frame training is hardest. At 4096 we trained the pipeline on PD12M crops (three seeds, a 512-pixel base) as a feasibility demonstration. The tiled stages peaked at 33.00 GiB allocated (set by the base; BF16, batch 24) and up to 61.7 GiB reserved (the FP32 run). A plain FP32 full-frame step of a seven-level frame-covering network (a memory comparator only) fit on no device we measured (A100 40 GB, H100 80 GB, H200 141 GB). With block or level checkpointing it fits on an H200 at batch 1 in FP32 or batch 2 in BF16, so we do not claim that full-frame training is impossible there. The uncurated samples at 4096 (Appendix P, Figures 10 to 12) include many incoherent textures, with a few coherent scenes, so we make no quality claim at 4096.

W2 and Q2 (residual or RGB tiles). Residual tiles do not measurably beat RGB tiles. Both targets were compared on the same generated 64-pixel bases at 512, with the same residual network architecture, training settings, training budget and compute per step (attention off in the stage networks). Ratios are image target over residual target, so values above 1 favor the residual target, with 90% intervals (paired in the two tiled rows):

Second stageFID-50kNative patch FID
Tiled, five seeds1.053 [0.930, 1.192]1.059 [0.917, 1.223]
Tiled, ten additional pairs1.104 [0.801, 1.522]1.175 [0.762, 1.811]
Frame-covering0.933 [0.860, 1.012]0.894 [0.808, 0.988]

All ten additional pairs are included, among them runs at FID-50k 155.7 and 96.6; the fifteen tiled pairs pooled give FID-50k 1.087 [0.886, 1.333] (90%). Training curves at fixed checkpoints (five seeds per target, FID-10k) put image over residual above 1 at all six checkpoints, from 1.276 at 0.28M to 1.048 at 5.6M training images or tile equivalents. We make no claim that the residual target improves quality or training efficiency; the revision presents the residual as the parameterization that makes the sum exact.

Reviewer_bUoaOpenReview ↗

Response to Authors

Thank you for the additional results at 1024×1024. The comparison between RGB and residual targets under matched training conditions addresses my second question.

I have one remaining question about 2048×2048. Table 1 reports a substantial reduction in the memory required for a single training step at this resolution. I understand that no quantitative quality evaluation is available. Have you tried sampling at 2048×2048 with the 1024×1024 checkpoint, keeping the base resolution and tile size fixed? This would double the upsampling factor and could affect the detail recovered by the residual model. If you have tried this setting, a few uncurated samples would help me assess the result. If it remains untested, could you clarify whether the current model supports this change at inference time?

Area_Chair_CVsdOpenReview ↗

Please help to join the discussion

Dear Reviewer bUoa,

The discussion deadline is approaching. Please help to join the discussion and offer the reviewer's opportunity to further respond to your concerns.

Thanks!

AC

Reviewer_xYPgOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents Tiled Residual Flow, a VAE-free model that splits an image into a low-res base and a high-res residual. The base is generated with a full-image flow model, and the residual is made tile by tile using a shared local flow, then stitched back together with overlap-add.

Critical strengths
  1. The paper uses an exact base/residual split and drops the learned decoder, which directly tackles the memory problem in high-res pixel-space generation. I think the problem is well chosen and the motivation makes sense.
  2. The core idea of the "global low-resolution + local high-res residual" is clean, natural, and exactly reversible, which feels genuinely new.
  3. The memory numbers are impressive and well reported. The abstract directly shows that training memory drops by 10.7× at 512×512 and 167× at 2048×2048 compared to full-frame baselines, so memory scaling is a real headline result.
  4. The paper stays focused. It builds step by step around three ideas — exact transform, tile locality, and memory bounds — and the structure is easy to follow.
Critical weaknesses
  1. Local tile generation could hurt global consistency. The low-res base carries global information, but the residual is still made per tile. On harder scenes, text, crowded objects, or images that need long-range structure, this may not hold up well.
  2. The numbers are not yet rock solid. The paper itself admits that: the 5-layer baseline was run with only two seeds, the tiled model with three seeds, baseline FID jumps around a lot between checkpoints, and both the M=64 choice and best-checkpoint pick are tied to the eval split. So "matches the baseline" is probably true directionally, but not yet statistically tight.
  3. The evidence is narrow. Most of the good quality results come from 512×512 FFHQ against a matched full-frame baseline. Higher resolutions and open-domain scenes are not well covered, which limits how convincing the story is.
  4. It is far from current practice. Tiled Residual Flow requires a whole new decoder-free, image-space stack — a base flow plus a tiled residual flow for both training and sampling. That makes it interesting, but also hard to drop into systems people already use.
  5. The bar for results feels modest. Tiled Residual Flow reads more like a "here is how memory scales" paper than one that actually beats existing methods on high-res output quality.
  6. The evaluation is not fully rounded. There are no user studies, so the claim of "no clear quality loss" really only holds within the specific benchmarks and evaluation matrices the paper uses. Human perceptual evidence would help close that gap.
  7. Tiled avoids using a learned decoder through exact decomposition, but in doing so it effectively shifts the challenge of recovering information lost in the low-resolution base to the local residual flow. For high-fidelity generation, this problem may be harder than the paper suggests, because many critical details are not merely local high-frequency components, but depend on broader structural and semantic context.
  8. The paper specifies the downsampling operator clearly, but does not isolate its effect experimentally. Since the exact base-residual split is defined by the downsample-upsample round trip, the method may be more sensitive to this design choice than the current ablations reveal.
Questions for authors

see above comments

3 comments
AuthorsOpenReview ↗

Response to Reviewer xYPg (1/2): W1 to W4

Thank you for the detailed review. The general response gives the main comparison and the withdrawn claims; the memory ratios quoted in your review are restated there for the whole pipeline. W5 to W8 follow in the next comment.

W1 (global consistency). We agree, and Section 6 now states this limitation: the residual network sees a 160-pixel window (a 128-pixel tile with a 16-pixel halo), so wider structure must come from the base. What we measured:

  • Tile boundaries. With one noise field shared by all tiles, as in every model of the main comparison, a classifier separates boundary crops from interior crops at 49.8% to 50.4% accuracy (chance is 50%), against 74% to 82% with independent noise per tile. A check on the 1024 models found no seam but had limited power, so we do not claim that seams are absent.
  • Structure across tiles. On man-made scenes (W3), detected straight lines break at tile boundaries 9.4% more often than at a shifted virtual grid (ratio 1.094 [1.073, 1.116], 90% interval).
  • Text. We have no text test.

W2 (statistics). We agree: the submitted comparison could not support "matches", and we withdraw that claim. The main comparison now reports independent training runs with 95% intervals, 4 tiled against 12 full-frame at 512 and 10 against 10 at 1024, each evaluated at its final checkpoint (general response). Bases and full-frame models are evaluated with post-hoc weight averaging, and held-out FID and KID agree in direction with FID-50k at both resolutions (Appendix A.3).

W3 (scope of the evidence). 1024 is now part of the main comparison (general response). Two more datasets at 512, as ratios of tiled over full-frame:

DatasetFID-50kNative patch FID
Man-made scenes: 42,388 SA-1B images of buildings, interiors and signage (five seeds per family, 90% intervals)1.361 [1.302, 1.422]1.488 [1.422, 1.556]
AFHQ, class-conditional, labels to the base only / to both stages (eight seeds per configuration)1.74 / 1.692.59 / 2.44

Both use a 64-pixel base and a residual network without the unshuffle, not the 512 pipeline of Table 2; the man-made scenes use FFHQ's training settings. On AFHQ, class accuracy is 98.5% / 98.8% (same order) against a classifier ceiling of 99.93%. Samples are in Figures 6 and 7. Training at 4096 is a feasibility demonstration (reply to bUoa). We have no quality results at 2048.

W4 (use with existing systems). Integration with latent and text-conditioned systems is untested (Section 6). The second stage needs only a low-resolution image, so the base is interchangeable in principle, and class conditioning through the base works (AFHQ, W3). We applied one trained second stage (a 1024 residual network for 256-pixel bases, without the unshuffle, used as trained) to 256-pixel samples of a public pretrained generator (R3GAN, FFHQ-256):

Upsampling to 1024Native patch FIDFID-50k
Our second stage13.3 [13.1, 13.5]9.47
Real-ESRGAN21.816.3
Bicubic28.48.81
Base-only upsampling6329.8

The interval is a 90% interval from half-sample image replicates; three second-stage seeds give 13.1 to 14.3. The base dominates FID-50k. Caveats: the generator was pretrained on all 70,000 FFHQ images, including our reference and held-out images; after conversion to the base format its samples show a conditioning shift in the highest frequency band evaluated; training compute is unmatched. It is a capability demonstration, not a comparison with the full-frame model.

AuthorsOpenReview ↗

Response to Reviewer xYPg (2/2): W5 to W8

W5 (level of the results). We agree and make no state-of-the-art claim: these are small models trained on 63,000 images. The paper's claims are the memory bound and the measured comparison with a full-frame model at about the same training GPU-hours (general response).

W6 (human evaluation). We ran no user study and make no perceptual-parity claim. Beyond FID, FD-DINOv2 and KD-DINOv2 are lower for the tiled model at both resolutions (general response). At 512, precision (0.927 [0.899, 0.956]) and sFID (1.094 [1.040, 1.150]) favor the full-frame model, and recall is 0.978 [0.927, 1.032] (ratios of tiled over full-frame, 95% intervals; Appendix A.3). Appendix P (Figures 4 to 12) shows uncurated samples as qualitative context.

W7 (what the residual stage must recover). We agree that the residual stage carries more than local detail. Fine detail is where the tiled model is behind: its native patch FID is 1.198 and 1.195 times the full-frame model's at 512 and 1024 (general response). These controls fit your point, although none of them isolates locality.

Real bases. FID with held-out real bases in place of generated ones (7,000 samples against all 70,000 images; each stage trained on 25.6M samples):

Resolution, baseReal basesGenerated bases
512, 128-pixel3.2927.23
512, 64-pixel12.5637.28
1024, 128-pixel3.8528.05
1024, 256-pixel2.4929.61

Base quality and the mismatch between real and generated conditioning both contribute to the gap, so it does not estimate either stage's share. With the 64-pixel base, FID on real bases stays at 12.56 (35.6 for one of the twelve runs). On man-made scenes it stays at 24.1 against a real-image floor of 8.2 (4,239 samples against the training images). Figure 9 shows samples on real bases at 1024 with a 256-pixel base.

Frame-covering second stage. On the same generated 64-pixel bases at 512 (five seeds, attention off in the stage networks, 90% paired intervals), a frame-covering second stage of similar parameter count scored 0.828 [0.769, 0.891] times the tiled stage's FID-50k and 0.752 [0.677, 0.835] times its native patch FID. It also differs in depth, width, conditioning, schedule and compute. The residual network's own U-Net run on the whole frame scored 0.966 [0.898, 1.039] times the tiled stage's FID-50k.

W8 (downsampling operator). We ran this ablation. Both stages were trained with a narrower and a wider Gaussian than the default (sigma 1, 5 taps; Appendix A.3). Ratios are to the default, with three paired seeds and 90% intervals:

OperatorFID-50kNative patch FID
Sigma 0.6, 5 taps0.909 [0.862, 0.958]0.810 [0.743, 0.883]
Sigma 1.6, 9 taps2.322 [2.116, 2.548]2.308 [2.100, 2.535]

The wider operator more than doubles both metrics, and the narrower one lowers them. The setting is 512 with a 128-pixel base trained for 100,000 steps and a two-level residual network without the unshuffle, not the pipeline of Table 2. This does not recommend an operator or establish robustness beyond the two tested.

AuthorsOpenReview ↗

Additional results for Reviewer xYPg: W2 and W3

W2 (statistics). The four tiled runs at 512 in Table 2 were part of the comparison that settled the tiled model's settings. To measure the result on runs that took no part in that choice, we trained eight more tiled runs at 512 with new seeds. They use the same settings and training budget, and each is evaluated at its final checkpoint against the same twelve full-frame runs. Ratios are tiled over full-frame with 95% Welch intervals on log scores, as in Table 2:

Tiled runs at 512FID-50kNative patch FID
Table 2, 4 runs0.904 [0.879, 0.930]1.198 [1.154, 1.243]
New seeds, 8 runs0.918 [0.893, 0.944]1.237 [1.193, 1.284]
All 12 runs0.914 [0.892, 0.936]1.224 [1.187, 1.262]

On the eight new runs, FD-DINOv2 and KD-DINOv2 are 0.815 [0.798, 0.832] and 0.754 [0.737, 0.772] times the full-frame model's (Table 2: 0.817 and 0.753), and median training GPU-hours are 44.3 against 43.7. The new runs reproduce the result of Table 2.

W3 (scope of the evidence). The man-made scene result in our reply used a 64-pixel base. On the same 42,388 SA-1B images, a tiled model with a 128-pixel base and a two-level residual network with the unshuffle has lower FID-50k and DINOv2 distances than the full-frame model. Both families use the 512 configurations of Table 2 with three changes. Neither uses horizontal flips. The tiled model's residual network has two levels instead of three. Its base and residual networks train for 1,030,000 and 200,000 steps instead of 730,000 and 400,000. We compare five runs per family and report ratios of tiled over full-frame with 95% intervals:

MetricRatio
FID-50k0.908 [0.880, 0.937]
Native patch FID1.214 [1.167, 1.263]
FD-DINOv20.876 [0.866, 0.887]
KD-DINOv20.846 [0.829, 0.864]

The measured training GPU-hours are 46.3 against 42.9 (medians).

Both results are in the revised PDF: Section 5.1 for W2, and Section 5.1 and Appendix H for W3.

Comments on the paper

4 comments
AuthorsOpenReview ↗

Response timeline

Thank you for the reviews. We are completing the requested experiments and will post our full response and a revised PDF by Friday, November 13.

AuthorsOpenReview ↗

General response: main results, changes since the submission, and withdrawn claims

We thank the three reviewers. The revised PDF is uploaded. We withdraw the claim that the tiled model matches a full-frame model at no FID cost; the comparison below replaces it.

Main comparison (Section 5.1, Table 2). FFHQ, both families trained on the same 63,000 images. Pairs are tiled / full-frame. Ratios are tiled over full-frame with 95% Welch intervals on log scores; below 1 favors the tiled model.

Resolution5121024
Tiled model128-pixel base, three-level residual network256-pixel base, two-level residual network
Independent runs4 / 1210 / 10
FID-50k, geometric mean18.99 / 21.0021.73 / 20.34
FID-50k ratio0.904 [0.879, 0.930]1.068 [1.021, 1.117]
Native patch FID, geometric mean20.72 / 17.3026.31 / 22.01
Native patch FID ratio1.198 [1.154, 1.243]1.195 [1.096, 1.303]
FD-DINOv2 ratio0.817 [0.794, 0.842]0.831 [0.812, 0.850]
KD-DINOv2 ratio0.753 [0.731, 0.776]0.775 [0.747, 0.804]
Training GPU-hours44.4 / 43.7264.6 / 285.5
Training FLOPs ratio0.7070.937
Model-step memory floor, GiB (BF16)0.33 / 3.130.81 / 11.96

FID-50k: clean-fid, 50,000 samples against all 70,000 FFHQ images, which include the training images; at 512, held-out FID and KID are also lower for the tiled model, 0.942 [0.924, 0.961] and 0.783 [0.741, 0.826] (Appendix A.3). Native patch FID: FID on 299-pixel crops at native resolution, from the first 10,000 samples; it measures fine detail. FD-DINOv2 and KD-DINOv2: Fréchet and kernel distances in DINOv2 features. GPU-hours: medians of measured wall-clock per run, both stages, including multi-GPU overhead.

At 512 the tiled model samples 3.3 times faster (steady state, 0.133 against 0.437 s per image), partly because the residual network takes 50 Euler steps against 100 and uses a pixel unshuffle.

Memory (Section 4, Table 1). The submission's ratios of 10.7 to 166.9 times compared one residual tile with a full frame. The revision reports the model-step floor of the whole pipeline: the larger of a one-image base step and a one-tile residual step. With a fixed 128-pixel base it stays at 0.33 GiB (BF16) at every image size measured, up to 8192. In FP32 at 2048 it is 0.65 GiB (128-pixel base) or 0.93 GiB (256-pixel base) against 59.2 GiB for a seven-level frame-covering network, about 91 and 64 times lower. Limits: the floor excludes the trainer's full-resolution data buffers; sampling memory still grows with image area; at a fixed compression ratio the base grows with the image; and with the 256-pixel base at 1024 the peak reserved training memory was 102.13 GiB against 105.14 GiB for the full-frame model (cost under a memory cap: reply to bUoa).

What changed since the submission. The submission trained on 30,000 images and measured FID on 3,500 samples; the revision reports FID-50k and native patch FID from independent runs, each evaluated at its final checkpoint. The full-frame models are attention U-Nets whose receptive field covers the frame. In this setup, tiled pipelines with a 100,000-step base, as in the submission, and a residual network without a pixel unshuffle scored FID-50k 25.00 at 512 with a 128-pixel base (19% above the full-frame model's 21.00); at 1024 they were 27% to 35% above the full-frame model. The revised tiled model trains its base longer (730,000 steps at 512, 1,440,000 at 1024); its residual network applies a 2x2 pixel unshuffle and has three levels at 512. At 512 it has 17.09M parameters against 7.60M, so the parameter counts are no longer matched. Appendix A.3 lists the other differences.

Also withdrawn or restated.

  • "No FID cost" and "matches"; level FID at equal training compute; the tiled model's lead between 7 and 15 GPU-hours.
  • FID 56.8 against 57.8: now in Appendix A.1, with the same checkpoints scored by FID-50k in Appendix A.2.
  • The diversity, low-resolution, edge and color statistics as evidence of a quality advantage.
  • The one-tile memory ratios (10.7, 41.9 and 166.9 times); that tiling is required at 2048; the crossover of sampling memory between 512 and 1024.
  • 2.06 times the baseline's GPU-hours: it applies only to the 30,000-image comparison (Appendices A.1 and E).
  • That halving the base resolution halves FID: in the setup above the ranking reverses (Section 5.3).
  • That the residual did not help in a 50k-step run at 1024; that a latent model's quality is bounded by its decoder.

Also new in the PDF. Section 6 (limitations); Appendix A.3 (training and evaluation details, held-out metrics, FLOP counting); Appendix P (uncurated samples, Figures 4 to 12).

AuthorsOpenReview ↗

Revised TL;DR

A VAE-free two-stage flow: exact sum of a low-resolution base and residual tiles. Model-step memory is set by base and tile sizes, not image size. At 512, similar GPU-hours and 29% fewer training FLOPs: 10% lower FID-50k, lower DINOv2 distances.

Area_Chair_CVsdOpenReview ↗

Reviewers, please help to join the discussion.

Dear reviewers,

The authors have uploaded the rebuttal. Please help to join the discussion and the further process.

Thanks!

Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.