- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper presents Tiled Residual Flow, which generates an image in two stages. A global flow model first produces a low-resolution image. A shared local flow model then generates residual patches to fill in what the upsampled image is missing. These patches are blended and added to the image, with no learned decoder needed. Processing patches in small batches keeps the residual network’s activation memory bounded as resolution increases.
Critical strengths
- the proposed methods can save memory, which is useful for practical usage.
Critical weaknesses
- Both the qualitative results in appendix and the quantitative results in table 2 are not attractive. Therefore, I am still concerned about the effectiveness of the propose methods.
- I think authors should provide more analysis why we need another residual flow to learn details. Currently, diffusion model is actually a model to iteratively refine a figure and therefore it is strange to introduce an additional flow to do the same thing.
Questions for authors
see weaknesses.
2 comments
Thank authors for the explanation.
For weakness 2, I am still not convinced. I expect authors to provide some theoretical analysis on this method to support the effectiveness and generalization. Besides, the qualitative results in the appendix don't make this method acceptable. This means there still exist some problems on effectiveness of the proposed method.
I highly recommend authors to provide theoretical analysis to show the effectiveness and generalization of the proposed method.
Currently, I will maintain my score.
Response to Reviewer yFHu: results and the role of the second flow
Thank you for the review.
W1 (results). At 512 the tiled model's FID-50k is now 18.99, against 49.7 for the submitted checkpoints scored with FID-50k (Appendix A.2). It is 10% lower than the full-frame model's 21.00 at about the same training GPU-hours (ratio 0.904 [0.879, 0.930]; general response). The full-frame model keeps better fine detail: native patch FID is 20.72 against 17.30 at 512, and at 1024 the tiled model's FID-50k and native patch FID are 1.068 and 1.195 times the full-frame model's. We withdraw the parity claim. For the qualitative results, Appendix P shows uncurated samples; Figures 4 and 5 show the models of Table 2 at 512 and 1024.
W2 (why a second flow). The second flow is a spatial split that bounds model-step memory, not a second refinement of the same image. The first flow generates a small image that carries the global layout. The second generates only what the upsampled base lacks, one tile at a time, and the two add exactly, with no learned decoder. No network runs on the full frame at full resolution, so this memory, excluding full-resolution data buffers, is set by the base and tile sizes and not by the image size: at 1024 the floor is 0.81 GiB against 11.96 GiB for a full-frame step (general response). It is not universally necessary: a single full-resolution flow keeps better fine detail (W1). Figure 8 shows the same generated bases without and with the second stage, and the reply to bUoa (W1 and Q1) gives the numbers; Section 5.3 and Appendix G report how far an exact base alone is from the data. The precedent is cascaded diffusion (SR3 and Cascaded Diffusion Models; Section 2), whose second stage also conditions on an upsampled low-resolution image; ours generates the exact additive residual, tile by tile. A quality advantage from predicting residual tiles over image tiles is not established (reply to bUoa, W2 and Q2).