- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
CHARM combines Hilbert-order autoregressive image generation with recursive compression of the KV cache. Completed spatial blocks are represented by learned summary tokens, maintained through an online power-of-four partition. The paper establishes a uniform Hölder-locality property of Hilbert traversal and derives (O(n^2\log n)) attention-score computation and (O(\log n)) cached entries for an (n\times n) latent grid. Experiments evaluate generation quality and saturated throughput on ImageNet at 256 and 512 resolutions, with additional continuous-token experiments.
Critical strengths
The method provides a substantial quality–throughput improvement. In Table 1, recursive compression increases CHARM-H throughput by 5.37× at 256 resolution and 12.74× at 512 resolution, while changing FID from 1.56 to 1.64 and from 1.78 to 1.87, respectively. The benchmark explicitly measures saturated generator-only throughput, and the appendix describes common hardware, precision, guidance usage, and an optimized uncompressed baseline.
The spatial partition and online cache mechanism form a coherent design. The complexity analysis includes the additional summary tokens, and Appendix A.4 explains predictor alignment and the training-time visibility pattern. Results across two model scales and resolutions support the practical usefulness of this approach.
Critical weaknesses
The experiments do not fully isolate the contribution of Hilbert ordering or its interaction with compression. The ImageNet-256 comparison rows in Table 2 match RAR’s scan-order ablation, whose raster result uses randomness annealing. Appendix A.5 does not establish whether the Hilbert experiment matches the relevant annealing, architecture, augmentation, and guidance choices. Separately, Table 2 varies order without compression, whereas Table 3 varies compression only under Hilbert order. Consequently, the specific synergy claimed in lines 471–475 remains insufficiently tested. Applying the same recursive compression to Morton/Z-order would be particularly informative because its aligned power-of-four blocks also correspond to square regions.
The interpretation of the locality analysis is stronger than the established coverage result. Proposition 1 provides a sufficient lower bound; failure to satisfy positive Hölder locality does not imply the absence of all nontrivial coverage guarantees. Moreover, Eq. 10 is zero for integer radii (r\leq3), so it does not establish high coverage at these local scales. Using the traversal in Eq. 9, for example, the coverage at (n=16,r=1,t=34) is (1/9). This does not contradict the theorem, but it illustrates why optimality of the Hölder exponent should be distinguished from superior actual coverage. This distinction matters because coverage is a central explanation offered for the generation improvements.
Questions for authors
Which Table 2 results were newly trained, and which were taken from RAR? Please specify whether randomness annealing, backbone conditioning, data augmentation, and guidance schedules were matched between the Hilbert and comparison models.
How does the same recursive summary-cache mechanism perform under Morton/Z-order, using a matched backbone and training budget? This would help distinguish the benefit of Hilbert continuity from that of hierarchical spatial compression.
Can you provide actual coverage profiles across neighborhood radii and generation progress, and clarify which coverage comparisons follow from the theory? Please also qualify the abstract’s statement about raster and zigzag lacking nontrivial guarantees.
I look forward to the rebuttal and would be open to raising my score if it resolves the concerns about matched controls and the interpretation of the coverage guarantee.
Related-work suggestion for the revision: Section 2 could more clearly distinguish complementary routes to efficient generation. Please briefly discuss AiM (Scalable Autoregressive Image Generation with Mamba) and H2O, and cite them where relevant. AiM uses a Mamba backbone for AR image generation, while H2O studies heavy-hitter and recent-token KV retention in language models. Relating these to CHARM would clarify the differences between backbone efficiency, cache selection, and hierarchical summary compression.
Thanks for the rebuttal. I maintain my positive score.