- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes QuadTok, which organizes discrete visual tokens in a quadtree to allocate representational capacity by region while retaining spatial correspondence. The encoder uses LPIPS reconstruction benefits estimated through complementary probing trees to select regions for refinement. A Kinship Causal Mask restricts information flow among tree nodes, and the decoder reconstructs hierarchically. The generator predicts visual codes in breadth-first order under a topology supplied in advance; it does not predict the tree structure itself. Validation covers ImageNet/COCO reconstruction, class-conditional generation at 256/512 resolution, component ablations, and zero-shot regional control with frozen models. The default reconstruction setting uses an average of 230 tokens and achieves ImageNet rFID of 1.46; the 947M generator achieves gFID of 2.08 (Tables 1 and 2).
Critical strengths
-
The structural design is well motivated, with direct evidence for its component benefits.
Section 3 integrates regional position, parent–child dependencies, and hierarchical residual decoding. In Table 4, using the same 697M generator and CFG, adding the mask reduces gFID from 3.17 to 2.23 and increases PSNR from 19.16 to 20.37, although rFID changes slightly from 1.41 to 1.46.
This control demonstrates the value of structural constraints more directly than rankings across different models and is a principal reason to support acceptance.
-
Regional allocation has measurable practical benefits.
The guided tree in Table 5 achieves rFID/PSNR of 1.46/20.37 with 230 tokens, close to or better than the full 320-token tree's 1.50/20.39. Table 11 compares LPIPS, MSE, and their combination at approximately 230 tokens. Table 3 additionally fixes the budget at 192 tokens, freezes the models, and uses random topologies as a control, with improved localization metrics in all four directions.
Together, reconstruction and layout-control results support the usefulness of explicit spatial representations beyond generation FID alone.
Critical weaknesses
-
Spatial-control validation mainly supports coarse positional shifts, and the magnitude of control remains insufficiently characterized.
Table 3 and Appendix F evaluate four half-planes—left, right, top, and bottom—and compare prescribed and random topologies at the same 192-token budget. This is a targeted control. However, Box hit rates improve by only 4.3–7.1 percentage points and Mask rates by 4.9–7.9 points, with some absolute hit rates still below 50%.
Hit rates establish that more subjects enter the target half-plane, but do not distinguish substantial movement from small boundary crossings by samples already near the boundary, or directly establish localization accuracy for arbitrary target regions. Since missed detections count as failures, 50% should also not be interpreted as a universal chance level here.
The evidence supports using topology to steer subject position, but the magnitude and applicable granularity of this spatial control should be clearer. Appendix H's discussion of multi-object limitations does not replace analysis of the current single-subject setting.
-
Dependencies among localization metrics and stability across classes require further examination.
Grad-CAM gains in Table 3 exceed Box/Mask gains, but the random-topology baseline already has pronounced directional biases: 29.6%/70.4% for left/right and 33.9%/66.1% for top/bottom. This suggests that the values may depend on the classifier's response structure, and their agreement with actual changes in subject position needs further explanation.
SAM 2 also uses Grounding DINO boxes as prompts, so Box and Mask share a localization source and cannot be treated as fully independent validations. Appendix F covers 50 classes with 20 images per class per condition, totaling 1,000 images. Class-level gains, detection-success breakdowns, and uncertainty are not reported, leaving it unclear whether improvements extend broadly across classes or are concentrated in some readily detected classes.
This gap affects the robustness and generalizability of the spatial-control results.
-
Generation quality and efficiency have not yet been connected in a directly comparable joint evaluation.
Appendix E on pages 17–18 provides timing on the same hardware at matched model scales, including absolute latency with decoding, and Figure 4 on page 6 shows the quality cost of changing token budgets. These are valuable results. However, timing uses randomly initialized LlamaGen weights, the 775M QuadTok model is used only for timing, and the XL quality results in Table 2 come from a 697M model. These comparisons do not directly establish acceleration at matched quality.
Gains at nearly equal token budgets are only about 1.09–1.20×, which the paper also attributes to implementation differences. Larger speedups mainly occur with fewer tokens and should be paired with the corresponding quality points. Table 10 further shows slower adaptive pretokenization. Although this cost can be amortized through caching and generation requires no image-dependent search, the benefit depends on the use case.
The paper should therefore clarify the quality levels and generation volumes at which variable representational capacity translates into actual time savings, rather than reporting best quality and speed at specified budgets separately.
Questions for authors
-
Control magnitude and scope of the conclusions:
Using the existing generated images from Table 3, could you report distributions of Box centers or Mask centroids along the target direction and displacement relative to random topologies, in addition to half-plane hit rates?
Do the improvements mainly reflect substantial subject movement toward the target region or changes in hits for samples near the boundary?
Please use these results to explain the control granularity supported by the evidence and distinguish the conclusions of the prescribed-half-plane experiments from finer layout control.
-
Metric reliability and stability across classes:
Could you provide class-level breakdowns of the Box/Mask gains in Table 3, uncertainty estimates obtained by resampling classes, and detection-success rates for each condition?
Do the larger Grad-CAM gains agree with Box/Mask positional changes, or are they concentrated in a small number of classes?
Given SAM 2's dependence on DINO boxes, please explain how shared detection errors affect interpretation of the two metrics.
-
Paired validation of quality and efficiency:
What topology-expansion probabilities, actual mean token counts, and CFG parameters are used by each QuadTok model in Table 2?
Could you pair the existing quality points in Figure 4 with generation-plus-decoding latency for the corresponding checkpoints and compare against LlamaGen configurations with quality results, identifying which quality levels actually yield speedups?
Please explain the comparability of the 775M timing model and the 697M quality model, and distinguish the one-time adaptive pretokenization cost from subsequent generation costs.
-
Comparison with RAR, PAR, FAR, and BAR:
Beyond the existing results for VAR, xAR, FlowAR, and other methods, could you add experimental comparisons and methodological discussion of RAR (Q. Yu et al., 2025), PAR (Wang et al., 2025), FAR (H. Yu et al., 2025), and BAR (Tang et al., 2026)? Would like to see additional ImageNet comparisons with all four methods in gFID, parameter count, token count, sampling rounds, and measured generation-plus-decoding latency or throughput. Even if time constraints prevent additional experiments, I would still appreciate a detailed comparative discussion of the methods above.
5 comments
W2/Q2. Metric reliability and class stability.
Thank you for these questions. Because 20 images per class give wide per-direction intervals, we regenerated 200 images per class and condition with new seeds (10,000 per condition) and re-evaluated all three metrics; we will update Table 3 with these results.
| Target | Grad-CAM | Box | Mask |
|---|---|---|---|
| Left | +28.7 [24.4, 32.8] | +5.6 [3.5, 7.6] | +5.8 [3.8, 7.7] |
| Right | +18.4 [15.8, 20.9] | +6.2 [4.2, 8.1] | +6.2 [4.3, 8.1] |
| Top | +26.6 [22.7, 30.4] | +5.3 [3.3, 7.4] | +5.2 [3.1, 7.2] |
| Bottom | +20.2 [17.2, 23.2] | +6.4 [4.5, 8.4] | +6.2 [4.3, 8.2] |
| Pooled | +23.5 [20.9, 25.7] | +5.9 [5.0, 6.7] | +5.8 [4.9, 6.7] |
Every interval excludes zero, including Left Box and Top Mask, whose intervals on the original 20 images per class included zero. The pooled gains agree with those of the original images (+21.3, +5.8, and +6.5 pp).
Class stability. Out of 50 classes, gains are positive in 46–48 per direction for Grad-CAM, 39–44 for Box, and 42–45 for Mask. If the Box or Mask gain were confined to half of the classes, about 37.5 classes per direction would show a positive gain; summed over the four directions, we observe 169/200 for Box and 175/200 for Mask, against about 150 under that alternative. The gains are therefore not confined to a few classes. Grounding DINO detects the target in 99.8% of the new images, so no class is poorly detected, and the gains cannot come from a subset of readily detected classes. We will add per-class gains and per-condition detection counts to the appendix.
Agreement between metrics. All three metrics move toward the target in every direction, by different amounts, because they locate different things. Grad-CAM locates class-discriminative evidence, which refinement affects most directly; Box and Mask locate the detected object, whose global position is set by the coarse codes and so moves less (W1). We therefore claim agreement in the direction of the change, not in its size, and will describe Grad-CAM in Table 3 as locating discriminative evidence rather than the subject.
Grad-CAM baseline asymmetry. The random-topology split (29.6/70.4 and 33.9/66.1 in Table 3; 27.5/72.5 and 32.4/67.6 on the new images) reflects where the center of mass of ResNet-50's 7×7 CAM falls relative to the midline. On the original images, counting the center column of the CAM grid as left, instead of splitting it at its center, flips the left/right split to 68/32, yet the gains stay between +17.5 and +31.3 pp in every direction. Each gain compares prescribed and random topologies under the same classifier, so a fixed classifier bias enters both terms; we therefore interpret Grad-CAM through its gains, not its absolute hit rates.
Box and Mask share one pipeline. SAM 2 is prompted with the Grounding DINO box, so a wrong box propagates to the mask. Box and Mask are therefore two summaries of one localization pipeline, not independent confirmations, and we will revise the paper accordingly to avoid overclaiming. Grad-CAM, in contrast, comes from a separate model, an ImageNet-trained ResNet-50 classifier, and shares no component with the detection pipeline. Because two independently built localizers both show the shift toward the target in every direction, their agreement is a valid cross-check of the direction of the effect.
W3/Q3. Paired quality and efficiency. The main results are in Part 2 of the General Response. Every two-level row of Table 2 uses p = 0.75 (256 tokens in expectation; 255.0 measured), and the three-level row uses p1 = 0.95 and p2 = 0.70 (about 988 tokens); all rows use the power-cosine CFG schedule rising from 1 to 20 (mean 10.5). That part pairs each quality point with generation-plus-decoding latency against LlamaGen at the same scale, and explains how the 775M timing model relates to the 697M quality model. QuadTok is both better and faster than LlamaGen at every scale; at matched quality, QuadTok-L at p = 0.5 is 2.02× faster than LlamaGen-XL (1.50× from the smaller model, 1.35× from fewer tokens).
One-time vs. per-sample cost. Adaptive pretokenization costs 3.3 ms more per image than LlamaGen's VQ (6.19 vs. 2.89 ms), about 70 A100-minutes per pass over ImageNet-1K. It is paid once when codes are cached for training. Sampling under a supplied or predicted tree needs no probe. Against LlamaGen-XL at matched quality, the one-time cost is recovered after about 21K generated images.
Q4. RAR, PAR, FAR, and BAR. As discussed in Part 1 of the General Response, these designs are orthogonal to QuadTok. They change how a fixed token set is ordered, parallelized, or transformed, whereas QuadTok changes the token set itself, so each can be applied on top of QuadTok's token sequence. We deliberately keep a plain tokenizer and a plain AR generator so that the gains over LlamaGen can be attributed to spatial adaptivity. We give method-specific details below.
- RAR (Yu et al., 2025) trains a raster-order generator on randomly permuted orders, annealed back to raster order, over a fixed 16×16 grid of discrete tokens.
- PAR (Wang et al., 2025) splits the token grid into regions, generates one initial token per region sequentially, and then generates the tokens at the same position in different regions in parallel (4 or 16 per step). This reduces LlamaGen's 576 steps (24×24 tokens) to 147 or 51.
- FAR (Yu et al., 2025) generates continuous tokens from low to high frequency; each of its 10 steps predicts a full-image frequency level with a diffusion head.
- BAR (Tang et al., 2026) treats tokens as projections onto basis vectors and learns an orthogonal linear transform of the token sequence end to end, on top of MAR and xAR with continuous tokens.
RAR and PAR keep a uniform spatial grid and change the order or the number of rounds. FAR and BAR are coarse-to-fine in frequency or in a learned basis, where each token describes the whole image. QuadTok is coarse-to-fine in space and gives more tokens to selected regions. Each method maps onto QuadTok's sequence: RAR-style order randomization within each BFS level, PAR-style parallel decoding of children of different parents (which interact only through their parents in the tokenizer), and frequency or basis reordering within the tokens of each level.
For reference, published ImageNet 256×256 results are listed below, followed by LlamaGen and QuadTok timed by us under one protocol. The tokenizers, token counts, latent types, and speed protocols of the published methods differ, so those rows are system-level references, not matched comparisons:
| Method | Tokens | Params | Steps | gFID | Reported speed |
|---|---|---|---|---|---|
| RAR-L / XL | 256, discrete | 461M / 955M | 256 | 1.70 / 1.50 | – / 8.3 img/s (A100, FP32, batch 128) |
| PAR-L-4× / XL-4× | 576, discrete | 343M / 775M | 147 | 3.76 / 2.61 | 3.38 / 4.94 s per image (A100, batch 1) |
| PAR-XXL-16× | 576, discrete | 1.4B | 51 | 3.02 | 2.28 s per image |
| FAR-L / H | continuous | 427M / 812M | 10 | 3.45 / 3.21 | not reported |
| BAR-H (on xAR-H) | continuous | 1.1B | – | 1.15 | not reported |
| LlamaGen-L / XL | 256, discrete | 343M / 775M | 256 | 3.80 / 3.39 | 8.51 / 12.72 s per batch of 32 (A100, BF16, CFG) |
| QuadTok-L, p = 0.75 (0.5) | 256 (192), discrete | 343M | 256 (192) | 2.93 (3.25) | 8.50 (6.30) s per batch of 32 (A100, BF16, CFG) |
| QuadTok-XL, p = 0.75 (0.5) | 256 (192), discrete | 697M | 256 (192) | 2.23 (3.01) | 10.36 (7.65) s per batch of 32 |
Scope. As stated in the Scope paragraph of the General Response, all four methods are orthogonal to QuadTok: they change how a token sequence is generated, not which tokens represent the image, so their techniques transfer to QuadTok's sequence as described above and could further improve its results. The controlled comparison is QuadTok versus LlamaGen, which shares our codebook, architecture, and timing protocol (last rows): QuadTok improves gFID at both scales and, at p = 0.5, is still better while 1.35× faster at L. We will add the four methods to the related work with this discussion.
Response to Reviewer mHoT
We thank the reviewer for the detailed requests. We analyzed the per-image data behind Table 3 (8,000 images: 8 conditions × 50 classes × 20 samples). CIs are 95% two-stage bootstrap intervals that resample classes and then images within each class.
W1/Q1. Magnitude and form of the shift. In all four directions, the box-center distribution shifts toward the target (KS p < 0.05), not just samples near the boundary; the mean shift is significant in three of them. Mean signed displacement of the box center into the target half (px on 256-px images; random → prescribed):
In three of four directions, most gained Box hits land at least one coarse cell (32 px) inside the target half. Two factors cap the Box/Mask hit rate. First, the topology does not change the coarse codes, which set the global layout (response to 6i7B, W1), so it moves box centers only through the fine tokens: 3–6 px on average, i.e., 1–2% of the image width. Second, 29% of box centers lie within 1 px of the vertical midline, because these objects span nearly the full width, so their box center cannot move.
With the coarse codes held fixed (paired experiment in our response to 6i7B, W1), box centers still move 5.3 px [4.2, 6.4] toward the target, matching the unpaired estimate above, and 8.8% of pairs move by at least one coarse cell toward it, while the fine level changes about a third of the subject mask (IoU 0.65). The topology thus places detail and, to a smaller extent, structure in the target half.
These results support half-plane control of where detail goes, with a smaller effect on the subject's extent, not placement of the subject in arbitrary target regions. We also acknowledge that this is not a standard layout-to-image evaluation. QuadTok is trained class-conditionally on ImageNet, which has no layout annotations, so we built a proxy: a prescribed topology refines the target half, and off-the-shelf Grad-CAM, Grounding DINO, and SAM 2 measure where the subject lands. The proxy is meant to test whether the topology carries a usable control signal without any layout training, not to benchmark layout-to-image generation. Within this scope, the evidence is consistent: in the re-evaluation with 10,000 images per condition (W2), every metric improves in every direction with intervals that exclude zero. We therefore believe these results show that the topology enables coarse, zero-shot control of where detail goes, and we will revise Sec. 4.3 and the captions of Fig. 1(c) and Fig. 5 to give a precise statement.