acceptodds
Under review as a conference paper at ICLR 2027

ScanAR: Parallel Row-Scan Decoding for Efficient Image Generation in Unified Autoregressive Models

Abstract

Unified multimodal autoregressive (Unified AR) models represent text and images in a shared token sequence and generate all modalities with a common next-token prediction objective. However, this raster-order decoding requires \(HW\) sequential decisions for an \(H \times W\) image, overlooking the inherent two-dimensional structure of visual tokens and creating a major inference bottleneck. To address this problem, we propose ScanAR, a row-scan parallel autoregressive method for efficient image generation within Unified AR models. Instead of modifying the generation head or introducing additional parameters, ScanAR accelerates decoding by redefining the predecessors of visual tokens, enabling seamless integration with existing Unified AR models. Specifically, the first visual row follows the standard left-to-right dependency, while each subsequent row is generated in parallel by conditioning each token on its vertically aligned predecessor in the previous rows. For nonvisual tokens, the standard autoregressive rule is unchanged. In this way, ScanAR reduces the sequential decoding from \(HW\) to \(H+W\), supporting both training from scratch and lightweight post-training without modifying the backbone, vocabulary, or output head. Extensive experiments on Emu3.5 demonstrate that ScanAR improves the GenEval score by 1.88 points and achieves a \(27.7\times\) speedup for \(512\times512\) image generation through post-training. On U0, ScanAR accelerates image generation while retaining its interleaved and video generation capabilities.

Primary area:
generative models
Keywords:
Unified multimodal autoregressive model, model generation acceleration

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_eBhqOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ScanAR adapts unified autoregressive models to generate image tokens row by row. The first row is generated sequentially, while subsequent rows are predicted in parallel using the preceding row’s hidden states. The method retains the backbone, vocabulary, and output head, and supports both post-training and training from scratch. Experiments cover Emu3.5 text-to-image generation, ImageNet class-conditional generation, and U0 multimodal generation.

Critical strengths
  1. The method introduces substantial parallelism without adding prediction heads or changing the tokenizer. Keeping the same output interface is useful for models that alternate between text and images, and the paper explains how training and inference handle the first row consistently.

  2. The Emu3.5 results show a substantial practical gain: generation time falls by a lot, while GenEval increases. The vLLM experiment also reports higher throughput and lower peak memory than FlashAR.

Critical weaknesses
  1. Preserving unified multimodal capabilities is the main distinction from prior acceleration methods, but the quantitative evidence is narrow. U0 is evaluated on embodied-scene transfer using one automated judge; interleaved generation and video are supported mainly by examples. This does not establish how reliably the adapted model preserves text quality, modality transitions, or temporal consistency.

  2. The ImageNet results show a more mixed quality-speed tradeoff than the main text suggests. At XXL scale, ScanAR reaches FID 3.46 vs FlashAR’s 2.79, with only a small throughput advantage, 65.3 vs 63.4 images/s. ScanAR also trains from scratch for 300 epochs, whereas FlashAR adapts a pretrained model for 25 epochs. The emphasis on the best Inception Score obscures these differences.

Questions for authors
  1. How many examples were used for the U0 evaluation, and what is the variability of the judge scores? Were missing views or incomplete outputs present, given that the evaluation prompt does not penalize mismatched view counts?

  2. Can you provide quantitative comparisons for interleaved text-image and video generation, including output validity and quality before and after adaptation? These would directly support the paper’s main distinction from image-only acceleration methods.

8 comments

We thank the reviewer for the positive assessment. In particular, the reviewer noted that keeping the same output interface is useful for models that alternate between text and images, and that the paper explains how training and inference handle the first row consistently.

W1 / Q2: Quantitative evidence for unified capabilities; interleaved and video.

Preserving unified capability is guaranteed by design. ScanAR leaves the text and structural-token path unchanged: causal predecessors, causal attention, and the same head (Eq. 3). After each image, decoding returns to standard causal mode over the same cache (Sec. 3.4). Modality switching is therefore performed by the original mechanism.

Interleaved generation: paired quantitative comparison. We compare the ScanAR-adapted U0 with the official raster-AR Xiaomi-Robotics-U0-Sequence model on 80 embodied interleaved subtask-planning cases from 9 robot data sources. Each case gives a task instruction and initial observations; the model must write the subtask plan and generate the observation images for each step (13.8 images per case on average). Both models run in the same inference engine with identical sampling settings and seeds, so the only difference in how images are produced is raster vs. row-scan visual decoding. All metrics were fixed before generation and are reported in full (mean over cases; Δ = paired ScanAR − raster with 95% bootstrap CI):

MetricRaster U0-SeqScanARΔ [95% CI]
Valid output (finished, all images decoded) ↑81.3%91.3%+10.0 pp [+1.3, +20.0]
Number of steps = requested ↑100%98.8%−1.3 pp [−3.8, 0.0]
Correct text/image alternation per step ↑100%98.8%−1.3 pp [−3.8, 0.0]
Image fidelity to GT, DINOv2 cosine ↑0.6700.654−0.017 [−0.040, +0.007]
Image fidelity to GT, LPIPS ↓0.3970.408+0.011 [−0.004, +0.027]
Image fidelity to GT, PSNR ↑15.1315.06−0.07 [−0.37, +0.23]
Scene consistency with initial observation, DINOv2 ↑0.6160.577−0.039 [−0.061, −0.018]
Temporal consistency between adjacent steps, DINOv2 ↑0.6900.672−0.019 [−0.037, +0.000]
Subtask-text exact match with GT ↑53.6%80.1%+26.4 pp [+17.8, +35.3]
Generation time per image ↓57.3 s5.2 s11.1× faster end-to-end

The structure of the interleaved output is preserved: ScanAR finishes the sequence and alternates text and images as reliably as the raster model, with differences of at most 1.3 points in step count and alternation. Image fidelity to the ground-truth observations is statistically indistinguishable on all three metrics. Scene and temporal consistency are slightly lower (by 0.04 and 0.02 in DINOv2 cosine), consistent with the small consistency gap on WorldArena below, while the full interleaved sequence, including text, is generated 11.1× faster. Since ScanAR does not change the text path, we attribute the difference in subtask-text accuracy to the post-training data rather than to the decoding scheme, and we do not claim it as a benefit of ScanAR.

Video: quantitative results on the public WorldArena benchmark. We evaluate video generation on the WorldArena RobotWin 2.0 test set (1,000 episodes), using the official protocol with 11 metrics that cover visual quality (aesthetic and image quality), temporal consistency (subject and background consistency, motion smoothness, photometric consistency), motion (dynamic degree, flow score), and VLM-judged task metrics (instruction following, interaction quality, perspectivity). We compare the ScanAR-adapted U0 with the raster U0 video model from which it is initialized, on the same episodes and prompts:

MetricRaster U0ScanAR
Aesthetic Quality41.5041.00
Image Quality54.1653.16
Subject Consistency79.0877.32
Background Consistency86.2184.95
Motion Smoothness95.2494.43
Photometric Consistency1.762.60
Dynamic Degree73.0767.58
Flow Score82.3874.97
Instruction Following80.2083.12
Interaction Quality74.2877.40
Perspectivity94.6296.32
Overall69.3268.44

ScanAR stays within 0.9 points of the raster model in overall score, while each 28×37-token frame is decoded in 65 sequential steps instead of 1,036 (16× fewer). It scores higher on all three task-level metrics judged by the VLM (instruction following, interaction quality, perspectivity), and its visual-quality and consistency metrics are within about 1–2 points of the raster model. The main difference is a lower amount of motion (dynamic degree and flow score). These results show that ScanAR preserves the video-generation capability of the unified model on a public benchmark.

Q1: U0 evaluation size, score variability, missing views.

Evaluation size. Table 2 is computed on 300 paired prompts: 150 single-dimension and 150 multi-dimension scene edits (2–5 changed attributes). The prompts come from five embodied sources (AgiBotWorld-Beta, GenieSim, Intern-A1, mibot, OXE). Both models are evaluated on the identical prompt set, with the same judge and the same prompt template (Appendix B.1).

Variability. Per-sample scores (mean ± std over samples):

DepthBackgroundForegroundLightingImage QualityMulti-ViewOverall
U07.44 ± 1.007.60 ± 1.397.27 ± 1.427.50 ± 1.638.05 ± 0.577.52 ± 1.147.52 ± 0.80
ScanAR7.50 ± 0.917.36 ± 1.807.21 ± 1.427.50 ± 1.687.76 ± 0.717.60 ± 1.027.49 ± 0.84

Because both models share the same prompts, we also report the paired per-prompt difference in the overall score: −0.03, 95% bootstrap CI [−0.13, +0.06]. The interval contains zero, so the overall quality of the two models is statistically indistinguishable. ScanAR is numerically higher on Depth Faithfulness (+0.06) and Multi-View Consistency (+0.08), the two dimensions that most directly test conditional geometric control.

Missing or incomplete outputs. Every one of the 300 prompts produced a valid output for both models. In all 600 judge explanations, no view-count mismatch between depth maps and generated views is reported, and the single-view fallback rule (multi-view consistency = 10) is never triggered. The special rule in our prompt therefore had no effect on any score.

W2: ImageNet quality-speed trade-off at XXL.

The purpose of the ImageNet experiment is to show that ScanAR can be fitted both by training from scratch (LlamaGen, four scales) and by post-training an existing model (Emu3.5, U0). Its comparison point is therefore the from-scratch methods with the same 300-epoch budget. Against these, ScanAR achieves the highest IS at every size and FID close to raster LlamaGen, with 8× fewer sequential steps. FlashAR uses a larger total budget (300 pretraining epochs plus 25 post-training epochs) and starts from a converged raster model.

We will present FID first in the main table, as suggested.

ScanAR's main claim concerns post-training of unified models. There, under the identical FlashAR protocol, ScanAR is better than FlashAR on both quality (GenEval 82.36 vs. 80.29) and latency (4.69 s vs. 5.68 s).

Reviewer_eBhqOpenReview ↗

Response to rebuttals

Thanks for adding the video and text-image tests. The U0 details also help answer my questions. The new results are useful, but scene consistency and some video motion scores still go down. Keeping the same output setup does not mean all abilities stay the same. Please explain these drops and show the ImageNet FID results clearly. I will keep my score at 3.

Reviewer_JNWFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this work, the authors propose a new method named ScanAR to improve the inference efficiency in a unified autoregressive system. To achieve higher inference speed for visual tokens (keeping the decoding of textual tokens unchanged), ScanAR first autoregressively generates each token in the first row, and then all tokens in a row sequentially. By doing so, ScanAR reduces the decoding steps from H*W to H+W, achieving significant speed improvement. Besides speed, extensive experiments also demonstrate good performance.

Critical strengths
  1. I do appreciate the authors’ good presentation. While lots of papers now create nonsense terms, making the submissions hard to read, I can see this submission has a good presentation and is well organized.

  2. Extensive experiments are conducted.

  3. It is an interesting topic: how to better deal with 2D images in the framework of next-token prediction.

Critical weaknesses
  1. I can see the comparison is not fair. The authors claim ScanAR can improve EMU3.5 by 1.88 in the abstract, as shown in Table 4. That is not fair. The authors should also finetune EMU3.5 on the same datasets for the same number of iterations. From the current comparison, we cannot tell whether the improvement comes from the scan strategy or the additional training data. I would consider the benefits to be from the dataset, and the reason can be found in Table 5. The scanning strategy cannot help the counting / position ability, while we can see the main improvement comes from counting / position.

  2. Missing important baselines like Infinity [1].

  • [1] Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesis
  1. I did not see a good motivation or mathematical proof that this is a good scanning strategy. Otherwise, we can come up with arbitrary scanning method, and just need to prove it can work without good motivation behind.
Questions for authors
  1. WHat is the motivation for KL loss in Eq (5)? From my understanding, directly SFT the original base model to ScanAR also works and can adapt quickly. I did not see any ablation for KL loss in the submission.

  2. I would not consider Table 1 is a good summary, in terms of these collumns: "No extra parameters", "Raster-order progression", "Support unified output". Taking VAR as an example. VAR is not based on causal AR, hence "No extra parameters" is not suitable. "Raster-order progression" is not well-defined. In terms of from low-resolution to high resolution, VAR can also be considered as raster-order. VAR also has the ability of "Supporting unified output" (even VAR did not implement it).

  3. What if we scale the image resolution to 1k? Will there be some performance degration or hard to be trained?

6 comments

We thank the reviewer for appreciating the presentation and organization of the paper and the extensiveness of the experiments.

W1: Is the GenEval gain from the post-training data?

We would like to clarify what the paper claims. ScanAR is an acceleration method, and its quality should be compared with other post-training acceleration methods under the same data, not with continued raster training. Under the FlashAR protocol, BlockDiffusion, FlashAR, and ScanAR are all post-trained on the same 80M-token data (Table 4), and the GenEval result in the paper serves exactly this comparison: ScanAR obtains 82.36, versus 80.29 for FlashAR and 73.83 for BlockDiffusion, with the lowest latency. Since the data is shared, this difference among acceleration methods cannot be attributed to the data. Keeping the quality of the original model is a by-product.

As a reference, a raster fine-tune with exactly our data, initialization, and schedule reaches 81.23 (2K steps), and ScanAR reaches 81.76 (2K steps). ScanAR therefore keeps the quality of a raster model trained on identical data, while BlockDiffusion and FlashAR fall below even the original model.

Model (same 80M-token post-training data)Sequential steps per 512² imageGenEval ↑
Emu3.5-Image (baseline, no post-training)1,02480.48
Raster fine-tune (2K steps)1,02481.23
ScanAR (2K steps)6481.76
BlockDiffusion (50K steps)6473.83
FlashAR (50K steps)6380.29
ScanAR (20K steps)6482.36

Row-parallel decoding also does not hurt the relational categories: ScanAR's Counting (69.06) and Position (80.00) are on par with or above those of the matched raster fine-tune (66.50 and 77.95, 5-seed means). Counting and relative position are determined by the global layout of the image. In ScanAR, this global layout, which is carried by all previously generated rows, stays fully visible exactly as in raster AR. The only context ScanAR gives up is the few left tokens of the current row (at most W − 1). These are local neighbours and have almost no influence on how many objects there are or where they are placed. In addition, each committed row is computed with full within-row attention (Sec. 3.2), so later rows condition on states that encode complete rows.

W2: Infinity.

Infinity generates an image by coarse-to-fine next-scale prediction, the paradigm introduced by VAR, rather than by next-token prediction. Its bitwise tokenizer, infinite-vocabulary classifier, and bitwise self-correction are built within this next-scale framework. This paradigm is already covered in the paper through VAR (Table 1 and Table 7). We will add Infinity to the related work next to VAR (Sec. 2.2).

For quality, the table below compares the GenEval scores reported by Infinity (Han et al., 2025, Table 1) with ScanAR on Emu3.5-Image (Table 5).

MethodPrompt rewritingTwo Obj.PositionColor Attr.Overall
Infinity-2B✓85.0049.0057.0073.00
ScanAR (Emu3.5-Image)×87.8880.0068.0082.36

ScanAR is higher on every category Infinity reports, and on the overall score by 9.36 points. This comparison is not fully controlled, and the differences cut both ways. Infinity's scores use prompt rewriting, while ours use the raw GenEval prompts. Infinity is trained from scratch specifically for text-to-image generation, with about 1.19B image–text training samples, whereas ScanAR post-trains on about 80K text–image pairs. On the other hand, Infinity-2B is much smaller than the 34B Emu3.5-Image base.

W3: Motivation for this scan order.

ScanAR's order is not arbitrary. It is the most conservative change to raster AR that allows a full row to be generated in parallel:

  1. Every conditioning token is still a raster predecessor. For token (i, j), ScanAR conditions on all rows < i. All of these tokens precede (i, j) in raster order. ScanAR's context is therefore a subset of raster AR's context, and the only tokens dropped are the left neighbours in the same row. Token order, spatial indices, and position ids are unchanged. This is why a pretrained raster checkpoint can be adapted with light post-training (20K steps on Emu3.5).
  2. The predecessor is the nearest fully committed state. At row i, the closest token whose state has seen a complete row is the token directly above, at (i−1, j). Using it as the predecessor keeps each column spatially aligned with its prediction source (Eq. 2).
  3. The output interface is unchanged. The same head predicts text and both causal and row-scan visual tokens. No position-query tokens or extra heads are needed. Arbitrary orders such as random-order AR require such additions (e.g., RandAR in Table 1).

Under these three constraints, which are all required for unified AR, the row-scan order follows naturally. It reaches a depth of H+W, compared with HW for raster AR.

Q1: Motivation for the KL term; would plain SFT work?

We agree that plain SFT can adapt the model, and the CE term is indeed active throughout training. The KL term serves a different purpose: it lets an existing raster model fit the row-scan prediction mode quickly. At the start of post-training, the row-scan predecessor states are new to a model trained with raster predecessors. A frozen raster teacher gives a dense, full-distribution target for these states, which carries much more information per position than a one-hot CE label. This pulls the new predictions towards the pretrained model's distribution in a few steps. Its fast-fitting role is why the term is used only at the start of post-training. When training from scratch there is no pretrained distribution to fit, so no KL is used, and the ImageNet results confirm that the objective is learnable without it. We conduct Emu3.5-Image post-train distillation ablation for 2000 steps to show the effect, the results in GenEval show that distillation helps model fit faster.

ModelSingle Obj.Two Obj.CountingColorsPositionColor Attr.Overall
ScanAR w/o Distillation (2K steps)98.1286.8766.2591.7677.2564.7580.83
ScanAR w/ Distillation (2K steps)98.7587.3771.2590.1677.5065.5081.76

Q2: Table 1 columns and VAR.

The columns describe compatibility with an existing unified raster AR model, which is the setting of the paper. We will make the definitions explicit in the caption.

  • No extra parameters: no parameters added on top of the raster AR model being accelerated. VAR needs a multi-scale tokenizer and scale-level conditioning, and it cannot reuse a raster checkpoint.
  • Raster-order progression: tokens are produced over the same H×W grid in raster index order. VAR's coarse-to-fine progression runs over different token maps at different resolutions, which is a different notion. We will rename the column to "Preserves raster token grid and order" to avoid ambiguity.
  • Support unified output: compatibility with a shared next-token interface over text and image. VAR replaces next-token prediction with next-scale prediction. Combining it with text generation would require a new formulation, and this has not been shown.

Q3: Scaling to 1K resolution.

Our experiments already show that ScanAR keeps quality as the resolution grows:

  • 256² (16×16 tokens): on ImageNet with LlamaGen, ScanAR's FID is within 0.06–0.37 of raster AR, and better at XL (Table 7).
  • 512² (32×32 tokens): on Emu3.5 with GenEval, ScanAR exceeds the raster model (82.36 vs. 80.48; Table 5).
  • 1K (≈1 megapixel, 4,096 visual tokens): the U0 embodied-scene transfer benchmark of Table 2 is generated at about one megapixel per output (e.g., 2048×512 multi-view canvases, i.e. 128×32 = 4,096 visual tokens, the same token count as a 1024² image). There, ScanAR is statistically indistinguishable from the original U0 under GPT-5.4 scoring (7.49 vs. 7.52; paired difference −0.03, 95% CI [−0.13, +0.06]).

Across a 16× increase in the number of visual tokens (256 → 4,096), ScanAR's quality relative to raster AR does not degrade.

ScanAR also already runs at about 1K resolution. Our U0 experiments use roughly 1024² images (e.g., 52×78 token grids), with a mean ScanAR generation latency of 17.4 s per image. We also post-trained Emu3.5 on 1024² data (64×64 tokens), where ScanAR's mean latency is 18.4–19.3 s. The relative benefit grows with resolution: at 64×64 tokens, ScanAR needs 128 sequential steps, compared with 4,096 for raster AR (32× fewer).

Training also scales. With FlexAttention, a 16K-token ScanAR step takes about 47.1 s, compared with about 44.0 s for causal AR on the same 8 GPUs.

Our comparison with other acceleration methods is at 512², the resolution of the FlashAR protocol, where published raster, BlockDiffusion, and FlashAR numbers exist. At 1K, no published acceleration baselines exist, so the reference is the original model itself (Table 2 above). Qualitative 1K results from the adapted U0 model are shown in Figures 6 and 7.

Reviewer_RqgQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes ScanAR, a row-scan decoding scheme for unified autoregressive models that generate both text and image tokens. The first visual row is decoded left-to-right, while each subsequent row is generated in a single parallel step, with every token conditioned on the hidden state of the token directly above it. This reduces the decoding depth for an H×WH \times W visual grid from to H+WH+W. The method requires no changes to the backbone, vocabulary, or output head, and is implemented through a modified attention mask, a joint training objective, and optional distillation from a raster-AR teacher. The approach can be applied either through lightweight post-training or by training a model from scratch.

Critical strengths

The proposed row-scan dependency structure is easy to understand, requires no architectural modifications, and preserves the unified autoregressive formulation. The method naturally applies to visual segments within interleaved multimodal sequences and supports both post-training adaptation and training from scratch.

Critical weaknesses
  • The paper does not sufficiently position itself relative to several highly relevant directions in accelerated autoregressive image generation. In particular, ZipAR is not discussed despite sharing many of the properties highlighted in Table 1, including parameter-free acceleration and applicability to unified models. More broadly, comparisons against methods such as LPD, ARPG, and recent speculative decoding approaches are either missing or incomplete. The novelty claim would therefore benefit from a more comprehensive discussion of related work and a stronger justification of what ScanAR uniquely contributes (L44-50 in introduction is hard to understand).

  • While ScanAR achieves impressive speedups, the generation-quality results are less conclusive. On ImageNet, the main paper emphasizes Inception Score, whereas FID, precision, and recall are reported only in the Appendix. There, ScanAR underperforms the raster LlamaGen baseline on FID for three of four model sizes. On GenEval, most of the reported improvement comes from large gains in Counting and Position, while other categories remain unchanged or decline. Furthermore, important controls are missing, including a raster Emu3.5 baseline fine-tuned under the same post-training setup. The evidence supporting preservation of unified capabilities is also limited, relying primarily on qualitative examples and a single GPT-5.4 based evaluation, without standard text or multimodal understanding benchmarks.

Questions for authors
  • Why is ZipAR not cited or compared? How does ScanAR differ conceptually and empirically, particularly regarding within-row dependencies, training requirements, and applicability to unified models?
  • Please include FID, precision, and recall in the main paper. Why does ScanAR perform worse than the raster LlamaGen baseline on FID for three of the four model sizes?
  • Can you include a raster Emu3.5 model fine-tuned using the same data, number of training steps, and distillation schedule used for ScanAR? This would help isolate the effect of the decoding strategy from the effect of post-training.
  • Can you evaluate the post-trained model on a standard understanding or text benchmark, as well as an additional text-to-image benchmark (e.g., DPG, T2I-CompBench, or human evaluation), to support the claim that unified capabilities are preserved?
  • The reported 27.7×27.7 \times speedup exceeds the 16×16 \times reduction in decoding steps (1024 to 64). Could you explain where the additional speedup comes from?
  • Why do the relative throughput gains diminish as model size increases (e.g., compared with FlashAR)? Does this indicate that the benefits are increasingly dominated by system-level optimizations rather than algorithmic improvements?
  • What accounts for the memory reduction relative to FlashAR (138.8 GB vs. 110.7 GB)? Which components contribute most to the savings?
  • Were all latency, throughput, and memory numbers measured within the same evaluation framework and on the same hardware? Several baselines number appear to be copied from original papers, making direct comparisons difficult to interpret.
10 comments

We thank the reviewer for the detailed questions and for acknowledging that ScanAR is easy to understand, needs no architectural changes, and preserves the unified AR formulation. We address each concern with evidence below.

Q1: ZipAR.

We thank the reviewer for the pointer and will cite ZipAR and add it to Table 1. ZipAR is a training-free method for raster AR models. Its goal is related to ours, so we evaluated it directly on the same raster LlamaGen checkpoints with the same evaluator.

The two methods differ in a fundamental way:

  • Within-row dependency. ZipAR keeps strict left-to-right causality inside every row. Row i+1 can start only after row i has produced w tokens. Its depth is therefore bounded below by the row length W and grows with the window w. ScanAR predicts an entire row in one step.
  • Measured depth. With the official ZipAR code on a 16×16 grid, ZipAR needs 60 (w=2), 88 (w=4), or 144 (w=8, its default) model forwards, compared with 256 for raster AR. ScanAR needs 32, which is 1.9× to 4.5× fewer than ZipAR.
  • Regular execution. In each ZipAR forward, a varying number of tokens from different rows is written at non-contiguous cache positions. In every ScanAR step, exactly one contiguous row is appended. This fixed shape allows ScanAR to be captured in a static CUDA graph (Appendix B.3.1) and served efficiently.

Empirical comparison (ImageNet 256², 50K samples, ADM evaluator, official ZipAR code). Every method is reported at its best CFG scale. Raster LlamaGen and ScanAR are taken from Table 7. ZipAR throughput is measured with the protocol of Appendix B.3.1 (single H20-3e, batch 512, BF16, static CUDA graph) using its official implementation.

ModelMethod (best CFG)TrainingModel forwardsFID ↓IS ↑P/R-F1 ↑Throughput (img/s) ↑
GPT-BRaster LlamaGenfrom scratch2565.46193.60.594117.9
GPT-BZipAR (w=8, default)training-free1445.69192.90.568158.2
GPT-BZipAR (w=4)training-free8816.5989.80.574227.6
GPT-BZipAR (w=2)training-free6055.9628.90.464288.5
GPT-BScanARfrom scratch325.59244.30.568669.4
GPT-LRaster LlamaGenfrom scratch2563.80248.30.63947.1
GPT-LZipAR (w=8, default)training-free1443.69226.60.63660.8
GPT-LZipAR (w=4)training-free886.77168.30.62485.0
GPT-LZipAR (w=2)training-free6033.9054.10.564106.6
GPT-LScanARfrom scratch323.86295.20.637235.2

The comparison shows a clear depth–quality trade-off for training-free parallel decoding. With its default window (w=8), ZipAR reaches ScanAR's quality level, but it needs 144 forwards, 4.5× ScanAR's 32. As the window shrinks towards ScanAR's depth, ZipAR's quality collapses: with 88 forwards its FID is 3× (B) and 1.8× (L) worse than ScanAR's, and with 60 forwards it degrades to an FID of 56.0 (B) and 33.9 (L). ScanAR keeps this quality at 32 forwards, with the highest IS of all methods at both scales.

Speed. At its default window (w=8), the only setting where ZipAR matches ScanAR's quality, ZipAR reaches 158.2 img/s (B) and 60.8 img/s (L). ScanAR is 4.2× (B) and 3.9× (L) faster. As the window shrinks, ZipAR becomes faster but its quality collapses, and even at w=2 (FID 55.96 / 33.90) ScanAR is still 2.3× (B) and 2.2× (L) faster. The measured speedups track the reduction in model forwards (4.5×, 2.75×, and 1.9× fewer than ZipAR at w=8, 4, and 2), so ScanAR's speed advantage comes from its lower decoding depth. In addition, every ScanAR step appends one contiguous row, whereas each ZipAR forward appends a varying number of tokens from different rows at non-contiguous cache positions under an explicit attention mask. For ZipAR we keep its official decoding code and only let PyTorch use its efficient attention kernels and skip a redundant copy of the KV cache. Neither change alters the computation, and both make ZipAR faster.

The reason is that ZipAR asks a raster-trained model to predict tokens whose left context is not yet available. ScanAR instead trains the model on exactly the row-scan conditionals it uses at inference time. This is why ScanAR requires training while ZipAR does not, and the table shows that the training is what makes high parallelism possible without losing quality.

Q2: FID/precision/recall in the main paper; FID vs. raster LlamaGen.

We will move Table 7 into the main text. Space was the only reason it was in the appendix.

On the comparison itself: ScanAR reduces sequential steps by 8× (256 → 32), and its FID stays within a small margin of raster LlamaGen: B 5.59 vs. 5.46, L 3.86 vs. 3.80, XL 3.34 vs. 3.39 (better), XXL 3.46 vs. 3.09. P/R-F1 is also close (e.g., L 0.637 vs. 0.639; XL 0.648 vs. 0.648). Both models are trained from scratch for 300 epochs. A small FID gap at an 8× depth reduction is the expected trade-off for any parallel decoder; for example, PAR is worse than LlamaGen at every size in the same table.

Following the standard protocol of LlamaGen, VAR, and FlashAR, all ImageNet results are reported with the CFG scale tuned for each method.

Q3: Raster Emu3.5 fine-tuned on the same data, steps, and distillation.

ScanAR is an acceleration method, so its quality should be compared with other post-training acceleration methods, not with continued raster training. The post-training uses a small amount of data (80M tokens, compared with 150B tokens and 940K steps for the base model; Table 4) only to fit the existing weights to row-scan prediction. Under the FlashAR protocol, BlockDiffusion, FlashAR, and ScanAR are all post-trained on exactly this data, and this is the comparison the GenEval result in the paper serves: ScanAR is the best of the three in both quality and latency. Keeping the quality of the original model is a by-product of an acceleration method that loses nothing.

As a reference, we also ran the control requested by the reviewer: a raster fine-tune of Emu3.5 with exactly our data, initialization, seed, batch size, and learning-rate schedule for 2K steps. Continued raster training is not an acceleration method; it shows the quality that the data can give a raster model.

Model (same 80M-token post-training data)Sequential steps per 512² imageGenEval ↑
Emu3.5-Image (baseline, no post-training)1,02480.48
Raster fine-tune (2K steps)1,02481.23
ScanAR (2K steps)6481.76
BlockDiffusion (50K steps)6473.83
FlashAR (50K steps)6380.29
ScanAR (20K steps)6482.36

Among the acceleration methods trained on the same data, ScanAR is the only one that loses no quality: BlockDiffusion and FlashAR fall below even the original model, while ScanAR matches a raster model fine-tuned on identical data, with 64 instead of 1,024 sequential steps.

Q4: Preservation of unified capabilities; understanding, text, DPG, T2I-CompBench.

We would like to point out that the preservation of unified capabilities is already supported quantitatively, not only by qualitative examples. We report three unified-generation settings on U0 and text-to-image generation on Emu3.5.

Embodied-scene transfer on U0 (Table 2). On 300 paired prompts scored by the same multimodal judge with the same template (Appendix B.1), ScanAR and the original U0 obtain 7.49 vs. 7.52 overall. The paired per-prompt difference is −0.03 with a 95% bootstrap CI of [−0.13, +0.06], so the two models are statistically indistinguishable. A multimodal judge is used because this task has no reference image.

Interleaved image-text generation. On 80 embodied subtask-planning cases (13.8 generated images per case on average), we compare the ScanAR-adapted U0 with the post-trained official raster-AR Xiaomi-Robotics-U0-Sequence in the same inference engine:

MetricRaster U0-SeqScanARΔ [95% CI]
Valid output (finished, all images decoded) ↑81.3%91.3%+10.0 pp [+1.3, +20.0]
Number of steps = requested ↑100%98.8%−1.3 pp [−3.8, 0.0]
Correct text/image alternation per step ↑100%98.8%−1.3 pp [−3.8, 0.0]
Image fidelity to GT, DINOv2 cosine ↑0.6700.654−0.017 [−0.040, +0.007]
Image fidelity to GT, LPIPS ↓0.3970.408+0.011 [−0.004, +0.027]
Image fidelity to GT, PSNR ↑15.1315.06−0.07 [−0.37, +0.23]
Scene consistency with initial observation, DINOv2 ↑0.6160.577−0.039 [−0.061, −0.018]
Temporal consistency between adjacent steps, DINOv2 ↑0.6900.672−0.019 [−0.037, +0.000]
Subtask-text exact match with GT ↑53.6%80.1%+26.4 pp [+17.8, +35.3]
Generation time per image ↓57.3 s5.2 s11.1× faster end-to-end

ScanAR keeps the interleaved structure and the fidelity of the generated observations while generating the full sequence 11.1× faster.

Video generation (public WorldArena benchmark). On the RobotWin 2.0 test set (1,000 episodes, official 11-metric protocol), ScanAR scores 68.44 overall versus 69.32 for the post-trained U0 it is initialized from. It is higher on all three VLM-judged task metrics (instruction following 83.12 vs. 80.20, interaction quality 77.40 vs. 74.28, perspectivity 96.32 vs. 94.62), while each frame is decoded in 65 instead of 1,036 sequential steps.

MetricRaster U0ScanAR
Aesthetic Quality41.5041.00
Image Quality54.1653.16
Subject Consistency79.0877.32
Background Consistency86.2184.95
Motion Smoothness95.2494.43
Photometric Consistency1.762.60
Dynamic Degree73.0767.58
Flow Score82.3874.97
Instruction Following80.2083.12
Interaction Quality74.2877.40
Perspectivity94.6296.32
Overall69.3268.44

Text-to-image on Emu3.5 (Tables 4–5). GenEval shows that the adapted model keeps, and in fact improves, compositional fidelity (82.36 vs. 80.48).

Preservation by design. ScanAR does not change how text is generated. Text and structural tokens keep their standard causal predecessors and causal attention (Eq. 3), and the vocabulary and output head are untouched. After the image, decoding returns to standard causal mode over the same KV cache (Sec. 3.4). The interleaved examples in Figures 1, 4, and 5 show the adapted U0 model writing multi-step text plans and switching modalities on its own.

On additional benchmarks: Emu3.5-Image-34B is the image-generation checkpoint of Emu3.5, and its established evaluation protocol, used by FlashAR and followed by us, is GenEval. Understanding benchmarks are not part of this protocol. We kept GenEval so that every number is directly comparable with the published raster, BlockDiffusion, and FlashAR results. The understanding or text ability can be measured through the interleave generation results.

Reviewer_bFWbOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper is about making image generation in unified autoregressive models faster. The problem is that these models draw an image token by token in normal reading order, so a picture takes as many steps as it has pixels, which is slow. The proposed method, ScanAR, does not change the network at all; it only changes the order of reading out image tokens. The first row is made left to right as before, and every row after that is made in parallel, where each new token uses the token directly above it, in the previous row, as its guide. This cuts the number of steps sharply, adds no new parameters and no extra generation head, keeps the same left-to-right feel, and keeps one shared stream for text and images. During training it learns both the normal and the row-scan ways to attend at once, with a small copy of the first row to set the starting state, plus a gentle distillation from the original slow model. Results are shown on a strong post-trained model with a big speedup and about the same quality, on an embodied-scene setting, and on class-conditional image generation from scratch at several model sizes, with examples of text-to-image, editing, and video. The aim is to show it goes much faster while keeping the output good.

Critical strengths

The first strength is that it is a clean drop-in change. Rather than rebuild the model or add a new head, it only changes the order the image is read out. That keeps the whole model intact and makes acceleration something you can add to a model you already trained, not just to a special new one. For a speed paper this matters a lot, because it decides whether the method is actually usable in practice.

The second strength is that it is shown to keep capability while going faster, across several different tasks and model sizes. Holding the quality roughly the same on text-to-image and related tasks, while getting a large speedup and scaling across sizes, gives believable evidence that the faster reading order does not badly damage the output.

Critical weaknesses

The first weakness is that the promise of keeping or improving quality is not even. Overall scores hold up, but on the harder relational cases, like several objects together or matching attributes, the results drop a bit. These are the places where making a whole row at once should weaken the model, since row members see each other less. The paper treats this as noise rather than testing whether the method quietly hurts the ability to relate things, which is exactly what its main claim depends on.

The second weakness is that the speed and comparison story is not fully clean. The reported speedup is larger than the basic depth argument alone would give, and the method still does extra work to set up its starting state, so the real cost is not fully laid out. On the quality side, some methods it is compared with are trained under different budgets, and one close competitor still wins on image quality, so the head-to-head is not apples-to-apples.

Questions for authors

First, can you explain the big speedup against the simple depth argument, and show where the time actually goes, including the extra work to set up the starting state?

Second, how much does each added training piece, the first-row copy, the mixed attention, and the distillation, actually help, and could some be removed while keeping quality?

Third, the drops on multi-object and attribute cases suggest the parallel rows may weaken relating things to each other. Do you have any test that checks this directly?

3 comments

We thank the reviewer for the positive assessment and for recognizing ScanAR as a clean, drop-in change that keeps the model intact.

W1 / Q3: Do parallel rows weaken relational ability?

We thank the reviewer for this question. We address the two categories the reviewer points to, Two Obj. and Color Attr., and test the row-interaction hypothesis directly on the generated images.

The post-training data lowers these categories for every model trained on it. To separate the effect of the data from that of decoding, we fine-tuned the raster Emu3.5-Image model on exactly the same post-training data, with the same initialization and schedule, and without any change to its decoding (2K steps):

GenEval categoryEmu3.5-ImageRaster fine-tune (same data, 2K steps)BlockDiffusionFlashARScanAR
Two Obj.94.9590.9188.8991.9287.88
Color Attr.70.2568.0058.4464.0068.00

The post-training data itself lowers both categories: the raster model drops on them even though its decoding is unchanged, and every model trained on this data inherits the drop.

Direct test of within-row interaction. If tokens in the same row "see each other less", relations between objects that occupy the same rows should suffer most. GenEval's Position category contains both cases: "left of / right of" places the two objects side by side, so they share image rows, while "above / below" stacks them across rows. On ScanAR's generations, same-row relations are not the weak point: they are more accurate than cross-row relations, 84.24% for left/right (184 images) versus 76.39% for above/below (216 images).

The design explains why relations are preserved. Compared with raster AR, ScanAR's context for token (i, j) differs only by the left tokens of the current row, at most W − 1 tokens. All previously generated rows remain fully visible and carry the global layout of the image, and each committed row is computed with full within-row attention (Sec. 3.2). Consistent with this, the two layout-level relational categories rise clearly: Position from 73.00 to 80.00 and Counting from 53.75 to 69.06 (Table 5). We will add this per-category analysis to the paper.

W2 / Q1: Speedup vs. depth argument; cost of the start-up (commit) step.

The commit step is already counted. For a 32×32 grid, ScanAR uses 32 causal steps for the first row, 1 non-sampling commit step, and 31 parallel row steps, for a total of 64 forward passes. This is the number reported in Table 9. The commit step is one forward over the W first-row positions that reuses the prefix KV cache. Its temporary KV is discarded, so it adds no cache memory (Sec. 3.4). None of the 64 forwards recomputes the prompt. A row-sized forward costs only about 1.2× a single-token forward in our measurement below, because decoding cost is dominated by reading the weights and the KV cache, so the commit step adds about 1/64 of the decoding time.

Where the extra speedup comes from. All baseline latencies in Tables 4 and 9 (raster Emu3.5, BlockDiffusion, FlashAR) are the numbers published in the FlashAR paper. We measured ScanAR ourselves under exactly the same configuration as published there. The published raster latency of 130.10 s comes from the official Emu3.5 implementation, which decodes with batch size 1. With classifier-free guidance, each of its decoding steps therefore consists of two sequential forward passes. Under the FlashAR configuration, the conditional and unconditional branches of the accelerated model are packed into one batch-2 forward per step. The wall-clock ratio against the published raster number therefore reflects both the reduction in sequential steps and this difference in how CFG is executed. We will state the measurement setting of each cited number in the table caption.

To show the algorithmic contribution of ScanAR on its own, we additionally ran the raster Emu3.5-Image model in the same inference code as ScanAR. Both use the same batch-2 CFG, the same KV cache and attention kernels, the same H20-3e GPU, and the same prompts and sampler (8 prompts, CFG 5). This raster decoder reproduces the reference next-token distributions up to bf16 precision: the per-branch logit difference is 0.5% of the logit range on average.

Method (same implementation)Sequential forwards per 512² imageLatency per image
Raster AR (Emu3.5-Image)1,05564.6 s
ScanAR644.69 s

The controlled speedup is 13.7×. The raster count is 1,024 visual tokens plus 31 end-of-row tokens, so the reduction in sequential forwards is 16.5×. The remaining factor of about 1.2 is the cost of a ScanAR row step, which processes 32 query tokens instead of one (about 74 ms vs. 61 ms per forward, prefill included). When both methods run in the same code, the measured speedup is therefore below the step ratio, exactly as the depth argument predicts. We will add this matched measurement to the paper next to the published numbers.

The same holds on ImageNet: with an identical implementation on the same H20-3e GPU (GPT-B, batch 1, BF16, eager, same sampler), raster AR runs at 0.394 img/s and ScanAR at 2.487 img/s, a 6.3× speedup for an 8× step reduction (256 → 32).

W2: Training budgets and the FlashAR comparison.

The two settings serve different purposes, and in both the comparison is fair or favours the baselines.

Purpose of the ImageNet experiment. We train LlamaGen-based ScanAR from scratch to show that the row-scan factorization is not only reachable by fine-tuning a raster model, but can also be fitted directly from random initialization at four scales. ScanAR therefore supports both regimes, training from scratch and post-training (Sec. 3.3). For this purpose, the right comparison is the from-scratch methods with the same 300-epoch budget (LlamaGen, PAR, NAR). FlashAR is listed for completeness, and its budget is actually larger: it starts from a 300-epoch pretrained LlamaGen and post-trains for 25 more epochs, 325 epochs in total. With less total training, ScanAR achieves the highest IS at every size and an FID within 0.06–0.37 of raster LlamaGen (better at XL: 3.34 vs. 3.39).

Purpose of the Emu3.5 experiment. This is the post-training setting, which is the main target of the paper. Here, under the identical FlashAR protocol with the same data and fewer steps (20K vs. 50K), ScanAR outperforms FlashAR on both GenEval (82.36 vs. 80.29) and latency (4.69 s vs. 5.68 s).

Q2: Contribution of the first-row copy, mixed attention, and distillation.

Each component has a specific, separate role:

  • First-row copy: correctness of the factorization. The first row is generated causally, so its causal state h^causal_{0,j} sees only tokens 0..j. If row 1 were predicted from these states, token (1, j) could not see the first-row tokens to its right. Row 1 would then follow a different conditioning rule from all later rows, which see the entire previous row. The copy produces the fully visible state h̄₀ in the same forward pass, with no extra parameters. At inference it costs one of the 64 steps.
  • Mixed attention: training efficiency. Methods that change the prediction order usually need an arbitrary visibility pattern, which is implemented as a dense 4-D attention mask. ScanAR's mask is different: the causal part (text, structure tokens, first row) and the row-scan part are both made of contiguous, aligned row blocks. This lets us express the whole mask as a block-sparse FlexAttention BlockMask and train in one fused kernel, without materializing a 4-D mask (Appendix C). Table 10 shows the effect. At 4K tokens, a ScanAR step takes 10.5 s, compared with 10.1 s for plain causal AR and 16.2 s with a dense 4-D mask. At 8K tokens, the dense mask runs out of memory, while ScanAR takes 21.3 s, compared with 20.2 s for causal AR. In other words, ScanAR trains at almost the cost of a standard AR model.
  • Distillation: fast fitting from existing weights. When post-training a pretrained raster model, the row-scan predecessor states are new to the model. The frozen raster teacher gives a dense full-distribution target that pulls the new row-scan predictions towards the pretrained model's distribution, so the model adapts quickly. It is therefore used only at the start of post-training, followed by CE only. Distillation is not needed to make the objective learnable. All ImageNet models are trained from scratch without any distillation and reach competitive FID and IS at four scales. We conduct Emu3.5-Image post-train distillation ablation for 2000 steps to show the effect, the results in GenEval show that distillation helps model fit faster.
ModelSingle Obj.Two Obj.CountingColorsPositionColor Attr.Overall
ScanAR w/o Distillation (2K steps)98.1286.8766.2591.7677.2564.7580.83
ScanAR w/ Distillation (2K steps)98.7587.3771.2590.1677.5065.5081.76

Comments on the paper

1 comment

We thank all reviewers for their careful reading and constructive feedback. We are glad that the reviewers found ScanAR clean and drop-in (bFWb, eBhq), easy to understand and requiring no architectural modification (RqgQ), and well presented (JNWF), and that it preserves the unified AR interface (all reviewers).

We first restate the scope of the paper. ScanAR targets Unified AR models, where text, control tokens, and images share one vocabulary, one backbone, and one next-token head. Under this setting, an acceleration method must (i) keep the output head and vocabulary unchanged, (ii) keep raster token indices and position ids so that an existing checkpoint can be adapted, and (iii) apply to every image segment inside an interleaved stream. ScanAR satisfies all three. It reduces the sequential depth of an H×W image from HW to H+W, and it leaves text and structural tokens on the standard causal path (Eq. 3).

ScanAR is an acceleration method. In the post-training setting, the pretrained raster model (Emu3.5-Image, U0) is the model being accelerated, and a small amount of data is used only to fit its weights to row-scan prediction. The relevant comparison is therefore with other post-training acceleration methods under the same data and protocol, not with continued raster training, which does not accelerate anything. Under the FlashAR protocol, where all methods are post-trained on the same 80M-token data, ScanAR obtains the best GenEval (82.36, vs. 80.29 for FlashAR and 73.83 for BlockDiffusion) with the lowest latency (Table 4). This is the comparison that the GenEval result in the abstract serves; that ScanAR also keeps the quality of the original model is a by-product of an acceleration method that loses nothing.

During the rebuttal period we are adding three experiments:

  1. ZipAR on the same raster LlamaGen checkpoints, evaluated with the same ADM evaluator (see RqgQ Q1).
  2. A raster Emu3.5 fine-tuned with exactly our data, initialization, seed, batch size, and learning-rate schedule (see RqgQ Q3).
  3. Post-trained scanAR without Distillation (see bFWb Q2).

And two additional evaluation (see eBhq W1/Q2):

  1. Interleave image-text generation Evaluation
  2. Video generation Evaluation

These three experiments have demonstrated:

  1. The differences and advantages of ScanAR compared to ZipAR.
  2. ScanAR keeps the quality of a raster model trained on identical data while decoding in 64 instead of 1,024 sequential steps
  3. The effectiveness of distillation.

Two additional quantitative evaluation results have demonstrated that scanAR can maintain the unified output capability of the AR model from both theoretical and practical perspectives.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.