acceptodds
Under review as a conference paper at ICLR 2027

CanvasFlash: Efficient Autoregressive Image Generation with Semantic Representation

Abstract

Long visual sequences impose substantial training and decoding costs on high-resolution autoregressive (AR) image generation. We propose CanvasFlash, a novel approach that reduces this cost by exploiting the redundancy within visual representations. We depart from the reconstruction-oriented design in favor of a semantic image tokenizer that serves as a high-level information bottleneck, and delegate pixel rendering to a diffusion decoder. The resulting token distribution becomes highly concentrated: diverse tokens cluster in semantically informative regions, whereas broad regions share a common Canvas Token, indicating that only a fraction of the sequence is needed to characterize an image for semantic representation. Leveraging this sparsity, we propose Canvas-Sparse Training, which processes of the tokens required by full-sequence training. We also propose Canvas-as-Draft Decoding, which uses the Canvas Token as a parameter-free draft and achieves up to speedup in AR decoding. With reinforcement learning, CanvasFlash combines these efficiency gains with competitive image generation performance. More broadly, CanvasFlash highlights the redundancy carried by reconstruction-oriented visual representations, and validates semantic tokenization as a practical route toward efficient autoregressive image generation.

Primary area:
generative models
Keywords:
Image Generation, Unified Model, Multimodal Learning, Reinforce Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_FZsxOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents CanvasFlash, which uses a semantic image tokenizer that naturally produces many Canvas Tokens. The paper exploits this structure through sparse AR training and a simple Canvas-token draft strategy for faster decoding, while using a diffusion decoder for final rendering. CanvasFlash highlights the redundancy carried by reconstruction-oriented visual representations.

Critical strengths
  1. The core efficiency idea is simple and technically appealing. Canvas Sparsity gives a natural signal for removing redundant tokens during training and reusing the Canvas Token for speculative decoding without an extra draft model. This idea is interesting, but also makes the lack of convincing end-to-end efficiency measurements more consequential.

  2. The method achieves strong generation quality despite aggressive token reduction. The paper reports competitive benchmark results while retaining only a fraction of the original training tokens, suggesting that the representation preserves useful semantic information.

Critical weaknesses
  1. The main efficiency claim is not validated end-to-end. The reported 3.09× decoding speedup excludes diffusion rendering, while training efficiency is mainly expressed through retained token count rather than wall-clock time, throughput, or compute. Since efficiency is the central contribution, this missing system-level evidence is a critical weakness.

  2. The contribution of Canvas Sparsity to final quality is not well isolated. DPG-Bench improves from 83.95 after SFT to 89.69 after GRPO, so a large part of the final performance gain comes from reinforcement learning rather than the proposed sparse representation itself. This makes it difficult to judge whether the proposed mechanism alone offers a strong enough quality–efficiency trade-off.

Questions for authors
  1. Could the authors report end-to-end latency, including the diffusion decoder, and actual training throughput or GPU-hours with and without Canvas-Sparse Training?

  2. Please provide the main efficiency-quality comparison before GRPO, so that the contribution of Canvas Sparsity can be evaluated independently.

Reviewer_Wjr9OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CanvasFlash uses a semantic image tokenizer to separate autoregressive semantic planning from diffusion-based pixel rendering. Many positions map to a shared Canvas Token. The method reduces their training supervision and attention context, and uses repeated Canvas Tokens as drafts for parallel verification during decoding. GRPO further improves the complete system's generation quality. The paper reports retaining 41% of training tokens, up to 3.09x faster AR token decoding, and DPG-Bench 89.69 at 1024 x 1024 resolution. Ablations examine the two training operations, draft-window size, and reinforcement learning.

Critical strengths
  1. The same observed token redundancy supports both training reduction and decoding acceleration without a separate draft model. Table 2 provides useful controls for label removal and attention-KV removal.
  2. The window-size analysis explains the decoding trade-off, and the complete system achieves competitive generation results. Table 4 also makes the additional quality gain from GRPO visible.
Critical weaknesses
  1. Semantic preservation under high sparsity: Figure 4 illustrates semantic reconstruction, while the quantitative results mainly evaluate the complete generation system. Which aspects of object counts, attributes, and spatial relations are preserved or altered in the existing source/reconstruction pairs? Please annotate a few existing pairs, noting preserved content and visible omissions or changes, with Canvas fractions if available. This is a qualitative assessment of the combined tokenization-and-rendering pipeline.
  2. Assumptions behind the prior correction: Equation (7) adds log(1/alpha) to the Canvas logit to compensate for reduced label supervision, while the method also removes Canvas KV context. Which conditional distribution is this odds rescaling intended to correct? Please state the label-sampling assumptions and explain how Canvas KV removal affects that distribution. Table 2 provides relevant empirical evidence.
  3. Reproducibility of the AR speedup: The 3.09x figure excludes diffusion rendering, but the timing GPU, inference batch size, precision, and absolute AR latency are missing. Section 4.3 gives sampling defaults without explicitly linking Table 3 to the RL checkpoint. Which checkpoint and sampling settings were used for Table 3? Please report the available timing protocol and absolute AR latency from the existing records.
  4. Training cost and recipe reproducibility: Table 2 matches training-token budgets and therefore exposes sparse training to more images; Table 4 adds quality gains from GRPO. What training-cost information and configurations are available for the Table 2 runs and the GRPO stage? Please include available training-time records, the data mixture, and the GRPO steps, group size, and reward weights.
Questions for authors

Please refer to weaknesses.

Reviewer_akc5OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

CanvasFlash combines autoregressive (AR) semantic modeling with diffusion rendering through a semantic image tokenizer. It exploits repeated Canvas tokens to enable sparse training and speculative decoding without an auxiliary draft model. The design is coherent and offers some novelty, with promising generation quality and AR decoding speedups.

Critical strengths
  1. Coherent representation-driven optimization. The method exploits the same token redundancy to reduce both training computation and sequential decoding overhead, rather than combining unrelated acceleration techniques.

  2. Meaningful decomposition of training redundancy. Separately examining Canvas tokens as prediction targets and attention context is more informative than pruning solely by token identity. Table 2 provides preliminary evidence for both forms of redundancy.

  3. Simple, draft-model-free speculative decoding. Canvas-as-Draft uses fixed Canvas proposals and target-model verification while preserving the sequential sampling distribution under the same target model and sampling settings. Its speedup does not rely on relaxed acceptance criteria.

  4. Competitive full-system quality. CanvasFlash achieves the highest DPG-Bench score in Table 1 and outperforms the listed open-source models on UniGenBench English, showing that its representation and acceleration strategies can coexist with strong generation quality.

Critical weaknesses
  1. Incomplete Efficiency Evaluation

Retaining 41% of training tokens and achieving up to 3.09× faster AR decoding do not directly establish system-level efficiency.

For training, this work lacks measured throughput, wall-clock time, or GPU-hours under matched hardware and clearly specified budgets. Shorter sequences reduce theoretical computation, but practical gains depend on sequence packing, communication overhead, and hardware utilization.

For inference, this work appropriately states that the speedup excludes diffusion rendering. Nevertheless, an efficiency-focused image generation system should report absolute AR and diffusion runtimes, end-to-end latency, and evaluation conditions, including hardware and batch size. Without these measurements, the practical impact of the AR speedup remains unclear.

  1. Limited Evidence on the Origins and Information Preservation of Canvas Sparsity

Canvas sparsity is central to the method, yet its support primarily consists of token visualizations, semantic reconstruction examples, and downstream results. Appendix B reports a similar pattern with another VLM, providing useful preliminary evidence of generalization but not explaining how the sparsity arises.

Repeated assignment to one codebook entry could reflect effective semantic compression, partial codebook collapse, or loss of fine-grained information. The current experiments do not adequately distinguish these possibilities.

This work is suggested to report codebook utilization, token entropy, Canvas frequency, and consecutive run-length distributions. Controlled experiments varying encoder initialization, supervision objectives, and codebook size would help explain the phenomenon. Information preservation should also be evaluated on text, counting, and small objects, rather than primarily on major objects and layouts.

  1. The Relationship Between Sparse Training and Inference Needs Clarification

This method combines three distinct operations: reducing supervision on Canvas labels, masking Canvas keys and values, and removing sequence positions. Their implementation and interaction are not sufficiently specified.

First, next-token prediction shifts inputs relative to targets. A Canvas input does not necessarily imply that its associated prediction can be discarded. Figure 3 recognizes this distinction, but the formal definitions should explicitly map input indices to target indices.

Second, training masks Canvas KV entries, whereas inference generates the full grid. The paper should clarify whether inference uses the same attention policy and how this policy interacts with draft verification and KV caching.

Third, the prior correction in Equation (7) is justified for class subsampling but does not automatically compensate for changes caused by context pruning. An independent ablation is needed, along with the order in which prior correction, CFG, temperature, and top-p are applied.

These ambiguities do not demonstrate that the method is incorrect, but they weaken reproducibility and the precision of its theoretical claims.

  1. Insufficient Attribution of Final Quality Gains

In Table 4, CanvasFlash scores 83.95 on DPG-Bench after SFT, below the standalone diffusion model’s 86.55, and reaches 89.69 after RL. This demonstrates the importance of RL to the final system, but does not isolate the benefit of semantic AR modeling or establish superiority over diffusion models optimized with comparable budgets.

The system also follows X-Omni’s semantic supervision and AR–diffusion division of labor while relying on strong pretrained components. More controlled comparisons are needed to separate the contributions of the tokenizer, Canvas strategies, data scale, and RL.

A particularly useful baseline would use identical components and training settings without Canvas acceleration. Reporting Canvas statistics and decoding speed before and after RL would also establish whether the efficiency gains persist after RL.

Questions for authors
  1. Under specified hardware, batch size, and sampling settings, what are the absolute AR, diffusion, and end-to-end latencies for standard decoding and Canvas-as-Draft? How many GPU-hours does sparse training save?
  2. Does Canvas-Sparse Training preserve quality at a fixed number of training samples, rather than only a fixed token budget?
  3. What quantitative evidence distinguishes useful semantic compression from harmful information loss? How sensitive is Canvas sparsity to codebook size and encoder initialization?
  4. Can you provide training pseudocode specifying input–target alignment, position-removal rules, and the training and inference KV policies?
  5. Has prior correction been independently ablated? In what order are prior correction, CFG, temperature, and top-p applied?
  6. How do Canvas frequency, run lengths, and decoding speed change after RL? What gains remain over an X-Omni-style baseline with matched components and training budgets?

Comments on the paper

5 comments

Dear Reviewers,

The authors have posted their rebuttal. Please read it carefully, assess whether they address your concerns, and leave a follow-up comment for the authors.

Thank you for your time and constructive feedback.

AC

Dear Reviewers,

The authors’ rebuttal is now available. Please review it, indicate whether your concerns have been addressed, and leave a follow-up comment for the authors.

Thank you for your time and constructive feedback.

Best,
AC

AuthorsOpenReview ↗

Summary of Revisions and Author Responses

We thank the reviewers for their careful reading and the AC for handling our submission. In the revision, we add one key experiment and a set of clarifications to respond to every point raised. All added text is marked in blue.

New experiment. The most frequently raised concern was that our efficiency claim was reported for the AR stage alone. We therefore measured the end-to-end pipeline including diffusion rendering, and obtain a 2.06× speedup at 1024x1024, with consistent gains across resolutions and draft-window sizes.

Added details. We also supplement details that were previously omitted: training pseudocode specifying input--target alignment, position removal, and the training/inference KV policy (Appendix B); the scope of the prior correction and the sampling order (Sec. 3.3, 4.3); the matched 98M-token budget of the Table 2 (Sec. 4.4); and the data sources and GRPO configuration (Sec. 4.1, Table 7).

Per-reviewer summary. Reviewer Wjr9 gave a positive assessment and raised valuable questions. We answered each and incorporated the corresponding material into the revised paper. Reviewer FZsx raised two concerns—missing end-to-end efficiency evidence and attribution of quality gains to GRPO. The new end-to-end measurement addresses the first, and the RL-free ablation in Table 2 addresses the second, so we believe both can be resolved. Reviewer akc5 raised mainly clarification requests on measurement protocol and implementation detail. We have supplied the requested details and tightened the corresponding discussion, and we respectfully note that these points concern reporting completeness rather than the validity of the method.

We hope the added experiment and clarifications resolve the reviewers' concerns and allow the paper's contribution to be assessed fairly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.