acceptodds
Withdrawn from ICLR 2027

EAR: Efficient AutoRegressive Text-to-Image Synthesis from a Game-Theoretic View

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Conventional autoregressive text-to-image synthesis approach rely on next-token prediction, which would hinder inference efficiency due to their sequential, predefined token generation order. To enable high parallelization without sacrificing generation quality, we revisit autoregressive text-to-image models from a theoretical perspective by analyzing the generation order of visual tokens. We also propose the Efficient AutoRegressive text-to-image synthesis model (EAR), which provides a principled explanation of how visual tokens are generated in autoregressive models, while still supporting fine-grained image synthesis with efficient inference. Specifically, drawing inspiration from Game theory, we suggest employing Shapley value to evaluate the interactions between regions that arise from different order steps. Moreover, we present a novel text-to-image synthesis framework that facilitates rapid generation, with the guidance of both global representation and intermediate local embedding extracted from a pretrained text encoder. Extensive experiments, including quantitative evaluations and qualitative visual comparisons, on the public benchmarks show that our EAR model achieves lower latency than previous autoregressive text-to-image models without sacrificing image quality. We hope that the EAR model will offer novel insights into the interpretability of autoregressive models, thereby providing valuable benefits to researchers in this field.

Primary area:
generative models
Keywords:
Game theory, Text-to-Image Synthesis

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ELBHOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper presents EAR, an autoregressive text-to-image generation framework that enables parallel prediction of grouped visual tokens. The method introduces a game-theoretic formulation based on Shapley values and mutual information to model dependencies among visual token groups and guide their generation order. A neural network is employed to predict the generation schedule, while a Mixed Visual Autoregressive Transformer (MVAT) uses positional query tokens to generate multiple visual tokens in parallel. The framework also incorporates global and local textual representations extracted from a pretrained CLIP encoder. Experiments are conducted on COCO and MJHQ to evaluate generation quality and inference efficiency, including comparisons with existing generative models and ablations on different generation ordering strategies.

Critical strengths

The paper addresses a well-motivated problem in autoregressive image generation: sequential decoding limits inference efficiency, while naive parallel prediction may disrupt spatial dependencies among visual tokens. Exploring dependency-aware generation ordering to balance parallelism and generation quality is a meaningful research direction.

Critical weaknesses

1.The ordering objective in Eq. (6) appears to be permutation-invariant for a fixed grouping, since both mutual information and the spatial distance weights in Eq. (5) are symmetric. It is therefore unclear how this loss can guide the claimed generation order, and the reported experiments do not establish that the learned ordering is actually induced by the proposed theoretical objective.

  1. Eq. (5) directly expresses the Shapley value using weighted mutual information, but the paper does not establish a mathematical relationship between these two quantities. A rigorous derivation or justification is needed to support the proposed game-theoretic formulation.

3.Table 1 significantly exceeds the right page margin, which is a conspicuous formatting issue in one of the paper's central experimental results. Such an obvious issue raises concerns about the overall care taken in preparing the manuscript.

Questions for authors
  1. Given the symmetry of mutual information and spatial distance weights, Eq. (6) appears to be permutation-invariant for a fixed grouping. Could the authors clarify how this objective provides a learning signal for the generation order and demonstrate that the learned ordering is driven by this loss?

  2. Could the authors provide a mathematical derivation or theoretical justification for expressing the Shapley value as weighted mutual information in Eq. (5)?

Reviewer_SBnqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

EAR proposes a parallel autoregressive text-to-image model with a learned token-grouping and generation schedule. Its auxiliary objective is motivated by Shapley values and mutual information. The generator combines a flat visual-token representation with global and token-level CLIP conditioning. Experiments compare image quality and efficiency against existing models and evaluate alternative orderings and text-conditioning variants.

Critical strengths
  1. Flexible parallel text-to-image generation. EAR combines learned token grouping with parallel decoding on a flat visual-token representation, without requiring a scale-wise token hierarchy.
  2. Text-conditioning ablations support the hybrid design. The model uses global and token-level CLIP representations. In Table 7, adding token-level guidance improves MJHQ FID from 5.6 to 5.2 and GenEval from 0.59 to 0.62.
  3. The reported quality–efficiency trade-off is promising. The 445M-parameter model achieves COCO FID 11.9, MJHQ FID 5.2, and GenEval 0.62, with reported throughput of 32.68 images/s at 512 × 512 resolution. Appendix A.12 also quantifies its additional training cost relative to a standard AR baseline.
Critical weaknesses
  1. The scheduling-loss derivation is incomplete. Equation (5) specifies pairwise MI scores, but the coalition value function required by Equation (11) is not defined. Theorem 1 establishes sufficient dummy-player conditions without proving the claimed equivalence to Equation (6), making it difficult for me to verify the central game-theoretic justification. Question: Please define the coalition game, identify the loss actually optimized, and establish the equivalence under explicit assumptions or describe Equation (11) as a heuristic surrogate.
  2. The learned scheduler is insufficiently specified for reproduction. Figure 2 depicts text conditioning, but the construction of token-wise group predictions is unclear. Appendix A.4 lists fixed group sizes without explaining quota-constrained assignment or gradient propagation through the discrete schedule; these are essential operations of the proposed method rather than peripheral implementation details. Question: Please provide training and sampling pseudocode specifying the scheduler inputs, assignment procedure, and gradient estimator or relaxation.
  3. The ordering ablations do not isolate the proposed auxiliary objective. Table 3 supports EAR’s scheduling design over the tested heuristics, but changes both the scheduling procedure and the use of the MI objective. The rank-correlation and semantic-priority analyses provide useful consistency evidence, yet do not establish that the game-theoretic regularizer contributes beyond learning a schedule. Question: Please compare with an otherwise matched learned scheduler without the MI objective, explain how this control is trained, and report variability for the ordering variants.
  4. Some efficiency results lack a consistent configuration description. Figure 7 makes latency depend on Shapley sampling count, although Appendix A.12 states that the MI estimator is training-only. Table 9 also uses eight steps at 256 × 256 resolution, whereas Appendix A.4 selects twenty, without reporting the corresponding eight-step quality. Question: Please identify the operation timed in Figure 7, report quality for the eight-step setting, and specify which components are included in generation timing.
Questions for authors

see weaknesses

Reviewer_xveBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes EAR, an efficient autoregressive text-to-image synthesis framework that aims to improve inference speed without sacrificing image quality. The core idea is to rethink the generation order of visual tokens in AR models from a game-theoretic perspective, using Shapley values to quantify interdependencies among token groups. Based on this analysis, the authors design a learnable generation ordering schedule that enables parallel prediction of token groups while preserving strong dependencies.

Critical strengths

Having reviewed an earlier version of this work, I appreciate the substantial improvements in the experimental evaluation compared with the earlier version. The additional random and raster-scan ordering baselines, analyses of learned generation schedules, and reporting of training costs and implementation details address several of my previous concerns. These additions provide a clearer picture of the scheduling mechanism and its computational trade-offs.

Critical weaknesses

The specific contribution of the game-theoretic regularizer remains insufficiently established. The paper lacks a controlled ablation using the same learnable ordering network with and without LMIL_{\mathrm{MI}}, making it difficult to separate the regularizer’s contribution from that of the scheduler or other design choices. The motivation for preferring a game-theoretic formulation over simpler scheduling objectives also remains unclear, and the connection between Theorem 1 and Equation (11) requires stronger justification. Although EAR-loc incorporates LPD’s locality-aware ordering, other relevant approaches, such as RandAR, PAR, and ARPG, are not evaluated as controlled ordering baselines; random grouping alone does not reproduce these methods.

Questions for authors

I think the core issue is why we need to use game theory to solve this problem. The paper does not clearly explain its motivation. What specific scheduling property does the game-theoretic regularizer enforce, and what evidence shows that this property cannot be achieved equally well by a simpler objective? If these questions are not answered, why should we use a game-theory-based method instead of a traditional method? Similar works, such as RandAR, PAR, and ARPG, are not included as comparison baselines.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.