- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes OAR, a post-training procedure for any-order autoregressive image generators. A pretrained model assigns each training image a specialized generation order by greedily selecting confident positions under teacher forcing. The model is then fine-tuned on these image–order pairs, with optional order annealing. Experiments evaluate generation quality, arbitrary-order conditioning, and text-conditioned generation.
Critical strengths
-
The same-backbone, matched-budget comparison provides useful evidence for specialized orders. On ImageNet, OAR achieves 2.17 FID, compared with 2.34 for annealed raster specialization and 2.39 for the original random-order baseline (Table 1). This isolates the choice of target order more convincingly than comparisons across different architectures.
-
The paper evaluates whether specialization preserves the flexibility of the original model. Inpainting and outpainting performance remains close to RandAR, while the human evaluation provides complementary evidence: OAR is preferred in 16 of 21 randomly selected image pairs. The pair-level sign test appropriately accounts for the limited number of evaluated pairs.
Critical weaknesses
-
The inference-cost tradeoff is insufficiently characterized for the main ImageNet model. Greedy decoding queries all remaining positions at each step. Parallel evaluation and KV caching can reduce latency, but do not establish that its cost is comparable to a prescribed order. Table 9 reports latency only for the separate six-layer text-conditioned model. A matched RandAR-XL timing comparison is needed to assess the practical value of the main quality improvement.
-
The inpainting/outpainting comparison against MAR and MaskGIT lacks important protocol details. Table 3 gives the mask-coverage range, but does not fully specify mask construction, the MAR variant/checkpoint, or task-specific sampling settings for the baselines. The results support retention relative to RandAR more clearly than they establish the broader cross-model superiority.
Questions for authors
-
What are OAR and RandAR-XL’s latency, throughput, and peak memory under identical hardware, precision, batch size, and decoding settings? Results at 256 and 128 steps would clarify the quality–cost tradeoff.
-
Which checkpoints and sampling configurations were used for Table 3? How were masks constructed, and was masking performed before image encoding or directly on latent tokens?
-
In Appendix C.3, what justifies cancelling the marginal V-entropy terms between Equations (6) and (7)? Content-dependent ordering can change the marginal distributions of the selected-token random variables. Please clarify the conditioning on positions, or distinguish the measured cross-entropy difference from the claimed V-information difference.
3 comments
Thank you for the detailed responses and additional experiments. I appreciate the authors' efforts to address my concerns, particularly the new inference-time measurements and clarification of the theoretical analysis.
The additional latency results provide a more complete picture of the trade-off. While the overhead is relatively small at low batch sizes, it becomes substantial at larger batch sizes (up to 3.35× latency overhead). The significant increase in token-forward computations also raises concerns about the practical efficiency of OAR, especially in high-throughput settings.
The clarifications regarding the completion protocol and V-information analysis are helpful. However, I believe the inference cost remains an important limitation that needs to be considered alongside the reported quality improvements.
Overall, the rebuttal has addressed several of my concerns, but the quality-efficiency trade-off is still not entirely convincing. I therefore maintain my original rating.
Thank you for the follow-up and for recognizing the additional measurements and theoretical clarification. Your feedback has helped us make the practical trade-off more concrete.
OAR improves the same RandAR-XL backbone from FID 2.39 to 2.17 while retaining zero-shot completion. The fine-tuning stage uses at most approximately 5% of the pretraining epochs, with the architecture unchanged.
The human study directly compares OAR against its pretrained RandAR-XL baseline in a controlled A/B test. Each pair contains one OAR image and one RandAR-XL image, generated with the same class, seed, guidance, and temperature on the same architecture. The pairs are uncurated, drawn from randomly sampled classes, and presented with randomized left/right placement. Across 51 raters, OAR is preferred over RandAR-XL in 64.3% of judgments and wins the majority vote in 16 of 21 pairs (two-sided pair-level sign test, p=0.027). This provides evidence that the gain is perceptible to people as well as reflected in FID.
Table 1 provides two useful controls. With the same backbone, fine-tuning budget, and annealing schedule, specialized orders achieve FID 2.17 versus 2.34 for raster specialization. Under the same greedy inference policy, fine-tuning improves FID from 3.64 to 2.17. Together, these comparisons support the value of the specialized trajectories and show that the trained model improves under a fixed decoder.
The matched A100/H100 benchmarks identify a relevant practical regime: OAR/RandAR latency is 1.01–1.16× at batches 1–8. At batch 32, it rises to 2.80–3.35×. We agree that the additional candidate-query computation is an important limitation for high-throughput generation and will make this batch dependence explicit in the revision.
OAR also retains parallel decoding: halving the steps from 256 to 128 changes FID from 2.17 to 2.25, an increase of 0.08 (Table 4). This preserves much of the full-step quality while allowing fewer decoding rounds.
We hope these controls and retained capabilities clarify OAR's contribution: specialized orders improve an existing generator through short post-training while preserving its completion flexibility. We will incorporate the cost measurements and their practical scope into the revision. If a focused additional analysis would help assess the remaining concern, we would welcome your guidance and be happy to provide it.
Runtime Results, Completion Protocol, and V-Information
Thank you for the thoughtful review and for recognizing the controlled evidence and retained flexibility. Your questions help us clarify practical costs, improve reproducibility, and sharpen the theoretical interpretation.
1. Matched RandAR-XL inference cost.
We benchmarked both 775M implementations on one 80GB A100 or H100: bf16, PyTorch 2.10.0+cu128, CFG 1.75, temperature 1, top-k 0, top-p 1, seed 0, and identical classes. Both use 256 steps or 128 with the same cosine parallel schedule. Synchronized generation timing includes KV allocation and excludes VQ decoding. Timing uses random weights; trained quality is evaluated separately at CFG 3.4.
Mean batch latency in seconds:
256 steps
128 steps
After two warmups, we used 15 repetitions through batch 16 and 10 at batch 32. SD was at most 2% of the mean, except A100 OAR at batch 1/256 steps (5%).
OAR/RandAR latency ratios are 1.01–1.16× through batch 8, 1.56–1.86× at 16, and 2.80–3.35× at 32 across both GPUs/schedules. Throughput is batch divided by latency; H100 batch-32 OAR/RandAR throughput is 1.55/4.40 images/s at 256 steps and 2.59/8.70 at 128.
Peak allocated memory, including weights, on A100 at 256 steps (GiB):
H100 memory differs by at most 0.02 GiB; 128-step memory is within about 0.03 GiB of the 256-step values.
At 256 steps, OAR/RandAR uses 33,407/512 token-forwards per image and an estimated 52.4/0.79 TFLOP (about 66×); at 128, 32.8/0.99 TFLOP (about 33×). Transformer forwards account for 96–98% of profiled time; confidence, selection, and sampling take 2–4%. Parallel queries use GPU parallelism: approximate profiled A100 batch-32 busy fractions are 53% for OAR and 12% for RandAR. Together with RandAR's eager-loop launch overhead, this helps explain the smaller latency ratio.
Table 4 separately reports OAR FID 2.17 at 256 steps and 2.25 at 128. We will add these cost measurements with their setup. The practical trade-off depends on batch size.
2. Completion protocol and interpretation.
Table 3 uses ImageNet 256×256 with shared random masks covering 25–75%. RandAR uses its released XL checkpoint; OAR fine-tunes that backbone. For MAR and MaskGIT, we used released ImageNet checkpoints and followed the default sampling settings in their repositories. All methods used the same images and masks. OAR conditions on visible tokens and fills masked positions greedily.
Thank you for highlighting the need for fuller protocol details. We will specify mask geometry, the encoding/masking sequence, model releases, guidance, temperature, steps, and schedules in the revision.
We agree that the controlled results support retention relative to RandAR: inpainting FID 2.10 versus 2.14 and outpainting 3.17 versus 3.21, without completion-specific training. We view these small differences as supporting evidence of preserved flexibility and will clarify the limits of the broader cross-model comparison.
3. Marginal V-entropy in Appendix C.3.
Thank you for identifying the missing conditioning. Content-dependent ordering can change the target marginal at a fixed decoding rank. We will replace the rank-wise argument with a pooled spatial-target comparison: sample J uniformly over the N positions, independently of the image and its text condition, and set Y=X_J. For each order π, define C_π as the text condition, target position J, and the ground-truth position–token prefix preceding J.
Both orders then predict the same target distribution: P(J=j,Y=y)=P(X_j=y)/N. For one common predictive family V satisfying optional ignorance, the shared marginal term cancels by construction:
=H_V(Y\mid C_r)-H_V(Y\mid C_s).$$ Here s denotes specialized and r raster order. This pooled identity replaces the sum of rank-specific information terms. We will distinguish the family-optimal V-entropy from fitted-model teacher-forced cross-entropy and state the scoring model and normalization of the measured difference. This retains V-information as the motivation for usable context under finite predictive capacity, with the empirical quantity labeled precisely. Thank you again. Your suggestions have helped us characterize specialization costs, identify missing protocol details, and clarify the information-theoretic argument.