acceptodds
Withdrawn from ICLR 2027

Next-Dense-Stride Prediction for Multimodal Autoregressive Visual Modeling

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

We introduce DenseAR, a new generative paradigm that reformulates autoregressive image generation as coarse-to-fine next-dense-stride prediction using a compact single-scale tokenizer. Our key insight is that the quality gain of coarse-to-fine generation needs no multi-resolution tokenizer—the prediction order on one compact grid can provide it. This addresses two limitations of existing autoregressive models at once: the slow inference of raster-order autoregression, which DenseAR avoids by predicting multiple tokens in parallel, and the heavy cost of multi-scale approaches, which need long, multi-resolution token sequences to achieve coarse-to-fine prediction. Building on our efficient framework and the flexibility of autoregressive modeling, we further extend DenseAR to a unified model that handles multiple modalities and imaging tasks within a single backbone. We validate DenseAR on both medical and natural images. On multi-contrast brain MRI, a single DenseAR model unifies cross-modal translation, modality-conditioned generation, and tumor segmentation, while remaining competitive with task-specific methods. On ImageNet, DenseAR improves class-conditional generation quality (FID and IS) over both a single-grid baseline without stride ordering and a multi-scale tokenizer-based baseline.

Primary area:
generative models
Keywords:
Generative modeling, Medical image analysis, Multimodal model

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_pSecOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

DenseAR generates images on a single token grid using a progressively denser spatial order. It starts with widely spaced positions and fills the remaining positions in parallel groups. The method also uses task and modality markers to combine MRI generation, cross-modal translation, and segmentation in one model. Experiments compare the method with visual generation baselines on ImageNet and specialist and generalist models on BraTS.

Critical strengths
  1. Coarse-to-fine ordering on a single grid is a straightforward way to introduce spatial structure without a multi-scale tokenizer. The comparison with ARPG is useful because model size, decoding steps, and reported memory are matched. DenseAR improves FID at both tested scales.

  2. The medical experiments demonstrate a useful combination of tasks within one backbone. DenseAR performs well on translation and remains competitive on generation and segmentation. The subject-level data split is also clearly stated.

Critical weaknesses
  1. Essential supporting material is unavailable in the main paper. The text repeatedly refers to appendices for the stride schedule, training configurations, task mixture, guidance settings, and controlled ablations, which is inconvenient. Central claims should be part of the main paper.

  2. The efficiency argument relies on decoding-step counts and memory figures, with ImageNet baseline measurements taken from another paper. No wall-clock latency or throughput is reported. DenseAR uses more decoding steps than VAR, so lower memory does not by itself establish faster generation. Moreover, the improvement over the closest matched baseline is modest.

Questions for authors
  1. Under the same hardware and implementation conditions, what are the latency and throughput of DenseAR, ARPG, and VAR at comparable quality? Are the FID improvements over ARPG consistent across sampling seeds?
Reviewer_Hsz9OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents DenseAR, an autoregressive visual generation framework based on next-dense-stride prediction. Experiments on ImageNet and BraTS-2023 demonstrate competitive generation quality, reduced memory consumption compared with multi-scale autoregressive models, and the feasibility of performing multiple medical imaging tasks using a single model.

Critical strengths
  1. The proposed next-dense-stride ordering provides a relatively simple mechanism for introducing coarse-to-fine generation without requiring a multi-scale tokenizer.
  2. The framework integrates image generation, cross-modal translation, and segmentation into a single autoregressive architecture.
Critical weaknesses
  1. The central contribution is a deterministic coarse-to-fine token ordering strategy applied to an existing parallel autoregressive framework. Related approaches, particularly ARPG and PAR, have already explored parallel prediction of spatially distributed tokens.
  2. The multimodal experiments are restricted to a single medical imaging dataset (BraTS-2023) and 2D slices, limiting the evidence for generalization across imaging modalities, datasets, and clinical scenarios.
Questions for authors
  1. Given the close relationship between DenseAR and ARPG, can the authors provide more substantial evidence that next-dense-stride prediction offers consistent benefits beyond the relatively small FID improvements reported in Table 2?
  2. During training, tokens are conditioned on preceding tokens within the same stride stage, whereas parallel inference removes these dependencies. How does this discrepancy affect generation quality and consistency?
  3. Can the authors compare the unified DenseAR model against independently trained DenseAR models for each medical imaging task under comparable training budgets?
Reviewer_J6i4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

My understanding is that DenseAR changes the order in which an autoregressive model predicts image tokens. It starts with spread-out positions and then fills the gaps. It uses one token grid per image and predicts groups in parallel during decoding. Task and modality markers let one medical model generate MRI slices, translate between contrasts, and predict tumor masks. Separate models handle ImageNet. The authors report better generation scores than random-order ARPG at matched model sizes and decoding steps. The medical experiments include a stride-order ablation and comparisons with specialist and generalist models.

Critical strengths
  1. I find the medical ablation useful. The stride order adds spatial structure to the existing parallel decoder while keeping one grid per image. Supplementary Table 3 reports medical generation FID of 10.54 without stride and 8.50 with stride. I see this as evidence that the ordering choice helps under the stated matched setup.
  2. I also see value in using one model for several medical tasks. It covers generation, all twelve single-source translation directions, several source combinations, and segmentation. It needs no separate prediction head for each task. This common interface could be useful, even though nnU-Net has higher reported segmentation Dice.
Critical weaknesses
  1. I am not yet sure that the ImageNet comparison separates the benefit of the ordering rule from extra training. The paper says compute is matched with ARPG. However, the supplement's GETTING_STARTED file says the released models receive another 180,000 or 240,000 training steps from ARPG checkpoints. If those models produced Table 2, extra training could explain some of the FID gains of 0.14 and 0.09. Identifying the evaluated runs could resolve this concern. If they used the documented continuation, a random-order control with the same starting weights and additional updates would help separate the two effects.
  2. I could not reconcile the medical data split in Section 4.1 with the supplied loaders. The paper describes 1,000 training, 11 validation, and 240 test subjects. The loaders use all 1,251 subjects for generation and 1,188 for translation. From my reading, these settings would let segmentation-test images enter other training tasks. I cannot tell whether the reported runs used these settings. This also does not establish that test masks were used. The segmentation script selects slices with at least 2,000 ground-truth tumor pixels for its headline score. The paper mentions filtering, but I am unclear which slices were retained and whether all methods used the same set. These details affect how I interpret performance on held-out subjects.
Questions for authors
  1. Which runs produced Table 2, and what initialization and total training did each receive? If DenseAR continued from ARPG weights, was random-order ARPG continued for the same additional updates?
  2. What subject lists were used for each medical task and tokenizer training? Did any segmentation-test subject contribute images through another task?
  3. What mask-area cutoff and retained slice counts produced each segmentation result? Were all methods scored on exactly the same slices?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.