- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces VISTA, a training-free, gradient-based test-time alignment framework for next-scale visual autoregressive (VAR) text-to-image models, aiming to improve attribute binding and spatial relationships. At selected early, coarse scales of Infinity, VISTA optimizes the continuous scale input through backpropagation. It uses compositional losses computed from differentiable cross-attention maps, combining CONFORM-style binding and PSG-style planar objectives with a newly proposed boundary-containment depth loss. To support optimization during stateful multi-scale sampling, the method incorporates KV-cache isolation, per-token gradient normalization, scale-aware step sizes, and an adaptive step budget. Experiments on T2I-CompBench and GenEval show substantial improvements over Infinity-2B/8B: the targeted average increases from 0.480 to 0.575 on 2B, while GenEval Position improves from 0.25 to 0.745. ImageReward scores and human preferences for compositionality also show improvements.
Critical strengths
-
The paper identifies an important gap: while test-time attention steering has been widely explored in diffusion models, it remains relatively underdeveloped for next-scale VAR models. The paper also clearly explains why existing diffusion-based approaches cannot be directly applied.
-
The proposed stabilization techniques, including cache isolation, per-token gradient normalization, and scale-aware step sizes, are each designed to address specific challenges arising from stateful multi-scale sampling.
-
The framework supports modular objectives, allowing existing attention-based losses from diffusion models to be incorporated easily. Among 17 tested configurations, 15 outperform the original backbone.
-
The improvements are substantial and consistent across random seeds, including a +20% gain in the targeted average on 2B. Notably, Infinity-2B with VISTA even outperforms the original 8B model.
-
The experimental evaluation is relatively comprehensive, covering human studies with statistical significance tests, ablation experiments, and runtime and memory analysis. The paper also acknowledges limitations such as increased latency and reduced aesthetic quality.
Critical weaknesses
-
The methodological novelty is relatively limited. The binding and planar losses are directly adopted from CONFORM and PSG, respectively. The primary contribution lies in adapting existing objectives to VAR models through several engineering techniques, along with a heuristic depth objective that provides relatively small improvements (+0.047 on 2B and +0.010 on 8B for 3D). The claim of being the “first gradient-based” approach also needs more careful discussion in relation to SSG and VPG, as their comparisons rely only on results reported in the original papers.
-
The baseline comparisons are not matched in computational cost or experimental protocol. VISTA is evaluated over 4 seeds, whereas BoN and TTS-VAR use only one, and TTS-VAR is tested only with N = 1 and 2. Interestingly, BoN (N = 8) performs worse than the original backbone, without further explanation. The paper also lacks comparisons under matched wall-clock time and does not investigate whether search-based methods could be combined with gradient-based steering.
-
The adaptive step-budget scheduler appears to have no practical effect on 8B. With τ = 1e-12, the appendix indicates that neither early stopping nor budget reduction is activated, meaning every scale uses all 5 optimization steps. Moreover, there is no adaptive-vs-fixed ablation on 2B to demonstrate the scheduler's effectiveness. The values of τ also differ by nine orders of magnitude across backbones, while lr_base differs by 9×, both selected through “brief manual exploration”. This raises questions about the method's robustness and the amount of hyperparameter tuning required for other backbones.
-
The generalizability of the method remains unclear beyond Infinity. All experiments are conducted on Infinity with ScaleKV, and the approach depends on explicit text cross-attention and a dependency parser. The framework also does not address numeracy or non-spatial relationships, while improvements on the Complex category are quite limited (0.384 → 0.398).
Questions for authors
-
Could you compare the adaptive scheduler with a fixed N_max budget on 2B? Was the prompt subset used to manually tune τ and lr kept separate from the evaluation prompts?
-
Why does BoN (N = 8) perform worse than the original backbone? Could you provide comparisons with BoN or TTS-VAR under matched wall-clock time and identical random seeds?
-
Could you evaluate SSG and VPG using the same backbone and evaluation protocol to enable a more direct comparison?
-
Have you tested VISTA on VAR architectures beyond Infinity, particularly models that do not provide explicit text cross-attention?
5 comments
Generality, attention requirements, and constraint coverage. Switti and Infinity share the same core VAR process of generating discrete token grids from coarse to fine. Switti uses a different tokenizer and current-scale self-attention while retaining explicit text cross-attention. VISTA raises Switti’s seven-category CompBench mean by 13.3%, including Numeracy, and its three-category GenEval mean by 36.4% (Tables 1–2 and Appendix G). The uncompressed Infinity‑2B reference scores 0.2% higher than the ScaleKV baseline, showing that the cache-compressed baseline starts at a comparable level (Appendix F).
The current implementation requires differentiable text cross-attention. For extracting constraints, we use a simple dependency parser. Fixed templates would suffice for many benchmark prompts. Parsing lets us use the same extraction rules for prompts phrased outside those templates (Section 5).
The CountCluster extension (Lee et al., 2025, arXiv:2508.10710) raises Infinity‑2B Numeracy by 6.6%. With counting disabled, full-category Numeracy and Non-Spatial scores are nearly unchanged. Few mixed prompts received binding updates, and a dedicated non-spatial objective remains unevaluated (Appendix F, Tables 9–10). The Complex gain is modest (3.6%), consistent with the greater difficulty of satisfying several constraints in the same image.
Adaptive versus fixed scheduling on Infinity‑2B. On 1,800 T2I-CompBench validation prompts with two seeds, a fixed five-step budget reaches 0.5712, within 0.0021 of adaptive scheduling (0.5733), without using a gradient threshold. Adaptive scheduling uses 3.3% fewer updates and 1.8% less generation time than the five-step policy. The second fixed policy also scores 0.5712 and matches the adaptive run’s 52,145 optimizer updates, controlling for the total number of updates (Appendix L.2, Table 17).
Hyperparameter sensitivity and the Infinity‑8B scheduler. We added a sweep over five learning rates and four step budgets on 400 prompts to show how composition and image quality vary across settings beyond the defaults (Appendix L.1, Figure 6). We selected learning rates and step budgets on T2I-CompBench training subsets and ran the component ablation on validation prompts.
The scheduler checks gradient norms before per-token normalization. On Infinity‑8B, the raw norms fell below the Infinity‑2B threshold of 10⁻³, so we lowered it to 10⁻¹² to allow updates. At that setting, Infinity‑8B effectively took all five steps per eligible scale. The reported efficiency gains are for Infinity‑2B (Appendix L). Our compute budget precluded a threshold sweep.
Thank you again for your review. We added experiments addressing your concerns about baseline comparisons, adaptive versus fixed scheduling, and transfer beyond Infinity (Generality). With discussion closing soon, we’d appreciate your reading our response and reconsidering your recommendation in light of the new evidence.
Thank you for the detailed review. We added a fixed-budget ablation, corrected the BoN result, reran TTS-VAR across four seeds, and extended the evaluation to Switti. We address your four concerns below.
Novelty, the depth objective, and comparison with SSG and VPG
Our work addresses a technically challenging problem because the discrete, multi-resolution, and stateful nature of visual autoregressive generation makes directly transferring existing alignment methods particularly difficult. To our knowledge, VISTA is the first to transfer compositional cross-attention losses originally defined for diffusion models to discrete next-scale visual autoregressive generation. We demonstrate this with CONFORM for binding, PSG for planar relations, and CountCluster for numeracy. Each scale predicts a larger token grid conditioned on earlier scales. VISTA updates the continuous input through the frozen transformer before sampling that grid, so the intervention changes the tokens available to all finer scales. In Infinity, cache isolation preserves the sampling state during auxiliary forwards. The new ablation shows that the choice of gradient normalization materially affects the result: the targeted mean is 0.5733 with per-token normalization, 0.5318 with global L2, and 0.5283 with raw gradients (Sections 3.1–3.4 and Table 17).
We narrowed the priority claim to the mechanism introduced by VISTA. To our knowledge, VISTA is the first method to backpropagate explicit compositional cross-attention losses through a frozen next-scale visual autoregressive generator to update its continuous scale inputs before discrete token sampling. SSG uses a closed-form logit update to reinforce coarse-to-fine refinement, and VPG adjusts logits to improve consistency with the generated visual prefix. Both evaluate compositional performance, but their guidance rules do not explicitly encode constraints such as attribute binding or spatial relations.
We reproduced SSG on Infinity‑2B with four T2I‑CompBench seeds. Binding scores improved, both spatial scores declined, and Overall remained essentially flat (Appendix N, Table 21). We could not locate a public VPG implementation, so Table 22 reports the authors’ GenEval results under their rewritten-prompt protocol, including gains on Two Object and Position. A matched comparison with VPG remains open. In our paired evaluation against the same Infinity‑2B backbone, VISTA improves the targeted T2I‑CompBench mean by 19.8%, with an 88.9% relative gain on 2D Spatial.
Depth objective. On the full T2I-CompBench 3D Spatial category, VISTA improves Infinity‑2B by 11.6% (0.405 → 0.452), compared with 6.7% for BoN and 3.5% and 4.9% for TTS-VAR at N=1 and N=2, respectively, under the reported settings. VISTA also improves Infinity‑8B by 2.4% (0.414 → 0.424, Table 1).
To test whether the choice of depth loss matters, we compared five depth objectives on the same 3D Spatial subset with the rest of the steering pipeline held fixed. Boundary containment achieved the highest score (0.495, +12.8% over the unsteered baseline of 0.439), followed by peak dominance (0.487, +11.0%, Appendix E, Table 8). This supports our choice of depth objective on the tested subset. On 3D Spatial, global soft IoU between noun attention maps fell by 16.1%, and the category’s image benchmark score rose by about 14% in the attention audit. The lower overlap shows that VISTA separates the noun attention maps more strongly. (Appendix I, Table 13).
BoN, TTS-VAR, and matched compute
We rewrote the BoN evaluation pipeline and reran the eight-candidate baseline, generating eight complete trajectories and selecting the image with the highest ImageReward score. The corrected BoN result is 0.514 Overall, up from the Base score of 0.470 (Table 1). TTS-VAR at (N=1) and (N=2) now uses four seeds on the same Infinity-2B backbone, prompts, and evaluator as VISTA and Base. It reaches 0.494 and 0.502, respectively, versus 0.546 for VISTA at the reported settings (Table 1 and Appendix M, Table 19).
At the settings profiled in Table 20, VISTA with ScaleKV took 6.68 s, compared with 19.01 s for TTS-VAR at N=1 and 36.37 s at N=2. In the benchmark comparison, VISTA achieved 0.546 Overall, versus 0.494 and 0.502 for the two TTS-VAR settings (Table 1). The approaches could be combined by using search to retain promising coarse prefixes and applying VISTA before sampling subsequent scales. We leave that combination for future evaluation.