- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates whether the earliest scales in text-to-image visual autoregressive models need text conditioning, and uses its findings to improve best-of-N generation. Attribute probes and conditioning interventions suggest weak object-level attribute binding at the coarsest scales. The proposed unconditioned anchors (UA) sample the first two scales without text conditioning to increase candidate diversity without additional generation steps. Early attention pruning (EAP) then uses cross-attention signals to discard candidates before intermediate-image decoding, followed by preview verification in a cascade. Experiments on Infinity and Switti report diversity, mean quality, oracle best-of-N scores, and actual selection performance on compositional benchmarks. The central claim is that early unconditioning incurs little mean-quality loss while the combined pipeline improves the quality–compute trade-off.
Critical strengths
-
Some settings show useful efficiency gains. On Infinity-2B, T2I-CompBench with IVL, the cascade improves the score from approximately 0.524 to 0.538 over standard preview BoN while reducing computation by 27% and latency by 29%. The equal-compute curves provide further support.
-
The attribute probes control for object identity. The noun-only baseline separates attribute information in visual features from predictable object–attribute associations. Scale-specific conditioning interventions further support the method’s motivation.
The efficiency gains support practical value, while the controlled probes provide an empirical motivation for how early scales are treated.
Critical weaknesses
-
UA’s distinct contribution over simple alternatives remains unclear. Table 17 gives random-token mean/oracle scores of approximately 0.469/0.645 versus 0.468/0.650 for UA, without comparing actual final selection. Weak guidance also reaches 0.5351 versus UA’s 0.5380 in Table 3. Thus, higher oracle scores alone do not establish a meaningful practical advantage from the proposed mechanism.
-
The evidence does not yet establish negligible quality loss or reliable useful diversity. UA lowers mean scores on the reported Infinity-8B and Switti tasks, including Switti T2I from 0.4608 to 0.4539. EAP separately reduces GenEval2 GM from 0.3651 to 0.3540 on the same UA pool, an acknowledged pruning trade-off. Equal-compute gains are valuable but do not establish that these capability losses are minor. Layout-Vendi can also reward incorrect layouts, and final-selection gains are mixed across models and selectors. Paired uncertainty, failure severity, and diversity among valid candidates are needed to substantiate the central claims.
Questions for authors
-
How do random tokens and UA compare with the same final selector and matched candidate counts and compute, alongside standard sampling and weak guidance?
-
Building on Tables 14–16, please report paired difference intervals, the fraction of prompts with substantial degradation under a justified criterion, and key failure categories. Additional generation seeds should test the stability of the small gains and losses.
-
Does diversity increase among candidates that satisfy the prompt? Please assess candidate validity and diversity within the valid subset, with a justified criterion. A targeted independent or blinded assessment of selected-image quality could also reveal artifacts missed by compositional scores.
-
How were the pruning scale and attention layers chosen, and were the preliminary GenEval2 prompts independent of the final test set?
10 comments
Rebuttal (2/6)
W2 / Q2. Paired-difference intervals, the fraction of prompts with substantial degradation under a justified criterion, key failure categories, and additional generation seeds.
(1) Paired-difference intervals for Tables 14–16. In the revised PDF, we have added paired 95% bootstrap intervals over prompts to every row of Tables 14–16 (now 15–17 in the updated PDF). Below, we show a simplified version focusing on UA’s effect on the pool mean, EAP’s effect on the selected image, and the resulting test-time scaling gain:
Table R9. Paired differences by category: UA’s effect on the pool mean, EAP’s effect on the selected image, and the test-time scaling gain.
| Subtask (prompts) | UA: pool mean | EAP: selected image | TTS gain |
|---|---|---|---|
| color (300) | −0.027 [−0.041, −0.013] | +0.001 [−0.007, +0.010] | +0.015 [−0.002, +0.032] |
| shape (300) | +0.013 [−0.001, +0.026] | −0.008 [−0.023, +0.008] | +0.039 [+0.017, +0.061] |
| texture (300) | +0.001 [−0.013, +0.014] | +0.005 [−0.002, +0.013] | +0.026 [+0.008, +0.045] |
| spatial (300) | +0.015 [−0.001, +0.031] | +0.002 [−0.017, +0.020] | +0.074 [+0.045, +0.106] |
| 3D-spatial (300) | −0.011 [−0.020, −0.003] | +0.018 [+0.006, +0.029] | +0.003 [−0.013, +0.020] |
| non-spatial (300) | −0.001 [−0.002, −0.000] | +0.000 [−0.001, +0.001] | +0.001 [−0.001, +0.002] |
| numeracy (300) | +0.000 [−0.011, +0.011] | −0.011 [−0.022, +0.001] | +0.040 [+0.017, +0.063] |
| complex (300) | −0.008 [−0.013, −0.003] | −0.001 [−0.006, +0.004] | +0.001 [−0.007, +0.009] |
| T2I-CompBench overall (2,400) | −0.002 [−0.006, +0.002] | +0.001 [−0.003, +0.005] | +0.025 [+0.018, +0.031] |
| object (800) | −0.004 [−0.010, +0.002] | +0.006 [+0.001, +0.012] | +0.020 [+0.011, +0.030] |
| count (800) | −0.010 [−0.017, −0.002] | −0.002 [−0.011, +0.006] | +0.048 [+0.033, +0.063] |
| attribute (721) | +0.007 [−0.001, +0.014] | −0.005 [−0.015, +0.005] | +0.043 [+0.026, +0.060] |
| position (519) | +0.010 [−0.004, +0.024] | −0.011 [−0.025, +0.003] | +0.088 [+0.060, +0.117] |
| verb (82) | −0.006 [−0.039, +0.023] | +0.016 [−0.017, +0.054] | +0.061 [−0.001, +0.124] |
| GenEval2 AM (800) | −0.001 [−0.006, +0.004] | −0.001 [−0.005, +0.003] | +0.042 [+0.033, +0.052] |
| GenEval2 GM (800) | +0.001 [−0.006, +0.008] | −0.011 [−0.020, −0.001] | +0.065 [+0.046, +0.084] |
- UA: pool mean: (UA − standard), averaged over the 16 candidates. A statistically significant drop appears only in a few categories: color, 3D-spatial, non-spatial, complex, and GenEval2 count.
- EAP: selected image (UA cascade − UA BoN@s9), i.e., the effect of pruning on the selected image. Pruning makes no T2I-CompBench category and no GenEval2 skill significantly worse, and it even yields significant gains on 3D-spatial and object atoms. Its only significant loss is the GM drop the reviewer cites (0.3651 → 0.3540), which vanishes under AM. This is because GM multiplies the atom scores, so a single wrong atom pulls a sample’s score toward zero. A 30% cut is aggressive for GenEval2, and a smaller cut loses less (App. C.2). Even so, at equal compute, the cascade still outperforms best-of-N (Fig. 4 in the main paper).
- TTS gain: our method (UA cascade, N = 16) minus standard best-of-7, both with the IVL verifier. Best-of-7 is the cheapest standard best-of-N that uses at least as much compute as our cascade (cost computed as in Fig. 4). Despite significant mean losses in some categories, the practical test-time scaling gains are significant in most categories, including GenEval2 count and GM, where UA and EAP respectively incur significant costs. Gains are not significant on color, 3D-spatial, non-spatial, complex, and verb
Rebuttal (3/6)
(2) Criterion for substantial degradation. We define substantial degradation as losing the ability to solve a prompt: best-of-N fails on a prompt when none of its candidates satisfies it, regardless of how good the selector is. A candidate is valid when it satisfies the prompt under the benchmark's own criterion. On GenEval2, a candidate is valid when every atom is judged correct (Soft-TIFA atom score ≥ 0.5). On T2I-CompBench, we use the five categories whose scores can determine whether a candidate satisfies the prompt: spatial (detector score > 0), numeracy (every count correct), and color, shape, and texture (BLIP-VQA score ≥ 0.8; the conclusions hold for any threshold between 0.25 and 0.8). These cover 1,500 prompts in total. The other three categories have no such threshold: 3D-spatial is a continuous depth-based score, non-spatial is a CLIP similarity, and complex is an average of three different scorers. These are well-covered by the mean. A prompt is lost when standard sampling solves it at least k times out of N = 16 and UA never does, and gained in the reverse case. A larger k indicates a more severe loss.
As a reference, we compute the chance rate on the same 16 + 16 candidates: we pool each prompt's 32 candidates and randomly split them into two sets of 16 (2,000 permutations). This gives the loss rate expected if UA's candidates were interchangeable with standard ones. Each cell shows lost / gained rates, with the chance loss rate in parentheses. *: lost exceeds chance (permutation p < 0.05).
Table R10. Prompts lost / gained by UA, relative to chance.
| Model, benchmark (prompts) | k ≥ 1 | k ≥ 2 | k ≥ 3 | k ≥ 4 |
|---|---|---|---|---|
| Infinity-2B, GenEval2 (800) | 4.0 / 7.5% (4.7%) | 1.5 / 3.4% (1.4%) | 0.8 / 2.0% (0.5%) | 0.1 / 1.5% (0.2%) |
| Infinity-2B, T2I-CompBench (1,500) | 4.1 / 8.3% (4.7%) | 1.3 / 4.8% (1.5%) | 0.7 / 2.9% (0.6%) | 0.5* / 1.9% (0.2%) |
| Infinity-8B, GenEval2 (800) | 5.6 / 8.6% (5.1%) | 2.8* / 4.4% (1.5%) | 1.9* / 2.8% (0.6%) | 1.0* / 2.2% (0.3%) |
| Infinity-8B, T2I-CompBench (1,500) | 1.9 / 5.7% (2.7%) | 0.6 / 3.6% (1.0%) | 0.3 / 2.0% (0.3%) | 0.0 / 1.4% (0.1%) |
| Switti, GenEval2 (800) | 4.5 / 5.1% (4.7%) | 1.4 / 1.9% (1.5%) | 0.8 / 0.6% (0.5%) | 0.1 / 0.4% (0.1%) |
| Switti, T2I-CompBench (1,500) | 3.4 / 4.6% (4.0%) | 1.2 / 1.5% (1.3%) | 0.4 / 0.7% (0.5%) | 0.1 / 0.3% (0.2%) |
- UA loses prompts at the chance rate in 20 of the 24 cells. Simply redrawing the pool loses just as many.
- On Infinity, gained exceeds lost in every cell, e.g., 4.8% gained vs. 1.3% lost of T2I-CompBench prompts at k ≥ 2 on Infinity-2B.
- On Switti, both losses and gains remain at the chance level.
EAP. The same question applies to pruning: does EAP's 30% cut remove every valid candidate of a prompt? For the UA pool that the cascade prunes, we count the prompts with at least k valid candidates, none of which survives the cut. The chance rate can be computed exactly: cutting the same number of candidates at random removes all v valid ones with probability C(c, v) / C(16, v), where c is the number of candidates cut. Values in parentheses are the rates for a random cut of the same size. *: above that rate (exact one-sided test, p < 0.05).
Table R11. Prompts whose valid candidates are all removed by EAP, relative to a random cut.
| Model, benchmark (prompts) | k ≥ 1 | k ≥ 2 | k ≥ 3 | k ≥ 4 |
|---|---|---|---|---|
| Infinity-2B, GenEval2 (800) | 1.5% (2.7%) | 0.0% (0.4%) | 0.0% (0.1%) | 0.0% (0.0%) |
| Infinity-2B, T2I-CompBench (1,500) | 1.9% (2.4%) | 0.1% (0.4%) | 0.0% (0.1%) | 0.0% (0.0%) |
| Infinity-8B, GenEval2 (800) | 2.9% (2.9%) | 0.6% (0.5%) | 0.1% (0.1%) | 0.0% (0.0%) |
| Infinity-8B, T2I-CompBench (1,500) | 1.3% (1.5%) | 0.3% (0.2%) | 0.1% (0.0%) | 0.1% (0.0%) |
| Switti, GenEval2 (800) | 2.1% (2.7%) | 0.0% (0.4%) | 0.0% (0.0%) | 0.0% (0.0%) |
| Switti, T2I-CompBench (1,500) | 2.0% (2.6%) | 0.3% (0.5%) | 0.2% (0.1%) | 0.1%* (0.0%) |
- In 23 of the 24 cells, EAP removes all valid candidates no more often than a random cut does; the single exception involves 2 of 1,500 prompts.
- When a prompt has two or more valid candidates, EAP almost never removes them all (0.0–0.6%).
Rebuttal (1/6)
We thank the reviewer for the thorough and constructive review. All new results below are computed from the stored candidate pools of the paper (Infinity-2B, N = 16 unless stated). We address each point below.
W1 / Q1. Comparing ways to widen the coarsest anchors: random tokens, weak guidance, and UA.
Setup. We compare four candidate pools on all 2,400 T2I-CompBench prompts: standard sampling, random tokens, weak guidance (w=1), and UA (w=0). Values in parentheses are paired differences from UA, with 95% bootstrap confidence intervals over prompts:
Table R8. UA vs. simple alternatives under the same selector (T2I-CompBench, N = 16).
The gain comes from where the anchors are widened, not from a particular way of widening them. Our claim is not that UA is the only way to widen the anchors. Sec. 3 identifies the coarsest scales as ones that do not need the text condition; intervening there widens the candidate pool, and the wider pool leads to a better selected image. Random tokens, weak guidance, and UA are three such interventions. Among them, UA reaches the highest oracle, significantly above both alternatives, and the highest selected-image score with both verifiers, although after selection the three are statistically tied. Random tokens and UA both select a significantly better image than standard sampling with both verifiers. We use UA because it is the simplest: it changes only a guidance weight and injects no codes.
In the revised PDF, we have added the selected-image columns to Tab. 25.