acceptodds
Under review as a conference paper at ICLR 2027

Seeds Teach Each Other Through Prompt Tokens: Reward-Free Compositional Text-to-Image Self-Improvement

Abstract

When a text-to-image model samples one compositional prompt with several seeds, different samples get different requirements right: one has the correct count, another the correct spatial relation. Reward and preference fine-tuning score each image as a whole, so they raise the probability of the best complete image and discard the correct parts of the others, and the parts cannot be mixed in pixel space because each seed lays out its objects differently. We observe that joint-attention transformers already provide positions that correspond across seeds: the prompt tokens. After one joint-attention block, the state of each prompt token has absorbed what its own sample rendered for that word, while its position denotes the same word in every sample. VoTEx (Visual-to-Text Exchange) uses this correspondence during training. For samples of a prompt, the next block lets every image query of a target sample attend to the prompt-token states of all samples, with a logit shift that keeps attention unchanged when the samples coincide. The resulting prediction is a dense target that we distill into the ordinary single-sample model, so architecture and inference cost stay the same. We prove that query-wise reading covers requirements with samples under a fixed evidence margin, whereas whole-image selection needs exponentially many, and that the shift is the unique scale-preserving correction. On SD3.5 Medium, FLUX.1-dev, and Qwen-Image-2512, VoTEx improves six compositional benchmarks by 2.2 to 8.9 points without any reward model, with all 18 run-level 95% intervals above zero. Filling the group with copies or other prompts removes at least 90% of the gain, and masking the sample that rendered a missed requirement correctly lowers accuracy on it four times more than masking an unrelated sample.

Primary area:
generative models
Keywords:
Text-to-Image Generation, Self-Improvement, Trajectory Coordination, Generative Models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_qMZBOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

VOTEX uses multiple samples of the same compositional prompt to construct self-distillation targets. After a joint-attention block incorporates each sample’s visual information into its prompt-token states, the next block lets image queries read prompt-token banks from all samples. A normalization term preserves attention under duplicated banks. The resulting predictions supervise an ordinary single-sample model, retaining its inference architecture and cost. Evaluation covers three backbones, six benchmarks, component controls, and donor-intervention experiments.

Critical strengths
  1. The cross-sample correspondence is appealing and technically concrete. Prompt-token positions offer a shared interface despite different image layouts. The normalized group read and dense distillation provide a plausible way to exploit complementary samples without retaining group computation during deployment.

  2. The reported empirical evaluation is extensive. Improvements are reported across three backbones and six benchmarks, with repeated training runs, human evaluation, and several informative ablations. In particular, the visual-write and cross-prompt controls help test whether the proposed information path contributes to the gains.

Critical weaknesses
  1. The group-size results contain unresolved reporting discrepancies. Table 3 reports scores of 51.4 and 58.3 at G=1 and G=4, whereas Figure 16 shows approximately 20.4 and 26.7 without identifying a different evaluation setting. The two G=4 budget tracks also report different results despite apparently identical training budgets, weakening the scaling analysis.

  2. The helpful-donor intervention is insufficiently specified. A donor selected as helpful in one bank is identified by the same index in another bank with disjoint seeds. Under independent, exchangeable sampling, that index need not remain helpful. Without the reported role-repetition rates and selection details, the masking contrast does not clearly establish selective transfer of correct content.

Questions for authors
  1. Please reconcile Table 3, Figure 16, and the G=4 budget results by specifying their datasets, metrics, checkpoints, and individual runs.

  2. How are donor roles established in the intervention bank, and what are the role-repetition rates and uncertainty estimates for the masking contrast?

3 comments
AuthorsOpenReview ↗

Response to Reviewer qMZB (1/2): group-size numbers reconciled

(1/2) Group-size results

Thank you for a review that is both kind and exacting. Your summary states the method more crisply than our own introduction (prompt-token positions as the shared interface, the normalized group read, dense distillation with unchanged deployment), and you identified exactly the two places where a reader could not verify what we wrote. We took both seriously; this post answers W1/Q1 and the next answers W2/Q2.

The three sets of numbers measure different things, and one budget was misstated. You are right that the submission did not let a reader reconcile them. Here is what each one is.

  • Table 3 of the submission (Table 4 in the revision): GenEval2 test prompts, official GM, validation-selected checkpoint, mean of three training runs (seeds 17, 29, 43).
  • Figure 16: Soft-TIFA GM on the held-out calibration prompts that select GG (App. B.2). These are deduplicated Pick-a-Pic prompts with more atoms per prompt, so the same checkpoints score about 20.4 at G=1G=1 and 26.7 at G=4G=4. The caption did not say this; it now does.
  • The fixed-trajectory sentence (55.40, 56.77, 54.31): GenEval2 test GM of the final checkpoint after the trajectory budget, not of the validation-selected checkpoint.

The two G=4G=4 tracks did not share a budget. As App. B.1 described them they would have, and your reading was the correct one. The description was wrong: the fixed-trajectory track holds 4,800 trajectories per run, the budget of G=2G=2 at 300 updates, so G=4G=4 gets 150 updates and G=8G=8 gets 75. Figure 16(c) already shows this design, since the two tracks share their G=2G=2 point and separate only at G=4G=4 and G=8G=8. We corrected App. B.1 and rewrote the paragraph in Sec. 4.4.

Every run under one protocol (new Table 8, App. C.4; GenEval2 test GM, validation-selected checkpoints):

TrackGGUpdatesRun 17Run 29Run 43MeanFinal ckpt. (submission)
fixed updates130051.3351.4151.3751.4
fixed updates230055.7156.3255.9456.055.40
fixed updates430058.0058.3558.5858.3
fixed updates830058.1257.6457.8657.9
fixed trajectories415057.0256.8157.4757.156.77
fixed trajectories87555.3654.7355.1255.154.31

The fixed-trajectory G=2G=2 entry is the fixed-update G=2G=2 configuration (same three runs). The G=4G=4 fixed-update runs are the main SD3.5 runs, and their gains over the base score of 51.34 reproduce the interval [6.24, 7.70] reported in Sec. 4.2. At G=1G=1 the loss is identically zero, so the parameters never move; those runs differ from 51.34 only through non-deterministic attention kernels at generation time.

What the corrected numbers say. At an equal budget of 4,800 trajectories, G=4G=4 still beats G=2G=2 (57.1 against 56.0), so the choice of G=4G=4 is not a side effect of extra rollouts. G=8G=8 falls to 55.1 because the same 4,800 trajectories then cover 600 prompts in 75 updates, against 2,400 prompts in 300 updates at G=2G=2: a larger group raises coverage per prompt (Theorem 1), and at a fixed budget it pays for that with prompts and updates. Validation-selected checkpoints score at or above the final checkpoints in every case, so the earlier sentence understated the fixed-trajectory track rather than overstated it.

Changes in the paper: Sec. 4.4 (rewritten paragraph), App. B.1 (budget), Figure 16 caption (prompts and score), App. C.4 with Table 8.

AuthorsOpenReview ↗

Response to Reviewer qMZB (2/2): donor roles, repetition rates, and uncertainty

(2/2) Donor roles in the masking experiment

Roles are now assigned inside the bank that is read. Your argument is correct as stated: if the intervention bank were an independent, exchangeable redraw, a donor index chosen in another bank would carry no role, and the helpful and unrelated masks would have the same expected effect. We answer in two parts.

(a) What the submission's design did, with the repetition rates. The intervention bank kept each donor's initial latent and redrew only the target's; "disjoint generation seeds" referred to the target seeds, which the text should have said (App. B.4 now does). Because the donors are not redrawn, roles mostly persist, but not always:

SD3.5FLUX
attempts22142147
helpful index correct again81.8%80.2%
unrelated index wrong again86.0%85.3%
donor accuracy on target-missed atoms (independent-redraw reference)44.1%44.7%

So the old contrast (Table 6) mixes attempts in which the role held with attempts in which it did not, and some of its target atoms were no longer missed in the new bank, which is why its full-read accuracy (75.84%) is higher than the recoveries below. It is a diluted estimate, and we no longer use it as the main evidence.

(b) The in-bank design (new Table 3, Sec. 4.3). For each GenEval2 prompt the frozen teacher draws G=4G=4 samples, and each serves once as target. An atom is missed when the target's local read, with the same noise, gets it wrong. Donor roles come from the donors' own images in the same group, so nothing is transferred. We keep cases with 1 or 2 of the 3 donors correct (1880 on SD3.5, 1832 on FLUX) and mask one donor's text bank at every step.

Recovery of missed atoms (%)SD3.5FLUX
full read53.755.0
helpful donor masked24.626.1
incorrect donor masked (matched on other atoms)49.751.3
helpful bank replaced by another prompt's20.221.9
contrast, pp [95% interval]25.1 [22.4, 27.8]25.2 [22.3, 28.1]

Intervals resample prompt clusters 20,000 times, since the four targets of a prompt share donors. The helpful mask removes 7.3 and 7.8 times as much recovery as the incorrect-donor mask; averaging over the group would predict similar drops. Table 9 adds the attention side: with only one of three donors correct, the image queries of the missed atom place 0.448 of their relevant text mass on it against 0.276 on each incorrect donor (SD3.5).

Changes in the paper: Sec. 4.3 rewritten around Table 3; App. B.4 corrected; App. C.5 with the repetition rates and Tables 9 and 10; the abstract now says "at least four times", which holds for both designs.

For the AC discussion, in brief. (i) The mechanism you found appealing now has a direct test at each step: copies or other prompts remove at least 90% of the gain (Table 5), masking the specific donor that rendered a missed requirement removes most of its recovery (Table 3), and attention favors that donor even when outnumbered (Table 9). (ii) Every group-size run is listed under one protocol (Table 8). (iii) With the strongest baselines on all three backbones, VoTEx stays best on GenEval2, T2I-CompBench++, and DPG, and on GenEval among held-out methods, without a reward model (Table 1).

Your questions led to the two clearest new pieces of evidence in the revision. If they now read as resolved, we would be grateful if you would consider raising your score and supporting the paper in the discussion.

Thank you for the rebuttal. My concerns have been addressed, and I will maintain my original rating.

Reviewer_bTzLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper first motivates by observing different seeds satisfy the given prompt each in a different way. They focus on this complementarity arising from different seeds in a text-to-image generative model, and try to seek how to leverage different advantages from multiple seeds. They propose a method of changing the attention mechanism for predicting text token at a specific transformer block. When calculating the text token, the method is to input concatenated keys, which come from different seeds and in this way, they argue that text token is aggregating image patches from multiple seeds in a way that leads to more faithful text-to-image generation. They distill the teacher model using group attention mechanism to a student model, alleviating the requirement of multiple seeds during inference time. They empirically show benefits across several benchmarks, compare mostly with other GRPO methods, and across different generative models.

Critical strengths

The idea of using multiple seeds and complement each other's missing part is interesting and seems like a sound approach. In text-to-image generation task, text or the prompt can actually act as an anchor to generation trajectory corresponding to multiple seeds, and using this as a semantic anchor in the proposed group attention seems to be plausible. Distillation to a student model is a nice idea to by pass the requirement of multiple seeds during inference time. Also, the empirical improvement seems convincing and some of the qualitative figures (e.g. fig. 6, 19, 20) is interesting to look at.

Critical weaknesses

I understand the general motivation behind the proposed group attention mechanism, but could the authors provide more justification or argument why this mechanism would identify regions that have successfully done faithful generation and complement each other's missing part while not doing in the other way (suppressing correctly generated ones)?

Can the authors include the comparison with other baselines for FLUX, Qwen in Tab. 1? Can you also compare with other methods in Tab. 2? It's bit difficult to assess the significance of the work without more comprehensive comparison with other post-training methods. Currently, the tables are showing zero or only 1-2 comparisons with other post-training methods, while there is a vast existing literature on this field.

Additionally, I found some of the description in the main text or presented figures are often hard to follow. I believe presenting simplified version of alg. 1 and deferring some figures with minor demonstration and statement of theorems to appendix could refine the exposition.

Questions for authors

Does the proposed method yield improvements even for relatively easy prompts? Furthermore, how does the performance gains become more pronounced as prompt difficulty increases? Other questions are described in the weaknesses part.

3 comments
AuthorsOpenReview ↗

Response to Reviewer bTzL (1/3): why the read favors correct evidence

(1/3) Why the group read favors correct regions and rarely overwrites them

Thank you for the review. You described the idea in the terms we had in mind (the prompt as a semantic anchor across trajectories, and distillation to avoid multiple seeds at inference), and you asked the question that matters most for the method: nothing in the read is told which donor is right, so why should it copy correct content rather than spread errors? You are right that the submission argued this only indirectly. The revision answers it with a mechanism and with direct measurements.

Short answer. The read does not know correctness, and the Limitations paragraph now says so. Three properties make it favor correct evidence and protect regions the target already renders correctly; each is measured below on the frozen teacher (GenEval2, G=4G=4).

1. Matching, not averaging. Each image query scores every prompt-token state of every sample. A token whose sample rendered its word correctly has absorbed image features that agree with the word, and the pretrained block, trained on matched text and images, scores a query that renders that word higher against such a state than against one whose rendering contradicts it. Measured with only one of three donors correct, so that the correct evidence is outnumbered:

SD3.5FLUX
relevant text mass on the correct donor0.4480.458
on each incorrect donor0.2760.271
correct donor receives the largest share (chance 0.333)0.5710.560
missed atom recovered41.0%42.6%

On SD3.5, recovery is 53.2% when the correct donor gets the largest share and 24.7% otherwise, so recovery follows the preference.

2. The target's own evidence stays in the read. The image bank remains local, the target's own text bank is one of the GG banks, and the −log⁡G-\log G shift keeps the text weight at the scale of the single-sample model. If all donors agree with the target, the output equals the base model's exactly (Eq. 2). A correct atom therefore changes only when the group disagrees with it:

Teacher: correct atom turns wrong (SD3.5)rate
all target-correct atoms1.7%
no donor wrong on the atom0.5%
one / two donors wrong2.4% / 4.8%
all three donors wrong11.4%

The same teacher recovers 45.1% of the atoms the target misses, so it fixes far more than it breaks.

3. Distillation keeps only what is predictable. The student regresses onto the conditional mean of the teacher's correction over donor draws (Eq. 5), so a wrong donor drawn by chance enters only through its average effect. In the deployed model, against the base:

SD3.5FLUXQwen-Image
missed atoms fixed29.4%30.6%24.8%
correct atoms broken3.2%3.0%1.7%

A masking test confirms the direction: removing the donor that renders a missed atom correctly lowers its recovery by 29.1 points on SD3.5, while removing an incorrect donor lowers it by 4.0 (Table 3).

Changes in the paper: Sec. 4.3 (new paragraph and Table 3), App. C.5 ("Why correct donors receive more mass", Tables 9 and 10), and the Limitations paragraph, which states that the read can be pulled the wrong way when all donors are wrong.

AuthorsOpenReview ↗

Response to Reviewer bTzL (2/3): stronger baselines on every backbone and per category

(2/3) More post-training baselines in Tables 1 and 2

The three strongest baselines now run on every backbone. You are right that the submission compared FLUX with two methods and Qwen-Image with one, which made the significance hard to judge. In the submission each baseline ran only on backbones its official code supports. In the revision, the three strongest reward-based methods on SD3.5 (DGPO, DiffusionNFT, Flow-GRPO) run on all three backbones: we port DGPO and DiffusionNFT to FLUX and Qwen-Image with their official rewards and hyper-parameters and the rollout count of their SD3.5 runs (marked ∗^*). On SD3.5 we add CoMat and Flow-GRPO trained with an atom-level partial-credit VQA reward, the closest runnable analogue of VQA-feedback methods such as SILMM. All methods are rescored with the same evaluators. Selected columns of the new Table 1 (full table in the PDF):

GenEval2DPGGenAITIFA
SD3.5 strongest prior baseline57.5 (DGPO)86.4 (DGPO)67.0 (TreeGRPO)82.6 (DiffusionNFT)
CoMat$^*$54.685.664.981.8
Flow-GRPO (atom-VQA)56.886.768.183.6
VoTEx58.388.067.283.1
FLUX DGPO$^*$58.485.968.683.9
DiffusionNFT$^*$58.185.768.984.1
VoTEx60.787.169.984.7
Qwen-Image DGPO$^*$75.690.676.688.5
DiffusionNFT$^*$75.390.476.888.7
VoTEx77.191.777.789.5

Across all added rows, VoTEx remains best on GenEval2, T2I-CompBench++, and DPG on every backbone, and on GenEval among the methods for which it is held out, with no reward model; on FLUX and Qwen-Image it also stays ahead on GenAI-Bench and TIFA. The atom-level VQA reward is the strongest added baseline on SD3.5: it leads on GenAI-Bench and TIFA, whose evaluators are close to its own VQA reward model, and trails VoTEx on GenEval2 by 1.5 points. We report this as it came out, since it shows where a per-atom reward helps and where a dense, per-region target helps more. We ran CoMat and the atom-level reward on SD3.5, and the final version will extend them to FLUX and Qwen-Image.

Table 2 for every method (new Table 12, App. C.7, all 12 categories). The pattern follows what each training signal checks. Methods trained with the GenEval detector reward gain most on numeracy, spatial relations, and DPG Count, and on FLUX and Qwen-Image they lead several single-object categories (color, attribute); on SD3.5, CoMat and the atom-level VQA reward lead on color, shape, texture, entity, and attribute, which are judged one object at a time. VoTEx leads on spatial relations, numeracy, complex 2D relations, DPG Global, Relation, and Count on all three backbones, the scene-level requirements where another sample most often supplies what one image misses (Sec. 4.2). For instance, on SD3.5 numeracy VoTEx reaches 45.5 against 45.3 for DGPO, and on color CoMat reaches 83.2 against 81.6 for VoTEx.

Training cost. Table 13 adds GPU-hours for every method and reruns the strongest baselines at the GPU-hours of VoTEx; our reply to Reviewer bNYD gives the numbers. In short, VoTEx trains in 10.6 GPU-hours per run on SD3.5, and at that budget DGPO reaches 53.6 on GenEval2.

Changes in the paper: Sec. 4.1 (baselines and porting), Sec. 4.2 (paragraph on the added rows), Table 1, App. C.7 with Table 12, App. C.8 with Table 13.

AuthorsOpenReview ↗

Response to Reviewer bTzL (3/3): exposition and gains by prompt difficulty

(3/3) Exposition, and easy versus hard prompts

The method section now starts from a short algorithm, and secondary material moved to the appendix. We followed both of your suggestions. A six-line Algorithm 1 at the start of Sec. 3 states the whole procedure (roll out the group, write, read, take the dense target, distill, deploy one sample), and the full pseudocode stays in App. B. The normalization theorem (now Theorem 2) and the duplicated-bank figure moved to App. A.2, and the student-residual figure moved to App. C.1. Sec. 3.2 keeps one sentence on why the −log⁡G-\log G shift is the unique correction. We kept Theorem 1 in the main text because the abstract's coverage claim rests on it, and the space we freed went to the evidence you and the other reviewers asked for. If a particular figure or paragraph still reads as hard to follow, please point us to it and we will rework it.

Q1: Yes, the gain is positive on easy prompts, and it grows with difficulty. We sort GenEval2 prompts into quartiles by the base model's GM on four held-out seeds, disjoint from the evaluation seeds so that sorting does not add regression to the mean (new Table 15, App. C.9). Gains in GM points with run-level 95% intervals:

Quartile of base difficultySD3.5FLUXQwen-Image
Q1 (easiest)2.31 [1.59, 3.03]2.58 [1.63, 3.53]0.94 [0.26, 1.62]
Q26.026.473.12
Q39.849.625.69
Q4 (hardest)9.719.737.61
already solved (subset of Q1)0.10 [-0.83, 1.02]0.26 [-0.68, 1.21]0.07 [-0.62, 0.77]
all prompts (Sec. 4.2)6.977.104.34

Three things stand out. First, the easiest quartile improves on every backbone, with intervals above zero. Second, the gain grows several-fold from Q1 to the two hardest quartiles, because harder prompts leave more correct atoms spread over different samples, the quantity DcompD_{\mathrm{comp}} measures. Third, on prompts the base model already solves (every atom correct on all four held-out seeds; 54, 61, and 143 prompts), the change stays within run-level noise: with no missed atom there is nothing to exchange, and the read changes little, close to the duplicate-bank case in which it changes nothing. For SD3.5 the gain levels off between Q3 and Q4, since more atoms of the hardest prompts have no correct donor at all (Table 9, c=0c=0), while the stronger Qwen-Image still gains most on its hardest quartile.

Changes in the paper: Algorithm 1 and Sec. 3.2 (exposition), App. A.2 and C.1 (moved material), Sec. 4.2 (difficulty sentence), App. C.9 with Table 15.

Closing. Your review pushed us to measure the mechanism directly (part 1), to run the strongest baselines on every backbone (part 2), and to check easy prompts (above). We believe the paper is clearer and better supported as a result. If these answers address your concerns, we would be grateful if you would reconsider your score.

Reviewer_bNYDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes VoTEx, a reward-free post-training method for compositional text-to-image generation. It starts from a simple observation: when you sample the same prompt with several seeds, different samples get different requirements right, but you can't combine the correct parts in pixel space because each seed has a different layout. So the authors use prompt tokens as the shared coordinates. After one joint-attention block, each prompt token has absorbed what its own sample rendered for that word. In the next block, the image queries of a target sample attend to the prompt tokens of all G samples. This group-conditioned prediction is then distilled into the ordinary single-sample model, so inference cost does not change. On three backbones and six benchmarks, the gains are 2.2 to 8.9 points.

Critical strengths
  • Using prompt tokens as a layout-independent correspondence between samples is the key idea, and I find it convincing. It avoids the problem that the same patch index shows unrelated content in different samples.

  • The model incurs no extra inference cost, because the group teacher is distilled into a normal single-sample model. This makes the method a practical alternative both to reward-based fine-tuning and to inference-time group methods.

  • Gains hold on three quite different backbones and six benchmarks, with three training runs and proper intervals.

Critical weaknesses
  1. The exponential sample-complexity argument in Section 2 applies to finding a fully correct image, but does not establish that reward-based learning generally requires exponentially many samples. With partial-credit rewards (e.g., atom-level VQA scores), a model can receive useful learning signals even when no sample satisfies all K requirements. The paper should distinguish the difficulty of whole-image selection from that of optimization with decomposable rewards.

  2. The union coverage in Figure 2 is susceptible to evaluator false positives because it takes a maximum over G samples. Since D_comp is derived from these judgments, its reliability also depends on evaluator accuracy. A human-verified subset or a sensitivity analysis under realistic false-positive rates would strengthen the central complementarity claim.

  3. The method relies on attention to select useful evidence from different samples, but it is unclear how reliably attention distinguishes correct from incorrect realizations of the same requirement. Although the donor-masking experiments provide supporting evidence, a quantitative analysis of whether attention preferentially selects correct donors over incorrect ones would strengthen the proposed mechanism.

  4. The empirical comparisons focus primarily on reward-based post-training methods. Other relevant approaches, such as SILMM and CoMat, are discussed but not evaluated in Table 1. A comparison with these methods, or an explanation of why direct comparisons don’t make sense would be useful. Comparisons with inference-time group methods under clearly stated compute budgets would also clarify the trade-off between training-time distillation and test-time collaboration.

Questions for authors
  1. Why is the GenEval column empty for baselines in Table 1?
  2. Could you report training GPU-hours for VoTEx and each baseline? It’d be good to know if the gains over reward-based methods hold under a comparable budget.
  3. How much of the union coverage and D_comp comes from evaluator false positives?
  4. Do image queries put more attention on same-prompt donors that satisfy the requirement than on those that don't, especially for counting and spatial relations? What happens when most donors are wrong, for example when three of four samples draw the wrong count?
4 comments
AuthorsOpenReview ↗

Response to Reviewer bNYD (1/4): partial-credit rewards and the GenEval column

(1/4) Partial-credit rewards (W1) and the empty GenEval column (Q1)

Thank you for a review that reads the paper the way we hoped it would be read. Your summary is exact, you identified prompt tokens as the key idea and found it convincing, and each of your four weaknesses points at a claim that we stated more strongly than we had tested. We addressed all four with new evidence and with changes to the text; this post covers W1 and Q1, and the next three cover W2/Q3, W3/Q4, and W4/Q2.

W1: You are right that the exponential count applies to whole-image selection, not to reward learning in general. Eq. 1 counts the samples needed to observe one complete success. That governs best-of-GG selection or a reward that credits only complete prompts. With an additive atom-level reward, a model receives a signal even when no sample satisfies all KK requirements. The revision now separates the two cases:

  • Sec. 2 limits Eq. 1 to whole-image selection and states that a decomposable reward gives signal without complete successes.
  • Proposition 1 (App. A.4) quantifies the decomposable case. For R=K−1∑qzqR=K^{-1}\sum_q z_q and a score-function estimator with a mean baseline, the squared signal-to-noise ratio of the per-atom gradient is at most vq/∑q′≠qvq′v_q/\sum_{q'\ne q}v_{q'}, which is 1/(K−1)1/(K-1) for equal atom rates. A per-atom signal of fixed quality therefore needs a number of samples linear in KK: polynomial, not exponential.
  • What remains is credit assignment. One scalar per image weights every region alike (the rank-one obstruction of Sec. 3.2), whereas VoTEx regresses a dense target that differs by position. The Limitations paragraph now states that against atom-level rewards the advantage is credit assignment plus no reward model, not an exponential gap.

We also tested it. Table 1 now includes Flow-GRPO trained with an atom-level partial-credit VQA reward on SD3.5:

SD3.5GenEvalGenEval2DPGGenAITIFA
Flow-GRPO (atom-VQA)68.356.886.768.183.6
VoTEx70.658.388.067.283.1

It is the strongest added baseline and leads on GenAI-Bench and TIFA, the benchmarks whose evaluators resemble its VQA reward model. On GenEval2, GenEval, T2I-CompBench++, and DPG it stays behind VoTEx, which uses no reward model. This is the pattern the credit-assignment argument predicts: per-atom rewards fix the exponential sample cost but still return one number per image.

Q1: Why the GenEval column was empty for baselines. Three of the five baselines (Flow-GRPO, DGPO, DiffusionNFT) use the GenEval evaluator inside their training reward, so GenEval is not a held-out test for them, and in the submission we left the column empty rather than mix held-out numbers with trained-on numbers. That decision should have been explained in the caption. The revision fills the column for every method and marks the non-held-out cells in grey with ‡, excluded from bold:

GenEval (SD3.5)score
Flow-GRPO‡ / DGPO‡ / DiffusionNFT‡92.4 / 94.1 / 92.9
MixGRPO / TreeGRPO (held out)66.1 / 65.6
VoTEx (held out)70.6

The ‡ values sit near the published 0.95, 0.97, and 0.98 of these methods, measured at their own resolution; among held-out methods VoTEx leads on every backbone.

Changes in the paper: Sec. 2 (scope of Eq. 1), App. A.4 (Proposition 1), Limitations, Table 1 caption and GenEval column, Sec. 4.2.

AuthorsOpenReview ↗

Response to Reviewer bNYD (2/4): evaluator false positives and complementarity

(2/4) Evaluator false positives in union coverage and DcompD_{\mathrm{comp}} (W2, Q3)

Complementarity survives both a stricter judge and a sensitivity analysis under generous false-positive rates. You are right that the union coverage takes a maximum over GG samples, so a single false positive among four samples marks an atom as covered, and DcompD_{\mathrm{comp}} inherits that bias. Following your second suggestion, we ran a sensitivity analysis, and we added a second, independent judge. Both use all 800 GenEval2 prompts and the same base samples as Sec. 2 (new Table 11, App. C.6).

(a) Strict labels from two judges. A second VLM judge from a different model family answers every official atom question, and an atom counts as correct only if both judges say so. This removes every false positive that the two judges do not share.

(b) Simulated false positives where they matter most. A false positive can create complementarity only when it is the single sample that covers an atom. We flip each atom covered by exactly one of the four samples to wrong with probability ff and average over 1,000 draws. We go up to f=30%f=30\%, a deliberately generous range for binary atom questions.

DcompD_{\mathrm{comp}} (points)SD3.5FLUXQwen-Image
official judge (submission)15.4014.606.69
agreement of the two judges91.8%92.4%94.6%
strict labels (both judges)14.1212.926.03
simulated, f=10%f=10\%14.5213.796.36
simulated, f=20%f=20\%13.6112.956.02
simulated, f=30%f=30\%12.7012.145.71
cross-prompt reference0.040.070.03

How to read it. The bias has the sign you anticipated: strict labels lower union coverage more than best-member coverage (on SD3.5, UU goes to 91.36 and BB to 77.24), so DcompD_{\mathrm{comp}} shrinks. It shrinks modestly, because inflating UU requires an atom that the other three samples all miss and that still draws a false positive. Three conclusions survive the corrections:

  1. Complementarity is not an evaluator artifact. Strict labels keep at least 88.5% of DcompD_{\mathrm{comp}}, and even f=30%f=30\% keeps at least 82.5%, more than 80 times the cross-prompt reference of 0.03 to 0.07 points. That reference uses the same judge with samples that share no prompt, so it bounds what judge noise alone produces.
  2. The ordering of the backbones is unchanged under every correction (SD3.5 and FLUX well above Qwen-Image), and it is the ordering of their gains in Table 1, so the link between unused complementary evidence and gain in Sec. 4.2 does not rest on evaluator errors.
  3. The gains themselves do not depend on this evaluator: they are measured on the deployed model with each benchmark's official evaluator, and the blind human study already in the submission (Sec. 4.5) prefers VoTEx for full-prompt fidelity with probability 0.633 to 0.656. Figure 8 (App. B.3) shows the spread directly: four samples of one prompt render different requirements correctly, with the deciding regions cropped per sample.

Changes in the paper: Sec. 2 reports the corrected DcompD_{\mathrm{comp}} next to the official values, and App. C.6 with Table 11 gives the second judge, the simulation, and both coverages.

AuthorsOpenReview ↗

Response to Reviewer bNYD (3/4): does attention prefer correct donors, and what if most are wrong?

(3/4) Attention to correct versus incorrect donors (W3, Q4)

Yes, image queries put more attention on donors that satisfy the requirement, including for counts and spatial relations, though less sharply there. We ran the quantitative analysis you suggested on the frozen teacher (GenEval2, G=4G=4, every prompt, each sample once as target). For each atom the target misses under its local read, we count how many of the three donors render it correctly (cc), take the image queries that render the atom (top 5% by own-bank mass on its prompt positions), and split their relevant-position text mass over the three donor banks (new Table 9, App. C.5).

Attention when only one of three donors is correct (uniform share = 0.333):

Share of relevant text massSD3.5 correctSD3.5 each incorrectFLUX correctFLUX each incorrect
all atoms0.4480.2760.4580.271
count0.3970.3010.4020.299
spatial0.4180.2910.4370.281
attribute0.4970.2520.4930.254

The correct donor gets the largest share in 0.571 of these cases on SD3.5 and 0.560 on FLUX (chance 0.333), and recovery is 53.2% when it does against 24.7% when it does not (SD3.5). The preference is weakest for counts, whose evidence is spread over several object instances, so no single prompt position carries it.

What happens when most samples are wrong. Your example, three of four samples drawing the wrong count, appears in two forms.

The target misses the atom and only one donor renders it (3 of 4 samples wrong). Recovery by the number of correct donors:

cc correct donors0123
SD3.5 recovery5.8%41.0%68.9%84.0%
FLUX recovery6.2%42.6%70.1%84.9%

With c=1c=1 the atom is still recovered in 41.0% of cases on SD3.5, and for counts in 32.7% (37.8% for spatial relations). With c=0c=0 nothing in the group can supply it, which Figure 14 shows in images and the Limitations paragraph states.

The target renders the atom correctly and all three donors are wrong. The group read turns it wrong in 11.4% of such cases on SD3.5 and 12.5% on FLUX, against 1.7% over all correct atoms. It does not flip with the majority, because the target's own image bank and text bank stay in the read (Eq. 2), and the student learns only the average correction over donor draws (Eq. 5). In the deployed SD3.5 model, 3.2% of correct atoms turn wrong while 29.4% of missed atoms are fixed (Table 10).

A causal check. Masking the donor that renders the missed atom lowers its recovery by 29.1 points on SD3.5, against 4.0 for masking an incorrect donor matched on the other atoms (Table 3; contrast 25.1 pp, 95% interval [22.4, 27.8]).

Changes in the paper: Sec. 4.3 (rewritten paragraph and Table 3), App. C.5 with Tables 9 and 10, and the Limitations paragraph, which states that the read can be pulled the wrong way when all donors are wrong.

AuthorsOpenReview ↗

Response to Reviewer bNYD (4/4): SILMM and CoMat, inference-time groups, and GPU-hours

(4/4) Other method families (W4) and training budget (Q2)

CoMat is now in Table 1; SILMM cannot run on these backbones, so we evaluate its closest analogue. CoMat replaces the reward model with image-to-text concept matching, and we port it to SD3.5 with its official loss weights. SILMM improves a large multimodal model with questions and answers that the same model generates, and its continuous variant relies on the model's own understanding head; SD3.5, FLUX, and Qwen-Image have no such head. The closest runnable analogue is per-atom VQA feedback from an external model, which is the Flow-GRPO (atom-VQA) row. On SD3.5:

SD3.5GenEval2DPGcolor (T2I)
CoMat$^*$54.685.683.2
Flow-GRPO (atom-VQA)56.886.782.6
VoTEx58.388.081.6

CoMat is strongest on per-object attributes (it leads the color column of Table 12), consistent with a caption-matching signal. The final version will add both to FLUX and Qwen-Image.

Inference-time group methods at stated compute (new Table 14). Per delivered image on SD3.5, wall-clock relative to one ordinary sample (A100, batch one):

SD3.5GenEval2DPGcompute/image
base51.3484.01.00x
VoTEx student58.3188.01.00x
best-of-4, VQAScore selector56.286.44.19x
VoTEx teacher at inference, G=4G=459.2888.31.00x

Best-of-4 recovers part of the spread between samples but pays about four generations per image. The VoTEx teacher at inference is itself an inference-time group method: it keeps all four images, adds about one point over the student at almost no extra compute per image, but requires four images to be generated jointly; distillation removes that constraint. Group Diffusion also denoises a group jointly, but it must first be trained with shared attention on each backbone and targets image quality (FID) rather than compositional faithfulness, so App. C.8 discusses it instead of porting it.

Q2: Training GPU-hours and a matched budget (new Table 13). Per run, 8 A100-80GB GPUs, BF16, including rollouts and teacher or reward-model inference:

SD3.5GPU-h/runGenEval2 at VoTEx's GPU-hours
VoTEx10.658.3
DGPO57.653.6
DiffusionNFT66.953.2
Flow-GRPO1386.451.8

On FLUX and Qwen-Image VoTEx uses 29.2 and 37.7 GPU-hours, fewer than any baseline in Table 13 on the same backbone. The most efficient reward-based method, DGPO, needs 5.3 to 5.5 times as many to converge, and when stopped at the VoTEx budget on SD3.5 the three strongest baselines keep less than half of their gain on GenEval2 and DPG. VoTEx is cheap because it runs 300 updates, needs no reward-model inference, and takes one teacher forward per sample. So the gains over reward-based methods hold, and widen, at a comparable budget.

Summary across our four replies. Each of your weaknesses now has a direct test in the paper: Proposition 1 and an atom-level VQA baseline (W1), complementarity under strict labels and simulated false positives (W2), attention and masking with roles defined in the same group (W3), and CoMat, best-of-4 selection, and GPU-hours (W4). Your review shaped a large part of the revision, and we are grateful for it. If these results resolve your concerns, we would be thankful if you would reconsider your score.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of the revision for all reviewers and the AC (1/2)

(1/2) Summary of the revision

We thank Reviewers qMZB, bTzL, and bNYD for reviews that were careful, specific, and generous with their time. Each review pointed to a place where the paper asked the reader to trust us instead of letting them check, and the revised PDF is a better paper because of it. All changes are in blue; table and section numbers below refer to the revision. The new Table 3 moves the submission's Tables 3 to 6 to Tables 4 to 7, and Tables 8 to 15 are new.

What the reviewers agreed on. All three reviewers found the central idea sound: prompt-token positions give a correspondence between seeds that does not depend on layout. Reviewer qMZB called it "appealing and technically concrete", Reviewer bNYD wrote that it is "the key idea, and I find it convincing", and Reviewer bTzL described the prompt as "a semantic anchor" for the group. All three also noted that distilling the group into the ordinary model keeps inference cost unchanged, and qMZB and bNYD singled out the breadth of the evaluation (three backbones, six benchmarks, repeated runs, a human study, and source controls).

Main changes

  1. Group-size numbers reconciled (qMZB). The submission printed three sets of group-size numbers that used different prompts, metrics, or checkpoint rules, and App. B.1 misstated the fixed-trajectory budget (4,800 trajectories, not 9,600). Sec. 4.4, App. B.1, and the Figure 16 caption are corrected, and the new Table 8 (App. C.4) lists every run under one protocol.
  2. Donor roles assigned inside the bank that is read (qMZB, bNYD, bTzL). New Table 3 (Sec. 4.3) masks helpful or incorrect donors of the same group and reports the contrast with a prompt-cluster 95% interval. Tables 9 and 10 (App. C.5) add attention shares when only one of three donors is correct, recovery by the number of correct donors, how often a correct atom is broken, and role-repetition rates for the submission's design.
  3. Wider baseline coverage (bTzL, bNYD). The three strongest reward-based baselines on SD3.5 (DGPO, DiffusionNFT, Flow-GRPO) now run on every backbone, with DGPO and DiffusionNFT ported where the official code lacks support. On SD3.5, Table 1 adds CoMat and Flow-GRPO trained with an atom-level partial-credit VQA reward. The GenEval column is filled for all methods, with ‡ marking the cells that are not held out, and Table 12 gives all 12 categories for every method.
  4. Compute (bNYD). Table 13 reports training GPU-hours of every method and reruns the three strongest baselines on SD3.5 at the GPU-hours of VoTEx. Table 14 compares best-of-4 selection and the VoTEx group teacher with the student at stated compute per image.
  5. Evaluator false positives (bNYD). Table 11 recomputes union coverage, best-member coverage, and DcompD_{\mathrm{comp}} on all 800 GenEval2 prompts with strict labels from two independent VLM judges, and under simulated false positives on the atoms covered by a single sample.
  6. Partial-credit rewards (bNYD). Sec. 2 now limits the exponential count to whole-image selection, and Proposition 1 (App. A.4) shows that an additive atom reward needs a number of samples linear in KK. The remaining advantage is credit assignment, which the Limitations paragraph now says plainly.
  7. Prompt difficulty (bTzL). Table 15 reports the gain by quartile of base difficulty, defined on held-out seeds.
  8. Presentation (bTzL). A six-line version of the algorithm now opens Sec. 3; the full algorithm stays in App. B. The normalization theorem and two supporting figures moved to the appendix, which made room for the evidence above and a Limitations paragraph.

The main text still ends on page 10. The title is unchanged, and the abstract changes by two words ("at least four times"). Every added experiment checks a claim the submission already made; none adds a new claim.

Part (2/2) below summarizes the new numbers in one table.

AuthorsOpenReview ↗

Summary of the revision (2/2): new results at a glance

(2/2) New results at a glance

ConcernNew evidence (revision)Result
Group-size runs (qMZB)Table 8: every run, one protocolAt 4,800 trajectories, G=4G=4 (150 updates) reaches 57.1 against 56.0 for G=2G=2; G=8G=8 reaches 55.1
Does the read use the right donor? (qMZB, bNYD)Table 3: roles assigned in the same groupMasking the helpful donor drops recovery by 29.1 pp on SD3.5 (FLUX 28.9); masking an incorrect donor drops it by 4.0 pp (3.7). Contrast 25.1 pp [22.4, 27.8]
Attention when most donors are wrong (bNYD, bTzL)Tables 9, 10With 1 of 3 donors correct: 0.448 of the relevant mass on it vs 0.276 per wrong donor; atom recovered in 41.0% of cases
Does the read break correct atoms? (bTzL)Table 10Teacher: 1.7% of correct atoms turn wrong (11.4% if all donors are wrong); deployed student: 3.2% broken vs 29.4% of missed atoms fixed
More baselines (bTzL, bNYD)Table 1, Table 12VoTEx stays best on GenEval2, T2I-CompBench++, DPG, and (among held-out methods) GenEval on all backbones; on SD3.5 the atom-level VQA reward leads on GenAI-Bench and TIFA and trails on GenEval2 by 1.5 points
GPU-hours (bNYD)Table 13VoTEx 10.6 / 29.2 / 37.7 GPU-h per run; stopped at that budget, DGPO reaches 53.6 GenEval2 on SD3.5 (VoTEx 58.3)
Inference-time groups (bNYD)Table 14 (SD3.5)Best-of-4 with VQAScore: 56.2 GenEval2 at 4.19x compute; student 58.31 at 1.00x
Evaluator false positives (bNYD)Table 11Strict two-judge DcompD_{\mathrm{comp}} = 14.12 / 12.92 / 6.03; with 30% simulated false positives it keeps at least 82.5%
Easy vs hard prompts (bTzL)Table 15SD3.5 gain 2.31 (easiest quartile) to 9.84 / 9.71 (two hardest); already-solved prompts 0.10 [-0.83, 1.02]
Partial-credit rewards (bNYD)Sec. 2, Prop. 1Additive atom rewards need samples linear in KK, not exponential; the remaining gap is credit assignment

Two points matter most for the overall assessment. First, the evidence that the gain comes from exchanging complementary content now rests on tests in which each step can be checked: copies or other prompts remove at least 90% of the gain (Table 5), masking the specific donor that rendered a missed requirement removes most of its recovery (Table 3), and attention favors that donor even when it is outnumbered (Table 9). Second, the comparison no longer depends on which methods happened to support which backbone: the three strongest reward-based baselines run on every backbone, and on SD3.5 they are also compared at equal training GPU-hours.

We are grateful for the reviewers' engagement and will gladly run any further check that would help the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.