- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper uses averaged upper-layer attention as a proxy for token dependency in masked generative models. It applies this proxy in two ways: (i) Dep2C-Credit, which accumulates discounted, prompt-relevant attention from future masked queries to each committed token and uses it to reweight the image-level GRPO advantage across steps and tokens; and (ii) Dep2C-Decoding, which reweights confidence by attention degree and greedily keeps strongly co-attending masked tokens out of the same parallel commit. The method is simple, needs no extra forward passes, and gives consistent gains on Show-o and Lumina-DiMOO.
Critical strengths
- This paper presents a clear and intuitive idea. The paper frames token dependency as a single concept that plays opposite roles: emphasize strong inter-step dependency during training, and avoid strong intra-step dependency during parallel decoding. The framing is easy to follow, and both components reuse attention that is already computed.
Critical weaknesses
- The credit signal sums over more future steps for early-committed tokens, so it systematically assigns higher weights to early tokens regardless of their actual dependency. This makes the step-level allocation similar in effect to timestep-reweighting methods such as MaskFocus [1], yet the paper does not compare against them. In addition, a simple baseline that assigns higher weights to earlier timesteps should also be included, to show how the different weights for tokens within the same timestep matter.
[1] G. Zhang et al., "MaskFocus: Focusing Policy Optimization on Critical Steps for Masked Image Generation," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026.
Questions for authors
- As mentioned in the weaknesses, the authors should add comparisons with (1) MaskFocus and (2) a timestep-only baseline that assigns higher weights to earlier timesteps.
- Figure 6 shows the training collapses. Does Dep2C-Credit still collapse at ≈4.4K steps using other seeds? Can the authors give some explanation on what causes the collapse in both methods?
- The main results compare Dep2C at 4K steps with Mask-GRPO at 3K steps (Fig. 6). Can the authors report the main tables under a matched training budget (like 3K for both)?
3 comments
Response— Part 2/2
Q2. Training degradation across seeds and possible causes
Question: Figure 6 shows the training collapses. Does Dep2C-Credit still collapse at approximately 4.4K steps using other seeds? Can the authors give some explanation on what causes the collapse in both methods?
The decline at approximately 4.4K steps in Figure 6 is an observation from that particular training run, not a fixed failure threshold of the method. Across our Show-o experiments covering two datasets and two reward settings, as well as the Lumina-DiMOO experiments, Dep2C delays degradation relative to the corresponding baselines. This supports consistency across the tested settings. These experiments are not, however, repetitions with different training seeds under an identical configuration; variation in the degradation onset across training seeds has not yet been systematically quantified.
One possible explanation is the interaction between noisy credit assignment under trajectory-level feedback and continued policy updates. Tokens within the same trajectory differ in their contributions to downstream predictions and final reward. Applying trajectory-level feedback to these decisions may reinforce some actions that did not actually contribute to the high reward. As updates accumulate, the resulting changes can propagate through the contexts of subsequent generation steps and affect training stability.
Dep2C refines credit assignment using downstream dependency information and may mitigate this process. Credit weighting itself, however, does not directly constrain the magnitude of policy updates and therefore cannot guarantee the elimination of long-run training instability. This is a plausible mechanism rather than an experimentally established causal explanation. Our empirical conclusion is that Dep2C delays degradation in the tested settings, not that it completely prevents collapse.
Q3. Comparison under a matched training budget
Question: The main results compare Dep2C at 4K steps with Mask-GRPO at 3K steps (Fig. 6). Can the authors report the main tables under a matched training budget (like 3K for both)?
Yes. Table 7 already reports results for all variants at 3,000 training steps on Lumina-DiMOO. The exact checkpoints used in the main tables are 3,100 steps for Mask-GRPO and 3,800 steps for Dep2C. These were selected by inspecting the online training success curves and choosing saved checkpoints from the high-success phase preceding clear degradation. The main tables report performance under this selection criterion, whereas Table 7 specifically evaluates a fixed training step count. Selected metrics from Table 7 are reproduced below:
| Configuration (all trained for 3K steps) | GenEval | Counting | Position | Attr. | UniPercept Mean |
|---|---|---|---|---|---|
| Mask-GRPO | 0.88 | 0.86 | 0.87 | 0.72 | 53.42 |
| + Dep2C-Credit | 0.90 | 0.93 | 0.92 | 0.75 | 53.42 |
| + Dep2C-Decoding | 0.88 | 0.89 | 0.87 | 0.74 | 53.65 |
| + Both components | 0.91 | 0.93 | 0.91 | 0.77 | 53.68 |
At the same 3K-step training budget, Credit alone increases GenEval from 0.88 to 0.90; the complete Dep2C method reaches 0.91, improving over the baseline by 3 percentage points. Counting, Position, and Attribute Binding improve by 7, 4, and 5 percentage points, respectively. Thus, Dep2C's gains on these metrics do not depend solely on the additional training steps used in the main tables.
We will highlight these matched-budget results in the revision and distinguish curve-guided checkpoint selection in the main Lumina-DiMOO tables, the matched 3K-step comparison in Table 7. This will allow readers to assess performance under a matched training budget separately from performance with curve-guided checkpoint selection.
Dear Reviewer gt3V,
As the discussion period draws to a close, we would like to briefly follow up on our responses above. Thank you again for the time and care you have put into reviewing our paper. Your questions helped us better examine whether the gains come from timestep weighting, within-step credit allocation, or differences in training budget.
In our responses, we provided additional MaskFocus comparisons, discussed the ablations separating step-level and within-step allocation, and highlighted the matched 3K-step results. We also clarified the checkpoint selection procedure and the limits of our evidence regarding training degradation across seeds.
We would greatly appreciate hearing whether these responses address your concerns. If any explanation or comparison remains unclear, please let us know. We will be monitoring the discussion closely and will respond as promptly as possible to any follow-up questions.
Thank you again for your thoughtful feedback and consideration.
Response — Part 1/2
We thank the reviewer for recognizing the clarity and intuition of Dep2C's unified dependency perspective. We address the concerns about timestep weighting, training degradation, and training budgets using supplementary comparisons and the existing ablations.
Q1. MaskFocus and the role of within-step credit allocation
Question: As mentioned in the weaknesses, the authors should add comparisons with (1) MaskFocus and (2) a timestep-only baseline that assigns higher weights to earlier timesteps.
Earlier-committed tokens have a longer interval over which future influence is accumulated, so the accumulation horizon can affect the distribution of credit across steps. However, credit is not determined solely by commitment time: it also depends on attention-derived dependency strength and the relevance of future masked positions to prompt entities. We use a MaskFocus comparison and hierarchical allocation ablations to examine the effects of step-level optimization and within-step token credit, respectively.
Comparison with MaskFocus at the same training step count. We reproduce MaskFocus on Show-o with K=2 and K=6 critical steps. The original MaskFocus setting selects six critical steps from a 64-step generation trajectory, approximately 10% of the total. Since our training rollouts use 20 decoding steps, K=2 approximately preserves this selection ratio, whereas K=6 preserves the original number of critical steps. These configurations examine proportional adaptation and retention of the original count, respectively.
Both configurations are evaluated at 400 training steps, matching the checkpoint step count in the main Show-o experiments. For each MaskFocus configuration, evaluation uses 64 decoding steps, CFG=3.5, and seed=42, with 2,212 images generated and evaluated across 553 prompts. The table combines the main-table results with the supplementary MaskFocus results.
Scores are rounded to two decimal places for consistency with the main tables; the unrounded overall scores for the two MaskFocus configurations are 0.7343 and 0.7310. This comparison provides additional method-level evidence at the same training step count. Dep2C's observed advantages are particularly apparent in compositional categories such as Two Objects, Counting, Position, and Attribute Binding. We next use controlled ablations to examine the two levels of credit allocation.
Separating step-level weighting from within-step credit allocation. All variants in Table 8 use a matched 1K-step training budget and are evaluated with 16 decoding steps at CFG=0:
The step-budget-only variant retains dependency-derived step weights while using uniform allocation within each step. The within-step-only variant instead fixes equal budgets across steps and varies token credit within each step, directly testing whether token-level differentiation contributes beyond step-level reweighting.
Table 10 further shows that, with equal step budgets, both moderate within-step allocation strengths (βs=0.25,0.5) outperform uniform allocation, whereas excessive concentration at βs=1 reduces the score to 0.4074. Table 9 likewise shows that increasing step-budget concentration does not continually improve performance, supporting the role of moderate credit allocation.
Together, these experiments address whether the gains arise solely from upweighting earlier timesteps: with the total budget of each step fixed, within-step allocation increases GenEval from 0.4266 to 0.4607, a gain of 3.41 percentage points; adding within-step allocation to step-level allocation further increases the score from 0.4604 to 0.4738, a gain of 1.34 percentage points. The former variant does not increase the total weight of earlier steps. Thus, under this evaluation setting, the gains cannot be fully explained by early-timestep reweighting; distinguishing tokens within the same step according to downstream dependencies provides additional value.