- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Conflux unifies visual understanding and generation in a discrete diffusion model. It progressively refines tokens using codebook geometry and learns from both autoregressive and continuous image teachers. AR teachers supervise its autoregressive view, with consistency training transferring this knowledge to the diffusion view; the image teacher helps improve its editing outputs. Results on 22 benchmarks, together with ablation studies, show gains in reasoning, image editing, and visual quality.
Critical strengths
- Well-motivated distillation design. The paper identifies a concrete supervision mismatch and supports its proposed AR-to-diffusion transfer with multiple objective ablations and consistent reasoning gains.
- Effective progressive discrete refinement. The geometry-aware corruption process improves visual quality over retrained diffusion baselines, demonstrating a practical approach to stronger unified discrete models.
Critical weaknesses
- Incomplete attribution of gains. Large-scale training contributes substantially to the overall results. Image-editing comparisons also conflate teacher refinement with on-policy updating, leaving the latter’s independent contribution unclear.
- Limited evidence for the magnitude and robustness of refinement gains. The fixed-model gain at 64 forward passes is modest, while additional refinement can hurt at low budgets. Missing uncertainty estimates limit confidence in small visual-quality advantages.
Questions for authors
- Were all Figure 5 variants matched in initialization, data, training compute, and inference settings? How should their larger gains be reconciled with the modest fixed-model gains in Table 6?
- Does the proposed editing method outperform teacher-refined targets generated from a fixed earlier student checkpoint under matched budgets? This would isolate the benefit of on-policy updating.
- Can you provide uncertainty estimates for HPSv3 and ImgEdit, and clarify whether the reported compute includes teacher rollouts and ablations?
Q1 Were all Figure 5 variants matched in initialization, data, training compute, and inference settings? How should their larger gains be reconciled with the modest fixed-model gains in Table 6?
Yes, they have matched initialization, data, and training compute. In terms of the gap, there are two interpretations. First, Figure 5 shows ablation runs with a short schedule which is 20k steps, while the full stage 1 training takes 400k steps and is considerably more expensive, hence the setup is different. These results show that self-refinement is more data efficient and converges faster under apples-to-apples comparison. Second, we find that even if refinement is not enabled at inference, it still outperforms vanilla mask diffusion, suggesting self-refining training is helpful even if at inference we set K=1 and never update tokens after initial commits. This is because the training objective and the distribution of training-time corruption is different. Self-refinement task is “easier” than one-step mask-to-token prediction, offering smoother distribution and training dynamics.
Q2 Does the proposed editing method outperform teacher-refined targets generated from a fixed earlier student checkpoint under matched budgets?
Yes. In revision Table 4, we added an off-policy baseline which uses refined targets on frozen students, which result in worse performance.
Q3.1 Can you provide uncertainty estimates for HPSv3 and ImgEdit.
For HPSv3, we report 95% confidence interval of different refinement settings in revision Table 7, copied below
For ImgEdit , we report in revision table 6
Uncertainty on ImgEdit. We report 95% confidence intervals computed using 3 independent seeds.
Q3.2 Whether the reported compute includes teacher rollouts and ablations?
Yes
W1 Incomplete attribution of gains.
Given a final checkpoint performance, the exact factorization of improvements of OPD, self-refining diffusion, and increased training data for each task is difficult to separate because they are introduced at different training stages discussed in Section 4.1. In particular, the self-refining diffusion vs vanilla mask diffusion is studied in the pretraining stage 1 whose data is mere image-text pairs. Image editing data is introduced in stage 2 SFT, and OPD is performed in stage 3. Running multiple setups across the entire pipeline is computationally expensive, so we organize our ablation studies into grouops of component matched settings at respective training stages.
Specifically, self-refining diffusion is ablated in the pretraining stage on the text-to-image task (Figure 5), where we match the initialization, data, and training schedule. Image editing is added to stage 2, which already underwent considerable training with self-refining diffusion in stage 1. OPD is further introduced in stage 3, where we compare our proposed OPD with off-policy alternatives (Table 4) under matched compute. These experiments are initialized from the same stage 2 checkpoint. At this stage, it becomes impossible to have a fair, non-self refining baseline unless we conduct another full scale training of stage 1, 2.
While the performance gain cannot be quantitatively factorized, we emphasize that each design is accompanied by an ablation study at respective stages with matched compute and data, which establishes their effectiveness independent of other components and data scaling. In summary, self-refining ablations are conducted in stage 1 with a 20k schedule on matched initialization, schedule, and data. OPD ablations are conducted in stage 3 initialized from the same checkpoint of stage 2 training (Conflux-Base) which has self-refinement enabled, so the 3.81→4.17 ImgEdit gain in table 4 completely attributes to cross-paradigm OPD.
W2 Limited evidence for the magnitude and robustness of refinement gains
See our response in Q1,Q2, where we show the improvements are statiscally significant.