- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper identifies the mismatch between nominal denoising timestep and the actual state of a sampled latent in unified multimodal models. It introduce StateWise, a state-calibrated conditioning framework for unified multimodal image generation and editing. The key idea of the method is to set the nominal noise level specified by the input timestep apart from the actual noise level of the evolving latent. A state head predicts a signed noise-level discrepancy relative to the nominal timestep and a gated adapter uses this prediction to update the DiT conditioning at each denoising step. An identity constraint suppresses unnecessary updates when the latent matches the timestep. The VAE and MLLM remain frozen, while the latent tokenizer, conditioning interface, calibration adapter, SANA-1.5 DiT, and editing-specific source-conditioning modules are trained.
As for the contributions, the paper’s main contribution is an explicit training scheme for teaching an MLLM to recognize latent-state deviations. Besides, it also provides a gated condition-calibration mechanism and integrates it into a unified generation-and-editing model using native VAE latents.
For about the validation, the final model achieves 0.90 on GenEval, 77.4 on GenEval2, and 86.22 on DPGBench. It obtains 4.28 overall on ImageEdit and 7.38 overall on GEdit-Bench-EN. In the controlled ablation, native-VAE noise-aware conditioning, noise-level augmentation, and state calibration improve GenEval from 0.86 to 0.87, 0.88, and 0.89, respectively; the final calibration step further improves counting from 82.81 to 84.38 and position from 82.25 to 84.00. Fixed-timestep trajectory tests show that the state head consistently distinguishes earlier, current, and later latent states.
In conclusion, I lean toward weak acceptance because the method is generally coherent, the controlled ablation provides a consistent signal, and the main concerns appear experimentally addressable without changing the core method.
Critical strengths
-
The paper identifies and isolates a specific conditioning failure mode and explicits state–time discrepancy modeling. StateWise keeps the nominal timestep fixed while varying the latent’s actual noise level, trains a state query to predict this discrepancy, and uses a gated adapter to update the DiT condition. The identity constraint further suppresses updates for matched states. This is important and valuable because it distinguishes state calibration from generic noisy-latent conditioning and makes the proposed mechanism directly testable.
-
The ablation studies of the paper provide evidence that the proposed training components contribute to generation quality, while the framework is integrated into a unified model that supports both generation and editing. Within the ablation setting, native-VAE noise-aware conditioning improves GenEval from 0.86 to 0.87, noise-level augmentation raises it to 0.88, and state calibration raises it to 0.89; the final step improves counting from 82.81 to 84.38 and position from 82.25 to 84.00. The final system also obtains 0.90 on GenEval, 77.4 on GenEval2, 4.28 overall on ImageEdit, and 7.38 overall on GEdit-Bench-EN.
Critical weaknesses
-
The contribution of state calibration is not isolated at the final model scale. The proposed final system uses Qwen3-VL-4B, three-stage training, and a Stage 3 mixture with editing data. However, the component ablation in Table 4 evaluates InternVL3-2B with only Stage 1/2 data and a shorter training schedule. Although Table 4 includes a no-calibration control, it is available only in this smaller setting. Therefore, I have a concern that the reported 0.88-to-0.89 improvement does not establish that state calibration drives the final model’s gains rather than other changes in the model and training setup.
-
The evidence for the proposed calibration mechanism is indirect. In D.1, the forward-noised and recursively sampled latents share text conditions but not the same clean latent or noise realization. Therefore, the 10.4–10.8% gap in noise-prediction norms does not isolate a state mismatch. In D.2, the final model only shows that the state head responds monotonically to earlier and later latents at a fixed timestep. It does not measure prediction accuracy or the resulting condition correction, and it does not evaluate the final model without calibration on generation and editing. The paper also does not report the extra inference cost of recomputing the Qwen3-VL-4B conditioning path at every sampling step.
Questions for authors
Clarification Requests:
- It would be appreciated if the authors can provide a final-scale matched ablation using Qwen3-VL-4B and SANA-1.5, with the same three-stage data and optimization budget, comparing no calibration, noise-level augmentation only, full calibration, and separate identity/state-loss ablations to eliminate my concerns.
- What is the state head’s held-out prediction error on synthetic offset pairs and on actual sampled trajectories? Please report MAE/RMSE or correlation, the distribution of gate values and correction norms, and an oracle-delta_u* or shuffled-state control.
- Can the forward-noised versus sampled-latent diagnostic be repeated with the final SANA-1.5 model at 1024 resolution using paired prompts/images and a direct measure of effective state or SNR discrepancy?
- What are the end-to-end latency, memory, FLOPs, and throughput costs under the reported 20-step CFG sampling procedure, compared with the corresponding no-calibration model? Please also clarify whether the conditional and unconditional MLLM paths are both recomputed at every step.
Suggestion for future work: A useful follow-up would compare StateWise with latent correction, time-shift sampling, and noise-awareness guidance under a matched end-to-end inference budget, including the cost of recomputing the MLLM condition.