- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes ASPO an adaptive self-correction policy for interleaved text-image generation. It allows a multimodal model to choose between refining a generation, a re-generation fallback, and termination. A new CorrectionBench is introduced to evaluate this task. Against multiple image generation baselines it receives superior quality scores.
Critical strengths
1.) The overall idea of introducing an adaptive self-correction policy appears a useful contribution. Formulating the solution as a learning problem across "refinement", "re-generation fallback", and "termination" also makes sense and is solid.
2.) Introduces CorrectionBench to evaluate the specified task.
Critical weaknesses
1.) I have various theoretical concerns with how the paper learned a tractable policy term for each image-emitting decision. Essentially, ASPO wishes to maximize the expected reward E[R(x_final)], via a surrogate loss L_IM. Is this surrogate aligned with the desired objective? Additionally, by adding noise to an intermediate latent subsequent velocity predictions may receive inputs that differ from what was seen during pre-training. So, what prevents RL fine-tuning from damaging the learned velocity field? While it makes intuitive sense to some degree, a more formal theoretical analysis would provide greater clarity.
2.) ASPO naturally uses more compute across multiple fallbacks/refinements, is it fair to compare it against single-turn image generation models? How would ASPO fare against naive Best-of-N generation with similar compute? The key point is, are improvements over baselines simply due to more compute?
3.) CorrectionBench gives a flawed context asking ASPO to correct it. However, given an image that is correct, it does not test if ASPO can choose “terminal” rather than unnecessarily refining/regenerating the image.
Questions for authors
Please see Critical Weaknesses 1-3, which contain my main questions