- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates whether generation can help understanding in unified models.
The authors investigate this question through a task called caption discrimination. They use the understanding and generation branches to compute scores, such as likelihood, and then train a linear model to combine these scores and select the correct caption between two similar captions. This design does not require generating an actual image.
They find that generation can enhance understanding, although the improvement depends on the capability of the baseline understanding model.
Critical strengths
- The caption discrimination setting does not require generating an actual image and can be adapted to different unified model architectures.
Critical weaknesses
-
The evaluation is limited to a single task, caption discrimination, which is a relatively simple binary classification problem for unified models. More complex and practical tasks, such as visual question answering and multimodal reasoning, are not investigated.
-
The paper proposes 12 Understanding Readouts and 4 Generation Readouts for score computation, but does not provide experiments to justify the choice of these numbers.
Questions for authors
Please see weaknesses.