- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
FLAT jointly trains a multimodal encoder and image-to-text and text-to-image decoders. Shared register tokens yield image and text representations with a variable-length prefix, aligned by contrastive learning and used for generation. The paper evaluates retrieval, captioning, and image synthesis; its fine-tuned image generator reaches about 0.83 on GenEval.
Critical strengths
The shared ordered-token design gives a concrete mechanism for using the same representation across retrieval and both generation directions.
Loss ablations and token-length sweeps reveal a useful distinction: retrieval is relatively stable at short lengths, whereas compositional image generation needs more tokens.
Critical weaknesses
The best retrieval, captioning, and image-generation results use separate task-specific fine-tuned checkpoints. They should not be read as the performance of a single unchanged model.
The value of joint training remains difficult to isolate because the main comparisons change backbone, data, or training budget; pretraining uses 65.7 million image–text pairs.
Flexible length has a substantial quality cost for generation: fine-tuned GenEval rises from 0.49 at one token to 0.83 at 16 tokens. Measured latency and peak-memory trade-offs are needed to assess its practical value.
Questions for authors
Can you report a backbone-, data-, and compute-matched comparison with separately trained or two-stage retrieval and generation models?
For each token length, what are measured latency, peak memory, retrieval, captioning, and image-generation results?
Which headline results come from the shared pretrained checkpoint, and which require task-specific fine-tuning?