$\text{“Draw Me a Moon that Looks Like a Boat”}$: Evaluating Visual Simile Canonicity in Text-to-Image Models

When a text-to-image (T2I) model is asked for "a curved moon", it will draw a crescent with ease. However, when it is prompted with a simile such as "a moon that looks like a boat", it often draws a plain boat, a moon beside a boat, a full moon, or a moon with apparent masts and sails. Such difficulty is because *"looks like"*'s implication shifts with every pair of things being compared to in the prompt, e.g, a moon that resembles a boat takes a boat's curve, while a cherry that resembles a ruby takes a ruby's color. To bridge this gap so that T2I models can better handle such prompts with implicated resemblance, we thus formalize the **Visual Simile Generation (VSG)** task: given a simile in form of "$⟨A⟩$ that look(s) like $⟨B⟩$", a T2I model must: 1., install $⟨A⟩$ as the dominant subject, 2., identify and transfer the implicated attribute from the comparison object $⟨B⟩$ onto the subject $⟨A⟩$, and 3., render no concrete features of $⟨B⟩$ literally. We curate **VSG-Bench**, consisting of 549 simile prompts spanning 13 resemblance relations over four axes. At evaluation, each prompt licenses the one attribute of $⟨B⟩$ a generation is expected to transfer, and names the concrete features of $⟨B⟩$ it must leave out. We also establish the composite CanonScore metric, which scores a generation on each of the three requirements separately: $C{S}_{anchor}$ for the subject, $C{S}_{resem}$ for the transferred attribute, and $C{S}_{bound}$ for staying clear of $⟨B⟩$'s concrete features. We benchmark 16 recent T2I systems spanning various architectures. We find no current T2I system can satisfy all three requirements simultaneously. More signifcantly, corroborated by human raters, our full paradigm effectively exposes **the resemblance-literalization tradeoff**, that the T2I systems conveying the intended resemblance attribute best are also the ones literalizing the most of the comparison objects.