- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper introduces Prompt-Side Aesthetic Forecasting (PSAF), which predicts how visually appealing AI-generated images will be from their text prompts alone. The authors build a large dataset and develop AesPrompt, a model that estimates both expected image quality and how much it may vary. AesPrompt outperforms existing methods and helps select better prompts before generating images. However, the dataset distribution, scorer, and problem formulation have limitations.
Critical strengths
-
the proposed method can predict aesthetic quality before image generation, thus reducing the cost of rendering and discarding images. It requires only prompt text, without access to the generator’s internals, so adding little overhead
-
as getting reliable prompt-level labels requires generating many images per prompt, which is expensive. The paper's adaptive Bayesian response sampling optimizes the overall budget and quality and effectively cuts the cost for building the dataset
-
the supplmentary code seems to be comprehensive to reproduce the reported test metrics
Critical weaknesses
-
as the main data is from a single closed sourced image model (MJ), I have a major concern over the soundness of this work. Different image models, versions, are known to have different characteristics, preferences. The experimental data built on top of single source, single model doesn't prove it generalizes to other generators. In cross-generator adapation table 2 results, it shows the transfer is weak and it's worse on newer models. a generator-specific approach which requires per-generator training makes it less attractive in practice given how frequently new models are released and how slow/expensive it could be to collect generator specific data and retrain
-
Same applies to the HPS which seems to be the main scorer. Table4's results on pickscore/clip score which are both a few years old and the it's evaluating on stable diffusion 1.5 which is certainly an outdated model. The paper would be more convincing if it also evaluated with current scorers and current generators in 6.2 exp to understand if the gain is consistent with new models/new & different scorers
-
this problem formulation does not cover image to image part and agentic workflows where a multi-turn image editing conversation is the prior. both have fast growing usage and presence in different mainstream applications, and both are more challenging/interesting
-
text to image has the culture aspect consideration and aesthetic preferences vary across cultures. how does this approach account for this diversity, and could learning from a particular generator and preference scorer reinforce or amplify cultural biases, favoring some visual traditions over others?
Questions for authors
-
Is the HPS (v3?) the main scorer for training dataset? This work may be learning to predict HPSv3's own biases rather than general human aesthetics. would need a better (small scale) exp similar to 6.2 to understand if the gain is consistent with newer models and other new scorers (or human preference)
-
how does it compare with an agentic prompt-optimization work like GPT image 2.5 thinking (chatgpt.com/images)? basically an LLM to optimize the prompt iteratively to achieve best aesthetics with better generability (observe, improve)