- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper constructs an image retargeting benchmark, which covers diverse input aspect ratios, instance counts, foreground layouts, and retargeting-sensitive content. The paper further introduces an evaluation protocol comprising four scores, including Semantic Faithfulness (SMF) and Layout Faithfulness (LOF), Scene and Texture Consistency (STC) and Structural and Visual Fidelity (SVF).
Critical strengths
The construction of 5,000 input images with 20,000 instance-level annotations (across 4 target aspect ratios) and 20,000 pixel-level reference outputs is a substantial engineering effort. The human-in-the-loop annotation pipeline (bounding, refinement, quality check) is thoughtfully designed and the matching accuracy (93–95%) is reported transparently. Evaluating seven representative methods (from classical cropping/scaling/seam carving to modern generative approaches like PruneRepaint, InGAN, GPDM, GPNN) across four aspect ratios provides useful empirical insights into the trade-offs between content preservation, layout fidelity, and visual quality.
Critical weaknesses
The pixel-level reference outputs are generated using Gemini 2.5 Flash, and the STC/SVF scores are also computed using Gemini. The reference outputs inherit any biases or systematic errors of the model. The reproducibility is compromised since results depend on a proprietary API that may change over time. The paper does not analyze how alternative generative models (e.g., other vision-language models) would produce different references or scores.
Questions for authors
Can the authors provide statistics on scene types, object categories, complexity levels, and demographic diversity (if human faces are present)? This is important for assessing the benchmark's coverage. Since the matching algorithm is not perfect, could erroneous matches artificially inflate or deflate metric scores for certain methods? Has the authors conducted any error analysis on the matching pipeline?