acceptodds
Under review as a conference paper at ICLR 2027

Amplify What You Gaze At: Target Saliency Boosting in Text-to-Image Generation

Abstract

Text-to-image generation has advanced in controlling what, where, and how objects appear, yet how visual attention is distributed among objects remains largely unexplored. In this paper, we introduce Target Saliency Boosting, a new task aimed at boosting the visual saliency of a specific object during text-to-image generation without requiring any visual priors. Our key insight is that visual saliency is inherently relative: boosting the saliency of a target object also depends on the global saliency distribution across all objects in the scene. Based on this insight, we propose GazeME, a lightweight framework that uses saliency-marked prompts, inserting learnable marker tokens around object descriptions to indicate which objects to visually emphasize or suppress. To learn these markers, we construct a saliency-semantics dataset that associates objects in image-prompt pairs with object-level saliency scores, and propose Saliency Prior Marker Activation (SPMA), a saliency-aware stochastic marker activation strategy that exploits relative saliency relationships for robust training. During inference, GazeME automatically inserts appropriate markers into the prompt, thereby directly enhancing the visual saliency of the target object. Extensive experiments demonstrate that GazeME effectively boosts target saliency while preserving both semantic alignment and image quality.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
visual saliency, text-to-image generation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_cvLWOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Target Saliency Boosting, a task that aims to increase the visual saliency of a specified object during text-to-image generation without requiring explicit saliency maps. The authors propose GazeME, which uses learnable focus and unfocus marker embeddings to control the relative saliency of objects. To train these embeddings, the paper constructs an automatically annotated dataset using SAM3 and AAM to obtain object-level saliency scores, and introduces Saliency Prior Marker Activation (SPMA) to generate saliency-aware marked prompts. The pretrained generative model is frozen, and only four marker embeddings are optimized using the standard flow matching objective. Experiments on FLUX.1-dev and Stable Diffusion 2.1 report improvements in target saliency over several baselines.

Critical strengths
  1. The idea of jointly enhancing target objects and suppressing non-target objects is reasonable.
  2. The proposed method is lightweight as optimizing only four marker embeddings while keeping the generative model frozen.
  3. The experimental evaluation is relatively comprehensive.
Critical weaknesses
  1. Previous works such as DreamArtist[1] and Prompt Sliders[2] have already explored using learnable text embeddings for positive and negative control of image generation. GazeME follows a similar approach by learning focus and unfocus embeddings while keeping the generative model frozen. The main difference is that GazeME uses AAM scores to assign these markers and introduces SPMA during training. It is unclear how much these additional designs improve upon existing embedding-based control methods.
  2. AAM is used both to label the training data and to calculate the main evaluation metric, Target-NSS. Although an additional evaluation with TranSalNet and a user study are provided, the user study only includes 17 prompt–target object pairs and only compares GazeME with FLUX-Editing.
  3. In Table 2, the w/o SPMA setting only activates the marker of the most salient object. This also removes the training of unfocus markers, so it is ambiguous whether the improvement comes from SPMA or simply from using both focus and unfocus markers.

[1] Dong, Z., Wei, P. and Lin, L., 2025. DreamArtist: Controllable One-Shot Text-to-Image Generation via Positive-Negative Adapter: Z. Dong et al. International Journal of Computer Vision, 133(10), pp.7037-7053.

[2] Sridhar, D. and Vasconcelos, N., 2024, September. Prompt sliders for fine-grained control, editing and erasing of concepts in diffusion models. In European Conference on Computer Vision (pp. 17-31). Cham: Springer Nature Switzerland.

Questions for authors
  1. Section 3.2 states that only four marker embeddings are optimized, while it's unclear how these tokens are initialized and incorporated into the architecture.
  2. Equations (3) and (4) rely on the nearest higher or lower saliency score, but the paper does not explain how to handle cases where multiple objects have the same saliency score.
  3. How much of the Target-NSS improvement comes from increasing the target object's saliency, and how much comes from suppressing the saliency of other objects?
Reviewer_4Hf4OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a framework called GazeME for Target Saliency Boosting in text-to-image generation, which aims to enhance the visual saliency of a specified object without requiring explicit saliency maps. GazeME consists of three key components: a saliency-semantics dataset with object-level saliency annotations, learnable marker tokens for emphasizing or suppressing objects, and a Saliency Prior Marker Activation (SPMA) strategy for learning relative saliency relationships.

Critical strengths
  1. This paper introduces an interesting task called Target Saliency Boosting, which enables users to control the visual prominence of specific objects during text-to-image generation.

  2. The proposed method is lightweight and only requires optimizing a small number of learnable marker embeddings without fine-tuning the pretrained generative model.

  3. Extensive experiments are conducted to evaluate the performance of the proposed method.

Critical weaknesses
  1. This paper uses AAM to annotate the training dataset and also adopts it for the main evaluation. Although Table 5 provides results using TranSalNet, the improvement over the strongest baseline becomes relatively small (0.5638 vs. 0.5571). More independent evaluations, such as eye-tracking experiments or additional human gaze annotations, would strengthen the conclusions.

  2. The qualitative results are somewhat limited. Only two examples are presented in Figure 3, with three additional examples in the supplementary material. More qualitative comparisons involving diverse objects, complex scenes, and different spatial layouts should be provided to better demonstrate the generalization ability of the proposed method.

  3. During inference, GazeME applies the <unfocus> marker to all non-target objects, which may affect their visibility or semantic fidelity. Moreover, the CLIPScore of GazeME (0.2820) is lower than that of the base model (0.2890). More object-level evaluations are needed to demonstrate that target saliency can be enhanced without losing or significantly altering other objects.

  4. In Table 2, removing SPMA means that only the highest-saliency object is activated during training. However, this setting does not isolate the contribution of stochastic marker activation from that of bidirectional marker supervision. Comparisons with deterministic activation of both focus and unfocus markers, as well as random marker activation, would be helpful.

  5. The Prompt Engineering baseline simply appends "focused on [o]" to the original prompt, while GazeME both emphasizes the target object and suppresses other objects. A stronger prompt-based baseline incorporating both instructions should be considered. In addition, comparisons with a standard textual inversion approach would help clarify the advantages of the proposed marker learning strategy.

Questions for authors

Please see Weaknesses.

Others:

The proposed method relies on an LLM to identify non-target objects during inference. What is the additional computational overhead of this process? How would incorrect object identification affect the final generation results?

Reviewer_y9oqOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper describes a generative model which takes into account object saliency. It is validated against several baseline models both quantitavely and qualitatively.

Critical strengths

There is very few state of the art in this domain and I think that the relation between human attention and the generated images is very important.

Critical weaknesses

The validation is done with models which do not really do the same thing due to lack of state of the art. Also CLIPscore metric is less good showing some less good semantic homogeneity.

Questions for authors

The paper is qui well written, what would be interesting is to better explain the pipeline using an example of an image so that it is easier to understand the input and outputs of each sub-part of the pipeline. It is also not very clear for me how computational expensive the whole process needs. It would be great to more clearly separate what is learnt once and what can be finetuned.

An interesting work would be to check how increasing 1 object saliency decreases the others (and not only the unfocused). Does this techniques decrease the distractors and their cognitive load in an image compared to not using saliency ? Any model reducing people cognitive load is valuable.

The applications are also not clearly suggested and it might be an important point.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.