acceptodds
Under review as a conference paper at ICLR 2027

BridgePrompt: Bridging the Implicit Meaning Gap in Text-to-Image Generation

Abstract

Text-to-image (T2I) models can render explicit prompts well, but often fail when the desired image depends on meaning that is only implicit in the user's request, such as figurative meaning, entity knowledge, physical consequences, or scientific relations. We call this failure mode the implicit visual meaning gap. We view it as a conditioning-interface problem: the prompt wording may not make explicit the visual semantics that the generator must realize, even when those semantics are recoverable from the request. This motivates a prompt-side objective of making the intended visual meaning explicit before generation. We propose BridgePrompt, which keeps both the language prompter and T2I generator frozen and optimizes only a reusable natural-language instruction using feedback on generated images from a frozen vision-language evaluator. Candidate instructions are scored by how well their generated images match the original prompt, rather than by the rewritten text itself, while inference requires only one refinement and one generation. Conditioning the generator on benchmark-provided intended-meaning annotations raises the score to , supporting the view that many failures can be repaired by making missing semantics explicit before generation. With a frozen 1.5B prompter, BridgePrompt improves SD3.5 Medium on T2I-ReasonBench from to , outperforms zero-shot GPT-4o refinement, and transfers without modification to other reasoning benchmarks and T2I models.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Text-to-Image Generation, Prompt Optimization, Multimodal Reasoning, Vision-Language Models, Semantic Grounding

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_5kNeOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies failures of text-to-image models when a prompt implies visual content through idioms, references, or physical reasoning rather than stating it explicitly. It proposes bridgeprompt, which uses GEPA to optimize a reusable instruction for a frozen language-model rewriter using image-level feedback from a frozen VLM. Experiments on T2I-ReasonBench and transfer settings evaluate whether the learned instruction improves reasoning fidelity without modifying the generator.

Critical strengths
  1. The paper addresses a real problem in text-to-image generation. T2I models sometimes struggle with prompts that require interpreting implicit meanings, resolving indirect references, or understanding physical and scientific relationships.

  2. The improvements on T2I-ReasonBench are substantial. BridgePrompt raises the score of SD3.5 Medium from 46.8 to 74.6, a gain of 27.8 points. The improvements are consistent across all four categories, with particularly strong gains on idiom and entity reasoning.

Critical weaknesses

Thank you for the additional experiments and clarifications. The cross-generator and transfer results partially address my concerns about scope and relevance, but the central methodological concern (W1) remains.

What is addressed. The Qwen-Image experiments are the most convincing new evidence for W3. Generator-specific recompilation reaches 80.5 (+13.0 over raw prompting) on a modern model with an LLM text encoder, which shows the formulation remains useful beyond SD3.5 Medium. I appreciate the honest reporting that direct transfer barely exceeds GPT-4o zero-shot refinement (73.5 vs 73.1) and degrades textual-image-design (77.1 → 67.9); this tempers the "reusable instruction" claim and is the right framing. For W2, the R2I-Bench (0.47 → 0.63) and WISE (0.45 → 0.71) transfer results are a reasonable test of distribution shift. I look forward to the released prompt IDs.

What remains unaddressed (W1). The response restates the formulation but does not provide the evidence I asked for. The central claim is that image-level feedback is the key ingredient, yet there is no ablation isolating image-level from text-level feedback — a comparison the paper itself flags as missing (§5.1 / Table 8). Two further issues compound this:

  1. The instruction guard (Table 6) carries most of the optimization gain. Without it, the score drops to ~55.4, only 1.5 points above the seed instruction (53.9). The bulk of the 53.9 → 74.6 improvement therefore depends on a hand-written engineering filter rather than the optimized instruction text. The rebuttal does not discuss this.
  2. There is still no comparison with prompt-refinement methods that train or iterate a rewriter (RePrompt, Promptist, OPT2I). The relevant question for a "frozen-prompter" claim is whether a trained rewriter closes the same gap more cheaply.

The GPT-4o-authored instruction (59.6 vs 74.6) is a useful new baseline and supports that optimization matters, but it does not isolate what is being optimized toward — image feedback versus instruction search.

Assessment. The problem is real, the gains are substantial, and the transfer and cross-generator evidence is stronger than I expected going into the rebuttal. However, the methodological contribution is still primarily an application of GEPA to T2I reinforced by a hand-written guard, and the single ablation that would establish image-level feedback as load-bearing is absent. I maintain my score as 2.

Questions for authors

No further questions

1 comment

Response to Reviewer 5kNe

We thank the reviewer for recognizing the importance of implicit visual reasoning and the improvements across all four T2I-ReasonBench categories. We address the concerns about novelty, generalization, and newer generators below.

W1 / Q1: Technical contribution beyond applying GEPA

BridgePrompt addresses the implicit visual meaning gap through instruction-level optimization. Many prompts convey an intended meaning without explicitly stating the visual relationships needed for generation. We treat this as a conditioning-interface problem and optimize the instruction guiding a frozen rewriter to make those relationships explicit.

Unlike instance-specific prompt optimization, BridgePrompt learns one instruction shared across inputs, using feedback from generated images. Both the prompter and generator remain frozen, and inference requires only one rewrite and one generation. The optimization cost is thus amortized across unseen prompts.

The results support this formulation:

  • Initial instruction: 53.9
  • GPT-4o-authored instruction: 59.6
  • Optimized instruction: 74.6
  • Hidden-constraint recovery: 19.3% to 40.1%

The 20.7-point gain over the initial instruction demonstrates the value of optimization beyond simply providing a plausible instruction. Best-of-50 sampling, CFG search, and length-control diagnostics further indicate that the gains cannot be explained solely by increased sampling or verbosity.

Our contribution is not a new GEPA algorithm, but a reusable, image-grounded optimization formulation that improves how implicit visual meaning is communicated to a frozen generator.

W2 / Q2: Generalization beyond T2I-ReasonBench

We evaluate the learned instruction on two additional benchmarks without reoptimization. On R2I-Bench, the Commonsense Reasoning (695 prompts) and Causal Reasoning (151 prompts) subsets show an improvement from 0.47 to 0.63. On WISE, the Cultural Commonsense (400 prompts) and Natural Science (300 prompts) subsets improve from 0.45 to 0.71.

These evaluations follow the respective official protocols and test distinct prompt distributions. The results demonstrate that the instruction transfers beyond the categories used during optimization. We will release the evaluated prompt IDs with our code for reproducibility.

W3 / Q3: Relevance to contemporary image generators

Our evaluation also includes SDXL, Sana-1.5, FLUX.1-dev, and Qwen-Image, extending beyond SD3.5 Medium.

On Qwen-Image, raw prompting scores 67.5, GPT-4o refinement 73.1, and direct BridgePrompt transfer 73.5. Optimizing the instruction specifically for Qwen-Image raises performance to 80.5, a 13.0-point gain over raw prompting.

These results show that stronger generators still benefit from making implicit visual constraints explicit. They also distinguish direct transfer from generator-specific optimization, which achieves larger gains.

Direct transfer does reduce textual-image-design performance from 77.1 to 67.9, indicating that instruction effectiveness depends on the target generator. Nevertheless, the aggregate improvements and stronger generator-specific result support the relevance of our formulation beyond SD3.5 Medium.

The qualitative examples highlight the failure mode. The quantitative cross-generator results, particularly the Qwen-Image evaluation, provide the stronger evidence that the proposed formulation remains useful beyond the main SD3.5 Medium setting.

Reviewer_JjvgOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that some failures to capture implicit meaning in text-to-image generation arise because the conditioning text does not explicitly describe the intended visual content. BridgePrompt freezes both the rewriting model and the image generator, then uses image-based feedback and GEPA to optimize a reusable rewriting instruction. Inference requires one rewrite followed by image generation. The method improves SD3.5 Medium’s T2I-ReasonBench score from 46.8 to 74.6 and transfers across benchmarks and generators.

Critical strengths
  1. The diagnostic experiments help distinguish sampling limitations from conditioning problems. Best-of-50 sampling and CFG search provide little improvement, while rewriting the text produces much larger gains. The length-control experiment also shows that the benefit is not simply due to adding more description.
  2. The deployment cost is modest. No model weights are fine-tuned, and inference adds only one rewriting step using a 1.5B model.
  3. The transfer results strengthen the practical case. The same instruction works on unseen benchmarks and different generators, and substantial gains remain when the optimization evaluator is changed.
Critical weaknesses
  1. The energy-based derivation is weakly connected to the implementation. The paper introduces a Boltzmann distribution and Metropolis–Hastings acceptance probabilities, but the actual GEPA optimizer does not implement this sampling process or inherit its guarantees.
  2. Direct comparisons with existing prompt-optimization methods are missing. The main baselines are the original prompt and zero-shot rewriting, which are insufficient to establish an advantage over Promptist, RePrompt, or general-purpose prompt optimizers.
Questions for authors

How does the method perform on explicit attribute and spatial-relation benchmarks such as GenEval? Can it preserve deliberately literal or surreal creative intent?

1 comment

Response to Reviewer Jjvg

We thank the reviewer for recognizing the diagnostic evidence, the low inference overhead, and the transfer results. The Best-of-50, CFG-search, and length-control experiments support our argument that the gains arise from improving the conditioning information, rather than simply increasing sampling or prompt length.

W1: Energy-based interpretation and its relationship to GEPA

The energy-based formulation is intended as a conceptual characterization of instruction quality, not as a description of the implemented optimizer.

Let J(s)J(s) denote the reward of instruction ss, and define its energy as E(s)=−J(s)E(s)=-J(s). For a reference distribution ρ(s)\rho(s) and β>0\beta>0, consider

πβ(s)∝ρ(s)exp⁡(βJ(s)).\pi_\beta(s) \propto \rho(s)\exp(\beta J(s)).

Equivalently, this distribution solves the following KL-regularized optimization problem:

πβ=argmax⁡q(Eq[J]−β−1KL(q∥ρ))\pi_\beta = \operatorname*{argmax}_{q} \Big( \mathbb{E}_q[J] - \beta^{-1}\mathrm{KL}(q\Vert\rho) \Big)

This favors high-reward instructions while regularizing against a reference distribution. GEPA does not explicitly optimize this objective.

BridgePrompt uses GEPA's feedback-informed reflection to propose instruction revisions and its empirical evaluation and selection procedure to identify effective candidates. GEPA does not sample from the Gibbs distribution above, implement Metropolis–Hastings transitions, or inherit convergence guarantees from this characterization. The mathematical formulation serves as an interpretation; the method's effectiveness is established by the experiments, not by a claimed equivalence to energy-based sampling.

W2: Comparisons with prior prompt optimizers

The main distinction from Promptist, RePrompt, and instance-specific optimization methods is what is optimized and how the result is deployed.

BridgePrompt optimizes a single instruction for a frozen rewriting model. Once selected, that instruction is reused across inputs without per-example search or policy fine-tuning. This differs from methods that train rewriting policies or iteratively optimize individual prompts.

Published results from these approaches use different objectives and experimental settings, so they cannot be treated as matched evidence on implicit visual reasoning. Instead, our evaluation isolates the role of instruction optimization within a fixed pipeline.

Best-of-50 sampling and CFG search yield limited improvements compared with rewriting, while the length-control analysis indicates that the gains are not merely attributable to longer descriptions. Together, these findings support our hypothesis that recovering and expressing missing visual constraints is more important than simply increasing sampling effort or prompt verbosity.

We position BridgePrompt as a distinct and complementary optimization formulation, rather than claiming that an existing optimizer has been replaced by a universally superior one.

Q1: GenEval and preservation of deliberately literal or surreal creative intent

We evaluate BridgePrompt on GenEval to assess whether optimizing for implicit visual reasoning compromises explicit compositional fidelity.

MethodAttribute BindingSpatial Relations
SD3.5 Medium0.530.24
BridgePrompt (Ours)0.540.23

BridgePrompt aims to recover implicit visual meaning while preserving explicitly stated constraints, rather than imposing arbitrary interpretations. Deliberately literal or surreal prompts present a distinct challenge because the intended interpretation may differ from conventional semantic associations. Our instruction explicitly encourages intent preservation, although the current evaluation does not isolate these creative cases.

Reviewer_u2kkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents BridgePrompt, a prompt refinement approach for addressing implicit visual meaning in text-to-image generation, including indirect entity references and physical or scientific reasoning. It keeps the language rewriter and image generator frozen and uses GEPA with VLM feedback on generated images to optimize a reusable rewriting instruction. At inference time, it performs one prompt rewrite followed by one image generation. On 400 held-out T2I-ReasonBench prompts, BridgePrompt achieves 74.6 with SD3.5 Medium, compared with 53.9 using the same 1.5B rewriter under the initial instruction. Additional experiments examine human preferences, evaluator sensitivity, and transfer across benchmarks and image generators.

Critical strengths
  • The method achieves substantial improvements without modifying the underlying models. With the same frozen 1.5B prompter and image generator, optimizing the rewriting instruction improves T2I-ReasonBench accuracy from 53.9 to 74.6, outperforming the GPT-4o-authored instruction baseline of 59.6. The optimized instruction can then be reused across prompts, requiring only one rewrite and one generation at inference time.

  • The optimized instruction shows useful transfer across benchmarks and image generators. Without further optimization, BridgePrompt improves performance on R2I-Bench and WISE and also produces gains on SDXL, Sana-1.5, and FLUX.1-dev. These results suggest that the learned instruction is not limited to the benchmark or generator used during optimization.

Critical weaknesses
  • The methodological novelty is limited. BridgePrompt mainly applies the existing GEPA optimizer to prompt rewriting with image-level feedback, an idea closely related to prior visual-feedback-based T2I prompt optimization methods. The additional energy-based formulation in Section 5.1 does not substantially strengthen the methodological contribution, as it is not used in the actual optimization and provides no new algorithm or theoretical guarantee. It is therefore unclear what BridgePrompt contributes beyond adapting existing techniques to this setting.

  • The experimental baselines are not sufficiently competitive. The main experiments use SD3.5 Medium and compare BridgePrompt primarily with zero-shot rewriting methods, including GPT-4o. Even the stronger GPT-4o-authored instruction in Table 8 is not iteratively optimized. Without direct comparisons with existing optimized T2I refinement methods such as OPT2I and RePrompt, it is difficult to assess the improvement over prior approaches.

  • Some experimental claims need stronger support. The reported 46.8 base score comes from the original benchmark rather than a matched run on the new test split, making the claimed 27.8-point improvement difficult to verify directly. The cross-generator results are also uneven: the transferred instruction lowers Qwen-Image's textual image design score from 77.1 to 67.9, raising concerns about its generalizability. The human evaluation, which covers only 24 prompts, provides limited additional evidence.

  • The presentation is unnecessarily theoretical in places. Section 5.1 introduces an energy-based interpretation with Metropolis–Hastings sampling, although the actual optimization relies entirely on GEPA. Since this formulation does not lead to a new algorithm or change the implemented method, its lengthy discussion adds complexity without a clear contribution.

Questions for authors
  • How does BridgePrompt compare with GEPA using text-only feedback under the same optimization budget? Such a comparison would help determine whether the improvements come from image-level feedback or instruction optimization itself.

  • Can you report raw-prompt SD3.5 scores on the identical 400 held-out prompts at CFG 4.5, 28 steps, and seeds {0,1,2}, or document that 46.8 matches this setup? Please also specify the R2I-Bench/WISE subsets, judges, and generation settings. This would resolve which headline and transfer differences are directly comparable.

2 comments

Response to Reviewer u2kk [Part 1]

We thank the reviewer for recognizing the substantial gains achieved with a frozen 1.5B prompter, the efficiency of a reusable instruction, and the transfer results across benchmarks and generators. We clarify the technical contribution and address the experimental concerns below.

W1: Methodological novelty

The contribution of BridgePrompt is the optimization formulation, not a new search algorithm. We identify the implicit visual meaning gap as a conditioning-interface problem: a user prompt may convey an intended meaning without explicitly specifying the visual constraints required to depict it. BridgePrompt addresses this by optimizing the system instruction that governs how a frozen language model translates implicit meaning into visually actionable text.

This differs from optimizing individual prompts. BridgePrompt learns one reusable instruction through image-grounded evaluation, then applies it across inputs with a single rewriting pass and one generation. Neither the prompter nor the image generator is fine-tuned, and no iterative optimization is required at inference time.

The controlled results demonstrate that this formulation is consequential. With the same frozen 1.5B prompter, the initial instruction achieves 53.9, a GPT-4o-authored instruction 59.6, and the optimized instruction 74.6. Hidden-constraint recovery also increases from 19.3% to 40.1%, directly supporting our hypothesis that making implicit visual requirements explicit improves generation fidelity.

Thus, the contribution is not simply applying GEPA to another task. It is identifying an actionable failure mode, defining an image-grounded instruction-level optimization target, and demonstrating that the resulting reusable instruction substantially improves visual reasoning without changing model weights or adding test-time search.

W2: Comparisons with optimized prompt-refinement methods

OPT2I, RePrompt, and Promptist are relevant prior work, but their published evaluations do not provide directly comparable measurements of the question studied here.

The distinction lies in the optimization target, supervision, and deployment cost. OPT2I performs iterative, instance-specific optimization; RePrompt trains a rewriting policy; and Promptist focuses primarily on prompt enhancement under a different objective. BridgePrompt instead optimizes a shared instruction for a frozen rewriter using feedback grounded in generated images.

Some of these methods provide public implementations. However, directly importing their reported scores would conflate different benchmarks, generators, evaluators, and optimization budgets. A matched comparison would require adapting and evaluating them under the same T2I-ReasonBench protocol, including the cost of any per-instance optimization.

Our controlled comparisons address a complementary question: holding the prompter and generator fixed, how much does optimizing the reusable instruction improve implicit visual reasoning? The improvement from 53.9 to 74.6, together with the GPT-4o-authored instruction baseline and diagnostic experiments, directly answers that question.

Response to Reviewer u2kk [Part 2]

W3: Experimental support, cross-generator behavior, and human evaluation

The 46.8 raw-prompt score is taken from the original T2I-ReasonBench report and should not be interpreted as a matched result on our held-out 400 prompts. The controlled improvement established by our experiments is 53.9 to 74.6, or 20.7 points, using the same prompter and evaluation setup. We will make this distinction explicit in the presentation.

The Qwen-Image results separate direct transfer from generator-specific optimization:

  • Raw prompts: 67.5
  • GPT-4o refinement: 73.1
  • Direct BridgePrompt instruction transfer: 73.5
  • Qwen-Image-specific instruction optimization: 80.5

Direct transfer improves over raw prompting, while reoptimizing the instruction for Qwen-Image yields a substantially larger gain. These are distinct experimental settings, and we will label them accordingly.

The reviewer also notes the textual-image-design regression from 77.1 to 67.9 under direct transfer. This result is important: instruction transfer is not uniformly beneficial, particularly when the target generator has different capabilities. It does not negate the positive aggregate transfer result; rather, it motivates distinguishing instruction transferability from generator-specific instruction optimization. Our claim is that the optimization formulation remains effective across the evaluated generators, not that one instruction is universally optimal.

Our human evaluation study includes 24 prompts, 13 annotators, and 312 judgments. BridgePrompt receives 53.2% preference, compared with 34.9% for GPT-4o refinement and 11.9% for neither. This provides an independent preference signal consistent with the automatic evaluation. We present it as supporting evidence, while the larger benchmark evaluations remain the primary quantitative basis for our conclusions.

W4: Energy-based interpretation

Section 5.1 is intended to provide an interpretation of instruction optimization, not to introduce a separate sampling algorithm. In particular, GEPA does not implement Metropolis–Hastings, and the paper should not suggest otherwise.

We will replace the extended discussion with the concise variational characterization provided in our response to Reviewer Jjvg. This preserves the conceptual motivation while making clear that the empirical contribution does not depend on a sampling or convergence guarantee.

Q1: Comparison with GEPA using text-only feedback

We would like to clarify the distinction between the representation of feedback and the information used to construct it.

GEPA already receives textual feedback in BridgePrompt. However, that feedback is produced by evaluating generated images with a frozen vision-language evaluator. Consequently, the optimization is grounded in visual outcomes even though the feedback communicated to GEPA is textual.

We interpret the requested text-only baseline as one in which the evaluator assesses the original and rewritten prompts without examining generated images. This is a meaningful alternative feedback source, rather than simply a different feedback format.

The current experiments establish the effectiveness of optimizing a reusable instruction using image-grounded feedback. A matched-budget text-only evaluator would further isolate the contribution of visual grounding. The absence of this ablation does not change what the existing experiment demonstrates: a frozen rewriter, guided by an instruction selected using generated-image outcomes, achieves a substantial controlled improvement in visual reasoning fidelity.

Q2: Raw-prompt evaluation and transfer protocols

Raw-prompt baseline. The reported 46.8 score is from the original T2I-ReasonBench paper and is not a matched evaluation on our held-out 400 prompts. Our controlled comparison is 53.9 to 74.6 (+20.7 points), using the same frozen prompter and generator. We will distinguish these results explicitly rather than treating the external reference as a matched baseline.

R2I-Bench and WISE protocols. For transfer evaluation, we follow the respective benchmarks' official evaluation protocols. On R2I-Bench, we evaluate the learned instruction on the Commonsense Reasoning subset (695 prompts) and the Causal Reasoning subset (151 prompts), and BridgePrompt improves the score from 0.47 to 0.63. On WISE, we evaluate on the Cultural Commonsense subset (400 prompts) and the Natural Science subset (300 prompts), and BridgePrompt improves the score from 0.45 to 0.71. These evaluations use the same learned instruction without benchmark-specific reoptimization. We will release the exact evaluated prompt IDs and implementation details alongside our code for reproducibility.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.