acceptodds
Under review as a conference paper at ICLR 2027

Draw What LLMs Intend: Prompt Alignment Is Not Enough for Text-to-Image Generation

Abstract

In Text-to-Image (T2I) generation, which image should we generate among multiple candidates that all satisfy the same prompt? T2I generation is no longer a prompt-to-image black box, but a pipeline in which an upstream model resolves semantics before a generator renders the image. This shift matters because prompts are structurally under-specified and permit multiple images that all preserve the same prompt anchors. Yet prior work primarily measures visual quality or prompt-image alignment, leaving unaddressed whether the generator faithfully realizes the semantic decisions formed by its upstream model. We argue that, among prompt-satisfying images, the visual commitment formed by the upstream model should be carried through to the final image; we call this problem Latent Intent Transfer (LIT). In this paper, we study LIT as a new axis for evaluating and improving T2I generation systems. We formulate LIT as the second-stage faithfulness problem of a modern T2I pipeline: whether the generator realizes the upstream model's resolved visual commitments. For evaluation, we introduce two metrics, intent win rate and open-slot commitment accuracy, along with DaLI-Bench, a controlled open-slot benchmark where multiple prompt-valid images remain possible. Guided by the four desired properties, we propose DaLI, a plug-and-play method for improving existing T2I pipelines. Extensive experiments show that DaLI consistently improves LIT while preserving prompt-anchor fidelity and visual quality.

Primary area:
generative models
Keywords:
Text-to-Image Generation, Latent Intent Transfer, Prompt Alignment

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_yAYZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Latent Intent Transfer (LIT), the problem of whether the generator in a T2I pipeline realizes the visual commitment resolved by its upstream model when the prompt leaves visual properties under-specified. To evaluate LIT, it proposes two metrics, intent win rate (IWR) and open-slot commitment accuracy (OpenAcc), and DaLI-Bench, which is constructed by masking seven types of visual attributes from existing prompt-alignment benchmarks. It also proposes DaLI, which trains only the bridge modules with a DPO-style loss on preference pairs derived from the upstream model, together with prompt-anchor and quality regularization. DaLI is evaluated on ELLA, Bifrost-1, Janus, and Janus-Pro, with cross-model and human judges used for robustness analysis.

Critical strengths
  • The paper points out a real gap in existing evaluation. Quality metrics assess overall image quality and prompt-alignment benchmarks check whether explicit prompt anchors are satisfied, so neither can tell which image is preferable when several images satisfy the same prompt. Focusing on the upstream-to-generator stage rather than on prompt rewriting or refinement is a clear distinction from prior work, and this problem setting is the main reason I find the paper worth considering.

  • DaLI-Bench provides a controlled setting for under-specification by masking attribute spans from existing prompt-alignment benchmarks. DaLI is applied to four pipelines from two categories (two latent-bridged hybrids and two unified architectures) while training only the bridge modules on a single GPU (Table 2), which makes the setup practical to reproduce.

Critical weaknesses
  • The necessity of following the upstream model's commitment is not sufficiently justified. Although the paper explicitly distinguishes LIT from user intent (Appendix G.2), Section 3.1 motivates the problem using the user's desired image, while human evaluation relies on intent inferred from the prompt (Appendix C.3). The paper does not convincingly explain why the upstream commitment should be the target, particularly when it may amplify model biases. Since this is the premise of the proposed formulation, it is my main concern.

  • The evaluation does not clearly establish that the improvements come from upstream-specific intent transfer. Cross-IWR judges infer intent independently from the open prompt, rather than using the upstream model's actual commitment. Despite low cross-model commitment agreement (48.0%, Table 4), external judges and humans show similar preferences for DaLI (54.1–61.1%, Table 3). This raises the possibility that the gains reflect general visual preferences rather than upstream-specific intent. Without comparison to training using existing human-preference signals (e.g., ImageReward or PickScore), the necessity of upstream-based supervision remains unclear.

Questions for authors
  • (Clarification) Why is faithfulness to the upstream model's visual commitment a desirable objective independently of user preference, especially when the commitment is not visible to users?
  • (Clarification) Given that cross-IWR judges infer intent from the open prompt rather than the upstream commitment, how do these results support upstream-specific intent transfer?
  • (Additional result) How does DaLI compare with the same selective adaptation using human-preference reward signals (e.g., ImageReward or PickScore), particularly in self-IWR, cross-IWR, and OpenAcc?
3 comments

We sincerely thank the reviewer for the insightful review and constructive feedback. We address every concern below and have revised the manuscript accordingly. All changes in the revised manuscript are highlighted in blue for ease of reference.


[W1/Q1] Why Follow the Upstream Model?

[A1] We agree that the upstream model’s commitment does not necessarily represent the user’s ideal output. However, within a multi-stage T2I pipeline, the upstream model is the component responsible for interpreting the prompt and communicating its resolved visual decisions to the downstream generator. If the generator fails to realize those decisions, the components are internally misaligned: the pipeline does not faithfully execute its own intermediate representation, even when the final image remains plausible or prompt-valid. LIT isolates and measures this inter-component faithfulness, while whether the upstream decision itself reflects the user’s preference remains the separate user-to-upstream alignment problem.


[W2/Q2] Cross-IWR Does Not Prove Specificity.

[A2] We agree that cross-IWR and human-IWR do not by themselves establish upstream specificity, because these judges do not observe the designated interpreter’s commitment; their role is external robustness rather than target identification. Absolute IWR values are also not directly calibrated across judges because their tie rates differ substantially, but this does not resolve the reviewer’s broader specificity concern. On the commitment-disagreement subset, DaLI changes agreement of Bifrost-1 with the designated interpreter (Qwen2.5-VL-7B) by +4.2 percent points, versus -2.1 percent points for the alternative interpreter (InternVL3.5-4B), yielding a specificity difference of 6.3 percent points. We accordingly describe the learned signal as interpreter-specific alignment rather than using cross-IWR itself as evidence of specificity.


[Q3] Missing Human-Preference Baseline.

[A3] To isolate the contribution of upstream supervision, we are training the same selective bridge adaptation using upstream, external-VLM (Claude), ImageReward, and PickScore labels while holding the candidate pairs, objective, and training budget fixed. Before adaptation, we verified that these alternatives provide meaningfully different supervision: their agreement rates with the upstream preference labels are 55.5% for Claude, 57.7% for ImageReward, and 55.4% for PickScore. Training and evaluation of the matched variants are currently in progress, and we will update this response with their self-IWR, cross-IWR, and OpenAcc results as soon as all runs are completed. This controlled comparison will directly determine whether upstream supervision provides an interpreter-specific advantage over generic model- or human-preference reward signals.

Reviewer_yAYZOpenReview ↗

Response to author

I thank the authors for the response. I appreciate that the authors acknowledge that cross-IWR and human-IWR do not establish upstream specificity, and the analysis on the commitment-disagreement subset is a helpful first step, although it is limited to one pipeline and one alternative interpreter. The controlled comparison with ImageReward, PickScore, and external-VLM labels is exactly the experiment needed to assess whether upstream supervision is necessary, and I look forward to the results.

Thank you for the follow-up and for recognizing the value of the commitment-disagreement analysis. We agree that its current scope (one pipeline and one alternative interpreter) should be interpreted as targeted evidence rather than a comprehensive demonstration across all architectures.

For the matched training-label comparison, the results are now complete and we attach them below. We train the same selective bridge adaptation with identical candidate pairs, objectives, and training budgets, varying only the source of the preference labels. All IWR values compare each adapted model against the same unadapted Bifrost-1 baseline; self-IWR is evaluated by the designated upstream model ϕ\phi, cross-IWR by InternVL-3.5-4B, and Δ\DeltaOpenAcc against the commitment elicited from ϕ\phi. Brackets indicate 95% confidence intervals.

Label SourceSelf-IWR [%]Cross-IWR [%]Δ\Delta OpenAcc [%]
ϕ\phi (DaLI)55.8 [54.0, 57.5]54.1 [51.3, 56.9]+3.1 [+0.8, +5.4]
Claude (Sonnet 5)47.6 [46.0, 49.2]48.2 [46.2, 50.2]-1.0 [-3.3, +1.3]
ImageReward50.2 [48.7, 51.8]50.5 [47.9, 53.1]+0.3 [-1.9, +2.6]
PickScore49.3 [47.3, 51.4]46.4 [43.5, 49.3]-0.5 [-2.9, 1.9]

Only upstream-ϕ\phi supervision yields consistent improvements across all three metrics: both self-IWR and cross-IWR are significantly above 50%, and its OpenAcc gain is the only one whose confidence interval excludes zero. In contrast, the external-VLM and human-preference reward signals do not reproduce the OpenAcc improvement under the otherwise matched adaptation procedure. These results suggest that the reported LIT gain depends on supervision from the designated upstream interpreter ϕ\phi rather than bridge adaptation with generic preference labels alone.

Reviewer_MhuTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Latent Intent Transfer (LIT), a new perspective on text-to-image generation that emphasizes faithfully transferring the upstream language model’s implicit visual decisions to the downstream generator, beyond conventional prompt alignment and visual quality. To evaluate LIT, the authors propose two metrics, Intent Win Rate (IWR) and Open-Slot Commitment Accuracy (OpenAcc), along with DALI-BENCH, a benchmark designed to assess under-specified visual attributes. They further develop DALI, a plug-and-play adaptation framework that improves latent intent alignment while preserving prompt fidelity and visual quality. Experiments across diverse T2I pipelines demonstrate the effectiveness of the proposed approach.

Critical strengths
  1. The paper highlights a potentially meaningful limitation of conventional T2I evaluation: multiple generated images may satisfy the same prompt while differing in their consistency with the visual decisions made by an upstream model. This perspective provides a useful motivation for investigating alignment within multi-stage generative pipelines.
  2. The authors introduce two complementary metrics, IWR and OpenAcc, and construct DALI-BENCH to evaluate under-specified visual attributes.
Critical weaknesses
  1. The novelty of this paper seems limit. The proposed DALI framework primarily combines established techniques, including DPO-style pairwise preference optimization, cosine-similarity regularization, output consistency regularization, and selective adaptation of bridge modules.
  2. The definition of latent intent is conceptually ambiguous, and its distinction from model preferences is insufficiently established. The paper treats the upstream model's pairwise image preferences and filled prompts as observations of its latent visual intent. However, these preferences are brought by the upstream model. Can it represent the raw intention? If cannot, why the generator should faithfully realize the visual commitments by the upstream model, minimizing the discrepancy in Stage 2? Consequently, it remains unclear whether the proposed framework genuinely measures and transfers latent intent or simply aligns the generator with the upstream model's observable preferences.
  3. The experimental comparisons are insufficient to isolate the benefits of the proposed approach. Although the authors evaluate multiple T2I pipelines, most quantitative comparisons are conducted against their corresponding unadapted versions. There are limited comparisons with alternative preference-learning approaches, established diffusion alignment methods, and other parameter-efficient adaptation strategies under matched training conditions.
Questions for authors

The most important question is the difference between the previous T2I generation evaluation method and the proposed method. It seems that this work just additionally consider the upstream model. What is the different between this work and other prompt learning or prompt optimization works?

1 comment

We sincerely thank the reviewer for the insightful review and constructive feedback. We address every concern below and have revised the manuscript accordingly. All changes in the revised manuscript are highlighted in blue for ease of reference.


[W1/Q1] Limited Methodological Novelty and Comparison with Prompt Optimization.

[A1] DaLI builds on established preference optimization, regularization, and parameter-efficient adaptation components, and we do not claim that these components are individually new generic techniques. Our primary contribution is the LIT problem and evaluation framework, while DaLI is a lightweight interface-level instantiation that localizes both supervision and intervention at the upstream-generator bridge. DaLI demonstrates that the identified mismatch can be at least partially mitigated without full-model retraining and with modest computational overhead. More broadly, we aim to establish upstream-to-generator consistency as an important consideration in the future evaluation and development of T2I systems. We claim that the main contribution of the paper is not the algorithmic approach itself, but the research question, formulation, evaluation framework, and an on-the-fly method to minimally fulfill the desired properties an optimal T2I framework should follow.


[W2] Ambiguous Latent Intent.

[A2] Observable pairwise preferences and filled prompts should not be interpreted as a raw human intention or as a directly observed hidden variable. We clarify that the target is operationally defined as a reproducible commitment elicited from a designated upstream interpreter under a fixed protocol; LIT then evaluates conditional consistency between that target and downstream generation. This objective does not assert that the commitment is universally correct, user-preferred, or free from bias. DaLI therefore does not correct an upstream interpreter that formed an inappropriate commitment. Our claims are therefore limited to upstream-conditioned interface alignment in the second stage rather than recovery of an unobserved intention.


[W3] Insufficient Matched Baselines.

[A3] We do not claim universal superiority over diffusion-specific alignment methods that are not directly applicable to all evaluated pipeline families. Nevertheless, we agree that a detailed ablation study would clarify the impact of each proposed idea. Together with the existing parameter-matched generator-LoRA comparison in Table 5, we provide a specified analysis on commitment-disagreement subset to check the specificity of our approach. On the commitment-disagreement subset, DaLI changes agreement of Bifrost-1 with the designated interpreter (Qwen2.5-VL-7B) by +4.2 percent points, versus -2.1 percent points for the alternative interpreter (InternVL3.5-4B), yielding a specificity difference of 6.3 percent points. We accordingly describe the learned signal as interpreter-specific alignment rather than using cross-IWR itself as evidence of specificity.

Reviewer_Eu57OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies latent intent transfer (LIT): whether a text-to-image generator realizes visual commitments made by its upstream model when the prompt leaves some attributes unspecified. It introduces DALI-Bench, two evaluation metrics, and DALI, which combines preference optimization and reference-based regularization while adapting lightweight bridge modules. Experiments evaluate commitment fidelity, prompt alignment, visual quality, and adaptation efficiency across several generation pipelines.

Critical strengths
  • The paper identifies a useful evaluation problem. Distinguishing prompt satisfaction from consistency with an upstream model’s visual choices is a meaningful perspective on multi-stage generation. DALI-Bench provides a controlled setting for examining seven types of unspecified visual attributes.
  • The reported results are encouraging across several architectures. OpenAcc improves on ELLA, Bifrost-1, Janus, and Janus-Pro. Cross-model and human evaluations show consistent overall preference trends, and Table 5 provides useful evidence for the trade-off achieved by bridge adaptation and regularization.
Critical weaknesses
  • The algorithmic contribution is relatively limited. DALI combines a reference-relative DPO-style objective, regularization of conditioning vectors and generator predictions, and bridge adaptation with frozen backbones (§5). These designs build on established approaches, including Diffusion-DPO [1], T2I-Adapter [2], and VLM-based feedback for prompt optimization [3]. Table 5 supports the practical design choices, but the methodological insight specific to LIT remains unclear. Although the problem formulation may offer independent value, the experiments do not isolate the contribution of upstream-generated preference labels relative to external-VLM labels under the same adaptation procedure.

  • Preservation of prompt-following capabilities beyond DALI-Bench remains unclear. Prompt-anchor fidelity is an explicit objective (§4.2), but its evaluation primarily uses prompt-anchor accuracy on DALI-Bench. COCO-30K FID and perceptual-quality metrics provide useful quality checks, but do not establish preservation of compositional and semantic prompt-following abilities under other prompt distributions. A before-and-after comparison on an independent standard benchmark would help establish the scope of this preservation claim.

  • The practical advantage over direct prompt completion remains unclear. OpenAcc already elicits explicit completions from the upstream model, making completion a natural alternative. In Table 11, DALI with the original prompt achieves 48.6 OpenAcc versus 47.8 for completion alone. Combining both reaches 49.9, a 2.1-point gain over completion alone, supporting complementarity. However, the reported confidence intervals concern within-method changes from original to completed prompts, not these direct between-method contrasts. Beyond the adaptation efficiency reported in Table 2, a comparison with completion overhead would help clarify when the additional preference-data construction and training are worthwhile.

Questions for authors

See Weaknesses.

References

[1] Wallace et al. Diffusion Model Alignment Using Direct Preference Optimization. CVPR, 2024.

[2] Mou et al. T2I-Adapter: Learning Adapters to Dig Out More Controllable Ability for Text-to-Image Diffusion Models. AAAI, 2024.

[3] Yang et al. Self-Rewarding Large Vision-Language Models for Optimizing Prompts in Text-to-Image Generation. Findings of ACL, 2025.

1 comment

We sincerely thank the reviewer for the insightful review and constructive feedback. We address every concern below and have revised the manuscript accordingly. All changes in the revised manuscript are highlighted in blue for ease of reference.


[W1] Limited Algorithmic Novelty.

[A1] We agree that DaLI builds on established preference optimization, regularization, and parameter-efficient adaptation components, and we do not claim that these components are individually new generic techniques. Our primary contribution is the LIT problem and evaluation framework, while DaLI is a lightweight interface-level instantiation that localizes both supervision and intervention at the upstream-generator bridge. DaLI demonstrates that the identified mismatch can be at least partially mitigated without full-model retraining and with modest computational overhead. More broadly, we aim to establish upstream-to-generator consistency as an important consideration in the future evaluation and development of T2I systems.


[W2] Out-of-Benchmark Prompt Preservation.

[A2] We agree that DaLI-Bench prompt accuracy does not by itself establish preservation of general compositional prompt following. We therefore evaluate Bifrost-1 and Bifrost-1+DaLI on all 553 GenEval prompts under a matched one-seed protocol: the overall score changes from 81.2% to 82.0% (Δ=0.8\Delta=0.8% [-0.3%, +2.0%]). Similar to the overall difference, the 95% confidence interval of prompt accuracy difference across all categories includes zero, which indicates the preservation of prompt accuracy.


[W3] Unclear Advantage over Prompt Completion.

[A3] We agree that prompt completion is a strong training-free baseline. However, they require additional LLM inference in the decoding phase, and may be considered as text-mediated systems in a broad point of view (in Figure 1(b)), where the prompt completion methods perform as an upstream LLM while the original T2I pipeline works as the generator. Prompt completion methods also target the first phase of the T2I generation pipeline (as stated in Appendix F.2), which is thus not the direct competitor of DaLI, but are complementary sources when jointly adapted. Additionally, we analyze the behavior of Bifrost-1 under completed prompts. Even though the prompt completion method filled out 76.3% of the open slots, only 25.4% of them are faithfully rendered in the image. These continuous failure modes even strengthen the importance of our research targeting the second stage, aligning the upstream LLM and the generator itself.

Reviewer_A1sAOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The authors argue that many modern text-to-image systems are no longer a direct text-to-diffusion pipeline; instead, an upstream large model ϕ\phi sits in front (rewriting the prompt, extracting hidden features, or serving as a unified multimodal backbone) and passes information to the downstream image generator G\mathcal{G} through a bridge module β\beta. User prompts are often under-specified (for example "a red bowl and a cup" leaves unspecified the cup's color, material, and background), while existing alignment metrics only check whether the bowl is red and whether a cup is present. The authors claim that the upstream model ϕ\phi forms some kind of internal visual commitment (latent intent) for these unspecified attributes, and that the downstream generator should faithfully render it. They name this two-stage alignment problem Latent Intent Transfer (LIT).

To evaluate LIT, the authors take existing datasets and remove one attribute word using lexical rules (seven categories: color, material, size, etc.) to construct DaLI-Bench, and design two metrics. The first, IWR, asks a judge model (by default the upstream model ϕ\phi itself) to make a pairwise preference judgment between two images given the open prompt and the type of the missing slot, and reports the win rate. The second, OpenAcc, first has ϕ\phi fill in the missing attribute in text, then asks an external judge whether the generated image matches that attribute.

Methodologically, the authors propose DaLI: freezing the upstream ϕ\phi and the generator G\mathcal{G} and fine-tuning only the bridge module β\beta. The loss has three parts: a DPO-style preference loss LprefL_{\text{pref}} based on the upstream model's preferences, a cosine regularizer LancL_{\text{anc}} that keeps the bridge output from drifting too far from the original conditioning, and a quality regularizer LqualL_{\text{qual}} that limits how much the generator's prediction changes.

Critical strengths
  • The perspective is novel. Splitting today's LLM-assisted text-to-image systems into a "user to upstream model" stage and an "upstream model to generator" stage, and asking whether the bridge layer transmits the upstream model's implicit tendencies, is a question worth discussing.
  • Broad evaluation coverage. The authors evaluate the LIT metrics on six systems spanning three categories — text-mediated (VPGen, RPG), latent-bridged (ELLA, Bifrost-1), and unified architectures (Janus, Janus-Pro) — and apply DaLI to the four systems in the latter two categories. On Bifrost-1 they additionally run a cross-judge study with six model judges and a human evaluation, and confidence intervals are reported for the main results.
  • Adaptation is cheap. Only the connector/bridge is tuned (0.1%–3.3% of parameters), and training runs on a single A100 with 1,024 calibration examples.
  • Appendix G.2 is quite candid about the risks, including whether latent intent is equivalent to user intent, reduced diversity, and bias amplification.
Critical weaknesses
  • Location: Section 1 (lines 074–075), Appendix C.1 (lines 864–875), Appendix C.3 (lines 1000–1048), Appendix G.2 This is my main concern: the commitment the paper wants to transfer is not the thing it actually measures. The main text says latent intent is the visual commitment formed inside the upstream model at generation time, but the commitment used in both evaluation and training is obtained by asking ϕ\phi separately, in a different call. When OpenAcc queries ϕ\phi, it tells ϕ\phi the target entity (e.g. parking meter) and the slot type (e.g. color) and asks it to pick one value, whereas at generation time the pipeline only sees "a photo of a parking meter". ELLA is the clearest case: according to the ELLA paper, it conditions only on Flan-T5-XL encoder hidden states, so there is no decoding step in the generation path and thus no step at which a specific color is sampled, yet OpenAcc's commitment comes from text produced by the T5 decoder; at training time, for such a text-only ϕ\phi (lines 871–875), the image generated from the completed prompt is used directly as the positive sample and the image from the original prompt as the negative sample (which also contradicts lines 865–866 stating that four candidate images per open prompt are sampled for ELLA for pairwise comparison), amounting to distilling ϕ\phi's text completion into the bridge. Even for VLM ϕ\phi's that can inspect images, training preferences come from querying ϕ\phi post hoc in a separate call after generation. Janus's image tokens and Bifrost-1's CLIP latents may well carry concrete visual information, but neither training nor evaluation (IWR, OpenAcc) reads them. Appendix G.2 redefines latent intent as an operational commitment induced by the prompt and ϕ\phi's priors; I find that wording more accurate, but under that definition what DaLI does is align the bridge to ϕ\phi's default priors rather than transfer a commitment the upstream model has already formed. The claims in the main text and the conclusions should be narrowed accordingly.

  • Location: Tables 3 and 4, Appendix C.3 (lines 989–990, 1050) Neither IWR nor the human evaluation can distinguish whether DaLI learned ϕ\phi's specific commitment or simply produces images that better match common sense. The IWR judge sees only the open prompt, the slot type, and two images; human annotators see only the open prompt and two images, and the Figure 6 caption even asks them to choose the image that better matches their own visual intent. Neither the external model judges nor the human annotators know ϕ\phi's commitment, so they can only judge from their own interpretation; in Figure 3(a) the judge's stated reason is that the background has sky and grassland and looks more natural. Table 3 points to the same issue: if DaLI had learned ϕ\phi-specific commitments, one would expect ϕ\phi itself to prefer DaLI's outputs at least as strongly as other judges do. Instead, self-IWR is 55.8 (95% CI [54.0, 57.5] in Table 9) and does not stand out: Qwen3-VL 32B is clearly higher (61.1, [58.0, 64.4]), and the other model judges and humans fall in an overlapping range (54.1–58.8 and 56.5). Judges differ in tie rates and discriminability, so this is only indirect evidence, but it fits the authors' own reading in Section 6.3 that the gain reflects preferences shared across models. Table 4 sharpens the distinction: cross-model agreement is fairly high for pairwise preference labels (74.8%) but much lower for the commitment values themselves (48.0% strict, 60.7% semantic). If DaLI's gain is recognized equally by all judges, it most likely lies in the shared part of these preferences, i.e. generic typicality, rather than in the commitments specific to ϕ\phi, which is what LIT claims to transfer. I suggest reporting OpenAcc separately on the subset of samples where cross-model commitments disagree; if DaLI only learned the shared default values, the gain on that subset should shrink substantially.

  • Location: Appendix G.2, Table 1 FID column, Table 8 Diversity is not evaluated. When the prompt does not specify the cup's color, different seeds producing different colors is normal behavior, and pushing the output toward the value ϕ\phi prefers with DPO is likely to reduce diversity. Appendix G.2 acknowledges this risk, but no diversity metric appears anywhere in the paper. In addition, after adding DaLI, FID gets worse on all four pipelines (ELLA 19.4→20.1, Bifrost-1 23.0→24.7, Janus 8.5→9.1, Janus-Pro 16.1→17.2), while MUSIQ and CLIP-IQA+ all improve. Each FID confidence interval individually contains zero, but all four pipelines move in the same direction, which is consistent with reduced diversity (higher per-image quality but poorer distribution coverage), although FID on COCO-30K can also shift for other reasons. I suggest adding a diversity evaluation (e.g. multi-seed LPIPS, entropy of the open attribute values), and toning down the "visual quality retention" claim.

  • Location: Appendix F.2 Table 11; the VPGen and RPG rows in Table 1 Compared with training-free prompt completion, DaLI's gain is small. On Bifrost-1, prompt completion alone raises OpenAcc from 45.5 to 47.8, while DaLI reaches 48.6 — only 0.8 points higher, with no confidence interval reported for that difference. I suggest making it a main baseline in the main text. Moreover, both the completion and the OpenAcc commitment come from Qwen2.5-VL 7B, so the value written into the completed prompt is very likely the same as the committed value; please report the agreement rate between the two. Separately, in Table 1, the text-mediated systems have the lowest OpenAcc of the three pipeline categories on average (VPGen 21.9, RPG 41.5; category mean 31.7, vs. 43.1 for latent-bridged and 44.3 for unified systems), with VPGen far below all others. In those systems the upstream output is text, so one can directly check whether the rewritten prompt states the evaluated attribute and whether the stated value matches the committed value, and then compute OpenAcc separately for each case; this would show whether failures come from the upstream not writing the attribute down or from the generator not rendering what was written. This analysis requires no training.

  • Location: Definition 2 (lines 236–239), Appendix C.2 (lines 910–912), Appendix E.1 (lines 1117–1118) The OpenAcc judge may coincide with the upstream model. Definition 2 requires the OpenAcc judge to be an external model distinct from ϕ\phi, to avoid using ϕ\phi both as the source of the commitment and as the evaluator. However, Appendix C.2 states that OpenAcc is evaluated in the same manner as prompt-alignment accuracy, and Appendix E.1 names Qwen2.5-VL-7B-Instruct as the prompt-alignment judge, which is exactly the upstream ϕ\phi of RPG and Bifrost-1. If so, OpenAcc on these two pipelines violates the paper's own definition, and Bifrost-1 is the pipeline used for the cross-judge, human, and prompt-completion studies. Please clarify which judge is used for OpenAcc on each pipeline, and if it is ϕ\phi, re-evaluate with an external judge.

  • Location: Section 6.3, Appendices C.2, C.3, F.5 (Minor) The human evaluation is not fully documented. The paper runs both preference and image-quality studies with 50 participants but does not say how they were recruited, whether they were compensated, or whether they gave informed consent, and there is no ethics statement. I suggest adding this information.

Questions for authors
  1. In a pipeline like ELLA that conditions only on encoder hidden states, does the representation passed from the upstream model to the bridge really contain a definite commitment for a single attribute? Can this be tested by probing the bridge input, or by fixing the upstream output and varying only the generator's random seed?
  2. In Table 3, ϕ\phi, which produced the training labels, does not show a stronger preference for DaLI's outputs than the external judges (Qwen3-VL 32B is clearly higher). Does this mean the IWR gain comes from generic typicality preferences rather than from ϕ\phi-specific commitments? Could you report OpenAcc separately on samples where cross-model commitments disagree?
  3. How much does generation diversity change before and after training DaLI, measured over multiple random seeds for the same open prompt (entropy of the open attribute values, LPIPS diversity)?
  4. Is DaLI's gain over training-free prompt completion (Table 11) significant? What is the agreement rate between the attribute value written into the completed prompt and the committed value used by OpenAcc? In text-mediated systems such as VPGen and RPG, does the rewritten prompt state the evaluated attribute, and what is OpenAcc in the stated vs. unstated cases?
  5. Definition 2 requires the OpenAcc judge to differ from ϕ\phi, while Appendix C.2 says OpenAcc is evaluated in the same manner as prompt alignment, whose judge in Appendix E.1 is Qwen2.5-VL-7B-Instruct. Is the OpenAcc judge for RPG and Bifrost-1 in fact their own upstream ϕ\phi?
2 comments

We sincerely thank the reviewer for the insightful review and constructive feedback. We address every concern below and have revised the manuscript accordingly. All changes in the revised manuscript are highlighted in blue for ease of reference.


[W1/Q1] Commitment–Measurement Mismatch.

[A1] We agree that our main-text wording overstated what the current protocol directly observes: a commitment elicited in a separate call is not a generation-time latent variable read from the same forward pass. We therefore clarify the scope of LIT as an operational, model-conditional target: whether downstream generation realizes a reproducible commitment elicited from the designated upstream interpreter under a fixed protocol. We also correct the inconsistency in Appendix C.1: ELLA uses a completed-prompt positive and an original-prompt negative, not four upstream-ranked candidates.

[W2/Q2] Upstream Specificity vs. Generic Typicality.

[A2] We agree that cross-IWR and human-IWR do not by themselves establish upstream specificity, because these judges do not observe the designated interpreter’s commitment; their role is external robustness rather than target identification. Absolute IWR values are also not directly calibrated across judges because their tie rates differ substantially, but this does not resolve the reviewer’s broader specificity concern. On the commitment-disagreement subset, DaLI changes agreement of Bifrost-1 with the designated interpreter (Qwen2.5-VL-7B) by +4.2 percent points, versus -2.1 percent points for the alternative interpreter (InternVL3.5-4B), yielding a specificity difference of 6.3 percent points. We accordingly describe the learned signal as interpreter-specific alignment rather than using cross-IWR itself as evidence of specificity.

[W3/Q3] Diversity Loss.

[A3] We investigate the diversity difference of Bifrost-1 before and after applying DaLI with attribute entropy and mean LPIPS. Under the same open prompts and four matched seeds, attribute entropy difference is +0.03 [-0.00, 0.06], while mean LPIPS difference is +0.003 [0.000, 0.007]. Neither metric decreases, providing no evidence of diversity loss in the evaluated setting. Nevertheless, we agree with the reviewer that diversity should be considered as a crucial factor within quality retention and therefore report perceptual quality, distributional quality, and prompt-valid diversity separately.

[W4/Q4] Weak Advantage over Prompt Completion.

[A4] We agree that prompt completion is a strong training-free baseline. However, they require additional LLM inference in the decoding phase, and may be considered as text-mediated systems in a broad point of view (in Figure 1(b)), where the prompt completion methods perform as an upstream LLM while the original T2I pipeline works as the generator. Prompt completion methods also target the first phase of the T2I generation pipeline (as stated in Appendix F.2), which is thus not the direct competitor of DaLI, but are complementary sources when jointly adapted. Additionally, we analyze the behavior of Bifrost-1 under completed prompts. Even though the prompt completion method filled out 76.3% of the open slots, only 25.4% of them are faithfully rendered in the image. These continuous failure modes even strengthen the importance of our research targeting the second stage, aligning the upstream LLM and the generator itself.

[W5/Q4] Text-Mediated Failure Decomposition.

[A5] We additionally decompose the failure modes of text-mediated systems (VPGen and RPG) according to whether the rewritten text states the evaluated attribute, whether the stated value matches the elicited commitment, and whether the generator renders the stated value. For VPGen, the attribute is stated in 57.1% of cases, with only 1.7% rendering accuracy when stated; the corresponding values are 78.8% and 19.4% for RPG, respectively. This separates upstream omission from a downstream rendering failure, which highlights the importance of the second-stage problem.

[W6/Q5] OpenAcc Judge Leakage.

[A6] We agree that the commitment source and OpenAcc image evaluator should be distinct and that our original description did not make the pipeline-specific configuration sufficiently clear. To remove this ambiguity, we re-evaluated OpenAcc [%] of the affected RPG and Bifrost-1 models using three independent judges InternVL3.5-8B, Qwen3-VL-8B, and LLaVA-OV2-8B. The results remain consistent regardless of the evaluator model.

JudgeRPGΔ\Delta Bifrost-1
Qwen2.5-VL-7B (ϕ\phi)19.31.1
InternVL3.5-8B21.22.2
Qwen3-VL-8B17.52.9
LLaVA-OV2-8B22.12.2

[W7] Incomplete Human-Study Documentation.

[A7] Thank you for identifying this omission. Fifty volunteers were recruited through convenience sampling from the authors’ personal and professional networks, and each participant received about USD 3.75 (in local currency) for completing the study. Before participation, they received an explanation of the study purpose, evaluation procedure, compensation, and research use of their responses, and voluntarily agreed to participate. The study was not submitted for prospective IRB review. We now disclose this explicitly and have added the recruitment, compensation, consent, and ethics details to Appendix C.3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.