acceptodds
Withdrawn from ICLR 2027

PE-OPSD: Internalizing Prompt Enhancement into Flow-matching Models via On-Policy Self-Distillation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

17 views

Abstract

Text-to-image users often provide short and underspecified prompts, while generative models rely on detailed textual conditions for reliable instruction following. Existing systems mitigate this mismatch with Prompt Enhancers (PEs), which rewrite raw prompts into richer descriptions at inference time, but this introduces extra cost and leaves prompt understanding external to the generator. In this work, we reinterpret prompt enhancement as privileged information and ask whether its benefit can be internalized into the generator itself. We propose Prompt-Enhanced On-Policy Self-Distillation () for text-to-image flow-matching models. During training, a student conditioned only on the raw prompt follows its own generation trajectory, while an enhanced-prompt teacher provides vector-field supervision at the states visited by the student. This on-policy formulation transfers enhanced-prompt generation behavior into the raw-prompt student without using enhanced prompts or offline-generated samples at inference time. After training, both the PE and teacher are removed. Experiments across multiple models and benchmarks show that improves prompt fidelity and visual quality while reducing inference-time prompt-rewriting overhead.

Primary area:
generative models
Keywords:
On-policy self-distillation, prompt enhancement, post-training, text-to-image generation, flow-matching model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_uatFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces PE-OPSD, a prompt-enhanced on-policy self-distillation framework for text-to-image flow-matching models that internalizes prompt-enhancement benefits into the generator, removing the need for an external prompt enhancer at inference. A student model samples on-policy trajectories from raw prompts, while an EMA teacher conditioned on enhanced prompts provides vector-field supervision at student-visited states. Experiments across multiple backbones, prompt enhancers, and benchmarks show improved prompt fidelity over SFT and off-policy distillation baselines without sacrificing inference efficiency, and ablation and out-of-domain evaluations support the method’s effectiveness and generalizability.

Critical strengths
  • Clear and Well-Organized Presentation. The paper is clearly written and well structured, with intuitive illustrations and detailed explanations that make the proposed method easy to understand.

  • Improved Inference Efficiency. The method eliminates the need for an external prompt enhancer at inference time, offering noticeable speedups while maintaining competitive generation quality. However, the practical necessity of this efficiency improvement remains debatable, as discussed in the weaknesses.

Critical weaknesses
  • Limited Methodological Novelty. The proposed approach mostly adapts existing on-policy self-distillation (OPSD) techniques to prompt enhancement. Its central contribution is to use enhanced prompts as privileged teacher conditions, and it does not introduce substantial algorithmic advances beyond established OPSD frameworks. Consequently, its methodological novelty appears modest.

  • Insufficient Motivation and Unexamined Controllability Trade-offs. The paper’s main rationale for internalizing PE is to reduce inference latency, but it neglects the value of explicit PE for flexible and controllable generation. Embedding PE into the generator may compromise adaptability to user-specific requirements, yet these trade-offs are neither discussed nor evaluated. Moreover, PE-induced latency may be acceptable in many practical production settings, so the necessity of the proposed approach is not adequately justified.

Questions for authors
  • Since on-policy self-distillation and privileged-teacher conditioning are already well explored, what fundamental methodological novelty does PE-OPSD offer beyond applying existing OPSD techniques to enhanced prompts?

  • What evidence establishes the necessity of removing inference-time PE when its latency may often be acceptable? Moreover, has the work examined whether internalizing PE compromises the generator’s flexibility and controllability, especially under detailed or dynamically changing user instructions?

Reviewer_p8kHOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper treats PE-enhanced prompts as training-time privileged information, aiming to internalize the benefit of inference-time prompt enhancement into the text-to-image generator so that only the raw prompt is needed at deployment. The student, conditioned on the raw prompt, rolls out its own sampling trajectory, while an EMA teacher conditioned on the enhanced prompt provides regression targets at the same student-visited states. Experiments on SD3.5-M, Z-Image, and Z-Image-Turbo, trained on GenEval/GenEval2 prompts, show better aggregate prompt fidelity than inference-time PE, SFT, and off-policy distillation, with results also reported on alternative PEs, larger models, and out-of-domain benchmarks.

Critical strengths
  1. The problem is practically relevant: converting the benefit of inference-time PE into a training signal removes the rewriting overhead at deployment.
  2. The method is simple, requiring neither additional text–image pairs nor modifications to the model's conditioning interface.
  3. The experiments cover 6 backbones, 3 PEs, 3 out-of-domain benchmarks, and a human study, with the SFT and off-policy baselines sharing the same initialization and training budget.
Critical weaknesses
  1. Limited technical novelty: OPSD has already been introduced to diffusion models by D-OPSD. The main change here is replacing the privileged condition with PE-generated text.
  2. Table 5 reports only Base and +Ours; Base+PE, SFT and off-policy distillation are missing on all out-of-domain benchmarks.
  3. Each raw prompt is paired with a single fixed enhanced prompt (defaulting to a photographic style), so the student regresses toward a single realization. Does generation diversity decrease? No diversity metric is reported.
Questions for authors
  1. What happens if the teacher uses the raw prompt with a higher CFG scale instead of the enhanced prompt? This control would confirm that the gain stems from the PE information rather than generic self-distillation sharpening.
  2. How does the EMA teacher perform under enhanced prompts at the end of training, and what are the final numbers of the frozen-teacher variant?
  3. Are the enhanced prompts used by Base+PE at test time also subjected to the consistency filtering and regeneration described in App. F.1?
  4. Please also see the Weaknesses.
Reviewer_G3g7OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Prompt-Enhancement On-Policy Self-Distillation (PE-OPSD), a method for transferring the benefits of prompt enhancement into a text-to-image flow-matching model, thereby eliminating the need for an external prompt enhancer at inference time. During training, the student generates trajectories conditioned on the original prompt, while an exponential moving average (EMA) teacher provides velocity predictions conditioned on an enhanced version of the same prompt. The student learns to match these predictions at its own visited states.

Critical strengths

1.The paper addresses a useful deployment problem. External prompt enhancement introduces additional inference latency and operational complexity. Internalizing its benefits into the generator is a meaningful objective, especially when the resulting model retains its original inference interface. 2. The approach is simple and broadly applicable. Using enhanced text as the teacher’s additional conditioning information avoids requiring a separate image-conditioned teacher or paired image supervision. The formulation applies naturally to flow-matching models, and the experiments cover both standard and step-distilled backbones. 3. The experimental comparison includes meaningful controls. The paper compares against inference-time prompt enhancement, synthetic-data supervised fine-tuning, and off-policy distillation. In particular, the off-policy comparison helps isolate the effect of collecting trajectories from the student. The paper also includes human evaluation, computational cost measurements, and EMA ablations.

Critical weaknesses
  1. The methodological novelty is limited relative to existing on-policy distillation frameworks. Student-generated trajectories, velocity matching, and EMA teachers are already established components, and D-OPSD also uses richer teacher conditioning to guide the student. The main distinction here is replacing that conditioning with enhanced text.
  2. Several improvements over strong baselines are too small to assess without statistical uncertainty. In Table 2, SD3.5-M achieves a GenEval 2 geometric mean accuracy of 0.226 versus 0.223 for off-policy distillation, while Z-Image achieves 0.406 versus 0.404 for inference-time prompt enhancement.
  3. The generalization results do not establish consistent improvements across capabilities. Out-of-distribution evaluations mainly compare PE-OPSD with the original model, leaving its advantage over competing methods unclear.
  4. The paper does not adequately evaluate whether prompt enhancement introduces unwanted generation preferences. The enhancement instructions in Appendix F.2 default to photographic imagery when the style is unspecified. Distilling such choices could make the model adopt particular styles or compositions even when the original prompt leaves them open.
  5. The relationship between training convergence and generation quality remains insufficiently understood. The EMA ablation shows that a decay of 0.9 produces rapidly decreasing training loss while generation quality deteriorates below the original model. This demonstrates that successful optimization of the matching objective does not necessarily indicate useful knowledge transfer.
  6. The efficiency comparison does not establish a consistent advantage under comparable computational budgets. Table 11 reports approximately 4.0 hours for PE-OPSD on Z-Image-Turbo, compared with 3.4 hours for SFT, including synthetic-data generation. Thus, the computational advantage depends on the model and training setting.
Questions for authors
  1. What is the precise methodological contribution beyond existing on-policy self-distillation frameworks?
  2. Are prompt-enhancement filtering procedures matched across training and evaluation baselines? Training enhancements are checked for consistency and regenerated when necessary. Does the inference-time prompt-enhancement baseline receive the same checking and regeneration procedure? If not, how much of the observed difference is attributable to this discrepancy?
  3. How does teacher quality evolve during training? Can the authors separately track the EMA teacher under enhanced prompts and the student under original prompts, including substantially longer training runs? Does the teacher retain an advantage throughout training, and how is the stopping point selected?
  4. Does internalization reduce conditional diversity or stylistic flexibility? What happens when several distinct but valid enhanced prompts are available for the same original prompt? Does sampling among these enhancements improve diversity or reduce unwanted defaults?
  5. Do the gains persist against competing methods on out-of-distribution benchmarks? Please evaluate inference-time prompt enhancement, SFT, and off-policy distillation on the same out-of-distribution tasks. How do the authors explain the category-level regressions in counting, color, material, spatial relations, and verbs?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.