acceptodds
Withdrawn from ICLR 2027

Lens: Rethinking Training Efficiency for Foundational Text-to-Image Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Training foundational text-to-image (T2I) models typically requires massive computational resources. We introduce Lens, a 3.8B-parameter T2I model that achieves performance competitive with, and in several cases surpassing, state-of-the-art models with more than 6B parameters across various benchmarks, while requiring significantly less training compute. For example, Lens requires only about 19.3% of the training compute used by Z-Image. The training efficiency of Lens stems from two key strategies beyond its compact model size. First, we maximize data information density per training batch by (i) training on Lens-800M, a dataset of 800M densely captioned image-text pairs whose captions are generated by GPT-4.1 and contain approximately 109 words on average, providing richer semantic supervision than conventional short captions, and (ii) constructing each batch from images with multiple resolutions and diverse aspect ratios, thereby enlarging the effective visual coverage of each optimization step. Second, we improve convergence speed through careful architectural choices, including adopting a semantic VAE that provides better latent representations and employing a strong language encoder that accelerates optimization while enabling multilingual generalization from English-only training data. After pre-training, we apply reinforcement learning with taxonomy-driven prompts (Lens-RL-8K) and structured reward rubrics to suppress artifacts and improve visual quality, a reasoner module with training-free system prompt search to better align user requests with the model, and distillation-based acceleration for 4-step inference. Through efficient training and systematic optimization, Lens generalizes to arbitrary aspect ratios from to and resolutions up to , and supports prompts in several commonly used languages. Thanks to its compact size, Lens generates a image in 3.15 seconds on a single NVIDIA H100 GPU, while its distilled turbo version performs 4-step generation in 0.84 seconds. We will release both the pre-trained and post-trained models.

Primary area:
generative models
Keywords:
image generation, foundation models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_44dwOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper investigates several design choices for training text-to-image diffusion models. The authors conduct extensive ablation studies on a customized dataset, covering image captioning, VAE architectures, text encoders, MAR training, and the scale of RL training data. Based on these experiments, the authors identify several factors that contribute to improved model performance, including dense caption supervision, stronger VAEs and language encoders, mixed-resolution training, and a large, diverse, and well-balanced RL prompt set.

Critical strengths
  1. Extensive ablation studies. The paper provides a relatively comprehensive set of ablations for text-to-image generation, offering insights into how different VAEs and text encoders perform within the proposed dataset and model architecture, even with a relatively compact model.

  2. Promising post-training performance. The model achieves promising results after post-training, demonstrating the potential of the proposed training recipe to improve the performance of a compact text-to-image diffusion model.

Critical weaknesses
  1. Limited novelty and generalizable insights. Many of the architectural ablations primarily compare existing VAEs and text encoders. While the experiments identify which components perform better in the proposed setting, they provide limited insight into why these components work better or what general principles can guide future model development. For example, the conclusion that stronger VAEs and text encoders improve performance is not particularly surprising. It remains unclear how researchers could apply these findings when new VAE or text encoder variants become available.

  2. Limited novelty of the training strategies and backbone architecture. Dense caption supervision and MAR training have already been explored in prior work, such as Qwen-Image and SD3. Similarly, the finding that scaling up the RL prompt set in terms of both size and diversity improves performance is relatively intuitive. The architecture is borrowed from SD3, and the text feature injection mechanism is similar to Hi-Dream.

  3. Unclear scope and insufficient evaluation of training efficiency. Although the paper is titled around training efficiency, it does not provide quantitative evidence demonstrating improved training efficiency, such as training time, computational cost, or the resources required to achieve a given performance level. Most of the ablation studies focus on generation quality, particularly GenEval, and the reported performance improvements appear to be largely attributable to post-training. Furthermore, it is unclear why few-step distillation, which primarily targets inference efficiency, is included in a study focused on training efficiency. The scope and central contribution of the paper therefore remain unclear.

  4. Incomplete and inconsistent evaluation metrics. The ablation studies rely heavily on GenEval, and a more comprehensive evaluation using additional metrics could provide a more convincing assessment of generation quality. In addition, it is unclear why Table 1 reports two different evaluation metrics. If multiple metrics are used throughout the ablation studies, reporting them consistently would make the comparisons more transparent and easier to interpret.

Questions for authors

Given the various factors investigated in the paper, how would the authors rank their relative importance for training a strong text-to-image diffusion model?

Reviewer_uUgEOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces an efficient way of training foundational text-to-image models. They do so by first preparing a dataset of 800M text-image pairs, thereafter improving on the image and text information via using different image resolution and aspect ratio during training as well as by extending the captions of the images using a prompt extender. The paper also studies different parts of the architecture design, like different captioners, different VAEs, etc. Finally, the paper proposes the final setup, which adopts RL on a curated dataset and achieves good performance using only 19% of the compute of prior methods.

Critical strengths
  1. The paper’s proposal of Lens-800M dataset and the Lens-RL-8K dataset for pretraining as well as RL optimization are quite nice!
  2. I appreciate the authors’ effort in doing broader ablations for many different parts of the setup, like the VAE, text encoder, captioner, etc.
Critical weaknesses
  1. I really appreciate the author’s effort in studying the effect of short and long captions. However, I am quite surprised by the results presented in Figure 4, where the model with longer prompts performs best on GenEval, which mainly consists of shorter prompts. I would assume that there would be a train-test discrepancy between the dense and long training prompts and the short evaluation prompts in GenEval. I would expect the evaluation to work for longer prompts if the GenEval prompts are somehow expanded (maybe via the reasoner); but then that would also mean that during training the reasoner is used; in that case, the shorter caption model would also benefit from a detailed caption from the reasoner and should behave in a much similar way to the long caption model. A clarification is extremely essential for this!
  2. I really like the fact that the ablation for which VAE to use is put under the microscope too in the paper. However, I find it a bit strange that for all other ablations the paper used 120k training steps whereas for the VAE ablation (in Figure 5) the final training step is 64k. What is the reasoning behind this? It would be really nice if the authors could comment on this.
  3. I find the architecture design of concatenating K layer information from the text encoder quite interesting. However, the paper shows no reasoning about the selection of the layers from the text encoder, how they should be selected for different encoders, and whether they even provide any benefit over using just the last layer features. It would really help practitioners and readers to understand the benefit of this design setup if more explanation is provided. I understand the time limits of the rebuttal, and as such, simple setups with text explanations are also enough!
  4. I really like the idea of mixed resolution training. However, I am not quite sure to understand how it works for patchification and thereafter batching (Appendix D.2 doesn't seem to provide much information). Do the authors change the patch size per image resolution? How are the batches constructed? Additionally, do the authors also provide the resolution and the aspect ratio as extra conditions to the model, similar to SDXL? It would be super nice if the authors could provide more information regarding this!
Questions for authors

I found the paper quite interesting. However, there are a few things that I am not so sure about, and would appreciate the authors take on these.

(Note: I totally understand the restrictions of the rebuttal, and simple text explanations and short experiments are more than enough)

  1. I am finding it a bit difficult to figure out what the core take-away message of the proposed method is. Is it the dataset? Is it the training? Or is it the architecture? I am struggling a bit because the proposed results feel like a design study. I am totally okay if this is the intended purpose. But I want the authors to point out directly what should be the thing that practitioners should look upon. Should we all start curating better datasets to gain efficiency? Because I didn't understand from the paper whether the 800M data points were necessary or if we could have had 10M and it would have been okay too, with the different architectural choices made! Or is it the RL finetuning which makes everything tick. So what I want from the authors in the rebuttal is to directly address the points that they feel are essential and also to please add them to their paper. I would be really looking forward to hearing about this in the rebuttal.
  2. As mentioned before, I really appreciate the ablations for the VAE, the text encoder, etc. However, I found that the VAE ablation was done up to 64K steps, whereas other ablations are done up to 120K steps (Weakness 2). It would be nice if the authors could clarify the reasoning behind this. On the subject of ablation, I feel the paper is missing the reasoning behind using K different layer features for the text encoder (Weakness 3), and it would be really nice if the authors provided a small experimental justification or text-only explanations regarding this.
  3. I found the results for GenEval for longer and shorter captions to be a bit confusing. GenEval itself is based on short captions, and it is surprising that the long caption model works better (Weakness 1). A bit of clarification from the authors would be really appreciated.

Overall, even though the paper is mostly a design choices paper, I still feel that it has many merits. But there are some clarifications that are missing in some parts. As such, I recommend Weak Rejection.

Reviewer_k25hOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the high computational cost of training foundational T2I models, arguing that training efficiency depends on three factors: model size, the information density of each training batch, and convergence speed. The authors present Lens, a MMDiT diffusion model that is as competitive with or better than larger (6B+) open-source models on OneIG, GenEval, LongText and CVTG, while using about 19.3% of Z-Image’s pre-training compute (192K A100-hours). The main contributions are (1) Lens-800M, an 800M-image corpus densely re-captioned by GPT-4.1 (2) controlled toy-scale ablations, evaluated by downstream generation quality rather than proxy metrics (3) an RL post-training stage using DiffusionNFT with binary GPT-4.1-mini rubric rewards on Lens-RL-8K (4) a 4-step distilled variant, Lens-Turbo, built on DMD2, decoupled-DMD and SenseFlow techniques and (5) a GPT-5.5 prompt-rewriting “reasoner” whose system prompt is refined by a training-free, LLM-driven search.

Critical strengths
  1. A strong, practically useful model and recipe at modest scale. The model is small, fast, and its reported benchmark results are competitive across compositional and text-rendering benchmarks.

  2. Generation-based, controlled ablations of design choices that are usually made by convention. The paper compares VAEs by downstream T2I learning curves rather than rFID or ImageNet proxies. It also compares text encoders on English and multilingual GenEval, caption density, RL prompt coverage, and reward functions.

Critical weaknesses
  1. The SOTA claims seem to come mostly from the reasoner and benchmark-specific prompting, not from the model itself. Without the reasoner (Table 6), Lens scores 0.843 on GenEval and 0.532 on OneIG-EN, comparable to Z-Image (0.840 / 0.546) and below Qwen-Image (0.868 / 0.539). With the same GPT-5.5 reasoner, Qwen-Image outperforms Lens on OneIG, LongText, and CVTG. Table 2 therefore does not support the claim that the 3.8B model surpasses larger models. The rewriting setup raises further concerns. The GenEval system prompt (Appendix F.5) is designed around detector-based verification and treats benchmark metadata as “the source of truth,” which would give Lens information unavailable to the baselines. The source of failure cases for the system-prompt search is also unspecified, raising the possibility of tuning on the test set. In addition, the RL rubrics closely mirror the dimensions measured by GenEval and CVTG, and the unusually high GenEval Position score (0.915 vs. at most 0.79 for baselines) suggests targeted optimization. All models should be compared either without a reasoner or with identical rewriting.

  2. The efficiency claim isn’t backed by a controlled comparison, and the cost accounting leaves out a lot. The 19.3% figure compares peak-FLOPS-normalized GPU-hours across different hardware against a model trained with different data and objectives, and there is no compute-matched baseline. It is therefore unclear whether the proposed factors explain the result or whether it simply reflects a smaller model trained for less. The accounting also excludes data preprocessing, which Table 11 shows is the dominant cost (576K V100 GPU-hours for filtering and $2.44M for captioning, versus about $800K for pre-training). It is further unclear whether text-encoder compute is included. The design choices are validated only at small scale, apparently with a single seed, and with no evidence that they compose or transfer to 3.8B. Since each component is drawn from prior work, the paper’s contribution rests on this efficiency analysis, which is its least rigorously supported aspect.

Questions for authors
  1. Could the authors provide a version of Table 2 in which all models are evaluated without a reasoner, or all models are evaluated with the identical reasoner and system prompt? For each baseline, please state whether its reported numbers were obtained with prompt rewriting and, if so, which rewriter was used.
  2. Does the GenEval reasoner receive the benchmark’s structured metadata (object classes, counts, colors, and spatial relations) at test time, as Appendix F.5 suggests? If so, please report results using the raw benchmark prompts only.
  3. On which data were the failure cases for the training-free system-prompt search collected? Were any benchmark prompts, or prompts derived from them, used during the search, and were the benchmark-specific system prompts (Appendices F.5–F.7) selected on the basis of benchmark scores?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.