- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents HiUni, a unified multimodal model extended directly from a pretrained VLM (Qwen3-VL-4B-Instruct). Understanding and generation share the same language-aligned ViT and the same dense Transformer backbone, and generation-specific computation is confined to the output side.
To handle the difficulty of optimizing diffusion in pixel space, the authors design a Pixel Diffusion Head. It removes block-wise AdaLN, injects the timestep through a time token, and keeps only a single timestep modulation before the output. To compensate for the spatial detail lost through patch merging and deep semantic abstraction, the authors introduce HiGate, which injects raw pixel features of the noisy image into the diffusion head through a token-wise gated residual.
The model is evaluated on understanding, text-to-image generation (including long text rendering in Chinese and English), and image editing. Its average understanding score is close to that of the base VLM, and its generation and editing results are comparable to larger unified models. The authors also compare dense and MoT backbones at the 2B scale and analyze the feature source and injection strength of HiGate.
Critical strengths
- I think this is a fairly solid empirical study. The three-stage training pipeline, the resolution schedule that grows from 256 to 2048, the per-stage hyperparameters, and the head architecture are all documented in the appendix in good detail. The engineering is clearly mature.
- The generation ability at the 4B scale is quite good and shows potential for scaling up. With 4B+1.3B parameters, HiUni reaches 47.98 on the Chinese Overall of Qwen-Image-Bench, 0.963 / 0.954 on LongText, and 6.70 / 6.81 on GEdit Overall. This is on par with larger models such as Bagel (7B+7B) and SenseNova-U1 (8B+8B), and I would be curious to see how this route performs at a larger scale.
- The paper is clearly written and easy to follow.
Critical weaknesses
- The paper reads more like an empirical study of pixel-space Transfusion design, and I do not see insights that match the "Harmonizing" in the title. To me, it is closer to a technical report for an industrial product and does not offer much novel insight. This direction has already been explored quite systematically: TUNA and Cheers study various unified representations and training recipes for understanding and generation, and architectures such as HiGate are themselves inspired by them. Some of the pixel-space training insights also come from JiT and related work. Feeding noisy pixels directly into the head to recover details has been done before in DiP [1] and DeCo [2]: the Patch Detailer Head in DiP takes both the DiT output and the original noisy pixel patch, and the pixel decoder in DeCo takes the full-resolution noisy image as input, yet the paper neither cites nor discusses these two works. The observation that removing block-wise AdaLN alleviates loss spikes in pixel space is interesting, but it feels more like an engineering measure for stable training than a new understanding of unified modeling.
- The discussion of related work is missing. RepFusion [3] also feeds noisy visual inputs through the MLLM's own visual pathway and uses the LLM outputs to drive a lightweight denoiser, which is quite close to the core idea of HiUni. Since the two works appeared close in time, I do not count this as a missing citation, but I suggest that the authors discuss the mechanistic difference so that readers can judge the boundary of the contribution.
- The ablations are too simple and incomplete. All ablations (Table 4, Table 5, Table 11) report only OneIG on the generation side, with no editing metrics such as ImgEdit or GEdit, and neither Qwen-Image-Bench nor LongText. For a unified model that emphasizes both generation and editing, a single benchmark is not enough to support the conclusions about the backbone choice and the HiGate design.
- The dataset information is too limited. Appendix A.5 only states that the understanding and generation corpora come from proprietary datasets, and that the editing data include private data produced by an internal pipeline in addition to public data. I understand that private data may not be easy to disclose, but some details must be provided, for example whether the authors used data distilled from external models (such as BLIP3o-60k, which is generated by closed-source models). I am also concerned that the improvement over other works in the main experiments may largely come from higher-quality synthetic data.
- Both the Intro and Sec. 4.3 present the dense vs. MoT comparison as an important argument. Yet in the only controlled comparison with the same initialization and the same recipe (Table 4), MoT-2B beats Dense-2B on both understanding (59.23 vs. 58.55) and generation (OneIG 0.345 vs. 0.291), and the authors still choose dense. Their argument is that, at a similar total parameter count, Dense-4B has clearly stronger understanding (65.04) and "competitive" generation (0.314). However, this overlooks the fact that Dense-4B is initialized from Qwen3-VL-4B while MoT-2B is initialized from Qwen3-VL-2B, so the two start from different places. Much of the understanding gap comes from the stronger base model rather than from the dense design itself, so this is not a fair comparison. Even so, Dense-4B still generates worse than MoT-2B.
[1] DiP: Taming Diffusion Models in Pixel Space, arXiv:2511.18822
[2] DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generation, CVPR 2026
[3] RepFusion: Leveraging Multimodal Priors for Denoising in Representation Space, arXiv:2606.14700
Questions for authors
See weaknesses.
- If I understand correctly, the paper maps the timestep through an MLP and uses the output to replace the embedding of the <time> token, which then enters the LLM as one token in the sequence context. JanusFlow uses a similar time token, but Transfusion and BAGEL add the timestep embedding directly to every patch. Why did the authors choose this design? Is there an ablation against other ways of injecting the timestep?
- Have the authors tried removing the understanding data and training only on generation and editing? This would surely hurt understanding, but could it improve generation and editing? I think this would help clarify whether understanding data is a help or a cost for generation.
9 comments
Response to Reviewer Smg9 Part [2]
[W3] Ablations evaluated only on OneIG for generation
We agree that evaluating multiple generation benchmarks provides stronger evidence for comparisons. In our preliminary experiments, we found OneIG to be the most consistent benchmark with generation quality and to provide clear differentiation between early checkpoints. We therefore selected it as the primary generation metric for the sake of brevity. Following the reviewer's suggestion, we have extended the evaluations of the Table 4 and Table 5 to Qwen-Image-Bench, LongText, and OneIG, covering both Chinese and English prompts.
Additional results for Table 4
| Model | Qwen-Image-Bench ZH | Qwen-Image-Bench EN | LongText ZH | LongText EN | OneIG ZH | OneIG EN |
|---|---|---|---|---|---|---|
| HiUni-Dense-4B | 17.50 | 17.23 | 0.109 | 0.085 | 0.338 | 0.289 |
| HiUni-Dense-2B | 17.15 | 16.98 | 0.100 | 0.028 | 0.319 | 0.262 |
| HiUni-MoT-2B | 18.50 | 18.37 | 0.157 | 0.141 | 0.358 | 0.331 |
| HiUni-Gen-4B | 23.22 | 22.65 | 0.361 | 0.230 | 0.409 | 0.371 |
Additional results for Table 5
| Configuration | Qwen-Image-Bench ZH | Qwen-Image-Bench EN | LongText ZH | LongText EN | OneIG ZH | OneIG EN |
|---|---|---|---|---|---|---|
| Pixel Feature | 17.50 | 17.23 | 0.109 | 0.085 | 0.338 | 0.289 |
| ViT Feature | 14.19 | 14.06 | 0.048 | 0.020 | 0.304 | 0.263 |
| No Gate | 12.71 | 12.13 | 0.014 | 0.004 | 0.284 | 0.249 |
The additional evaluations confirm the observations in Sections 4.3 and 4.4. HiUni-MoT-2B consistently outperforms HiUni-Dense-2B on generation, while HiUni-Dense-4B achieves substantially stronger understanding performance. The generation-only baseline performs best across all six generation metrics, further revealing the generation trade-off of joint training. For HiGate, Pixel Feature consistently outperforms ViT Feature and No Gate across all six generation metrics.
The Table 4–5 ablations are early-stage checkpoints trained before editing supervision is introduced. We therefore do not treat direct ImgEdit or GEdit evaluation of those checkpoints as a meaningful editing ablation. Instead, Table 3 evaluates the fully trained model, while Section 4.5 and Appendix E test the effects of adding editing supervision.
[W4] Data description and influence
We agree that training-data composition matters for interpreting cross-model results. Figure 3 and Appendix A.5 describe the main data categories, use of public and proprietary sources, and model-generated supervision. We provide further quantitative details here.
Data sources. We do not use BLIP3o-60k or similar public distilled image-generation datasets. The generation corpus mostly comprises real image–text pairs, internally synthesized images represent 3.9% and 5.3% of generation tokens in pretraining and SFT, respectively, and are used mainly for style control. The understanding corpus includes the training splits of datasets such as ScienceQA, Visual Genome, RefCOCO, COCO, and DocVQA, which are not included in the evaluation set. Approximately half of the understanding pre-training data contain model-generated captions, largely from Qwen-series models, and about 97% of understanding SFT examples contain model-generated reasoning trajectories to train a think-mode model. The editing corpus combines public datasets and internally generated edit pairs, as described in Appendix A.5.
Leakage checks. We perform text matching and image near-duplicate detection against the reported evaluation benchmarks. The detected image overlap rates were 1.2% for MMStar, 1.0% for MMBench, and below 0.5% for the other understanding benchmarks, primarily from shared underlying public image sources. For image generation, we detected 20 relative prompts in OneIG-EN (1.8%) and one in LongText-EN (0.6%), with none detected in the other evaluated generation sets.
Controlled Comparison. Since most compared models do not disclose their complete training data and recipe, we do not attribute differences in Tables 1–3 solely to architecture design. Controlled experiments in Section 4.3 use the same underlying training sources and configuration for Dense-2B and MoT-2B, differing in backbone sharing, while the single-task baselines probe task-mixture effects. These comparisons support architectural and optimization conclusions more directly than cross-paper leaderboard rankings.
Response to Reviewer Smg9 Part [3]
[W5] Dense versus MoT comparison
Before answering this question, we would like to clarify that our goal is not to establish the superiority of shared backbones over task-specific designs. We have revised the Introduction and Section 4.3 to acknowledge these results explicitly. Here, we would like to reiterate the motivation of HiUni and the purpose of our architectural comparisons.
First, HiUni aims to explore whether a pretrained VLM can support both understanding and generation without introducing task-specific backbone computation, rather than demonstrating that Dense is generally superior to MoT. This highly shared design allows both tasks to reuse the same pretrained backbone, providing a simpler alternative to task-specific architectures.
Second, the ability to inherit stronger pretrained representations is an intended advantage of the shared Dense design. Under a similar total parameter budget, HiUni-Dense-4B can directly reuse the pre-trained Qwen3-VL-4B backbone, whereas HiUni-MoT-2B uses two task-specific branches initialized from Qwen3-VL-2B. We acknowledge that this comparison involves different pre-trained initializations and is not intended to establish the inherent superiority of Dense. Instead, it highlights a practical advantage of backbone sharing that, a larger pre-trained VLM can be reused for both tasks within a similar total parameter budget.
Our central claim is consequently one of feasibility and design flexibility: a single shared backbone is a viable UMM alternative, even though specialized computation can improve generation performance with the same activated parameters.
[Q1] Why use a time token instead of adding timestep embeddings to each image patch?
We thank the reviewer for this question. The time-token conditioning follows prior models such as JanusFlow and Hunyuan-Image-3.0, which incorporate timestep embeddings as tokens into the Transformer sequence. We adopt this approach as a simple implementation choice compatible with our shared backbone. We have clarified the corresponding description and references in Section 3.2.
We have not conducted a controlled comparison with patch-wise timestep addition under otherwise identical settings. Since timestep conditioning is not the focus of HiUni, we do not claim that the time-token design is superior to alternative approaches, but regard it as an established implementation choice.
[Q2] Would removing understanding data improve generation performance?
Yes. To test this, we introduce HiUni-Gen-4B, which uses the same pretrained initialization, visual pathway, backbone, and 0.5B diffusion head as HiUni-Dense-4B, but trains on generation-only data rather than the joint task mixture. The experimental setup is an early comparison before introducing image editing.
As shown in Table 4, generation-only training improves the average OneIG score from 0.314 to 0.390. The additional evaluations in W3 show consistent improvements across all six generation metrics. In particular, OneIG ZH/EN increases from 0.338/0.289 to 0.409/0.371, with similar gains on Qwen-Image-Bench and LongText.
These results reveal two important insights. First, a pretrained VLM can be effectively adapted into a competitive image generation model, highlighting a promising direction for extending VLMs beyond visual understanding. Second, while HiUni focuses on introducing generation capabilities while preserving the understanding performance of pretrained VLMs, the stronger performance of HiUni-Gen-4B highlights the potential for further improving generation through more effective unified optimization, opening up opportunities to advance both capabilities together.
We thank the reviewer again for the valuable suggestions, and hope our additional experiments and revisions have adequately addressed the concerns.
Response to Reviewer Smg9 Part [1]
We sincerely thank the reviewer for the constructive feedback and for recognizing the value of our empirical study, the detailed training pipeline, and the promising generation performance at the 4B scale. We have revised the manuscript to better clarify our contributions, strengthen the architectural comparisons, and provide a more balanced analysis of the trade-offs in unified modeling. Our revisions and additional results are summarized below.
[W1] Insights into harmonizing understanding and generation
We agree that pixel-space diffusion and residual detail injection have precedents. The central question of HiUni is whether a pretrained VLM's semantic visual pathway and dense backbone can serve both understanding and generation, without a separate generative visual encoder or task-specific backbone computation. Our experiments lead to three insights.
First, a shared dense backbone enables effective reuse of pretrained VLM capabilities across understanding and generation. In the controlled 2B comparison, Dense and MoT have similar understanding performance (58.55 vs. 59.23), while MoT is stronger on generation (0.345 vs. 0.291). A 4B shared backbone, initialized from a larger pretrained VLM, offers a practical alternative with stronger understanding and competitive generation under a similar total parameter budget. Joint training also retains comparable understanding to an understanding-only baseline (65.04 vs. 64.58).
Second, semantic modeling and fine-grained visual reconstruction can be effectively decoupled. HiGate supplies raw noisy-pixel features only near the generation output, improving OneIG from 0.267 to 0.314 with little change in understanding (Section 4.4). This decouples the requirements of semantic modeling and fine-detail prediction without introducing task-specific backbone branches.
Third, architectural sharing does not guarantee uniformed positive transfer. Under a fixed total token budget, adding editing supervision increases average understanding from 62.64 to 64.49, while OneIG decreases from 0.538 to 0.528 (Section 4.5).
We therefore use harmonizing to describe supporting distinct tasks within a highly shared architecture, not to claim that all tasks mutually improve. These strengths and limitations provide a more precise characterization of highly shared UMMs. We believe that demonstrating this feasibility and identifying the benefits and limitations of sharing provides useful insights into unified multimodal modeling, laying the foundation for more mutually beneficial interactions between understanding and generation in the future.
[W2] Discussion of related work
We thank the reviewer for highlighting these closely related studies. We have expanded Section 2 to better position HiUni relative to prior work.
TUNA constructs a shared visual representation by cascading a VAE encoder with a representation encoder, while TUNA-2 adopts an encoder-free design based on direct pixel embeddings. Cheers combines a VAE encoder, a VAE decoder, and a ViT as the visual encoder, and injected the output features of VAE into the diffusion head through residuals. In contrast, HiUni focuses on directly reusing the pretrained VLM's language-aligned ViT, patch merger, and dense backbone, and injects raw noisy-pixel features at the output side.
DiP and DeCo demonstrate the benefit of preserving pixel-level details in image generation. DiP uses a Patch Detailer Head, while DeCo introduces a pixel decoder conditioned on semantic features. HiUni does not claim novelty in such mechanism, its distinction lies in integrating pixel-level detail injection into a unified model that shares the same pretrained visual encoder and dense backbone across understanding and generation.
RepFusion is particularly relevant because it also processes noisy visual inputs using a pretrained multimodal model. However, it conditions a separate diffusion Transformer in representation space. In contrast, HiUni processes noisy and clean images through the same visual encoder and dense backbone and predicts pixels through a lightweight task-specific head.
These comparisons clarify that HiUni contributes a novel approach to architectural sharing and pretrained VLM reuse for UMM, rather than claiming novelty in individual generation components. We have added the relevant references and clarified these connections in Section 2.