acceptodds
Under review as a conference paper at ICLR 2027

UniFacet: One Visual Space, Complementary Facets for Image Generation and Editing

Abstract

Unified image generation and editing require a visual representation that preserves both semantic structure and fine-grained image details. Existing unified tokenizers often fine-tune pretrained vision foundation models (VFMs) under semantic and reconstruction supervision. We introduce UniFacet, a unified visual tokenizer that reformulates this problem as completing a pretrained semantic representation with reconstruction-relevant details. UniFacet keeps the VFM frozen as a semantic anchor and learns a lightweight residual branch to recover missing image-specific information, while regularizing its magnitude to obtain the necessary reconstruction details with minimal modification to the pretrained semantic structure. A lightweight projection compresses the resulting unified representation into a compact reconstruction facet for image decoding. To provide complementary semantic information for generative modeling, we derive a semantic facet from the same unified representation and combine it with the reconstruction facet as the generative target. We instantiate UniFacet in UniFacet-1.5B, a unified model for text-to-image generation and instruction-guided image editing. The model uses the unified representation for source-image conditioning and jointly predicts the reconstruction and semantic facets for image synthesis. Experiments demonstrate high-fidelity reconstruction with largely preserved pretrained semantic capabilities, together with strong text-to-image generation and instruction-guided image editing performance.

Primary area:
generative models
Keywords:
visual tokenizer

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_iXCiOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2(+1)
Soundness
2
Read review
Paper summary

This paper presents UniFacet, a visual tokenizer for image generation and editing. It supplements a frozen visual encoder with a regularized residual branch to recover reconstruction-relevant details. The completed representation is used for source-image conditioning, while compact reconstruction and semantic facets form the generation target. Experiments evaluate reconstruction, semantic preservation, generation, and editing, with UniFacet-1.5B achieving 0.89 on GenEval and 7.47 on GEdit-Bench-EN.

Critical strengths
  1. The design of separation between source-image conditioning and generation targets is reasonable , since source conditioning needs to preserve image details, whereas generation benefits from a compact target that is easier to model.

  2. Table 6 provides a convincing comparison of the overall design. With the same backbone, training data, and training schedule, UniFacet improves GEdit-EN from 6.29 to 7.35 over the dual-stream baseline while halving the source-image token sequence. The reconstruction and understanding evaluations also support the feasibility of recovering image details while largely preserving semantic capabilities.

Critical weaknesses
  1. Although the additional branch is called a semantic facet, its semantic content is not directly evaluated. The understanding and alignment experiments assess the high-dimensional unified representation rather than the learned 16-dimensional projection. Table 5 shows that the branch helps generation, but does not establish what semantic information it retains or how that information contributes to the improvement.

  2. The benefit of jointly adapting the semantic projection is not sufficiently isolated. Table 5 lacks a fixed-projection baseline with the same dimensionality and training budget. The comparison in Appendix I addresses direct semantic alignment rather than this question. It therefore remains unclear whether joint adaptation is necessary or whether a fixed auxiliary representation would provide similar gains.

  3. After learning the semantic projection in the preliminary 50k-step training run, the paper states that the projection is reused, but does not explain whether the generator weights are also carried over. If they are reused, the final generator has already received this additional training. The authors should clarify which generator checkpoint is used to start the subsequent multimodal training.

Questions for authors
  1. What evidence shows that the learned 16-dimensional facet retains semantic information, and does joint adaptation outperform a fixed projection under matched dimensions and training budgets?

  2. Are the generator weights from the preliminary 50k-step run reused for subsequent multimodal training?

4 comments
AuthorsOpenReview ↗

Response to Reviewer iXCi

We sincerely thank the reviewer for the careful reading and constructive questions. In response, we add a direct probing analysis of the semantic facet (W1), clarify how the semantic projection is obtained and used (W2), and specify the checkpoint used to start multimodal training (W3).

W1 & Q1: What evidence shows that the learned 16-dimensional facet retains semantic information?

We directly analyze the frozen 16-d semantic facet zsemz_{\mathrm{sem}} used in UniFacet-1.5B, and compare it with the 64-d reconstruction facet zrecz_{\mathrm{rec}} and the 1280-d VFM features zvfmz_{\mathrm{vfm}}.

Qualitative evidence. This figure compares per-image PCA visualizations of zvfmz_{\mathrm{vfm}} and zsemz_{\mathrm{sem}}. Despite the compression to 16 dimensions, zsemz_{\mathrm{sem}} keeps the object-level layout of the VFM features. Objects such as people, horses, the bird and the teddy bear form coherent regions with clear boundaries against the background.

Quantitative evidence. We fit linear foreground/background probes on an eight-breed Oxford-IIIT Pet subset (256/64/192 train/val/test images). The tokenizer and projections are frozen; only the linear readouts are trained. We report:

  • mIoU: binary foreground/background mIoU.
  • Content-centered mIoU: the probe is refitted after subtracting the training-set mean feature at each spatial position, which removes positional priors.
  • Mask-transfer FG IoU: cross-image foreground-mask transfer via nearest-neighbor feature matching on 64 held-out same-breed pairs.
FeatureChannelsmIoUContent-centered mIoUMask-transfer FG IoU
zsemz_{\mathrm{sem}}1696.178.788.3
zrecz_{\mathrm{rec}}6472.256.539.3
zvfmz_{\mathrm{vfm}}128098.692.994.2
  1. zsemz_{\mathrm{sem}} nearly matches the VFM, whereas zrecz_{\mathrm{rec}} does not, despite having 4× more channels. The gap therefore reflects what each facet encodes rather than its capacity. In the segmentation examples, zsemz_{\mathrm{sem}} yields complete silhouettes nearly identical to those of zvfmz_{\mathrm{vfm}}, including thin parts such as the beagle's legs and tail, while zrecz_{\mathrm{rec}} masks are fragmented and follow texture rather than object extent (e.g., the striped blanket in the Egyptian Mau example).
  2. The signal is content-driven, not positional. After centering, zsemz_{\mathrm{sem}} still reaches 78.7%, 22.2 points above zrecz_{\mathrm{rec}}.
  3. zsemz_{\mathrm{sem}} is consistent across images. Same-breed mask transfer reaches 88.3% IoU with zsemz_{\mathrm{sem}} versus 39.3% with zrecz_{\mathrm{rec}}; such cross-image correspondence is characteristic of semantic rather than appearance features. The remaining gap to zvfmz_{\mathrm{vfm}}, mainly in the centered setting, is expected given the 80× compression.

How this relates to the generation gain. The decoder consumes only zrecz_{\mathrm{rec}}, so zsemz_{\mathrm{sem}} cannot help by adding decoding capacity. Together with the probes, Table 5 indicates that the gain comes from exposing object-level structure that zrecz_{\mathrm{rec}} lacks, rather than from adding latent channels: (i) the gain is insensitive to dsemd_{\mathrm{sem}} (0.70–0.71 for 8/16/32 channels); and (ii) widening zrecz_{\mathrm{rec}} from 64 to 128 channels lowers GenEval from 0.65 to 0.57, whereas adding a 16-d semantic facet raises it to 0.66.

We will add this analysis and the figures to the revision.

W2 & Q1: Is joint adaptation necessary, or would a fixed projection of the same dimension provide similar gains?

We first note that the semantic projection in UniFacet is also fixed during final training. It is adapted only during the first 30k steps of the preliminary run, and is then frozen together with its normalization statistics for all multimodal training of UniFacet-1.5B (Appendix F.2). The joint-adaptation step serves a single purpose: to learn adaptively a compact semantic projection whose features suit flow-matching generation. The question is therefore not whether the projection is fixed, but how its weights are obtained.

We considered two alternatives for obtaining this fixed projection:

  • Fixed PCA projection. CoReDi reports that replacing the jointly learned projection with a fixed PCA projection degrades generation quality. PCA retains the directions of largest variance in the upstream features, which need not be the directions most useful for generative modeling.
  • Projection learned with a separate semantic objective. Training the projection with another semantic objective (e.g., feature reconstruction or distillation) would require an additional training stage and data, and offers no guarantee that the resulting features suit generative modeling.

We therefore adopt the more natural option of learning the projection jointly with the generative model, following CoReDi. It selects the projection directly under the generative objective, at negligible cost: a single 1280→16 linear layer adapted for 30k steps.

Our main contribution is the semantic facet itself: it is derived from the unified space and generated jointly with the frozen reconstruction facet. The specific procedure used to obtain its projection is not part of this contribution. Table 5 and the probing results in our previous response support this design. We agree that a systematic comparison of projection strategies would be informative and will explore it in future work. We will also state explicitly in Sec. 3.1.2 that the adaptation strategy follows CoReDi.

W3 & Q2: Which generator checkpoint initializes the subsequent multimodal training?

Thank you for pointing out this ambiguity. No generator weights from the preliminary 50k-step run are carried over. The only transferred components are the semantic projector, a single linear layer, and its normalization statistics, both kept frozen. Curriculum phase C1 initializes the generator anew from Qwen2.5-1.5B-Instruct. The final generator therefore receives no training beyond the C1–C4 and SFT schedule in Table 8. We will state this initialization and parameter transfer explicitly in Appendix F.2.

We hope these responses address the reviewer's concerns, and we are happy to provide further clarification.

Dear Reviewer iXCi,

Thank you again for your careful review. With the discussion period drawing to a close, we would like to check whether our response has resolved your concerns. In it we added a direct probing analysis of the 16-d semantic facet, explained how its projection is obtained and used, and clarified that no generator weights are carried over from the preliminary run.

If anything remains unclear, or you would like further analysis, we would be glad to use the remaining time to provide it. Thank you for your time and consideration.

Best regards,

The Authors

Thank you for the detailed responses and for revising the unclear parts of the manuscript. My concerns have been addressed.

Reviewer_mu7GOpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents a unified visual tokenizer using a frozen VFM and a regularized residual branch. In addition, it presents a unified model for image generation and image editing

Critical strengths

The unified model with a smaller number of parameters achieves comparable performance on some metrics.

Critical weaknesses
  • The technical contribution seems limited. Prior work has proposed using a residual encoder to complement the frozen VFM features, such as [1-2], and using joint flow matching, such as [3, 4]. As for the dimension analysis of the latent feature, prior work has also investigated this, such as [5].

  • Experiments

  1. It may be difficult to isolate the contribution of UniFacet from the comparisons with prior methods in Tables 1, 2, and 3, as these methods appear to use different training data or settings.
  2. It would also be helpful to report throughput in images per second under the same inference conditions to demonstrate the efficiency benefits of UniFacet-1.5B compared with previous methods such as BAGEL.
  3. A comparison between the final UniFacet-1.5B model and previous methods on image understanding benchmarks is missing.

[1] Shi, Minglei, et al. "Latent diffusion model without variational autoencoder." International Conference on Learning Representations 2026.

[2] Liu, Siyu, et al. "Improving reconstruction of representation autoencoder." arXiv preprint arXiv:2602.08620.

[3] Kouzelis, Theodoros, et al. "Boosting generative image modeling via joint image-feature synthesis." Neural Information Processing Systems 2025.

[4] Kouzelis, Theodoros, Spyros Gidaris, and Nikos Komodakis. "Coevolving representations in joint image-feature diffusion." European Conference on Computer Vision 2026.

[5] Zhang, Shilong, et al. "Both semantics and reconstruction matter: Making representation encoders ready for text-to-image generation and editing." International Conference on Machine Learning 2026.

Questions for authors
  • If increasing the size of UniFacet-1.5B, e.g., to 7B, how is the performance?

  • Could the paper also report FID or FDD, like [2, 5]?

8 comments
AuthorsOpenReview ↗

Response to Reviewer mu7G

We thank the reviewer for the constructive feedback and for recognizing that UniFacet achieves competitive performance with a smaller backbone. We address each concern below.

W1: Technical contribution relative to [1–5].

We thank the reviewer for these references. We discuss [1, 2] in Appendix D.2 and build on [3, 4] in Sec. 3.1.2; we will add [5] to the related work. We do not claim residual completion, joint image–feature flow matching, or latent-dimension analysis as standalone novelties. Below we clarify how UniFacet differs in (1) task setting, (2) technical design, and (3) experimental findings.

(1) Task setting. [1–4] develop and evaluate latent spaces for class-conditional ImageNet generation, a single-task setting in which the latent only needs to be generated and decoded. UniFacet targets a unified model for text-to-image generation and instruction-guided editing, in which the same visual interface must also encode source images for editing and understanding. These roles have different requirements: source conditioning benefits from rich, high-dimensional information, whereas generation benefits from a compact target. Reconciling them within one visual representation is the problem UniFacet addresses, and it does not arise in the class-conditional setting of [1–4]. [5] targets T2I and editing as we do, but organizes semantics differently (see below).

(2) Technical design.

  • vs. residual completion [1, 2]: SVG and LV-RAE complete frozen VFM features with residual details and perform diffusion in the completed high-dimensional space. UniFacet uses the completed space zuniz_{\mathrm{uni}} only as what the model reads, and generates a compact 80-d target projected from it.
  • vs. joint image–feature modeling [3, 4]: in ReDi and CoReDi, the image latent and the semantic target come from separate VAE and VFM encoders. In UniFacet, the source-conditioning tokens and both generative facets come from a single completed space in one encoder pass. We adopt CoReDi's projection adaptation (Sec. 3.1.2) and do not claim it as a contribution.
  • vs. PS-VAE [5]: PS-VAE supervises a single compact latent to reconstruct both semantic features and pixels. UniFacet keeps zrecz_{\mathrm{rec}} purely reconstructive and exposes semantics through a separate facet derived from the same space.

(3) Experimental findings for generation and editing. Under controlled settings, we establish the following findings, which the ImageNet studies in [1–4] do not address:

  • Editing: single-stream zuniz_{\mathrm{uni}} conditioning outperforms BAGEL's ViT+VAE dual-stream conditioning with half the source tokens (GEdit-EN 7.35 vs. 6.29, Table 6). Removing the residual at inference drops the score to 3.78, whereas a norm-matched random perturbation only lowers it to 6.92, so the editing model relies on the recovered details.
  • Understanding: replacing zvfmz_{\mathrm{vfm}} with zuniz_{\mathrm{uni}} does not degrade understanding (Table 4), so one representation serves both understanding and editing.
  • T2I: the complementary semantic facet improves GenEval from 0.65 to 0.70, and from 0.57 to 0.66 with a 128-d zrecz_{\mathrm{rec}} (Table 5); what matters is which information the target exposes, not dimensionality alone. In our framework, it also outperforms a PS-VAE-inspired direct alignment of zrecz_{\mathrm{rec}} in both reconstruction (+0.16–0.33 dB PSNR) and generation (0.71 vs. 0.68, Appendix I), as well as REPA-style supervision (0.70 vs. 0.67).
  • System level: with 1.5B parameters, UniFacet-1.5B reaches 0.89 on GenEval and 7.47 on GEdit-Bench-EN, exceeding the 7B BAGEL by 0.95 on GEdit-Bench-EN with 3.0–5.4× higher throughput.

We will sharpen this positioning in Sec. 1 and Appendix D.

W2.1: Isolating the contribution of UniFacet from differences in training data and settings.

We agree that Tables 1–3 compare final systems trained with different data and recipes, and therefore do not by themselves isolate UniFacet's contribution. A fully matched comparison with these methods is, however, not feasible. Many of them are trained on large proprietary or unreleased datasets, and the baselines also differ from one another in data composition, scale, and training recipe, so there is no common setting to align with. For this reason, system-level comparisons in this area conventionally report results on standard benchmarks. To isolate UniFacet's contribution, we instead rely on controlled comparisons in which only the component under study changes, with the backbone, data, schedule, and evaluation held fixed (Sec. 4.4, Appendix F.4):

Controlled comparisonResult
Editing tokenizer, 300k steps (Table 6): BAGEL ViT / BAGEL ViT+VAE / UniFacetGEdit-EN 5.92 / 6.29 / 7.35 (N / 2N / N source tokens)
Source representation for understanding (Table 4): zvfmz_{\mathrm{vfm}} → zuniz_{\mathrm{uni}}Avg. 67.83 → 68.28 (RADIOv4), 66.58 → 66.68 (SigLIP2)
Generative target, 50k steps (Table 5): zrecz_{\mathrm{rec}} → (zrec,zsem)(z_{\mathrm{rec}}, z_{\mathrm{sem}})GenEval 0.65 → 0.70 (drecd_{\mathrm{rec}}=64); 0.57 → 0.66 (drecd_{\mathrm{rec}}=128)
Direct semantic alignment → complementary facet (App. I)GenEval 0.68 → 0.71; PSNR +0.16–0.33 dB
REPA-style supervision → joint generation (new, 50k steps)GenEval 0.67 → 0.70
Source condition at inference: zvfmz_{\mathrm{vfm}} / norm-matched random perturbation / zuniz_{\mathrm{uni}} (new)GEdit-EN 3.78 / 6.92 / 7.35

The Table 6 comparison is the most direct system-level control: with backbone, data, and schedule fixed and only the visual tokenizer changed, UniFacet improves GEdit-EN from 6.29 to 7.35 while using half as many source tokens. Table 1 evaluates all released tokenizers under one shared protocol (Appendix E.2). We will make these controlled comparisons more prominent in the revision.

W2.2: The efficiency benefits of UniFacet-1.5B compared with BAGEL

We thank the reviewer for this suggestion and now report throughput in images per second.

Setup. All models run on a single NVIDIA A100-40GB GPU with BF16 precision, batch size 1, 512×512 resolution, and 50 denoising steps, with classifier-free guidance (CFG) applied over the full sampling interval and each model's default text CFG scale. For editing, both models use the same source image and instruction. We report UniFacet with two CFG execution modes: sequential, which runs the guidance branches one after another as in BAGEL's official implementation, and packed, which batches them into a single forward pass and is used in our release. For each setting, we perform two warmup runs followed by 16 timed runs, measuring CUDA-synchronized wall-clock time and excluding model loading and image saving.

TaskModelCFG executionLatency (s/img)Throughput (img/s)Peak Allocated Memory (GiB)
T2IUniFacet-1.5Bpacked3.830.2617.87
T2IUniFacet-1.5Bsequential5.670.1767.80
T2IBAGELsequential11.730.08528.63
EditUniFacet-1.5Bpacked3.930.2548.00
EditUniFacet-1.5Bsequential5.830.1727.85
EditBAGELsequential20.490.04928.89
EditBAGEL*sequential13.170.07628.89

* BAGEL without image CFG (image CFG scale 1.0), i.e., without the additional guidance branch on the source image, so that it runs the same two forward passes per step as UniFacet. BAGEL's default editing setting requires image CFG (scale 2.0), which corresponds to the row above.

UniFacet-1.5B achieves:

  • ~2.1× / ~3.1× higher T2I throughput than BAGEL (sequential / packed CFG)
  • ~3.5× / ~5.2× higher editing throughput than BAGEL with its default setting, and ~2.3× / ~3.3× than BAGEL* without image CFG (sequential / packed CFG)
  • ~3.6–3.7× lower peak allocated GPU memory

These results demonstrate that UniFacet-1.5B provides substantial efficiency advantages in practical inference settings, in addition to competitive multimodal quality. We will include this throughput comparison in the revised manuscript.

W2.3: Understanding comparison of the final model.

We thank the reviewer for this suggestion. Since the paper focuses on generation and editing, the main text only includes the controlled understanding ablation (Table 4). We have now evaluated the final UniFacet-1.5B checkpoint against understanding-only and unified models of comparable scale on six benchmarks:

ModelSEEDMMStarMMBOCRBenchRWQAAI2DAvg
InternVL2-2B70.950.173.878.556.774.367.4
Show-o54.429.626.73.332.331.629.7
Harmon-1.5B65.235.750.211.648.055.444.3
Janus-Pro-1B66.943.160.249.054.566.956.8
UniFacet-UMM-1.5B (Ours)75.953.373.266.864.375.968.2

These results show that UniFacet-UMM-1.5B achieves strong image understanding performance compared with models of similar size.

We will add this comparison table to the revised paper.

Q1: Scaling UniFacet from 1.5B to 7B

We have not trained a 7B UniFacet model and therefore cannot provide a verified performance estimate at that scale. As acknowledged in Appendix C, larger-backbone experiments were outside our available computational budget.

Q2: Reporting FID or FDD.

We thank the reviewer for this suggestion. Of the two distribution-level metrics suggested, we report reconstruction FID (rFID), which complements the PSNR/SSIM results in Table 1. We evaluate rFID on all 50,000 ImageNet validation images at 512×512, using the same ground-truth preprocessing and the Clean-FID implementation for all models.

ModelTyperFID ↓
FLUX VAEreconstruction-oriented0.041
Qwen-Image VAEreconstruction-oriented0.075
NextStep VAEreconstruction-oriented0.090
UniFacet-RADIOv4 (64-ch)unified0.122
UniTok-Largeunified0.245
DualViTokunified0.461
UniLIP-1Bunified0.535

UniFacet achieves the lowest rFID among unified tokenizers, about 2× lower than the next best (UniTok-Large). It substantially narrows the gap to reconstruction-oriented VAEs (0.041–0.090) while using 4× fewer tokens (16× vs. 8× downsampling) and preserving pretrained semantics (CKNNA 0.996). This ranking is consistent with the PSNR/SSIM results in Table 1, where FLUX VAE also leads.

Generation FID/FDD. The generation FID/FDD reported in [2, 5] is measured on class-conditional ImageNet generation, where a fixed label set and a shared reference distribution make these metrics the standard. For text-to-image generation and editing, unified models are mainly compared on benchmarks such as GenEval and DPG-Bench (Table 2) and GEdit-Bench and ImgEdit (Table 3). gFID is less commonly reported in this setting, since it depends on the choice of prompt set and reference images and does not measure text alignment or instruction following. We therefore follow these benchmarks for generation and editing.

We will include the rFID results and evaluation details in the revision.

Reviewer_T9aJOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

UniFacet constructs a unified visual representation by adding a learned, magnitude-regularized residual to frozen vision-foundation-model features. It projects this representation into a compact reconstruction facet and learns a separate semantic facet jointly with a diffusion model, keeping the reconstruction pathway fixed. A BAGEL-style model initialized from Qwen2.5-1.5B uses the unified features for source-image conditioning and jointly generates both facets, decoding only the reconstruction facet. Validation covers reconstruction, preservation of visual-understanding performance, text-to-image generation, and instruction-guided editing.

Critical strengths
  1. UniFacet achieves strong reconstruction while largely preserving visual-understanding performance. It reaches 32.95 dB PSNR on COCO-512, compared with 26.60 for UniTok and 24.09 for DualViTok, supporting the usefulness of a shared representation for both semantics and reconstruction.

  2. A controlled comparison supports the practical benefit for editing. With the same backbone, data, and training schedule, UniFacet scores 7.35 on GEdit-Bench-EN versus 6.29 for dual-stream BAGEL, while requiring only one source-image sequence.

Critical weaknesses
  1. The key design choice is not directly tested. UniFacet’s main distinction is that its reconstruction and semantic facets come from the same completed representation. The experiments show that adding a semantic facet improves generation, but do not compare a semantic facet derived from zuniz_{\mathrm{uni}} with one derived from zvfmz_{\mathrm{vfm}}. This leaves unclear whether the shared completed space is necessary for the reported gain.

  2. The benefit of generating both facets is not fully established. The model generates zrecz_{\mathrm{rec}} and zsemz_{\mathrm{sem}} together, but only zrecz_{\mathrm{rec}} is used to decode the image. The results show that adding a semantic facet helps generation, but do not establish whether jointly generating it is better than using semantic supervision during training without generating that facet at inference.

  3. Editing perceptual quality remains a weakness. UniFacet scores 7.06 on English and 7.00 on Chinese GEdit-Bench perceptual quality, below several listed models.

Questions for authors
  1. What explains the large performance drop when zuniz_{\mathrm{uni}} is replaced by zvfmz_{\mathrm{vfm}} for source-image conditioning in Table 6? Because this is an inference-time swap that creates a train–test representation mismatch, a matched comparison controlling for that mismatch would help determine whether the drop reflects reliance on residual information or sensitivity to the changed input distribution.

  2. What is the benefit of jointly generating zrecz_{\mathrm{rec}} and zsemz_{\mathrm{sem}}, compared with generating zrecz_{\mathrm{rec}} alone and applying semantic supervision during diffusion training? This comparison would clarify whether the semantic facet must be sampled at inference to improve generation, or whether an auxiliary training objective could provide a similar benefit without expanding the generated representation.

  3. How much does the 512-pixel long-side limit affect the GEdit-Bench perceptual-quality scores? A controlled evaluation on square images or a resolution-matched comparison would help establish whether output resolution accounts for the observed gap in perceptual quality.

3 comments
AuthorsOpenReview ↗

Response to Reviewer T9aJ

We sincerely thank the reviewer for the thoughtful and detailed review. In response, we add an ablation that derives the semantic facet from zvfmz_{\mathrm{vfm}} (W1), a comparison with REPA-style semantic supervision (W2/Q2), a norm-matched perturbation control for Table 6 (Q1), and a resolution-matched PQ evaluation (W3/Q3).

W1: Is the shared completed space necessary for the generation gain?

We would like to clarify that the completed shared space and the downstream facets serve distinct roles. The completed representation zuniz_{\mathrm{uni}} provides an information-rich space containing both pretrained semantics and reconstruction-relevant details. Its value is particularly evident in image editing, where a single source-image token stream supplies both semantic and fine-grained information, enabling strong editing performance.

Each facet extracts information from this shared space according to its task. The semantic facet primarily captures semantic information inherited from zvfmz_{\mathrm{vfm}}. Our ablation shows that using zuniz_{\mathrm{uni}} or zvfmz_{\mathrm{vfm}} as the input to the semantic projection yields nearly identical gains in generation performance. This is consistent with our design: the shared space preserves information needed across tasks, while each facet selectively extracts what it needs. There is therefore no requirement for every facet to depend on the details introduced by residual completion.

W2 & Q2: Joint generation of zsemz_{\mathrm{sem}} vs. semantic supervision without generating it.

We thank the reviewer for suggesting this comparison. We trained a zrecz_{\mathrm{rec}}-only model with a REPA objective that aligns intermediate generator features with the frozen VFM features zvfmz_{\mathrm{vfm}}.  All other settings follow the Table 5 protocol (50k steps):

Generative configurationGenEval ↑
zrecz_{\mathrm{rec}} only0.65
zrecz_{\mathrm{rec}} only + REPA0.67
Joint (zrec+zsemz_{\mathrm{rec}} + z_{\mathrm{sem}})0.70

Semantic supervision helps (+0.02), but explicitly generating the semantic facet yields a larger gain (+0.05). These results support the effectiveness of using semantic features as an explicit joint generation target.
We will include this additional ablation in the revised manuscript.

W3: Editing perceptual quality on GEdit-Bench.

We thank the reviewer for this observation. We agree that PQ is the weakest of our GEdit-Bench metrics, as discussed in Appendix L. We attribute most of the gap to two factors, and we verify the second with a controlled experiment.

(1) Model scale. Every open-source model in Table 3 with a higher PQ than UniFacet is substantially larger, with 4–13× as many parameters.

(2) Training and output resolution. We believe this is the more important factor. The models with higher PQ are trained and run at around 1024 pixels or higher, whereas UniFacet, due to limited compute, caps the long side at 512 pixels in both training and inference (Appendix C; Limitations). PQ scores naturalness and visual artifacts, so it is directly sensitive to resolution; the official GEdit-Bench scoring prompt even lists blurred faces as an example artifact. A model run at 1024 pixels produces roughly 4× as many pixels, yielding finer texture, sharper edges, and clearer faces and text. This favors it on PQ independently of how well it follows the instruction.

(3) Controlled evidence for the resolution effect. To isolate this effect, we re-scored all 606 English GEdit-Bench outputs of Step1X-Edit v1.1 after downsampling them to a 512-pixel long side. PQ drops from 7.36 to 7.18 (−0.18), and by −0.22 for elongated images (aspect ratio >1.5). The 512-pixel cap alone thus explains a meaningful part of UniFacet's PQ gap (see Q3 for details).

Q1: The drop when zuniz_{\mathrm{uni}} is replaced by zvfmz_{\mathrm{vfm}} in Table 6.

Table 6 is intended to show that the complementary information recovered by residual completion remains important for editing, even though zuniz_{\mathrm{uni}} and zvfmz_{\mathrm{vfm}} are strongly aligned (CKNNA 0.996). Replacing zuniz_{\mathrm{uni}} with zvfmz_{\mathrm{vfm}} removes this information and sharply reduces editing performance, indicating that high feature alignment does not imply interchangeability as a source condition.

As the reviewer notes, this inference-time swap also introduces a train–test mismatch. To test whether the drop merely reflects generic sensitivity to perturbed inputs, we added a token-wise norm-matched random-perturbation control, which matches the magnitude of residual removal at each token while randomizing its direction. For each source token, we compute ri=zuni,i−zvfm,ir_i = z_{\mathrm{uni},i} - z_{\mathrm{vfm},i}, sample ϵi∼N(0,I)\epsilon_i \sim \mathcal{N}(0, I), and construct

ηi=∥ri∥2 ϵi∥ϵi∥2+ε,\eta_i = \|r_i\|_2 \, \frac{\epsilon_i}{\|\epsilon_i\|_2 + \varepsilon},

where ε\varepsilon is a small constant for numerical stability. We then condition on zuni,i−ηiz_{\mathrm{uni},i} - \eta_i. This perturbation has the same per-token magnitude as replacing zuniz_{\mathrm{uni}} with zvfmz_{\mathrm{vfm}}, but does not specifically remove the learned residual. Results on GEdit-Bench-EN:

Source condition at inferenceSC ↑PQ ↑Overall ↑
zuniz_{\mathrm{uni}} (as trained)8.356.887.35
zuni−ηz_{\mathrm{uni}} - \eta (norm-matched random)8.226.336.92
zvfm=zuni−rz_{\mathrm{vfm}} = z_{\mathrm{uni}} - r (residual removed)5.462.853.78

Under a perturbation of identical magnitude, the overall score decreases only from 7.35 to 6.92, whereas removing the learned residual reduces it to 3.78. Generic sensitivity to inference-time distribution shift therefore does not appear to be the primary cause of the drop. Instead, the result points to the loss of the complementary information carried by zuniz_{\mathrm{uni}}, supporting the value of residual completion for image editing. We will include this control and its analysis in the appendix and reference it from the discussion of Table 6 in Sec. 4.4.

Q3: How much does the 512-pixel long-side limit affect the GEdit-Bench perceptual-quality scores?

We thank the reviewer for this suggestion and ran the requested controlled, resolution-matched evaluation.

Setup. We used all 606 English GEdit-Bench outputs of Step1X-Edit v1.1. Each output was scored under two conditions:

  • Original: the released output with the official GEdit area normalization.
  • Capped: the same output downsampled to a 512-pixel long side, then given the same official normalization.

Results. Capping the resolution consistently lowers PQ:

SettingPQΔ
Original7.36—
Capped7.18−0.18

The 512-pixel long-side limit measurably lowers GEdit-Bench PQ (−0.18 overall). The drop grows with aspect ratio: −0.16 for moderate aspect ratios (1.1–1.5) and −0.22 for elongated images (>1.5), which lose the most pixels under the cap. Resolution therefore accounts for a meaningful part of the observed PQ gap. We will add this analysis to the appendix.

Dear Reviewer T9aJ,

Thank you again for your thoughtful and detailed review. With the discussion period drawing to a close, we would like to check whether our response has resolved your concerns. In it we added the semantic-facet ablation derived from z_vfm, a comparison with REPA supervision, a norm-matched perturbation control for Table 6, and a resolution-matched PQ evaluation.

If any question remains open, or you would like another control or analysis, we would be glad to run it in the remaining time. Thank you for your time and consideration.

Best regards,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Overall Response

We thank all reviewers for their careful reading. They recognized the strong reconstruction with largely preserved understanding (T9aJ), the controlled editing comparison with a halved source-token sequence (T9aJ, iXCi: 7.35 vs. 6.29 on GEdit-EN), the reasonable separation of source conditioning and generation targets (iXCi), and competitive performance with fewer parameters (mu7G). We summarize the main concerns below; details are in the individual replies. We have uploaded a revised PDF in which new and changed text is marked in blue.

1. Is the semantic facet useful, and what does it encode? (T9aJ, iXCi)

  • Generating vs. supervising (T9aJ W2/Q2): a zrecz_{\mathrm{rec}}-only generator with REPA-style supervision reaches 0.67 GenEval (0.65 without), whereas jointly generating zsemz_{\mathrm{sem}} reaches 0.70 (Appendix I.2).
  • Semantic content (iXCi W1/Q1): in linear foreground/background probes on Oxford-IIIT Pet, the 16-d zsemz_{\mathrm{sem}} reaches 96.1 mIoU, close to zvfmz_{\mathrm{vfm}} (98.6) and far above the 64-d zrecz_{\mathrm{rec}} (72.2); the same holds after removing positional priors (78.7 vs. 56.5) and for cross-image mask transfer (88.3 vs. 39.3). PCA visualizations are included (Appendix N).
  • Projection and initialization (iXCi W2/W3): following CoReDi, the semantic projection is adapted only during the first 30k steps of a preliminary run and then frozen. No generator weights are carried over; the generator is initialized anew from Qwen2.5-1.5B-Instruct (Appendix F.2).

2. Does the editing gain depend on the recovered residual? (T9aJ Q1) A token-wise norm-matched random perturbation, which matches the magnitude of residual removal but not its direction, lowers GEdit-EN only from 7.35 to 6.92, whereas removing the residual gives 3.78. The drop is therefore not mainly generic sensitivity to a train-test mismatch (Sec. 4.4, Appendix G.2).

3. Editing perceptual quality (T9aJ W3/Q3) Every open-source model in Table 3 with higher PQ is 4-13x larger. In a controlled re-scoring of Step1X-Edit v1.1 outputs, capping the long side at 512 pixels lowers PQ from 7.36 to 7.18 (-0.18, and -0.22 for aspect ratios above 1.5), so the 512-pixel cap explains part of the gap (Sec. 4.3, Appendix L).

4. Positioning relative to [1-5] (mu7G W1) [1-4] study latents for ImageNet generation, where the latent only needs to be generated and decoded. UniFacet targets a unified model for text-to-image generation and editing, where the same visual interface must also encode source images; this favors rich high-dimensional features for source conditioning but a compact target for generation. Technically, SVG/LV-RAE diffuse in the completed high-dimensional space, ReDi/CoReDi take image latents and semantic targets from separate VAE and VFM encoders, and PS-VAE supervises one compact latent for both semantics and pixels; UniFacet reads zuniz_{\mathrm{uni}} and generates a compact 80-d target projected from it. We added PS-VAE [5] and sharpened this positioning in Sec. 1, Sec. 2 and Appendix D.

5. Efficiency and understanding (mu7G W2.2/W2.3)

  • Throughput (A100-40GB, BF16, 512x512, 50 steps, batch size 1): UniFacet-1.5B is 2.1-3.1x faster than BAGEL for T2I and 3.5-5.2x for editing (2.3-3.3x against BAGEL without image CFG), with 3.6-3.7x lower peak memory (Appendix O).
  • Understanding: the final model achieves the highest average among the compared models across six benchmarks (68.2), versus 67.4 for InternVL2-2B, 66.9 for UniLIP-1B, 56.8 for Janus-Pro-1B, and 55.1 for Show-o2-1.5B. It leads on SEED-Bench, MMStar, RealWorldQA, and AI2D (Appendix M)..

6. Controlled vs. system-level comparisons (mu7G W2.1) Tables 1-3 compare final systems trained on different data, and a fully matched comparison is infeasible. The controlled evidence, with backbone, data, schedule and evaluation fixed, is Tables 4-6 and Appendix I: 6.29 to 7.35 GEdit-EN (Table 6), 0.65 to 0.70 GenEval (Table 5), and 0.71 and 0.70 GenEval for the complementary facet versus 0.68 for direct alignment and 0.67 for REPA (Appendix I).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.