acceptodds
Withdrawn from ICLR 2027

ReCoDe: Re-Conditioned Decoding for Faithful Controllable Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Controllable image generation conditions a diffusion model on spatial conditions, but only while the latent is being generated. The decoder that renders that latent never sees the condition, so structure is lost in the pixels: boundaries blur, thin lines break, and small objects fade. In this work, we propose to apply the condition twice, once to shape the latent and once in the decoder. Our method, dubbed Re-Conditioned Decoding (ReCoDe), denoises in pixel space under the original condition, and attaches to any conditioned generator without retraining it. Across three condition types, two datasets, and three controllers of a FLUX generator, ReCoDe overall outperforms condition-blind decoding of the fully denoised latent, improving pixel-space condition adherence on most metrics.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Conditional Generation, Pixel Diffusion Decoder, Conditioned Decoder, Conditioned Pixel Decoder

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_4n42OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

ReCoDe adds spatial conditioning to a frozen pixel-diffusion decoder for controllable image generation. A shared condition encoder feeds gated residuals into the patch stream and spatially varying affine modulation into the pixel stream. The condition paths are trained with flow matching using PiD-generated high-resolution targets, while the generator and decoder backbone remain frozen. Experiments use FLUX.1-dev with three controllers and segmentation, edge, and depth conditions.

Critical strengths

The paper addresses a useful interface: spatial constraints can be weakened when a controlled latent is rendered into pixels. Directly supplying the condition to the final synthesis stage is a plausible and modular remedy.

Critical weaknesses

The methodological contribution is limited. The approach largely combines an existing pixel-diffusion decoder with established gated residual conditioning and spatial affine modulation. Its main addition is the placement of spatial-condition adapters in the patch and pixel streams, while the technical advance beyond a straightforward conditioned PiD is not sufficiently established.

The visual quality and experimental setting are not convincing. The generated examples in Figure 3 remain visually unsatisfactory. Evaluation is limited to FLUX.1-dev with generation at 512×512 before high-resolution decoding, leaving the practical benefit over stronger generators and higher-resolution generation settings unclear.

Questions for authors
  1. What substantive technical contribution does ReCoDe offer beyond combining PiD with established conditioning mechanisms?

  2. Does ReCoDe provide convincing visual-quality improvements with stronger generators and higher base generation resolutions, given the unsatisfactory examples in Figure 3?

Reviewer_2W4dOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents ReCoDe, which addresses the loss of spatial fidelity in controllable image generation. It introduces a lightweight condition branch into a pretrained pixel-diffusion decoder, injecting spatial conditions at both patch and pixel levels. This enables the decoder to recover missing details and correct structural inconsistencies without retraining the original generator. Experiments across three condition types and three FLUX controllers show improved condition adherence and image quality.

Critical strengths
  • The paper identifies an overlooked bottleneck in controllable image generation and provides a new motivation for improving the decoding stage.
  • ReCoDe is lightweight and modular, requiring less than 1% additional decoder parameters without retraining the original generator.
  • Experiments across three spatial conditions and three controllers demonstrate consistent improvements in image quality and condition adherence.
  • The motivation of this paper that injecting the conditions into the decoder sound novel.
Critical weaknesses
  • The method description is not sufficiently clear or detailed. For example, in Figure 2, the roles of the patch stream and pixel stream, as well as which components are newly introduced versus inherited from the pretrained decoder, are not clearly explained. It is also unclear whether components such as the Conv Blocks are newly added and trained or are part of the original architecture. The textual explanations do not adequately clarify these implementation details, making the proposed architecture difficult to understand and reproduce.
  • The novelty of the proposed method appears limited, as injecting conditions into the decoder has been explored in prior work. Beyond combining existing components, the paper provides limited insights into why the proposed design is necessary or effective. For example, it is unclear whether combining patch tokens with per-pixel features provides substantial benefits over simpler alternatives, such as directly injecting the condition into the decoder. Similarly, the motivation for introducing separate patch and pixel streams in the generative decoder is not sufficiently justified. More detailed ablation studies and analyses are needed to demonstrate the necessity of these design choices and clarify the specific methodological contributions of ReCoDe.
  • The evaluation is limited to FLUX-based models and three spatial conditions, leaving generalization to other generators and conditioning types unclear.
  • Improvements are not consistent across all metrics, with some results showing limited gains over the vanilla pixel-diffusion decoder.
  • The qualitative comparisons in Figure 3 do not demonstrate a clear visual advantage of ReCoDe over the baseline methods. The generated images appear largely similar, making it difficult to visually verify the claimed improvements in structural fidelity and fine-detail preservation.
  • The paper lacks comprehensive ablation studies for the newly introduced components. In particular, it does not sufficiently isolate the contributions of the patch stream, pixel stream, and their respective condition injection mechanisms. Without these experiments, it is difficult to determine whether each component is necessary and how much it contributes to the overall performance improvement.
Questions for authors

please refer to the weakness. If you address my concerns, I will consider to raise my score.

Reviewer_gry1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a simple way of making a latent rectified flow model follow the condition more closely, by feeding the condition to the decoder in a latent rectified flow model. It does so by encoding the image condition (such as an edge map or a segmentation map) through a pixel stream and a patch stream and then modulating the residuals with multiplicative gating in the patch stream and activations with a SPADE connection in the patch stream. The paper argues that feeding in the condition to the decoder and making it generative are both important.

Critical strengths
  • The paper targets an important problem, namely that the generated images may not follow the condition closely.
  • The paper is well written.
  • The idea is simple.
Critical weaknesses
  • Since the paper replaces the input latent with the latent generated from the text prompt only (which would probably not agree with the condition at all), does the model in just learn to ignore the input latent and pay attention only to the condition? If so, does the model suffer from mode collapse, i.e., given the same condition, only one image can be generated, regardless of the noise? If not, why doesn't the model ignore the input latent?
  • The generated images in Fig. 3 seem to have weird high-frequency artifacts.
  • The resolutions of the results in Fig. 3 and 4 don't seem to be 2048x2048, even though the third contribution refers to 2048x2048 as the native output resolution.
  • In Table 2, the proposed method with the generative decoder does not seem to be much better at following the condition compared to the baseline with a deterministic decoder, which contradicts the second contribution, i.e., "We show that a decoder must be both generative and condition-aware: a deterministic decoder only repositions existing structure, and a condition-blind one invents detail that contradicts the condition."
  • For Table 2, the results for the proposed method with K = 16 and K = 28 are missing.
  • Some of the phrasing is unclear, e.g.,
    • On L93, it's unclear what "the last stage" means
    • On L114, it's unclear what "single fixed condition" means and how it's different from the proposed method
  • Minor: On L121, "restore" should be "recover"
Questions for authors

What happens if you were to increase or decrease the percentage of input latents that you replace with the text-only latent (on L232)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.