From RGB Generation to Dense Field Readout: Pixel-Space Dense Prediction with Text-to-Image Models

Large-scale text-to-image models are attractive backbones for dense prediction because RGB generation pretraining learns rich semantic, structural, and geometric priors. Representative generative and editing approaches reuse these priors by casting dense prediction as target generation: annotations such as depth, normals, alpha mattes, masks, and heatmaps are encoded into an RGB-trained VAE latent space and decoded back as image-like targets. This target-side interface is inherited from RGB synthesis rather than derived from the task. Unlike RGB synthesis, dense prediction asks for pixel-correct, task-native fields on the same image plane, not new RGB content to be rendered. Our starting point is that a pretrained DiT already organizes RGB inputs through a patch$\to$token$\to$patch lattice on the image plane. Each token therefore indexes a fixed output patch whose channels can carry task-native quantities instead of RGB appearance. We instantiate this idea as ReChannel. We retain the VAE encoder to preserve the DiT's input distribution, remove the target-side decoder, adapt the frozen DiT with per-task LoRA, and map each token to its $p\times p\times K_t$ pixel-space patch using a shared token-local linear head. The head has about 33K parameters, performs no spatial mixing, and is trained directly with pixel-space, task-native supervision. Using FLUX-Klein, we evaluate ReChannel on six dense prediction tasks and more than a dozen benchmarks. This minimal interface achieves state-of-the-art results on trimap-free matting, KITTI depth, and referring segmentation, while remaining competitive on normals, saliency, and pose. Under a matched 4B backbone, it is more accurate and up to 2.48$\times$ faster than latent, VAE-decoded, and edit output interfaces, while larger heads provide no further gains. These results show that dense prediction can inherit the prior of generative pretraining without inheriting its output interface.