RestoreSteer: Adaptive Flow Control for Faithful Image Restoration

Flow-based image editors provide strong generative priors for image restoration, but perceptually plausible outputs can deviate from the scene supported by a degraded observation, limiting restoration fidelity. We regard a restoration as faithful when it preserves scene information supported by the observation without reproducing the degradation itself. The available evidence and safely editable quantities change along the sampling trajectory: early states permit changes to scene formation but expose only coarse evidence; intermediate states provide more reliable boundaries while geometry remains editable; and late states support precise residual measurements but tolerate only small corrections. We introduce RestoreSteer, a training-free controller that matches the correction target and magnitude to these sampling phases. RouteCal redirects early scene formation using observation-supported structure and coarse appearance; StructAlign corrects persistent geometric disagreement after reliable boundaries emerge; and ObsRefine optimizes a bounded late latent residual through the frozen decoder and forward degradation model. Each stage evaluates bounded candidates before modifying the sampling state. RestoreSteer requires neither backbone retraining nor a fixed pseudo-clean target obtained by inverting the observation. Across six restoration tasks on Qwen-Image-Edit and FLUX.2, RestoreSteer improves PSNR, SSIM, and LPIPS over native sampling in every evaluated task-backbone setting. Component ablations further show complementary contributions from the three sampling phases.