Degradation-Adaptive Inverse Operator Learning for All-in-One Image Restoration

All-in-one image restoration aims to recover images corrupted by heterogeneous degradations using a single unified model. Most existing approaches directly predict clean images or residuals, often relying on degradation-specific prompts or additional conditioning modules to distinguish different restoration tasks. In this work, we explore a different perspective and formulate all-in-one restoration as input-conditioned inverse operator learning. Instead of directly regressing restored pixels, we predict spatially adaptive inverse operators that transform degraded observations into clean estimates. Based on this formulation, we propose OPIR, an all-in-one image restoration framework that infers degradation representations directly from the input and uses them to modulate a shared restoration operator. This input-conditioned modulation allows the model to adapt to heterogeneous degradations without requiring degradation labels or manually specified task prompts at inference. To progressively improve difficult regions, we further introduce an operator-uncertainty-guided refinement strategy, where the distribution and statistical responses of the predicted local operators are used to identify pixels that require stronger restoration. A second restoration stage then selectively refines these challenging regions. In addition, an efficient multi-scale operator implementation reuses compact predicted kernels across different receptive fields, substantially reducing the computational overhead of spatially adaptive filtering. Extensive experiments demonstrate that OPIR consistently achieves superior performance across multiple all-in-one restoration benchmarks, while also remaining highly competitive on task-aligned restoration settings.