Towards Autonomous Image Restoration with Fine-grained Agentic Control and History-grounded Restoration

Most generative image restoration methods perform restoration in a single round, without explicitly reassessing residual degradations, newly introduced artifacts, or reconstruction errors in the restored results. Recent agentic approaches introduce iterative assessment and replanning, but their control largely remains at the level of tool selection, with visual history primarily used for decision-making rather than restoration execution. To build an autonomous restoration framework, we propose AutoRestore, which tightly integrates VLM-based assessment with generative restoration through fine-grained agentic control and history-grounded restoration. At each round, AutoRestore uses a VLM to identify restoration requirements and generate region-specific instructions. These instructions are further translated into spatially varying uncertainty and uncertainty-driven modulation, enabling fine-grained restoration control. Meanwhile, we introduce history-grounded restoration, which preserves reference representations from the original input and intermediate results as persistent visual memory across rounds. Extensive experiments on multiple image restoration benchmarks demonstrate that AutoRestore achieves superior restoration quality and a better trade-off between fidelity and perceptual quality.