Operator-Adaptive Mean-Target Guidance for Linear Inverse Problems

Training-free samplers for inverse problems construct guidance from a residual evaluated at an evaluation target, a point along the generative trajectory. Since the measurement is defined from the clean image through a known operator, most existing methods form the residual at a clean-image estimate. Some methods instead map the measurement to the current sampling step and form the residual at the current state. Although existing methods construct guidance in different ways, they typically treat the evaluation target as given rather than derive it. We introduce Mean-Target Guidance (MTG), which derives operator-adaptive evaluation targets. A single frozen model evaluation produces a continuum of virtual targets. The gradient of a loss that aggregates their weighted squared residuals reduces exactly to one residual at their weighted mean, the mean-target. Inverse-variance weighting over the virtual targets yields a mean-target for each singular direction. While the mean-target determines where the residual is evaluated, a direction-specific damping factor controls how strongly that residual updates the current state. The construction applies to both diffusion and flow-matching backbones. MTG outperforms strong training-free baselines across standard linear inverse problems. An ablation study separates the effects of the mean-target and damping factor. We train the evaluation targets and update strengths, highlighting the importance of where the residual is evaluated and how strongly it acts on the state. Code is available at https://anonymous.4open.science/r/MTG2027.