acceptodds
Withdrawn from ICLR 2027

Flow-Filter: Plug-and-Play Image Smoothing with Flow Matching

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

13 views

Abstract

Structure-aware image smoothing seeks to remove textures while preserving structures, but balancing texture suppression and structural fidelity remains challenging. Handcrafted priors offer explicit local geometric constraints yet risk over-smoothing, while pretrained generative priors capture rich natural-image statistics but may retain or reconstruct unwanted textures. To address this challenge, we propose Flow-Filter, a plug-and-play image smoothing framework that uses a pretrained Flow Matching model as a generative structural prior for structure–texture decomposition and requires no task-specific training or fine-tuning. We further introduce a joint gradient structural prior that couples first- and second-order differential responses, allowing global natural-image modeling from the generative prior to complement explicit local geometric constraints. To adapt FM to selective texture removal, we couple its denoising process with progressive ADMM-based decomposition and introduce per-step clean–noise endpoint modulation to reduce structural drift and persistent texture residuals. Extensive experiments demonstrate that Flow-Filter suppresses complex textures more effectively while preserving salient structures and fine geometric details, achieving a favorable balance between smoothing strength and structural fidelity. Our results show that pretrained Flow Matching models can go beyond their conventional role in generation and serve as effective plug-and-play priors for selective image filtering.

Primary area:
optimization
Keywords:
image smoothing, flow matching, plug-and-play, generative prior, structure-texture decomposition, ADMM, texture filtering

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_EbtsOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary
  • This paper proposes Flow-Filter, a plug-and-play framework for Structure-Aware Image Smoothing. It uses a pretrained pixel-level Flow Matching model as a global generative structure prior, combined with first-/second-order mixed-difference MCP gradient geometry and RGB group-sparse texture priors, enabling structure-texture decomposition without task-specific fine-tuning.
  • Key components include Flow-ADMM Level Matching, which links the Flow perturbation scale to the ADMM penalty; Clean-Endpoint Anchoring and Noise-Endpoint Refresh to reduce structure drift and prevent residual texture accumulation; and 2D Cyclic Patch-Phase Shifting to alleviate patch-boundary artifacts.
  • Experiments on texture filtering, JPEG artifact removal, NKS synthetic benchmarks, detail enhancement, and HDR tone mapping show a better balance between structure preservation and texture removal than conventional optimization-based and supervised/fine-tuned methods.
Critical strengths
  1. Well-motivated generative-prior integration: Rather than using the generative model as an end-to-end smoother, which may introduce texture hallucination or structural distortion, the paper uses it as a source of global structural guidance and combines it with explicit differential priors within a unified ADMM framework.
  2. Effective solution to the deployment challenge of patch-based generative priors: The proposed 2D cyclic Patch-Phase Shifting (PPS) mechanism addresses phase-locked boundary artifacts that can accumulate during multi-step iterations with patch-based generative models such as JiT. Importantly, it requires neither additional retraining nor network evaluations, and the ablation study clearly demonstrates its effectiveness in removing stitching artifacts.
Critical weaknesses
  1. Limited quantitative evaluation on real-world natural images: Most quantitative results are based on 50 clip-art images for JPEG artifact removal and 200 synthetic textures from the NKS dataset. Evaluation on complex natural images and real-world multi-scale textures relies mainly on qualitative comparisons, with limited evidence from large-scale user studies or downstream tasks such as edge detection and salient object segmentation.

  2. High computational cost limits practical and interactive applications: As reported in Table 4, processing a relatively small image of approximately 400×350400\times350 resolution takes about 3.5 seconds, making the method orders of magnitude slower than real-time or sub-second smoothing approaches. The paper does not explore low-step sampling (e.g., 5–10 steps) or distillation-based acceleration.

Questions for authors
  1. Hyperparameter robustness and adaptivity: If a single fixed set of hyperparameters is used across all tasks and test images, how much does performance degrade on the NKS and JPEG artifact removal benchmarks? Is there an input-adaptive strategy for automatically estimating these parameters, rather than relying on task-specific manual tuning?

  2. Potential for few-step acceleration: The current inference requires up to 50 model evaluations. Does the proposed decomposition remain effective when the sampling steps are drastically reduced to 3–5 steps?

  3. End-to-end benefits for downstream tasks: Can the authors provide quantitative results on downstream tasks using the smoothed images, demonstrating practical advantages over conventional real-time filters?

Reviewer_XMA9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The authors propose an image smoothing method that combines a pre-trained flow model’s generative prior together with a classic gradient-based smoothing approach. The authors claim that applying classic approaches can risk over-smoothing (e.g. omitting small letters in images), while using generative priors can help preserve the naturalness of the smoothed image. The proposed algorithm optimizes a structure map S and texture map T decomposition of an image f such that f = S + T. A constrained optimization is done iteratively via ADMM, and attempts to balance gradient-based smoothing iterations, and maximizing the adherence of the structure component S to the generative prior. The authors further propose 'Flow-ADMM' denoising-level alignment to align the flow denoising schedule with the optimization iterations. They also propose a 'noise-endpoint refresh' scheme which aids the flow model to carry over information from previous optimization iterations by utilizing the previous step’s flow-model's noise predictions. The authors showcase qualitative and quantitative evaluations on the tasks of Texture filtering, and JPEG compression artifact removal, and provide various ablation studies.

Critical strengths
  • The authors developed a novel method for image smoothing by creatively adapting a pre-trained flow matching model to regularize the optimization process.
  • The paper showcases impressive qualitative results, as well as competitive quantitative performance (JPEG artifact removal and texture filtering).
Critical weaknesses
  • Lack of clear method overview and presentation: The paper lacks a clear high-level overview of the method. It is very hard to follow and understand the big picture. Several technical components are introduced, as well as a complex optimization scheme, without explaining how everything should interact together. I suggest adding an overview method diagram figure of the method and referencing it throughout the method section. Figure 2 shows different intermediate steps without explaining what drives them. Figure 3 and 4 focus on specific mechanisms and do not do much to explain the overall framework. All in all, the heavy focus on implementation details makes it hard to see the core contributions of the proposed method.
  • Missing ablations. A) First order and second order MCP optimization is proposed but not ablated against first order only optimization. B) Sensitivity to hyperparameters is not evaluated (ρ0,μ,ξ,γ,α0,η,K\rho_0, \mu, \xi, \gamma, \alpha_0, \eta, K).
Questions for authors
  • Texture regularization: The regularizer in eq. 6 assumes that the separated texture map should be sparse. But in many cases textures can be dense with and take up large spatial space (e.g. grass leaves or fur). How does this regularization fare when applied to such examples?
  • How does runtime of Flow-Filter compare with the competing methods?
  • How would the hyperparameter configuration be selected for in-the-wild images (without calibrating them per image as in fig 5 and 6)?
  • minor remark - Background has one subsection which is redundant.
Reviewer_qbLNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Flow-Filter, a plug-and-play image smoothing framework combining a pretrained JiT generative prior, gradient regularization, and texture decomposition, with additional endpoint and patch-shifting strategies. It is evaluated on synthetic texture-filtering data, JPEG artifact removal, and several image-processing applications. While the results show potential, the effectiveness of the proposed component interactions and their generalization across diverse textures remain insufficiently validated.

Critical strengths
  1. The visual results on the presented examples are promising, showing good texture suppression while preserving important structural details.
  2. The method uses a pretrained JiT model without task-specific training, providing a practical plug-and-play framework.
  3. The paper evaluates the method on multiple image-processing tasks and includes quantitative comparisons on synthetic texture filtering and JPEG artifact removal.
Critical weaknesses
  1. Modeling assumption: The regularizer promotes sparse residuals but does not explicitly model texture properties, making its suitability for dense textures unclear.
  2. Conflicting objectives: CEA, NER, and gradient regularization may impose competing constraints, yet their interactions and hyperparameter dependencies are insufficiently investigated.
  3. Insufficient ablation: Ablations are mainly qualitative, with limited quantitative evidence of individual component contributions and their complementarity.
  4. Limited real-world validation: The main paper presents few qualitative examples, with unclear data sources and selection criteria.
  5. Presentation clarity: The relationships between the optimization formulation, intermediate variables, and implemented operations are not always clearly explained.
Questions for authors

What is the rationale for modeling potentially dense and repetitive textures using a pixel-wise sparsity penalty? How does the proposed framework balance CEA's tendency to preserve the input, including unwanted textures, against the regularizer's suppression of texture residuals?

Quantitative joint ablations and hyperparameter sensitivity analyses would help establish whether CEA, NER, and the texture regularizer work complementarily or require delicate balancing to achieve reliable structure–texture separation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.