- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes FIRM, a measurement-conditioned flow-matching method for imaging inverse problems. Through a variational characterization of the posterior mean, the authors motivate incorporating explicit data-consistency updates into the conditional velocity field. The resulting architecture uses an unrolled half-quadratic splitting module and is trained end-to-end. Experiments on five image restoration tasks across two datasets demonstrate competitive reconstruction quality and reduced inference cost.
Critical strengths
1.Clear motivation and a coherent connection between theoretical analysis and architectural design.
The paper addresses how measurement information should enter the generative process by connecting posterior-mean estimation, variational optimization, and conditional velocity parameterization. Explicitly applying the measurement operator within each velocity-field evaluation gives the architecture a clear model-based structure. The theoretical characterization provides a reasonable foundation for integrating optimization unrolling with conditional flow matching.
2.Broad experimental evaluation with competitive reconstruction quality and inference efficiency.
The evaluation covers multiple restoration tasks and includes larger test sets, architectural ablations, and mismatch experiments. The capacity-matched CelebA comparisons provide convincing evidence of effectiveness. In the reported CelebA deblurring configuration, FIRM requires 10 network evaluations and 0.24 seconds, compared with 500 evaluations and 13.03 seconds for Flower5-OT, demonstrating a substantial inference benefit.
Critical weaknesses
1.Differences in comparison conditions leave some uncertainty about the source of the improvements.
In the conditional-flow ablation, matching the sampling steps gives FIRM more internal network evaluations, while matching evaluation counts changes the sampling steps and the corresponding distortion–perception operating point. Some AFHQ-Cat models also have greater capacity than the flow baselines. The existing ablations provide positive evidence, but a fuller discussion of these differences would help clarify the contribution and scope of operator embedding.
2.The relationship between sampling steps and the distortion–perception trade-off could be explained more thoroughly.
The paper demonstrates how changing the sampling steps affects reconstruction metrics, but some settings exhibit non-monotonic behavior, including simultaneous deterioration in PSNR and LPIPS. Discussing these observations in relation to conditional-mean estimation, posterior sampling, and numerical approximation would strengthen the experimental analysis.
Questions for authors
1.Could the authors clarify which conclusions are supported by comparisons at matched sampling steps versus matched network evaluation counts? Based on the existing ablations, how should we interpret the respective contributions of explicit operator embedding and additional internal computation?
2.Could the authors further explain how changing the sampling steps affects the reconstruction distribution, and discuss possible reasons for the simultaneous deterioration in PSNR and LPIPS in some settings?
4 comments
2.The relationship between sampling steps and the distortion–perception trade-off could be explained more thoroughly. The paper demonstrates how changing the sampling steps affects reconstruction metrics, but some settings exhibit non-monotonic behavior, including simultaneous deterioration in PSNR and LPIPS. Discussing these observations in relation to conditional-mean estimation, posterior sampling, and numerical approximation would strengthen the experimental analysis.
Numerical explanation of the non-monotonic behavior. The deterioration previously observed at results from the velocity scaling used during sampling. The velocity is singular at , so training uses the denominator (Eq. 21). During sampling, either or can be used; the two coincide for . For , using reduces the magnitude of the final updates, leaving residual source noise and degrading both PSNR and LPIPS. Using eliminates this deterioration and yields a consistent distortion–perception tradeoff (PSNR / LPIPS):
| CelebA, deblurring | CelebA, SR | AFHQ-Cat, deblurring | AFHQ-Cat, SR | |
|---|---|---|---|---|
| 2 | 36.04 / 0.029 | 34.39 / 0.028 | 29.89 / 0.229 | 29.10 / 0.167 |
| 4 | 35.78 / 0.028 | 33.96 / 0.026 | 29.42 / 0.188 | 28.50 / 0.141 |
| 10 | 35.03 / 0.023 | 33.13 / 0.022 | 28.72 / 0.145 | 27.75 / 0.115 |
| 25 | 34.19 / 0.018 | 32.50 / 0.020 | 28.16 / 0.119 | 27.25 / 0.104 |
| 50 | 33.80 / 0.016 | 32.11 / 0.019 | 27.86 / 0.114 | 27.02 / 0.101 |
| 100 | 33.57 / 0.016 | 31.91 / 0.019 | 27.70 / 0.111 | 26.90 / 0.102 |
As increases, PSNR decreases while LPIPS generally improves, with both metrics approaching a plateau at larger . This is consistent with a transition from posterior-mean-like reconstructions toward perceptually realistic samples. We will use for all results with and clarify this numerical choice in App. F.1.
Posterior fidelity versus distortion–perception tradeoff. Since the exact posterior is unknown for natural images, PSNR and LPIPS alone cannot establish posterior fidelity. We therefore evaluated FIRM on a Gaussian random field prior over images, for which the posterior is available in closed form (random inpainting; App. F.4).
| Distance to posterior mean | to exact posterior ↓ | |
|---|---|---|
| 2 | 0.055 | 0.829 |
| 10 | 0.506 | 0.360 |
| 25 | 0.791 | 0.162 |
As increases, the samples move away from the posterior mean while approaching the exact posterior, with the distance decreasing by approximately fivefold from to . This provides direct evidence of improved posterior fidelity and complements the distortion–perception trends observed on natural images.
Could the authors clarify which conclusions are supported by comparisons at matched sampling steps versus matched network evaluation counts? Based on the existing ablations, how should we interpret the respective contributions of explicit operator embedding and additional internal computation?
Please see our response to W1. In short:
- Matched : FIRM is the better estimator along the same trajectory, with higher PSNR and SSIM in every setting and lower LPIPS in 7 of 10.
- Matched NFEs: at equal cost, FIRM has higher PSNR and SSIM on every task; the conditional flow's lower LPIPS reflects its more perceptual operating point ().
- No-DC control (same network, NFEs, and ): removing only the data-consistency step removes the gain, so the improvement comes from the explicit operator embedding, not from additional internal computation.
Could the authors further explain how changing the sampling steps affects the reconstruction distribution, and discuss possible reasons for the simultaneous deterioration in PSNR and LPIPS in some settings?
Please see our response to W2. We quantify how increasing moves the reconstruction distribution away from the posterior mean and toward the exact posterior, using the distance to the posterior mean and the 2-Wasserstein distance to the exact posterior. We also explain how the velocity scaling used during sampling caused the simultaneous deterioration in PSNR and LPIPS, and show that correcting it eliminates this behavior.
We thank the authors for their detailed and thoughtful response. The new ablation clarifies the contribution of the paper. These additions address our main concerns.
Follow up on
Thank you for considering our rebuttal. We’re glad the new ablation clarified our contribution and that the additions addressed your main concerns. We’d be happy to provide any further details that could support a stronger recommendation.
We thank the reviewer for the feedback. In response, we have added a capacity-matched baseline without the data-consistency step (W1, Q1), an analysis of sampling steps from N=1 to N=100, and a comparison with the exact posterior using a Gaussian random field (W2, Q2). See our point-by-point response below.
We disentangle these factors through three controlled comparisons, each holding a different condition fixed (CelebA, PSNR / SSIM / LPIPS, better values in bold).
Capacity. On CelebA, all comparisons above are capacity-matched (34.5M, the size of the flow baselines' prior). On AFHQ-Cat, FIRM is larger than the flow baselines' prior (31.0M) but smaller than I2SB (552.8M): 59.9M, and 239.4M for deblurring, the hardest setting at 256×256, where smaller estimators underfit (App. D.1).