- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes PCAR-Flow, a physics-calibrated generative method for metalens image reconstruction. Metalens measurements suffer from highly non-uniform recoverability across spatial positions and frequencies. To address this, the authors introduce "effective local observability," a metric combining optical transfer, sensor noise, and local image statistics to quantify measurement support. The pipeline first uses a deterministic base reconstructor to recover well-observed structures. Then, it applies conditional flow matching exclusively to unresolved high-frequency wavelet residuals. Finally, a reliability-gated admission module controls how much generated detail is injected into the final image. Theoretical analysis links observability to posterior uncertainty under a local Gaussian model. Experiments demonstrate that PCAR-Flow outperforms strong baselines (e.g., DMDiff) on the M1 dataset and successfully generalizes to a different metalens system (M2) without retraining.
Critical strengths
-
Strong physical grounding and elegant framework: The concept of "effective local observability" provides a principled way to bridge physical optics and generative models. By restricting the flow matching to high-frequency residuals and gating them based on physical cues, the method avoids hallucinating details where the measurement already provides strong support. This effectively balances perceptual quality and physical consistency.
-
Impressive cross-metalens generalization: The experiment transferring the M1-trained model directly to the M2 metalens system (using only the new calibration package) is a critical strength. It proves that the learned prior allocation mechanism is genuinely physics-aware and not simply overfitting to a specific lens's degradation pattern.
Critical weaknesses
-
Reliance on ground-truth oracle for gate training: The admission gate is supervised using an oracle admission map (Eq. 21), which requires computing the least-squares scale between the proposal and the true target residual. While upstream modules are frozen, relying on a ground-truth-dependent oracle during Stage C training raises concerns about whether the gate will generalize optimally at inference time when no ground truth exists. The gap between oracle supervision and real-world test-time behavior needs more justification.
-
Fragility of the local Gaussian assumption: The theoretical interpretation of observability relies heavily on a local Gaussian model for both the latent state and noise. However, natural images contain sharp edges and highly non-Gaussian local structures. The paper acknowledges this briefly in the limitations, but given that observability is the core signal driving the entire pipeline, its accuracy in non-Gaussian regions could silently degrade the gating performance.
Questions for authors
-
Regarding the gate training (Stage C): Have you tried training the admission gate purely via reinforcement learning or differentiable physics losses (e.g., optimizing and LPIPS directly) without the oracle map ? If so, how does the performance compare?
-
How sensitive is the overall pipeline to small, realistic errors in the calibration package (e.g., slight misalignment in the PSF grid)? Does the gate naturally suppress bad generations in these cases?
-
The method excludes the LL band from the flow matching. Could applying a very lightweight flow to the LL band further improve low-frequency color/contrast recovery without significantly harming ?
5 comments
At ±5% displacement, gated admission reduces LPIPS from 0.1511 to 0.1337 and from 0.0618 to 0.0436 compared with always-on admission. At ±10%, the corresponding values decrease from 0.1616 to 0.1402 and from 0.0724 to 0.0501. Meanwhile, mean gate activation decreases from 0.362 to 0.330 as the displacement increases, indicating a more conservative admission response under larger calibration misalignment.
Although reconstruction quality still degrades with increasing misalignment, gated admission consistently yields lower perceptual error and forward inconsistency than unrestricted residual injection at both perturbation levels. These results demonstrate that the reliability gate mitigates the adverse effects of PSF-grid misregistration by regulating residual admission, rather than correcting the underlying calibration error.
Rebuttal Table W2. Sensitivity to Spatial PSF-Grid Misalignment.
| PSF-grid displacement | Admission | LPIPS ↓ | ↓ | Mean gate |
|---|---|---|---|---|
| ±5% | Gated | 0.1337 | 0.0436 | 0.362 |
| Always-on | 0.1511 | 0.0618 | 1.000 | |
| ±10% | Gated | 0.1402 | 0.0501 | 0.330 |
| Always-on | 0.1616 | 0.0724 | 1.000 | |
Q3. LL-Band Selection and Reliability-Guided Low-Frequency Generation
We retain the LL band in the deterministic pathway to preserve low-frequency structures supported by the calibrated measurements. As shown in Supplementary Table 13, unrestricted LL generation slightly improves LPIPS (0.1289 to 0.1261), but reduces PSNR from 30.62 to 30.14 dB and increases from 0.0398 to 0.0619, indicating a trade-off between perceptual improvement and measurement consistency.
To examine whether this trade-off can be mitigated, we further evaluated a lightweight LL residual branch with three admission strategies: always-on, fixed scaling, and reliability-gated admission. The deterministic base and high-frequency flow were kept frozen, with identical LL residual proposals across the three configurations; only the LL admission strategy was varied. As shown in Rebuttal Table W3, reliability-gated admission provides the best balance among the tested LL configurations. Compared with always-on admission, it improves PSNR from 30.38 to 30.59 dB and reduces from 0.0529 to 0.0416 (21.4%), LL-MAE from 0.0249 to 0.0231 (7.2%), and from 3.64 to 3.36 (7.7%), with only a marginal LPIPS increase (0.1268 to 0.1271).
However, compared with the original HF-only PCAR-Flow, the reliability-gated LL variant provides only a modest LPIPS improvement (0.1289 to 0.1271), while slightly reducing PSNR (30.62 to 30.59 dB) and increasing (0.0398 to 0.0416). Its LL-MAE and also remain higher (0.0231 vs. 0.0197 and 3.36 vs. 3.22), with inference time increasing from 0.98 to 1.11 s per image. Thus, although reliability gating substantially mitigates the adverse effects of unrestricted LL generation, the additional branch does not improve all reconstruction criteria. We therefore retain the LL band in the deterministic pathway in the default PCAR-Flow, while demonstrating the potential of reliability-guided admission as a more controlled alternative to unrestricted low-frequency generation.
Rebuttal Table W3. Effect of Lightweight LL Residual Modeling and Admission Strategies.
| Configuration | LL Admission | PSNR ↑ | LPIPS ↓ | ↓ | LL-MAE ↓ | ↓ | Time (s) ↓ |
|---|---|---|---|---|---|---|---|
| Original HF-only PCAR-Flow | -- | 30.62 | 0.1289 | 0.0398 | 0.0197 | 3.22 | 0.98 |
| Lightweight LL flow | Always-on | 30.38 | 0.1268 | 0.0529 | 0.0249 | 3.64 | 1.09 |
| Lightweight LL flow | Fixed scaling | 30.51 | 0.1276 | 0.0462 | 0.0238 | 3.49 | 1.09 |
| Lightweight LL flow | Reliability-gated | 30.59 | 0.1271 | 0.0416 | 0.0231 | 3.36 | 1.11 |
Q4. Reliability of Effective Observability under Non-Gaussian Structures and Spatially Varying PSFs
The posterior interpretation of effective observability in Appendix A.3 is derived under a local plug-in Gaussian model for the latent coefficients and measurement noise, yielding . This is a local statistical approximation, not a claim that natural images are globally Gaussian or that the identity holds exactly for arbitrary non-Gaussian structures. To examine its empirical reliability beyond relatively smooth regions, we further stratified the evaluation by local image gradients and spatial PSF variation (Rebuttal Table W4).
The calibration accuracy degrades in more challenging regions. Specifically, the calibration MAE increases from 0.0408 in low-gradient regions to 0.0718 in high-gradient regions, accompanied by a decrease in Spearman correlation from 0.937 to 0.846. In regions with rapid PSF variation, the MAE further reaches 0.0826 and the correlation decreases to 0.785. The corresponding degradation rates are 10.9%, 19.4%, and 22.7%, respectively. These results expose the limitations of the local approximation under complex image and optical conditions, rather than implying uniformly reliable observability estimates across all regions.
Importantly, the admission gate does not rely on alone. It jointly incorporates baseline uncertainty , confidence , and forward disagreement (Eq. 16), providing complementary reliability information for residual allocation. Table 5 supports their respective contributions: removing , , or increases from 0.0398 to 0.0460, 0.0452, and 0.0493, respectively. Thus, observability serves as one component of a multi-cue admission mechanism, rather than a standalone estimate that determines residual injection. We will clarify the scope of the local Gaussian interpretation and include the region-wise analysis to make these limitations explicit.
Rebuttal Table W4. Region-Wise Observability Calibration and Reconstruction Fidelity.
| Region | Calibration MAE ↓ | Spearman ↑ | Degradation Rate ↓ |
|---|---|---|---|
| Low-gradient (bottom 80%) | 0.0408 | 0.937 | 10.9% |
| High-gradient (top 20%) | 0.0718 | 0.846 | 19.4% |
| Rapid PSF variation (top 10%) | 0.0826 | 0.785 | 22.7% |
Clarifications Regarding Gated Supervision, Calibration Robustness, Band Selection, and Observability Assumptions
In response to the reviewer's concerns regarding oracle supervision, calibration errors, and local statistical assumptions, we provide further explanations of the relevant mechanisms and analyze their applicability and limitations through controlled experiments.
Q1. Contribution of Oracle Admission Supervision and Alternative Gate-Training Objectives
Eq. (21) defines the oracle admission target as the clipped local least-squares coefficient between the generated residual and the reference residual. This target is used exclusively during Stage C training, where the base reconstructor and residual flow are frozen and the generated proposals are detached. The gate is therefore trained to estimate how much of each proposed residual should be admitted, with Eq. (22) further incorporating physical-consistency and monotonicity constraints. At inference, the oracle target and reference image are not required; the gate predicts admission coefficients solely from the observable cues Ωs, Ub, Cb, and rb.
To assess whether the gate can be trained without oracle admission targets, we compare two alternative objectives using the same frozen upstream networks and gate architecture (Rebuttal Table W1). The first directly combines perceptual and physical-consistency losses, while the second uses only physical-consistency and monotonicity losses. The perceptual-physical objective achieves 30.39 dB PSNR and 0.1357 LPIPS, approaching the original oracle-supervised result (30.62 dB and 0.1289, respectively), but with a higher Ephys of 0.0442 versus 0.0398. In contrast, physical-consistency and monotonicity losses alone reduce Ephys to 0.0380, but also lower the mean gate activation to 0.117, with PSNR and LPIPS deteriorating to 29.84 dB and 0.1530. This indicates that optimizing physical consistency without a direct perceptual objective tends to produce overly conservative residual admission, limiting the recovery of useful details.
The comparison shows that differentiable objectives can train the gate without an oracle admission map, but do not fully reproduce the balance achieved by oracle-supervised training. The local least-squares target provides explicit spatial supervision for residual allocation, while the physical-consistency and monotonicity terms constrain its effect on the measurements. Importantly, this supervision is confined to training and introduces no additional reference-image requirement at inference.
Rebuttal Table W1. Gate Training without Oracle Admission Targets.
Q2. Robustness to Calibration Mismatch and PSF-Grid Misregistration
Table 6 and Figure 6 evaluate complementary forms of calibration mismatch. Table 6 considers joint moderate calibration drift, physical remounting, and thermal drift, while Figure 6 reports a controlled PSF-perturbation sweep. Under joint moderate drift, the mean gate activation decreases from 0.382 to 0.255, whereas LPIPS and Ephys increase from 0.1289 to 0.1544 and from 0.0398 to 0.0602, respectively. Physical remounting and thermal drift exhibit similar reductions in gate activation. These results suggest that the gate becomes more conservative under degraded calibration, although the remaining reconstruction errors indicate that admission control cannot fully compensate for an inaccurate forward model.
To specifically examine spatial misregistration of the PSF calibration grid, we introduce controlled coordinate offsets of ±5% and ±10% of the nominal grid spacing (Rebuttal Table W2). The measured inputs and learned network parameters are kept unchanged, while the spatial coordinates associated with the PSF calibration samples are perturbed. For each offset, the calibration-dependent forward operator and reconstruction cues are recomputed, producing the corresponding base reconstruction and residual proposals. We then compare gated and always-on admission using the same base reconstruction and residual proposals within each perturbation condition, with only the admission map changed (gs=1 for always-on admission). This paired comparison isolates the effect of residual admission under each calibration error.
For physical-consistency evaluation, we use the nominal, unperturbed calibration operator rather than the perturbed operator supplied to the reconstruction pipeline. Thus, Ephys measures the discrepancy relative to the nominal imaging system, without using the evaluation operator as an inference-time input.