acceptodds
Under review as a conference paper at ICLR 2027

Frequency Matters: Frequency-Aware Decoupling and Harmonized Enhancement for High-Fidelity Medical Image Super-Resolution

Abstract

Single-image super-resolution (SISR) in medical imaging is hindered by the low-frequency bias of deep learning and the distinct semantic roles of mid- and high-frequency components, which uniform enhancement strategies fail to address. We propose the Frequency-Aware Representation Enhancer (FARE), a plug-and-play module that explicitly decouples and enhances these frequency bands. FARE comprises a High-Frequency Semantic Amplifier (HFSA), which combines directional wavelet decomposition with gradient-informed gating, and an Adaptive Mid-Frequency Decoupler (AMFD), which applies patch-level STFT with a learnable Gaussian band-pass mask. Their outputs are fused via cross-frequency negotiated gating. Extensive evaluations on multiple medical image benchmarks demonstrate consistent improvements over state-of-the-art methods.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Super-resolution

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_bmgvOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper observes medical image super-resolution and proposes the Frequency-Aware Representation Enhancer (FARE), a plug-and-play module designed to improve mid- and high-frequency representations. The method integrates frequency-specific enhancement with adaptive fusion to improve reconstruction quality. Experiments across multiple medical imaging datasets, SR backbones, and task settings demonstrate generally positive improvements, supported by additional ablation.

Critical strengths
  • The proposed FARE framework provides a reasonable frequency-aware design that explicitly enhances mid- and high-frequency representations for medical image super-resolution.

  • The experiments across multiple datasets and backbones demonstrate generally positive improvements. The extensive ablation studies and additional evaluations further support the effectiveness and applicability of the proposed method.

Critical weaknesses
  • The methodological novelty is somewhat limited. The proposed framework mainly combines existing frequency decomposition, structural guidance, and cross-frequency gating techniques, while its technical differences from prior work are not sufficiently clarified.

  • The performance improvements on several strong backbones are relatively marginal. Although the Appendix provides statistical significance analysis in one setting, the reliability of these improvements across independent training runs remains insufficiently evaluated.

  • The computational overhead is relatively substantial. As shown in Appendix Table 6, FARE significantly increases the parameters, FLOPs, and inference latency, while the performance gains on several strong backbones are modest. Further experimental verification is required.

  • The validation of the core frequency-selection and fusion mechanisms remains insufficient. The Gaussian mask in AMFD is formulated using spectral magnitudes rather than explicit frequency coordinates, and the ablation studies do not isolate the contribution of FusionGate from the combined HFSA and AMFD modules.

Questions for authors

Could the authors clarify how the Gaussian mask in Equations (4)–(5) selectively isolates mid-frequency components and provide visualizations of the learned frequency masks?

Could the authors provide an ablation combining HFSA and AMFD without FusionGate, combine with a computation-matched comparison, to better isolate the contribution of the fusion mechanism?

5 comments
AuthorsOpenReview ↗

Response to Reviewer bmgv (1/2)

We thank the reviewer for the positive assessment of our design and experiments, and for the constructive suggestions. Table and figure numbers refer to the revised PDF.

W1. "The methodological novelty is somewhat limited ... technical differences from prior work are not sufficiently clarified."

FARE is built on a principle that prior frequency-aware SR does not address. In medical images, mid frequencies carry contextual tissue textures, whereas high frequencies carry sharp boundaries (Appendix G.1). Their balance also varies strongly across images (Fig. 1). Prior methods contrast low with high frequencies or treat all high frequencies as one pool. FARE instead separates the two bands, adapts the mid band per patch, and lets the bands gate each other. Each difference is tested (PSNR, in-house/Kvasir):

  • vs. wavelet-based SR (Huang et al., 2017; Jeevan et al., 2024). HFSA handles only the HF band, using directional DTCWT magnitudes gated by a gradient prior. DTCWT beats Haar wavelets by 0.12/0.16 dB in the same structure (Table 7).
  • vs. Fourier losses (Fuoli et al., 2021). These losses supervise only the output spectrum, whereas FARE models the bands inside the network. FARE outperforms the Fourier-space loss on both SwinIR and ATD (Table 4).
  • vs. fixed frequency filters. AMFD selects the mid band per patch in the magnitude domain (Eq. 5). A coordinate-based mask with the same per-patch parameters is 0.05/0.04 dB worse (Table 4).
  • vs. simple fusion. FusionGate couples the bands by mutual sigmoid gating and beats plain addition by 0.07/0.05 dB (Table 4).

The two branches are also non-redundant: the NMI between their responses is only 0.0747 ± 0.0016 (Appendix G.2).

W2. "The performance improvements on several strong backbones are relatively marginal. ... the reliability of these improvements across independent training runs remains insufficiently evaluated."

  • Reliability across runs. The submission reports 52 baseline-vs-FARE comparisons over 10 backbones, 5 datasets and 3 degradation settings (Tables 1–3, 7). Each is a separately trained pair of models. FARE improves SSIM and LPIPS in all 52 and PSNR in 51 (one tie), and it never lowers any metric. A two-sided sign test gives p < 10⁻¹⁵ for each metric. Appendix J now reports this test, including the two MambaIR comparisons.
  • Why PSNR gains look small. PSNR averages over all pixels, while the boundaries and textures FARE restores occupy few of them. On the in-house blind set, PSNR gains are 0.03–0.40 dB (Table 1). On the same runs, however, residual spectral energy falls by 27–39% and spectral KL divergence by 20–26% for all six backbones (Fig. 4). The visual results agree (Figs. 3, 5, 6).
  • Larger gains where there is headroom. In MRI k-space SR, the Transformer backbones gain 0.29–0.75 dB and DiffMSR gains 0.34/0.38 dB (Table 3). ESC gains 0.65 dB in-house (Table 7).

W3. "The computational overhead is relatively substantial."

We report the overhead openly (Table 6: 65 → 96 ms per image for ATD). It is justified on three counts.

  • Linear scaling. FARE adds fixed-window FFTs, DTCWT and convolutions, all linear in the number of pixels. The backbone's asymptotic complexity is therefore unchanged.
  • Use case. FARE targets image reading and diagnostic assistance, not real-time interaction (Appendix H.1). An extra 31 ms per image is small in that workflow.
  • Gains not bought with capacity. The new controls keep essentially the capacity of the full FARE (Table 4; PSNR, in-house/Kvasir). Yet removing spectral selection costs 0.11/0.12 dB, and a coordinate-based mask with the same parameters costs 0.05/0.04 dB. Addition instead of FusionGate costs 0.07/0.05 dB, and Haar instead of DTCWT costs 0.12/0.16 dB (Table 7). The gains thus follow the mechanism, not the parameter count.

References

  • Huang, H., He, R., Sun, Z. and Tan, T. (2017). Wavelet-SRNet: A wavelet-based CNN for multi-scale face super resolution. In ICCV, pp. 1689–1697.
  • Jeevan, P., Srinidhi, A., Prathiba, P. and Sethi, A. (2024). WaveMixSR: Resource-efficient neural network for image super-resolution. In WACV, pp. 5884–5892.
  • Fuoli, D., Van Gool, L. and Timofte, R. (2021). Fourier space losses for efficient perceptual image super-resolution. In ICCV, pp. 2360–2369.
AuthorsOpenReview ↗

Response to Reviewer bmgv (2/2)

W4/Q1. "Could the authors clarify how the Gaussian mask in Equations (4)–(5) selectively isolates mid-frequency components and provide visualizations of the learned frequency masks?"

The Gaussian acts on the spectral magnitude by design, and it isolates mid frequencies in three steps. First, feature spectra are dominated by low frequencies, so magnitude decays with radial frequency on average. Second, the largest magnitudes therefore form the low-frequency core, and the smallest form the high-frequency noise floor. Third, a Gaussian centered at intermediate magnitudes suppresses both ends and passes the band in between.

The magnitude domain also adds adaptivity. Since (w, b) is predicted per patch, the band follows each patch's content. Since each coefficient is weighted by its own magnitude, the band is also not confined to a radially symmetric ring. A coordinate-based mask with the same per-patch parameters is confined to one. We verify this with a controlled test (ATD, blind ×4; now in Table 4):

AMFD spectral maskIn-house PSNR / SSIM / LPIPSKvasir PSNR / SSIM / LPIPS
Uniform, M ≡ 1 (no spectral selection)29.43 / 0.7491 / 0.140034.69 / 0.8770 / 0.1108
Coordinate-based Gaussian29.49 / 0.7529 / 0.137734.77 / 0.8770 / 0.1107
Magnitude-based Gaussian (ours)29.54 / 0.7541 / 0.135934.81 / 0.8774 / 0.1105

Without spectral selection, FARE falls to the HFSA-only level (29.43 vs. 29.42 dB, in-house). The coordinate-based mask recovers only part of the gain, and the magnitude-based mask is best on every metric and on both datasets. We report this test instead of a mask visualization, because a visualization shows where a mask acts but not whether the selection matters.

W4/Q2. "Could the authors provide an ablation combining HFSA and AMFD without FusionGate, combine with a computation-matched comparison ...?"

Yes. As requested by the reviewer, we replace FusionGate (Eq. 7–8) with plain addition of the HFSA and AMFD outputs and keep all else fixed. The two variants differ only by the two 1×1 convolutions of Eq. (7) per block, so their computation is essentially matched (ATD, blind ×4; now in Table 4):

VariantIn-house PSNR / SSIM / LPIPSKvasir PSNR / SSIM / LPIPS
HFSA only29.42 / 0.7492 / 0.140234.67 / 0.8770 / 0.1108
AMFD only29.44 / 0.7502 / 0.139234.77 / 0.8774 / 0.1108
HFSA + AMFD, addition29.47 / 0.7523 / 0.138534.76 / 0.8772 / 0.1108
HFSA + AMFD, FusionGate (ours)29.54 / 0.7541 / 0.135934.81 / 0.8774 / 0.1105

FusionGate beats addition on every metric and on both datasets. With addition, the combined model barely improves on the better single branch: +0.03 dB in-house and no gain on Kvasir (34.76 vs. 34.77 dB). With FusionGate, the gain is 0.10/0.04 dB. FusionGate therefore contributes on its own.

We hope these results address the reviewer's concerns, and we would be grateful if the reviewer would consider raising the score.

As the rebuttal deadline approaches, We greatly look forward to hearing back from you. If there are additional questions or points that require further elaboration, we welcome them and will do our best to respond promptly and effectively before the deadline.

Reviewer_XdMcOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper proposes FARE, a plug-and-play module designed to enhance medical image super-resolution by explicitly attending to mid- and high-frequency details. FARE comprises two key components: a High-Frequency Semantic Amplifier (HFSA), which combines directional wavelet decomposition with gradient-informed gating, and an Adaptive Mid-Frequency Decoupler (AMFD), which applies patch-level STFT with a learnable Gaussian band-pass mask. The method is evaluated across multiple SR baselines on diverse medical datasets, including ultrasound, endoscopy, and MRI. In addition to standard spatial metrics, the authors introduce frequency-domain evaluations to better assess spectral alignment. Comprehensive ablation studies confirm the effectiveness of each component.

Critical strengths
  1. The experiments in this paper are well-designed. The authors evaluate FARE on six representative super-resolution baselines across multiple medical datasets. The ablation studies are also complete, validating the individual contributions of HFSA and AMFD, the necessity of the gradient prior, and comparisons against common alternative attention modules. These results confirm that FARE works well as a plug-and-play module that integrates into different models without much architectural changes. Furthermore, the paper is well-written and includes both quantitative metrics and intuitive visual comparisons alongside frequency-domain evaluations, making the overall validation clear and convincing.

  2. In addition to PSNR, SSIM, and LPIPS, the authors introduce frequency-domain metrics to evaluate spectral alignment between reconstructed and ground-truth images.

Critical weaknesses
  1. Applying super-resolution to endoscopy and ultrasound images is questionable. Modern endoscopy images already provide high optical resolution, and the real clinical bottlenecks are illumination, occlusion, and motion blur rather than spatial resolution. For ultrasound, spatial resolution is constrained by acoustic physics rather than digital grid size. Therefore, super-resolution on these two modalities may hallucinate non-existent anatomical structures. Additionally, BSRGAN degradation is designed for natural images. It is unsuitable for medical images, particularly for capturing the modality-specific physical degradation processes of ultrasound. Furthermore, no downstream tasks such as segmentation are evaluated. PSNR and SSIM alone are insufficient to demonstrate that the model can improve real clinical diagnosis. Also, in many cases, the performance improvements are not particularly significant.

  2. The novelty of this work is incremental. Equations 7 and 8 are essentially equivalent to standard cross-attention. In the HFSA module, the application of DTCWT for super-resolution is well-established. Utilizing gradients to guide the network toward structural regions is also common. Furthermore, generating a mask via concatenation, convolution, and sigmoid constitutes the most standard spatial attention mechanism. Similarly, AMFD follows the standard STFT procedure. The relatively notable contribution is the affine reparameterization trick to ensure numerical stability when optimizing the Gaussian mask in the frequency domain. However, this remains an engineering practice rather than a theoretical breakthrough.

Questions for authors
  1. The evaluation relies solely on standard metrics (PSNR, SSIM, LPIPS), which do not clearly indicate whether the model benefits actual clinical diagnosis. Given that super-resolution models can hallucinate non-existent anatomical details, have the authors evaluated FARE on downstream clinical tasks? Such experiments would help demonstrate whether the enhanced images are genuinely useful for diagnosis.

  2. How does FARE compare to other frequency-based SR methods or frequency-based losses? It would help to see this comparison to understand where FARE fits among existing approaches.

  3. Could integrating the method with linear attention-based methods, such as MambaIR or Restore-RWKV lead to similar improvements?

7 comments
AuthorsOpenReview ↗

Response to Reviewer XdMc (1/3)

We thank the reviewer for recognizing our experimental design, complete ablations and frequency-domain evaluation. We believe the main concerns stem from a misunderstanding of what SR is used for here and of how FARE works. Table and figure numbers refer to the revised PDF.

W1-a. "Applying super-resolution to endoscopy and ultrasound images is questionable."

We respectfully disagree, because the argument conflates two different problems.

  • Acquisition artifacts vs. delivered resolution. Illumination, occlusion and motion blur are acquisition artifacts, and acoustic physics bounds what a probe can acquire. SR fixes neither. It addresses the resolution of the delivered image. Small sensors limit this resolution, as in capsule endoscopes, and scan conversion, resampling and compression reduce it further. Solving one problem does not remove the other.
  • The target is the real image. The HR reference is always the native clinical image. FARE is therefore trained and evaluated to reproduce detail that the device actually acquired, never to exceed a physical limit.
  • SR for these modalities is an established direction. EndoL2H (IEEE TMI 2020) and MDA-SR (MICCAI 2023) address capsule endoscopy, and DMUISR (PMB 2024) addresses fetal ultrasound. Under the same protocol, ATD+FARE outperforms EndoL2H and DMUISR in PSNR and SSIM in every setting (×4; now in Table 8):
Dataset, settingMethodPSNRSSIMLPIPS
Kvasir, blindEndoL2H32.680.86730.1385
ATD+FARE34.810.87740.1105
Kvasir, non-blindEndoL2H36.340.91140.0421
ATD+FARE40.130.94170.0335
HC18, blindDMUISR27.520.74690.1483
ATD+FARE28.760.80120.1669
HC18, non-blindDMUISR31.090.83770.0597
ATD+FARE33.930.90440.0778

DMUISR, a diffusion model, reaches lower LPIPS on HC18, but its PSNR is 1.24–2.84 dB lower. This is the perception–distortion trade-off of generative SR (Blau & Michaeli, 2018). Clinical reading requires detail that is both visible and faithful.

Clinical value. In EUS, lesion-layer localization and invasion-range assessment depend on how clearly the wall layers and lesion boundaries can be read (Appendix H.2). FARE sharpens exactly these boundaries and textures (Figs. 3, 6), so these structures become easier to read and localize.

References

  • Almalioglu, Y., Ozyoruk, K. B., Gokce, A., et al. (2020). EndoL2H: Deep super-resolution for capsule endoscopy. IEEE Transactions on Medical Imaging, 39(12):4297–4309.
  • Liu, T., Chen, Z., Li, Q., et al. (2023). MDA-SR: Multi-level domain adaptation super-resolution for wireless capsule endoscopy images. In MICCAI, pp. 518–527.
  • Liu, T., Han, S., Xie, L., Xing, W., Liu, C., Li, B. and Ta, D. (2024). Super-resolution reconstruction of ultrasound image using a modified diffusion model. Physics in Medicine & Biology, 69(12):125026. (DMUISR)
  • Blau, Y. and Michaeli, T. (2018). The perception-distortion tradeoff. In CVPR, pp. 6228–6237.
AuthorsOpenReview ↗

Response to Reviewer XdMc (2/3)

W1-b. "... super-resolution on these two modalities may hallucinate non-existent anatomical structures."

This risk concerns mainly generative SR (GANs, diffusion), which samples detail from a learned prior. Our models do not. All endoscopy and ultrasound models, with or without FARE, are deterministic regressors trained with a pixel-wise L1 loss (Appendix C). Such a regressor predicts a central estimate of the plausible HR images, so its typical error is over-smoothing, not invention. FARE adds no generative prior or adversarial loss. Two analyses confirm that the recovered detail is real.

  • Spectral match to the ground truth (Fig. 4). FARE raises the mid/high-frequency energy ratio for all six backbones. At the same time, the spectra move closer to the ground truth: residual energy falls by 27–39%, KL divergence by 20–26%, and spectral correlation rises. The added energy is thus recovered content.
  • No speckle amplification (Appendix L.2). On the noisy, low-SNR tail of the ultrasound test inputs (Fig. 7), FARE does not systematically raise the estimated noise level or the high-frequency energy ratio. SNR also shows no consistent decrease.

W1-c. "BSRGAN degradation is designed for natural images. It is unsuitable for medical images ..."

The blind setting is not meant to replicate ultrasound physics. It tests robustness to unknown, mixed degradations (blur, noise, resampling, compression) that occur along clinical imaging chains (Section 3.1). It is also not our only setting: the non-blind setting and physically grounded k-space undersampling for MRI complement it. FARE improves in all three settings (Tables 1–3), so its benefit does not hinge on one degradation model.

W1-d / Q1. "PSNR and SSIM alone are insufficient ... the performance improvements are not particularly significant." (and Q1 on downstream tasks)

We agree, which is why the paper does not rely on them alone. As the reviewer notes among the strengths, we also report four frequency-domain metrics (Fig. 4) and visual comparisons on every modality (Figs. 3, 5, 6).

PSNR and SSIM are insufficient because they average over the whole image (Appendix D). Medical images contain large homogeneous regions (Section 1), while the boundaries and textures that FARE restores occupy few pixels. Their recovery therefore barely moves these averages. On the in-house blind set, for example, PFT gains only 0.03 dB in PSNR, while its residual spectral energy falls by 37% (Fig. 4).

Even by these metrics, the gains are systematic. Across 52 separately trained baseline-vs-FARE pairs, FARE improves SSIM and LPIPS in all 52 and PSNR in 51 (one tie). A two-sided sign test gives p < 10⁻¹⁵ for each metric (Appendix J).

Downstream evaluation answers a different question. A downstream score depends on the task model and its training as much as on the SR output, so it cannot isolate what SR recovers. We therefore evaluate against the ground-truth image, in pixel, perceptual and spectral terms.

AuthorsOpenReview ↗

Response to Reviewer XdMc (3/3)

W2. "The novelty of this work is incremental."

We respectfully disagree. The review judges each operator in isolation, but every architecture is built from standard operators. The contribution lies in the principle and in how the operators serve it.

Our principle is specific to medical SR. Mid frequencies carry contextual tissue textures, whereas high frequencies carry sharp boundaries, so the two must be modeled separately (Appendix G.1). Existing frequency-aware SR contrasts low with high frequencies or treats all high frequencies as one pool, and therefore mixes these cues. HFSA and AMFD separate them, and FusionGate recombines them. We test each choice below.

  • "Equations 7 and 8 are essentially equivalent to standard cross-attention." They are not. Cross-attention computes query–key similarities and normalizes them with a softmax; Eq. (7)–(8) do neither. Each branch produces an element-wise sigmoid gate through a 1×1 convolution. This gate modulates the other branch at a cost linear in the number of pixels. To avoid confusion, the revised Section 2.4 now calls these gating weights. The gate matters: plain addition lowers PSNR by 0.07/0.05 dB (in-house/Kvasir) and worsens every metric (Table 4).
  • DTCWT. We do not claim DTCWT as new. We use it for its six directional subbands and approximate shift invariance (Section 2.2, Appendix I). The choice matters: Haar wavelets in the same structure are 0.12/0.16 dB worse (Table 7).
  • HFSA gating. The gate is standard; its input is not. It combines directional DTCWT magnitudes with a normalized gradient prior that separates structural HF responses from noise. Removing the prior (grad = 1) worsens SSIM and LPIPS in all four settings of Table 4 and never improves PSNR.
  • AMFD. The novelty is content-adaptive band selection, not the STFT: (w, b) is predicted per patch, and the Gaussian acts on the spectral magnitude. The affine reparameterization only makes this selection stable to train. The selection is what matters. Without it (M ≡ 1), FARE falls to the HFSA-only level (29.43 vs. 29.42 dB). A coordinate-based mask reaches 29.49 dB, and ours reaches 29.54 dB (in-house, Table 4).

Q2. "How does FARE compare to other frequency-based SR methods or frequency-based losses?"

The paper already contains this comparison, and as requested by the reviewer, the revision extends it (PSNR, in-house/Kvasir).

  • Frequency-based loss. Replacing FARE with the Fourier-space loss of Fuoli et al. (2021) gives 29.43/34.70 dB on ATD, versus 29.54/34.81 dB for FARE. On SwinIR, it gives 29.39/34.51 dB, versus 29.44/34.55 dB (Table 4).
  • Wavelet variant. Haar wavelets inside FARE are 0.12/0.16 dB worse than DTCWT (Table 7).
  • Frequency-based SR method (new). ATD+FARE exceeds DMUISR, a wavelet-integrated diffusion model, by 1.24/2.84 dB on HC18 (blind/non-blind; Table 8).

Q3. "Could integrating the method with linear attention-based methods, such as MambaIR or Restore-RWKV lead to similar improvements?"

Yes. FARE acts on the output of each residual group, so it is independent of the token mixer inside the group (Section 2.1). The submission already covers 10 backbones with diverse token mixers (Tables 1–3, 7). They range from CNNs and window attention to graph aggregation and a diffusion model.

As requested by the reviewer, we added MambaIR (Guo et al., ECCV 2024), a linear-complexity state-space backbone. We keep its official ×4 classical-SR configuration and insert one FARE module after each residual state-space group (RSSG). This is the same position as in all other backbones (Section 2.1, Fig. 2). The FARE design is unchanged; only its channel width matches MambaIR's feature dimension. Both models are trained from scratch with the protocol of Appendix C and evaluated at the final checkpoint (blind ×4; now in Table 7):

ModelIn-house PSNR / SSIM / LPIPSKvasir PSNR / SSIM / LPIPS
MambaIR29.28 / 0.7480 / 0.141134.52 / 0.8755 / 0.1112
MambaIR + FARE29.42 / 0.7513 / 0.137234.63 / 0.8761 / 0.1107

FARE improves MambaIR on every metric and on both datasets (+0.14/+0.11 dB PSNR; −0.0039/−0.0005 LPIPS), consistent with the other 10 backbones.

We hope these clarifications resolve the concerns about our clinical goal and technical design. We respectfully ask the reviewer to reconsider the rating.

References

  • Fuoli, D., Van Gool, L. and Timofte, R. (2021). Fourier space losses for efficient perceptual image super-resolution. In ICCV, pp. 2360–2369.
  • Guo, H., Li, J., Dai, T., Ouyang, Z., Ren, X. and Xia, S.-T. (2024). MambaIR: A simple baseline for image restoration with state-space model. In ECCV.
Reviewer_eSFMOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FARE, a plug-and-play module for medical image super-resolution, designed to be inserted after each residual group. The HFSA branch combines the high-frequency sub-bands of a Dual-Tree Complex Wavelet Transform (DTCWT) with a gradient prior to predict a spatial mask that modulates the features. The AMFD branch applies an FFT to overlapping feature patches and uses a learnable Gaussian mask to extract “mid-frequency” components. The two branches are fused via bidirectional gating. The authors integrate FARE into ten backbones and evaluate it on endoscopic ultrasound, endoscopy, fetal ultrasound and MRI data under blind, non-blind and k-space downsampling settings. Experimental results demonstrate consistent effectiveness of the proposed method.

Critical strengths
  1. The motivation is clear, and the module design aligns well with it.
  2. The evaluation is extensive, covering CNN and Transformer backbones, one diffusion-based model (DiffMSR), and three imaging modalities.
  3. The ablations are fairly complete, the computational overhead is reported explicitly, and the in-house data are split by patient with IRB approval.
Critical weaknesses
  1. As formulated, AMFD is not a band-pass filter. In Eq. 5 the Gaussian takes the spectral magnitude, not the frequency position, as its variable. It therefore gates coefficients by magnitude, and its “center” and “bandwidth” are magnitude statistics, unless w and b (predicted by 1×1 convolutions) vary with frequency position, which the paper does not state. There is no comparison against a fixed radial band-pass mask and no visualization of the learned bands, so the “adaptive mid-frequency” claim is unsubstantiated.

  2. Gains are not controlled for model capacity. FARE increases ATD from 20.3M to 32.2M parameters and from 1.52T to 2.06T FLOPs (Table 6). Yet the gains on Transformer backbones in Tables 1–2 are mostly below 0.2 dB, and some are 0.00–0.03 dB. A parameter-matched comparison is required.

  3. The non-blind baselines appear under-trained. In Table 2, RCAN trails SwinIR by 6.8 dB on Kvasir. FARE adds 1.08–2.38 dB to RCAN on all three datasets and 1.77 dB to HAN on HC18, whereas Transformer backbones gain at most about 0.5 dB. Were the baselines retrained with the same schedule?

  4. FusionGate is never isolated. Tables 4–5 lack an “HFSA+AMFD without FusionGate” variant, so the negotiated-fusion claim has no supporting result.

  5. No variance is reported. No standard deviations or multiple seeds are given, and many SSIM/LPIPS differences appear only at the fourth decimal place. The Wilcoxon test covers only PFT on the in-house blind setting, and its per-image pairing ignores correlation among images from the same patient.

  6. Data partitioning is unclear. For the three 2D datasets, Appendix C describes only an 8:2 train/validation split, and no separate test set is described. It is not stated whether HC18 is split by patient, or whether the MRI 7:1:2 split is at the volume or slice level.

  7. Clinical utility is not demonstrated. The paper presents BSRGAN as simulating clinical degradation, but it is a generic model and the same pipeline generates training and test inputs. There is no real low-quality data, downstream task, or clinician assessment.

Questions for authors
  1. What exactly is the variable of the Gaussian in Eq. 5, and are w and b frequency-dependent?
  2. What are the results of a parameter-matched baseline and of HFSA+AMFD without FusionGate?
  3. What training schedule and curves were used for RCAN/HAN?
  4. What are the split level (patient/volume/slice) and sample counts per dataset, and are multi-seed results available?
5 comments
AuthorsOpenReview ↗

Response to Reviewer eSFM (1/3)

We thank the reviewer for the detailed review and for recognizing our clear motivation, extensive evaluation, ablations and explicit overhead reporting. Table and figure numbers refer to the revised PDF.

W1/Q1. "As formulated, AMFD is not a band-pass filter."

We respectfully disagree. To answer Q1: the variable of the Gaussian in Eq. (5) is the spectral magnitude M_j = |X_j|. The parameters w and b do not depend on frequency position; one (w, b) pair is predicted per patch. Both choices are deliberate.

  • Magnitude selection is band selection. Feature spectra are dominated by low frequencies (Section 1), so magnitude decays with radial frequency on average. The largest magnitudes thus form the low-frequency core (including DC), and the smallest form the high-frequency noise floor. A Gaussian centered at intermediate magnitudes suppresses both ends and passes the band in between. Hence μ and σ set the center and width of a soft band-pass filter.
  • The magnitude domain adds adaptivity. The passband follows each patch's own spectrum, as motivated by Fig. 1. It is also not confined to a radially symmetric annulus, because each coefficient is weighted by its own magnitude. A coordinate-based Gaussian lacks this freedom, even with the same per-patch (w, b).
  • The band is distinct from HF. AMFD responses share little information with the DTCWT high-frequency responses (NMI 0.0747 ± 0.0016; Appendix G.2).

As requested by the reviewer, we compare with a radial band-pass mask (ATD, blind ×4; now in Table 4). Our radial mask is stronger than a fixed one, because its center and width are predicted per patch, as in AMFD. We also add a control without spectral selection.

AMFD spectral maskIn-house PSNR / SSIM / LPIPSKvasir PSNR / SSIM / LPIPS
Uniform, M ≡ 1 (no spectral selection)29.43 / 0.7491 / 0.140034.69 / 0.8770 / 0.1108
Coordinate-based Gaussian (same parameterization, on radial frequency)29.49 / 0.7529 / 0.137734.77 / 0.8770 / 0.1107
Magnitude-based Gaussian (ours, Eq. 5)29.54 / 0.7541 / 0.135934.81 / 0.8774 / 0.1105

Without spectral selection, FARE falls to the HFSA-only level (29.43 vs. 29.42 dB, in-house). AMFD therefore helps only through what it selects. The coordinate-based mask recovers only part of the gain, and the magnitude-based mask is best on every metric and on both datasets. We report this controlled test instead of a mask visualization. A visualization shows where a mask acts, but not whether the selection matters.

W2/Q2 (capacity). "Gains are not controlled for model capacity. ... A parameter-matched comparison is required."

As requested by the reviewer, the new controls provide parameter-matched comparisons (Table 4; PSNR, in-house/Kvasir). Each has the same or slightly fewer parameters than the full FARE. Together, they show that capacity does not explain the gains.

  1. Same capacity, no selection. The uniform-mask control keeps both branches and FusionGate; only the small (w, b) predictor is unused. It gains only 0.03/0.08 dB over ATD, versus 0.14/0.20 dB for the full FARE.
  2. Same parameters, different mask. The coordinate-based mask has exactly our parameter count, yet it is 0.05/0.04 dB worse.
  3. Same structure, different fusion or transform. Addition instead of FusionGate removes only two 1×1 convolutions per block, yet loses 0.07/0.05 dB. Haar instead of DTCWT loses 0.12/0.16 dB (Table 7).
  4. More parameters alone do not help here. HAT has about 1.75× the parameters of SwinIR (20.8M vs. 11.9M at ×4; Chen et al., 2023). Under the same protocol, it is not better: 29.28 vs. 29.33 dB in-house and 34.20 vs. 34.46 dB on Kvasir (Tables 1, 7).

Why some PSNR gains are small. PSNR averages over all pixels, while the detail FARE restores occupies few of them (General Response, point 2). On the in-house blind set, for example, PFT gains only 0.03 dB in PSNR. Its residual spectral energy, however, falls by 37% (3521 → 2217; Fig. 4). Where backbones leave more headroom, the PSNR gains are larger: 0.29–0.75 dB for the Transformer backbones in MRI k-space SR (Table 3).

References

  • Chen, X., Wang, X., Zhou, J., Qiao, Y. and Dong, C. (2023). Activating more pixels in image super-resolution transformer. In CVPR, pp. 22367–22377.
AuthorsOpenReview ↗

Response to Reviewer eSFM (2/3)

W3/Q3. "The non-blind baselines appear under-trained. ... Were the baselines retrained with the same schedule?"

Yes. Every model, with or without FARE, is trained from scratch with one protocol (Appendix C). It uses 64×64 LR patches, batch size 16, Adam, an L1 loss and 250k iterations. The learning rate starts at 2×10⁻⁴ and is halved at 150k/200k/225k/240k. RCAN and HAN follow exactly this schedule, and nothing is tuned per backbone. Each baseline and its FARE counterpart thus share data, schedule and budget, so their difference isolates the module.

The larger gains on CNNs are expected. RCAN and HAN rely on local convolutions and lack long-range spatial modeling, which AMFD's patch-level STFT supplies (Section 1). Transformers already model long-range dependencies and leave less headroom. Our conclusion does not rest on CNNs, however. On the Transformer backbones alone, FARE improves PSNR in 33 of 34 comparisons (one tie) and SSIM and LPIPS in all 34 (Tables 1–3, 7).

W4/Q2 (FusionGate). "FusionGate is never isolated."

As requested by the reviewer, we replace FusionGate (Eq. 7–8) with plain addition of the two branch outputs and keep everything else fixed. The two variants differ only by the two 1×1 convolutions of Eq. (7) per block (ATD, blind ×4; now in Table 4):

VariantIn-house PSNR / SSIM / LPIPSKvasir PSNR / SSIM / LPIPS
HFSA only29.42 / 0.7492 / 0.140234.67 / 0.8770 / 0.1108
AMFD only29.44 / 0.7502 / 0.139234.77 / 0.8774 / 0.1108
HFSA + AMFD, addition29.47 / 0.7523 / 0.138534.76 / 0.8772 / 0.1108
HFSA + AMFD, FusionGate (ours)29.54 / 0.7541 / 0.135934.81 / 0.8774 / 0.1105

FusionGate beats addition on every metric and on both datasets. With addition, the combined model barely improves on the better single branch: +0.03 dB in-house and no gain on Kvasir (34.76 vs. 34.77 dB). With FusionGate, the gain is 0.10/0.04 dB. Summing the two bands thus does not make them cooperate, whereas the gating does, as intended (Section 1).

W5/Q4 (variance). "No variance is reported."

Our conclusions rest on consistency across independent runs, not on any single run or fourth-decimal difference. The submission reports 52 baseline-vs-FARE comparisons (Tables 1–3, 7), each a separately trained pair of models. FARE improves SSIM and LPIPS in all 52 and PSNR in 51 (one tie). Under the null hypothesis of no effect, a two-sided sign test gives p < 10⁻¹⁵ for each metric. Appendix J now reports this test, including the two MambaIR comparisons.

The test has two strengths. It spans all backbones, datasets and settings, whereas repeated seeds would probe one configuration. It also treats each training run as one sample, so within-patient correlation cannot affect it. The per-image Wilcoxon test on PFT (Appendix J) is complementary. It is significant for all three metrics, even though the mean PSNR gain there is only 0.03 dB.

W6/Q4 (splits). "Data partitioning is unclear."

As requested by the reviewer, Section 3.1 and Appendix C now state the split unit of each dataset. All splits are fixed and shared by each baseline and its FARE counterpart:

  • In-house EUS (1,658 images): 8:2, split by patient.
  • HC18 (1,334 images from 551 women): 8:2, split by subject.
  • Kvasir (4,000 images, 8 classes): 8:2 within each class.
  • BraTS2018 and IXI: 7:1:2 at the volume level, using the central 100 axial slices of each volume.

For the 2D datasets, the 20% portion is the held-out test set reported in all tables. The original Appendix C called it "validation". It is used neither for training nor for checkpoint selection.

AuthorsOpenReview ↗

Response to Reviewer eSFM (3/3)

W7. "Clinical utility is not demonstrated."

We respectfully disagree. An SR method is clinically useful to the extent that it restores real anatomical detail, and the paper demonstrates this.

  • What FARE restores. Clinicians read boundaries and textures, such as the wall layers and lesion margins in EUS, for lesion-layer localization and invasion-range assessment (Appendix H.2). FARE makes these structures clearer on every modality (Figs. 3, 5, 6). The restored content is also closer to the ground-truth spectrum (Fig. 4).
  • BSRGAN. The blind setting tests robustness to unknown, mixed degradations (blur, noise, resampling, compression; Section 3.1). It does not model a specific scanner. Bicubic downsampling and physically grounded k-space undersampling for MRI complement it. FARE improves in all three settings, so its benefit does not hinge on one degradation model. Synthetic degradation for training and testing is the standard SR protocol, because pixel-aligned real LR–HR medical pairs are rarely available.
  • No speckle amplification. The blind LR inputs of the two ultrasound test sets include a heavy tail of noisy, low-SNR images (Fig. 7). On these tail cases, FARE does not systematically raise the estimated noise level or the high-frequency energy ratio. SNR also shows no consistent decrease (Appendix L.2).
  • Dedicated medical SR methods. Under the same protocol, ATD+FARE exceeds EndoL2H on Kvasir by 2.13/3.79 dB PSNR (blind/non-blind) and DMUISR on HC18 by 1.24/2.84 dB (Table 8).
  • Reader studies and downstream tasks evaluate a clinical workflow. Their outcome depends on the reading protocol and the task model as much as on the SR output. They are therefore a separate validation question, beyond the scope of an SR method paper.

We hope these answers resolve the reviewer's concerns. The mask design and the fusion module are now backed by a clear rationale and controlled experiments. The gains are reliable across 52 independent comparisons. We respectfully ask the reviewer to reconsider the rating.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response to All Reviewers

We thank all reviewers for their careful reading. All three found the paper clear and easy to follow. Reviewer eSFM found the motivation clear, and Reviewers XdMc and bmgv found the experiments and ablations thorough. Several concerns stem from a misreading of our goal or design, so we address the shared points here. Each reviewer thread is also self-contained. All table and figure numbers refer to the revised PDF. There, Fig. 4 (frequency-domain evaluation) and Fig. 5 (MRI visual comparison) have swapped places relative to the submission.

1. New experiments, as requested by the reviewers (ATD, blind ×4; now in Table 4). Each control changes only the spectral mask or the fusion rule.

VariantIn-house PSNR / SSIM / LPIPSKvasir PSNR / SSIM / LPIPS
ATD baseline29.40 / 0.7484 / 0.141234.61 / 0.8764 / 0.1108
FARE, uniform mask (no spectral selection)29.43 / 0.7491 / 0.140034.69 / 0.8770 / 0.1108
FARE, coordinate-based Gaussian mask29.49 / 0.7529 / 0.137734.77 / 0.8770 / 0.1107
FARE, addition instead of FusionGate29.47 / 0.7523 / 0.138534.76 / 0.8772 / 0.1108
FARE (magnitude-based mask + FusionGate)29.54 / 0.7541 / 0.135934.81 / 0.8774 / 0.1105
  • Spectral selection is essential. The uniform mask keeps both branches and FusionGate, yet gains only 0.03/0.08 dB over ATD (PSNR, in-house/Kvasir). The full FARE gains 0.14/0.20 dB.
  • Magnitude beats coordinates. A coordinate-based mask with the same per-patch parameters is 0.05/0.04 dB worse.
  • FusionGate beats addition, by 0.07/0.05 dB.
  • Capacity does not explain the gains, because all three controls have essentially the same capacity as the full FARE.

As requested by Reviewer XdMc, we also tested MambaIR. FARE improves it on every metric (+0.14/+0.11 dB; Table 7). The revision further adds a comparison with two dedicated medical SR methods (Table 8) and a speckle analysis on ultrasound (Appendix L.2).

2. Our goal is the faithful recovery of fine anatomical detail. PSNR, SSIM and LPIPS are reference indicators.

FARE restores boundaries and textures, which occupy few pixels. PSNR and SSIM average over the whole image (Appendix D), and medical images contain large homogeneous regions (Section 1). Recovered detail therefore moves these averages only slightly. LPIPS, in turn, rewards perceptual plausibility, which generative models can reach at the cost of fidelity (Table 8; Blau & Michaeli, 2018). We therefore also measure detail recovery directly. On the in-house blind set, PSNR rises by only 0.03–0.40 dB across six backbones (Table 1). On the same runs, FARE cuts the residual spectral energy by 27–39% and the spectral KL divergence by 20–26% for every backbone (Fig. 4). The visual comparisons agree on every modality (Figs. 3, 5, 6).

3. The gains are systematic, not noise.

The submission reports 52 baseline-vs-FARE comparisons over 10 backbones, 5 datasets and 3 degradation settings (Tables 1–3, 7). Each is a separately trained pair of models. FARE improves SSIM and LPIPS in all 52 and PSNR in 51 (one tie), and it never lowers any metric. If FARE had no effect, each comparison would be equally likely to go either way. A two-sided sign test rejects this with p < 10⁻¹⁵ for each metric. The two MambaIR comparisons follow the same pattern, and Appendix J now reports this test.

4. Two clarifications.

  • AMFD. The Gaussian in Eq. (5) acts on the spectral magnitude by design. Magnitude decays with frequency on average, so a Gaussian centered at intermediate magnitudes passes an intermediate band. It is therefore a soft band-pass filter. Because (w, b) is predicted per patch, the band also adapts to each patch (Section 2.3).
  • FusionGate. Eq. (7)–(8) are element-wise sigmoid gates computed by 1×1 convolutions. There is no query–key similarity and no softmax, so FusionGate is not cross-attention. The revised Section 2.4 now calls them gating weights.

5. Revised PDF (all changes in blue). New results: Table 4 and Section 3.3 (controls), Table 7 (MambaIR), Appendix J (sign test), and Appendix L (Table 8, Fig. 7). Clarified text: Section 2.3 (magnitude-domain band-pass), Section 2.4 (gating terminology), Section 3.1 (degradation setting, MRI split), and Appendix C (data splits, training protocol).

References

  • Blau, Y. and Michaeli, T. (2018). The perception-distortion tradeoff. In CVPR, pp. 6228–6237.
AuthorsOpenReview ↗

Update: downstream classification and segmentation (Appendix L.3)

Following Reviewer XdMc's suggestion, the revised PDF adds downstream-parity checks on Kvasir (Appendix L.3, Table 9). Each downstream model is trained on HR images and kept fixed. It is then tested on HR, bicubic, ATD and ATD+FARE versions of the same blind ×4 test inputs. ATD+FARE improves every downstream metric over ATD. Eight-class classification accuracy rises from 53.3% to 60.1%, and polyp-segmentation Dice from 38.8% to 52.2%, while HD95 falls from 67.1 to 46.7 px. These results also bear on the downstream question of Reviewer eSFM (W7) and the overhead question of Reviewer bmgv (W3): FARE's small PSNR gains understate its practical benefit.

AuthorsOpenReview ↗

Summary of Rebuttal and Discussion

Dear ACs and Reviewers,

We thank you for your time and constructive feedback on our submission.

We are encouraged that, following the rebuttal, all three reviewers (eSFM, XdMc, bmgv) raised their ratings from weak reject to weak accept. Reviewers eSFM and XdMc noted that our responses and additional experiments resolved most of their concerns. Reviewer XdMc also raised the soundness score and retracted two initial objections. We also appreciate the reviewers' recognition of our clear presentation (eSFM, XdMc, bmgv), our clear motivation and reasonable frequency-aware design (eSFM, bmgv), and our extensive, well-designed experiments and ablations across backbones, datasets and modalities (eSFM, XdMc, bmgv).

During the rebuttal, we conducted additional experiments and analyses and revised the paper in the following areas (all changes in blue):

  • Downstream benefit (XdMc, eSFM). We added downstream-parity checks on Kvasir, confirming that the detail restored by FARE benefits downstream analysis. A classifier and a segmenter are trained on HR images and kept fixed. On identical blind ×4 inputs, ATD+FARE improves every metric over ATD: 8-class accuracy rises from 53.3% to 60.1%, polyp Dice from 38.8% to 52.2%, and HD95 falls from 67.1 to 46.7 px (Appendix L.3, Table 9).
  • Validation of the frequency mechanism (eSFM, bmgv). We added capacity-matched controls, confirming that the gains come from frequency selection and gated fusion, not from extra parameters. Without spectral selection, only 0.03/0.08 dB of the 0.14/0.20 dB PSNR gain remains (in-house/Kvasir). A coordinate-based mask loses 0.05/0.04 dB, and plain addition instead of FusionGate loses 0.07/0.05 dB (Table 4). We also clarified that the magnitude-domain Gaussian acts as a per-patch soft band-pass filter (Section 2.3).
  • Statistical reliability (eSFM, bmgv, XdMc). We added a sign test over all 54 baseline-vs-FARE comparisons, each a separately trained pair (11 backbones, 5 datasets, 3 degradation settings), confirming that the gains are systematic. FARE improves SSIM and LPIPS in all 54 and PSNR in 53 (one tie), with p < 10⁻¹⁵ for each metric (Appendix J). PSNR also understates the restored detail: on the in-house blind set, residual spectral energy falls by 27–39% for all six backbones (Fig. 4).
  • Cost–benefit (XdMc, bmgv). We showed that the overhead (+11.9M parameters and +31 ms per image for ATD; Table 6) scales linearly with image size and buys gains that capacity alone does not. HAT has 75% more parameters than SwinIR, yet it is not better under our protocol (Tables 1, 7), whereas FARE delivers the downstream gains above.
  • Positioning and generality (XdMc, bmgv). We clarified FARE's principle: mid frequencies carry tissue texture, and high frequencies carry boundaries. Each design choice beats its standard alternative: DTCWT over Haar wavelets (Table 7), FARE over a Fourier-space loss, and gating over addition (Table 4). FARE also improves MambaIR on every metric (+0.14/+0.11 dB; Table 7). Reviewer XdMc found these comparisons "adequately addressed".
  • Medical SR baselines and robustness (XdMc, eSFM). ATD+FARE outperforms two SR methods designed for endoscopy and fetal ultrasound by 1.24–3.79 dB PSNR (Table 8). A speckle analysis shows that FARE does not systematically amplify noise, even on the noisiest ultrasound inputs (Appendix L.2). FARE also improves under all three degradation settings (blind, bicubic and k-space), so its benefit does not hinge on one degradation model.
  • Protocol clarity (eSFM). We clarified that all models, including RCAN and HAN, share one training schedule, and we now state the split of every dataset, e.g., by patient for the in-house data and by volume for MRI (Section 3.1, Appendix C).

We believe these revisions and additional experiments have substantially strengthened our paper. We really appreciate your time and effort in coordinating the review process.

Sincerely,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.