acceptodds
Under review as a conference paper at ICLR 2027

Spectral Profiling for Complementary Dual-Branch Capacity Allocation in One-Step Diffusion Super-Resolution

Abstract

One-step diffusion super-resolution enables efficient generative restoration by replacing iterative sampling with a single denoising pass. A fixed U-Net exhibits heterogeneous spectral responses across layers, yet existing dual-LoRA methods typically retain uniform ranks across adapted layers. We propose Frequency-Response-Guided Diffusion Super-Resolution (FreqDiSR), which profiles a frozen U-Net by measuring the Fourier high-frequency response of each layer and converts the resulting profile into a static adaptation capacity map. Under a fixed total rank budget for the two branches at each profiled module, decreasing responses increase the capacity of the pixel branch, whereas increasing responses increase the capacity of the semantic branch, coupling the measured spectral ordering with their respective fidelity and perception objectives. Haar-based decoder modulation and low-frequency supervision extend the same separation of structure and detail to feature processing and training objectives without introducing additional denoising steps. Allocation controls with matched budgets favor the measured correspondence over inverse mappings, depth-based mappings, and permutations on perceptual metrics, while subset and degradation analyses show that the spectral profile is stable. On DRealSR, FreqDiSR improves PSNR by and reduces LPIPS by over PiSA-SR. Across three benchmarks, it achieves the lowest LPIPS and the lowest DISTS on RealSR and DRealSR.

Primary area:
generative models
Keywords:
one-step diffusion super-resolution, spectral profiling, low-rank adaptation, parameter-efficient adaptation, real-world image restoration

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_jNeLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FreqDiSR, a spectral-guided capacity allocation framework for one-step diffusion-based image super-resolution. Building upon a dual-branch LoRA architecture, the method uses high-frequency energy statistics extracted from a frozen U-Net to allocate complementary ranks between pixel-oriented and semantic-oriented branches under a fixed capacity budget. The framework further incorporates wavelet-based high-frequency modulation and a low-frequency structural constraint. Experiments on super-resolution benchmarks report improvements in reconstruction and perceptual quality, supported by component ablations, alternative allocation strategies, and statistical analyses.

Critical strengths

1.Rather than simply increasing adaptation capacity, the paper considers how a limited LoRA budget should be distributed between branches with different restoration objectives. This is a meaningful problem for parameter-efficient adaptation of diffusion-based super-resolution models.

2.The proposed framework achieves improvements over the underlying dual-branch baseline, and the authors include several controlled allocation strategies, including inverse, depth-based, and shuffled assignments. These experiments provide preliminary evidence that the allocation pattern matters, although they do not fully establish the necessity of the proposed spectral criterion.

Critical weaknesses
  1. The contribution of spectral-guided capacity allocation is not convincingly isolated from the other components of the framework.

The paper attributes its performance improvements to spectral-guided capacity allocation, but the overall framework simultaneously introduces multiple modifications, including frequency-based rank assignment, wavelet modulation, and additional structural supervision. Although component ablations and alternative allocation strategies are provided, the evidence does not fully establish how much of the improvement can be attributed specifically to the proposed spectral allocation mechanism.

In particular, the comparison against uniform rank allocation is not strictly controlled, as it also changes the rank configuration of modules outside the spectral allocation scheme. This leaves an important confounding factor in what should be the most direct baseline for evaluating the central contribution. Moreover, the improvements associated with the allocation strategy are relatively modest, and the relevant controlled experiments do not adequately account for training variability.

Consequently, while the complete framework appears effective, the current results do not convincingly establish that spectral-guided allocation itself provides a substantial and reproducible advantage over simpler capacity allocation strategies.

  1. The central assumption linking spectral characteristics to branch-specific capacity requirements remains insufficiently justified.

The proposed method relies on the assumption that the high-frequency energy of frozen U-Net features provides a meaningful indication of how adaptation capacity should be distributed between pixel-oriented and semantic-oriented branches. However, feature frequency statistics and the actual marginal benefit of allocating additional LoRA capacity are fundamentally different quantities.

The paper demonstrates that the spectral measurements are relatively stable and that certain alternative allocation patterns perform worse. Nevertheless, these observations do not directly establish that the proposed spectral criterion captures the underlying capacity requirements of the two branches. It remains unclear whether the observed benefits arise from a meaningful relationship between spectral characteristics and adaptation needs, or from a particular allocation heuristic that happens to work under the evaluated configuration.

This concern is particularly important because the proposed allocation rule constitutes the primary methodological novelty of the paper. Without stronger evidence connecting the spectral measurements to actual adaptation benefits, the method remains insufficiently motivated beyond its empirical performance on the reported setup.

Questions for authors
  1. Can the authors demonstrate that spectral-guided rank allocation provides a substantial and reproducible advantage over a strictly controlled uniform allocation baseline? In particular, could the authors compare against uniform allocation while keeping all modules outside the proposed spectral allocation scheme unchanged, under identical parameter budgets and training conditions? Given the relatively modest gains in the allocation-specific ablations, evidence across multiple training seeds would be important for establishing the robustness of the claimed contribution.

  2. What evidence supports the assumption that high-frequency feature energy is a meaningful indicator of branch-specific adaptation capacity requirements? The current analysis primarily establishes the stability of spectral statistics rather than their relationship to the actual benefits of allocating additional capacity. Could the authors provide direct evidence that the proposed spectral measurements predict the relative effectiveness of allocating capacity to the pixel and semantic branches, rather than merely correlating with layer characteristics or architectural structure?

  3. To what extent does the proposed spectral allocation rule generalize beyond the specific architecture and capacity configuration considered in the paper? Since the allocation mechanism is presented as the main methodological contribution, it would be important to establish whether its advantages persist under different capacity budgets or model configurations, rather than being specific to the evaluated setup.

Reviewer_WhixOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

FreqDiSR profiles the spectral responses of a frozen U-Net to divide each spatial module's LoRA rank budget between pixel and semantic branches for one-step super-resolution. It supplements this allocation with Haar-based decoder modulation and low-frequency supervision. Benchmark comparisons report improved restoration over PiSA-SR at similar latency, while matched-budget allocation controls and calibration-subset analyses assess the spectral strategy.

Critical strengths

(1) Measured, Inverse, Depth-based, and Permuted mappings use the same total LoRA parameter budget. Compared with the mean of three permuted profiles, Measured reduces LPIPS from 0.2918 to 0.2845 and DISTS from 0.2129 to 0.2082. The rank maps obtained from three independent calibration subsets agree on 224 of 226 modules in every pairwise comparison. These results support the reproducibility of the spectral configuration and the importance of its correspondence to individual modules.

(2) On DRealSR, the main table reports a 0.62 dB PSNR improvement and a 0.0140 LPIPS reduction over PiSA-SR. A separate unified paired re-evaluation on 93 image pairs supports this advantage, with a 95% confidence interval of [0.0043, 0.0185] for the LPIPS improvement. The reported latency is 0.1355 seconds per image, compared with 0.1351 seconds for PiSA-SR. These findings support practical utility, although they do not imply improvement on every metric.

Critical weaknesses

(1) About spectral allocation

In Table 7, the full configuration has 8.208704M LoRA parameters, whereas All-module rank-4 has 8.111424M. The difference comes from manually assigned ranks in the additional 32 non-spatial modules. This comparison therefore changes both spectral allocation and the capacity of the non-spatial modules.

In addition, the choice of α\alpha across ranks in ΔW=(α/r)BA\Delta W=(\alpha/r)BA is unspecified, so changes in capacity may also change the update scaling. The LPIPS and DISTS improvements over the uniform configuration are only 0.0040 and 0.0014, respectively, while the central allocation controls use a single training seed. Bootstrapping over test images and demonstrating calibration-set stability cannot replace validation across repeated training runs. A more tightly controlled uniform baseline and repeated training are needed to establish that this marginal gain is reliable.

(2) Insufficient comparisons

The dual-branch framework already exists in PiSA-SR, and offline rank allocation using activation statistics already exists in AIRA. This paper instead uses spatial spectral statistics to divide capacity between two branches within the same module. That distinction is meaningful, but a direct comparison with a non-spectral activation-based allocation signal is missing.

The main table's lowest-LPIPS claims are correct within the listed comparison set, but broader SOTA performance is not established. For example, the original TSD-SR and TVT (Fine-structure Preserved Real-world Image Super-resolution via Transfer VAE Training) papers report DIV2K-Val LPIPS values of 0.2673 and 0.2773, respectively, compared with 0.2783 here. Although these cross-paper numbers may not establish a fair ranking, they identify omitted strong baselines that could affect the conclusion.

Questions for authors

Across multiple training seeds, does spectral allocation outperform a uniform-rank baseline when the non-spatial ranks and LoRA scaling are held fixed?

At matched parameter and training budgets, does spectral profiling outperform allocation based on a simple activation statistic, such as variance?

Under a common evaluation protocol, how does the full method compare with TSD-SR and TVT in restoration quality?

Reviewer_5As9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

FreqDiSR proposes to profile the frozen U-Net’s layer-wise high-frequency responses and use them to redistribute a fixed LoRA rank budget between fidelity- and perception-oriented branches. It further introduces Haar-based decoder modulation and low-frequency supervision to separate structural preservation from detail synthesis. The method is efficient and empirically competitive, although the gains attributable specifically to spectral rank allocation are relatively modest compared with those of the auxiliary Haar-based components.

Critical strengths
  1. FreqDiSR uses the measured high-frequency response of each layer to allocate capacity between the pixel and semantic branches, instead of applying a uniform rank or a hand-crafted depth-based schedule. Since the two branch ranks sum to a fixed budget for each profiled module, the method mainly changes how capacity is divided rather than simply increasing the parameter count.
  2. The three components operate at complementary levels: Fourier profiling determines where pixel or semantic capacity is allocated, Haar modulation controls which frequency components are adjusted during decoding, and the low-frequency anchor constrains the output structure. This gives the overall method a reasonably unified motivation.
Critical weaknesses
  1. The full method combines response-guided rank allocation, decoder feature modulation, and low-frequency supervision. However, the ablations suggest that much of the PSNR improvement comes from the latter two components, while the gain from the proposed rank allocation over uniform rank is relatively modest. This makes it difficult to determine how much of the final improvement should be attributed to the paper’s central allocation strategy.
  2. The allocation rule remains largely heuristic. The assumption that layers with stronger high-frequency responses should receive more semantic rank is intuitive but not theoretically established.
Questions for authors
  1. Could the authors provide a more controlled decomposition of the gains from rank allocation, Haar modulation, and low-frequency supervision, including their interactions? In particular, how much improvement remains when only the proposed rank allocation is introduced under an exactly matched parameter budget?
  2. How sensitive are the results to the total rank budget, allocation contrast, and minimum/maximum rank assigned to each module? It would also be informative to evaluate more extreme allocations. For example, when the spectral profile strongly favors the semantic branch, what happens if the entire available rank budget is assigned to that branch, or vice versa for the pixel branch? Such limiting-case comparisons could better demonstrate whether the proposed intermediate allocation provides a meaningful trade-off.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.