acceptodds
Withdrawn from ICLR 2027

Hybrid Generalization-Aware Parameter Selection Test-Time Adaptation for All-in-One Image Restoration

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

All-in-one image restoration (AiOIR) methods achieve impressive performance across diverse degradations by learning a unified restoration model. However, they remain vulnerable to test-time distribution shifts. Existing test-time adaptation (TTA) methods stabilize online parameter updates via Fisher-based selective adaptation, yet this selection criterion is inherently one-sided: it only identifies which parameters to preserve from updates, but fails to distinguish which of the remaining parameters merit adaptation to the target degradation. To address this limitation, we propose Hybrid Generalization-aware Parameter Selection (HGPS), a budget-neutral, per-tensor selection principle. Specially designed to complement Fisher stability, HGPS employs cross-view gradient agreement as a proxy for target-relevant update utility. Within each tensor’s original update budget, HGPS hard-protects high-Fisher elements, assigns part of the update slots to parameters with consistent cross-view gradients, and fills the rest with low-Fisher elements. Furthermore, the mask is computed once before adaptation and can be plugged into Fisher-based TTA pipelines without altering their optimization or restoration schedule. Extensive experiments across multiple restoration tasks and distribution shifts demonstrate that HGPS consistently improves state-of-the-art restoration TTA, surpassing the strongest existing TTA method by 2.69 dB PSNR on Kodak24.

Primary area:
generative models
Keywords:
All-in-one image restoration, test-time adaptation, Hybrid Generalization-aware Parameter Selection

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_tdxZOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Hybrid Generalization-aware Parameter Selection (HGPS), a test-time adaptation strategy for all-in-one image restoration under distribution shifts. The method combines Fisher-based parameter importance with cross-view gradient agreement to identify parameters that are both safe and beneficial to update. Specifically, HGPS preserves high-importance parameters while selecting adaptation parameters based on gradient consistency within a fixed update budget.

Critical strengths
  1. Exploring test-time adaptation for all-in-one image restoration is an interesting and meaningful direction, as it enables restoration models to adapt to unseen degradations and improves their generalization to real-world scenarios.

  2. The proposed method is flexible and plug-and-play, allowing easy integration into existing Fisher-based test-time adaptation frameworks without modifying their original optimization objectives or adaptation pipelines.

Critical weaknesses
  1. The experimental evaluation is not sufficiently comprehensive. The proposed method is only validated on three backbone architectures, and comparisons with several recent state-of-the-art all-in-one image restoration methods, such as MoCE-IR, are missing.

  2. The computational complexity analysis is insufficient. The paper does not provide quantitative comparisons of FLOPs, GPU memory consumption, or actual test-time adaptation latency.

  3. The real-world experimental validation remains insufficient. More comprehensive evaluations across diverse real-world degradation distributions are needed to further demonstrate the robustness and generalization capability of the proposed method.

  4. The proposed cross-view gradient agreement score relies on dividing eight augmented views into two fixed groups, as described in Eq. (4). What is the motivation behind this specific grouping strategy? How sensitive is the parameter selection and restoration performance to different view partitions?

Questions for authors

See the above Weaknesses parts.

1 comment

We thank the reviewer for the comments. The revised PDF directly addresses W1, W2, and W4 with new experiments; we respectfully discuss W3 below.

Reply to W1 (three backbones; MoCE-IR missing). MoCE-IR (CVPR 2025) is now included as a fourth backbone in both protocols (Tables 1-2). HGPS improves MoCE-IR+DCTTA on all four three-task targets (family average +0.49 dB); the gain is especially pronounced in the five-task setting, where HGPS improves MoCE-IR+DCTTA on all five targets and raises the family average by +1.28 dB - the largest within-family improvement in this setting. We also note that the adopted backbones are themselves recent peer-reviewed all-in-one restoration models: StarIR (IEEE TPAMI 2026), PromptIR (NeurIPS 2023), DFPIR (CVPR 2025), and MoCE-IR (CVPR 2025). The host framework DCTTA (CVPR 2026) itself was evaluated on three backbones for both the three-task and five-task protocols; our 4-backbone x 2-protocol evaluation exceeds that coverage while keeping every within-family factor (pipeline, budget, optimizer, seed) strictly controlled, which is what licenses attributing the gains to the selection criterion alone.

Reply to W2 (computational complexity). New Sec. 4.5 / Table 4 reports a controlled profiling comparison on Kodak24: GG is computed once before adaptation (138.5 s for S=24S=24 samples); the per-iteration adaptation cost is unchanged (adaptation-time ratio 1.038); end-to-end time increases by 17.26% (8.04 s/image); peak allocated GPU memory does not increase (20.63 vs 20.70 GiB). Since the extra cost is a one-time selection pass rather than per-iteration inference, wall-clock latency and memory are the deployment-relevant metrics for TTA.

Reply to W3 (real-world validation). We respectfully clarify that our target domains are already predominantly real-world: O-HAZY and DenseHaze (real haze), SDD (real defocus blur), and VILNC-Indoor3 (real low-light) - four real-world degradation distributions spanning both protocols - plus a cross-modality generalization study on real T1-weighted MRI scans (App. E.1, Table A.3). Because our contribution is a parameter-selection criterion rather than a new restoration model, we prioritized strictly controlled comparisons (identical pipeline, budget, and seed within each family) over benchmark breadth. We will further extend real-world evaluations in the camera-ready version.

Reply to W4 (fixed two-group partition). App. C / Table A.1 evaluates 5 alternatives: two balanced random partitions, a group-swap control, a rotation-vs-reflection partition, and a four-group generalization of GG (Eq. A.4). Although GG rankings can change substantially (Spearman 0.19-0.82), the adapted outputs deviate from the default partition by at most 0.063 dB (Rain12) and 0.198 dB (SDD), and the scoring cost is unchanged (5.4-6.5 s/sample). We also added the design rationale: both default groups mix rotations and reflections, so neither group observes a single symmetry type. We now state explicitly that GG should not be interpreted as partition-invariant, and that directional degradations (rain, motion blur) require care since reflections flip the degradation orientation (Sec. 5).

We hope the new experiments and clarifications address your concerns, and we would be grateful for a re-evaluation of the score.

Reviewer_AC4VOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes Hybrid Generalization-aware Parameter Selection (HGPS), a test-time adaptation method for all-in-one image restoration under distribution shifts. It argues that Fisher-based methods such as DCTTA identify parameters that should be protected but do not distinguish which remaining parameters are useful to update. HGPS addresses this by preserving the existing per-tensor update budget while selecting some update candidates using cross-view gradient agreement: parameters whose gradients are consistent and sufficiently large across two groups of geometric views receive update slots, while the highest-Fisher parameters remain protected and the remaining slots go to low-Fisher parameters. The method is evaluated with three restoration backbones across three-task and five-task settings, where the authors report improvements over DCTTA on most benchmarks. The paper also reports that removing the Fisher protection guard sharply reduces performance, and identifies weak pseudo-supervision in very dark scenes as a limitation.

Critical strengths
  1. Addresses a gap in Fisher-based parameter selection.: Fisher scores can identify parameters that should be protected, but they do not indicate which remaining parameters are worth updating. HGPS adds a criterion that selects parameters whose gradients have sufficiently large magnitudes and consistent directions across different image transformations.

  2. Isolates the effect of parameter selection.: HGPS retains the existing adaptation procedure (including the number of updates per parameter tensor, adaptation loss, optimizer, number of optimization steps, and post-update restoration procedure) and changes only the parameter-selection criterion. This makes it easier to attribute performance differences to the selection strategy under the paper’s experimental setup.

Critical weaknesses
  1. The method relies on fixed design choices and may be less effective with weak pseudo-supervision.: The mixing ratio is a global hyperparameter, with the default assigning half of the update slots to G. Yet the best PSNR in the experiments is obtained at λ=0.375\lambda = 0.375, and the optimal ratio may vary across tasks or parameter groups. The paper also reports limited gains for all adaptation methods on the particularly challenging VILNC-Indoor3 dataset, and notes that consistency-based pseudo-labels may provide a weak signal. The results therefore leave open how reliably GG identifies useful updates across other datasets and models.

  2. The paper has substantial clarity and consistency issues in its figures, notation, and equations.: Figure 1(a) is difficult to interpret: several colors and the dashed lines are not explained clearly. Figure 2 uses LaL_a and LsL_s without defining them in the figure or connecting them to the losses defined in the paper. In particular, it is unclear whether LaL_a refers to LadaptL_{\text{adapt}} or LaugL_{\text{aug}}, and whether LsL_s denotes LsVL_{sV} or a student-consistency term in Eq. A.2. The diagram also does not distinguish the two terms in Eq. A.2, making the loss being illustrated ambiguous.
    There also appears to be a sign error in Eq. (3). The text says parameters with Mt,i=1M_{t,i}=1 are protected and restored to their pretrained values θt,i0\theta_{t,i}^{0}, but the equation is written as : θt,i−Mt,iθt,i0+(1−Mt,i)θt,i\theta_{t,i}-M_{t,i}\theta_{t,i}^{0}+(1-M_{t,i})\theta_{t,i} Substituting Mt,i=1M_{t,i}=1 yields −θt,i0-\theta_{t,i}^{0}, which flips the sign instead of restoring the pretrained weight. To match the stated behavior and the algorithm’s definition of 1 as “restore,” the first term appears to require a plus sign. The authors should clarify the intended equation and whether the implementation follows it.

Questions for authors
  1. Would adapting λ\lambda by task or layer improve performance over a single global value? The paper uses a fixed global ratio for assigning update slots to G. Could the authors compare this with task-specific or layer-specific ratios (for example, across denoising, deraining, dehazing, deblurring, and low-light tasks, or across encoder, decoder, and prompt parameters)?

  2. How sensitive is G to the choice of transformation groups? The method computes gradient agreement using two fixed groups of eight transformations. Would performance hold under alternative groupings or with other transformations, such as color changes, crops, or noise? This would help establish whether the results depend on the particular grouping used.

  3. What is the additional computation required to calculate G, and what performance gain does it provide relative to that cost? The paper states that adaptation-time computation is unchanged but that an additional cross-view score calculation is required beforehand, involving a teacher forward pass and backpropagation through both groups for each sample. Please quantify this overhead (for example, in FLOPs, runtime, or GPU-hours) and compare it directly with the resulting performance gains.

  4. How should readers interpret the qualitative restoration results, and what evidence shows that HGPS improves restoration quality? The paper provides few visualizations for a restoration method. In Fig. A.1, the purpose of the hazy example is unclear, and the bear example appears to show worse results for DFPIR+HGPS than for DFPIR+DCTTA. Could the authors provide additional representative examples and explain how the visual results support the claimed improvements?

1 comment

We thank the reviewer for the careful reading, especially of the notation and figures. All four questions are addressed in the revised PDF.

Reply to W2 (figures, notation, equations). (i) Figure 2's LsL_s/LaL_a are now explicitly defined in the caption and in App. A as the two terms of Ladapt=Ls+λteaLa\mathcal{L}_ {\mathrm{adapt}} = L_s + \lambda_ {\mathrm{tea}} L_a, distinct from the one-time Fisher-scoring loss Laug\mathcal{L}_ {\mathrm{aug}} (Eq. 1). (ii) Figure 1 has been redrawn for readability. Panel (a) now has an explicit legend: each element is colored by its cross-view score GG - green for consistent, non-vanishing gradients (updates that generalize to the target degradation) and orange for inconsistent or near-zero gradients (updates that contribute little) - and the dashed vertical lines mark the hard-guard boundary (top-FF elements, frozen) and the KtK_t update-slot split between the high-GG and low-FF branches. The face icons summarize the takeaway: combining FF (stability) with GG (update utility) yields better restoration, while FF-only selection performs worse; the caption now spells out each visual element. (iii) On the "sign error" in Eq. (3): we would like to gently clarify a notation misreading - the symbol is a long assignment arrow (LaTeX \longleftarrow), which is typeset from an arrowhead plus an extension bar that visually resembles a minus sign. The equation reads θt,i⟵Mt,iθt,i0+(1−Mt,i)θt,i\theta_ {t,i} \longleftarrow M_ {t,i}\theta^{0}_ {t,i} + (1-M_ {t,i})\theta_ {t,i}, so substituting Mt,i=1M_ {t,i}=1 yields θt,i0\theta^{0}_ {t,i} exactly as the text states, and the implementation follows it. To remove any ambiguity we will typeset it with \leftarrow in the revision.

Reply to W1 & Q1 (global mixing ratio). A single global α\alpha is deliberate: it is the only new hyperparameter HGPS introduces, keeping the rule budget-neutral and directly comparable to the host. The sweep (Table 3) shows a wide plateau - every α∈[0.125,0.875]\alpha \in [0.125, 0.875] beats TIPS by >=1.39 dB, and α=0.5\alpha=0.5 lies within 0.07 dB of the α=0.375\alpha=0.375 peak - so the gain does not hinge on tuning. Per-task/per-layer ratios are promising but introduce new hyperparameters; we list this as future work. The weak gains on VILNC-Indoor3 apply to all adapted variants and are acknowledged as a limitation in Sec. 5.

Reply to Q2 (transformation groups). App. C (Table A.1) tests 5 alternative groupings of the 8 views, including a four-group variant: outputs deviate by <=0.198 dB. On new transform types: the pseudo-label is built in canonical coordinates via inverse transforms, which requires invertible, pixel-aligned geometric views - crops break pixel alignment, and color/noise jitter perturbs the degradation itself (harmful for dehazing/deraining). Reusing the host's dihedral views also keeps the comparison with +DCTTA strictly controlled.

Reply to Q3 (overhead). New Sec. 4.5 / Table 4 (controlled Kodak24 rerun): one-time GG scoring takes 138.5 s (5.77 s/sample); selection is 4.02x DCTTA's; per-iteration adaptation cost is unchanged (time ratio 1.038); end-to-end +17.26% (8.04 s/image); peak memory does not increase (20.63 vs 20.70 GiB). Since GG is a one-time pre-adaptation pass, wall-clock and memory are the deployment-relevant metrics.

Reply to Q4 (visual evidence). Figure 1(c) is redrawn, and App. E provides per-backbone comparisons (Fig. A.1) and a cross-modality MRI example (Fig. A.2). We would like to note that single-image visual comparisons are inherently anecdotal for dense regression: any individual example (such as the bear) can look better or worse by chance, for either method. The method-level evidence is therefore the average over each full test set, where +HGPS matches or improves every column of the DFPIR family (Table 1). Upon publication, we will publicly release the full set of per-image visual results for all benchmarks and backbones.

We appreciate your detailed feedback and hope the revision addresses it.

Reviewer_CSh9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates test-time adaptation for unified image restoration under distribution shifts, focusing on parameter selection under a limited parameter update budget. To address the limitation that Fisher-based importance measures do not explicitly assess the utility of parameter updates, the authors propose HGPS, which estimates update utility using gradient consistency and magnitude across two sets of geometric views under teacher-generated pseudo-supervision. While keeping the number of updated parameters in each parameter tensor unchanged and protecting high-Fisher parameters, HGPS combines high-utility and low-Fisher selection strategies to determine which parameters to update. The method is evaluated on three unified image restoration models: StarIR, PromptIR, and DFPIR.

Critical strengths

The paper argues that low Fisher importance does not necessarily imply high update utility. Accordingly, the authors distinguish parameter protection from update utility and assess the latter using cross-view gradient agreement and magnitude. The motivation is clear and reasonable, leading to a targeted refinement of Fisher-only parameter selection. Experiments on StarIR, PromptIR, and DFPIR demonstrate the effectiveness of the proposed method.

Critical weaknesses
  1. Cross-view gradient agreement is central to HGPS’s assessment of parameter update utility. However, the rationale for the fixed partition A={0,3,4,7} and B={1,2,5,6} is insufficiently explained. Since the grouping affects the teacher pseudo-labels within each group, gradient agreement estimates, and ultimately parameter selection, it may substantially influence performance. Have alternative partitions or different numbers of view groups been evaluated?
  2. The settings and analysis of key hyperparameters could be clarified further. Both F and G are accumulated over S target samples, yet the value of S is not specified. Moreover, the experiments use a fixed protection ratio, leaving unclear whether HGPS’s gains over Fisher-only selection persist across different update budgets. Since this ratio also determines the number of hard-protected parameters, further justification and sensitivity analysis would strengthen the paper.
Questions for authors
  1. Regarding the first weakness, could the authors analyze the extent to which different view transformations and grouping strategies affect the rankings induced by G and the final performance? Also, would directional degradations, such as rain streaks and motion blur, have an impact?
  2. The paper uses cross-view gradient agreement and gradient magnitude to select parameters that are “more worthwhile to update.” This idea is understandable, but the relationship between G and the authors’ claimed target-domain update utility remains largely heuristic. Could additional intermediate analyses be provided to demonstrate its role?
2 comments

We thank the reviewer for the constructive comments. The revised PDF addresses both weaknesses: Appendix C (view-partition sensitivity), Appendix D (update-budget sensitivity), and Section 4.5 (overhead) are new.

Reply to W1 & Q1 (fixed partition; effect of grouping on rankings and performance). We added a dedicated study (App. C, Table A.1): two balanced random partitions, a group-swap control, a rotation-vs-reflection partition, and a four-group generalization of GG (Eq. A.4), evaluated on Rain12 and SDD. GG rankings can change substantially (Spearman 0.19-0.82), yet the adapted outputs are stable: max |dPSNR| = 0.063 dB (Rain12) and 0.198 dB (SDD) vs. the default partition. We also clarify the design rationale: the default groups ∅,t,vh,vht/v,h,vt,ht\\{\varnothing,t,vh,vht\\}/\\{v,h,vt,ht\\} each mix rotations and reflections, so neither group sees a single symmetry type. On directional degradations: reflections do flip rain-streak orientation, and we now state this explicitly in the Limitations; empirically our largest gain is on deraining (Rain100H, +2.35 dB on PromptIR), so the effect is benign in the regimes we evaluate.

Reply to W2 (unspecified S; fixed protection ratio). SS is the full target test set - all test images are used for the one-time scoring (e.g., S=24S=24 on Kodak24); this is now stated in Sec. 4.5 and App. D. On ρ\rho: we inherited ρ=0.6\rho=0.6 from the host pipeline rather than tuning it. The new App. D (Table A.2) repeats the strictly controlled comparison at ρ∈0.5,0.6,0.7,0.8\rho \in \\{0.5,0.6,0.7,0.8\\}: HGPS improves over Fisher-only selection at every budget, by +2.94/+2.69/+1.74/+0.36 dB, with consistent SSIM gains. Notably, HGPS's peak in this diagnostic sits at ρ=0.5\rho=0.5 (24.684 dB), so the inherited default is conservative, not cherry-picked.

Reply to Q2 (the G-utility link remains heuristic). We agree this link deserves scrutiny, and this is exactly what the controlled design is for: within each backbone family, +DCTTA and +HGPS share the pipeline, budget, optimizer, and seed, so performance gaps are attributable to the selection criterion alone. Three pieces of intermediate evidence now triangulate GG's role: (i) the α\alpha-sweep - every guarded hybrid (α>0\alpha>0) beats TIPS by >=1.39 dB; (ii) the Gen-only collapse (-7.5 dB without the Fisher guard), showing GG is useful only as a complement; (iii) App. C separating ranking stability from output stability. A fully mechanistic account of per-element update utility is an interesting direction, and we note it as future work.

We hope these additions resolve your concerns and would appreciate a re-evaluation.

We thank the reviewer for raising this important concern. We agree that the original submission could have articulated the novelty and significance of HGPS more clearly.

Significance. Existing all-in-one image restoration models can degrade substantially when the test-time degradation distribution differs from the training distribution. Instead of retraining a restoration backbone for each new target distribution, HGPS adapts an existing pretrained model using only unlabeled target images, without accessing source data or ground-truth supervision. Under controlled comparisons, HGPS improves the average performance over the adapted baseline across all four backbones in both the three-task and five-task protocols. These results suggest that the significance of HGPS lies in providing a parameter-selection principle for improving existing restoration models under distribution shift, rather than in introducing another restoration backbone.

Novelty. Our contribution lies neither in parameter freezing alone nor in the generic use of gradient information. The key idea is to disentangle two complementary questions that Fisher-only selection conflates: which parameters should be protected, and which of the remaining parameters are worth updating for the target domain. Fisher importance FF addresses the first question, while our element-wise cross-view score GG addresses the second by measuring both update-direction agreement and gradient magnitude under teacher-generated pseudo-supervision.

To the best of our knowledge, HGPS is the first method to use element-wise cross-view gradient agreement as a pre-adaptation parameter-selection criterion for source-free all-in-one image restoration TTA. Unlike online gradient-alignment approaches such as GraTa, which adjust update directions or learning rates during adaptation, HGPS computes GG once before adaptation to determine which parameter elements are eligible to retain optimizer updates. The Fisher hard guard and the high-GG/low-FF allocation then operate within exactly the same per-tensor update budget as the baseline.

The controlled experiments support the principle that “safe to update” is not equivalent to “useful to update.” Replacing Fisher-only selection with HGPS improves average performance under an identical pipeline and update budget. Moreover, the Rain100H ablation shows that removing the Fisher guard causes severe performance degradation in Gen-only adaptation. Together, these results demonstrate that FF and GG play complementary rather than interchangeable roles.

We hope this clarifies that HGPS contributes a distinct parameter-selection principle for restoration TTA, supported by controlled cross-backbone experiments and ablation results.

Comments on the paper

1 comment

We thank all reviewers for their constructive feedback.

Significance. HGPS adapts pretrained all-in-one restoration models using only unlabeled target images, without source data or target ground truth. The two protocols collectively cover five restoration tasks: denoising, dehazing, deraining, deblurring, and low-light enhancement. Under controlled within-backbone comparisons, HGPS improves the mean PSNR and SSIM across target domains for all four backbones in both protocols, demonstrating its effectiveness across multiple restoration tasks and distribution shifts.

Novelty. To the best of our knowledge, HGPS is the first source-free all-in-one image restoration TTA method to use element-wise cross-view gradient agreement for pre-adaptation parameter selection, and to combine it with a Fisher hard guard in a hybrid selection rule. Specifically, the Fisher score FF first identifies stability-critical elements that are excluded from GG-based selection and protected during adaptation. Our score GG then measures cross-view update-direction agreement and gradient magnitude under teacher pseudo-supervision to identify target-relevant updates. HGPS combines high-GG and low-FF elements within exactly the same per-tensor update budget as the baseline, forming a hybrid Fisher- and gradient-guided parameter-selection strategy for TTA.

Controlled improvements under an identical pipeline and update budget, together with the hybrid-ratio and Fisher-guard ablations, support the complementary roles of FF and GG: Fisher protection stabilizes adaptation, while cross-view gradient agreement improves the allocation of the remaining update capacity.

In light of the clarified novelty and the controlled cross-task and cross-backbone evidence, we respectfully invite the reviewers to reconsider their assessments and ratings.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.