acceptodds
Under review as a conference paper at ICLR 2027

GSAIOIR: Exploring Gaussian Fields for All-in-One Image Restoration

Abstract

Gaussian primitives have advanced 3D scene representation and have shown promise for image-plane modelling. We investigate whether explicit Gaussian primitives can serve as a shared output interface for image restoration and downstream prediction. Existing formulations address task-specific objectives, motivating all-in-one restoration as a setting for testing this interface across degradation operators. Two challenges arise. At fixed primitive density, larger images require more Gaussians, motivating crop-wise prediction with limited global context. Additive rendering depends on accumulated weight, whereas normalised blending with nonnegative weights confines a single RGB field to a convex average of its colours. We introduce GSAioIR, a task-conditioned Gaussian restoration framework. A bounded full-image view and crop-box geometry provide aligned context for local primitive prediction. Three parallel fields produce coarse RGB and signed residuals at the middle and fine scales. Independent normalisation removes uniform weight-scale dependence. Residual summation then permits corrections outside the coarse field's colour convex hull, while query-key matching makes fine-scale aggregation content-dependent. Experiments support restoration across five degradation types. Downstream transfer extends the backbone to segmentation, detection, depth, and surface-normal prediction. These results support the applicability of 2D Gaussian representations across restoration and downstream vision tasks.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
all-in-one image restoration, 2D Gaussian representation, Gaussian rasterisation, global-to-local conditioning, moment correction

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_GF4SOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates whether Gaussian representation can serve as a shared output for various image restoration tasks and downstream prediction, and proposes GSAioIR, a Gaussian-based framework for task-conditioned all-in-one image restoration. To handle larger images with reduced memory usage, the method combines global context with cropwise processing. It employs three parallel Gaussian fields with normalised rendering and content-guided weighting (CGW). Experiments demonstrate better performance than Gaussian-based baselines.

Critical strengths
  • The paper extends Gaussian-based image restoration to a task-conditioned all-in-one setting, proposing a unified framework to handle multiple degradation types.
  • The proposed method outperforms Gaussian-based IR baselines across multiple restoration tasks.
Critical weaknesses
  • While the paper aims to investigate whether Gaussian outputs benefit image restoration and downstream tasks, it lacks comparison that isolates the contribution of Gaussian representations from other design choices. Given that the proposed method does not consistently outperform competitive dense-output methods in quantitative comparisons, the motivation for adopting Gaussian representations needs further justification.
  • Several components show only marginal improvements in average performance, making their individual contributions somewhat unclear.
Questions for authors
  • Could you compare the proposed Gaussian-based framework against dense RGB prediction while keeping all other components unchanged?
3 comments

Response to Reviewer GF4S

Dear Reviewer GF4S,

Thank you for your careful review. We added output controls and a CGW capacity ablation. Manuscript revisions are marked in blue.

Weakness 1. Gaussian contribution and motivation

While the paper aims to investigate whether Gaussian outputs benefit image restoration and downstream tasks, it lacks comparison that isolates the contribution of Gaussian representations from other design choices. Given that the proposed method does not consistently outperform competitive dense-output methods in quantitative comparisons, the motivation for adopting Gaussian representations needs further justification.

Response. Gaussian primitives combine explicit spatial supports and attributes in 3D reconstruction. We investigate whether this formulation can support accurate restoration across degradations and extend to semantic prediction and geometric estimation. Spatial supports describe where predictions contribute; task-specific attributes/readouts determine what is predicted.

We added a parameter-matched dense RGB control sharing features and task conditioning. GSAioIR gains 0.2512 dB on the five-task mean and remains competitive with strong dense methods. The control compares complete output systems, including C/M/F, rather than the kernel alone. Restoration and downstream results support the feasibility of this formulation and its potential as a common interface across low- and high-level vision.

Downstream control. With the same frozen Task3 base and attribute projection/readouts, we compare GSAioIR's Gaussian branch with static dense convolution. Four tasks train jointly for 12,000 updates, batch 32 per stream. Tests use CamVid233/Penn–Fudan25/NYUv2 654; each row uses one checkpoint. AP averages IoU 0.50–0.95; AP50 uses 0.50.

MethodmIoU ↑ (%)AP ↑ (%)AP50 ↑ (%)AbsRel ↓Normal error ↓ (°)
GSAioIR42.8840.5385.100.278034.46
Static dense convolution42.4135.7681.770.283034.43

GSAioIR gains 4.77 AP points; normal error is 0.03° higher. Both retain restoration because the base is frozen. Sharing one learned geometry remains a research goal.

Manuscript: Abstract/Introduction, pp. 1–2; §4.4/Table 5, p. 9.

Weakness 2. Individual component contributions

Several components show only marginal improvements in average performance, making their individual contributions somewhat unclear.

Response. These controls retain the Gaussian reconstruction pipeline, with retraining allowing the remaining paths to adapt. CGW only adjusts fine-field aggregation through 648 parameters. Its original gain is 0.0330/0.1091 dB on mean/Rain100L; five-setting averaging makes task-specific gains less visible.

We added Expanded CGW to compare weighting capacity. Each query/key projection uses 80→192→192→80→64→32→4, with 150,888 parameters combined and four-dimensional matching retained.

Task3 variantQ/K parametersMean PSNR ↑Mean SSIM ↑Rain100L PSNR ↑
No CGW031.86040.9109436.7175
Base64831.89340.9110236.8266
Expanded CGW‡150,88831.94270.91536.8834

The added run reports +0.0493 dB over Base, making gains and projection costs explicit. CGW remains a refinement. Original controls use 25 epochs, 111,275 updates, seed 3402; ‡ is the added capacity run. The mean weights SOTS, Rain100L and CBSD68 σ15/25/50 equally.

The original Progressive control omitted preceding-coarse conditioning. Its incorrect configuration is replaced by the updated 31.3074 dB result, not a full serial restoration comparison. Three fields gain 0.0913/0.2693 dB on mean/Rain100L over one Gaussian RGB field. Width 128 slightly exceeds Base at 31.9018 dB, so larger heads are not established as necessary.

Manuscript: §4.5, pp. 9–10; Table 6, p. 10 adds the capacity ablation. §3.4, p. 5 describes aggregation.

Question 1. Matched dense RGB comparison

Could you compare the proposed Gaussian-based framework against dense RGB prediction while keeping all other components unchanged?

Response. We retain local/global features and task conditioning, replacing Gaussian prediction/rendering with a width-336, seven-block RGB decoder. Gaussian/dense RGB have 27,847,797/27,846,723 parameters. PSNR is in dB; ΔPSNR is Gaussian minus dense RGB. The mean weights five tasks equally.

OutputSOTSCBSD68 σ25Rain100LGoProLOLv1Mean
Gaussian28.600031.361936.116528.373123.345029.5593
Dense RGB28.816030.919135.627428.402022.776029.3081
ΔPSNR−0.2160+0.4428+0.4891−0.0289+0.5690+0.2512

Manuscript: §4.1, pp. 7–8; Table 3, p. 8.

Dear Reviewers,

Thank you again for your time and consideration of our submission.

We would like to kindly follow up on our previous responses to the reviewers' comments and concerns. We have made our best efforts to address the raised issues through detailed point-by-point responses and revisions to the manuscript.

We fully understand that this is a busy period and sincerely appreciate the time and effort you have devoted to the review process. If you have had an opportunity to look over our responses, we would be very grateful for any further feedback or guidance. Should any concerns remain or further clarification be helpful, we would be more than happy to provide additional explanations.

Thank you once again for your valuable time and consideration. We sincerely appreciate your efforts in handling our submission.

I appreciate the authors' detailed response and the matched dense RGB comparison I requested. Gaussian outputs appear feasible for all-in-one restoration. However, restoration is the paper's central focus, and even in the matched comparison the benefit is mixed. In my view, the downstream results do not fully explain why Gaussian outputs should be preferred for restoration itself, and I still find the rationale for adopting them insufficient. The ablation differences for several components also remain marginal. I therefore maintain my original score.

Reviewer_cBwaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes GSAIOIR for all-in-one five-task image restoration. The author introduces task-conditioned restoration, global-to-local context, independently normalised RGB and signed residual field, and content-dependent weighting with query–key matching to modulate fine-scale aggregation.

Critical strengths
  1. The all-in-one restoration across five different degradation types is s a challenging yet important task, which GSAIOIR addresses.
  2. GSAIOIR outperform other Gaussian-based method and achieves competitive results with dense-output methods.
  3. The authors conduct comprehensive analysis on the design choices in Gaussian restoration and other transfer tasks such as segmentation and detection.
Critical weaknesses
  1. Currently, I don't fully understand the advantage of Gaussian representation vs. dense pixel output for restoration tasks. I am wondering what is the performance for the matched dense RGB output head using the same system.
  2. GSASR-R is a reduced-capacity adaptation and trained with significantly fewer number of iteration, so it doesn't isolate the architectural comparisons.
  3. The differences between ablated components in Table 5 are small, within few decimals.
Questions for authors

Please see comments above.

3 comments

Response to Reviewer cBwa

Dear Reviewer cBwa,

Thank you for your careful review. We added output controls, an expanded GSASR-R baseline and a CGW capacity ablation. Revisions are marked in blue.

Q1. What is the advantage of Gaussian representation, and how does it compare with a matched dense RGB head?

Gaussian explicitly parameterises spatial supports and task attributes. GSAioIR is competitive with state-of-the-art dense baselines (Tables 1–2); task-specific fields/readouts extend it to semantic prediction and geometric estimation. A common interface across low- and high-level vision remains a research goal.

Same-system restoration control. We retain local/global features and task conditioning, replacing Gaussian prediction/rendering with a width-336, seven-block convolutional RGB decoder. Gaussian/dense RGB have 27,847,797/27,846,723 parameters. This compares complete output systems, including C/M/F. PSNR is in dB; ΔPSNR is Gaussian minus dense RGB. The mean weights the five tasks equally.

OutputSOTSCBSD68 σ25Rain100LGoProLOLv1Mean
Gaussian28.600031.361936.116528.373123.345029.5593
Dense RGB28.816030.919135.627428.402022.776029.3081
ΔPSNR−0.2160+0.4428+0.4891−0.0289+0.5690+0.2512

Gaussian improves three tasks; dense RGB remains higher on dehazing and deblurring.

Frozen-base extension. With the same frozen Task3 base and attribute-projection/readout design, four tasks train jointly for 12,000 updates, batch 32 per stream. Tests use CamVid233/Penn–Fudan25/NYUv2 654; each row uses one checkpoint. AP averages IoU 0.50–0.95; AP50 uses 0.50.

Added branchNew parametersmIoU ↑ (%)AP ↑ (%)AP50 ↑ (%)AbsRel ↓Normal error ↓ (°)
Gaussian5.107M42.8840.5385.100.278034.46
Static dense convolution5.109M42.4135.7681.770.283034.43

Both retain restoration with the base frozen. Gaussian improves mIoU/AP/AP50/AbsRel, with slightly higher normal error.

Manuscript: Table 3, p. 8 and §4.4/Table 5, p. 9.

Q2. Does the reduced capacity and shorter training of GSASR-R prevent an architectural comparison?

We agree that the original adaptation did not isolate architecture. Expanded GSASR-R uses 20.8M parameters, the Task5 pool and global batch 64. Its configured schedule is 50 epochs (157,000 updates), including 15 warmup epochs. We report its best available checkpoint.

ModelSOTSCBSD68 σ25Rain100LGoProLOLv1Mean
Expanded GSASR-R (best)26.779129.326029.098225.705819.216626.0251
GSAioIR28.600031.361936.116528.373123.345029.5593

Q1 supplies the shared-feature, parameter-matched control. The GSASR-R comparison still includes conditioning and optimisation differences.

Manuscript: Expanded GSASR-R replaces the Task5 9M row in Table 2, p. 5; the additional comparison is in §4.1/Table 3, pp. 7–8.

Q3. Why are the component differences in Table 5 so small?

These controls retain the Gaussian restoration pipeline; retraining lets remaining paths adapt. CGW's 648 parameters only reweight fine-field contributions. Five-setting averaging also reduces the visibility of task-specific gains.

We added Expanded CGW to examine capacity. Each query/key projection uses 80→192→192→80→64→32→4 (150,888 parameters combined), retaining four-dimensional matching.

Task3 variantQ/K parametersMean PSNR ↑Mean SSIM ↑Rain100L PSNR ↑
No CGW031.86040.9109436.7175
Base64831.89340.9110236.8266
Expanded CGW‡150,88831.94270.91536.8834

The expanded run reports +0.0493 dB over Base; original CGW gains 0.0330/0.1091 dB on mean/Rain100L over no CGW. The new comparison makes gains and parameter costs explicit. Original controls use 25 epochs, 111,275 updates, seed 3402; ‡ is the additional capacity run. The mean weights SOTS, Rain100L and CBSD68 σ15/25/50 equally.

The original Progressive control omitted preceding-coarse conditioning. We replaced its incorrect configuration: the updated result is 31.3074 dB, with 36.6732 on Rain100L. This remains a field-conditioning control, rather than a full serial restoration cascade.

Three fields gain 0.0913/0.2693 dB on mean/Rain100L over one Gaussian RGB field. Width 128 reaches 31.9018 dB, slightly above Base; larger heads are therefore not established as necessary. CGW is presented as a lightweight refinement.

Manuscript: the original Table 5 is now Table 6, p. 10; §4.5, pp. 9–10 adds Expanded CGW and explains these comparisons.

Dear Reviewers,

Thank you again for your time and consideration of our submission.

We would like to kindly follow up on our previous responses to the reviewers' comments and concerns. We have made our best efforts to address the raised issues through detailed point-by-point responses and revisions to the manuscript.

We fully understand that this is a busy period and sincerely appreciate the time and effort you have devoted to the review process. If you have had an opportunity to look over our responses, we would be very grateful for any further feedback or guidance. Should any concerns remain or further clarification be helpful, we would be more than happy to provide additional explanations.

Thank you once again for your valuable time and consideration. We sincerely appreciate your efforts in handling our submission.

Thank you for the detailed rebuttal. The result of expanded GSASR-R provides a clearer comparison where GSAioIR outperforms it. GSAioIR proves the ability of all-in-one restoration, but the justification and insights of Gaussian representation vs. dense RGB remain vague to me, which should be added in future revisions. Besides, I share the similar concern with Reviewer GF4S on the limited ablation improvements. Therefore, I incline to maintain my score.

Reviewer_eiHKOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper explores utilizing explicit 2D Gaussian representations for all-in-one image restoration. The proposed method, GSAioIR, combines global and three levels of local features to produce coarse, middle, and fine-grained Gaussian fields. Independent normalization and content-guided weighting are introduced to control the output range and produce the restored image. Experiments on five image restoration tasks and four downstream tasks demonstrate the effectiveness of the proposed method.

Critical strengths
  1. Statement and motivation. This paper extends 2D Gaussian to all-in-one image restoration using a shared framework to address various types of degradation. This is a worthwhile research problem an doffers an alternative to conventional pixel-based predicition.
  2. Resonable method design. The proposed framework contains several designs: Global-local context incorporation provides additional cues for local reconstruction; Independent normlization stablized the output brightness; Signed residual field enables the model to apply both positive and negative corrections to the initial output. The proposed modules effectively tackl specific issues in a coherent framework.
  3. Clear and sufficient experiments. The proposed method achieves competitive results across five restoration tasks. The authors conduct corresponding ablation studies. The fine-tuning experiments on downstream tasks demonstrate the potential of the Gaussian output interface to other semantic and geometric prediction tasks.
Critical weaknesses
  1. The retionale for the RGB-residual division is nor sufficiently justified. Although this paper explains the motivation of introducing signed residuals: citing constraints related to the color convex hull, it does not demonstrate why this design is necessary. There is no explanation why and how the coarse field is tasked with basic RGB and the middle and fine fields predict residuals, rather than a different allocation. In Figure 2 and Figure 16, the coarse RGB field appears almost gray, while the fine residual field captures the most image structure and color. In Table 19, the authors also conclude that "Find residual contain much more spatial cariation". This suggests that the fine-level residual is actually carrying the full RGB reconstruction.
  2. Limited performance gain. As shown in the ablation study, the performance obtained from CGW and the proposed multi-field design is limited. Furthermore, given that the entire pipeline is quite complex and contains many modules, the results in Table 1 and Tbale 2 do not demonstrate consistent advantage over state-of-the-art dense prediction approaches.
  3. In Equation 1, the input to EgE^g include XgX^g and tt. However, in Figure 2, tt is not provided as input to EgE^g.
Questions for authors

Please refer to the weaknesses.

2 comments

Response to Reviewer eiHK

Dear Reviewer eiHK,

Thank you for your careful review and for recognising the value of Gaussian outputs for all-in-one restoration. We have added a CGW capacity ablation and clarified the manuscript. Revisions are marked in blue.

Q1. Why assign RGB to C and residuals to M/F? Does the nearly gray C indicate that F carries most reconstruction?

Field allocation. C/M/F denote Gaussian support scales and prediction grids, rather than prescribed frequency bands. They are predicted in parallel, rendered independently, and summed. C supplies a bounded RGB reference; signed M/F allow corrections beyond its colour convex hull. A residual can therefore carry substantial colour and spatial variation. This is a reconstruction parameterisation, rather than a claim that each field must recover a fixed type of image content.

Evidence concerning C. In the inspected Task5 E40 checkpoint, C has means of 0.4978/0.5001/0.5002 on SOTS/Rain100L/CBSD68 σ25. Fixed-weight interventions on 668 native 160×160 centre crops give the following ΔPSNR relative to the unchanged checkpoint, without retraining.

InterventionSOTSRain100LCBSD68 σ25
Replace C with 0.5; retain M/F+0.0588−0.0091−0.0014
Remove M; retain C/F−0.2574−2.3315−0.5332

We agree that F carries most visible spatial variation here. Replacing C with 0.5 has little effect, whereas removing M substantially hurts deraining. We now describe C as a bounded reference and M/F as complementary additive corrections. These diagnostics do not establish the optimality of the allocation.

Manuscript: §4.3, pp. 8–9; rendering formulation in §3.5, p. 6.

Q2. Are the small gains from CGW and multiple fields justified by the added complexity? Why are the results not consistently better than dense approaches?

Component contribution and capacity. CGW only adjusts fine-field aggregation. Its original two 80→4 query/key projections add 648 parameters, with their positive inner product modulating Gaussian weights before normalised aggregation. To examine projection capacity, we added Expanded CGW: each projection uses 80→192→192→80→64→32→4, with 150,888 parameters combined and the same four-dimensional matching output.

Task3 variantQ/K parametersMean PSNR ↑Rain100L PSNR ↑
No CGW031.860436.7175
Base64831.893436.8266
Expanded CGW‡150,88831.942736.8834

The original CGW gains 0.0330 dB on the mean and 0.1091 dB on Rain100L. Expanded CGW reports a further 0.0493 dB mean gain over Base, with mean SSIM 0.915. The capacity ablation adds a direct comparison between omitted, lightweight, and larger weighting branches; the observed benefits remain incremental. C/M/F adds 427,784 parameters over one Gaussian RGB field and improves mean/Rain100L PSNR by 0.0913/0.2693 dB.

Original controls use 25 epochs and seed 3402; ‡ is the additional capacity run. The mean equally weights SOTS, Rain100L and CBSD68 σ15/25/50. We present CGW as a lightweight aggregation refinement.

Comparison with dense approaches. Tables 1–2 contain strong, established dense restoration models. GSAioIR instead predicts explicit Gaussian supports and attributes before rendering pixels. The geometric and aggregation constraints introduce a different reconstruction parameterisation whose benefits vary with degradation. Competitive results establish its feasibility, while the strongest dense methods remain better on some settings.

In the same-system output control, shared features and task conditioning give approximately 27.85M parameters each. Gaussian reaches 29.5593 dB, versus 29.3081 dB for dense RGB, a 0.2512 dB five-task mean gain. Gaussian is higher on denoising, deraining and low light; dense RGB is higher on dehazing and deblurring. This compares complete output systems, including C/M/F. Task-specific downstream fields further demonstrate extension to semantic prediction and geometric estimation; a common interface across low- and high-level vision remains the research direction.

Manuscript: §4.5, pp. 9–10 and Table 6, p. 10 add Expanded CGW; §3.4, p. 5 clarifies fine-field aggregation. The dense output control is in Table 3, p. 8.

Q3. Why does Figure 2 omit a task-conditioning input that appears in Equation 1?

Thank you for identifying this ambiguity. Task condition tt enters both local and global encoders, with task embeddings learned internally. The revised Figure 2(a) and caption, p. 3, make these paths and HGPD conditioning explicit, consistent with Equation 1. This corrects the presentation of the implemented model.

Dear Reviewers,

Thank you again for your time and consideration of our submission.

We would like to kindly follow up on our previous responses to the reviewers' comments and concerns. We have made our best efforts to address the raised issues through detailed point-by-point responses and revisions to the manuscript.

We fully understand that this is a busy period and sincerely appreciate the time and effort you have devoted to the review process. If you have had an opportunity to look over our responses, we would be very grateful for any further feedback or guidance. Should any concerns remain or further clarification be helpful, we would be more than happy to provide additional explanations.

Thank you once again for your valuable time and consideration. We sincerely appreciate your efforts in handling our submission.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.