acceptodds
Under review as a conference paper at ICLR 2027

PSG: Preserving Activation Structure for Low-Bit Image Restoration

Abstract

Post-training quantization (PTQ) is a practical route to deploying image restoration models, yet restoration fidelity degrades sharply at ultra-low bit widths. A key challenge is channel-wise outliers, which are particularly pronounced in Mamba-based models and substantially inflate the activation range. Suppressing these extremes can tighten the range, but is not a reliable remedy, as even limited removal can severely degrade reconstruction quality. We show that these outliers are not random spikes, but exhibit strong persistence: their dominant direction remains stable across inputs, and even within each input, the response along this direction is effectively shared across tokens. We term this shared directional component the pedestal. Crucially, the pedestal not only inflates the range, but also defines a direction highly sensitive to quantization error, making it a structure that should be preserved rather than suppressed. Building on this, we propose PSG, a PTQ framework that combines Pedestal Shifting and GRILL. Pedestal Shifting keeps the pedestal outside the quantizer while reducing the activation range, and GRILL calibrates quantization bounds for the residual activations and weights through closed-form updates. Across diverse image restoration tasks and architectures, PSG consistently outperforms existing methods, delivering up to dB higher PSNR at 2 bits, smaller models, and fewer bit-operations, while deploying as standard TensorRT engines with no additional inference overhead. Our code is available at: https://anonymous.4open.science/r/PSG-AD60.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Post-training Quantization, Image Restoration

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_WKAVOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses how to improve restoration fidelity with ultra-low bit widths in post-training quantization (PTQ). Through controlled analysis, it observed a key characteristic that some special outliers, named as pedestal, define a direction highly sensitive to quantization error, making it a meaningful structure. Practical strategies, including Pedestal Shifting for pedestal isolation and Grill for quantization bounds calibration are proposed. Experiments are extensive, showing consistent and nontrivial improvement over existing methods.

Critical strengths
  1. Regarding pedestal, the paper provides compelling empirical evidence that a persistent dominant activation component is approximately constant across tokens yet highly sensitive to quantization-induced distortion.

  2. The PSG strategies, including Pedestal Shifting and Grill, are well grounded and effective.

  3. Experiments, both quantitative and qualitative, are extensive and demonstrate substantial improvements, particularly at ultra-low bit widths.

Critical weaknesses
  1. The mechanistic evidence described in Sec.3 and Figure 1 is uneven across architectures and datasets. Although Sec. 3.2 reports directional perturbation results averaged across Mamba-based restorers, the detailed token-wise constancy experiments in Appendix B are restricted to MambaIR ×2 on Set5, and the actual quantization-error decomposition is reported only for MambaIR. Extending these analyses to other backbones and datasets would strengthen the generality of the proposed explanation.

  2. The baseline results exhibit an unexplained non-monotonic trend across bit widths. In Table 8, SPR²Q at W3A3 outperforms its W4A4 counterpart on all five benchmarks for MambaIR ×2, including gains of 0.557 dB on Urban100 and 0.715 dB on Manga109. This does not necessarily indicate an error, but warrants clarification of the baseline reproduction protocol.

Questions for authors
  1. Could the authors extend the token-wise constancy and quantization-error decomposition analyses to additional backbones and datasets?

  2. Could the authors explain the non-monotonic SPR²Q results in Table 8?

Given satisfactory clarifications of the two points above, the reviewer is willing to raise the score further.

3 comments
AuthorsOpenReview ↗

Response to Reviewer WKAV (1/2): Constancy and error decomposition beyond MambaIR (W1, Q1)

We thank the reviewer for describing the pedestal as "a meaningful structure" supported by "compelling empirical evidence", and Pedestal Shifting and GRILL as "well grounded and effective". This is the chain the paper is built on: the analysis identifies a structure that should be preserved, and the method is designed around preserving it. As requested, we extended the two analyses behind this structure, token-wise constancy and quantization-error decomposition, to more backbones, tasks, and test sets.

Summary. Together, the two analyses now cover four backbones, including the Transformer, three tasks, and seven test sets. The findings of Sec. 3 carry over. The response along e1e_1 stays functionally constant at the gated sites, and the e1e_1-aligned error stays a small part of the distortion with a disproportionate effect on fidelity.

 

Token-wise constancy (Table R8).

We repeated the joint replacement of Appendix B on the full-precision models, where the token-dependent coefficient ctc_t along e1e_1 is replaced by a single constant (Eq. 2). The gate uses the same threshold as in the paper (τ=0.1\tau=0.1).

  • Gated sites. The replacement changes PSNR by up to 0.058 dB, on SwinIR as well as on the Mamba backbones. The response along e1e_1 is functionally constant where the gate applies.
  • All sites. The same replacement costs more, up to 4.79 dB. The sites where one constant is not enough are those the gate rejects. This is most visible on MambaIRv2-light and on SwinIR denoising, where the rejected sites vary strongly across tokens.
BackboneTaskTest setGated sitesAll sites
MambaIRSR ×2Set5−0.014−0.180
B100−0.052−0.147
Denoising σ=25BSD68−0.012−0.144
McMaster−0.020−0.239
JPEG CAR q=10LIVE1−0.014−0.317
MaIRSR ×2Set5−0.040−0.160
B100−0.050−0.141
Denoising σ=25BSD68−0.004−0.089
McMaster−0.010−0.147
MambaIRv2-lightSR ×3Set5−0.058−1.053
B100−0.047−0.954
SwinIRSR ×2Set5−0.027−0.126
B100−0.042−0.179
Denoising σ=25BSD68−0.021−3.913
McMaster−0.027−4.785
JPEG CAR q=10LIVE1−0.022−0.381

Table R8. PSNR change (dB) under joint replacement of ctc_t by a single constant. The first row is the setting of Table 7(a). MambaIRv2-light is released for SR only.

 

Quantization-error decomposition (Table R9).

We repeated the decomposition of Sec. 3.2 on the three other backbones. The activation quantization error is split into its component along e1e_1 (ϵ∥\epsilon_\parallel) and the rest (ϵ⊥\epsilon_\perp), as in Eq. 3, and each is removed in turn.

  • ϵ∥\epsilon_\parallel remains a small part of the distortion, below 9% in Table R9.
  • Its effect on fidelity is disproportionate. Removing it recovers up to 65.6% of the lost fidelity. Removing ϵ⊥\epsilon_\perp, which accounts for over 90% of the distortion, recovers less in Table R9. The disproportion holds on SwinIR, the Transformer backbone, as well as on the Mamba backbones.
  • On MaIR, removing ϵ⊥\epsilon_\perp lowers fidelity. The e1e_1-aligned error alone is there at least as harmful as the full error.
BackboneTaskTest setϵ∥\epsilon_\parallel share of distortionRecovered, ϵ∥\epsilon_\parallel removedRecovered, ϵ⊥\epsilon_\perp removed
MaIRDenoising σ=15BSD684.71%+14.8%−13.1%
McMaster4.41%+12.1%−9.6%
Kodak244.79%+13.8%−10.4%
MambaIRv2-lightSR ×2Set57.18%+22.0%+3.0%
B1008.55%+25.4%−0.6%
Urban1008.04%+11.2%+0.3%
SwinIRSR ×2Set53.09%+65.6%+28.5%
B1004.68%+57.3%+25.1%
Urban1003.45%+36.2%+19.7%

Table R9. Recovered: share of the lost fidelity restored when one error component is removed. MambaIR is reported in Sec. 3.2.

 

Revision (done). Both analyses are in Appendix B.1–B.2; Sec. 3.1 and 3.2 refer to them.

AuthorsOpenReview ↗

Response to Reviewer WKAV (2/2): Non-monotonic SPR²Q results in Table 8 (W2, Q2)

Summary. We verified our SPR²Q setup by reproducing the reported ×2 results on MambaIRv2-light under the original protocol (Table R5). On MambaIR ×2, the W4A4 model degrades during distillation under the released learning rate. Reducing only the learning rates by 10× restores W4A4 above W3A3 on all five benchmarks, while PSG remains ahead by at least 0.66 dB. The same reduction does not improve W3A3 or W2A2.

 

Protocol: official code, released settings.

For MambaIR, we retain the optimization settings released for SPR²Q's MambaIRv2-light backbone: Adam, learning rate 10⁻², batch 8, 12,000 + 500 iterations, at every bit width. The released configuration keeps the rate constant, whereas the paper states cosine annealing. We follow the release, as for every baseline. Only two things differ from it, both shared by all methods: the quantization scope (Sec. 5.1) and the use of the final checkpoint (response to Reviewer 2JMm). With its original scope and checkpoint selection, the same code reproduces the reported MambaIRv2-light results (Table R5).

SPR²Q's reported evaluations and released configurations cover lightweight backbones. On MambaIRv2-light, SPR²Q is the strongest baseline at W4A4 at every scale of our tables, with W4A4 above W3A3 (Tables 2 and 9). MambaIR is substantially larger (82.0 MB versus 3.4 MB, Table 4(a)), and no configuration is released for it. We therefore transferred the released settings without backbone-specific tuning.

 

W4A4 degradation on MambaIR ×2 is resolved by reducing the learning rate.

The official code logs Set5 PSNR during distillation. The W4A4 model drops 0.49 dB below its calibrated start within the first 1,000 iterations, never fully recovers, and ends 0.19 dB below it.

Under the same rate, more weights move to a different quantization level at W4A4 (Table R10). SPR²Q trains low-rank rectifiers (1.66M parameters on MambaIR) jointly with the quantizer bounds. Their weight increments are added to the weights before quantization. The trained increments have a similar size relative to the weights at both bit widths. On the finer 4-bit grid, they move 24.2% of the weights to a different level, against 10.9% at W3A3. During distillation, Set5 PSNR rises by 1.16 dB at W3A3 and falls by 0.19 dB at W4A4. These observations led us to test whether a rate that suits W3A3 is too large at W4A4.

SPR²Q on MambaIR ×2, released rateW3A3W4A4
Rectifier weight increment, relative to weight norm8.5%8.2%
Weights moved to a different quantization level10.9%24.2%
Change in Set5 PSNR during distillation (dB)+1.16−0.19

Table R10. Statistics of the final checkpoints.

Reducing the learning rates by 10× restores the expected ordering (Table R11). We re-ran W4A4 with the learning rates divided by 10 and nothing else changed. The final checkpoint now improves on its calibrated start, and W4A4 is above W3A3 on all five benchmarks. PSG leads this row by at least 0.66 dB and every other baseline of this setting by at least 0.32 dB, so the conclusion of Table 8 does not depend on it.

MambaIR ×2, PSNR (dB)Set5Set14B100Urban100Manga109
SPR²Q W3A3, released rate36.96132.65931.44430.01536.657
SPR²Q W4A4, released rate36.79432.51531.31729.45835.942
SPR²Q W4A4, rate ÷ 1037.21033.08331.70931.04237.507
PSG W4A438.20234.10432.37033.22539.191

Table R11. Bold: the re-run with the smaller rate.

The same reduction does not improve W3A3 or W2A2 (Table R12). The smaller rate lowers W3A3 by up to 1.39 dB and W2A2 by less than 0.05 dB. We therefore retain the originally reported W3A3 and W2A2 results.

×2, change with rate ÷ 10 (dB)Set5Set14B100Urban100Manga109
W4A4+0.416+0.568+0.392+1.584+1.565
W3A3−0.814−0.432−0.361−0.887−1.393
W2A2−0.045−0.018−0.013−0.009−0.026

Table R12. SPR²Q on MambaIR ×2.

 

PSG uses one setting everywhere.

It trains only the bounds, with one setting for every backbone, task, and bit width (App. C).

 

Revision (done). Table 12 (Table 8 of the submission) now reports the reduced-rate SPR²Q W4A4 row with SSIM, marked †, and Appendix C states the setting and the reason. We also ran cosine annealing: the W4A4 model degraded again, so it is not reported.

AuthorsOpenReview ↗

Revised PDF uploaded.

We have uploaded the revised paper (changes in blue) and updated the "Revision" paragraphs of our two responses above with the locations. The extended analyses are in Appendix B.1 and B.2 (Tables 8 and 9), and the updated SPR²Q row is in Table 12 (Table 8 of the submission). We hope the revision addresses your concerns and would appreciate it if you could take a look.

Reviewer_2JMmOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

Post-training quantization of image restoration models degrades sharply below 4 bits, largely because of channel-wise activation outliers that are pronounced in Mamba-based restorers and cannot simply be clipped without a large loss in fidelity. The authors show that these outliers share a dominant direction that is stable across inputs, effectively constant across tokens, and highly sensitive to quantization error, call its shared component the pedestal, and propose PSG, which combines Pedestal Shifting, which removes the pedestal before quantization and restores it in full precision through offline folding, with GRILL, which calibrates uniform quantization bounds for the residual activations and weights through closed-form grid-constrained Lloyd-Max updates, followed by a distillation step that refines only the bounds.

Critical strengths

PSG achieves large and consistent gains at 2 and 3 bits across three Mamba backbones, SwinIR, and three restoration tasks, reaching higher PSNR than the strongest baseline in 2-bit denoising.

The method is motivated by a diagnostic analysis showing that the error component along the dominant direction makes up only a tiny fraction of the total activation distortion, yet removing it recovers a large part of the lost restoration quality.

The pedestal shift and its compensation are folded into adjacent static operators, and the appendix verifies that the folded network produces practically unchanged outputs while also describing an exact variant with negligible cost.

Critical weaknesses

The argument that the pedestal should be preserved rather than suppressed rests only on a comparison against clipping, whereas the actual competing approach, channel-wise shifting as in OS+, also removes the offset before quantization and restores it in full precision. As a result, the sensitivity analysis justifies not clipping the outliers but does not explain why a single-direction pedestal should outperform per-channel shifting.

Pedestal Shifting is essentially a gated version of the channel-wise shift in OS+ that is constrained to a single direction, and the only reason given for its advantage is more stable estimation rather than the persistence or sensitivity findings that the paper emphasizes. In the component ablation, Pedestal Shifting on its own yields a clearly smaller improvement than GRILL, so the main conceptual contribution appears incremental relative to OS+ and accounts for only a minor share of the overall gain.

Evidence for the pedestal structure comes almost entirely from Mamba models, mostly from a single configuration, and even within these models the constancy gate admits fewer than half of the candidate sites and only a small fraction in the lightweight backbone. The Transformer experiments report only final accuracy without showing that a pedestal exists or how many sites are shifted, so the claim that the structure is general to image restoration is not established and the gains outside Mamba may come from GRILL alone.

Several key design choices are heuristic, including a fixed gate threshold on the coefficient of variation that appears to be tuned on a test benchmark, the size of the calibration set, the midrange estimator, and the restriction to a single direction. Since shifting at every site performs worse than not shifting at all, the benefit of the method depends on these empirically chosen settings, yet no principled criterion or cross-model validation is provided for them.

Questions for authors

How were the baselines configured in terms of calibration data, iteration budgets, and hyperparameters, and can their originally reported results be reproduced in a shared setting? Please also explain why some baselines produce nearly identical numbers at the lowest bit width and why OS+ becomes the strongest baseline in several low-bit settings.

With GRILL and distillation held fixed, how does replacing the pedestal with an OS+ style per-channel shift, or removing the shift entirely, compare across bit widths and backbones, and how much do the baselines gain when Pedestal Shifting is added to them?

7 comments
AuthorsOpenReview ↗

Response to Reviewer 2JMm (1/3): Pedestal Shifting and channel-wise shifting (W1, W2, Q2)

We thank the reviewer for the detailed review and for recognizing the gains at ultra-low bit widths and the diagnostic analysis behind the method. Below, "L" denotes line numbers of the original submission.

Summary. PSG is Pedestal Shifting and GRILL together, and both are contributions of the paper. The design of Pedestal Shifting follows from the three findings of Sec. 3, not from estimation stability alone. The controlled comparison the reviewer requested supports this design: under the same GRILL calibration and 4,000 iterations of bound optimization, Pedestal Shifting keeps a consistent margin over both an OS+-style per-channel shift and no shift.

 

What the clipping comparison shows, and where OS+ stands (W1). The clipping analysis (L56–59, App. A) makes one point: the extreme channels cannot simply be removed, because restoration depends on what they carry. Shifting respects this, and we present channel-wise shifting such as OS+ as an established tool (L69–71). The paper then compares against it directly, in Table 1(a) and with OS+ as a baseline in all result tables (L416–418).

 

The design follows from the findings of Sec. 3 (W1, W2).

Stable estimation is not the only reason given for the advantage of Pedestal Shifting. The paper constrains the shift to the pedestal because of the persistence and sensitivity findings (L64–75, L283–284):

  • Persistence across inputs: one vector for all inputs. The dominant direction is shared across inputs (L161–165), so the vector can be fixed offline from a few images. Per-channel extremes are far less consistent, and the OS+-style vector varies across calibration sets (Table 1(a)).
  • Token-wise constancy: one vector for all tokens. The response along e1e_1 is functionally constant across tokens (L167–179), so subtracting a single vector removes it from all tokens at once, and the fold restores it in full precision. A component that varies across tokens could not be removed by a fixed offset. The gate verifies this property at each site (Eq. 7).
  • Sensitivity: why the pedestal is preserved. Errors along e1e_1 affect fidelity disproportionately (Sec. 3.2), which also holds on the other backbones (Table R9, response to Reviewer WKAV). The pedestal therefore bypasses the quantizer, with the magnitude that minimizes the residual range along e1e_1 (Eq. 6).

 

Controlled comparison (Q2, W2).

GRILL and distillation are fixed and only the shift changes: no shifting, OS+-style per-channel shifting, or Pedestal Shifting. The ablation of the paper uses W3A3 MambaIR (Table 5(a)), so we extend it here to three other backbones and bit widths, including the lightweight model and the Transformer. Table R3 gives the means, and a supplementary comment gives all per-benchmark results.

BackboneBitNo shiftOS+-stylePedestal
MaIR ×2W2A233.997 / 0.930434.126 / 0.931134.367 / 0.9332
W3A334.962 / 0.937335.017 / 0.937835.092 / 0.9386
MambaIRv2-light ×3W2A228.712 / 0.831828.690 / 0.831228.924 / 0.8364
W3A329.412 / 0.843329.580 / 0.847729.739 / 0.8523
W4A430.425 / 0.862130.463 / 0.864130.552 / 0.8661
SwinIR ×3W2A230.128 / 0.859630.383 / 0.864430.494 / 0.8662
W3A331.026 / 0.874131.198 / 0.877831.236 / 0.8787

Table R3. PSNR (dB) / SSIM, mean over the five SR benchmarks.

  • Pedestal Shifting is above both alternatives in every setting, in PSNR and SSIM.
  • Over the OS+-style shift, the margin grows toward lower bit widths. It is up to 0.16 dB at W3A3 and up to 0.24 dB at W2A2. Over no shift, the gain is up to 0.37 dB.
  • An unconstrained per-channel shift is not always a gain. On MambaIRv2-light at W2A2 it is below no shift.

 

Margin over OS+ and share of the gain (W2).

  • The margin over the OS+-style shift is consistent. In Table R3, the same GRILL calibration and 4,000 iterations of bound optimization fit the bounds to each shift. The margin is small, but it holds in every setting. Bound fitting does not fully make up for the choice of shift, which is what the analysis of Sec. 3 determines.
  • The share depends on the bit width and model. GRILL is also part of PSG (L87–90), so Table 5(a) divides the gain between two of our own components. It is measured at 3 bits, where the conventional calibration baseline does not collapse (L452–455). At 2 bits on MambaIR, the larger part comes from Pedestal Shifting (part 2/3, Table R6).

 

Revision (done). The opening of Sec. 4.1 now states how the three findings of Sec. 3 lead to the design, and the controlled comparison is in Appendix B.3 (Table 10).

AuthorsOpenReview ↗

Response to Reviewer 2JMm (2/3): Baseline configuration and reproduction (Q1)

Summary. All baselines run their official code with the released settings. We made three adaptations: one quantization scope and the final checkpoint for every method, and an SR calibration for the two baselines released for other domains. Under the original protocol, our implementation reproduces the numbers reported for SPR²Q. Near-identical numbers at W2A2 come from a shared initialization, and the lead of OS+ on MambaIR comes from its shift.

 

Configuration (Table R4).

All methods draw calibration and distillation data from the training set of each task (App. C).

MethodCalib. imagesOptimization (released setting)
OS+32 (official: 128)training-free shift and scale search
PTQ4VM32 (official: 256)100 epochs, AdamW, lr 5e-4 (smoothing scales: 1e-2)
2DQuant324,000 iterations, batch 16, Adam, lr 1e-2
HarmoQ325,000 iterations, batch 8, Adam, lr 1e-2, step decay
SPR²Q3212,000 + 500 iterations, batch 8, Adam, lr 1e-2 (SwinIR: 4,000 + 500, batch 16)
PSG8gradient-free GRILL, then 4,000 iterations, batch 16, Adam, lr 1e-2

Table R4. Baseline rows follow the released code.

Where a release differs from its paper in iterations, batch size, or learning-rate schedule, we follow the release.

 

What we adapted.

Everything else follows the releases.

  • One quantization scope. No baseline defines a scope that covers our whole benchmark: none of the released scopes covers MambaIR or MaIR. We therefore use one rule for all four backbones (L407–410) and apply it to every method, PSG included, so that each row is the same quantized network.
  • Final checkpoint. The released SR settings validate on Set5, a test benchmark. To keep test data out of checkpoint selection, we report the final checkpoint for every method, including ours.
  • Calibration of OS+ and PTQ4VM. Both were released for other domains, so we give them the 32 calibration images of the SR baselines. With its official min-max calibration, OS+ collapses on restoration models at low bit widths, so we also give it the conventional bound search (DOBI), which raises its results.

 

Reproduction of the reported numbers (Table R5). MambaIRv2-light is the one backbone we share with a baseline paper (SPR²Q). With its official scope and checkpoint selection, our implementation reproduces the reported ×2 results within 0.06 dB in PSNR and 0.0004 in SSIM. The numbers in our tables differ from these through the first two adaptations.

BitSet5Set14B100Urban100Manga109
Reported437.72 / 0.958933.27 / 0.915631.94 / 0.896431.53 / 0.922338.03 / 0.9754
Ours437.68 / 0.958833.33 / 0.916031.94 / 0.896231.51 / 0.922238.08 / 0.9754
Reported235.97 / 0.949531.98 / 0.902030.95 / 0.882728.55 / 0.881934.39 / 0.9599
Ours235.97 / 0.949931.94 / 0.902430.94 / 0.882728.57 / 0.882334.36 / 0.9603

Table R5. SPR²Q on MambaIRv2-light ×2, PSNR (dB) / SSIM. Reported: Table 1 of the SPR²Q paper.

 

Near-identical numbers and the lead of OS+ at W2A2 (Table R6).

Both concern SR on the Mamba backbones at W2A2, where the bound initialization and the shift make the difference.

  • Near-identical numbers: a shared initialization. SPR²Q and HarmoQ add weight-side corrections to the framework of 2DQuant (L291–294), and in their released code all three start the bounds from the same initialization (DOBI). On the Mamba backbones they stay within 0.3 dB in mean PSNR at W2A2 in every SR table, against up to 1.0 dB on SwinIR (Table 11).
  • The calibrated bounds matter even after distillation. With no shift, a GRILL start is up to 2.8 dB above the DOBI start in mean PSNR after the same distillation (MaIR ×2: "No shift" in Table R3 against 2DQuant in Table 9).
  • The lead of OS+: its shift. OS+ is the only baseline that removes channel-wise offsets before quantization, and it leads the baselines on MambaIR at W2A2 at all three scales (Table 8). At ×2, PSG without the shift is at the level of the shift-free baselines and below OS+, and the pedestal adds 2.5 dB on average and up to 4.7 dB on Manga109.
MambaIR ×2, W2A2ShiftSet5Set14B100Urban100Manga109
HarmoQnone34.97731.38730.47727.90332.665
PSGnone34.96331.34730.46027.84732.561
OS+per-channel35.27031.62230.62428.27733.525
PSGpedestal37.33332.92531.67430.58537.304

Table R6. PSNR (dB). Baseline rows are from Table 8.

 

Revision (done). Appendix C now has Table R4 (Table 11), the three adaptations, and the reproduction.

AuthorsOpenReview ↗

Response to Reviewer 2JMm (3/3): Generality beyond Mamba and design choices (W3, W4)

Summary. The pedestal structure holds beyond Mamba, and the gain there is not from GRILL alone. On SwinIR, the constancy and sensitivity findings of Sec. 3 carry over, and the shift adds up to 0.37 dB with GRILL fixed. The gate is meant to pass only the sites where one fixed vector can represent the response. The single direction and the midrange are derived, the calibration size has limited effect, and the one empirical value, the gate threshold, is confirmed by cross-model validation on held-out images.

 

Evidence beyond one configuration (W3).

The constancy and error-decomposition analyses of Sec. 3 now cover four backbones, three tasks, and seven test sets. The full tables are in our response to Reviewer WKAV (Tables R8 and R9). On SwinIR, replacing the response along e1e_1 by a constant changes PSNR by up to 0.042 dB at the gated sites. The e1e_1-aligned error is below 5% of the distortion there, yet removing it recovers up to 65.6% of the lost fidelity. The pedestal is also stable across calibration sets on SwinIR, with a cosine similarity of 0.998 (Table R1, response to Reviewer D2Cy).

 

Shifted sites, and why the gate passes only part of them (W3).

On SwinIR, the gate passes 27 of the 72 candidate sites on average over tasks, and up to 44. This is a larger share than on the lightweight Mamba backbone (44 of 160, App. C). Shifting only these sites is intended.

  • Where the shift is applied matters. As the reviewer notes, shifting every site can be worse than not shifting at all (Table 1(b)). A fixed vector represents the response only where it is constant across tokens (App. B: −0.180 dB at all sites against −0.014 dB at the gated sites, and likewise on SwinIR in Table R8). The gate selects these sites, and the others are left to GRILL, as stated in our limitations (L476–478).
  • The gated sites are enough for a gain that GRILL alone does not give. With GRILL and distillation fixed, Pedestal Shifting raises SwinIR ×3 by 0.37 dB at W2A2 and 0.21 dB at W3A3. Even on the lightweight backbone, where 44 of 160 sites pass, it adds up to 0.33 dB (Table R3).

 

Single direction and midrange are derived, not heuristic, and the calibration size has limited effect (W4).

  • Single direction. e1e_1 is the only sensitive direction whose response is constant across tokens. The median CV is 0.06 for e1e_1 and above 64 for e2e_2 to e4e_4, and extending the pedestal to them degrades quality (L188–191, Table 7(b)).
  • Midrange. It is the closed-form minimizer of the residual range along e1e_1 (Eq. 6, L245–257), not a tuned estimator. The choice follows from the analysis of Sec. 3: it preserves the sensitive component by leaving the smallest possible range of it to the quantizer.
  • Calibration size. Performance is saturated from 2 images, with a change of 0.06 dB from 2 to 64 (Table 5(b)).

 

Gate threshold: one empirical value, validated on held-out images across models (Table R7).

The gate criterion follows from token-wise constancy: the shift is valid only where one constant represents the response (L263–269, Eq. 7). A CV of at most 0.1 means that at least 99% of the energy of the response along e1e_1 is constant across tokens. The value itself is the one empirical setting among these choices, a single τ=0.1\tau=0.1 (L279–280, App. C). For consistency with the analyses of App. A and App. B, the paper reports its sensitivity on Set5. Table R7 adds the same sweep on 20 DIV2K validation images outside every calibration set, for two further backbones.

SR ×2, W2A2τ=0\tau=00.050.100.15∞\infty
SwinIR34.47834.81735.03734.73934.752
MaIR34.08834.56335.19235.09934.980

Table R7. PSNR (dB) on the held-out images, calibration only. τ=0\tau=0: no shift. τ=∞\tau=\infty: every site.

The same trend holds on held-out images and across backbones. As on MambaIR in Table 1(b), 0.1 is the best value on both backbones, above no shift and above shifting every site. Every tested threshold from 0.05 to 0.15 is also above no shift, so the gain does not rest on the exact value.

 

Revision (done). Appendix B.1 and B.2 (Tables 8 and 9) contain the extended analyses, and Appendix C gives the SwinIR gate statistics. The held-out sweep (Table R7) is now Table 1(b) of the main text, in place of the Set5 sweep. Sec. 4.1 refers to it and states that the gate is selective by design.

Reviewer_D2CyOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PSG, combining Pedestal Shifting and GRILL for low-bit image restoration. The authors identify a persistent, quantization-sensitive dominant direction in activations, preserve its contribution in full precision, and optimize residual quantization grids through alternating closed-form updates.

Critical strengths

Interesting observation that activation outliers exhibit a persistent dominant direction with high sensitivity to perturbation. Strong performance across restoration tasks, particularly at extremely low bit widths.

Critical weaknesses

Limited evidence of the benefit over conventional per-channel shifting. Since the dominant direction is extracted from an uncentered second-moment matrix, it may largely reflect the activation mean at sites satisfying the CV gate. The proposed pedestal is essentially a fixed channel-wise shift computed from the calibration set, yet Table 1(a) compares only against per-channel midrange shifting, without a per-channel mean-shifting baseline.

PSG’s Pedestal Shifting uses essentially the same shift-and-fold reparameterization as DFQ, including full-precision bias compensation and a related coefficient-of-variation-based criterion for selecting channels. The main differences, which is estimating a rank-1 pedestal along the dominant direction, appear to be incremental adaptations. A DFQ-style baseline using empirical per-channel shifts, CV-based gating, and bias correction is needed to establish the specific benefit of the proposed pedestal formulation.

Questions for authors
  • Could the authors provide cosine similarity, relative l2 distance, and downstream performance when using activation per-channel mean under CV gate satisfaction as Table.1 (a)?
  • Could the authors provide cosine similarity between pedestal and per-channel mean?
  • Could the authors clarify the methodological differences between Pedestal Shifting and DFQ’s high-bias absorption?
3 comments
AuthorsOpenReview ↗

Response to Reviewer D2Cy (1/2): Pedestal vs. per-channel mean (Q1, Q2, W1)

We thank the reviewer for the precise and constructive comments, and for recognizing the persistent, perturbation-sensitive direction as an interesting observation. We have added the requested mean-shifting and DFQ-style comparisons on four backbones. Below, "L" denotes line numbers of the original submission.

Summary. The per-channel midrange is range-optimal but unstable. The per-channel mean is stable but not range-optimal. The pedestal has both properties: it shares its direction with the mean, and takes the range-optimal magnitude along the direction that Sec. 3 identifies as sensitive. In the requested comparisons, it outperforms mean shifting (up to 0.66 dB) and a DFQ-style baseline (up to 1.83 dB, part 2/2), with lower quantization error along this direction.

 

Why Table 1(a) compares against the midrange (W1).

Under per-tensor quantization, extreme channels widen the pooled activation range (L53–56). Per-channel midrange shifting minimizes this pooled range and provides a natural range-based reference among prior channel-wise shifts (L69–71). Table 1(a) shows that the pedestal is far more stable than this reference across calibration sets. The per-channel mean is the complementary reference, stable but not range-optimal, and we compare against it below.

 

Mean comparison (Q1, Q2).

Table R1 follows the setting of Table 1(a), with the pedestal and per-channel mean applied at the same gated sites.

BackboneCos. ↑Dist. (%) ↓Cos (p,m)PSNR ↑e1e_1 error ratio
MambaIR0.9971 / 0.99667.69 / 7.810.999935.429 / 35.3791.19×
MaIR0.9958 / 0.99559.17 / 9.160.999532.975 / 32.5191.73×
MambaIRv2-light0.9976 / 0.99768.68 / 6.590.999733.754 / 33.0911.60×
SwinIR0.9980 / 0.99756.81 / 6.810.999733.973 / 33.5152.71×

Table R1. Paired entries: pedestal / mean. Cos. and Dist. measure consistency across calibration sets, following Table 1(a). The fourth column compares the pedestal and mean vectors. Error ratio: e1e_1-aligned quantization error of the mean divided by that of the pedestal.

Direction and stability (Q1, Q2): The alignment is expected from our construction. We use the uncentered second moment to retain channel-wise offsets (L156–157). Let μ\mu and Σ\Sigma denote the activation mean and covariance, and let ct=xt⊤e1c_t=x_t^\top e_1, with mean cˉ\bar c and standard deviation σc\sigma_c. Since e1e_1 is the leading unit eigenvector of M=μμ⊤+ΣM=\mu\mu^\top+\Sigma,

λ1(M)=cˉ2+σc2≥∥μ∥22.\lambda_1(M)=\bar c^2+\sigma_c^2\ge \lVert\mu\rVert_2^2.

For nonzero μ\mu and neglecting the numerical stabilizer δ\delta, the constancy gate gives

∣e1⊤μ∣∥μ∥2=∣cˉ∣∥μ∥2≥11+CV(c)2≥11+τ2≈0.995,τ=0.1.\frac{|e_1^\top\mu|}{\lVert\mu\rVert_2} =\frac{|\bar c|}{\lVert\mu\rVert_2} \ge \frac{1}{\sqrt{1+\mathrm{CV}(c)^2}} \ge \frac{1}{\sqrt{1+\tau^2}} \approx 0.995, \qquad \tau=0.1.

The high alignment in Table R1 is therefore consistent with the gate. The pedestal and mean also show broadly comparable stability across calibration sets. Thus, the distinction from mean shifting lies in the range-based choice of shift magnitude and its quantization benefits, rather than superior estimation stability.

Performance (Q1): Directional alignment does not make the shifts interchangeable. The pedestal improves PSNR over mean shifting by 0.456–0.663 dB on MaIR, MambaIRv2-light, and SwinIR. MambaIR is nearly tied, with a 0.050 dB difference.

The mean removes the average coefficient cˉ\bar c along e1e_1, whereas the pedestal uses the midrange:

c⋆=arg⁡min⁡amax⁡t∣ct−a∣=max⁡tct+min⁡tct2,p=c⋆e1.c^\star =\arg\min_a \max_t |c_t-a| =\frac{\max_t c_t+\min_t c_t}{2}, \qquad p=c^\star e_1.

This choice minimizes the maximum absolute residual coefficient and the pooled range of the e1e_1-aligned residual (Eq. 6). The lower quantization error along this direction is a separate empirical result: Table R1 shows reductions by factors of 1.19–2.71, consistent with the motivation to protect the sensitive directional component identified in Sec. 3.2. The smallest error reduction coincides with the near tie in PSNR on MambaIR.

 

Revision (done). Table 1(a) now includes per-channel mean shifting; it is reported on MambaIRv2-light, where both properties are clearly visible. Sec. 4.1 has a new paragraph, "Relation to per-channel shifts". Appendix B.3 gives the alignment bound (Eq. 11) and the cross-backbone summary of Table R1 (both shifts at the same gated sites).

AuthorsOpenReview ↗

Response to Reviewer D2Cy (2/2): Pedestal Shifting vs. DFQ high-bias absorption (Q3, W2)

Summary. The shift-and-fold mechanism is shared with DFQ, while the shift differs in objective, structure, and criterion. The pedestal outperforms a DFQ-style baseline incorporating the three components requested by the reviewer on all four backbones, by up to 1.83 dB.

 

The shift-and-fold compensation mechanism is shared, and we do not claim it as our contribution. Both methods compensate for the shift through the next layer, y=W(x−z)+Wzy=W(x-z)+Wz. Our paper identifies channel-wise shifting as an established tool (L69–71) and cites OS+ for offline compensation (L235–237).

 

The methods differ in objective, shift structure, and gating criteria (Q3).

  • Objective. DFQ absorbs large channel biases arising from weight equalization. Pedestal Shifting preserves a persistent activation component identified as particularly sensitive to quantization error (Sec. 3).
  • Structure and magnitude. DFQ computes shifts independently from each channel's mean and standard deviation, retaining a 3σi3\sigma_i margin in shifted channels. We construct a site-wise vector p=c⋆e1p=c^\star e_1 from the shared dominant direction, with c⋆c^\star minimizing the aligned component's residual range (Eq. 6).
  • Criterion. Both use CV-type conditions, as the reviewer notes. DFQ's channel-wise condition, βi>3γi\beta_i > 3\gamma_i, identifies positive shifts under its approximate ReLU-preserving rule. Our site-wise condition selects sites where the response along e1e_1 is nearly constant across tokens and can be represented by a single constant (Eq. 7, L263–267).

DFQ-style baseline (W2). We implemented the empirical per-channel shifts, CV-based gating, and bias correction specified by the reviewer. Our sites have no BN and no intervening ReLU, so we use calibration-derived statistics and the sign-symmetric rule zi=sign⁡(μi)max⁡(0,∣μi∣−3σi)z_i=\operatorname{sign}(\mu_i)\max(0,|\mu_i|-3\sigma_i). A related variant without the margin, per-channel mean shifting under our gate, is compared in Table R1.

BackbonePSNR: pedestal / DFQ-stylee1e_1 error ratio
MambaIR35.429 / 35.2468.1×
MaIR32.975 / 32.6897.7×
MambaIRv2-light33.754 / 31.9276.3×
SwinIR33.973 / 33.8085.3×

Table R2. Same setting as Table R1. Error ratio: e1e_1-aligned quantization error of the DFQ-style baseline divided by that of the pedestal.

The pedestal outperforms the DFQ-style baseline on all four backbones, by 0.165–1.827 dB, and its e1e_1-aligned quantization error is lower by factors of 5.3–8.1, consistent with its focus on the sensitive component.

 

Revision (done). DFQ is now cited in Sec. 1, 2.2, and 4.1, and the opening of Sec. 4.1 states how our shift is determined differently.

AuthorsOpenReview ↗

Revised PDF uploaded.

We have uploaded the revised paper (changes in blue) and updated the "Revision" paragraphs of our two responses above with the locations. The parts most relevant to your comments are Table 1(a), the new paragraph "Relation to per-channel shifts" in Sec. 4.1, and Appendix B.3. We hope the revision addresses your concerns and would appreciate it if you could take a look.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Follow-up on our responses and revised manuscript

Dear Reviewers,

We sincerely thank you for your time and thoughtful comments, which have helped us strengthen the paper.

The revisions include clearer distinctions from prior shifting methods, a more explicit connection between our analysis and method, and broader supporting analyses. Changes are shown in blue, and our responses to each reviewer provide further clarifications and references to the relevant sections in the revised manuscript.

As the discussion period is closing, we would be grateful if you could review the revised manuscript and let us know whether our responses have addressed your concerns. We are happy to clarify any remaining points. 

Thank you again for your time and consideration.

Best regards, 

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.