acceptodds
Under review as a conference paper at ICLR 2027

SOLA-SR: Learning Scale-Ordered Latent Subspaces for Arbitrary-Scale Super-Resolution

Abstract

Latent representations have driven significant recent advances in large language models, and advance efforts to adapt such representations to computer vision have inspired new paradigms for image generation. However, discrete latent representations constrain the modeling of naturally continuous image features, especially in super-resolution (SR), where reconstruction requires richer, finer-grained image features. Inspired by COLA-DLM’s separation of continuous latent representations from conditional decoding, we propose Scale-Ordered Latent Subspaces for Arbitrary-Scale Super-Resolution (SOLA-SR). SOLA-SR separates shared image encoding from scale-dependent selective decoding, extracting image features in a single encoding pass and progressively exposing them during decoding as the target scale increases. Specifically, a learnable orthogonal transformation reorganizes feature channels while preserving their spatial layout. Continuous scale gating selects a compact subset of feature dimensions at lower scales and progressively activates additional channel groups as the scale increases. We algebraically fold the selected subspace’s linear mapping into the decoder’s first query layer, thereby skipping computations associated with inactive input dimensions. A single encoding thus supports deterministic reconstruction at multiple scales without iterative latent-variable sampling. By coupling scale-dependent representation organization with selective decoding, SOLA-SR offers a path toward combining fine-grained scale control, reconstruction quality, and inference efficiency in arbitrary-scale super-resolution. Experiments indicate an overall trend toward faster end-to-end inference and distinguish the roles of representation organization and iterative latent-variable generation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
arbitrary-scale super-resolution, image restoration, implicit neural representations, scale-ordered latent subspaces, efficient decoding

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_FhhqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This article introduces SOLA-SR, which organizes feature channels into nested subspaces through learnable orthogonal transformations, and dynamically selects some channels based on the super-resolution scale to reduce decoding computations at lower scales. The method is combined with a LIIF-style decoder and implements selective decoding through algebraic transformations. Experiments on several standard datasets and different upscaling ratios have verified both the reconstruction quality and running efficiency, and the numerical consistency between dense and selective decoding is also reported.

Critical strengths
  1. The method design is simple and theoretically solid. Selective decoding is based on precise algebraic transformations rather than approximate calculations, and experiments have confirmed its consistency with dense decoding.
  2. The scale-related feature organization is somewhat novel. A shared image encoding can support different scales while reducing the decoder's computation at lower magnifications.
Critical weaknesses
  1. The core ablation doesn't fully prove the role of the learned orthogonal basis. In Table 4, the learned basis is optimized separately after freezing the encoder/decoder, while the identity/random baseline doesn’t undergo fully matched joint training, so the performance boost can’t be clearly attributed to the orthogonal basis itself.
  2. Efficiency advantages are limited and there’s a lack of thorough comparison with recent methods. At ×4 and higher magnifications, the gate is almost fully open and the computational gain is minimal; moreover, there’s no fair comparison with recent efficient ASSR methods. Therefore, it’s not yet sufficient to demonstrate that the efficiency advantage is competitive.
Questions for authors

Is it possible to provide a completely fair ablation: the identity, random orthogonal, and learned orthogonal settings all use the same encoder/decoder, loss, training budget, and scale distribution for joint training?

Could you also add a fair efficiency comparison with recent efficient ASSR methods, and clearly report the full training setup and parameter count?

In the appendix, the number of different scales and the number of experiment pairs seem inconsistent; could you clarify the evaluation protocol?

4 comments

Thank you for reading our paper carefully and pointing out where it could be improved. Below, we address the concerns you raised and respond to each of your questions.

We have also revised the manuscript. Please look for \textcolor[RGB]{166,75,0}{\text{dark-orange text}} or \textcolor[RGB]{166,75,0}{\underline{\text{dark-orange underlining}}} to identify revisions addressing your comments.

For Critical Weaknesses:

1. The core ablation does not sufficiently establish the role of learning the orthogonal basis. In Table 4, the learned basis is optimized separately with the encoder and decoder frozen, while the identity and random-basis controls do not undergo fully matched joint training. The performance gains therefore cannot be clearly attributed to the orthogonal basis itself.

Our response:

Thank you for pointing out this limitation of Table 4. We jointly trained the encoder and decoder with an identity basis (T5), a fixed random orthogonal basis (T6), or a learned orthogonal basis (T4). The three variants used the same initialization, gate, data, loss, scale distribution, and training budget across three matched seeds.

DatasetScaleT4 − T5 ΔY-PSNR (dB)T4 − T6 ΔY-PSNR (dB)T4 − T5 ΔLPIPST4 − T6 ΔLPIPS
Set5×2+0.194 ± 0.006+0.177 ± 0.007−0.00024 ± 0.00005−0.00041 ± 0.00005
Set5×3+0.073 ± 0.014+0.052 ± 0.005+0.00085 ± 0.00076−0.00367 ± 0.00029
Set5×4+0.003 ± 0.002+0.022 ± 0.009−0.00265 ± 0.00033−0.00443 ± 0.00043
Set14×2+0.137 ± 0.018+0.111 ± 0.025−0.00048 ± 0.00028+0.00018 ± 0.00038
Set14×3+0.041 ± 0.002+0.022 ± 0.003+0.00176 ± 0.00081−0.00380 ± 0.00058
Set14×4+0.015 ± 0.002+0.016 ± 0.002−0.00240 ± 0.00046−0.00642 ± 0.00025
B100×2+0.080 ± 0.003+0.066 ± 0.001−0.00103 ± 0.00035+0.00008 ± 0.00039
B100×3+0.028 ± 0.002+0.013 ± 0.002−0.00123 ± 0.00100−0.00486 ± 0.00055
B100×4+0.004 ± 0.003+0.013 ± 0.001−0.00369 ± 0.00048−0.00421 ± 0.00034
Urban100×2+0.245 ± 0.008+0.182 ± 0.006−0.00176 ± 0.00006−0.00129 ± 0.00012
Urban100×3+0.064 ± 0.005+0.054 ± 0.004−0.00064 ± 0.00041−0.00142 ± 0.00020
Urban100×4+0.037 ± 0.003+0.038 ± 0.002−0.00235 ± 0.00030−0.00203 ± 0.00037

The learned basis achieved comparative performance than both fixed bases in all 12 dataset–scale comparisons after matched joint training, supporting its contribution within the gated model.

You can find the clarification of which components are jointly trained in the dark-red text with a dark-blue underline in §3.1 (p. 3, lines 154–157). The separate frozen-backbone study is described in the plain black text in §4.3 (p. 8, lines 430–431).

2. The efficiency advantage is limited, and comparisons with recent methods are insufficient. At ×4 and higher scales, the gate is almost fully open, leaving little computational benefit. Fair comparisons with recent efficient ASSR methods are also missing. The evidence is therefore insufficient to establish a competitive efficiency advantage.

Our response:

We address the measured efficiency gain and the comparison with recent ASSR methods separately.A and B.

A. Selective decoding yields its largest savings at lower magnifications.

We compared Dense and Exact with identical weights over 200 scales, reporting the results below

Encoding modeScale rangeDense mean/frame (ms) ↓Exact mean/frame (ms) ↓Time reduction
Independent×2.01–×3.5088.35882.1357.0%
Independent×3.51–×4.00129.352124.0694.1%
Reuse×2.01–×3.5050.48443.65613.5%
Reuse×3.51–×4.0090.95284.3227.3%

Above ×3.5, all channels are active and the first-layer width saving is zero. The remaining measured time difference reflects the complete Dense and Exact execution paths. The larger reductions at lower scales are the benefit targeted by selective decoding.

B. Recent methods establish the quality and efficiency comparison.

We evaluated quality on the same images at ×2, ×3, and ×4, and timed across 200 scales on the same experiment settings. In accordance with your requirements, we have incorporated the recent efficient ASSR method as a comparison.

MethodParameters (M)Avg. Y-PSNR ↑Avg. LPIPS ↓Independent (s) ↓Encoding reuse (s) ↓
SOLA-SR Exact22.32329.6860.131918.52610.803
AnySR–EDSR2.29729.3010.200412.681—
DIIF–RDN small22.30528.9790.222410.723—
HIIF–RDN23.70929.1370.189134.80227.278
GSASR–RDN41.19829.1570.187898.16189.682
Thera RDN-Air21.98229.4180.202218.2739.590

These results show that SOLA-SR reduces inference time through selective decoding while delivering competitive performance among the evaluated ASSR methods.

The added comparison with recent ASSR methods and the accompanying discussion appear in the dark-orange text with a dark-blue underline around Table 4 (p. 8, lines 410–425).

For Questions:

1.Could you provide a fully fair ablation in which the identity transform, fixed random orthogonal transform, and learned orthogonal transform are all jointly trained using the same encoder, decoder, losses, training budget, and scale distribution?

Our reply:

OK,we trained the identity (T5), fixed random orthogonal (T6), and learned orthogonal (T4) variants with the same encoder and decoder architecture, initialization, gate, data, loss, scale distribution, and training budget.

DatasetScaleT4 − T5 ΔY-PSNR (dB)T4 − T6 ΔY-PSNR (dB)T4 − T5 ΔLPIPST4 − T6 ΔLPIPS
Set5×2+0.194 ± 0.006+0.177 ± 0.007−0.00024 ± 0.00005−0.00041 ± 0.00005
Set5×3+0.073 ± 0.014+0.052 ± 0.005+0.00085 ± 0.00076−0.00367 ± 0.00029
Set5×4+0.003 ± 0.002+0.022 ± 0.009−0.00265 ± 0.00033−0.00443 ± 0.00043
Set14×2+0.137 ± 0.018+0.111 ± 0.025−0.00048 ± 0.00028+0.00018 ± 0.00038
Set14×3+0.041 ± 0.002+0.022 ± 0.003+0.00176 ± 0.00081−0.00380 ± 0.00058
Set14×4+0.015 ± 0.002+0.016 ± 0.002−0.00240 ± 0.00046−0.00642 ± 0.00025
B100×2+0.080 ± 0.003+0.066 ± 0.001−0.00103 ± 0.00035+0.00008 ± 0.00039
B100×3+0.028 ± 0.002+0.013 ± 0.002−0.00123 ± 0.00100−0.00486 ± 0.00055
B100×4+0.004 ± 0.003+0.013 ± 0.001−0.00369 ± 0.00048−0.00421 ± 0.00034
Urban100×2+0.245 ± 0.008+0.182 ± 0.006−0.00176 ± 0.00006−0.00129 ± 0.00012
Urban100×3+0.064 ± 0.005+0.054 ± 0.004−0.00064 ± 0.00041−0.00142 ± 0.00020
Urban100×4+0.037 ± 0.003+0.038 ± 0.002−0.00235 ± 0.00030−0.00203 ± 0.00037

After matched joint training, the learned basis showed better overall performance than both fixed bases across dataset–scale comparisons.

2.Could you add a fair efficiency comparison with recent efficient ASSR methods and explicitly report the full training settings and parameter counts?

Our reply:

A. The efficiency comparison covers recent ASSR methods.

MethodParameters (M)Avg. Y-PSNR ↑Avg. LPIPS ↓Independent (s) ↓Encoding reuse (s) ↓
SOLA-SR Exact22.32329.6860.131918.52610.803
AnySR–EDSR2.29729.3010.200412.681—
DIIF–RDN small22.30528.9790.222410.723—
LM-LIIF–EDSR-b1.38629.4600.20733.0032.578
HIIF–RDN23.70929.1370.189134.80227.278
GSASR–RDN41.19829.1570.187898.16189.682
Thera RDN-Air21.98229.4180.202218.2739.590

Compared with the same-weight Dense path, Exact reduces sequence time by 6.1% with independent encoding and 11.1% with encoding reuse while preserving reconstruction quality. The broader comparison places SOLA-SR’s measured quality and efficiency alongside recent methods.

B.The training setup settings for both models.

SettingMatched protocol
Training data780 DIV2K training images
Input and queries48×48 LR input, 192×192 HR crop, 64×64 query region
Batch and scalesBatch 4; scales sampled from [2,4)[2,4), with ×4 every 20 steps
Training budget20k decoder-only warm-up steps, then 80k joint steps
OptimizerAdamW, weight decay 0, gradient clipping at 1
Learning ratesWarm-up 10−410^{-4}; joint stage 10−510^{-5} for encoder and decoder and 10−410^{-4} for SOLA-SR’s Cayley parameters, with stage-wise cosine decay
Loss weightsCharbonnier 1, LPIPS 0.02, color 0.01; scheduled Sobel term

3.The number of scales and the number of experimental pairs reported in the appendix appear inconsistent. Could you clarify the evaluation protocol?

Our reply:

We checked the experiment server and confirmed that the appendix retained an outdated total count of experimental pairs. We confirm that all experiments compared within each table used the same evaluation protocol. To clarify the counts, each scale uses 219 pairs, giving a total of 657 image–scale pairs per method across the fixed scales ×2, ×3, and ×4. Appendix Table 17 still reported the earlier total of 876 pairs across four scales. This was a counting error, which we have now corrected. Thank you for pointing it out.

For the evaluation pair count, please consult the plain black text in §4.1 (p. 6, lines 275–280), which reports 657 integer-scale image–scale pairs.

Reviewer_d6tjOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes SOLA-SR, an arbitrary-scale super-resolution framework that organizes shared image features into scale-ordered latent subspaces. Based on an RDN encoder and a LIIF-style coordinate decoder, the method introduces a learnable orthogonal transformation, parameterized through the Cayley transform, to reorganize feature channels. A predefined scale-dependent gating schedule progressively activates additional channel groups as the target scale increases. At inference time, the selected subspace mapping is algebraically folded into the decoder's first query layer to avoid computation on inactive feature dimensions while preserving equivalence to dense decoding. The framework is trained using pixel, perceptual, color, and gradient losses. Experiments on Set5, Set14, B100, and Urban100 report competitive reconstruction quality and reduced inference time relative to the LIIF baseline in the evaluated settings.

Critical strengths
  • Competitive reconstruction performance across multiple scales.\textbf{Competitive reconstruction performance across multiple scales.} The method achieves competitive PSNR and SSIM, together with consistently favorable LPIPS results, on the evaluated benchmarks. The experiments include integer and non-integer magnification factors, as well as scales beyond the configured channel-release range.
  • Measured inference improvements over the LIIF baseline.\textbf{Measured inference improvements over the LIIF baseline.} Table 3 reports lower inference time than LIIF-RDN under both independent encoding and encoding reuse. The paired Dense–Exact comparisons further demonstrate that selective decoding provides measurable latency reductions at smaller scales without requiring separate model training.
Critical weaknesses

Major Weaknesses

  • Uncontrolled comparisons.\textbf{Uncontrolled comparisons.} SOLA-SR uses additional LPIPS, color, and gradient losses, while the baselines follow different training recipes. Without loss ablations or a loss-matched LIIF baseline, the reported improvements, particularly in LPIPS, cannot be attributed to the proposed representation. This is especially concerning at ×4, where all channels are active and the orthogonal transformation cancels out in the forward pass.

  • Unclear scale-ordered representations.\textbf{Unclear scale-ordered representations.} The nested subspaces are defined by a fixed gating schedule rather than learned. Table 5's correlations do not establish scale-specific feature specialization, since channel activation is already designed to increase with scale. It remains unclear what information different channel groups represent and why orthogonal reorganization is preferable to alternative learned projections. Orthogonality also does not prevent information loss after truncation.

  • Incomplete ablation.\textbf{Incomplete ablation.} Table 4 omits the original pretrained LIIF performance without gating. Therefore, the improvement from the learned orthogonal basis may simply recover information lost through channel truncation rather than improve the original representation. Moreover, this key ablation uses only 20 images from the DIV2K training split rather than held-out validation images, raising concerns about generalization and potential overlap with pretraining data.

  • Limited efficiency benefits.\textbf{Limited efficiency benefits.} The proposed method reduces computation only in the decoder's first query layer, primarily at smaller scales. At ×4 and above, all channels are active and the computational savings disappear, despite larger scales requiring more expensive reconstruction. Table 3 also lacks a direct quality–latency trade-off, making the practical benefit unclear, especially considering the additional training.

  • Unclear motivation for Table 9.\textbf{Unclear motivation for Table 9.} The HR-conditioned posterior has access to information unavailable in the original SR problem, so its improvement does not establish the benefit of latent organization. The LR-conditioned generation results also do not isolate the contribution of iterative sampling. Since conventional ASSR methods already perform deterministic reconstruction, the relevance of these experiments and the comparison with COLA-DLM to the main contribution remains unclear.

Minor Weaknesses

  • Limited generalization and novelty discussion.\textbf{Limited generalization and novelty discussion.} The method is only demonstrated with an RDN–LIIF architecture, without evaluation using other common backbones such as EDSR or SwinIR. Its distinction from existing resource-adaptive methods, particularly AnySR, and earlier nested representations is also insufficiently discussed. The conceptual connection to COLA-DLM appears weak, as shared encoding and conditional decoding already exist in LIIF.

  • Presentation issues.\textbf{Presentation issues.} The related-work section largely repeats the introduction, while Figure 2 does not clearly explain the relationship between the notation and decoder operations. Table 5's interpretation is insufficiently justified, and the qualitative comparisons in Figure 4 do not consistently show an obvious advantage over GSASR-RDN.

Questions for authors
  • Can the authors provide a loss-matched RDN–LIIF baseline with the same training budget, along with ablations of the LPIPS, color, and gradient losses? How much of the reported improvement remains?
  • What is the original pretrained LIIF performance without gating in Table 4? Why is the ablation evaluated on only 20 DIV2K training images rather than held-out validation images, and were these images used during pretraining?
  • What evidence shows that the learned channel groups contain scale-specific information rather than merely adapting to a predefined gating schedule? Can the authors compare different channel budgets, channel orderings, and alternative learnable projections?
  • Why is computation reduced only in the first query layer, especially when the savings disappear at larger scales? Can the authors provide a quality–latency–memory comparison and report the additional training cost?
  • What hypothesis is Table 9 intended to validate? How do the HR-conditioned and LR-conditioned experiments independently isolate representation organization and iterative latent generation?
  • Does the proposed method generalize to other encoders, such as EDSR or SwinIR, and what distinguishes it from existing resource-adaptive ASSR methods?
10 comments

Thank you for reading our paper carefully and pointing out where it could be improved. Below, we address the concerns you raised and respond to each of your questions.

We have also revised the manuscript. Please look for \textcolor[RGB]{31,78,121}{\text{dark-blue text}} or \textcolor[RGB]{31,78,121}{\underline{\text{dark-blue underlining}}} to identify revisions addressing your comments.

For Critical Weaknesses:

1. The comparison does not control for training variables. SOLA-SR adds LPIPS, color, and gradient losses, while the baselines use different training recipes. Without loss ablations or a loss-matched LIIF control, the reported gains, especially in LPIPS, cannot be attributed to the proposed representation. This concern is particularly important at ×4, where all channels are active and the orthogonal transform cancels in the forward pass.

Our response:

We address this concern in two parts, covering the training comparisons and the full-gate behavior at ×4.

The comparison does not control for training differences.

We trained LIIF-RDN and SOLA-SR with the same losses, data, and training schedule across three matched seeds.

DatasetsScaleLIIF-RDN Y-PSNR ↑LIIF-RDN Y-SSIM ↑LIIF-RDN LPIPS-Alex ↓SOLA-SR Y-PSNR ↑SOLA-SR Y-SSIM ↑SOLA-SR LPIPS-Alex ↓
Set5×238.0300.96030.039838.4090.96230.0399
Set5×334.4990.92580.063034.8830.93110.0627
Set5×432.2900.89350.100632.8160.90270.1002
Set14×233.8500.91890.069634.3040.92410.0692
Set14×330.4090.84220.117830.7780.85090.1172
Set14×428.6550.78090.180328.9650.79210.1798
B100×232.2570.89990.112032.4520.90330.1117
B100×329.0940.80450.161429.4110.81300.1608
B100×427.5560.73330.230827.8460.74680.2303
Urban100×232.6460.93330.047833.5170.94060.0477
Urban100×328.7160.86360.100629.3330.87580.1001
Urban100×426.6000.80070.160527.1650.81790.1601

Against the retrained LIIF-RDN baseline, th SOLA-SR scores remain comparative performance.

At ×4, all channels are active and the orthogonal transform cancels in the forward pass.

As you point out, the orthogonal transform and its inverse cancel at ×4, and the query branch reads the encoder features directly. However,SOLA-SR jointly trains the encoder, orthogonal basis, and decoder across scales, so the full-width query uses weights learned under scale-dependent gating and can yield different results from a separately trained LIIF-RDN.

Dataset (×4)LIIF-RDN (T2) Y-PSNR ↑ (dB)SOLA-SR Y-PSNR ↑ (dB)LIIF-RDN (T2) Y-SSIM ↑SOLA-SR Y-SSIM ↑LIIF-RDN (T2) LPIPS ↓SOLA-SR LPIPS ↓
Set532.29032.8160.89340.90270.10080.1002
Set1428.65028.9650.78090.79210.18010.1798
B10027.55627.8460.73310.74680.23080.2303
Urban10026.59727.1650.80060.81790.16020.1601

Thus, full gate-cancellation does not imply identical performance for separately trained SOLA-SR and LIIF-RDN models.

You can find the joint-training clarification in the dark-red text with a dark-blue underline in §3.1 (p. 3, lines 154–157). The fully open query path and training losses are described in §§3.5–3.6 (p. 5, lines 249–269).

2. The meaning of the scale-ordered representation is unclear. The nested subspaces are defined by a fixed gating schedule rather than learned. The correlations in Table 5 do not establish scale specialization of the channels, because channel activation is designed to increase with scale. It remains unclear what information the different channel groups represent and why orthogonal reorganization is preferable to other learnable projections. Orthogonality also does not prevent information loss after truncation.

Our response:

Firstly,we would like to clarify the role of the fixed gate. In §3.3–3.4 (p. 4, Eqs. 1 and 4), M(s)M(s) determines when basis coordinates are activated, while the learned orthogonal transform TT determines their directions in the original feature space through zs=T⊤M(s)Tzz_s=T^\top M(s)Tz.

And then,we address this concern in two parts.

A. The gate fixes the activation order, while training learns the channel basis.

We jointly trained the learned, identity, and fixed random orthogonal bases with the same gate, data, loss, and budget across three seeds. All variants use the same 32-channel budget.

DatasetLearned − Identity ΔY-PSNR (dB)Learned − Fixed Random ΔY-PSNR (dB)
Set5+0.194 ± 0.006+0.177 ± 0.007
Set14+0.137 ± 0.018+0.111 ± 0.025
B100+0.080 ± 0.003+0.066 ± 0.001
Urban100+0.245 ± 0.008+0.182 ± 0.006

At this fixed budget, learning the basis improves reconstruction over either fixed basis.

B. Table 5’s correlations alone do not establish what the channel groups represent.

We kept the ×3 checkpoint and active-channel budget fixed, then reordered the groups or removed equally weighted, equally wide active groups.

DatasetFixed Random Order ΔLPIPSReverse Order ΔLPIPSDrop Group 0 − Drop Group 1 ΔLPIPS
Set5+0.0156 ± 0.0013+0.0207 ± 0.0016+0.0164 ± 0.0009
Set14+0.0211 ± 0.0012+0.0252 ± 0.0018+0.0208 ± 0.0009
B100+0.0358 ± 0.0006+0.0412 ± 0.0021+0.0341 ± 0.0014
Urban100+0.0123 ± 0.0008+0.0134 ± 0.0011+0.0133 ± 0.0002

Which groups are selected affects reconstruction at the same budget. These output tests establish functional differences between groups.

The dark-blue addition in §3.3 (p. 4, lines 205–207) distinguishes the roles of the fixed gate and the learned basis.

3. The ablation is incomplete. Table 4 omits the performance of the original pretrained LIIF without gating. The improvement from learning an orthogonal basis may therefore merely recover the loss caused by channel truncation rather than improve the original representation. Moreover, this key ablation uses only 20 images from the DIV2K training set rather than independent validation images, raising concerns about generalization and possible overlap with the pretraining data.

Our response:

A. Table 4 omits the ungated pretrained LIIF reference.

We recovered the ungated result under Table 4’s protocol, using the same frozen EDSR-baseline-LIIF model, 20 development images, and eight off-grid scales.

ConfigurationRGB PSNR ↑ (dB)RGB SSIM ↑LPIPS ↓
Original LIIF, no gate28.9330.68030.1164
Identity basis + gate24.5650.78410.2107
Fixed random orthogonal basis + gate21.3670.76260.1771
Learned orthogonal basis + gate30.969 ± 0.0050.8789 ± 0.00010.1204 ± 0.0002

The learned basis substantially improves reconstruction over the fixed-basis gated controls and brings performance close to the ungated reference.

In the revised manuscript, the ungated LIIF reference added in response to your comment appears as the dark-blue row in Table 5 (p. 9, line 437).

B.Datasets clarifying

We thank the reviewer for raising the data provenance question. Table 4 describes its 20 images as development images. We divided the 800 DIV2K training images into 780 for fitting the orthogonal basis and 20 for evaluation, and confirmed that none of those 20 images entered basis fitting. The pretrained LIIF backbone’s training pool did include them, so Table 4 evaluates basis learning within this development setting.

And there were some well methods devided 800 images into 2 parts.Please see the tables below.

StudyTraining within DIV2K Train_HRInternal evaluationRelevant scopePaper Link
SOLA-SR Table 4Cayley basis fitted on IDs 0001–0780IDs 0781–0800, 20 development imagesThe 20 images were excluded from basis fitting. They fall within the official LIIF backbone’s reported 800-image pretraining pool.Ours
RL-Restore, CVPR 2018First 750 imagesRemaining 50 imagesAn internal split of the 800 training imagesPaper
OWAN, CVPR 2019First 750 imagesRemaining 50 imagesAn internal split used for the restoration comparisonPaper
FYE-SR, ACM MobiCom 2024700 images for its Model Split Learner100 imagesIts SR models were described as pretrained on DIV2KPaper
Reviewer_yqDYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SOLA-SR targets arbitrary-scale super-resolution. An RDN encoder computes one feature map per image. A learnable per-pixel orthogonal channel transform (Cayley-parameterized) rotates the features. A fixed, piecewise-linear gate then releases nested channel groups as the scale grows: 32 channels are always on, and four groups of 8 open between ×2 and ×4. A LIIF decoder reconstructs the image from the gated features. At inference, the subspace projection is folded into the first LIIF layer, so that layer processes only the active channels. The paper motivates the design by analogy to COLA-DLM. Experiments cover integer and non-integer scales up to ×35.2 on Set5, Set14, B100 and Urban100, timing over 200 consecutive scales, an orthogonal-basis ablation on a frozen pretrained LIIF, a numerical check that the dense and folded paths agree, and a study contrasting representation organization with LR-conditioned latent generation.

Critical strengths
  1. A different place to put scale adaptation. Most ASSR methods adapt the upsampler, implicit function or renderer to the scale. This paper instead organizes the latent channels so that low scales use a compact prefix and higher scales expose more groups. This is a reasonable and somewhat new angle for ASSR, and it is the main reason the paper is of interest.
  2. A correct and well-verified folding derivation. Folding the projection into the first decoder layer is exact. The dense and folded paths agree to about 1e-6, and the ×4 outputs are bitwise identical. The efficiency claim, though small, rests on a correct derivation.
Critical weaknesses

The method removes information, yet it reports better quality, with no explanation. The paper's motivation concerns only structure and efficiency. Mechanistically, the design should not improve quality. At low scales the gate only removes channels. The orthogonal transform T can be absorbed by the jointly trained encoder. For s ≥ 4 the model is architecturally identical to LIIF-RDN (Appendix E). Under matched training it should at best match LIIF-RDN, so the reported gains (e.g., Set5 ×4: 32.82 vs. 32.51 dB) are more likely due to its different training losses (LPIPS, gradient, color). The authors themselves note that "the more distinctive benefits are structural" (L325–327). Neither a same-recipe LIIF-RDN baseline nor a joint-training ablation without T is provided.

Questions for authors
  1. Given that gating only removes information at low scales and the model is identical to LIIF-RDN for s ≥ 4, what explains the reported quality gains over LIIF-RDN?
  2. How does LIIF-RDN perform when retrained with exactly the same losses, data and schedule as SOLA-SR?
3 comments

Thank you for reading our paper carefully and pointing out where it could be improved. Below, we address the concerns you raised and respond to each of your questions.

We have also revised the manuscript. Please look for \textcolor[RGB]{139,30,45}{\text{dark-red text}} or \textcolor[RGB]{139,30,45}{\underline{\text{dark-red underlining}}} to identify revisions addressing your comments.

    For Critical Weaknesses:

The method removes information, yet it reports better quality, with no explanation. The paper's motivation concerns only structure and efficiency. Mechanistically, the design should not improve quality. At low scales the gate only removes channels. The orthogonal transform TT can be absorbed by the jointly trained encoder. For s≥4s \geq 4 the model is architecturally identical to LIIF-RDN (Appendix E). Under matched training it should at best match LIIF-RDN, so the reported gains (e.g., Set5 ×4\times 4: 32.82 vs. 32.51 dB) are more likely due to its different training losses (LPIPS, gradient, color). The authors themselves note that "the more distinctive benefits are structural" (L325–327). Neither a same-recipe LIIF-RDN baseline nor a joint-training ablation without TT is provided.

Our response:

Thank you for reading the paper closely and for pointing out the need for a matched LIIF-RDN baseline and a jointly trained control without the learned transform. We address your concerns in three parts below.

A. Why does learning the basis matter when low-scale gating discards feature channels?

We jointly trained the encoder and decoder for each variant with the same gate and training recipe across three matched seeds, then evaluated all three on the same experiment setting(at ×2 scale) with 32 active channels.

ConfigurationY-PSNR ↑ (dB)paired gain of T4 (dB)
Identity Basis + Gate (T5)32.350 ± 0.009+1.167 ± 0.009
Fixed Random Orthogonal Basis + Gate (T6)32.413 ± 0.004+1.104 ± 0.004
Learned Orthogonal Basis + Gate (T4)33.517—
Matched LIIF-RDN, Ungated (T2)32.644 ± 0.004+0.873 ± 0.004

With the same 32-channel budget, the learned basis retains feature directions that yield higher performance than either fixed basis.

B. What does cancellation at full gating imply for the comparison with a separately trained LIIF-RDN?

SOLA-SR has the same forward form as LIIF-RDN. However,the two models are trained differently, and we explain this distinction in more detail in our response under Questions for Authors below. Thank you for raising this point.Let’s continue…….SOLA’s encoder and decoder are jointly trained with the gated basis across scales, so its predictions can differ from those of a separately trained LIIF-RDN.To certificate,we evaluated the three variants jointly trained under the same protocol(at ×4 scale) across three matched seeds, where all 64 channels are active and the orthogonal transform and its inverse cancel.Results of ours experiment are as follow.

ConfigurationY-PSNR ↑ (dB)paired gain of T4 (dB)
Identity Basis + Gate (T5)26.549 ± 0.009+0.616 ± 0.009
Fixed Random Orthogonal Basis + Gate (T6)26.548 ± 0.007+0.617 ± 0.007
Learned Orthogonal Basis + Gate (T4)27.165—

The matched-training comparison shows that SOLA-SR and LIIF-RDN still differ after separate training.

You can find our clarification of joint training in the dark-red text with a dark-blue underline in §3.1 (p. 3, lines 154–157). The fully open query path is described in §3.5 (p. 5, lines 249–259).

C. How does matching the training recipe affect the reported quality comparison with LIIF-RDN?

DatasetsScaleLIIF-RDN Y-PSNR ↑LIIF-RDN Y-SSIM ↑LIIF-RDN LPIPS-Alex ↓SOLA-SR Y-PSNR ↑SOLA-SR Y-SSIM ↑SOLA-SR LPIPS-Alex ↓
Set5×238.0300.96030.039838.4090.96230.0399
Set5×334.4990.92580.063034.8830.93110.0627
Set5×432.2900.89350.100632.8160.90270.1002
Set14×233.8500.91890.069634.3040.92410.0692
Set14×330.4090.84220.117830.7780.85090.1172
Set14×428.6550.78090.180328.9650.79210.1798
B100×232.2570.89990.112032.4520.90330.1117
B100×329.0940.80450.161429.4110.81300.1608
B100×427.5560.73330.230827.8460.74680.2303
Urban100×232.6460.93330.047833.5170.94060.0477
Urban100×328.7160.86360.100629.3330.87580.1001
Urban100×426.6000.80070.160527.1650.81790.1601

Under matched experimental conditions, SOLA-SR delivers competitive reconstruction quality against LIIF-RDN, and the gains observed without additional manual hyperparameter tuning support the contribution of its scale-dependent subspace design.

The explanation in §3.3 (p. 4, lines 190–207) describes how the orthogonal transform reorganizes features before the gate selects channels.

    For Questions:

1.Given that gating only removes information at low scales and the model is identical to LIIF-RDN for s ≥ 4, what explains the reported quality gains over LIIF-RDN?

Our reply:

Thank you for your question.

First, we clarify how SOLA-SR builds on LIIF-RDN. LIIF-RDN provides a mature foundation for studying feature use across magnification scales through feature extraction and coordinate querying. We use its RDN [CVPR 2018] and LIIF [CVPR 2021] components without treating the entire LIIF-RDN model as a frozen module. RDN extracts spatially organized features from the low-resolution image, while LIIF reconstructs pixels from selected features, query coordinates, and pixel cell sizes. Building on these components lets us focus on SOLA-SR’s representation organization, including which features to use and how to reduce the corresponding computation.

Second, we clarify the quality comparison. We jointly train the encoder, decoder, and orthogonal transform. In the paper’s reported comparisons, SOLA-SR achieves better results than LIIF-RDN. Table 1 presents the fixed-scale results, and Table 2 presents the non-integer-scale results. Following your question, we also completed a comparison under the same experimental settings and provide its detailed results in our response to the next question.

We retrained LIIF-RDN using the same data, losses, and schedule as the controlled SOLA-SR run, then compared its three-seed mean with the SOLA-SR scores.

2.How does LIIF-RDN perform when retrained with exactly the same losses, data and schedule as SOLA-SR?

Our reply:

We retrained LIIF-RDN with the same losses, data, and training schedule as in our controlled SOLA-SR experiments, and report its results alongside the SOLA-SR scores.

DatasetsScaleLIIF-RDN Y-PSNR ↑LIIF-RDN Y-SSIM ↑LIIF-RDN LPIPS-Alex ↓SOLA-SR Y-PSNR ↑SOLA-SR Y-SSIM ↑SOLA-SR LPIPS-Alex ↓
Set5×238.0300.96030.039838.4090.96230.0399
Set5×334.4990.92580.063034.8830.93110.0627
Set5×432.2900.89350.100632.8160.90270.1002
Set14×233.8500.91890.069634.3040.92410.0692
Set14×330.4090.84220.117830.7780.85090.1172
Set14×428.6550.78090.180328.9650.79210.1798
B100×232.2570.89990.112032.4520.90330.1117
B100×329.0940.80450.161429.4110.81300.1608
B100×427.5560.73330.230827.8460.74680.2303
Urban100×232.6460.93330.047833.5170.94060.0477
Urban100×328.7160.86360.100629.3330.87580.1001
Urban100×426.6000.80070.160527.1650.81790.1601

Using the retrained LIIF-RDN as the reference, the SOLA-SR remain stronger overall across the datasets and scales shown.

For the SOLA-SR training objective used as the reference for the matched comparison, please consult the plain black text in §3.6 (p. 5, lines 265–269).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.