acceptodds
Under review as a conference paper at ICLR 2027

SWA-IC: Multirate Learned Image Compression with Sliding Window Attention

Abstract

Learned image compression has surpassed conventional codecs in rate–distortion performance, but many leading methods require a separately trained model for each operating point. Supporting multiple rates therefore increases training and storage requirements and complicates deployment. We present SWA-IC, a single image compression model supporting 128 fine-grained operating points with state-of-the-art rate–distortion performance. We extend sliding window attention from autoregressive context modeling to the analysis and synthesis transforms, establishing a common SWA-based design for feature transformation and spatial context modeling. We give the synthesis transform direct access to context features for image reconstruction and equip SWA with learned attention sinks to suppress uninformative contributions. Growing channel groups and generalized Gaussian entropy modeling strengthen the parallel context model, while channel-wise gains control rate with shared network weights across all operating points. On Kodak, Tecnick, and CLIC, SWA-IC achieves BD-rate savings of , , and , respectively, relative to VTM 23.10, obtaining the lowest BD-rate among the evaluated single-rate and multirate codecs. At medium and large tested resolutions, it also matches or improves on the decoding latency of previous state-of-the-art image codecs.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
learned image compression, multirate compression, sliding window attention, entropy modeling, transformer networks

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_st8mOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SWA-IC, a multirate image compression model supporting 128 operating points. It combines sliding window attention, context feature reuse, growing channel groups, and generalized Gaussian entropy modeling to improve rate–distortion performance.

Critical strengths
  1. Strong results against recent compression methods.
  2. Good coverage of benchmarks and runtime comparisons.
  3. Useful ablations, especially for context feature reuse.
Critical weaknesses
  1. The novelty is limited. Most components are adaptations of existing methods.
  2. The much larger training dataset raises concerns about comparison fairness.
  3. Some ablations change model capacity, making the reported gains difficult to isolate.
  4. The 165M-parameter model also weakens the storage-efficiency motivation.
Questions for authors
  1. Would the improvements hold with matched training data and model capacity?
  2. Can the authors provide parameter-matched ablations for channel grouping and context feature reuse?
  3. What is the actual storage advantage over MixCompress, considering the full model size?
3 comments
AuthorsOpenReview ↗

Response to Reviewer st8m

We thank the reviewer for the feedback and the concrete questions.

W1: Novelty. We agree that the components build on prior work. The contribution is what their combination shows: sharing all network weights across 128 rates reaches state-of-the-art rate–distortion performance; reusing context features for synthesis gives a large gain; and the bimodal GGD shapes (Sec. 4.2.2) may explain why GGD helps more here than reported by Zhang et al. (2025).

W2/Q1: Training data. Retraining on ImageNet-1k and a 400k-image subset with otherwise identical settings is under way (see the general comment).

W3/Q2: Capacity-matched ablations. Capacity-matched controls are in training; the general comment lists the parameters and decoding complexity of the original variants and their controls.

W4/Q3: Storage. We agree that the storage argument applies to sets of single-rate models and will clarify this. One 165.1M-parameter checkpoint covers all 128 operating points; six checkpoints of the compared single-rate codecs need 397–716M.

MixCompress-MoD is slightly smaller (156.6M) because of its lighter cascaded LALIC backbone (66.1M); its rate-dependent experts and conditioning add 90.4M (+137%). SWA-IC shares all transform and context-model weights across rates, with 237.6k rate-specific parameters (+0.14%; gains and per-rate factorized priors). Our non-cascaded backbone, following DCVC-RT, uses wide features at low resolution and thus more parameters for a given compute budget; experts would add to it. At similar size, SWA-IC saves 5.00%, 5.04%, and 3.05% bitrate over MixCompress-MoD (Kodak, Tecnick, CLIC), offers 128 instead of six operating points, and decodes faster: MixCompress-MoD reports slightly slower decoding than LALIC, which is 1.7–3.4× slower than SWA-IC.

AuthorsOpenReview ↗

Update: Response to Reviewer st8m

The new results are in (details in the general comment):

W2/Q1: Training data. SWA-IC trained on ImageNet-1k is on par or slightly better than the main model (+0.07% / −0.34% / −0.59% on Kodak / Tecnick / CLIC), as is a 400k-image subset (Sec. 4.1, App. A.7).

W3/Q2: Capacity-matched ablations. With one additional synthesis block, removing the context features still costs 8.50% (8.53% before). With equal channel groups at matched parameters, the gap is 5.19% (9.62% before). Table 2 lists parameters and decoding complexity for every row.

W4/Q3: Storage. The abstract and introduction now state that the storage argument applies to sets of single-rate models, and quantify the overhead of MixCompress's rate adaptation. App. A.5 gives the total sizes.

Do the new results, together with the storage clarification, address your concerns? We are happy to clarify anything that remains open.

The rebuttal addresses most of my concerns. I will raise my rating from 2 to 3.

Reviewer_DWh8OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SWA-IC is a single multirate image codec that covers 128 operating points with shared weights, controlled by banks of channel-wise gains. The transforms follow DCVC-RT's non-cascaded design, using 2D sliding window attention blocks (3x3 window) with learned attention sinks. The context model adapts P-SWA to images: a four-step diagonal spatial schedule times three channel groups gives 12 sequential stages. It uses growing channel groups (28, 56, 172) and a generalized Gaussian likelihood with beta in [1, 7]. The synthesis transform also receives the channel-module features alongside the decoded latent. The model is trained for 12 epochs on the winter-2021 ImageNet release on one H100. It reports BD-rates of -18.44 / -24.86 / -18.53% vs VTM 23.10 on Kodak / Tecnick / CLIC, the best among the compared single-rate and multirate codecs, and decodes a 2048x2048 image in 0.36 s on an RTX 3090. Validation includes the authors' own VTM runs, common-interval BD-rates, local rate-difference curves, runtime protocol checks, from-scratch ablations, and an analysis of rate allocation and GGD shapes.

Critical strengths
  1. Practical result: one 165M-parameter checkpoint with 128 rate points matches or exceeds the best single-rate codecs on three datasets, with competitive decoding latency at medium and large resolutions. If this holds under controlled training data (see Weakness 1), it is useful for deployment because it removes the need for one model per rate.

  2. The evaluation is unusually trustworthy. The authors run VTM 23.10 themselves with documented settings, use explicit overlap intervals (plus common-interval and middle-third BD-rates in Table 7), and show where the method wins and loses through local rate curves. They also validate the AMP timing protocol against FP32 and time DCVC-RT with and without its CUDA extensions. This is a main reason I lean positive.

Critical weaknesses
  1. The training data is not comparable to that of the baselines. Twelve epochs at about 1.2M iterations and batch size 128 imply ~12.8M images per epoch, i.e. ImageNet-21k scale. The single-rate baselines' published numbers come from much smaller training sets (e.g. Flickr30k for TCM, about 100k ImageNet images for MLIC++). The margins over the strongest baselines are small: on Kodak, -18.44 vs -18.43 for HPCM Large is a tie; the gap is 0.42 points on CLIC and 1.34 points over CMIC on Tecnick. They are also sensitive to the integration interval (the Kodak margin becomes 0.94 under Table 7's common interval). The state-of-the-art claim may therefore be partly a data effect.

  2. Neither the cost of multirate training nor the source of the largest gains is isolated. There is no single-rate SWA-IC, so we cannot see what shared-weight training costs this architecture, or whether the high-rate decline comes from rate adaptation. The two largest ablation effects are confounded with capacity. "No growing groups" (+9.62%) changes hidden widths and parameter count (A.1). "No context features for synthesis" (+8.53%) removes 1024 channels of decoder-side features. Since most components come from prior work (SWA/P-SWA, DCVC-RT, gain-based rate control, GGD, attention sinks, wavelet shortcuts, ELIC-style groups), the insight beyond strong engineering is limited.

Questions for authors
  1. Which images, and how many, were used for training? Can you report SWA-IC trained on ImageNet-1k only, or a strong baseline retrained on your data?
  2. Can you report a single-rate SWA-IC at 3-4 lambda values with the same data and schedule?
  3. Can you give parameters and decoding MACs for each ablation row, and capacity-matched controls for the channel-grouping and synthesis-context ablations?
  4. At what PSNR do the local rate curves of SWA-IC cross those of HPCM Large and CMIC at the high-rate end?
3 comments
AuthorsOpenReview ↗

Response to Reviewer DWh8

We thank the reviewer for the detailed review and for recognizing the care taken in our evaluation.

W1/Q1: Training data. We train on 13.1M images (ImageNet winter 2021 release), consistent with your estimate. Retraining on ImageNet-1k and a 400k-image subset with the same number of iterations is under way (see the general comment).

W1: Margins. Differences between VTM-anchored BD-rates over different PSNR intervals are not direct comparisons (App. A.6). Measured directly, SWA-IC saves 0.43% bitrate against HPCM Large on Kodak, 0.72% against HPCM Large on CLIC, and 1.75% against CMIC on Tecnick. Over the common intervals of Table 7, the margins grow to 0.94, 1.86, and 1.07 points (Kodak, Tecnick, CLIC), and to 1.88, 2.66, and 1.92 over the middle third, so the advantage holds regardless of the integration interval.

W2/Q2: Cost of multirate training. A single-rate SWA-IC takes about 165 GPU hours per λ\lambda (same as multirate). Our compute is currently committed to the retraining runs for the shared data and capacity concerns (see the general comment), so training single-rate models for several λ\lambda is not feasible within the discussion period. If resources become available, we can instead fine-tune the multirate checkpoint at single rates for 4 epochs and compare each with the multirate curve at equal PSNR. In an earlier convergence check, continuing multirate training for 4 more epochs gave no further improvement, so any gain from such fine-tuning would reflect specialization rather than additional training. A larger gain at the highest rate would then point to rate adaptation as the cause of the high-rate decline.

W2/Q3: Capacity of ablation variants. The general comment lists the parameters and decoding complexity of both variants and their capacity-matched controls.

W2: Novelty. We agree that the components build on prior work. The contribution is what their combination shows: sharing all network weights across 128 rates reaches state-of-the-art rate–distortion performance; reusing context features for synthesis gives a large gain; and the bimodal GGD shapes (Sec. 4.2.2) may explain why GGD helps more here than reported by Zhang et al. (2025).

Q4: Crossing points. At the high-rate end, SWA-IC's local rate curve crosses that of HPCM Large at about 37.35 dB on Kodak, stays ahead of it over the whole range on Tecnick, and meets it at HPCM Large's highest rate point (38.45 dB) on CLIC. It crosses that of CMIC at about 37.30, 38.36, and 38.05 dB on Kodak, Tecnick, and CLIC. Except for the CLIC point against HPCM Large, these crossings lie between widely spaced rate points of the baselines, so their positions rely on the Akima interpolation and are uncertain.

AuthorsOpenReview ↗

Update: Response to Reviewer DWh8

The new results are in (details in the general comment):

W1/Q1: Training data. SWA-IC trained on ImageNet-1k is on par or slightly better than the main model (+0.07% / −0.34% / −0.59% on Kodak / Tecnick / CLIC), as is a 400k-image subset. The results are not a data effect (Sec. 4.1, App. A.7). The direct pairwise savings over the strongest baselines are now in Sec. 4.2.1.

W2/Q3: Ablations. Table 2 now lists parameters and decoding kMACs/px for every row. With one additional synthesis block, removing the context features still costs 8.50% (8.53% before). With equal channel groups at matched capacity, the gap is 5.19% (9.62% before).

W2/Q2: Single-rate SWA-IC. The shared cluster unfortunately did not free up enough compute for the single-rate runs, so we cannot provide them within the discussion period. The paper makes no claim about single-rate performance. Sec. 4.2.1 now states as a hypothesis that the high-rate decline may partly stem from sharing network weights across a wide rate range, based on the existing comparison of MixCompress with its single-rate backbone.

Do these results address your concerns about the training data and the ablations? We are happy to answer any remaining questions.

Thanks for the response and the additional experiments. These address my main concerns about the training data and ablations. I understand the compute constraints on the single-rate comparison. I will keep my current score.

Reviewer_2sP1OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This work proposes an image compression method with continuous variable-rate capability, outperforming all existing methods—including state-of-the-art (SOTA) fixed-rate image compression algorithms. The approach extends "sliding window attention" (SWA) into both the analysis and synthesis transforms, while enabling the synthesis transform to directly access context features.

Critical strengths
  1. The method supports variable-rate compression while simultaneously surpassing the performance of existing fixed-rate approaches.
  2. The introduction of context features for the synthesis transform yields significant performance gains.
  3. The decoding speed is highly competitive, outperforming most existing methods.
Critical weaknesses
  1. In the P-SWA context model shown in Figure 1, the output of "Spatial Module 1" is fed directly into the "Masked Transformer Block" within the accumulator. Does this introduce a risk of information leakage from the encoder to the decoder? If so, is this information accounted for in the bit-rate calculation?
Questions for authors
  1. While the method is impeccable in terms of performance and efficiency, could it encounter issues when deployed across diverse hardware platforms—for instance, due to inconsistencies in computational precision?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 2sP1

We thank the reviewer for the positive assessment and the precise questions.

W1: Thank you for raising this. Figure 1 does not distinguish the accumulator's strict mask from the non-strict masks of the spatial modules; we will mark the strict masking in Figure 1 and its caption. The accumulator's cross-attention is strictly masked: each position attends only to Spatial Module 1 outputs from earlier decoding steps, excluding its own step. These outputs depend only on symbols the decoder has already reconstructed (Eq. 6, Sec. 3.3), so encoder and decoder compute identical features. The only information sent ahead is the hyperprior z~\tilde{z}, whose bits are included in all reported rates; all rates are measured from actually decoded bitstreams.

Q1: This is a fair point, and a limitation SWA-IC shares with most learned codecs, including all compared single-rate baselines. The GGD scale and shape use quantized lookup tables, but the network that selects the entries runs in floating point, so numerical differences between devices can break decoding.

AuthorsOpenReview ↗

Update: Response to Reviewer 2sP1

We have revised Figure 1 and its caption as announced: the accumulator's strictly masked cross-attention is now marked in green and distinguished from the non-strict masks of Spatial Modules 1 and 2. We hope this resolves the concern and are happy to answer further questions.

Thanks for the response. My concerns have been addressed. I will keep my current score.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response to All Reviewers

We thank the reviewers for their constructive feedback. We are glad they found the paper clear and valued that a single model with 128 operating points matches or exceeds the best single-rate codecs (all reviewers), the gains from reusing context features for synthesis (2sP1, st8m), the competitive decoding speed (2sP1, DWh8), and the thorough evaluation (DWh8, st8m).

Two concerns are shared by Reviewers DWh8 and st8m. We address both with new experiments.

  1. Training data. SWA-IC is trained on the ImageNet winter 2021 release (13.1M images), which is larger than the training sets of the single-rate baselines. To isolate the effect of the training data, we retrain SWA-IC on the ImageNet-1k training set (1.28M images) and a random subset (400k images) of it. We train for the same number of iterations (1.2M), so only the data changes.

  2. Capacity of ablation variants. We are rerunning the ablations that change model capacity with matched capacity. The variant with equal channel groups receives a wider SwiGLU in the channel blocks, and the variant without context features one additional synthesis block. Parameters and decoding kMACs/px, relative to the full model (165.1M, 1219.6 kMACs/px):

    • Equal channel groups: 154.6M (−6.4%), 1178.2 kMACs/px (−3.4%); capacity-matched control: 165.1M (0.0%), 1219.5 kMACs/px (−0.01%)
    • No context features: 162.2M (−1.8%), 1208.3 kMACs/px (−0.9%); capacity-matched control: 166.4M (+0.8%), 1274.0 kMACs/px (+4.5%)

All runs are in training, currently at epochs 3–6 of 12 (or the equivalent iterations for the ImageNet-1k runs). So far, the logged training metrics show no qualitative differences from the original runs at the same training stage. We will report the results and update the paper once training completes, before the end of the discussion period.

AuthorsOpenReview ↗

Update: General Response to All Reviewers

We have uploaded a revised paper; changes are highlighted in blue. All runs have finished.

Training data. Retraining SWA-IC on smaller training sets with the same 1.2M iterations gives on-par or slightly better results. BD-rate relative to the main model (negative is better):

Training setImagesKodakTecnickCLIC
ImageNet winter 2021 (main model)13.1M0.00%0.00%0.00%
ImageNet-1k1.28M+0.07%−0.34%−0.59%
Random ImageNet-1k subset400k−0.13%−0.22%−0.51%

SWA-IC's advantage therefore does not stem from its larger training set.

Capacity-matched ablations. ΔBD-rate on CLIC relative to the full model (165.1M parameters, 1219.6 decoding kMACs/px):

VariantParamsDec. kMACs/pxΔBD-rate
No context features for synthesis162.2M1208.3+8.53%
No context features, +1 synthesis block166.4M1274.0+8.50%
Equal channel groups154.6M1178.2+9.62%
Equal channel groups, matched (m=6.52m=6.52)165.1M1219.5+5.19%

Added capacity does not compensate for the missing context features. For the channel groups, about half of the original gap was due to capacity; the growing groups remain clearly better.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.