acceptodds
Under review as a conference paper at ICLR 2027

PRISM: Progressive Region-Informed State Modeling for Overfitted Image Compression

Abstract

Overfitted codecs compress each image by fitting a lightweight decoder and a hierarchical latent tailored to the content. Given the inherent trade-off between model expressiveness and parameter rate, existing methods largely focus on representation design and optimization, leaving explicit modeling of dependencies across the latent hierarchy underexplored. We reveal that maintaining and reusing a shared contextual state across latent grids can improve both entropy modeling and image synthesis. Motivated by this, we introduce **P**rogressive **R**egion-**I**nformed **S**tate **M**odeling (PRISM) for overfitted image compression, which progressively encapsulates cross-scale context into a compact state as entropy modeling proceeds through the latent hierarchy. Gated, region-informed updates adapt the state to spatially varying dependencies while selectively retaining information from previously decoded grids for subsequent probability estimation. The accumulated state is further supplied to the synthesis network, where the retained context complements the latent representations for image reconstruction. Experiments on Kodak and CLIC2020 demonstrate state-of-the-art rate-distortion (RD) performance among overfitted codecs, with BD-rate reductions of 9.56% and 20.35% over VTM-19.1, respectively, approaching strong codecs such as LIC-HPCM at substantially lower decoding complexity. These gains extend across diverse overfitted codecs, highlighting the potential of PRISM as a general paradigm for improving both RD performance and convergence.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Implicit neural representation, source coding, overfitted image compression, low-complexity image codec, context state recycling.

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_HMdHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses overfitted image compression, where each image is encoded by jointly fitting a lightweight decoder and a hierarchical multi-resolution latent, with model parameters themselves contributing to the bitrate. The authors argue that existing overfitted codecs reuse decoded latent values but discard the contextual features formed during entropy modeling, and that preserving these features in a shared state could improve both entropy modeling and synthesis. They propose PRISM, which progressively aggregates the autoregressive entropy model's hidden features E_t into a compact persistent context state C_t (only c=2 channels) as decoding proceeds through the latent hierarchy.

Critical strengths
  1. The paper's insight is interesting: It accurately identifies that overfitted codecs have moved from the "increasing capacity" stage to "more efficiently utilizing compact image-specific representations," while explicit modeling of cross-grid dependencies has long been overlooked. The notion that "entropy model features are discarded after use" is a genuine and non-trivial entry point.

  2. The proposed architecture is highly lightweight with great application potential: The core module uses only 2 channels, and the memory gate involves merely T+1 scalar logits—Equation (8) shows that the entire state update plus gating requires only 61–75 parameters. This is a rare case where "architectural modifications introduce almost no additional rate or decoding overhead," which is precisely the property most needed for overfitted codecs.

  3. The experiments are solid: The authors conducted comprehensive and detailed ablation studies to verify different module sharing strategies. Compression performance and model parameter size were thoroughly compared against various baselines, yielding satisfactory results.

Critical weaknesses
  1. Theoretical support. Although the "state modeling" framework is appealing, the paper contains no theorems, propositions, or guarantees. The core update \bar{C}t = \delta C{t-1} + \alpha_t \Delta C_t (Eq. 1), the gating parameterization (Eq. 2), and the entropy residual correction (Eq. 4) are all empirically designed, without any information-theoretic or convergence justification. For ICLR, a framework named after "state modeling"—even with a rough analysis of "what the state retains" or "how errors accumulate across grids"—would significantly enhance persuasiveness.

  2. Novelty is combinatorial and needs clearer delineation. PRISM combines the hierarchical entropy model from Cool-chic 5.0, region-wise synthesis from MoRIC, and mature gated recurrence (LSTM/GRU/HGRN). The genuinely new point—accumulating ARM features into a reusable state—is homologous to existing context reuse in transform-based LIC (e.g., LIC-HPCM's hierarchical progressive context modeling, already cited by the authors). The paper should more precisely clarify how PRISM's reuse differs from these precedents, rather than simply stating "applying it to overfitted codecs."

  3. Although extensive experiments were conducted, the evaluation datasets are limited, including only Kodak and CLIC. It is recommended to introduce the Tecnick dataset and the DIV2K dataset for further testing. Additionally, only the PSNR metric was reported, which is insufficient; metrics such as MS-SSIM should also be considered.

  4. The bottleneck of encoding cost remains unresolved. PRISM still requires per-image iterative fitting (up to 100k main-stage updates), and each iteration must traverse the complete state sequence plus region synthesis. The claimed "5–7× fewer iterations" does not imply a proportional reduction in wall-clock time (as acknowledged by the authors in Appendix E.2). Therefore, the practical encoding cost of overfitted codecs is only alleviated, not resolved.

Questions for authors
  1. Provide theoretical support. Can we characterize what the state C_t retains across grids, or derive an upper bound on how reconstruction error accumulates as the state propagates?
  2. Supplementary ablation study: Of the 2.91/4.10 pp relative to Cool-chic 5.0, how much comes from the state mechanism itself, how much from the region-wise synthesis inherited from MoRIC, and how much from the interaction between the two? A matched ablation that "adds only PSM (without MoRIC) to Cool-chic 5.0" would resolve the ambiguity.
  3. Add experiments with the MS-SSIM metric and the Tecnick dataset. If possible, also include experiments on the DIV2K dataset to validate performance on high-resolution image compression.
  4. Add encoding/decoding latency experiments, reporting the average encoding and decoding time on Kodak.
8 comments

We thank the reviewer for the constructive feedback. In response, we have (a) provided an analytical characterization of progressive state accumulation; (b) clarified PRISM's distinctions from prior work along with additional ablations (Tables R.3.1 and R.3.7); (c) extended the evaluation to Tecnick, additional baselines, and MS-SSIM (Tables R.3.2-R.3.5); and (d) reported measured fitting and decoding times (Table R.3.6). We address each concern and question in detail below.

W1. (a) Following the suggestion, we provide an analytical characterization of the information retained in the context state across latent grids by explicitly expanding the linear transformations in the state update.

At grid tt, the state aggregates ARM features Et\mathbf E_t, and pooled region masks rt\mathbf r_t. Since GηG_{\boldsymbol\eta} is affine, the state update in Eq. (1) can be expressed as

ΔCt=WCCt−1+WEEt+Wrrt+bG,\Delta\mathbf C_t = \mathbf W_C\mathbf C_{t-1} +\mathbf W_E\mathbf E_t +\mathbf W_r\mathbf r_t +\mathbf b_G, where the weights WC\mathbf W_C, WE\mathbf W_E, and Wr\mathbf W_r act channel-wise on the incoming state, ARM features, and region features, respectively, and bG\mathbf b_G is the corresponding bias.

Let Pt\mathcal P_t and bU,t\mathbf b_{U,t} denote the linear component and bias of the resolution-alignment operator at grid tt. Combining the gated update with alignment yields the following theoretic result Ct=At(Ct−1)⏟Propagated previous state+qt⏟Current-grid contribution\begin{aligned} \mathbf C_t=\underbrace{\mathcal A_t(\mathbf C_{t-1})}_{\text{Propagated previous state}} +\underbrace{\mathbf q_t}_{\text{Current-grid contribution}} \end{aligned} This decomposition is explained as follows. Here, At(Ct)=Pt((δI+αtWC)Ct)\mathcal A_t(\mathbf C_t)=\mathcal P_t ((\delta\mathbf I+\alpha_t\mathbf W_C)\mathbf C_t) propagates previously accumulated information, while qt=αtPt(WEEt+Wrrt+bG)+bU,t\mathbf q_t=\alpha_t\mathcal P_t (\mathbf W_E\mathbf E_t+\mathbf W_r\mathbf r_t+\mathbf b_G) +\mathbf b_{U,t} introduces the current grid's projected features and the bias terms, I\mathbf I is the identity matrix.

Starting from C0=0\mathbf C_0=\mathbf 0, recursively expanding the updates gives: C1=q1\mathbf C_1=\mathbf q_1 and C2=A2(q1)+q2\mathbf C_2=\mathcal A_2(\mathbf q_1)+\mathbf q_2, C3=A3(A2(q1))+A3(q2)+q3\mathbf C_3 = \mathcal A_3\bigl(\mathcal A_2(\mathbf q_1)\bigr) +\mathcal A_3(\mathbf q_2)+\mathbf q_3. The final state is

CT=qT+AT(qT−1)+AT(AT−1(qT−2))+⋯+AT(AT−1(⋯A2(q1)⋯ )),\begin{aligned} \mathbf C_T ={}& \mathbf q_T +\mathcal A_T(\mathbf q_{T-1})\\\\ &+\mathcal A_T\bigl(\mathcal A_{T-1}(\mathbf q_{T-2})\bigr) +\cdots\\\\ &+\mathcal A_T\bigl( \mathcal A_{T-1}(\cdots\mathcal A_2(\mathbf q_1)\cdots) \bigr), \end{aligned}

This decomposition expresses CT\mathbf C_T as an accumulation of contributions qk\mathbf q_k from all latent grids, each incorporating ARM features and region information, where kk means latent grid index. The transition operators At\mathcal A_t determine how these contributions are transformed and retained across subsequent grids, thereby characterizing the mechanism of progressive context accumulation. This interpretation is related to recurrent aggregation in state-space models such as Mamba [1] and MambaIC [2].

[1] Gu, A. and Dao, T., 2023. Mamba: Linear-time sequence modeling with selective state spaces.

[2] Zeng, Fanhu, et al. Mambaic: State space models for high-performance learned image compression. Conference on Computer Vision and Pattern Recognition (CVPR). 2025.

W1. (b) We clarify that intermediate states condition subsequent entropy decoding, while only the final state is used for image synthesis.

Since no intermediate reconstructions are generated during state propagation, reconstruction error cannot be directly measured at each stage, making its accumulation error difficult to characterize. We therefore assess the contribution of each stage empirically through state visualization and ablation (Fig. 9, previously Fig 8).

W2. We agree that PRISM builds upon existing entropy models and region-wise synthesis. However, its key distinction lies in reusing entropy-model representations for both probability estimation and image synthesis under the limited model-rate and decoding-complexity constraints of overfitted compression.

Specifically, PRISM progressively reuses ARM features with minimal parameter overhead. Unlike LIC-HPCM, which primarily reuses hierarchical context for entropy estimation, PRISM further exposes the accumulated state to image synthesis, allowing both rate and distortion objectives to jointly optimize the shared representation. Our ablations (Table 3, previously Table 1) confirm the benefits of progressive accumulation and synthesis-side reuse beyond direct feature reuse or increased model capacity.

Furthermore, we have conducted additional ablations by incorporating progressive state modeling into other overfitted codecs (Cool-chic 5.0). As shown in Tables R.3.1, The consistent RD gains further demonstrate the effectiveness and generality of the proposed mechanism.

We have clarified these distinctions in the revised manuscript.

Table R.3.1 Effect of accumulated state and previous ARM feature only on complexity and RD performance.

SettingNN parameterDecoding complexityBD-rate
Arm+ifceupsamplingsynthesispsmTotal(MAC / pixel)(vs. w/o PSM)
CC5.0 w/o PSM108660735-188119430
CC5.0 (previous)1090607336219452053-0.81%
CC5.0 (Accumulated)1090607336719502058-2.15%
PRISM w/o PSM108660722-186821960
PRISM (Previous)1090607166419302280-0.72%
PRISM (Accumulated)1090607166919352286-1.98%

W3. We have extended our evaluation to Tecnick (Tables R.3.2-R.3.3), included additional baselines (Table R.3.4), and reported MS-SSIM results (Table R.3.5) to complement the PSNR-based evaluation.

Table R.3.2 Rate-distortion points on Tecnick dataset (first 10 images, 1200$\times$1200). CC5.0 and PRISM use λ∈2e−2,4e−3,1e−3,4e−4,1e−4\lambda \in \\{2e^{-2}, 4e^{-3}, 1e^{-3}, 4e^{-4}, 1e^{-4}\\} and 100k iterations.

Settingbpp/PSNR (dB)BD-rate (vs. VTM-19.1)
RD point123456-
HPCM-Large0.0541/32.5100.0766/33.8710.1095/35.0980.1605/36.4250.2370/37.7480.3722/39.227-25.39%
MLIC${++}$0.0554/32.6800.0824/33.9800.1197/35.2100.1854/36.5700.2718/37.7900.4053/39.090-19.43%
CC5.00.0414/31.1770.0932/34.3000.1977/36.9470.3384/38.8690.7270/41.807--21.13%
PRISM (ours)0.0419/31.5030.0925/34.5000.1920/37.1200.3273/38.9770.7144/41.978--26.28%

Table R.3.3 Rate-distortion points on Tecnick dataset (100 images, 1200$\times$1200). CC5.0 and PRISM use λ∈2e−2,4e−3,1e−3,4e−4,1e−4\lambda \in \\{2e^{-2}, 4e^{-3}, 1e^{-3}, 4e^{-4}, 1e^{-4}\\} and 10k iterations.

Settingbpp/PSNR (dB)BD-rate (vs. VTM-19.1)
RD point123456-
HPCM-Large0.0840/31.0120.1234/32.5250.1792/33.9730.2600/35.4890.3687/36.9470.5265/38.442-23.76%
MLIC${++}$0.0894/31.2750.1332/32.7210.1930/34.1000.2870/35.6220.3987/36.9130.5469/38.188-19.65%
CC5.00.0632/29.4330.1559/32.8600.3250/35.9430.5200/38.0570.9907/41.104--14.57%
PRISM (ours)0.0634/29.6740.1540/33.0760.3197/36.1300.5071/38.2140.9684/41.266--19.70%

Table R.3.4 More baselines on Tecnick dataset (100 images, 1200×1200).

Image codecELICSTFTCM-LargeMLIC${++}$WeConveneLALICHPCM-LargeCC5.0PRISM (ours)
BD-rate over VTM-19.1-6.49%-4.13%-13.75%-19.65%-10.59%-20.17%-23.76%-14.57%-19.70%

On Tecnick, PRISM outperforms LIC-HPCM on the first 10 images with 100k fitting iterations and outperforms MLIC${++}$ on all 100 images with only 10k iterations, demonstrating its potential for high-resolution compression. PRISM also maintains comparable performance to ELIC under MS-SSIM.

Table R.3.5 MS-SSIM BD-rate on Kodak, with VTM as the reference.

SystemBD-rate (MS-SSIM) ↓\downarrow
VTM0.00%
ELIC-46.94%
STF-49.56%
MLIC++-54.74%
LIC-HPCM-55.63%
PRISM (ours)-46.19%

Due to the limited rebuttal time and computational resources, we have not included DIV2K experiments. Nevertheless, our evaluation covers established benchmarks, with Kodak and CLIC2020 following the protocols commonly adopted in prior overfitted codecs [3,4]. In particular, our new Tecnick results provide additional evidence on high-resolution images, which are less frequently evaluated in overfitted compression due to the substantial per-image optimization cost.

[3] Ladune, Théo, et al. Cool-chic: Coordinate-based low complexity hierarchical image codec. International Conference on Computer Vision (ICCV). 2023.

[4] Kim, Hyunjik, et al. C3: High-performance and low-complexity neural compression from a single image or video. Conference on Computer Vision and Pattern Recognition (CVPR). 2024.

W4. To address the concern, we include the actual coding time across different architectures in Table R.3.6.

Table R.3.6 Average coding time for Kodak dataset using an NVIDIA RTX 3090 GPU and Intel Core i9-10980XE CPU @ 3.00GHz. Orange indicates GPU computation, blue indicates CPU computation and bold numbers highlight the best results.

ModelsEncoding timeDecoding timeBD rate
VTM-19.184.22 s\textcolor{blue}{84.22\text{ s}}274.16 ms\textcolor{blue}{274.16\text{ ms}}00
EVC (S/M/L)20.93/34.38/45.56 ms\textcolor{orange}{\mathbf{20.93/34.38/45.56}\text{ ms}}18.80/24.17/31.99 ms\textcolor{orange}{\mathbf{18.80/24.17/31.99}\text{ ms}}7.16%/1.05%/-0.73%
MLIC${++}$257.77 ms\textcolor{orange}{257.77\text{ ms}}334.03 ms\textcolor{orange}{334.03\text{ ms}}-14.96%
HPCM143.9 ms\textcolor{orange}{143.9\text{ ms}}159.8 ms\textcolor{orange}{159.8\text{ ms}}-19.92%
Cool-Chic 5.016.5 sec / 1k steps\textcolor{orange}{16.5\text{ sec / 1k steps}}251.1 ms\textcolor{blue}{251.1\text{ ms}}-6.65%
Cool-Chic 5.0 (PSM)22.1 sec / 1k steps\textcolor{orange}{22.1\text{ sec / 1k steps}}272.8 ms\textcolor{blue}{272.8\text{ ms}}-8.83%
PRISM26.5 sec / 1k steps\textcolor{orange}{26.5\text{ sec / 1k steps}}309.9 ms\textcolor{blue}{309.9\text{ ms}}-9.56%
PRISM (w/o PSM)20.3 sec / 1k steps\textcolor{orange}{20.3\text{ sec / 1k steps}}284.1 ms\textcolor{blue}{284.1\text{ ms}}-7.60%

Compared with PRISM without PSM, incorporating PSM increases decoding latency by 25.825.8 ms and encoding time by 6.26.2 s per 1,0001{,}000 iterations. However, Fig. 7 (previously Fig. 6) shows that PRISM achieves comparable RD performance to Cool-chic 5.0 on Kodak with approximately 5×5\times fewer encoding iterations. Accounting for the measured per-iteration costs, the estimated fitting time is 530530 s (20×26.520\times26.5) for PRISM and 1,6501{,}650 s (100×16.5100\times16.5) for Cool-chic 5.0, yielding a 3.1×\mathbf{3.1\times} speedup. Thus, under these settings, faster convergence more than offsets the additional per-iteration cost.

Overall, PRISM improves RD performance and encoding efficiency at the cost of moderately increased decoding latency. Further engineering optimizations may reduce the runtime overhead of PSM, as our current implementation uses unoptimized research code. We have incorporated these results and discussions into the revised manuscript.

Answers to questions:

Q1. In our response to W1, we analytically characterize the information retained across latent grids and examine the state's effects on bitrate and reconstruction quality (Fig. 9, originally Fig 8)).

Q2. Following the reviewer's suggestion, we add a matched ablation "CC5.0 + PSM (w/o MoRIC)" in Table R.3.7.

Table R.3.7 Ablation study on Kodak, with full PRISM as the BD-rate reference.

SystemBD-rate ↓\downarrow
PRISM (full)0.00%
CC5.0+2.95%
CC5.0 + PSM (w/o MoRIC)+0.74%
w/o State modeling+2.02%
w/o Synthesis state reuse+1.69%
w/o Learned memory gating (δ=1.0\delta=1.0, αt=1.0\alpha_t=1.0)+0.81%
w/o Region conditioning+0.62%

From the table, it is clear that the state mechanism alone provides approximately 22 percentage points of improvement, while regional synthesis inherited from MoRIC provides approximately 0.90.9 points. Combining both yields an overall improvement of approximately 33 percentage points. Moreover, PRISM achieves larger gains on high-resolution images, as evidenced by the CLIC2020 and Tecnick results (Tables R.3.2 and R.3.3).

To further examine the contribution of progressive accumulation, we include additional ablations in Table R.3.1. The results demonstrate the significant benefits of progressive context accumulation.

We have incorporated these ablations and discussion into the revised manuscript.

Q3. In our response to W3, we provided additional Tecnick evaluations, broader baseline comparisons, and MS-SSIM results (Tables R.3.2-R.3.5).

Q4. In our response to W4, we reported measured encoding and decoding time, and analyzed the overhead introduced by PSM (Table R.3.6)

We thank the reviewer again for the constructive feedback and welcome any further questions or suggestions.

Reviewer_eDsFOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PRISM, which progressively accumulates and reuses contextual features across latent grids for both entropy modeling and image synthesis in overfitted image compression. Experiments demonstrate strong RD performance, improved convergence, and relatively low decoding complexity.

Critical strengths

1.The proposed progressive state modeling provides an interesting approach to reusing hierarchical contextual information for entropy modeling and image synthesis.

2.PRISM achieves impressive RD performance, outperforming existing overfitted codecs while maintaining relatively low decoding complexity.

3.The experiments are comprehensive, including component ablations, convergence analysis, and evaluations across different overfitted codecs.

Critical weaknesses

1.Complexity-performance trade-off. PRISM introduces a noticeable increase in decoding complexity compared with Cool-chic 5.0, while its additional RD gains are relatively modest. It remains unclear whether similar improvements could be achieved by simply increasing the capacity of the baseline networks under the same computational budget.

2.Limited practical efficiency evaluation. The complexity analysis primarily relies on MACs/pixel and fitting iterations. Actual encoding and decoding times are not provided, making it difficult to assess the practical efficiency of the proposed method.

Questions for authors

1.Could the authors provide a complexity-matched comparison with Cool-chic 5.0, for example by increasing its ARM or synthesis network capacity under a similar computational budget?

2.Could the authors report actual encoding and decoding times under identical hardware conditions, particularly considering the additional state computations?

3 comments

We thank the reviewer for the valuable feedback. In response, we have (a) conducted additional comparisons of network capacity and progressive state accumulation (Tables R.2.1-R.2.2); (b) extended the RD evaluation to Tecnick (Table R.2.3) to further assess compression performance ; and (c) reported measured encoding and decoding times (Table R.2.4). We address each concern point by point below.

W1. To address this concern, we compare PRISM with Cool-chic 5.0 variants of increasing network capacity (HOP, our revised configuration, and VHOP) under comparable computational budgets. Our revised CC5.0 configuration uses a spatial ARM context size of 16 and a width of 58 for the first synthesis layer. The results in Table R.2.1 show that increasing network capacity alone does not match PRISM's RD performance, highlighting the benefit of effective context reuse beyond simply scaling network capacity.

Table R.2.1 Decoding complexity and BD-rate on the Kodak dataset.

ModelDecoding complexityBD-rateBD-rate
(MAC / pixel)(vs. VTM-19.1)(vs. CC5.0 HOP)
CC5.0 HOP1943-6.65%0
CC5.0 HOP + PSM2016-8.83%-2.15%
PRISM (state width=1)2211-8.97%-2.28%
CC5.0 (new configuration)2279-6.88%-0.21%
PRISM (state width=2)2286-9.56%-2.87%
PRISM (state width=4)2442-9.49%-2.72%
CC5.0 VHOP2874-5.91%+0.88%

To further isolate the benefit of progressive state accumulation, Table R.2.2 compares variants with similar computational budgets. Reusing ARM features alone achieves a BD-rate reduction of -0.72% relative to the baseline without PSM, while progressive state accumulation further increases this reduction to 1.98%, with only five additional parameters and six MACs/pixel. These results highlight the benefits of our design beyond simply increasing model capacity.

Table R.2.2 Effect of accumulated state and previous ARM feature only on complexity and RD performance.

SettingNN parameterDecoding complexityBD-rate
Arm+ifceupsamplingsynthesispsmTotal(MAC / pixel)(vs. w/o PSM)
CC5.0 w/o PSM108660735-188119430
CC5.0 (previous)1090607336219452053-0.81%
CC5.0 (Accumulated)1090607336719502058-2.15%
PRISM w/o PSM108660722-186821960
PRISM (Previous)1090607166419302280-0.72%
PRISM (Accumulated)1090607166919352286-1.98%

In addition, we emphasize that PRISM achieves BD-rate reductions of 2.87% on Kodak and 4.75% on CLIC2020 relative to Cool-chic 5.0, representing significant RD improvements over a leading overfitted codec. Notably, PRISM also achieves competitive RD performance with LIC-HPCM, a strong autoencoder-based neural codec, on CLIC2020.

To further evaluate performance on high-resolution images, we include additional results on Tecnick (Table R.2.3), where PRISM outperforms LIC-HPCM on the first 10-image subset. To the best of our knowledge, this is the first reported result of an overfitted codec surpassing LIC-HPCM on this subset. On 100 Tecnick images, PRISM also outperforms MLIC++ despite using only 10k fitting iterations due to rebuttal-time computational constraints, further demonstrating its potential for high-resolution compression.

We have incorporated these comparisons and discussions into the revised manuscript.

Table R.2.3 BD-rate (vs. VTM-19.1) on Tecnick (1200$\times$1200). CC5.0 and PRISM use λ∈{2e−2,4e−3,1e−3,4e−4,1e−4}\lambda \in \{2e^{-2}, 4e^{-3}, 1e^{-3}, 4e^{-4}, 1e^{-4}\}, with 100k iterations on the first 10 images and 10k iterations on all 100 images.

ModelBD-rate (vs. VTM-19.1)
First 10 images (100k iterations)100 images (10k iterations)
LIC-HPCM-25.39%-23.76%
MLIC${++}$-19.43%-19.65%
CC5.0-21.13%-14.57%
PRISM (ours)-26.28%-19.70%

W2. To address the concern, we include the actual coding time across different architectures in Table R.2.4.

Table R.2.4 Average coding time for Kodak dataset using an NVIDIA RTX 3090 GPU and Intel Core i9-10980XE CPU @ 3.00GHz. Orange indicates GPU computation, blue indicates CPU computation and bold numbers highlight the best results.

ModelsEncoding timeDecoding timeBD rate
VTM-19.184.22 s\textcolor{blue}{84.22\text{ s}}274.16 ms\textcolor{blue}{274.16\text{ ms}}00
EVC (S/M/L)20.93/34.38/45.56 ms\textcolor{orange}{\mathbf{20.93/34.38/45.56}\text{ ms}}18.80/24.17/31.99 ms\textcolor{orange}{\mathbf{18.80/24.17/31.99}\text{ ms}}7.16%/1.05%/-0.73%
MLIC${++}$257.77 ms\textcolor{orange}{257.77\text{ ms}}334.03 ms\textcolor{orange}{334.03\text{ ms}}-14.96%
HPCM143.9 ms\textcolor{orange}{143.9\text{ ms}}159.8 ms\textcolor{orange}{159.8\text{ ms}}-19.92%
Cool-Chic 5.016.5 sec / 1k steps\textcolor{orange}{16.5\text{ sec / 1k steps}}251.1 ms\textcolor{blue}{251.1\text{ ms}}-6.65%
Cool-Chic 5.0 (PSM)22.1 sec / 1k steps\textcolor{orange}{22.1\text{ sec / 1k steps}}272.8 ms\textcolor{blue}{272.8\text{ ms}}-8.83%
PRISM26.5 sec / 1k steps\textcolor{orange}{26.5\text{ sec / 1k steps}}309.9 ms\textcolor{blue}{309.9\text{ ms}}-9.56%
PRISM (w/o PSM)20.3 sec / 1k steps\textcolor{orange}{20.3\text{ sec / 1k steps}}284.1 ms\textcolor{blue}{284.1\text{ ms}}-7.60%

Compared with PRISM without PSM, incorporating PSM increases decoding latency by 25.825.8 ms and encoding time by 6.26.2 s per 1,0001{,}000 iterations. However, Fig. 7 (previously Fig. 6) shows that PRISM achieves comparable RD performance to Cool-chic 5.0 on Kodak with approximately 5×5\times fewer encoding iterations. Accounting for the measured per-iteration costs, the estimated fitting time is 530530 s (20×26.520\times26.5) for PRISM and 1,6501{,}650 s (100×16.5100\times16.5) for Cool-chic 5.0, yielding a 3.1×\mathbf{3.1\times} speedup. Thus, under these settings, faster convergence more than offsets the additional per-iteration cost.

Overall, PRISM improves RD performance and encoding efficiency at the cost of moderately increased decoding latency. Further engineering optimizations may reduce the runtime overhead of PSM, as our current implementation uses unoptimized research code. We have incorporated these results and discussions into the revised manuscript.

Answers to questions:

  • Q1. In our response to W1, we incorporated comparisons showing the impact of network capacity and state accumulation, along with additional results on Tecnick (Tables R.2.1-R.2.3), all supporting the superiority of PRISM.
  • Q2. In our response to W2, we reported measured encoding and decoding times, and analyzed the overhead introduced by PSM (Table R.2.4).

We thank the reviewer again for the constructive feedback and welcome any further questions or suggestions.

Thanks to the authors for their response. My concerns have been addressed. Although the contribution is relatively incremental for INR, the proposed idea is indeed interesting and effective. Therefore, I maintain my Weak Accept rating.

Reviewer_vfXYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PRISM, a framework for overfitted neural image compression that progressively updates a region-conditioned context state across latent grids and reuses it for both entropy modeling and image reconstruction. Experiments on Kodak and CLIC2020 report BD-rate reductions of 9.56% and 20.35% over VTM 19.1, respectively, while maintaining low decoding complexity. Ablation and cross-codec experiments further validate the proposed mechanism.

Critical strengths
  1. State-of-the-art compression performance with lightweight decoding. PRISM achieves state-of-the-art rate-distortion performance among existing overfitted image codecs, with BD-rate reductions of 9.56% and 20.35% over VTM 19.1 on Kodak and CLIC2020, respectively. Comparisons with recent overfitted codecs, including Cool-chic 5.0, and strong autoencoder-based codecs demonstrate a competitive rate-distortion–complexity trade-off.

  2. A well-motivated context-reuse mechanism with cross-codec validation. Reusing progressively accumulated entropy-model features for both probability estimation and reconstruction is technically reasonable. Component ablations support its effectiveness, while experiments on Cool-chic 5.0 and LotteryCodec demonstrate benefits across different codec architectures.

Critical weaknesses
  1. The proposed method is difficult to follow, with important implementation details insufficiently explained. While the background is clearly introduced, the description of PRISM itself is fragmented across the main text and appendix.

    • The connections between entropy decoding, state updates, regional synthesis, and mask-based merging are difficult to trace. Fig. 2 should explicitly identify the modules corresponding to the mathematical operators.
    • Although Table 2 specifies N=2 for all experimental configurations, the procedure for converting SAM proposals into exactly two non-overlapping regions covering the image is not explained.
    • There is an apparent dimensional inconsistency: Eq. (6) specifies c+T synthesis-input channels, where T includes hyperlatent grids, whereas Appendix D.1 and Table 2 indicate that only the main grids and context state enter synthesis directly.
  2. Additional ablations are needed to justify the specific architectural choices. The existing component-removal experiments demonstrate benefits, but leave several design choices insufficiently examined.

    • Table 1 includes an ablation without learned memory gating, but the replacement settings should be specified, particularly how the decay and update coefficients are fixed when learning is disabled.
    • All reported configurations use two regions and a two-channel state. Experiments varying the region count and state width would clarify their influence on compression performance and complexity.
    • Compare PRISM with a baseline that uses only the previous decoded grid's ARM features, suitably aligned, instead of the accumulated state. This would distinguish the benefit of progressive state accumulation from feature reuse alone.
  3. Actual encoding time is not evaluated.

    • Practical encoding time is important for per-image optimized codecs, but the paper reports convergence primarily in terms of optimization iterations.
    • Since different architectures can have different per-iteration costs, wall-clock encoding-time comparisons under comparable hardware are needed to establish the practical efficiency advantage.
  4. Minor presentation issues require further polishing.

    • Clearly identify the source image used in Fig. 8.
    • Clarify the color conventions in Fig. 2 and their relationship to the bitstream components and network modules.
    • Include a compact decoding workflow and essential configuration information in the main text to reduce reliance on the appendix.
    • Several figures and details essential to understanding the method are deferred to the appendix. Moving the detailed state-update diagram (Fig. 9), a compact decoding workflow, and key architecture settings into the main text would improve readability and reduce the need to repeatedly consult the appendix.
    • Correct the apparent typo “λ = 4 × 1^{-4}” in the caption of Fig. 26.

I am open to revising my assessment based on the authors’ clarifications, additional evidence, and points raised during the reviewer discussion.

Questions for authors

Please refer to the weaknesses.

4 comments

We thank the reviewer for the constructive feedback. In response, we have: (a) clarified the decoding workflow, region-partitioning procedure, and synthesis-input dimensions; (b) specified the fixed-gating configuration and added ablations on gating coefficients (Table R.1.1 & R.1.2), state width (Table R.1.3 & R.1.4), region count (Table R.1.5), and progressive state accumulation (Table R.1.6), together with their complexity analysis (Table R.1.7); (c) reported measured fitting and decoding times under matched hardware conditions (Table R.1.8) and analyzed practical encoding efficiency gains; and (d) corrected noted presentation issues.

We provide point-by-point responses to each concern below.

W1. (a) To clarify the connections among the components, we have added Section 3.5, “Decoding Workflow,” with algorithmic pseudocode detailing the decoding process.

We have also revised Fig. 2 to explicitly identify the modules corresponding to the mathematical operators.

(b) We have clarified the region-partitioning procedure in Appendix D. Specifically, SAM2 [1] automatically generates candidate masks without manual prompts. We select the highest-scoring mask covering 5-30% of the image as the foreground and use its complement as the background, ensuring two non-overlapping regions with complete image coverage. For N>2N>2, additional high-scoring masks are selected, with overlapping pixels excluded from subsequently selected regions.

[1] Ravi, Nikhila, et al. "SAM 2: Segment anything in images and videos" International Conference on Learning Representations, 2025.

(c) The reviewer is correct that only the main latent grids and context state enter synthesis, while hyperlatents are used exclusively for entropy modeling, as in [2, 3]. We have clarified this distinction by introducing TmainT_{\mathrm{main}} (main latent grid number) and ThyperT_{\mathrm{hyper}} (hyper latent grid number), where T=Tmain+ThyperT=T_{\mathrm{main}}+T_{\mathrm{hyper}}, and revising the synthesis-input dimension in Eq. (6) as c+Tmainc+T_{\mathrm{main}}. The corresponding revisions are highlighted in blue in the revised manuscript.

[2] Ballé, Johannes, et al. “Variational Image Compression with a Scale Hyperprior.” ICLR 2018.

[3] Ladune, Théo, et al. “Cool-chic 5.0: Faster Encoding and Inter-Feature Entropy Modeling for Overfitted Image Compression.” arXiv preprint arXiv:2605.02726, 2026.

W2. (a) We have clarified that the original ablation (Table 3, previously Table 1) fixes δ=1.0\delta=1.0 and αt=1.0\alpha_t=1.0. To provide a more comprehensive analysis, we additionally evaluate different fixed coefficients in Table R.1.1, showing that learned gating achieves the best BD-rate. For a more comprehensive analysis, Table R.1.2 reports the learned coefficients. Larger coefficients at lower rates indicate stronger reliance on accumulated context, whereas smaller coefficients at higher rates suggest reduced reliance on this context as individual latent grids carry richer information.

We have incorporated these new results and discussions into the revised manuscript.

Table R.1.1 Different decay and update coefficients settings on Kodak dataset (first 10 images).

Settingbpp/PSNR (dB)BD-rate (vs. learned)
lambda2e-24e-31e-34e-41e-4-
learned0.0888/27.8060.2161/31.2390.4636/34.9240.7118/37.5451.2628/41.3440
decay=1 update=10.0900/27.7840.2154/31.2130.4671/34.9110.7148/37.5331.2655/41.343+0.67%
decay=0.5 update=10.0914/27.8090.2191/31.2140.4632/34.8750.7182/37.5201.2685/41.324+1.47%
decay=0.5 update=0.50.0898/27.7970.2192/31.2170.4643/34.9030.7135/37.5181.2672/41.342+1.07%

Table R.1.2 Learned decay and update coefficients on Kodak (median over images).

lambda2e-24e-31e-34e-41e-4
decay0.680.620.530.570.48
update0.470.500.410.420.34

W2. (b) We provide additional ablations on state width (Tables R.1.3-R.1.4) and region count (Table R.1.5), evaluating both RD performance and decoding complexity.

Table R.1.3 Different state width effect on Kodak (region number=2).

Settingbpp/PSNR (dB)BD-rate (vs. VTM)
lambda2e-24e-31e-34e-41e-4-
State width=00.086/27.53010.220/30.90850.483/34.57580.757/37.25911.346/41.1777-7.60%
State width=10.087/27.60310.219/30.96610.478/34.60610.752/37.28921.335/41.1899-8.97%
State width=20.087/27.63940.218/30.97710.480/34.63840.750/37.30771.332/41.2138-9.56%
State width=40.090/27.70330.220/31.00930.480/34.63480.751/37.33011.335/41.2210-9.49%

Table R.1.4 Effect of the state width on complexity and RD performance.

SettingNN parameterDecoding complexityBD-rate
Arm+ifceupsamplingsynthesispsmTotal(MAC / pixel)(vs. State width=0)
State width=0108660722-186821960
State width=11088606994418912211-1.39%
State width=21090607166919352286-1.98%
State width=410946075012520292442-1.83%

As shown in Tables R.1.3-R.1.4, increasing state width improves RD performance up to c=2c=2, while further increasing it to c=4c=4 yields no additional gain at the cost of more parameters and greater optimization complexity. We therefore adopt c=2c=2 to balance RD performance and efficiency. Similarly, Table R.1.5 shows that N=2N=2 achieves the best RD performance on the five-image subset. Although finer partitioning allows greater spatial adaptation, the additional contour-coding overhead and decoding complexity can offset these benefits, particularly at low rates.

Table R.1.5 Effect of region numbers KK on complexity and performance on Kodak (5 images (kodim03, kodim13, kodim15, kodim19, kodim23), fixed channel dim=2).

SettingNN parameterDecoding complexityBD-rate
Arm+ifceupsamplingsynthesispsmTotal(MAC / pixel)(vs. KK=1)
KK=110906067167188820160
KK=21090607166919352286-0.34%
KK=31090607617119822555-0.05%

Considering the additional complexity and limited RD gains from larger state widths or more regions, we adopt c=2c=2 and N=2N=2 as a favorable trade-off between compression performance and complexity. We have incorporated these ablations and complexity comparisons into the revised manuscript.

W2. (c) Following the reviewer's suggestion, we compare progressive state accumulation with a baseline that reuses only the previous grid's ARM features (Tables R.1.6-R.1.7). Progressive accumulation achieves a lower BD-rate with nearly identical model size and decoding complexity, demonstrating its benefit beyond one-step feature reuse. We have incorporated this comparison and discussion into the revised manuscript.

Table R.1.6 Accumulated state vs. previous ARM feature only on Kodak (state width=2; region number=2 for PRISM).

Settingbpp/PSNR (dB)BD-rate (vs. VTM)
lambda2e-24e-31e-34e-41e-4-
CC5.0 (previous)0.087/27.48860.219/30.88460.478/34.56190.755/37.26051.347/41.2085-7.50%
CC5.0 (accumulated)0.085/27.56930.218/30.93170.480/34.59760.752/37.29521.341/41.2116-8.83%
PRISM (previous)0.089/27.56290.219/30.93070.482/34.62250.754/37.30131.341/41.2167-8.32%
PRISM (accumulated)0.087/27.63940.218/30.97710.480/34.63840.750/37.30771.332/41.2138-9.56%

Table R.1.7 Effect of accumulated state on complexity and RD performance.

SettingNN parameterDecoding complexityBD-rate
Arm+ifceupsamplingsynthesispsmTotal(MAC / pixel)(vs. previous)
CC5.0 (previous)10906073362194520530
CC5.0 (accumulated)1090607336719502058-1.33%
PRISM (previous)10906071664193022800
PRISM (accumulated)1090607166919352286-1.26%

W3 To address the concern, we report the actual coding time across different architectures in Table R.1.8.

Compared with PRISM without PSM, incorporating PSM increases decoding latency by 25.825.8 ms and coding time by 6.26.2 seconds per 1,0001{,}000 iterations due to the additional state updates and reuse. However, Fig. 7 (previously Fig. 6) shows that PRISM achieves comparable RD performance to Cool-chic 5.0 on Kodak with approximately 5×5\times fewer fitting iterations. Accounting for the measured per-iteration costs, the estimated encoding times are 530530 s (20×26.520\times26.5) for PRISM and 1,6501{,}650 s (100×16.5100\times16.5) for Cool-chic 5.0, yielding a 3.1×\mathbf{3.1\times} speedup. Faster convergence therefore offsets the additional computation per iteration.

Overall, PRISM improves RD performance and coding efficiency at the expense of moderately increased decoding latency. These measurements are based on research implementations, and further engineering optimization may reduce the PSM overhead. We have incorporated Table R.1.8 and this analysis into the revised manuscript.

Table R.1.8 Average coding time for Kodak dataset using an NVIDIA RTX 3090 GPU and Intel Core i9-10980XE CPU @ 3.00GHz. Orange indicates GPU computation, blue indicates CPU computation and bold numbers highlight the best results.

ModelsEncoding timeDecoding timeBD rate
VTM-19.184.22 s\textcolor{blue}{84.22\text{ s}}274.16 ms\textcolor{blue}{274.16\text{ ms}}00
EVC (S/M/L)20.93/34.38/45.56 ms\textcolor{orange}{\mathbf{20.93/34.38/45.56}\text{ ms}}18.80/24.17/31.99 ms\textcolor{orange}{\mathbf{18.80/24.17/31.99}\text{ ms}}7.16%/1.05%/-0.73%
MLIC${++}$257.77 ms\textcolor{orange}{257.77\text{ ms}}334.03 ms\textcolor{orange}{334.03\text{ ms}}-14.96%
HPCM143.9 ms\textcolor{orange}{143.9\text{ ms}}159.8 ms\textcolor{orange}{159.8\text{ ms}}-19.92%
Cool-Chic 5.016.5 sec / 1k steps\textcolor{orange}{16.5\text{ sec / 1k steps}}251.1 ms\textcolor{blue}{251.1\text{ ms}}-6.65%
Cool-Chic 5.0 (PSM)22.1 sec / 1k steps\textcolor{orange}{22.1\text{ sec / 1k steps}}272.8 ms\textcolor{blue}{272.8\text{ ms}}-8.83%
PRISM26.5 sec / 1k steps\textcolor{orange}{26.5\text{ sec / 1k steps}}309.9 ms\textcolor{blue}{309.9\text{ ms}}-9.56%
PRISM (w/o PSM)20.3 sec / 1k steps\textcolor{orange}{20.3\text{ sec / 1k steps}}284.1 ms\textcolor{blue}{284.1\text{ ms}}-7.60%

W4. We thank the reviewer for these suggestions and have addressed the presentation issues as follows:

(a) We have identified the CLIC2020 source image jared-erondu-21325 in the caption of the Figure.

(b) We have clarified the color conventions in Fig. 2. Blue and purple represent the region-specific latents and modules for regions 11 and NN, respectively; red denotes the shared modulator; green indicates the ARM and causal spatial context; and teal represents the context state and its update and reuse pathways.

(c)-(d) We have added algorithmic pseudocode and a summary table in the new Sec. 3.5 to clarify the decoding workflow and key architecture settings. We have also moved the detailed state-update diagram (previously Fig. 9) into the main text.

(e) We have corrected the typo in the caption of Fig. 26 to λ=4×10−4\lambda=4\times10^{-4}.

We thank the reviewer again for the constructive feedback and welcome any further questions or suggestions.

AuthorsOpenReview ↗

Discussion reminder

Dear Reviewer,

Thank you again for your thoughtful review and valuable feedback.

As the discussion phase is approaching its end, we would appreciate it if you could let us know whether any concerns remain or further clarification is needed, so that we may address them before the discussion closes.

Best wishes,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.