- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses overfitted image compression, where each image is encoded by jointly fitting a lightweight decoder and a hierarchical multi-resolution latent, with model parameters themselves contributing to the bitrate. The authors argue that existing overfitted codecs reuse decoded latent values but discard the contextual features formed during entropy modeling, and that preserving these features in a shared state could improve both entropy modeling and synthesis. They propose PRISM, which progressively aggregates the autoregressive entropy model's hidden features E_t into a compact persistent context state C_t (only c=2 channels) as decoding proceeds through the latent hierarchy.
Critical strengths
-
The paper's insight is interesting: It accurately identifies that overfitted codecs have moved from the "increasing capacity" stage to "more efficiently utilizing compact image-specific representations," while explicit modeling of cross-grid dependencies has long been overlooked. The notion that "entropy model features are discarded after use" is a genuine and non-trivial entry point.
-
The proposed architecture is highly lightweight with great application potential: The core module uses only 2 channels, and the memory gate involves merely T+1 scalar logits—Equation (8) shows that the entire state update plus gating requires only 61–75 parameters. This is a rare case where "architectural modifications introduce almost no additional rate or decoding overhead," which is precisely the property most needed for overfitted codecs.
-
The experiments are solid: The authors conducted comprehensive and detailed ablation studies to verify different module sharing strategies. Compression performance and model parameter size were thoroughly compared against various baselines, yielding satisfactory results.
Critical weaknesses
-
Theoretical support. Although the "state modeling" framework is appealing, the paper contains no theorems, propositions, or guarantees. The core update \bar{C}t = \delta C{t-1} + \alpha_t \Delta C_t (Eq. 1), the gating parameterization (Eq. 2), and the entropy residual correction (Eq. 4) are all empirically designed, without any information-theoretic or convergence justification. For ICLR, a framework named after "state modeling"—even with a rough analysis of "what the state retains" or "how errors accumulate across grids"—would significantly enhance persuasiveness.
-
Novelty is combinatorial and needs clearer delineation. PRISM combines the hierarchical entropy model from Cool-chic 5.0, region-wise synthesis from MoRIC, and mature gated recurrence (LSTM/GRU/HGRN). The genuinely new point—accumulating ARM features into a reusable state—is homologous to existing context reuse in transform-based LIC (e.g., LIC-HPCM's hierarchical progressive context modeling, already cited by the authors). The paper should more precisely clarify how PRISM's reuse differs from these precedents, rather than simply stating "applying it to overfitted codecs."
-
Although extensive experiments were conducted, the evaluation datasets are limited, including only Kodak and CLIC. It is recommended to introduce the Tecnick dataset and the DIV2K dataset for further testing. Additionally, only the PSNR metric was reported, which is insufficient; metrics such as MS-SSIM should also be considered.
-
The bottleneck of encoding cost remains unresolved. PRISM still requires per-image iterative fitting (up to 100k main-stage updates), and each iteration must traverse the complete state sequence plus region synthesis. The claimed "5–7× fewer iterations" does not imply a proportional reduction in wall-clock time (as acknowledged by the authors in Appendix E.2). Therefore, the practical encoding cost of overfitted codecs is only alleviated, not resolved.
Questions for authors
- Provide theoretical support. Can we characterize what the state C_t retains across grids, or derive an upper bound on how reconstruction error accumulates as the state propagates?
- Supplementary ablation study: Of the 2.91/4.10 pp relative to Cool-chic 5.0, how much comes from the state mechanism itself, how much from the region-wise synthesis inherited from MoRIC, and how much from the interaction between the two? A matched ablation that "adds only PSM (without MoRIC) to Cool-chic 5.0" would resolve the ambiguity.
- Add experiments with the MS-SSIM metric and the Tecnick dataset. If possible, also include experiments on the DIV2K dataset to validate performance on high-resolution image compression.
- Add encoding/decoding latency experiments, reporting the average encoding and decoding time on Kodak.
8 comments
Part2:
Furthermore, we have conducted additional ablations by incorporating progressive state modeling into other overfitted codecs (Cool-chic 5.0). As shown in Tables R.3.1, The consistent RD gains further demonstrate the effectiveness and generality of the proposed mechanism.
We have clarified these distinctions in the revised manuscript.
Table R.3.1 Effect of accumulated state and previous ARM feature only on complexity and RD performance.
| Setting | NN parameter | Decoding complexity | BD-rate | ||||
|---|---|---|---|---|---|---|---|
| Arm+ifce | upsampling | synthesis | psm | Total | (MAC / pixel) | (vs. w/o PSM) | |
| CC5.0 w/o PSM | 1086 | 60 | 735 | - | 1881 | 1943 | 0 |
| CC5.0 (previous) | 1090 | 60 | 733 | 62 | 1945 | 2053 | -0.81% |
| CC5.0 (Accumulated) | 1090 | 60 | 733 | 67 | 1950 | 2058 | -2.15% |
| PRISM w/o PSM | 1086 | 60 | 722 | - | 1868 | 2196 | 0 |
| PRISM (Previous) | 1090 | 60 | 716 | 64 | 1930 | 2280 | -0.72% |
| PRISM (Accumulated) | 1090 | 60 | 716 | 69 | 1935 | 2286 | -1.98% |
W3. We have extended our evaluation to Tecnick (Tables R.3.2-R.3.3), included additional baselines (Table R.3.4), and reported MS-SSIM results (Table R.3.5) to complement the PSNR-based evaluation.
Table R.3.2 Rate-distortion points on Tecnick dataset (first 10 images, 1200$\times$1200). CC5.0 and PRISM use and 100k iterations.
| Setting | bpp/PSNR (dB) | BD-rate (vs. VTM-19.1) | |||||
|---|---|---|---|---|---|---|---|
| RD point | 1 | 2 | 3 | 4 | 5 | 6 | - |
| HPCM-Large | 0.0541/32.510 | 0.0766/33.871 | 0.1095/35.098 | 0.1605/36.425 | 0.2370/37.748 | 0.3722/39.227 | -25.39% |
| MLIC${++}$ | 0.0554/32.680 | 0.0824/33.980 | 0.1197/35.210 | 0.1854/36.570 | 0.2718/37.790 | 0.4053/39.090 | -19.43% |
| CC5.0 | 0.0414/31.177 | 0.0932/34.300 | 0.1977/36.947 | 0.3384/38.869 | 0.7270/41.807 | - | -21.13% |
| PRISM (ours) | 0.0419/31.503 | 0.0925/34.500 | 0.1920/37.120 | 0.3273/38.977 | 0.7144/41.978 | - | -26.28% |
Table R.3.3 Rate-distortion points on Tecnick dataset (100 images, 1200$\times$1200). CC5.0 and PRISM use and 10k iterations.
| Setting | bpp/PSNR (dB) | BD-rate (vs. VTM-19.1) | |||||
|---|---|---|---|---|---|---|---|
| RD point | 1 | 2 | 3 | 4 | 5 | 6 | - |
| HPCM-Large | 0.0840/31.012 | 0.1234/32.525 | 0.1792/33.973 | 0.2600/35.489 | 0.3687/36.947 | 0.5265/38.442 | -23.76% |
| MLIC${++}$ | 0.0894/31.275 | 0.1332/32.721 | 0.1930/34.100 | 0.2870/35.622 | 0.3987/36.913 | 0.5469/38.188 | -19.65% |
| CC5.0 | 0.0632/29.433 | 0.1559/32.860 | 0.3250/35.943 | 0.5200/38.057 | 0.9907/41.104 | - | -14.57% |
| PRISM (ours) | 0.0634/29.674 | 0.1540/33.076 | 0.3197/36.130 | 0.5071/38.214 | 0.9684/41.266 | - | -19.70% |
Table R.3.4 More baselines on Tecnick dataset (100 images, 1200×1200).
| Image codec | ELIC | STF | TCM-Large | MLIC${++}$ | WeConvene | LALIC | HPCM-Large | CC5.0 | PRISM (ours) |
|---|---|---|---|---|---|---|---|---|---|
| BD-rate over VTM-19.1 | -6.49% | -4.13% | -13.75% | -19.65% | -10.59% | -20.17% | -23.76% | -14.57% | -19.70% |
On Tecnick, PRISM outperforms LIC-HPCM on the first 10 images with 100k fitting iterations and outperforms MLIC${++}$ on all 100 images with only 10k iterations, demonstrating its potential for high-resolution compression. PRISM also maintains comparable performance to ELIC under MS-SSIM.
Table R.3.5 MS-SSIM BD-rate on Kodak, with VTM as the reference.
| System | BD-rate (MS-SSIM) |
|---|---|
| VTM | 0.00% |
| ELIC | -46.94% |
| STF | -49.56% |
| MLIC++ | -54.74% |
| LIC-HPCM | -55.63% |
| PRISM (ours) | -46.19% |
Due to the limited rebuttal time and computational resources, we have not included DIV2K experiments. Nevertheless, our evaluation covers established benchmarks, with Kodak and CLIC2020 following the protocols commonly adopted in prior overfitted codecs [3,4]. In particular, our new Tecnick results provide additional evidence on high-resolution images, which are less frequently evaluated in overfitted compression due to the substantial per-image optimization cost.
[3] Ladune, Théo, et al. Cool-chic: Coordinate-based low complexity hierarchical image codec. International Conference on Computer Vision (ICCV). 2023.
[4] Kim, Hyunjik, et al. C3: High-performance and low-complexity neural compression from a single image or video. Conference on Computer Vision and Pattern Recognition (CVPR). 2024.
Part3:
W4. To address the concern, we include the actual coding time across different architectures in Table R.3.6.
Table R.3.6 Average coding time for Kodak dataset using an NVIDIA RTX 3090 GPU and Intel Core i9-10980XE CPU @ 3.00GHz. Orange indicates GPU computation, blue indicates CPU computation and bold numbers highlight the best results.
| Models | Encoding time | Decoding time | BD rate |
|---|---|---|---|
| VTM-19.1 | |||
| EVC (S/M/L) | 7.16%/1.05%/-0.73% | ||
| MLIC${++}$ | -14.96% | ||
| HPCM | -19.92% | ||
| Cool-Chic 5.0 | -6.65% | ||
| Cool-Chic 5.0 (PSM) | -8.83% | ||
| PRISM | -9.56% | ||
| PRISM (w/o PSM) | -7.60% |
Compared with PRISM without PSM, incorporating PSM increases decoding latency by ms and encoding time by s per iterations. However, Fig. 7 (previously Fig. 6) shows that PRISM achieves comparable RD performance to Cool-chic 5.0 on Kodak with approximately fewer encoding iterations. Accounting for the measured per-iteration costs, the estimated fitting time is s () for PRISM and s () for Cool-chic 5.0, yielding a speedup. Thus, under these settings, faster convergence more than offsets the additional per-iteration cost.
Overall, PRISM improves RD performance and encoding efficiency at the cost of moderately increased decoding latency. Further engineering optimizations may reduce the runtime overhead of PSM, as our current implementation uses unoptimized research code. We have incorporated these results and discussions into the revised manuscript.
Answers to questions:
Q1. In our response to W1, we analytically characterize the information retained across latent grids and examine the state's effects on bitrate and reconstruction quality (Fig. 9, originally Fig 8)).
Q2. Following the reviewer's suggestion, we add a matched ablation "CC5.0 + PSM (w/o MoRIC)" in Table R.3.7.
Table R.3.7 Ablation study on Kodak, with full PRISM as the BD-rate reference.
| System | BD-rate |
|---|---|
| PRISM (full) | 0.00% |
| CC5.0 | +2.95% |
| CC5.0 + PSM (w/o MoRIC) | +0.74% |
| w/o State modeling | +2.02% |
| w/o Synthesis state reuse | +1.69% |
| w/o Learned memory gating (, ) | +0.81% |
| w/o Region conditioning | +0.62% |
From the table, it is clear that the state mechanism alone provides approximately percentage points of improvement, while regional synthesis inherited from MoRIC provides approximately points. Combining both yields an overall improvement of approximately percentage points. Moreover, PRISM achieves larger gains on high-resolution images, as evidenced by the CLIC2020 and Tecnick results (Tables R.3.2 and R.3.3).
To further examine the contribution of progressive accumulation, we include additional ablations in Table R.3.1. The results demonstrate the significant benefits of progressive context accumulation.
We have incorporated these ablations and discussion into the revised manuscript.
Q3. In our response to W3, we provided additional Tecnick evaluations, broader baseline comparisons, and MS-SSIM results (Tables R.3.2-R.3.5).
Q4. In our response to W4, we reported measured encoding and decoding time, and analyzed the overhead introduced by PSM (Table R.3.6)
We thank the reviewer again for the constructive feedback and welcome any further questions or suggestions.
Part1:
We thank the reviewer for the constructive feedback. In response, we have (a) provided an analytical characterization of progressive state accumulation; (b) clarified PRISM's distinctions from prior work along with additional ablations (Tables R.3.1 and R.3.7); (c) extended the evaluation to Tecnick, additional baselines, and MS-SSIM (Tables R.3.2-R.3.5); and (d) reported measured fitting and decoding times (Table R.3.6). We address each concern and question in detail below.
W1. (a) Following the suggestion, we provide an analytical characterization of the information retained in the context state across latent grids by explicitly expanding the linear transformations in the state update.
At grid t, the state aggregates ARM features Et, and pooled region masks rt. Since Gη is affine, the state update in Eq. (1) can be expressed as
ΔCt=WCCt−1+WEEt+Wrrt+bG, where the weights WC, WE, and Wr act channel-wise on the incoming state, ARM features, and region features, respectively, and bG is the corresponding bias.
Let Pt and bU,t denote the linear component and bias of the resolution-alignment operator at grid t. Combining the gated update with alignment yields the following theoretic result Ct=Propagated previous stateAt(Ct−1)+Current-grid contributionqt This decomposition is explained as follows. Here, At(Ct)=Pt((δI+αtWC)Ct) propagates previously accumulated information, while qt=αtPt(WEEt+Wrrt+bG)+bU,t introduces the current grid's projected features and the bias terms, I is the identity matrix.
Starting from C0=0, recursively expanding the updates gives: C1=q1 and C2=A2(q1)+q2, C3=A3(A2(q1))+A3(q2)+q3. The final state is
CT=qT+AT(qT−1)+AT(AT−1(qT−2))+⋯+AT(AT−1(⋯A2(q1)⋯)),
This decomposition expresses CT as an accumulation of contributions qk from all latent grids, each incorporating ARM features and region information, where k means latent grid index. The transition operators At determine how these contributions are transformed and retained across subsequent grids, thereby characterizing the mechanism of progressive context accumulation. This interpretation is related to recurrent aggregation in state-space models such as Mamba [1] and MambaIC [2].
[1] Gu, A. and Dao, T., 2023. Mamba: Linear-time sequence modeling with selective state spaces.
[2] Zeng, Fanhu, et al. Mambaic: State space models for high-performance learned image compression. Conference on Computer Vision and Pattern Recognition (CVPR). 2025.
W1. (b) We clarify that intermediate states condition subsequent entropy decoding, while only the final state is used for image synthesis.
Since no intermediate reconstructions are generated during state propagation, reconstruction error cannot be directly measured at each stage, making its accumulation error difficult to characterize. We therefore assess the contribution of each stage empirically through state visualization and ablation (Fig. 9, previously Fig 8).
W2. We agree that PRISM builds upon existing entropy models and region-wise synthesis. However, its key distinction lies in reusing entropy-model representations for both probability estimation and image synthesis under the limited model-rate and decoding-complexity constraints of overfitted compression.
Specifically, PRISM progressively reuses ARM features with minimal parameter overhead. Unlike LIC-HPCM, which primarily reuses hierarchical context for entropy estimation, PRISM further exposes the accumulated state to image synthesis, allowing both rate and distortion objectives to jointly optimize the shared representation. Our ablations (Table 3, previously Table 1) confirm the benefits of progressive accumulation and synthesis-side reuse beyond direct feature reuse or increased model capacity.