acceptodds
Under review as a conference paper at ICLR 2027

Rethinking Generative Image Compression at Extremely Low Bitrates

Abstract

Generative image compression produces visually plausible reconstructions at low bitrates, yet their behavior as the rate approaches zero remains largely unexplored. When pushed below normal operating rates, representative codecs undergo semantic collapse: rather than gracefully losing source-specific detail, they produce malformed or unrecognizable content. Our analysis identifies two factors. As the bitrate decreases, reconstruction losses increasingly conflict with semantic objectives on gradients and visual results, while pixel-space and reconstruction-oriented VAE diffusion models become less efficient on semantic preservation. Guided by these findings, we introduce RAE-CoD, a compression-oriented diffusion (CoD) built in a representation autoencoder (RAE) space with direct alignment between compressed and source representations, preserving recognizable, naturally structured content for a image with as few as 16 bits. We evaluate this framework using five vision foundation models (VFM) and a blinded vision-language model protocol. On MSCOCO-30K, RAE-CoD stands out from all evaluation. At 0.001-0.008 bpp, it reduces relative VFM feature MSE and Fr\'echet Distance ratio by at least 25.7% and 69.1% over the best competitors. Meanwhile, semantic recognizability and quality of the reconstructions remain nearly constant while source consistency falls smoothly, replacing abrupt semantic collapse with a graceful transition toward unconditional generation. Code will be available.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Generative Image Compression, Extremely Low Bitrate, Diffusion Model

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_5mzrOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a generative image codec that operates at extremely low bitrates, where decoding becomes close to unconditional generation. The correlation between reconstruction-oriented and semantic training objectives is analyzed (Fig. 2, 3), and pixel space, VAE latent space, and RAE latent space are compared for zero-shot compression (Fig. 4, 5). The key observation from these analyses is that the RAE latent space preserves high-level semantics, as evaluated by vision foundation models (VFMs), better than pixel or VAE latent spaces. Based on this, a generative codec, RAE-CoD, consisting of an RAE, a latent codec, and a diffusion transformer (DiT) is proposed. The main distinction from previous generative image codecs is that it performs coding in the RAE latent space rather than in pixel or VAE latent space. Training follows the RAE training convention, with a newly introduced auxiliary loss to compensate for the training instability caused by the rate constraint. For the same reason, training proceeds progressively from high to low bitrates. For evaluation, VFM-based metrics and a VLM-based protocol, which separately assesses semantic recognizability, quality, and consistency (SR, SQ, SC), are used. With these, RAE-CoD achieves higher scores than recent generative image codecs, and it is also preferred by participants in a user study.

Critical strengths

S1: I understand the core claim of this paper to be that existing generative codecs suffer from semantic collapse at extremely low bitrates, and that RAE-space compression resolves this collapse. This claim is well supported by the experimental results. If semantic collapse had been evaluated only by distances or similarities in the VFM feature space, the evaluation could have fallen into circular reasoning, since the RAE itself is built on VFM representations. However, by also evaluating the models with the VLM-based protocol, in particular semantic consistency (SC), and showing the advantage of RAE-CoD there, the paper avoids this circularity, and the claim is well supported.

Critical weaknesses

W1: Compared with Pixel-CoD, VAE-CoD, and AEIC-ME, the proposed RAE-CoD achieves consistently better semantic consistency (SC), as judged by a VLM (Fig. 8). However, SC is still a model-based evaluation. To determine whether a higher SC actually delivers a better reconstruction to humans, which is the ultimate goal, the user study is the most important result. Yet the user study is conducted in a limited setting, and details about its protocol are largely missing.

  • First, it is not stated whether the original image was presented to participants together with the stimuli. Since the proposed model is a generative codec, the study should evaluate how well the reconstruction conveys the original image (i.e., a codec evaluation). Without the original, it may only measure how plausible the image itself looks (i.e., a generation-quality evaluation).

  • Second, the user study should at least be conducted at the original resolution rather than on 256×256 crops. On resized and center-cropped low-resolution images, high-level semantic differences tend to dominate low-level distortion in participants' ratings. I do not consider this an optional additional experiment; I consider it essential for establishing the value of this paper.

Questions for authors

I am willing to raise my rating if both concerns in W1 are resolved. If the original image was shown to participants together with the stimuli, and if the authors present user-study results at a resolution higher than 256×256 or convincingly show that results at 256×256 are sufficient.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 5mzr

We sincerely thank the reviewer for recognizing the importance of evaluating semantic preservation beyond VFM feature distances. We also appreciate the critical comments regarding the user study. We provide detailed clarifications on the experimental protocol and the rationale behind the resolution below.

W1, Q1: Implementation of the user study.

We would like to confirm that the original image was indeed presented to participants alongside the reconstructed images in every test.

Specifically, each test trial presented three 256×256 images on the same display: the original image, the reconstruction produced by RAE-CoD, and the reconstruction produced by a competing method. The original image was displayed at the top and explicitly labeled as the ground truth image, while the two reconstructed images were placed side by side underneath it. Their left-right positions were randomized across trials to avoid positional bias. Participants were instructed to select the reconstruction whose content was more consistent with the ground truth image shown above. No additional textual hints or method-specific information were provided in the interface. The voting results were automatically recorded to calculate the preference rates of RAE-CoD against each competing method.

We acknowledge that these details were not sufficiently explained in the original manuscript due to the page limit. We will provide a more explicit description of the user study implementation in the revised final version.

W1, Q2: Rationale for conducting the user study at 256×256 resolution.

We agree with the reviewer that, on 256x256 images, high-level semantic differences tend to dominate low-level distortion in participants' ratings. we would like to clarify that emphasizing high-level semantic consistency is intentional and directly aligned with the central objective of this work. Our user study is specifically designed to investigate semantic preservation under extremely low-bitrate compression, instead of assessing low-level distortion (which becomes less meaningful at extreme bitrates due to the intrinsic rate-distortion tradeoff). We believe that using 256×256 images is appropriate for this particular evaluation objective for two reasons.

First, at this resolution, the main semantic content of an image remains sufficiently recognizable, allowing participants to focus on whether the essential objects, scenes, and their semantic relationships are faithfully preserved. This is particularly relevant to our study, which aims to assess semantic consistency rather than fine-grained texture or pixel-level fidelity.

Second, higher-resolution images generally contain more spatial redundancy and fine-grained visual details. Under extremely low-bitrate compression, these additional details introduce substantial low-level distortions, which may influence human preferences independently of the general semantic quality and consistency that we intend to investigate. In contrast, evaluating images at 256×256 provides a more focused comparison of the semantic content conveyed by different reconstructions as demonstrated in Fig. 1 and Fig. 9, reducing the influence of fine-grained details on the assessment.

We acknowledge that conducting user studies at higher resolutions would provide additional insights. However, our current framework relies on the pretrained RAEv2 [1] DiT and decoder, whose publicly released checkpoints and code include only 256×256 resolution. Directly applying these models to higher-resolution images does not yield meaningful reconstructions, while finetuning RAEv2 DiT and decoder at higher resolutions with official 256x256 training strategies requires substantial computational resources and is beyond the scope of this work. Therefore, we leave high-resolution human evaluation as an important experiment for future work.

Reference:
[1] Singh et al., 2026, "Improved baselines with representation autoencoders".

Reviewer_5mzrOpenReview ↗

Response to Athors

Thank you for confirming that the original image was presented alongside the reconstructions in every trial. This resolves my first concern.

Regarding the resolution, while the emphasis on high-level semantic consistency is well aligned with the central objective of this work, it does not by itself justify restricting the evaluation to 256×256. The response notes that this resolution reduces the influence of fine-grained details on the assessment, but such a reduction is itself a source of bias in a human evaluation of a codec.

I fully recognize that the framework builds on the pretrained RAEv2 DiT and decoder, whose released checkpoints and code are limited to 256×256. For this reason, the absence of a high-resolution evaluation is better understood as a limitation of the current study than as an intentional design choice.

To be clear, I regard this paper as a strong contribution. If the authors clearly acknowledge this limitation of the experimental setting, I am willing to raise my rating.

AuthorsOpenReview ↗

Response to Reviewer 5mzr: The manuscript has been revised and updated

Thank you very much for your response and the suggestion. We acknowledge this resolution limitation on our current experimental setting, and we have revised the submission accordingly with all changes highlighted in blue.

Specifically,

  1. "Limitations" in Section 7 Conclusion now explicitly acknowledges that the current experimental settings are limited to 256x256 images.
  2. "Human Preference Study" in Section 5.2 Main Results now contains the detailed implementation of the user study as presented in the previous response.

Please let us know if you have any further concerns. We would be willing to address them promptly.

Reviewer_5mzrOpenReview ↗

Response to Athors

Thank you for the quick revision. It addresses my remaining concern, and I have raised my rating accordingly.

Reviewer_X9fdOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper investigates generative image compression in the regime between conventional operating rates and zero bits. The authors identify a failure mode they call semantic collapse: as bitrate decreases, existing codecs produce outputs that are neither source-consistent nor naturally formed—objects deform, entities disappear, and scene meaning changes abruptly.

They introduce RAE-CoD, a compression-oriented diffusion model built in the semantic space of a Representation Autoencoder (RAE). It uses DINOv3 as the clean representation target, a deep latent codec that fuses pixel and representation features into entropy-coded latents, and a conditional Decoupled Diffusion Transformer (DDT) that generates the source representation from the decoded condition. A deterministic auxiliary loss directly aligns the codec condition with the clean representation, bypassing the ambiguous signal from reconstruction objectives. Training is progressive: Stage I increases the rate penalty with LoRA on the pretrained DDT; Stage II fixes the rate and fine-tunes the full model jointly with the codec.

The paper also introduces a two-level evaluation protocol: VFM-based metrics for paired and distributional semantic distance, and a blinded VLM judge (Qwen3.5-9B) that disentangles semantic recognizability (SR), semantic quality (SQ), and semantic consistency (SC).

Critical strengths

1、The paper addresses a genuinely underexplored regime—between conventional operating rates and zero bits.

2、The gradient geometry analysis (Fig. 2) and the decoder-adaptation test (Fig. 3) provide concrete evidence that reconstruction objectives can oppose semantic objectives, especially at low rates. The zero-shot DiffC comparison across pixel/VAE/RAE spaces (Fig. 4–5, Fig. 13) isolates the effect of diffusion space cleanly.

3、RAE-CoD directly follows from the analysis: use a representation space (RAE), align the codec condition directly with the clean representation (L_aux), and avoid reconstruction-oriented supervision of the codec. The progressive training schedule (LoRA Stage I → full fine-tune Stage II) is sensible and well-documented.

4、The two-level protocol is a significant contribution. Separating SR, SQ, and SC with a blinded VLM judge, and normalizing with per-image JPEG/identity anchors, addresses a real gap: conventional metrics cannot distinguish "recognizable but wrong" from "collapsed." The VFM ensemble (Inception-v3, ConvNeXt-v2, DINOv2, SigLIP2, CLIP) covers diverse inductive biases, and the linear probing sanity check (Table 4) confirms each space carries high-level semantics.

Critical weaknesses

1、The paper does not compare against the strongest recent generative codecs at matched rates. The baselines are ResULIC, PerCo-SD, CoD, AEIC-ME, DiT-IC, Pixel-CoD, and VAE-CoD. Missing are StableCodec (Zhang et al., 2025), OSCAR (Guo et al., 2025), OneDC (Xue et al., 2025), and especially CoD-Lite (Jia et al., 2026a), which is designed for real-time one-step decoding. Several of these report operating points below 0.01 bpp. Without these, the claim of "strongest semantic fidelity in the evaluated range" is not fully established.

2、All quantitative results are at 256×256. The paper acknowledges this in limitations, but it is a significant restriction: semantic preservation at high resolution may behave differently (more tokens, more spatial structure to preserve, different rate-distortion tradeoffs). The 16-bit result at 256×256 is impressive, but it is unclear whether the same holds at 512×512 or 1024×1024.

3、The VLM-based metrics, while innovative, are not validated against human judgment at scale. The paper reports a human preference study (Fig. 10, 120 votes from 10 participants on Kodak), but this validates only overall preference, not the specific SR/SQ/SC decomposition. Do human raters agree with Qwen3.5-9B on what is "recognizable" vs. "consistent"? A correlation study between VLM scores and human ratings on a subset would strengthen confidence in the protocol. Without it, the SR/SQ/SC numbers are a diagnostic tool of uncertain calibration.

4、The rate at which RAE-CoD operates is extremely low, but the paper does not discuss the practical use case. At 0.000244–0.008 bpp, a 256×256 image costs 16–512 bits. This is far below any practical image compression scenario (e.g., thumbnails, previews, or semantic hashing). The paper frames this as "pushing the lower bitrate frontier," which is fair, but a discussion of what applications this enables (or why the regime matters) would help. Is this a step toward semantic communication, extreme thumbnail generation, or a theoretical probe?

5、The auxiliary alignment loss L_aux is applied to the codec condition c, but the paper does not ablate its placement. Table 1 ablates the type of auxiliary objective (none, pixel MSE, semantic, both), but not where it is applied (condition c vs. the generated output x̂₀ vs. intermediate features). Since the paper's central claim is that direct alignment of the codec condition with the clean representation is key, an ablation on the alignment target would strengthen the argument.

6、The one-step DiffC experiment is presented as a 20-bit result, but the paper acknowledges it is theoretical. Appendix C.3 states: "It does not yet serialize a standalone DiffC fragment or count its arithmetic-coder termination, metadata, byte-alignment, and container overhead. The 20-bit point should therefore be interpreted as a theoretical-rate analysis." This is honest, but Fig. 11 presents it alongside measured 16-bit results without a clear visual distinction. The paper should either (a) implement the serialization to report a real rate, or (b) clearly mark the 20-bit point as theoretical in all figures.

7、The progressive training schedule uses a specific set of rate weights {0.1, 2, 12, 16, 24, 32, 48}. The paper does not ablate this schedule. Would a different schedule (e.g., geometric progression, or fewer stages) work as well? The choice of 30K-iteration intervals is also not justified.

8、The 4-bit VQ codebook for the hyper latent z is a specific design choice. The paper notes that "at extreme rates, the factorized model for z can dominate the bits," motivating VQ. But it does not compare against alternatives (e.g., a stronger factorized model, or a different codebook size). An ablation on codebook size (2-bit, 4-bit, 8-bit) would be informative.

9、The pixel encoder architecture is described as "Inception-style depthwise convolutions and gated channel mixing" (following Zhang et al., 2025), but the paper does not analyze the contribution of individual components. Table 2 shows that fusing pixel and RAE features helps, but not why—which pixel-level features complement the RAE representation?

10、The internal guidance formula (x_base + 1.78·(x_full − x_base)) is adopted from Singh et al. (2026) without ablation. Is this guidance necessary? What happens with CFG instead?

11、The VLM judge is Qwen3.5-9B, a single model. The paper does not test robustness to the choice of VLM judge. Would GPT-4V, Claude, or Gemini produce similar SR/SQ/SC scores? A cross-model correlation study would strengthen the protocol.

12、The human preference study has only 10 participants and 120 votes per comparison. This is a small study by modern standards. The paper should report confidence intervals or statistical significance.

13、The paper does not report PSNR or MS-SSIM. While the paper correctly argues that these are not the right metrics for this regime, reporting them (even in an appendix) would allow comparison with rate-distortion-oriented codecs and clarify the perception-distortion tradeoff.

14、The FD ratio (FDr⁵) normalizes by an anchor codec, but the paper does not specify which anchor is used. Appendix A.1 mentions "an anchor codec" but does not name it. This should be clarified.

15、The paper claims "perfect realism" toward 16 bits, but does not quantify realism. FID is not reported. The VLM's SQ measures intrinsic quality, but a distributional realism metric (FID or KID) would complement it.

16、The decoder is 0.9B parameters (DDT) plus a 57.2M codec plus frozen DINOv3 and RAEv2 decoder. The total inference cost is not reported.

Questions for authors

See the weaknesses part

3 comments
AuthorsOpenReview ↗

Response to Reviewer X9fd (Part 1. W1-4)

We sincerely thank the reviewer for thorough evaluation of our work and for recognizing its contributions, particularly the investigation of semantic collapse at extremely low bitrates, the analysis of conflicting reconstruction and semantic objectives, and the proposed RAE-space compression framework. We have carefully considered all the comments and provided detailed responses and clarifications below.

W1. Comparison with StableCodec, OSCAR, OneDC, and CoD-Lite.

We would like to clarify that StableCodec [1], OSCAR [2] and CoD-Lite [3] are weaker baselines. Our experiments already include frontier generative compression methods, including CoD [4], AEIC-ME [5], and DiT-IC [6]. In particular, AEIC-ME and DiT-IC have demonstrated substantially better compression performance than earlier methods such as StableCodec and OSCAR in their respective original papers. Similarly, CoD-Lite is a lightweight variant of CoD, primarily designed to reduce model complexity and improve decoding efficiency. Since our work focuses on semantic preservation at extremely low bitrates rather than lightweight or real-time decoding, we consider the original CoD a more appropriate comparison.

Regarding OneDC [7], it is not specifically designed to investigate the extremely low-bitrate regime considered in our work. Nevertheless, we have evaluated RAE-CoD against OneDC at their lowest available operating points on MSCOCO-30K to provide an additional reference.

Methodbpp ↓RelMSE⁵ ↓COS⁵ ↑FDr⁵ ↓
OneDC0.003421.2350.4691.374
RAE-CoD (Ours)0.001980.7880.6210.304

W2. Evaluation at higher resolutions.

The resolution limitation primarily originates from the pretrained generative backbone. The initial RAE [8] reported the feasibility of training and deploying at 512×512 resolution. However, RAE-CoD relies on the pretrained RAEv2 [9] DiT and decoder, for which publicly released checkpoints and code are available only at 256×256 resolution. Directly applying these checkpoints to higher-resolution inputs does not produce meaningful reconstructions. Training RAEv2 DiT and decoder at higher resolutions also requires substantial additional resources and a dedicated large-scale pretraining effort, which is beyond our current computational budget and the scope of this work.

We would also like to emphasize that our central objective is to investigate semantic preservation at extremely low bitrates, rather than to establish a resolution-scalable generative codec. As acknowledged in the Critical Strength 1, this operating regime between conventional compression rates and zero bits is genuinely underexplored. RAE-CoD substantially alleviates semantic collapse even at 256×256 resolution and achieves meaningful semantic reconstruction at rates as low as 16 bits per image. We believe these results already provide important evidence for the central contribution of this work. Extending RAE-CoD to higher resolutions is therefore an important direction for future research.

W3. Human validation on VLM-based semantic metrics.

We do not claim that the VLM-based evaluation aligns with human perception, and such kind of validation is beyond our research scope.

Instead, VLM-based evaluation serves as one independent aspect of our complementary evaluation framework to analyze semantic quality and preservation at extreme bitrates that cannot be adequately characterized by conventional metrics alone. Specifically, our evaluation incorporates several complementary perspectives: (1) semantic distances and (2) distributional distances measured by VFMs, (3) VLM's perception of semantic properties, and (4) human preference. These evaluation dimensions provide complementary evidence for our central observation that existing generative codecs suffer from semantic collapse at extremely low bitrates, while RAE-CoD alleviates this issue.

W4. Practical value of RAE-CoD.

RAE-CoD can be viewed as a step toward semantic communication. For example, in deep-sea exploration and outer-space satellite communication, bandwidth can be extremely limited, and transmitting complete visual observations can be expensive. In such settings, transmitting a highly compact representation that preserves the essential semantic content of an observation, followed by generative reconstruction at the receiving end, could provide a promising alternative to conventional codec.

From a more fundamental perspective, our work also explores how many bits are required to preserve the essential semantic content of a visual observation. By pushing the bitrate towards the extreme lower bound (16 bits for 256x256 images), RAE-CoD provides a framework for investigating the relationship between semantic compression and generative reconstruction.

AuthorsOpenReview ↗

Response to Reviewer X9fd (Part 2. W5-12)

W5. Ablation on where Laux\mathcal{L}_{aux} is applied, such as the codec condition, generated output, or intermediate features.

We would like to clarify that Laux\mathcal{L}_{aux} is specifically designed to supervise the codec condition cc, while Table 1 already investigates the choice and effectiveness of this auxiliary objectives. Applying Laux\mathcal{L}_{aux} to other locations changes the objective and the role of this auxiliary loss. As illustrated in Fig. 6, the generated output x^0\hat{x}_{0} and intermediate features already receive their valid supervision through x-prediction flow matching and REPA, respectively.

W6. Presentation of Figure 11.

We will revise the presentation of Fig. 11 to explicitly distinguish the results, labeling the 16-bit point as a measured bitrate and the 20-bit point as a theoretical bitrate.

W7. Validation on the progressive training schedules.

Our progressive training schedule follows the implicit bitrate pruning strategy introduced in StableCodec [1], as already stated in Lines 276–278 of the original manuscript. Similarly, the 30K-iteration interval is an empirically selected training configuration that provides satisfactory performance in our experiments. These are not methodological contributions of this work.

W8. Ablation on 4-bit VQ codebook.

The 4-bit VQ codebook is an architectural design choice for controlling the minimum achievable bitrate (an exact minimum bitrate of 16 bits per 256×256 image), rather than a component introduced to improve semantic compression. As discussed in previous studies [1, 10, 11], factorized hyperprior models can become a significant source of bitrate overhead at extremely low rates. Our VQ design addresses this limitation by explicitly controlling the cost of the hyper latent. Exploring an optimal hyper-latent quantization configuration is not our current focus.

W9. Analysis on the pixel encoder.

The pixel encoder is inherited from prior work (StableCodec [1] has already investigated the role of this architecture) and is not the main methodological contribution of RAE-CoD. Table 2 also provides an ablation to validate its effectiveness.

W10. The internal guidance (IG) and CFG.

RAE-CoD follows the inference framework of RAEv2, including its internal guidance (IG) mechanism. The guidance strategy and its relationship to classifier-free guidance (CFG) have been investigated in the original RAEv2 framework. We conduct additional experiments on Kodak at three bitrate points, comparing inference with no guidance, with CFG, and with IG.

0.000244 bppRelMSE⁵↓COS⁵↑FDr⁵↓0.001829 bppRelMSE⁵↓COS⁵↑FDr⁵↓0.007848 bppRelMSE⁵↓COS⁵↑FDr⁵↓
None1.0410.4751.355None0.7370.6350.992None0.4660.7680.691
CFG1.0390.4811.341CFG0.7420.6350.993CFG0.4410.7800.662
IG1.0140.4871.339IG0.6880.6570.966IG0.4380.7820.648

W11. The VLM judge with GPT-4V, Claude, or Gemini.

We adopt Qwen3.5-9B, a representative and frontier open-source VLM, for our VLM-based evaluation. One important consideration is reproducibility: using an publicly available model allows the community to reproduce and further investigate our evaluation protocol without depending on proprietary APIs or potentially changing model-access conditions.

We acknowledge that evaluating the robustness of the proposed protocol across different VLM judges would be valuable. However, incorporating multiple proprietary VLMs, such as GPT-4V, Claude, and Gemini, is not necessary for establishing the central claims of this work and would introduce additional dependencies on closed-source evaluation systems.

W12. The confidence intervals or statistical significance of the user study.

Our user study follows the established evaluation setup of prior studies [1, 5, 6, 11, 12] to report only the preference ratio. We conduct pairwise preference comparisons between RAE-CoD and each competing method, and participants are asked to select the reconstruction whose content is more consistent with the reference image. The current votes have demonstrated a consistent preference for RAE-CoD, supporting its advantage in preserving semantic content under extremely low-bitrate compression.

AuthorsOpenReview ↗

Response to Reviewer X9fd (Part 3. W13-16)

W13. Report PSNR and MS-SSIM.

We supplement the PSNR and MS-SSIM results of RAE-CoD on Kodak as a reference. Note that low-level distortion metrics become less meaningful for quality assessment at extremely low bitrates, as discussed in our Figs. 2–5 and previous work [1, 2, 5, 6, 10]. At such low rates, similarly poor PSNR and MS-SSIM scores can correspond to fundamentally different reconstruction outcomes, including blurring, structural distortions, incorrect textures, or even completely altered semantic content. Consequently, PSNR and MS-SSIM alone provide little insight into quality evaluation, and become largely non-informative in the extremely low-bitrate regime considered in this work.

Bpp0.0002440.0005450.0011730.0018220.0038800.007828
PSNR10.2110.9511.0911.5012.6213.12
MS-SSIM0.08900.14350.17380.17740.25270.3545

W14. The anchor codec of FD ratio is not specified.

We would like to clarify that the anchor codec used for normalizing the FD ratio is the highest-bitrate operating point of AEIC-ME. This information is already stated in Lines 371–372 of the original manuscript.

W15. Report distributional realism metric like FID.

We would like to clarify that the proposed evaluation already includes FID. Specifically, Fig. 7 reports the Fréchet Distance (FD) computed in the Inception-v3 feature space, which corresponds to the conventional Fréchet Inception Distance (FID). Furthermore, our evaluation extends beyond Inception-v3 by reporting Fréchet distances in several additional feature spaces, including ConvNeXt-v2, DINOv2, SigLIP2, and CLIP. These complementary distributional metrics provide a broader assessment of the statistical similarity between reconstructed and original image distributions.

W16. Inference cost.

We measure RAE-CoD's encoding and decoding runtime on an NVIDIA A100 GPU using Kodak images at 256×256 resolution. The average encoding time is 44.21 ms per image, while the average decoding time is 1214.40 ms per image. Note that our main goal is to investigate generative compression at extremely low bitrates, rather than to optimize inference efficiency. We therefore report these runtime measurements as a reference.

Reference:
[1] Zhang et al., 2025, "Stablecodec: Taming one-step diffusion for extreme image compression".
[2] Guo at al., 2025, "OSCAR: One-Step Diffusion Codec Across Multiple Bit-rates".
[3] Jia et al., 2026, "CoD-Lite: Real-Time Diffusion-Based Generative Image Compression".
[4] Jia et al., 2025, "CoD: A Diffusion Foundation Model for Image Compression".
[5] Zhang et al., 2026, "Ultra-Low Bitrate Perceptual Image Compression with Shallow Encoder".
[6] Shi et al., 2026, "DiT-IC: Aligned Diffusion Transformer for Efficient Image Compression".
[7] Xue et al., 2025, "One-Step Diffusion-Based Image Compression with Semantic Distillation".
[8] Zheng et al., 2026, "Diffusion transformers with representation autoencoders".
[9] Singh et al., 2026, "Improved baselines with representation autoencoders".
[10] Jia et al., 2024, "Generative Latent Coding for Ultra-Low Bitrate Image Compression".
[11] Guo et al., 2025, "Generative Latent Video Compression".
[12] Mentzer et al., 2022, "Neural video compression using gans for detail synthesis and propagation".

Reviewer_ZYv9OpenReview ↗
Rating
4
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper considers image compression at extremely low bitrates (0.001 - 0.008 bits per pixel). A novel method is proposed based on RAE generation and compression in semantic latent spaces of semantic encoders like DINO, coupled with a pixel decoder that maps the semantic features to pixels. The paper follows the idea of earlier generative image compression work: to aim for perfect realism regardless of the bitrate, while ensuring maximum faithfulness to the original image as allowed by the bitrate. Compared to prior work, significant gains are obtained in semantic feature-based reconstruction (MSE in feature space, Frechet distance ratio), which seems consistent with defining the compression problem in semantic feature spaces. In addition VLM-as-a judge results are reported, which show smooth degradation of the consistency with the source, while maintaining image quality. Experiments compare the proposed approach to alternatives using either VAE latent space or direct pixel space generative models.

Critical strengths
  • The paper explores image compression at extremely low bitrates, beyond what has been explored in prior work as far as I know.

  • The proposed approach leverages recent advances in generative image models and combines this with a modified hyper-prior model, yielding improved compression performance at extremely low bitrates.

Critical weaknesses
  • It is not clear that a learned compression model generalize to other resolutions.

  • The authors do not comment on how compute scales with the image resolution, and include only compression results at 256 resolution.

  • The evaluations do not include runtime analysis of the proposed and compared-to methods.

  • The definition of the SC score is unclear (lines 320-323).

Questions for authors
  • Can you include results where a model trained at 256 resolution is deployed to compress higher resolution images.

  • Can you include additional results and analyses with models trained at 512 or higher image resolution.

3 comments
AuthorsOpenReview ↗

Response to Reviewer ZYv9

We sincerely thank the reviewer for the positive assessment of our work, particularly for recognizing its exploration of image compression at extremely low bitrates. Our point-by-point responses are provided below.

W1, Q1: Can RAE-CoD trained at 256 resolution generalize to higher resolution?

We have investigated the direct deployment of RAE-CoD, trained at 256×256 resolution, on higher-resolution images. Our experiments indicate that the model cannot be directly applied to higher resolutions (e.g., 512×512) while maintaining meaningful reconstructions. This limitation primarily stems from its reliance on the pretrained RAEv2 [2] DiT (provides generative priors) and decoder (remains frozen in RAE-CoD). Specifically, RAEv2 is only trained and validated at 256×256 resolution, corresponding to a 16×16 latent grid. Directly applying it to 512×512 images expands the latent grid to 32×32. Such a change introduces a substantial distribution shift which the original pretrained model has not been trained to handle. Consequently, directly deploying either the original RAEv2 or our RAE-CoD at 512×512 resolution does not produce meaningful reconstructions.

W2, Q2: Train and analyse RAE-CoD at 512 resolution or higher.

Regarding the feasibility of training at higher resolutions, the initial RAE [1] has reported successful training at 512×512 resolution, suggesting the RAE-based architecture itself is not fundamentally restricted to 256×256 inputs. In principle, RAE-CoD can be extended to higher resolutions. However, our approach relies on the sufficiently pretrained DiT and decoder from RAEv2, which only reports checkpoints and training for 256×256 resolution. Finetuning RAEv2 DiT and decoder at 512×512 resolution requires substantial additional computational resources (the original RAEv2 training setup on 256x256 images involves a batch size of 1024 and at least 32 NVIDIA H100 GPUs). Adapting such a large-scale training pipeline requires a substantial independent research effort and is beyond the scope of our current work.

We would also like to clarify the motivation for focusing on 256×256 resolution. From the perspective of image compression, high-resolution images generally contain more spatial redundancy and can therefore support more extreme compression. For instance, existing generative image compression methods [3, 4] have demonstrated that 2K and 4K images can retain relatively good perceptual reconstruction quality even at 0.005 bpp. In contrast, compressing 256×256 images at the same bitrate leads to severe semantic collapse and visual issues, as illustrated in Fig. 1 and Fig. 9. This makes 256×256 images particularly challenging and suitable for exploring compression efficiency at extremely low bitrates.

Motivated by this observation, we initially focus on compressing 256×256 images at bitrates as low as 0.000244 bpp. Extending RAE-CoD to higher-resolution images is an important direction for our ongoing research and future work.

W2, W3: How compute scales with resolution and runtime analysis.

For reference, we have supplemented the complexity measurements (MACs per pixel) and runtime latency comparisons at 256×256 resolution. All runtime measurements are conducted on an NVIDIA A100 GPU. RAE-CoD incurs relatively high decoding complexity due to iterative denoising, as our current main goal is to investigate the potential of generative compression at extremely low bitrates rather than efficiency.

MethodEnc. (KMACs/pixel)Enc. (s)Dec. (KMACs/pixel)Dec. (s)
PerCo-SD2,193.90.0566,904.30.88
ResULIC2,140.10.0517,864.90.63
CoD263.40.02115,677.51.09
AEIC-ME204.00.013,066.00.05
DiT-IC2,759.90.025,555.00.07
RAE-CoD (Ours)1,341.30.04152,380.71.21

W4: Detailed definition of the SC score.

We include detailed score implementation in Appendix Section B.1. The SC score is derived by combining (1) global scene similarity and (2) local semantic similarity which measures the similarity between textual descriptions of corresponding local semantic units. We acknowledge that the details could be described more explicitly in the current manuscript. We will further improve this description and release the evaluation code upon acceptance to facilitate reproducibility.

Reference:
[1] Zheng et al., 2026, "Diffusion transformers with representation autoencoders".
[2] Singh et al., 2026, "Improved baselines with representation autoencoders".
[3] Zhang et al., 2025, "Stablecodec: Taming one-step diffusion for extreme image compression".
[4] Shi et al., 2026, "DiT-IC: Aligned Diffusion Transformer for Efficient Image Compression".

Thanks for the response. My stance remains positive.

Regarding W1: I appreciate the honest response, even if it is a negative result that the proposed model does not directly generalize across resolutions: a clear requirement for any realistic image compression model.

Regarding W2: I agree that training a model requires substantial effort, and that 256 images are harder to compress. Still, showing only 256 resolution results is a major limitation of the work.

Regarding W3: thanks for adding this additional information, please add it to the manuscript.

Regarding W4: thanks for the clarification. Please also release checkpoints and training code along with the eval code for full transparency.

AuthorsOpenReview ↗

Response to Reviewer ZYv9

We appreciate your response and the positive assessment of our work. We have revised our submission accordingly to reflect the discussion during rebuttal, with all changes highlighted in blue.

  • Regarding W1 and W2, we have updated the "Limitations" in Section 7 Conclusion to explicitly acknowledge the experimental limitation with only 256 resolution results, and the limited generalization ability to higher resolutions. We will continue our investigation on this limitation in future work.

  • Regarding W3, we have added Appendix Section G to discuss the additional complexity measurement.

  • Regarding W4, we commit to releasing the checkpoints and all source code (both training and evaluation) upon acceptance.

Thank you once again for your effort in providing all the feedback!

Comments on the paper

1 comment
AuthorsOpenReview ↗

Follow-up on the rebuttal

Dear Reviewers,

We sincerely thank all your effort in assessment and providing feedback for our work. To address the raised concerns, we posted our responses for every point on November 10.

With fewer than three days remaining in the discussion period, we would appreciate it if you could kindly let us know whether our responses have adequately addressed your concerns. Also, if any additional questions arise. We would like to make sure there's enough time for any follow-up discussion.

Thank you again for your time and effort in reviewing our work.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.