- Rating
- 4(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a generative image codec that operates at extremely low bitrates, where decoding becomes close to unconditional generation. The correlation between reconstruction-oriented and semantic training objectives is analyzed (Fig. 2, 3), and pixel space, VAE latent space, and RAE latent space are compared for zero-shot compression (Fig. 4, 5). The key observation from these analyses is that the RAE latent space preserves high-level semantics, as evaluated by vision foundation models (VFMs), better than pixel or VAE latent spaces. Based on this, a generative codec, RAE-CoD, consisting of an RAE, a latent codec, and a diffusion transformer (DiT) is proposed. The main distinction from previous generative image codecs is that it performs coding in the RAE latent space rather than in pixel or VAE latent space. Training follows the RAE training convention, with a newly introduced auxiliary loss to compensate for the training instability caused by the rate constraint. For the same reason, training proceeds progressively from high to low bitrates. For evaluation, VFM-based metrics and a VLM-based protocol, which separately assesses semantic recognizability, quality, and consistency (SR, SQ, SC), are used. With these, RAE-CoD achieves higher scores than recent generative image codecs, and it is also preferred by participants in a user study.
Critical strengths
S1: I understand the core claim of this paper to be that existing generative codecs suffer from semantic collapse at extremely low bitrates, and that RAE-space compression resolves this collapse. This claim is well supported by the experimental results. If semantic collapse had been evaluated only by distances or similarities in the VFM feature space, the evaluation could have fallen into circular reasoning, since the RAE itself is built on VFM representations. However, by also evaluating the models with the VLM-based protocol, in particular semantic consistency (SC), and showing the advantage of RAE-CoD there, the paper avoids this circularity, and the claim is well supported.
Critical weaknesses
W1: Compared with Pixel-CoD, VAE-CoD, and AEIC-ME, the proposed RAE-CoD achieves consistently better semantic consistency (SC), as judged by a VLM (Fig. 8). However, SC is still a model-based evaluation. To determine whether a higher SC actually delivers a better reconstruction to humans, which is the ultimate goal, the user study is the most important result. Yet the user study is conducted in a limited setting, and details about its protocol are largely missing.
-
First, it is not stated whether the original image was presented to participants together with the stimuli. Since the proposed model is a generative codec, the study should evaluate how well the reconstruction conveys the original image (i.e., a codec evaluation). Without the original, it may only measure how plausible the image itself looks (i.e., a generation-quality evaluation).
-
Second, the user study should at least be conducted at the original resolution rather than on 256×256 crops. On resized and center-cropped low-resolution images, high-level semantic differences tend to dominate low-level distortion in participants' ratings. I do not consider this an optional additional experiment; I consider it essential for establishing the value of this paper.
Questions for authors
I am willing to raise my rating if both concerns in W1 are resolved. If the original image was shown to participants together with the stimuli, and if the authors present user-study results at a resolution higher than 256×256 or convincingly show that results at 256×256 are sufficient.
4 comments
Response to Athors
Thank you for confirming that the original image was presented alongside the reconstructions in every trial. This resolves my first concern.
Regarding the resolution, while the emphasis on high-level semantic consistency is well aligned with the central objective of this work, it does not by itself justify restricting the evaluation to 256×256. The response notes that this resolution reduces the influence of fine-grained details on the assessment, but such a reduction is itself a source of bias in a human evaluation of a codec.
I fully recognize that the framework builds on the pretrained RAEv2 DiT and decoder, whose released checkpoints and code are limited to 256×256. For this reason, the absence of a high-resolution evaluation is better understood as a limitation of the current study than as an intentional design choice.
To be clear, I regard this paper as a strong contribution. If the authors clearly acknowledge this limitation of the experimental setting, I am willing to raise my rating.
Response to Reviewer 5mzr: The manuscript has been revised and updated
Thank you very much for your response and the suggestion. We acknowledge this resolution limitation on our current experimental setting, and we have revised the submission accordingly with all changes highlighted in blue.
Specifically,
- "Limitations" in Section 7 Conclusion now explicitly acknowledges that the current experimental settings are limited to 256x256 images.
- "Human Preference Study" in Section 5.2 Main Results now contains the detailed implementation of the user study as presented in the previous response.
Please let us know if you have any further concerns. We would be willing to address them promptly.
Response to Athors
Thank you for the quick revision. It addresses my remaining concern, and I have raised my rating accordingly.
Response to Reviewer 5mzr
We sincerely thank the reviewer for recognizing the importance of evaluating semantic preservation beyond VFM feature distances. We also appreciate the critical comments regarding the user study. We provide detailed clarifications on the experimental protocol and the rationale behind the resolution below.
W1, Q1: Implementation of the user study.
We would like to confirm that the original image was indeed presented to participants alongside the reconstructed images in every test.
Specifically, each test trial presented three 256×256 images on the same display: the original image, the reconstruction produced by RAE-CoD, and the reconstruction produced by a competing method. The original image was displayed at the top and explicitly labeled as the ground truth image, while the two reconstructed images were placed side by side underneath it. Their left-right positions were randomized across trials to avoid positional bias. Participants were instructed to select the reconstruction whose content was more consistent with the ground truth image shown above. No additional textual hints or method-specific information were provided in the interface. The voting results were automatically recorded to calculate the preference rates of RAE-CoD against each competing method.
We acknowledge that these details were not sufficiently explained in the original manuscript due to the page limit. We will provide a more explicit description of the user study implementation in the revised final version.
W1, Q2: Rationale for conducting the user study at 256×256 resolution.
We agree with the reviewer that, on 256x256 images, high-level semantic differences tend to dominate low-level distortion in participants' ratings. we would like to clarify that emphasizing high-level semantic consistency is intentional and directly aligned with the central objective of this work. Our user study is specifically designed to investigate semantic preservation under extremely low-bitrate compression, instead of assessing low-level distortion (which becomes less meaningful at extreme bitrates due to the intrinsic rate-distortion tradeoff). We believe that using 256×256 images is appropriate for this particular evaluation objective for two reasons.
First, at this resolution, the main semantic content of an image remains sufficiently recognizable, allowing participants to focus on whether the essential objects, scenes, and their semantic relationships are faithfully preserved. This is particularly relevant to our study, which aims to assess semantic consistency rather than fine-grained texture or pixel-level fidelity.
Second, higher-resolution images generally contain more spatial redundancy and fine-grained visual details. Under extremely low-bitrate compression, these additional details introduce substantial low-level distortions, which may influence human preferences independently of the general semantic quality and consistency that we intend to investigate. In contrast, evaluating images at 256×256 provides a more focused comparison of the semantic content conveyed by different reconstructions as demonstrated in Fig. 1 and Fig. 9, reducing the influence of fine-grained details on the assessment.
We acknowledge that conducting user studies at higher resolutions would provide additional insights. However, our current framework relies on the pretrained RAEv2 [1] DiT and decoder, whose publicly released checkpoints and code include only 256×256 resolution. Directly applying these models to higher-resolution images does not yield meaningful reconstructions, while finetuning RAEv2 DiT and decoder at higher resolutions with official 256x256 training strategies requires substantial computational resources and is beyond the scope of this work. Therefore, we leave high-resolution human evaluation as an important experiment for future work.
Reference:
[1] Singh et al., 2026, "Improved baselines with representation autoencoders".