acceptodds
Under review as a conference paper at ICLR 2027

Fidelity Where You Look, Realism Everywhere: Attention-Guided Additive Fidelity Allocation for Generative Image Compression

Abstract

Generative image codecs produce realistic reconstructions, but spatially uniform distortion pricing cannot distinguish a harmless plausible substitution from the loss of source-specific detail in a face or a word. We introduce FLaRE, a one-step generative codec that adds fidelity on top of a realism floor according to a continuous spatial fidelity-need field. FLaRE spatially extends rate–distortion–perception (RDP) optimization and decomposes its objective into the baseline realism-codec objective and an excess-fidelity allocation problem. A training-only auxiliary decoder makes a dedicated message latent carry the requested source evidence without imposing strong local regression on the delivered reconstruction. Field-conditioned latent gating and scaling control where and how much message rate is spent; no field is transmitted, and the one-step decoding path does not vary with the field. Because the floor objective is retained from the realism codec, FLaRE at zero excess remains competitive with its backbone. Experiments show that coded rate lands where the field points, grows monotonically with its magnitude, and improves fidelity in the requested region without harming the background. For perceived quality, we instantiate the field with off-the-shelf predicted attention, motivated by fixation analysis showing that attention concentrates on faces, text, and other information-bearing content. In a subjective quality assessment with 37 observers, FLaRE improves MOS by 0.29–0.57 over OneDC and AEIC-ME, and at least 35 observers rated it higher on average than either competitor. Our code is included in the supplementary material.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
generative image compression, perceptual image compression, neural image codec, rate-distortion-perception, spatial fidelity allocation, diffusion-based codec, visual attention

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_nwHXOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces FLARE, a one-step generative codec that adds fidelity on top of a realism floor according to a continuous spatial fidelity-need field. Experiments show improved RPD performance in some reported settings.

Critical strengths

Addressing the distortion–perception (DP) trade-off under ultra low bitrate constraints in generative image compression, it is valuable to explore effective ROI-adaptive bitrate allocation strategies. This work employs an external human attention prediction model to identify regions requiring higher reconstruction fidelity and allocates more bits to these regions, thereby improving local reconstruction fidelity while maintaining global perceptual realism.

Critical weaknesses
  1. Although this paper focus on an interesting research topic, the proposed improvements are largely limited to spatially adaptive quantization within the AEIC-ME backbone. Compared with existing region-adaptive generative compression methods such as MRIDC and Relic, the architectural modifications are relatively limited.
  2. The experiments in the main text primarily focus on evaluating the effectiveness of FLaRE's fidelity-need field mechanism and presenting subjective evaluation results, while lacking key quantitative comparisons. Moreover, the existing qualitative analyses are limited to comparisons with AEIC-ME and OneDC, and the quantitative results in Figure 25 indicate that FLaRE does not demonstrate a clear advantage in global quality metrics.
  3. Appendix A.2 analyzes the per-channel bitrate distribution of the original AEIC-ME codec. However, this analysis does not provide sufficient evidence that retaining only the first 48 channels as the base group during FLaRE training and inference is sufficient to preserve a competitive realism floor at zero excess.
  4. This paper needs improvement in its writing and overall organization, as the presentation lacks sufficient logical structure and clarity. For example: (1) the "Our Contributions" section in the Introduction is overly verbose and lacks concise summarization; (2) The Introduction, Related Work, and Experiments sections frequently use semicolons to string together multiple ideas rather than presenting them in a logically structured manner; (3) In Sections 6.1 and 6.2, lengthy experimental descriptions are presented in single paragraphs without appropriate paragraph breaks, making them difficult to follow; (4) finally, many table and figure captions in the Appendix contain excessively detailed experimental descriptions, particularly Table 9 and Figures 24 and 25, which negatively affects readability.
  5. The main idea of spatial rate allocation for the RDP trade-off is very close to the `Wasserstein distortion' [1-3], which provides a standard evaluation paradigm or loss function construction. (For example, both extensive user studies and quantitative experiments are required; some standard evaluation protocols are preferred, such as CLIC with Elo scores. )

[1] Wasserstein distortion: Unifying fidelity and realism. [2] Good, cheap, and fast: Overfitted image compression with Wasserstein distortion [3] Frequency-Aware Perceptual Optimization for Low-Complexity Implicit Image Compression

Questions for authors
  1. Section 6.1 states that “Objective quality uses PSNR, LPIPS (Zhang et al., 2018), DISTS (Ding et al., 2022), and patch-FID over 256 × 256 crops. Baselines are the realism codecs AEIC-ME and OneDC (Xue et al., 2025), the generative codecs MS-ILLM (Muckley et al., 2023) and GLC (Jia et al., 2024), and the VVC intra anchor VTM (Bross et al., 2021)”, however, this claim is not well discussed in main text and connected to the corresponding results presented in Appendix Fig. 25. Moreover, Appendix G primarily discusses quantitative results on the 40-image subjective study set, without providing sufficient analysis of Fig. 25, making it difficult for readers to locate and understand these key quantitative evaluations. Given their importance, these results should be presented and discussed as part of the main experimental results rather than being relegated to the Appendix.
  2. The results in Fig. 25 indicate that FLaRE does not demonstrate an advantage over other methods. Why do the authors not provide further analysis of this observation? Additionally, why are reconstruction fidelity and perceptual realism not evaluated specifically within the regions indicated by the fidelity-need field in this comparison?
  3. The qualitative comparisons in the experimental section are limited to AEIC-ME and OneDC, which is insufficient. Given that Fig. 1 already includes comparisons with VTM, DiffC, DiT-IC, and PerCo, why are these methods not included in the qualitative comparisons in the main experiments? At least DiffC, DiT-IC, and PerCo (SD) have publicly available implementations.
  4. Fig. 5 uses three subfigures to validate Spatial Targeting, Monotonicity, and Effective Fidelity, respectively. Why not conduct these three related controlled experiments on the same image? Are the statistical curves on the right side of Fig. 5 based on the 40-image subjective study set? Moreover, the large-scale mechanism validation experiments are placed in the Appendix, while the caption of Fig. 5 does not explicitly specify that the statistical results are based on the 40-image set or adequately direct readers to Fig. 20 in the Appendix.
  5. FLaRE's inference efficiency advantage over MRIDC and Relic mainly stems from the one-step generation mechanism of its AEIC-ME backbone based on SD-Turbo. However, FLaRE does not offer an efficiency advantage over AEIC-ME itself. Moreover, generating the fidelity-need field relies on an external attention prediction model, which introduces additional computational overhead. According to Table 1, this process takes approximately 255 ms, nearly three times the codec's inference time. Although this overhead is reasonable, I suggest including it in FLaRE's overall inference cost rather than reporting it only in the footnote of Table 1. I also suggest moving the computational efficiency analysis to the Appendix, as computational efficiency is not a core advantage of FLaRE.
  6. The existing generative ROI compression methods discussed in the third part of Related Work, i.e., MRIDC and Relic, could be considered advances in rate–distortion–perception and generative compression. Moreover, the methods discussed in the region-adaptive coding part are relatively outdated. More recent works, such as C-ROI, TV-ROI, and Implicit Bit Allocation, should also be reviewed.
  7. Appendix A.2 aims to support a general conclusion that the 320-dimensional latent representation of AEIC-ME contains substantial redundancy, allowing 272 channels to be repurposed for a new information-coding task. However, the analysis demonstrates rate occupancy rather than representation redundancy in a strict sense. Moreover, the analysis is conducted only on Kodak, a very small dataset, and should be further validated across different image distributions, such as DIV2K and CLIC 2020. Additionally, the 48 channels in the analysis refer to the top 48 channels ranked by bitrate, whereas FLaRE appears to use a fixed partition of the first 48 channels as the base group during training and inference. This creates a logical gap that requires clarification. The authors should explain the relationship between the fixed channel partition and the offline rate-based ranking, and justify why the Top-48 rate concentration observed in AEIC-ME provides sufficient grounds for the channel split adopted in the retrained FLaRE model.
  8. Fig. 1 presents a qualitative comparison of different methods. The abbreviations of these baseline methods should be explicitly introduced in the Introduction or Related Work section, and the corresponding references should be included in the figure.
7 comments
AuthorsOpenReview ↗

Response to Reviewer nwHX (1/7)

We sincerely thank the reviewer for this careful and insightful review. Please find our point-by-point response below.

Weakness 1 — the architectural modification is small compared with MRIDC / Relic et al.

We appreciate this observation: our change stays inside the AEIC-ME backbone and is a smaller structural modification than MRIDC's or Relic et al.'s. We respectfully suggest that achieving region control with a smaller architectural change can also be beneficial. Both methods obtain region control by changing how the decoder executes: MRIDC masks the VQ tokens outside the ROI and predicts them before iterative diffusion decoding, and Relic et al. transmit a spatial timestep map and vary quantization, entropy coding and iterative denoising according to it. FLaRE, by contrast, is a coding mechanism that turns an external continuous field into bit allocation inside a one-step realism codec (a non-transmitted field that gates and scales a dedicated message latent, with training-only auxiliary supervision). Because the change sits in the encoder-side coding rather than in the decoder's execution, no iterative sampler or decoding graph is altered for region control and decoding stays at one step (Table 14 (Table 1 in the submission): MRIDC 20 steps, Relic et al. 5–50 steps). The added cost over AEIC-ME is about +1M parameters and +17 ms of codec time, and because the field's effect is absorbed into the hyperlatent, no map is transmitted as side information, unlike Relic et al. (Appendix Table 15 (Table 12 in the submission)). We agree with Q5 that the DeepGaze inference time needed to produce the field (about 255 ms on Kodak) belongs in FLaRE's overall inference cost, and Table 14 (Table 1 in the submission) now reports it that way. That map-generation cost would be the same for MRIDC or Relic et al. if they took the same attention map, so for the structural comparison here we quote the codec-side numbers without it.

Whether the simplicity cost us performance. On the digitized Kodak curves of the two papers (Fig. 28 (Fig. 25 in the submission)), FLaRE is ahead on whole-image PSNR, LPIPS and DISTS at every reported point where the rates overlap — though with neither code nor checkpoints public this is not a fully matched comparison.

In summary, the novelty of this paper is not a new decoder architecture but the mechanism that turns a non-transmitted continuous field into bit allocation, and the measured properties of that mechanism (the control characteristics, the region fidelity gain at equal rate, the subjective study). We hope this clarifies why we view the compact architectural change as a meaningful aspect of the contribution.

AuthorsOpenReview ↗

Response to Reviewer nwHX (2/7)

Weakness 2 / Question 1 / Question 2 — too little quantitative comparison in the body, no advantage in the global metrics of Fig. 25 (now Fig. 28), no separate evaluation inside the field's region

We appreciate the request for clearer quantitative evidence, an explanation of Fig. 28 (Fig. 25 in the submission), and separate quality measurements within the selected region. We have expanded the region-level comparisons and reorganized Sec. 6 accordingly.

Why global metrics show limited gains. At the study operating points, FLaRE sends 29–43% of its payload to the field's active area, a small fraction of the image. This deliberately prioritizes regional fidelity over globally averaged quality. Relative to AEIC-ME on the study set, global LPIPS changes by +0.052 / +0.018 and PSNR by −0.54 / +0.39 dB; on CLIC and DIV2K, LPIPS changes by +0.004 to +0.016, with comparable or favorable PSNR and DISTS. Importantly, zero-excess FLaRE is comparable to AEIC-ME at matched rate, indicating that the global differences arise when excess bits are directed to selected regions. We therefore do not claim superiority on global metrics at equal total rate.

Requested region-level baseline comparison. Appendix Table 10 (Table 9 in the submission) already reported region PSNR/LPIPS and background quality on three datasets. We have added matched-rate region PSNR, LPIPS, and DISTS for every Sec. 6.1 baseline, including DiT-IC, VTM, MS-ILLM, and GLC (Table 17). FLaRE improves region PSNR by 2.3–6.2 dB over those baselines, with lower region LPIPS and DISTS (Table 17). We interpret this cautiously: global codecs are not designed to concentrate rate within the specified region, so this alone does not establish superiority among ROI methods.

Controlled comparison among ROI designs. MRIDC and Relic et al. address similar regional objectives at overlapping rates, but their code and checkpoints are unavailable (Appendix E). Our original submission therefore relied on matched-total-rate subjective and qualitative comparisons. To address the reviewer's concern more directly, we additionally trained four variants from the same pretrained model, with matched rate weight, sparse field, and region definition (Table nwHX-1): (i) the field in the training loss only; (ii) + the field as a codec input; (iii) + the channel split and its gate; (iv) + the training-only auxiliary decoder.

Table nwHX-1 — In-region quality of variants (i)–(iv) at a matched total rate and at a matched in-region rate (40 study images)

at the same total rate, 0.0611 bpp (Fig. 6)

Design (model)Total bppIn-region bppConcentrationRegion PSNR (dB)Region LPIPSRegion DISTSRegion FIDRegion HF (source = 1)
(i) field in the training loss only0.06110.08621.923.820.2630.241213.80.28
(ii) + field as a codec input0.06110.33436.028.620.0990.167143.40.60
(iii) + channel split and gate0.06111.010918.930.600.0310.10270.20.77
(iv) + auxiliary decoder0.06111.043619.628.890.0250.08255.51.06

at the same in-region rate, 0.40 bpp (Fig. 32)

Design (model)Total bppIn-region bppConcentrationRegion PSNR (dB)Region LPIPSRegion DISTSRegion FIDRegion HF (source = 1)
(ii) + field as a codec input0.06510.40006.729.170.1010.174155.10.62
(iii) + channel split and gate0.04040.400011.327.870.0570.11375.60.71
(iv) + auxiliary decoder0.03930.400011.626.980.0460.10467.60.94

Concentration = the region's bit share ÷ its area share.

AuthorsOpenReview ↗

Response to Reviewer nwHX (3/7)

Weakness 2 / Question 1 / Question 2 — too little quantitative comparison in the body, no advantage in the global metrics of Fig. 25 (now Fig. 28), no separate evaluation inside the field's region (continued)

At 0.0611 total bpp, region FID improves 213.8 → 143.4 → 70.2 → 55.5, and DISTS 0.241 → 0.167 → 0.102 → 0.082. The split-and-gate step increases concentration from 6.0 to 18.9; at full field strength it holds background rate at 0.0236 → 0.0237 bpp, whereas (ii) raises it 0.0449 → 0.0524 bpp (Table nwHX-2). At a matched 0.40 in-region bpp, (ii) has higher region PSNR than (iv) (29.17 vs. 26.98 dB), but substantially worse FID (155.1 vs. 67.6). Variant (i) cannot reach that in-region-rate range even at its first nonzero strength; its relative high-frequency energy reaches only 0.32 (Figs. 6 and 32, both new in the revision). These comparisons identify the split and gate as the main source of spatially targeted allocation.

Table nwHX-2 — Bit allocation as the field strength a = η³ opens from 0 to 1 (40 study images)

Design (model)In-region bpp (η = 0 → 1)Background bpp (η = 0 → 1)Background changeConcentration
(i) field in the training loss only0.0710 / 0.1347 (no field input; two operating points)0.0416 / 0.0898–1.9 / 1.6
(ii) + field as a codec input0.0494 → 0.45830.0449 → 0.0524+16.7 %7.3
(iii) + channel split and gate0.0427 → 1.10060.0236 → 0.0237+0.4 %19.6
(iv) + auxiliary decoder0.0414 → 1.04410.0226 → 0.0223−1.3 %19.6

Auxiliary supervision (Table nwHX-3). Relative to (iii), (iv) trades 1.7 dB of region PSNR (30.60 vs. 28.89 dB) for better LPIPS (0.031 → 0.025), DISTS (0.102 → 0.082), and high-frequency energy (0.77 → 1.06). Direct supervision's region FID worsens beyond its best field strength on all three datasets: 69.8 → 70.4 (study), 61.9 → 63.5 (DIV2K), and 24.9 → 28.0 (CLIC). Auxiliary supervision instead maintains perceptual improvement as region rate increases. Together, the controlled results separate where the bits go (split and gate) from how their added fidelity affects perceptual quality (auxiliary supervision). Full results appear in Sec. 6.3 (new in the revision) and Appendix I.

Table nwHX-3 — Region perceptual metrics for auxiliary versus direct output supervision as the field strength is raised; each cell is Aux / Direct

Strength ηStudy set (40) Region FIDStudy set (40) Region DISTSDIV2K validation (100) Region FIDDIV2K validation (100) Region DISTSCLIC 2020 test (428) Region FIDCLIC 2020 test (428) Region DISTS
0 (no field)102.5 / 100.40.155 / 0.15688.7 / 87.10.168 / 0.16835.6 / 34.10.155 / 0.155
0.268.8 / 76.70.109 / 0.11661.9 / 65.80.115 / 0.12124.3 / 25.30.106 / 0.111
0.464.4 / 72.00.091 / 0.10149.9 / 61.90.095 / 0.10720.1 / 24.90.089 / 0.099
0.657.8 / 69.80.086 / 0.10046.8 / 62.0 ↑0.089 / 0.10418.8 / 26.0 ↑0.082 / 0.098
0.855.6 / 70.3 ↑0.083 / 0.102 ↑45.4 / 63.0 ↑0.085 / 0.105 ↑18.2 / 27.3 ↑0.078 / 0.100 ↑
1.055.5 / 70.4 ↑0.082 / 0.10145.0 / 63.5 ↑0.084 / 0.10517.9 / 28.0 ↑0.076 / 0.100 ↑

↑ marks a value that worsened relative to the previous strength; it occurs only under direct supervision.

We have also reorganized the experimental section. Our experiments are three of different kinds — the controlled comparison between designs, the subjective study, and the comparison with existing codecs — so Sec. 6.1's setup is now split in three, each stating its purpose, its data and the basis on which rates are matched. The controlled comparison between designs is now a section of the body (6.3; the four variants above), and the comparison with existing codecs is a separate, final section of Sec. 6 (6.6) that points directly at Fig. 28 (Fig. 25 in the submission), the BD-rates and Appendix Table 10 (Table 9 in the submission). That section states that global metrics are insensitive to where bits land, that an in-region comparison favours region-adaptive codecs, and why we are not ahead in Fig. 28 (Fig. 25 in the submission), together with what the zero-excess curve means.

Reviewer_NuKaOpenReview ↗
Rating
4(+1)
Confidence
2(+1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors present FLARE, a generative image coding scheme which adapts the source fidelity in a continuous spatial manner. They extend AEIC-ME to include base and message channel groups in the latent, with the message channel being gated and scaled based on a spatial field. An auxiliary decoder is added to the network and trained to act as a fidelity preserver. This auxiliary decoder is discarded during inference, and the one-step generative decoder is used without explicit transmission of the spatial field. They show that spatial adaptation is achieved via the field magnitude and enables local fidelity boosts using synthetic fields and large-scale image benchmarks. The method is also tested using DeepGaze-predicted fields. Other tests illustrate the method's interface on plate recognition, face identity preservation, and logo classification.

Critical strengths
  1. The authors show that spatial adaptation works in a controllable manner. The experiments demonstrate controllable spatial allocation: moving a synthetic field moves the message-bit allocation, increasing its magnitude increases the allocated rate, and fidelity improves in the requested regions. Comparisons are also made to a rate-matched codec with zero-field in Table 9 in order to distinguish gains from adding rate at a fixed checkpoint from comparisons at matched total rate. These controls and the accompanying results show that the spatial adaptation is a useful compression tool.

  2. The subjective study shows that the method is actually useful. In the subjective study, all observers test allstimuli and each comparison of FLARE to each competitor/rate shows that 35–36 observers prefer FLARE on average. These MOS gains, as reported, show that the practical benefit of local fidelity is important. In Table 11, random and centered fields with matched active area and field magnitude fail to reproduce the gains in attended-region fidelity, underscoring the importance of field placement.

Critical weaknesses
  1. The authors only partially show that the auxiliary supervision is beneficial. Figure 3 and Appendix E.3 shows the authors' results of comparing the auxiliary supervision and the direct output supervision, using uniform fields. This would show the authors' reasoning about regression and texture smoothing, but it is not clear how the two approaches would compare using the sparse attention fields of the main quality tests. Appendix H evaluates the channel split together with coded-path gating; it does not isolate the benefit of auxiliary supervision. Comparing the existing auxiliary-decoder and direct-output-supervision variants under the sparse study fields, at comparable total rates, would strengthen the evidence for this design choice.

  2. The paper and the release have discrepancies about important experimental details. Appendix A.1 describes the use of max pooling in training and Appendix A.3 states that the field activation mask is applied to the input before any analysis or scaling, for both training and evaluation. Meanwhile, the release's paper-to-code documentation says that the checkpoints were trained with average pooling and without pre-gating, and the training defaults provided agree with this. Additionally, Section 6.1 states that the 2K images were evaluated without tiling, while Table 9, Figure 25, and the inference code indicate that tiling is used for decoding. These discrepancies affect the field coverage or reconstruction, so the authors should reconcile them and clarify the configurations used to obtain the reported results.

Questions for authors
  1. Protocol clarification: Which pooling and pre-gating configurations were used for the released checkpoints and main ablations? Please reconcile Appendix A and the release documentation, and clarify any corrections needed for the reported results.

  2. Evaluation clarification: Does "without tiling" in Section 6.1 refer to just the encoding? Please state the encoding and decoding protocols for the 2K results and clarify which comparisons are made with the same protocols.

  3. Focused validation: Please also consider comparing the existing auxiliary-supervision and direct-output-supervision versions of the method using the sparse study fields and similar total rates, with reports of the local fidelity and overall/background quality. This would help show that the auxiliary design's advantage holds in the key setting.

Future work. Spatial prioritization also appears in Latent Wavelet Diffusion (ICLR 2026), where a wavelet-derived saliency map masks the diffusion training loss, although FLaRE tackles the different problem of allocating a source-dependent bit budget. Given Figure 2, comparing attention-derived fields with signal-derived fields (e.g., local wavelet energy) at matched active area and total rate would clarify what attention adds beyond local structural complexity.

5 comments
AuthorsOpenReview ↗

Response to Reviewer NuKa (1/3)

We sincerely thank the reviewer for this careful and insightful review. Please find our point-by-point response below.

Weakness 1 / Question 3 — the benefit of auxiliary supervision was not shown under the sparse fields, and Appendix H does not isolate it

We appreciate the reviewer identifying two gaps: the auxiliary-supervision comparison used uniform rather than sparse fields, and Appendix H did not isolate supervision from the split and gate. We therefore compared auxiliary versus direct output supervision under the sparse fields of the main evaluation, keeping the pretrained initialization, rate weight, split, gate, and field fixed. Only supervision differs: the auxiliary model uses a training-only decoder for the message latent, while the direct model applies an equally weighted field-weighted L2L_2 loss to the delivered image. We evaluated the 40 study images at a=η3a=\eta^3, η∈{0,0.2,…,1.0}\eta\in\{0,0.2,\dots,1.0\}, including region high-frequency energy relative to the source.

Results (Table NuKa-1). Direct supervision achieves 1.1 dB higher mean region PSNR (29.30 vs. 28.18 dB), but auxiliary supervision yields better region FID (60.4 vs. 72.7), LPIPS (0.033 vs. 0.043), and DISTS (0.090 vs. 0.106). At maximum strength, relative high-frequency energy is 1.06 with auxiliary supervision versus 0.77 with direct supervision. As more bits enter the region, auxiliary perceptual quality continues improving, whereas direct supervision reverses partway through. The trend also holds on DIV2K validation (100 images) and CLIC 2020 test (428 images; Appendix I, Table 20).

Table NuKa-1 — Auxiliary versus direct output supervision under the sparse field, as its strength is raised; each cell is Aux / Direct (40 study images)

Strength ηTotal bppIn-region bppRegion PSNR (dB)Region FIDRegion LPIPSRegion DISTSRegion HF vs. source
0 (no field)0.028 / 0.0290.041 / 0.04321.26 / 21.19102.5 / 100.40.124 / 0.1240.155 / 0.1560.62 / 0.67
0.20.037 / 0.0380.314 / 0.32126.58 / 27.3668.8 / 76.70.050 / 0.0620.109 / 0.1160.89 / 0.68
0.40.047 / 0.0490.627 / 0.64828.04 / 29.4364.4 / 72.00.035 / 0.0400.091 / 0.1011.05 / 0.79
0.60.054 / 0.0560.829 / 0.87128.55 / 30.2457.8 / 69.80.029 / 0.0330.086 / 0.1001.06 / 0.79
0.80.058 / 0.0610.963 / 1.01528.84 / 30.6155.6 / 70.3 ↑0.026 / 0.0310.083 / 0.102 ↑1.05 / 0.77
1.00.061 / 0.0641.044 / 1.10128.89 / 30.7955.5 / 70.4 ↑0.025 / 0.0300.082 / 0.1011.06 / 0.77

Bold marks each metric's best point along the dial; ↑ marks a value that worsened when the strength was raised further.

Background and whole-image effects (Table NuKa-2). Across the strength range, background bpp remains 0.0226 → 0.0223 (auxiliary) and 0.0236 → 0.0237 (direct); background PSNR varies by at most 0.08 dB, and LPIPS by 0.001. Between-model background differences remain similarly stable (PSNR about 0.03 dB, LPIPS within 0.003). Whole-image DISTS improves slightly for both (0.085 → 0.081 / 0.080) as region quality improves. Thus, the widening perceptual difference is concentrated in the selected region rather than the background.

Table NuKa-2 — Background and whole-image quality along the same dial; each cell is Aux / Direct, background = everything outside the region

Strength ηBackground bppBackground PSNR (dB)Background LPIPSWhole-image PSNR (dB)Whole-image LPIPSWhole-image DISTS
0 (no field)0.0226 / 0.023623.06 / 23.040.153 / 0.15022.92 / 22.900.152 / 0.1500.085 / 0.085
0.20.0224 / 0.023423.04 / 23.010.152 / 0.14923.11 / 23.090.149 / 0.1470.081 / 0.081
0.40.0221 / 0.023222.98 / 22.970.153 / 0.15023.08 / 23.080.150 / 0.1470.080 / 0.079
0.60.0221 / 0.023322.98 / 22.960.153 / 0.15023.08 / 23.080.149 / 0.1470.080 / 0.080
0.80.0221 / 0.023522.99 / 22.970.152 / 0.14923.09 / 23.090.149 / 0.1460.080 / 0.080
1.00.0223 / 0.023722.99 / 22.970.153 / 0.15023.09 / 23.090.150 / 0.1470.081 / 0.080

Interpretation. We do not regard direct supervision as intrinsically inferior: it achieves higher region PSNR and represents a valid distortion–perception choice. Our objective, however, is to improve fidelity selectively without reversing the realism codec's perceptual improvement as rate increases. The sparse-field results show that auxiliary supervision better satisfies this objective.

This comparison has been added to Appendix I (Table 20) as a table and a figure, and Fig. 3 now points at it from its caption.

AuthorsOpenReview ↗

Response to Reviewer NuKa (2/3)

Weakness 2 / Question 1 / Question 2 — the pooling, pre-gating and tiling descriptions disagree with the code and the numbers

Thank you for pinpointing the discrepancies between Appendices A.1 and A.3 and the released code documentation, and between the tiling statement in Sec. 6.1 and Table 10 (Table 9 in the submission), Fig. 28 (Fig. 25 in the submission) and the inference code. It is as you saw, and the released code documentation describes the actual configuration. The actual settings and the corrections follow.

1. Pooling and pre-gating (W2, Q1).

Actual settings. The released checkpoints and the main ablation models all use, in training, average pooling of the field onto the grid with a fixed threshold τ=10−3τ = 10^{-3}, and no pre-gating (no masking of the input before the analysis transform). At evaluation and inference they use max pooling, pre-gating and the per-operating-point threshold τqτ_q. Every number reported in the paper was measured with this evaluation configuration, and the released inference code uses the same one, so reproduction is unaffected.

Corrections. The reported numbers stand; only the descriptions change. We have retracted the sentence in Appendix A.3 saying that the mask is applied before analysis and scaling "in training and evaluation", and marked Eq. (8) as the inference mask while giving the training mask (average pooling, τ=10−3τ = 10^{-3}, no pre-gating) alongside it. We corrected the pooling description in Appendix A.1 and stated in Appendix B.1 that the configuration there is for inference only. We also added a paragraph to Appendix A.3 that puts the training and the evaluation settings in one table, matching the released code's paper-to-code document (docs/PAPER_TO_CODE.md).

2. Tiling (W2, Q2).

Protocol. "Without tiling" in Sec. 6.1 refers to the encode only, as you say. For 2K images the encode processes the whole frame at once, while on the decode side, for images above 7682768^2, the UNet and the VAE decoder use the tiling protocol of the base model AEIC-ME (latent tiles of 96 with an overlap of 32; VAE tiles of 160). The generator does not receive the field, and the synthesis transform is conditioned only on the scale map decoded from the hyperlatent, so tiled decoding is separate from the field processing.

Comparisons made under the same protocol, and the correction. In the 2K results of Table 10 (Table 9 in the submission), Fig. 28 (Fig. 25 in the submission) and Appendix G (CLIC 2020 test, DIV2K), FLaRE and AEIC-ME were both evaluated with this protocol (whole-frame encode, tiled decode above 7682768^2). The other baselines follow their own released inference settings, and we state that in Appendix G. The sentence in Sec. 6.1 now reads "2K images are encoded by FLaRE as single frames without tiling and decoded with the base codec's protocol, which tiles images larger than 7682768^2".

AuthorsOpenReview ↗

Response to Reviewer NuKa (3/3)

Question 4 — what attention adds over a signal-derived field (local structural complexity)

We agree that matching active area and total rate is essential to isolate the benefit of attention over local structural complexity. We conducted the suggested comparison.

Setup. Two signal-derived fields were used. (i) Local wavelet energy: the single-level Haar high-frequency energy of the input luminance, Gaussian-smoothed. (ii) LWD-style latent wavelet saliency: the formula of Latent Wavelet Diffusion [Sigillo et al., ICLR 2026] (single-level Haar high-frequency energy of the VAE latent, averaged over channels and upsampled to image size) applied to our own VAE latent (the public repository has no VAE weights, so we used ours, and instead of per-timestep binarization we used the area-matching threshold below). For both maps the threshold is adjusted per image so that the active area matches the attention field (DeepGaze); the three fields are then encoded with the same FLaRE model and the signal-derived results are interpolated to the attention field's total bpp. The evaluation set is the 40 images of the subjective study plus 100 COCO val images containing a face or text, and the face and text regions are defined by detector output on the source (InsightFace, EasyOCR).

Results (Table NuKa-3, Fig. 21 (new in the revision)). Table NuKa-3 reports the active-area IoU of the three fields, face and text recall, and the quality inside the face/text regions and over the whole image at the same total rate. First, the two signal-derived fields point almost entirely elsewhere than the attention field (active-area IoU 0.10–0.11). Second, only the attention field picks out the regions that carry the message: at matched area its face recall is about 12× chance while both signal-derived fields are near chance (0.74 vs. 0.10 / 0.13; chance = the active area, 0.061). Third, at the same total rate the quality inside the face and text regions is higher for the attention field at both operating points (PSNR +3.3 to +3.8 dB; LPIPS 0.092 / 0.056 vs. 0.175–0.178 / 0.095–0.096), while whole-image quality is comparable across the three fields (PSNR within 0.2 dB, LPIPS within 0.016). Since each of the signal-derived fields still raises the PSNR of its own region by 1.9–2.9 dB and barely changes the whole-image metrics, the codec is a neutral allocation device that sends bits wherever the field points. What attention adds is the selection of semantic regions such as faces and text, which local wavelet energy does not capture in the pixel domain or in the latent domain. Fig. 21 shows the active areas of the three fields side by side with reconstructions at the same operating point.

Table NuKa-3 — The attention field versus two signal-derived fields at matched active area and matched total rate (40 study + 100 COCO images)

DeepGaze field (ours)Local Haar energy fieldLWD-style latent wavelet saliency
Active-area IoU against DeepGaze—0.110.11
Face pixel recall0.740.100.13
Text pixel recall0.570.220.26
Face ∪ text PSNR (dB)24.521.221.2
Face ∪ text LPIPS (composited)0.0560.0960.095

Face/text metrics over the 111–114 images that contain such a region; recall = the fraction of face/text pixels falling in active cells, chance 0.061 at the shared active area. High-rate point λ4; λ16 behaves the same.

This comparison has been added to Appendix D as a table and a figure.

Reviewer_jjRJOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a generative image compression method that improves fidelity in attended regions through spatial fidelity-need weighting. Its main contribution is to separate excess fidelity requirements from the baseline generative compression objective, supervise a dedicated message latent through a training-only auxiliary decoder, and control the spatial allocation of additional information through latent gating and scaling. Experiments show that, given a fidelity-need field, the method improves objective fidelity in the specified regions and whole-image perceptual quality in a matched-rate subjective study. The supplementary experiments provide substantial empirical support for its spatial allocation mechanism and local fidelity improvements.

Critical strengths
  1. The problem is practically relevant. Generative image compression faces a genuine need to preserve fidelity in critical regions, which may require allocating additional bits to improve reconstruction quality. 2.The paper provides reasonably extensive experiments and evidence supporting the effectiveness of the proposed approach.
Critical weaknesses
  1. The problem is practically relevant, and the experiments support the effectiveness of the proposed mechanism. However, the paper lacks direct comparisons with other approaches designed to improve fidelity in selected regions, leaving the specific benefits of the proposed message representation and auxiliary supervision unclear. For example, PICO (What Matters in Practical Learned Image Compression) applies a strong L1 reconstruction loss and reduces GAN-based losses in detected text regions to improve text fidelity. Comparisons with such region-specific optimization strategies under matched region information and bitrate would help establish the advantages of the proposed design.
  2. Although the paper provides extensive experiments, the comparisons most relevant to assessing its contribution appear to be missing. ROI-conditioned coding, region-weighted losses, and base-layer-plus-regional-enhancement approaches could plausibly pursue the same objective. Comparisons using the same backbone, region information, and total bitrate would help establish whether FLaRE offers a better quality–rate trade-off. The existing comparisons with general-purpose codecs do not resolve this question, particularly when only FLaRE receives the target-region information.
  3. I find the interpretation of Fig. 3 insufficiently convincing. Adding a strong output-space MSE term to an already perceptually optimized generative codec can shift its operating point toward higher PSNR at the expense of perceptual quality. Consequently, obtaining better LPIPS/DISTS but lower PSNR than this directly supervised variant does not, by itself, demonstrate an improved trade-off over the original codec: the auxiliary approach may simply preserve more of the original model’s perceptual quality. Although baseline comparisons are provided elsewhere, Fig. 3 would be more informative with the original AEIC-ME and zero-excess FLaRE included as matched-rate references. This would help distinguish an improvement over the original operating frontier from a redistribution of distortion and perceptual quality.
  4. The abstract and introduction would benefit from clearer terminology. For example, “coded rate lands where the field points” does not clearly distinguish spatial bit allocation, message rate, and total bitrate. Likewise, “source fidelity is represented in the code” leaves it unclear whether “code” refers to the quantized latent representation or the final bitstream. Defining these terms more precisely would make the method and its claims easier to follow. Overall, I find the problem valuable and the proposed mechanism empirically supported. However, the paper does not yet convincingly establish its advantage over existing region-adaptive optimization or regional enhancement approaches. Matched-information, matched-rate comparisons are needed to determine whether the method provides a better fidelity–perceptual quality trade-off, together with clearer wording about the scope of its claims.
Questions for authors

1.Under the same fidelity-need field and total bitrate, what additional benefits do the dedicated message representation and auxiliary decoder provide over direct ROI-weighted supervision or a regional enhancement bitstream? Are there controlled experiments comparing these alternatives? 2.Does auxiliary supervision achieve a better fidelity–perceptual quality trade-off at matched bitrates than the original AEIC-ME and zero-excess FLaRE, rather than merely outperforming the variant with stronger output-space MSE supervision?

5 comments
AuthorsOpenReview ↗

Response to Reviewer jjRJ (1/5)

We sincerely thank the reviewer for this careful and insightful review. Please find our point-by-point response below.

Weakness 1 — no matched comparison with other methods that raise fidelity in selected regions (e.g. PICO)

We agree that matched comparisons with region-fidelity methods are important. A direct comparison with PICO was not feasible because its code and weights are unavailable and its operating rates are 3–23× higher than ours.

We instead conducted a controlled comparison under the same training loss (including the GAN term), incorporating a variant from PICO’s region-specific optimization family. Without an external guide at inference, this variant concentrates rate less effectively and yields lower ROI perceptual quality. Details appear under Weakness 2; the contributions of the dedicated message representation and auxiliary supervision are addressed under Weakness 2 and Question 1.

Comparison with an external method — C-ROI. We also compared against C-ROI, the one publicly available spatial fidelity-allocation model whose rate range overlaps ours, giving it our region directly as its ROI mask (on our side, the released FLaRE). The comparison is at the same number of bits inside the same region rather than at the same total rate, and it points the same way as the controlled experiment: C-ROI is 3.4 dB higher in region PSNR, but is well behind on perceptual quality and on high-frequency retention (region LPIPS 0.147 vs. 0.063, region DISTS 0.126 vs. 0.070, high-frequency retention 0.73 vs. 0.93). That is a natural outcome for a codec trained with MSE and a rate term and no perceptual loss. When the total rate is matched instead, C-ROI spends most of its bits in the ROI and its background collapses, whereas FLaRE's background stays at the level of its own zero-excess floor at the same rate. The setup, the full numbers and reconstructions at an equal total rate are in Appendix J (Fig. 35, Table 21).

AuthorsOpenReview ↗

Response to Reviewer jjRJ (2/5)

Weakness 2 — no controlled comparison with ROI-conditioned coding / region-weighted losses / base-plus-enhancement

We appreciate the request for a controlled comparison with region-weighted losses, ROI-conditioned coding, and base-plus-enhancement designs. We implemented these alternatives within the same architecture, using the same training images, DeepGaze field, loss (including GAN), and rate weight (Sec. 6.1, Appendix I). The active sparse-field area defines the region throughout.

Designs. (i) Region-weighted loss: the field weights the reconstruction loss but is not supplied to the codec at inference, as in the region-specific optimization family discussed in Weakness 1 (including PICO). (ii) ROI-conditioned coding: the same field additionally scales the precision of the entire latent, following the principle of C-ROI, TV-ROI, and Qmap-style methods. (iii) Base-plus-region enhancement: the latent is divided into an always-coded floor and a field-gated message group, while retaining the same direct reconstruction loss as (i) and (ii). Thus, (i)–(iii) isolate the use of region information. (iv) adds only auxiliary-decoder supervision to (iii); its effect is discussed in Weakness 3.

Region quality at matched total rate (Table jjRJ-1, upper block). As the field is introduced into the codec and then given a dedicated message representation, concentration rises from 1.9 to 6.0 to 18.9, while region DISTS falls from 0.241 to 0.167 to 0.102 and region FID from 214 to 143 to 70. At 0.0611 total bpp, (ii) spends only 0.33 bpp in the region, versus 1.01 bpp for (iii). This supports the contribution of the split representation beyond conditioning alone. The perceptual ordering also holds when comparing at matched in-region rate (Table jjRJ-1, lower block; Appendix I, Table 18). Adding auxiliary supervision in (iv) lowers region PSNR by 1.7 dB relative to (iii), but improves FID 70.2 → 55.5, DISTS 0.102 → 0.082, LPIPS 0.031 → 0.025, and relative high-frequency energy 0.77 → 1.06.

Table jjRJ-1 — Region quality of designs (i)–(iv) at a matched total rate and at a matched in-region rate (40 study images)

at the same total rate, 0.0611 bpp (Fig. 6)

Design (model)Total bppIn-region bppConcentrationRegion PSNR (dB)ROI LPIPSROI DISTSROI FIDROI HF (source = 1)
(i) field in the training loss only (region-weighted loss)0.06110.08621.923.820.2630.2412140.28
(ii) + field as a codec input (ROI-conditioned coding)0.06110.33436.028.620.0990.1671430.60
(iii) + channel split and gate (base layer + region enhancement)0.06111.010918.930.600.0310.102700.77
(iv) + auxiliary decoder0.06111.043619.628.890.0250.08255.51.06

at the same in-region rate, 0.40 bpp (Fig. 32)

Design (model)Total bppIn-region bppConcentrationRegion PSNR (dB)ROI LPIPSROI DISTSROI FIDROI HF (source = 1)
(ii) + field as a codec input (ROI-conditioned coding)0.06510.40006.729.170.1010.1741550.62
(iii) + channel split and gate (base layer + region enhancement)0.04040.400011.327.870.0570.113760.71
(iv) + auxiliary decoder0.03930.400011.626.980.0460.10467.60.94

Concentration = the region's bit share ÷ its area share.

AuthorsOpenReview ↗

Response to Reviewer jjRJ (3/5)

Weakness 2 — no controlled comparison with ROI-conditioned coding / region-weighted losses / base-plus-enhancement (continued)

Bit allocation as the field strength increases (Table jjRJ-2). With the split and gate (iii), background rate remains nearly constant (0.0236 → 0.0237 bpp; +0.4%), whereas conditioning the full latent (ii) increases it (0.0449 → 0.0524 bpp; +16.7%). Final concentration is 19.6 versus 7.3. Because the field and loss are matched, this difference isolates the split representation: outside the selected region, its gated message channels collapse to zero symbols while the floor remains coded. A single unsplit latent cannot be gated off without also removing background information. Design (i) has no inference-time field control (concentration 1.9). At the same region PSNR of 23.6 dB, (iii) requires 0.0318 total bpp, versus 0.0499 for (ii) and 0.0508 for (i) (Fig. 6 (new in the revision)).

Table jjRJ-2 — Bit allocation of designs (i)–(iii) as the field strength a = η³ opens from 0 to 1 (40 study images)

ModelIn-region bpp (η = 0 → 1)Background bpp (η = 0 → 1)Background changeConcentration
(i)0.0710 / 0.1347 (no field input; two operating points)0.0416 / 0.0898–1.9 / 1.6
(ii)0.0494 → 0.45830.0449 → 0.0524+16.7 %7.3
(iii)0.0427 → 1.10060.0236 → 0.0237+0.4 %19.6

The trends also hold on DIV2K validation (100 images) and CLIC 2020 test (428 images) (Appendix I, Figs. 33 and 34).

We have added the comparison, including tables and figures, to Sec. 6.3 (new in the revision) and Appendix I.

Weakness 3 — whether Fig. 3 shows an improved frontier or a redistribution, and the missing anchors

We thank the reviewer for this important distinction. Better perceptual metrics than a direct-MSE variant do not establish a better fidelity–perception frontier relative to the realism codec FLaRE extends. We do not claim to improve that frontier. Our goal is instead to allocate additional bits to selected regions while retaining the perceptual behaviour of the realism codec it extends.

We agree with the reviewer's interpretation that auxiliary supervision may preserve more of the original model's perceptual quality; this preservation is the intended property, rather than evidence of a superior frontier. Direct output supervision remains a valid fidelity–perception choice, but may shift the reconstruction toward distortion optimization as more region bits are added.

Controlled evidence (Fig. 3; Appendix E.4 (Appendix E.3 in the submission)). We compare (iii), the split-and-gated design with direct L2L_2 supervision, against (iv), the same design with training-only auxiliary supervision. Architecture, field, and rate weight are fixed. With the field off, the models are nearly identical (patch-FID 53.1 vs. 53.3, DISTS 0.089 vs. 0.088). As the uniform field opens, auxiliary supervision improves DISTS 0.089 → 0.032 and patch-FID 53.1 → 17.9 monotonically. Direct L2L_2 instead worsens DISTS 0.0455 → 0.0496 and patch-FID 22.5 → 23.6 beyond η=0.6η=0.6, although PSNR continues to rise and ends higher (33.33 vs. 31.97 dB). The sparse-field comparison confirms this behavior (Appendix I, Table 20).

Separately, the split-and-gate mechanism directs the additional bits to the requested region: background rate changes by only +0.4%, versus increasing concentration to 19.6, compared with 7.3 without the split (Weakness 2). Together, these results support targeted allocation with continued perceptual improvement, not a claim of frontier improvement.

References added to Fig. 3. As you suggested, the figure now carries both AEIC-ME and zero-excess FLaRE as matched-rate references; the zero-excess ladder is the released FLaRE at all five operating points with the field off, measured under the same protocol as the two variants, and the two ladders overlap (at matched rate, zero-excess is about +0.5 dB in PSNR and marginally behind on the perceptual metrics). The body and caption no longer read "a better trade-off". Both ladders stop at the variants' zero-excess points, since the public checkpoints do not reach the rates the open field spans; adjusting the rate weight of the official recipe alone did not get AEIC-ME there, and we will extend the references once we have converged points.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Thank You for Your Feedback and Our Apologies for the Delayed Response

Dear Reviewers,

We sincerely thank you for taking the time to provide thoughtful and insightful feedback on our work.

To address your concerns as thoroughly as possible, we have conducted additional experiments and revised the manuscript accordingly. Several controlled experiments required training new models, which took considerable time and unfortunately delayed our responses.

We apologize for the delay and kindly ask for your understanding. We greatly appreciate your patience and, once again, your valuable comments, which have helped us improve our work.

Sincerely, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.