acceptodds
Withdrawn from ICLR 2027

All-in-One: Joint Multimodal Image Compression via Shared 2D Gaussian Splatting

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Aligned visible and thermal infrared images observe related boundaries but differ in texture and radiometric response. Independent compression repeats spatial support, whereas rigid sharing can suppress modality-specific structure. We study this trade-off with \method, an explicit RGB–IR codec that stores shared 2D Gaussian anchors, a jointly coded appearance dictionary, and sparse infrared geometry corrections. Sparse appearance routes are optimized in their deployed topology, and the correction support is fixed before its values are learned. Reconstruction uses normalized 2D splatting without a neural synthesis network. Complete-stream accounting separates representational savings from payload estimates and evaluates RGB and IR quality independently. In a ten-pair LLVIP development study, sparse corrections improve recorded RGB/IR PSNR by 0.129/0.299 dB over hard sharing at an additional 0.0189 pair-bpp; both SSIM values also improve. These component results motivate a controlled study of support reuse, with dataset-level matched-rate and complete-encoding-cost comparisons specified separately from the development evidence.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
2DGS, Image Compression, Multimodal Image

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_tRnaOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates joint compression of aligned RGB and thermal infrared image pairs using a shared 2D Gaussian representation. The proposed method combines shared geometric anchors, a joint appearance dictionary with top-4 routing, and sparse IR-specific geometric corrections. The experimental evaluation compares hard geometry sharing with sparse correction on ten LLVIP image pairs. The correction mechanism improves checkpoint-level RGB and IR PSNR by 0.129 and 0.299 dB, respectively, while increasing the complete packet rate from 0.3967 to 0.4156 pair-bpp. Reconstruction quality is evaluated at model checkpoints, whereas bitrate is measured from separately serialized packets. The study does not report reconstruction quality obtained from decoding these packets.

Critical strengths

The paper presents a reasonably clear formulation for joint RGB–IR compression using shared 2D Gaussian representations. It also accounts for complete serialized packet sizes, including metadata, providing a transparent basis for assessing coding costs.

Critical weaknesses

1. Incomplete end-to-end rate–distortion evaluation.

PSNR/SSIM values are measured from model checkpoints, while bitrates come from separately encoded packets with different quantization rules. Without evaluating actual bitstream-decoded reconstructions, the reported rate–distortion performance remains unverified.

2. Lack of essential baselines and fair comparisons.

The paper provides neither independent RGB–IR compression baselines nor comparisons with existing joint compression methods. The absence of rate-matched evaluations and multi-rate RD curves prevents meaningful assessment of compression efficiency.

3. Insufficient validation of the correction strategy.

Correction support is selected using local reconstruction errors rather than estimated distortion reduction per coded bit. The effectiveness of this heuristic remains unclear without comparisons against alternative selection strategies or rate-aware correction allocation.

4. Limited experimental scale and missing ablations.

Experiments cover only ten LLVIP image pairs, limiting the generalizability of the findings. Critical design choices, including the correction ratio, shared routing, and error-based selection, lack systematic ablation.

5. Insufficient implementation details and reproducibility.

Important details of Gaussian initialization, anchor growth, optimization, and entropy coding remain insufficiently specified. These omissions make it difficult to independently reproduce and verify the proposed compression pipeline.

6. Limited and insufficiently demonstrated novelty.

The framework builds upon established Gaussian representations and cross-modal compression techniques. Without comparisons or ablations that isolate the benefits of the proposed combination, its distinct methodological contribution remains unclear.

Questions for authors
  1. Can the authors provide end-to-end rate–distortion results measured from actual bitstream-decoded reconstructions?
  2. At matched bitrates, what advantages does the proposed joint representation offer over independent and existing joint compression methods?
  3. Which specific components of the proposed framework provide measurable benefits over simpler alternatives, and how are these benefits experimentally validated?
Reviewer_Mh9WOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes All-in-One for jointly compressing aligned RGB–IR image pairs using shared 2D Gaussian geometry, a compact appearance dictionary with shared Top-4 routing, and sparse infrared position and scale corrections. Reconstruction uses normalized splatting without a neural synthesis network. Experiments on ten LLVIP pairs compare hard geometry sharing with sparse correction. The corrected configuration improves average RGB/IR PSNR by 0.129/0.299 dB at approximately 4.76% higher rate, with eight pairs improving across all four reported quality metrics.

Critical strengths

The method shares spatial support while allowing bounded IR geometry corrections, providing a concrete mechanism for accommodating disagreement between aligned modalities.

Critical weaknesses
  1. The evaluation compares only hard sharing and sparse correction at different rates. There is no independently coded reference, matched-rate hard-sharing comparison, or evaluation against existing RGB–IR codecs. The reported improvement therefore does not establish whether the proposed sharing strategy is more effective than alternative allocations of the same bit budget.

  2. The experimental evidence is too limited.

Questions for authors
  1. Why are there no comparisons with existing codecs?

  2. Both evaluated configurations already share geometry and appearance routes. There is no independent-representation baseline, and the separate-route and separate-dictionary controls motivated in §3.3 are absent. Does sharing actually improve the rate–distortion trade-off, or does it introduce constraints that the correction mechanism then partially compensates for?

  3. Why is the entire evaluation restricted to ten development pairs?

Reviewer_eZZkOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Aligned visible (RGB) and thermal infrared (IR) images observe related boundaries but differ in texture and radiometric response. Independent compression repeats spatial support, whereas rigid sharing can suppress modality-specific structure. This paper proposed an explicit RGB-IR codec that stores shared 2D Gaussian anchors, a jointly coded appearance dictionary, and sparse inferred geometry corrections. However, the experimental results provided in the manuscript do not fully demonstrate the effectiveness of the proposed algorithm. In addition, the phrase "All in One: Joint Multimodal Image Compression" in the title is misleading as the algorithm can be applied to data compression of different modal pairs, but the manuscript only studies RGB-IR data compression tasks.

Critical strengths
  • The proposed multimodal data alignment and compression scheme based on Shared 2D Gaussian Splatting for RGB-IR data joint compression task has certain innovation.
Critical weaknesses
  • Insufficient experimental comparison settings. The paper's motivation is that sharing spatial support across modalities saves bits versus compressing RGB and IR separately. The experiments compare only hard sharing vs. sparse correction, two configurations of the same shared representation. There is no independent-coding baseline (e.g., coding RGB and IR separately with the same 2DGS pipeline or a standard codec). Given the reported results, it is hard to judge whether sharing beats independent coding at matched rate.

  • There is no performance comparison with the existing baseline works such as CCEM, GaussianImage, JPEG, JPEG2000, BPG, or any learned codec. Cross-modality compression precedents (e.g., Lu et al., CVPR 2022, IR-conditioned visible coding) are also absent from the references (~13 total). The most relevant comparator is named but unmeasured.

  • Insufficient experimental scale and statistical validity The experimental results only provide evaluation data for 10 pairs of LLVIP images, a single anchor number of N=5100, a single configuration, and 2 operating points.

Questions for authors
  • What is the RD comparison results against independent coding?

  • Provide performance comparison results with CCEM, Lu et al. CVPR 2022 and a traditional codec like JPEG XL or BPG on the same LLVIP pairs.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.