acceptodds
Withdrawn from ICLR 2027

Recover Semantics, Not Codewords: An Asymmetric Paradigm for Vector-Quantized JSCC

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vector-quantized semantic communication commonly follows a recover-then-reconstruct paradigm: the receiver detects a transmitted symbol, retrieves its source codeword, and then reconstructs the image. This design resolves channel ambiguity through a discrete decision, although a codeword error need not imply poor reconstruction. We argue that faithful source reconstruction does not require exact codeword recovery. Our insight is to jointly learn which source representations become neighbors in the channel and how observations within those neighborhoods are reconstructed. We introduce AsymVQ-JSCC, which retains a discrete source vocabulary while coupling source-aware irregular QAM with continuous semantic recovery. The transmitter learns constellation coordinates from source relationships and refines their geometry through the image reconstruction objective. Building on this geometry, a continuous receiver maps raw observations into semantic latents, learning denoising jointly with the decoder. Bypassing hard detection and codebook lookup allows observations between constellation points to support reconstruction through latents beyond the source vocabulary. Each spatial token occupies one channel use, independently of vocabulary size. Across four image datasets, bandwidth ratios, and SNRs, AsymVQ-JSCC achieves competitive PSNR and perceptual quality with graceful degradation under both AWGN and Rayleigh fading. In a fixed-checkpoint intervention on Kodak, projecting receiver latents to the nearest source codeword, while leaving the transmitted symbols, channel observations, and model weights unchanged, reduces PSNR by 1.46–2.57 dB and increases LPIPS by 0.063–0.077 across the tested SNRs. These results provide direct evidence that reconstruction benefits from continuous receiver representations beyond the source vocabulary.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Joint Source-Channel Coding, Vector Quantization image compression, Non-uniform QAM

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_hUCpOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

For wireless image transmission over noisy channels, existing methods that transmit vector-quantized representations often rely on recovering discrete codewords before reconstruction. The paper proposes AsymVQ-JSCC, which learns the QAM constellation according to source representations and directly maps the received signal to the latent used for image reconstruction without hard detection or codeword lookup.

Critical strengths

The paper studies a meaningful problem in wireless image transmission, namely how to reconstruct images effectively when vector-quantized representations are affected by channel noise. Questioning whether exact codeword recovery is necessary is relevant because it directly concerns how channel observations should be used for reconstruction.

Critical weaknesses
  1. The proposed mechanism lacks necessary ablation studies. The paper mainly reports overall performance comparisons, but does not provide matched ablation experiments that isolate the proposed components. In particular, although the title emphasizes "recover semantics," the paper does not clearly define or directly evaluate what "semantics" means or how it differs from codeword recovery. The proposed continuous receiver is also closely related to existing soft decoding for noisy vector quantization [1] and neural reconstruction from noisy finite constellation observations [2], but these alternatives are not discussed or compared experimentally. A matched comparison among hard codeword recovery, conventional soft decoding, and the proposed continuous receiver is therefore necessary to establish both the effectiveness and novelty of the proposed mechanism.

  2. The experimental comparison does not provide a sufficiently fair or complete baseline. The separated baseline uses fixed rate-1/21/2 LDPC and 64-QAM over the entire SNR range. At high SNR, this introduces unnecessary channel redundancy and leaves fewer bits for source coding under the same channel-use budget, which can lower the achievable reconstruction quality of the separated system. At low SNR, 64-QAM with the same code rate can be too aggressive and lead to frequent LDPC decoding failures, producing a sharp waterfall effect. This setting can directly contribute to the abrupt transition of ProGIC+LDPC observed in Figure 3. Therefore, the current results cannot clearly determine whether the reported advantage comes from the proposed system or from a weak fixed-MCS separated baseline. A more appropriate evaluation is to adapt the modulation order and LDPC code rate to the channel condition using an MCS table [3], as adopted in recent image transmission studies [4,5]. The separated comparison also omits recent strong image codecs such as OneDC [6], DiT-IC [7], and EF-LIC [8]. Finally, the paper does not report model size, computation, or encoding and decoding latency, making it difficult to determine whether the reported gains come from the proposed communication design or additional computational cost.

References

[1] M. Skoglund and P. Hedelin, "Hadamard-Based Soft Decoding for Vector Quantization Over Noisy Channels," IEEE Transactions on Information Theory, vol. 45, no. 2, pp. 515–532, 1999.

[2] T.-Y. Tung, D. B. Kurka, M. Jankowski, and D. Gündüz, "DeepJSCC-Q: Constellation Constrained Deep Joint Source-Channel Coding," IEEE Journal on Selected Areas in Information Theory, vol. 3, no. 4, pp. 720–731, 2022.

[3] 3GPP, "NR; Physical Layer Procedures for Data," 3GPP TS 38.214, Table 5.1.3.1-1.

[4] Y. Huang and Z. Qin, "Image Semantic Communication With Quadtree Partition-Based Coding," IEEE Journal on Selected Areas in Communications, vol. 44, pp. 2765–2778, 2026.

[5] G. Zhang, P. Yang, Y. Cai, Q. Hu, and G. Yu, "From Analog to Digital: Multi-Order Digital Joint Coding-Modulation for Semantic Communication," IEEE Transactions on Communications, vol. 73, no. 6, pp. 4257–4271, 2025.

[6] N. Xue, Z. Jia, J. Li, B. Li, Y. Zhang, and Y. Lu, "One-Step Diffusion-Based Image Compression with Semantic Distillation," Advances in Neural Information Processing Systems, vol. 38, 2025.

[7] J. Shi, M. Lu, X. Li, A. Ke, R. Zhang, and Z. Ma, "DiT-IC: Aligned Diffusion Transformer for Efficient Image Compression," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 25719–25729, 2026.

[8] H. Cao, W. Guo, Z. Qin, and J. Han, "Efficient Learned Image Compression without Entropy Coding," in Proceedings of the 43rd International Conference on Machine Learning, PMLR 306, pp. 11158–11181, 2026.

Questions for authors
  1. We would like the authors to clearly define what “semantics” means in this paper and quantify the gain of recovering “semantics” rather than codewords. We also expect explicit ablation experiments comparing hard codeword recovery, conventional soft decoding, and the proposed continuous receiver, so that the contribution of the proposed mechanism can be directly identified.

  2. We would like the authors to provide a fairer and more complete separated-system comparison. The LDPC rate and QAM order should be selected according to an MCS table for each SNR, stronger recent image codec baselines should be included, and model size, computation, and encoding/decoding latency should be reported to clarify fairness and reproducibility.

Reviewer_SmemOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AsymVQ-JSCC, a vector-quantized joint source-channel coding framework that combines source-aware irregular QAM constellation learning with continuous receiver-side recovery. Instead of performing hard detection and codebook lookup, the receiver directly maps noisy channel observations to continuous latent representations for image reconstruction. The method jointly optimizes the source codec, constellation mapper, and receiver, and provides an affine reconstruction analysis to motivate continuous recovery beyond the source codebook. Experiments on four image datasets under AWGN and Rayleigh fading show competitive reconstruction quality across different SNRs and bandwidth ratios. Additional ablation experiments demonstrate the degradation caused by constraining receiver latents to the source codebook.

Critical strengths

The paper presents a conceptually clear asymmetric architecture that decouples the discrete transmission vocabulary from the continuous reconstruction space. By jointly optimizing the source-aware constellation mapper and the continuous receiver with the image reconstruction objective, the framework provides a coherent approach to improving reconstruction under channel noise without requiring hard detection and codebook lookup at the receiver.

Critical weaknesses
  1. Insufficient novelty analysis and comparison with related work. This paper proposes a JSCC framework combining source-aware irregular QAM constellation mapping with continuous reception. However, its overall design is closely related to prior work on channel denoising for digital semantic communications. For example, Mo et al. proposed SCDM (arXiv:2501.17876, 2025), which investigates score-based diffusion for channel denoising in digital semantic communication systems. Although the two approaches differ in their receiver mechanisms and constellation design, the paper neither adequately discusses this relevant work nor provides a direct experimental comparison. The existing CDM-JSCC baseline primarily addresses diffusion-based image reconstruction and cannot substitute for a comparison with SCDM. Consequently, the substantive incremental contribution of the proposed framework remains unclear.

  2. Limited evidence of practical applicability and system-level validation. The proposed receiver requires access to noisy I/Q observations before hard decisions, while the learned irregular QAM constellation may require modifications to conventional modulation and demodulation pipelines. The evaluation relies on simulated AWGN and Rayleigh fading channels, without real-world baseband or over-the-air experiments. Furthermore, the current ablations do not sufficiently isolate the contributions of continuous reception and learned constellation mapping. The integration requirements, computational overhead, and practical benefits of the proposed design therefore remain insufficiently established.

Questions for authors
  1. Comparison with prior work: How does AsymVQ-JSCC differ fundamentally from SCDM (Mo et al., 2025) in terms of receiver-side recovery and its underlying optimization objective? Can the authors provide a direct comparison under matched channel conditions and bandwidth budgets to clarify the advantages of the proposed approach?

  2. Ablation and component contributions: Appendix C.5 projects the continuous receiver's output onto the nearest source codeword at inference time, while keeping the model trained for continuous reception fixed. However, this does not provide a fair comparison with a hard-decision receiver trained to handle channel noise. A proper baseline should perform hard detection and codebook lookup, with the VQ-VAE codec trained under the same noisy-channel conditions to account for the resulting symbol errors. Could the authors provide independently trained end-to-end baselines covering regular and learned irregular constellations, each with both hard-decision and continuous reception? Such comparisons are necessary to isolate the contributions of the constellation mapper and continuous receiver.

  3. Practical receiver interface: What signal interface does the proposed receiver require in a practical digital communication system? Specifically, does deployment require access to pre-decision I/Q samples and modifications to the conventional modulation and demodulation chain to support the learned irregular constellation? Please clarify the assumed implementation and its practical overhead.

Reviewer_YmMfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose AsymVQ-JSCC. It keeps a discrete codebook at the transmitter and sends one learned 2D constellation point per spatial token. The receiver does not detect the symbol or look up the codeword. One linear map turns the noisy observation into a continuous latent, and the decoder reconstructs from those latents.

Critical strengths

The paper is clearly written. I did not find obvious language problems, and the derivations are correct under the assumptions. The problem is also meaningful. In VQ transmission, the receiver is usually required to recover the exact codeword before reconstruction, even though a good image does not require that hard decision. That distinction is worth making.

Critical weaknesses

I have some concerns about the experimental setup and conclusions. First, the authors did not test their main claim: On Kodak, removing channel noise and then projecting onto the codebook drops PSNR from 22.71dB to 20dB. The drop is larger than the range reported for the noisy SNRs. It shows that the decoder was trained to use continuous latents, but it does not show that asymmetric recovery beats a lookup receiver trained for the same codec and the same channel-use. There is also no check of whether the nearest codeword is the one was sent. Second. The perceptual improvement cannot be attributed to the receiver interface. AsymVQ-JSCC is trained with an LPIPS loss and an adversarial loss. The DeepJSCC checkpoints are selected by validation mean squared error. Under this difference in training objectives, a lower LPIPS value does not by itself establish the benefit of removing codebook lookup. Third. The contribution of the learned constellation is not isolated. The paper provides no comparison between the proposed constellation and a regular constellation when both use the same continuous receiver.

Questions for authors

My questions are listed in the Critical Weaknesses section above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.