- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper systematically investigates how different parity-check matrix representations affect neural decoding performance. The authors first demonstrate experimentally that algebraically equivalent parity-check matrices can lead to different neural decoding performance. They then propose using redundant dual parity-check relations and introducing observation-dependent reliability information for each relation. The paper conducts extensive controlled experiments, primarily based on CrossMPT.
I believe this research question is meaningful, and the experimental analysis is conducted carefully.
Critical strengths
-
Investigating the impact of different equivalent parity-check matrices on the performance of Transformer-based decoders is an important research direction. Previous works, such as MM-ECCT [1], have also explored related issues.
-
The experimental analysis is systematic and progressive. Through row-weight matching, NativeRepeat, Random-WM, and reliability-shuffling experiments, the authors attempt to distinguish different factors affecting decoding performance. The experimental design is relatively rigorous.
Critical weaknesses
-
The contributions of this paper are primarily empirical rather than algorithmic. The experiments demonstrate that the specific representation of a parity-check matrix affects neural decoding performance. However, the paper does not establish a general principle for determining which parity-check matrix representations are more suitable for neural decoders. For example, the large performance differences between equivalent parity-check matrices are partly attributable to variations in graph density, while the remaining performance differences become considerably smaller after controlling for row weights. In fact, a similar observation was already implicit in MM-ECCT. Therefore, the paper mainly identifies and analyzes the sensitivity of neural decoders to parity-check representations, but does not provide a clear answer to how better representations can be systematically constructed.
-
The proposed relation-selection method lacks consistent performance advantages. Although the experiments show that introducing distinct redundant parity-check relations can improve decoding performance, the authors' specific selection strategy is not consistently superior. On Gallager codes, Selected and Random-WM exhibit almost no significant performance difference. On MacKay codes, Random-WM even outperforms Selected across multiple random seeds. Furthermore, on BCH codes, there is also a substantial performance gap compared with the recent plug-and-play method proposed by Xiao et al. [2].
Questions for authors
The authors' investigation of how parity-check matrix representations affect neural decoding performance is relatively systematic and clearly involves a considerable amount of work. However, the final proposed method and its performance improvements are rather limited. Moreover, some of the conclusions are relatively straightforward in the context of decoding, which makes the overall contribution of the paper insufficiently compelling.
Please refer to the major weaknesses above for the specific concerns.
References
[1] S.-J. Park, H.-Y. Kwak, S.-H. Kim, S. Kim, Y. Kim, and J.-S. No. Multiple-Masks Error Correction Code Transformer for Short Block Codes. IEEE Journal on Selected Areas in Communications, 43(7):2518–2529, 2025.
[2] S. Xiao, W. Fang, and Q. Zhang. Drop-in Circulant Structural Priors for Transformer Decoding of Cyclic Codes. Proceedings of the 43rd International Conference on Machine Learning (ICML), PMLR 306:138689–138707, 2026.
2 comments
Response to Reviewer U3Tu (2/2)
3. Gap to Xiao et al. on BCH codes. The reference-scale experiments substantially change our BCH results: on BCH(63,36), Selected+ reaches at 4/5/6 dB, versus for published CrossMPT. Xiao et al.'s circulant prior exploits cyclic structure and therefore targets cyclic codes. Our relation vocabulary and confidence feature require only a parity-check matrix, and we apply them to non-cyclic LDPC, MacKay, and CCSDS codes as well. For cyclic codes, the set of low-weight dual relations is closed under cyclic shifts, so the two approaches may partly exploit the same structure; we cite Xiao et al. in Related Work and view the methods as complementary.
4. "Some conclusions are relatively straightforward." We accept that the high-level message is natural from a coding-theory viewpoint; indeed, the method is directly motivated by the dual code. Two results were not obvious to us in advance. First, at an equal number of relation tokens and equal compute, replacing repeated checks with distinct relations matters, so the gain is not simply from more tokens. Second, CrossMPT already receives every , yet explicitly aggregating them per relation still helps, while a 33-parameter placebo does not. This suggests that, at these scales, the backbone does not learn relation-level aggregation by itself.
We thank the reviewer again and are happy to provide further details.
Response to Reviewer U3Tu (1/2)
We thank the reviewer for separating two questions: whether neural decoders are sensitive to the parity representation, and whether we know how to construct a good one.
1. Empirical contribution; density; relation to MM-ECCT. We agree with the reviewer on the facts. Most of the dense-basis gap is explained by row density, and the residual under exact row-weight matching is modest: 1.28× at 5 dB on BCH(31,16) (Table 1), and 1.21±0.04× over 3–5 dB in the compact Learned(31,16) control (three seeds, identical initialization and data streams). We also agree that Multiple-Masks ECCT already implies that the choice of parity-check matrix matters. We therefore position basis sensitivity (C1) as a controlled diagnostic that motivates the method rather than as a central contribution.
On the construction principle: the method is inspired by the dual code in coding theory. Every dual codeword is a valid parity check, and classical decoders use multiple bases, redundant checks, and low-weight dual codewords because each provides a new check neighborhood (Section 4.1). In the soft-decoding view, the confidence of a relation u, ∏i∈supp(u)tanh(∣Li∣/2), decays with relation weight and is limited by its weakest bit. This predicts large redundancy gains where native checks are dense, and larger confidence gains where native checks are already sparse. Table 2 follows this pattern: BCH codes gain up to about 80× lower BER at 6 dB from redundancy, while LDPC(121,60) and CCSDS gain little from redundancy but about 22% from ρ at 5 dB.
2. The selection rule is not consistently better. We agree, and the revised paper no longer claims that it is. The evidence is mixed: on compact Gallager, Selected and Random-WM are essentially tied (0.71±0.40%); on MacKay with the 300k-step protocol (three seeds, five support draws), Random-WM is better by 11.85±2.31%; on MacKay at reference scale (Table 3), Selected is better by 6–10%.
Relation identities can therefore change BER at a fixed budget, rank, and row-weight histogram (by roughly 6–12%), but the direction of the effect varies across protocols. Our deterministic selector does not establish a consistently superior mechanism. We state this in Sections 6.3 and 7 and identify learned, observation-adaptive, or backbone-aware selection as a key direction for future work.
The effect that holds across all evaluated settings is the gain from confidence conditioning: it appears for both relation sets, improving Random-WM by 13–16% (Table 3), while a parameter-matched placebo gives no gain (Appendix Table 8). Same-weight shuffling of ρ shows that the correct assignment lowers BER by 57–96% on MacKay and 75–95% on Gallager (Appendix Table 10). Replacing repeated checks with distinct relations helps on most codes (24 of 27 points in Table 2, and for both supports on Gallager and MacKay), but not on LDPC(121,60).