acceptodds
Under review as a conference paper at ICLR 2027

Embedded Neural Belief Propagation: More Dimensions, Better Beliefs

Abstract

Recently, Transformer-based decoders have achieved state-of-the-art performance on short codes, surpassing conventional belief propagation (BP) decoders. However, processing high-dimensional representations through attention incurs substantial computational cost. In this paper, we propose Embedded Neural Belief Propagation (E-NBP), which combines the efficiency of BP with the expressive power of high-dimensional representations. E-NBP embeds channel information into d-dimensional vectors and propagates message vectors along Tanner-graph edges. A key design choice is to perform intra-dimensional check-node updates to maintain low complexity, while performing cross-dimensional variable-node updates to improve decoding performance. On CCSDS(128, 64) at Eb/N0 = 6 dB, E-NBP reduces BER by 98.1% relative to CrossMPT while requiring 73.7% fewer FLOPs than EfficientMPT. These findings show that vector messages enable BP to compete with Transformer-based decoders while retaining its computational efficiency.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
error correcting codes, neural decoding, neural belief propagation, LDPC codes, message passing, Tanner graph

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_1Wy2OpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Main Idea: The paper extends neural belief propagation by replacing scalar messages with high-dimensional vectors. Variable-to-check updates use learned linear transformations to mix information across dimensions, while check-to-variable updates retain the familiar BP sum-product operation independently within each dimension. A learned readout converts these vectors into bit decisions. The goal is to improve decoding through richer message representations while retaining BP’s structured, relatively efficient computation. Overall, I feel this is a genuinely new and interesting idea.

Perhaps the most striking part is the evaluation and the gains appear to be substantial. Authors test E-NBP across several code families (including BCH, Polar, LDPC etc), and show a) reduced BER across a broad range of SNR values; while b) reduced flops.

Critical strengths
  1. Idea is clean and seems novel. There is substantial interest in designing low-complexity error correcting codes for a wide variety of applications and the approach seems to be generally applicable.
  2. Easy to implement and shows gains in terms of performance (reduced BER) and complexity (lower FLOPS) in a wide range of conditions.
  3. Paper is written very well (though this is expected these days from a good paper :)
Critical weaknesses

None.

Questions for authors
  1. Choosing the message dimension: It would be good if authors can provide practical guidance for selecting d when applying E-NBP to a new code family, particularly how to balance dimension against the number of decoding iterations under a fixed computational budget? The dimension–iteration ablations are helpful, but it remains a bit unclear how the preferred choice depends on the code’s Tanner-graph structure.

  2. Intuition for when dimensionality expansion helps: Could the authors elaborate on when vector-valued messages are expected to provide the largest gains over scalar BP—for example, whether the benefit is particularly pronounced in graphs with short cycles, where correlated messages can contribute to persistent decoding errors? I am very interested in understanding "why" this helps; not just the empirical results themselves. Bringing the relevant insights and analysis from the appendix (which has some discussion) earlier into the main paper would strengthen the motivation and help readers understand why the proposed expansion is effective. If I missed a discussion sitting something else, please do point out!!

3 comments
AuthorsOpenReview ↗

Choosing Message Dimension and Iteration Count (Q1)

Reviewer comment

Choosing the message dimension: It would be good if authors can provide practical guidance for selecting d when applying E-NBP to a new code family, particularly how to balance dimension against the number of decoding iterations under a fixed computational budget? The dimension–iteration ablations are helpful, but it remains a bit unclear how the preferred choice depends on the code's Tanner-graph structure.

Response

We thank the reviewer for the positive evaluation of our work and for the insightful questions on practical use and the underlying mechanism. These comments led us to consider not only how E-NBP improves performance, but also how users can choose its configuration and understand the effect of dimension expansion. We have addressed both aspects with an iteration–dimension comparison on additional codes and a concrete case study of error recovery.

In addition to the existing BCH(63,45) analysis (Figure 6(a)), we compared combinations of the number of iterations ℓˉ\bar{\ell} and the message dimension dd on LDPC(49,24) and Polar(64,32). The table below reports BER at 5 dB and FLOPs per codeword.

(ℓˉ,d)(\bar{\ell}, d)LDPC(49,24) BERFLOPs (M)Polar(64,32) BERFLOPs (M)
(3,256)3.9e-479.23.5e-4139.0
(6,128)4.0e-540.75.4e-571.4
(12,64)2.0e-521.43.9e-537.5
(16,48)2.6e-516.64.8e-529.1
(24,32)2.9e-511.85.2e-520.6

All configurations satisfy ℓˉ×d=768\bar{\ell} \times d = 768, but their FLOPs differ due to the cost of the cross-dimensional transformations; we therefore report the actual FLOPs of each configuration. The newly trained (3,256), (16,48), and (24,32) models are evaluated at the best-validation checkpoint with early stopping, while (6,128) and (12,64) use the existing results. These results are also visualized in the iteration–dimension sweep plot.

On both codes, (12,64) achieves the lowest BER among the five configurations. More importantly, the two directions of deviation from (12,64) are highly asymmetric:

  • Too few iterations is costly. Reducing the iterations to 3 increases BER by about 20× on LDPC(49,24) and 9× on Polar(64,32), even with more FLOPs.
  • A smaller dimension is much less costly. Reducing the dimension to 32 increases BER by only about 1.4×, while halving the FLOPs.

In addition, since the FLOPs of E-NBP scale as ℓˉd2\bar{\ell}d^2, the message dimension is the more expensive axis. Figure 5(a) also shows that, once ℓˉ=12\bar{\ell} = 12 is secured, increasing dd steadily improves performance.

Practical guidance. Based on these observations, we suggest the following rule for a given compute budget:

  1. Secure enough iterations first (around 10–12 in our experiments).
  2. Then use the remaining budget to increase the message dimension dd.
  3. If the budget is tight, reduce dd rather than the number of iterations, since this causes only a modest performance loss.

Our main experiments follow this rule: (ℓˉ,d)=(12,64)(\bar{\ell}, d) = (12, 64) for short codes and (10,128)(10, 128) for long codes.

Dependence on Tanner-graph structure. This asymmetric trend is consistently observed across BCH, LDPC, and polar codes with different Tanner-graph structures, although the magnitude of degradation varies by code. We therefore expect the above rule to serve as a reliable starting point for a new code family, followed by a comparison of nearby configurations based on the actual FLOPs and BER on its Tanner graph.

AuthorsOpenReview ↗

Why Dimensionality Expansion Helps (Q2)

Reviewer comment

Intuition for when dimensionality expansion helps: Could the authors elaborate on when vector-valued messages are expected to provide the largest gains over scalar BP—for example, whether the benefit is particularly pronounced in graphs with short cycles, where correlated messages can contribute to persistent decoding errors? I am very interested in understanding "why" this helps; not just the empirical results themselves. Bringing the relevant insights and analysis from the appendix (which has some discussion) earlier into the main paper would strengthen the motivation and help readers understand why the proposed expansion is effective. If I missed a discussion sitting something else, please do point out!!

Response

We thank the reviewer for this thoughtful question. Due to the page limit, we were unable to move the existing appendix discussions into the main text. Instead, we strengthened the main text with a new case study that directly illustrates why dimension expansion helps, presented in Section 6.1 and the newly added Figure 5cc.

Specifically, on BCH(63,45), we analyze a frame that NBP fails to decode but E-NBP recovers, with the same six iterations. In this frame, we select two VNs that form a 4-cycle and compare the final LLRs of NBP with the dimension-wise readout contributions of E-NBP. This shows how the correct-direction contributions of some dimensions outweigh the incorrect-direction contributions of others, flipping the final decision to the correct one. This provides an intuitive explanation of why vector messages help: whereas a scalar message on a short cycle can only reinforce a single, possibly wrong belief, vector messages allow different dimensions to carry complementary evidence.

Regarding when the gains are largest, we observe that E-NBP provides similar gains over scalar BP across codes regardless of the number of short cycles in their Tanner graphs. We believe this rather highlights the universality of E-NBP: the benefit of vector messages is not limited to a particular graph structure, but extends consistently across diverse code families.

Related discussions can also be found in the following parts of the paper:

  • Section 6.1, Figure 5(b): E-NBP recovers more low-weight residual errors, which are commonly associated with trapping sets, than NBP and CrossMPT.
  • Appendix E, Figure 8: Different message dimensions dominate error correction at different iterations, showing that the dimensions play distinct roles.
  • Appendix F, Proposition 3 / Figure 9 (heatmaps, off-diagonal energy): E-NBP learns cross-dimensional coupling rather than acting as an ensemble of independent scalar decoders.
AuthorsOpenReview ↗

Closing Remarks

We thank the reviewer for the positive evaluation and the insightful questions on the practical use and underlying mechanism of E-NBP. Specifically, we have made the following revisions:

  • We added iteration–dimension comparisons on LDPC(49,24) and Polar(64,32), and provided practical guidance for choosing the configuration under a given compute budget.
  • We added a case study (Section 6.1, Figure 5cc) illustrating why vector messages help, showing how complementary contributions from different dimensions correct errors on a 4-cycle where scalar NBP fails.
  • We clarified that E-NBP provides consistent gains regardless of the number of short cycles, and pointed to the related analyses in Figure 5(b), Appendix E, and Appendix F.

In addressing the reviewer's comments, we were able to provide clearer practical guidance and a more intuitive understanding of why dimension expansion is effective. We hope that our responses further strengthen the reviewer's positive assessment of our work.

We sincerely thank the reviewer for the time and effort devoted to providing a thoughtful and constructive review.

Reviewer_Q4ovOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes E-NBP, which replaces scalar BP messages with vectors. The variable-node updates mix information across dimensions, while the check-node updates apply weighted BP operations separately in each dimension. The goal is to improve decoding without using attention. The experiments cover several code families and compare E-NBP with BP-based and Transformer-based decoders.

Critical strengths

The update rules are simple and easy to follow. Keeping the check-node operation close to BP while learning how to mix dimensions at variable nodes is a reasonable design.

The reported BER gains are large on several BCH and LDPC codes, and the experiments also include longer codes. Table 2 is useful: adding multiple message dimensions already helps a lot, and adding BEB_E gives a further clear improvement. This shows that the full result is not coming from the embedding alone.

Critical weaknesses

I'm not yet convinced by the comparison with existing vector-message decoders. GNN decoders already use high-dimensional messages, so the new part here is mainly the particular update rule. However, Appendix A says the GNN baselines are trained with eight or ten rounds but evaluated with only six, while the main short-code E-NBP results use twelve rounds. This may underestimate the GNN baseline. A comparison at the trained depth, together with matched-compute results, is important for judging the contribution. The embedding-only version is also a strong baseline. Since the full model adds parameters and computation, a comparison at a similar budget would help show how much the cross-dimensional updates add.

Questions for authors

Why are the GNN baselines evaluated with fewer rounds than they were trained with? Could you show results at their trained depth and compare the accuracy–compute trade-off? On a representative code, how does the full model compare with the embedding-only version at a similar parameter or compute budget? Could you provide measured latency or throughput alongside the operation counts?

5 comments
AuthorsOpenReview ↗

Accuracy–Compute Comparison with GNN (W1–W5, Q1)

Reviewer comment

I'm not yet convinced by the comparison with existing vector-message decoders. GNN decoders already use high-dimensional messages, so the new part here is mainly the particular update rule. However, Appendix A says the GNN baselines are trained with eight or ten rounds but evaluated with only six, while the main short-code E-NBP results use twelve rounds. This may underestimate the GNN baseline. A comparison at the trained depth, together with matched-compute results, is important for judging the contribution. Why are the GNN baselines evaluated with fewer rounds than they were trained with? Could you show results at their trained depth and compare the accuracy–compute trade-off?

Response

We thank the reviewer for the careful review and for suggesting concrete directions for comparison. Following these suggestions, we added a GNN comparison with matched iterations and dimension, together with measurements of inference throughput and GPU memory, to clarify both the performance and the cost of E-NBP.

The GNN results in our paper strictly follow the configurations of the original paper and its public source code (Cammerer et al., 2022): the models are trained with (ℓˉ,d)=(8,20)(\bar{\ell}, d) = (8, 20) for BCH codes and (10,16)(10, 16) for polar and LDPC codes, and evaluated with ℓˉ=6\bar{\ell} = 6. We kindly refer the reviewer to the original paper for the rationale behind this setting. Nevertheless, to ensure a fair comparison, we additionally trained and evaluated GNN with (ℓˉ,d)=(12,64)(\bar{\ell}, d) = (12, 64), the same configuration as E-NBP. This setting exceeds both the trained depth and the message dimension of the original configurations, and therefore gives GNN a stronger baseline.

The table below reports BER at 5 dB for both the original and the new GNN settings.

CodeGNN (original) BERGNN (12,64) BERE-NBP (12,64) BERGNN (12,64) FLOPs (M)E-NBP (12,64) FLOPs (M)
BCH(31,16)1.6e-31.7e-37.3e-548.615.3
BCH(63,45)2.4e-31.3e-39.0e-5121.740.1
Polar(64,32)3.0e-38.7e-43.9e-5116.837.5
Polar(64,48)1.8e-38.0e-42.1e-491.829.2
LDPC(49,24)3.2e-41.0e-42.0e-569.921.4

Under the same 12 iterations and dimension 64, E-NBP achieves lower BER with fewer FLOPs than GNN on all five codes. For example, on BCH(63,45), E-NBP reduces FLOPs by 67.0% and BER by 92.9%. Although both GNN and E-NBP use vector messages, these results show that retaining the BP update rules, as in E-NBP, improves both decoding performance and computational efficiency.

AuthorsOpenReview ↗

Comparison with Embedding-Only Baseline at a Similar Compute Budget (W6–W7, Q2)

Reviewer comment

The embedding-only version is also a strong baseline. Since the full model adds parameters and computation, a comparison at a similar budget would help show how much the cross-dimensional updates add. On a representative code, how does the full model compare with the embedding-only version at a similar parameter or compute budget?

Response

We thank the reviewer for this insightful suggestion. On LDPC(49,24), we compare E-NBP with (12,64) and Embed-Only-NBP with (12,751). Embed-Only-NBP is the embedding-only baseline, which excludes the trainable extrinsic transformation B_E and self-feedback transformation B_S. Since its cost grows only linearly in d, we increased its dimension to match the compute of E-NBP. The resulting FLOPs are 21.4M and 23.1M, respectively, so Embed-Only-NBP is given 7.9% more computation.

LDPC(49,24)E-NBP (12,64)Embed-Only-NBP (12,751)
Message dimension64751
Trainable parameters (K)255.1 (100.0%)1,814.5 (711.3%)
FLOPs/codeword (M)21.4 (100.0%)23.1 (107.9%)
Training time/epoch (s)36.4 (100.0%)140.5 (386.4%)
BER @ 5 dB2.0e-52.3e-5

Percentages are relative to E-NBP (100.0%).

At a similar compute budget, E-NBP achieves similar or slightly better BER than Embed-Only-NBP while using 7.1× fewer parameters and 3.9× faster training per epoch. To reach this level of performance without cross-dimensional updates, the message dimension must be increased by more than 10×, which substantially enlarges the model and makes training much slower. In contrast, the cross-dimensional updates of E-NBP use computation more efficiently, achieving comparable performance with a compact model that is far faster to train.

AuthorsOpenReview ↗

Measured Inference Throughput and GPU Memory (Q3)

Reviewer comment

Could you provide measured latency or throughput alongside the operation counts?

Response

We thank the reviewer for this suggestion. We have newly added measured inference throughput and peak GPU memory in Figure 4cc and (d) of the revised manuscript.

Inference throughput. Measured on an NVIDIA RTX A5500 with a batch size of 128, and reported in codewords/s (higher is better).

CodeECCTEfficientMPTCrossMPTE-NBP
BCH(63,36)13,48021,84214,51943,301
BCH(63,45)15,21124,99716,87361,357
Polar(64,48)15,60825,33119,20072,843
LDPC(49,24)16,30325,26816,758112,243
LDPC(121,60)4,74410,5426,60731,433
CCSDS(128,64)4,62210,5706,52541,319
WiMAX LDPC(576,432)OOM1,5736645,352
WiMAX LDPC(1056,880)OOM8803033,010
WiMAX LDPC(2304,1920)OOM311OOM1,378

Peak GPU memory. Memory required for a batch size of 128, in MiB (lower is better).

CodeECCTEfficientMPTCrossMPTE-NBP
BCH(63,36)330.780.5120.440.1
BCH(63,45)277.679.4107.446.5
Polar(64,48)277.179.0103.141.2
LDPC(49,24)260.369.7103.048.4
LDPC(121,60)1,218.0129.1344.747.5
CCSDS(128,64)1,277.3133.1345.356.4
WiMAX LDPC(576,432)OOM483.24,148.068.4
WiMAX LDPC(1056,880)OOM850.49,140.673.1
WiMAX LDPC(2304,1920)OOM1,817.1OOM97.3

OOM denotes out of GPU memory under the given setting.

The results in both tables are also visualized in Figure 4cc and (d) of the revised manuscript, and we kindly invite the reviewer to refer to them.

On every evaluated code, E-NBP achieves higher throughput and lower peak GPU memory than EfficientMPT, the most efficient Transformer baseline, with up to 4.4× higher throughput and 18.7× lower memory. E-NBP also outperforms CrossMPT and ECCT on both metrics wherever they run successfully. Notably, the memory of E-NBP stays below 100 MiB even on the length-2304 code, where ECCT and CrossMPT run out of memory.

Additionally, we present measurement results for the codes listed in Table 8: inference throughput and peak GPU memory.

Overall, these additional comparisons show that E-NBP is designed to be efficient not only in decoding performance, but also in computation, training cost, and actual GPU execution. The reviewer's concrete requests helped us present the advantages of E-NBP more clearly across these metrics. We will include the additional results and measurement settings in the revised manuscript.

Reviewer_SpyYOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes a variant of neural BP decoding, referred to as Embedded Neural BP to decode graph-based codes. In particular, the paper proposes a vector message-update rule, as opposed to the conventional scalar update. The authors show improvements over other neural-BP decoding baselines for several polar, BCH, and LDPC codes. The approach is of some interest, but the contribution is very minor.

Critical strengths

The proposed neural BP decoder improves on the selected baselines.

Critical weaknesses

The main weakness is that the contribution is too limited and lacks sufficient depth to warrant publication at ICLR. The shown improvements with respect to other neural BP decoders are promising, but the contribution is simply too narrow.

The approach itself is indeed quite straightforward. In my view, a simple idea has been expanded to a full ICLR paper. I find very little in the proposed approach that is surprising or conceptually substantial.

Note that, improving upon existing methods does not, by itself, warrant a publication, particularly at a venue such as ICLR. Indeed, a lot of purely engineering work can improve upon the state-of-the-art, without constituting a significant research contribution.

The construction is largely heuristic, with neither theoretical analysis nor novel insight. The paper does not demonstrate a substantial advance in scientific understanding or methodology.

The comparison is with standard BP and other neural BP decoders. For the polar and BCH codes this comparison makes little sense. There are far better decoders than BP (CRC+list decoding for polar codes and OSD for BCH codes).

Questions for authors

Please provide comparison results with state-of-the-art decoders, see my comment above.

5 comments
AuthorsOpenReview ↗

Contributions of E-NBP (W1–W9)

Reviewer comment

The main weakness is that the contribution is too limited and lacks sufficient depth to warrant publication at ICLR. The shown improvements with respect to other neural BP decoders are promising, but the contribution is simply too narrow. The approach itself is indeed quite straightforward. In my view, a simple idea has been expanded to a full ICLR paper. I find very little in the proposed approach that is surprising or conceptually substantial. Note that, improving upon existing methods does not, by itself, warrant a publication, particularly at a venue such as ICLR. Indeed, a lot of purely engineering work can improve upon the state-of-the-art, without constituting a significant research contribution. The construction is largely heuristic, with neither theoretical analysis nor novel insight. The paper does not demonstrate a substantial advance in scientific understanding or methodology.

Response

We thank the reviewer for taking the time to review our paper and for the constructive comments. We agree that an ICLR paper requires more than an engineering improvement, and we believe E-NBP makes a meaningful contribution to neural channel decoding. The reviewer's comments gave us the opportunity to clarify the contribution and novelty of our work, as summarized below. We would be grateful if the reviewer could take a look.

1. [Methodological contribution] E-NBP extends BP messages to vectors while keeping the BP update rules intact, adding only linear cross-dimensional mixing. Unlike GNN-based decoders, it does not introduce general-purpose neural networks (e.g., MLPs) into the message updates, and therefore preserves BP's key property of codeword symmetry. In this sense, E-NBP is a principled vector extension of BP, rather than a heuristic modification.

2. [Theoretical analysis] (newly added in Appendix F) To address the concern that the construction is heuristic, we have added a formal analysis in Appendix F. These results establish E-NBP as a distinct decoder class, separate from NBP, GNN-based decoders, and ensembles of BP decoders.

  • Proposition 1: E-NBP contains BP and NBP as special cases.
  • Proposition 2: E-NBP preserves codeword symmetry, which general MLP-based GNN decoders do not guarantee.
  • Proposition 3 / Figure 9 (heatmaps, off-diagonal energy): E-NBP does not reduce to an ensemble of independent branches; it actually learns cross-dimensional interactions.

3. [Novel insight] E-NBP provides new insight into the sources of neural decoding performance.

  • High-dimensional representation is a key factor. Embedding alone, without any cross-dimensional exchange, reduces BER by up to 88% relative to NBP (Table 2). This suggests that the gains of Transformer-based decoders may stem from high-dimensional representations rather than attention.
  • Message dimension is a new design axis. E-NBP with 12 iterations achieves performance that BP cannot reach even with 800 iterations (Figure 5(a)), showing that message dimension is a design axis distinct from the number of iterations.
  • E-NBP resolves persistent BP failures. It recovers trapping-set-related errors more effectively than NBP and CrossMPT (Figure 5(b)). In the newly added Figure 5cc, we present a concrete example in which scalar NBP fails but E-NBP succeeds, showing how complementary contributions from different message dimensions turn incorrect bit decisions into correct ones.

4. [Practical contribution] Without attention, E-NBP outperforms Transformer-based SoTA decoders in both performance and complexity. On CCSDS(128,64), it achieves a 0.60 dB gain over CrossMPT while requiring 73.7% fewer FLOPs than EfficientMPT.

In summary, E-NBP changes only the message representation of BP, without adding attention or general-purpose neural networks, yet this alone is sufficient to surpass Transformer-based SoTA decoders. We believe this finding is both surprising and conceptually meaningful: it identifies message dimension as a new design axis for BP decoders and shows that attention is not necessary for strong neural decoding, offering a meaningful advance for both the channel coding and machine learning communities.

AuthorsOpenReview ↗

Comparison with OSD and SCL Decoders (W10–W12, Q1)

Reviewer comment

The comparison is with standard BP and other neural BP decoders. For the polar and BCH codes this comparison makes little sense. There are far better decoders than BP (CRC+list decoding for polar codes and OSD for BCH codes). Please provide comparison results with state-of-the-art decoders, see my comment above.

Response

Following the reviewer's suggestion, we additionally evaluated OSD-2 on BCH, polar, and LDPC codes, and compared with SCL-4 on the polar code. Since the polar code in our benchmark does not include a CRC, we compare with standard SCL; the SCL result is taken from Appendix E, Table 4 of CrossMPT. All values below are BER.

CodeOSD-2Polar SCL-4CrossMPTE-NBP
BCH(63,45) @ 5 dB7.3e-6N/A2.7e-49.0e-5
Polar(64,32) @ 5 dB2.2e-52.3e-54.7e-53.9e-5
LDPC(121,60) @ 4 dB3.7e-5N/A3.2e-39.3e-4
WiMAX LDPC(576,432) @ 3.5 dB3.8e-3N/A8.3e-54.8e-5
ComplexityO(n3)O(n^3)O(nlog⁡n)O(n\log n)O(n2)O(n^2)O(E)O(E)

N/A indicates that polar SCL is not directly applicable. Complexity is given for fixed code rate, OSD order, SCL list size, number of iterations/layers, and message dimension; nn is the code length and EE is the number of Tanner-graph edges.

On the short codes, OSD-2 and SCL-4 outperform both CrossMPT and E-NBP. However, on the long WiMAX LDPC(576,432) code, E-NBP outperforms OSD-2 with linear rather than cubic complexity, showing that OSD does not scale well to long codes. In addition, SCL is applicable only to polar codes. Moreover, E-NBP achieves lower BER than CrossMPT, a universal decoder, on every code in the table. We therefore do not claim that E-NBP outperforms classical decoders on every code class; rather, we position E-NBP as an efficient, universally applicable decoder.

Beyond universality, E-NBP inherits the strengths of BP: it is hardware-friendly and has linear complexity on sparse graphs. At the same time, it overcomes a key limitation of BP, namely that its strong performance has been largely confined to LDPC codes, making E-NBP a new decoder class that is effective across BCH, polar, and LDPC codes.

The reviewer's comments helped us clarify the position of E-NBP relative to classical decoders. We will include these additional comparisons and the discussion in the revised manuscript.

AuthorsOpenReview ↗

Closing Remarks

We thank the reviewer for the careful review and the candid comments. Specifically, we have made the following revisions:

  • We clarified the contributions of E-NBP from structural, theoretical, conceptual, and performance perspectives.
  • We added a formal analysis (Appendix F) showing that E-NBP is a principled decoder class, distinct from NBP, GNN-based decoders, and ensembles of BP decoders.
  • We added a case study (Figure 5cc) showing how complementary contributions from different message dimensions correct errors that scalar NBP fails to resolve.
  • We added comparisons with OSD-2 and SCL-4, showing that E-NBP outperforms OSD-2 on a long LDPC code with linear complexity while remaining universally applicable.

In addressing the reviewer's comments, we were able to clarify the contribution and novelty of E-NBP beyond performance improvements. We respectfully ask the reviewer to reconsider the evaluation of our paper in light of our responses and additional results.

We sincerely thank the reviewer for the time and effort devoted to reviewing our work.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal: Addressed Concerns and the Position of E-NBP

We thank the AC and the reviewers for their careful and constructive reviews. We have responded to each reviewer in detail, and the reviewers' main concerns can be summarized into the following five issues.

  • Scope of contribution: Does replacing scalar updates with vector-message updates, a simple idea, constitute a well-grounded contribution beyond performance improvement?
  • Comparison with existing vector-message decoders: Is the GNN baseline underestimated, given that it was evaluated with fewer iterations than it was trained with?
  • Value of cross-dimensional updates: What do they add over an embedding-only configuration at a similar budget?
  • Practical execution cost: Do the FLOP reductions carry over to measured throughput and memory?
  • Position relative to classical decoders: How does E-NBP compare with classical decoders such as OSD and SCL?

1. Main concerns and responses

We addressed these concerns with additional experiments and analyses.

ConcernAdded evaluationKey result
Scope of contributionFormal analysis of the structural properties of E-NBPE-NBP contains BP and NBP, preserves codeword symmetry, and learns cross-dimensional interactions rather than acting as an ensemble of independent NBPs
Comparison with existing vector-message decodersRetrained GNN with the same 12 iterations and dimension 64 as E-NBPE-NBP achieves lower BER with about 3× fewer FLOPs on all five codes
Value of cross-dimensional updatesCompared with Embed-Only-NBP at similar FLOPs on LDPC(49,24)Embed-Only-NBP requires dimension 751, whereas E-NBP reaches comparable BER with dimension 64 (7.1× fewer parameters, 3.9× shorter training time per epoch)
Practical execution costMeasured throughput and peak GPU memory on nine codesUp to 4.4× higher throughput and 18.7× lower memory than EfficientMPT; below 100 MiB at length 2304, where ECCT and CrossMPT run out of memory
Position relative to classical decodersCompared with OSD-2 (BCH, polar, LDPC) and SCL-4 (polar)OSD-2 and SCL-4 are better on short codes; E-NBP outperforms OSD-2 on the long WiMAX LDPC code with linear complexity and applies across code families

2. Where E-NBP stands: is attention necessary?

Following ECCT (NeurIPS 2022), research on Transformer-based decoders has continued with CrossMPT (ICLR 2025) and EfficientMPT (ICLR 2026), yet the high computational cost of attention remains a barrier to practical deployment.

E-NBP asks the following question: Can we obtain the benefits of high-dimensional representations while retaining BP's structured computation? E-NBP answers this question in the affirmative by extending BP messages to vectors and adding only cross-dimensional mixing, a linear operation. This makes message dimension a new design axis for BP-based decoding, distinct from the number of iterations.

In other words, attention is not a prerequisite for strong neural decoding. Without attention, E-NBP keeps BP's updates intact and extends only the message representation, a simple structure with linear complexity that retains hardware-friendly computation. Even so, it delivers strong decoding performance across diverse code families, overcoming the long-standing limitation that BP's strong performance has been largely confined to LDPC codes. The simplicity of E-NBP is thus a strength rather than a limitation. By expanding representations on top of BP, which is already widely used in practical communication systems, E-NBP offers a meaningful advance for both the channel coding and machine learning communities.

Closing remarks

As an attention-free neural decoder that applies a single update structure across diverse code families, E-NBP opens a new direction between scalar neural BP and attention-based decoders, toward a general-purpose neural decoder. We respectfully ask that the paper be assessed in light of the central finding that BP-based computation combined with high-dimensional messages alone can deliver strong neural decoding without attention. We thank the AC and the reviewers again for their careful review and constructive feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.