acceptodds
Under review as a conference paper at ICLR 2027

BeliefFlow: Continuous-Time BP-Structured Neural Decoding for LDPC Codes

Abstract

Belief propagation (BP) decodes low-density parity-check (LDPC) codes through sparse local message passing. Recent Transformer-based decoders achieve strong performance using parity-aware attention without explicitly implementing BP's update rules. This raises the question of whether strong performance can be achieved while retaining BP's computational structure. We introduce BeliefFlow, which evolves vector-valued Tanner-edge states in continuous time using a shared neural vector field. Its local operators combine attention-based aggregation over destination-excluded neighborhoods with gated, element-wise box-plus operations. Training uses duration-aware loss reweighting to emphasize difficult examples, while randomized perturbations generate diverse retry trajectories at inference. Across six LDPC codes, the 0.34M-parameter decoder leads the comparison in 17 of 18 code/SNR settings, based on reported scores or lower bounds. These results show that expressive local updates and perturbation-based retries can deliver strong performance within a BP-structured architecture.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Neural channel decoding, LDPC codes, Belief propagation, Neural ordinary differential equations

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_gh54OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper provides a continuous time ODE perspective into neural Belief Propagation framework, using off-the-shelf ODE solvers which enhanced their iterative decoder performance. Superior performance is gained from their design, especially from the retrying mechanism, though the raw performance is already leading.

Critical strengths
  1. This method attains substantially better BER compared to other compared methods, across multiple LDPC family codes. The introduction of perturbation further improves their performance.
  2. Their introduction of the ODE formulation for Belief Propagation into neural BP decoder is rather new in this field, and gives rise to the introduction of ODE solvers.
Critical weaknesses
  1. The comparison could be more rigorous, with more details described in the 4-th question for the authors. SBECC is a general neural decoder, while the neural BP decoder baseline only includes ARBP while other neural BP decoders were proposed and could serve as better baselines for neural decoder. Moreover, perturbation’s contribution for improved results is substantial and insufficiently separated. As explained in the question, evaluation of the other methods without perturbation gives an additional advantage for BeliefFlow, and Table 2 only partially attributes to this.
  2. The necessity of bringing in ODE formulation is not compelling, see question 3.
Questions for authors
  1. Although Neural BP is light in computation and parameters, the authors did not present a qualitative comparison of either latency or computational cost in actual end-to-end decoding setups. These results might clarify the limitation claims at line 475.

  2. The effect of different NFE steps with different ODE solvers is not clear; one might hypothesize that RK4 superseded Euler because it requires more function evaluations for accuracy. The authors are encouraged to compare the two under the same NFE budget as well, or provide some insights into this.

  3. The necessity and practical benefit of the continuous-time formulation remain unclear. Equation (10) shows that Euler discretization yields a damped, weight-shared recurrent message-passing network, which can be formulated directly in discrete time with a damping coefficient.

  4. As the bank of positional embeddings is essentially frozen/learnable additive Gaussian perturbation, it is then rather to ask whether they are positional embeddings or serve as perturbation, and to propose a Gaussian noise at say 0.4σchannel0.4\sigma_{\text{channel}} level, with the same candidate and output merging setups for other compared methods. In my setups, the authors' reported -ln BER might be superseded by a 50-iteration BP decoder.

  5. The presentation of the authors' model setup is a bit unclear; readers shall resort to Figure 1, the appendix, and their source code for understanding the model's computation path. It is encouraged to present a standalone and complete description in the appendix.

  6. Is the training/testing set division a consequence of the invalidity of zero codeword symmetry for BeliefFlow? Or is the symmetry still retained?

  7. Can you make a comparison with perturbed BP decoding?

Some references for perturbation decoding:

[1] F. Leduc-Primeau, S. Hemati, S. Mannor, and W. J. Gross, "Dithered Belief Propagation Decoding," IEEE Transactions on Communications, 60(8), 2042–2047, 2012.

[2] J. Gao, H. Wang, and K. Tian, "Noise-Assisted List Decoding for 5G LDPC Codes," IEEE VTC-Fall, 2022.

10 comments

We would like to thank the reviewer for recognizing the decoding improvements and the novelty of our work. Your constructive comments have helped us improve the manuscript. We provide point-by-point responses to each of your concerns below.

If our responses resolve the reviewer’s concerns, we hope the reviewer could raise the score and support the paper in discussion. Given the paper’s current borderline standing, your support in the Reviewer-AC Discussion could be decisive for the final outcome.


Changelog of the manuscript

Before addressing the individual comments, we summarize the main changes from the original submission. In particular, the appendix now contains 12 subsections, listed below. These provide additional experimental results and implementation details to address the questions raised during review. The revised PDF has been uploaded and is available for review.

Contents of the Appendix

  • A.1 Detailed decoding performance of BeliefFlow (included more metrics like BER, BLER, block retry rate, etc)
  • A.2 Experiments on training with different solvers and step sizes
  • A.3 Coupling between decoder training and retry perturbations (explored what the retry perturbation should be like)
  • A.4 Results on different retry voting strategies (bitwise voting vs. framewise voting)
  • A.5 Effect of the kinetic-energy regularization term (which has a modest effect and is retained following common neural ODE practice)
  • A.6 Ablation study of variable-node attention and check-node gating (variable-node attention is much more important than check-node gating)
  • A.7 Comparison with perturbation-based BP decoders (compared with NMS and perturbed-NMS)
  • A.8 Comparison with OSD-based decoders (compared with OSD, BP-OSD under a comparable nominal candidate budget)
  • A.9 Preliminary results on longer codes (updated results after more training epochs, BF leads BP by a large margin on longer codes)
  • A.10 Confidence bounds when no bit errors are observed
  • A.11 Implementation details of the neural update (included missing default hyperparameters)
  • A.12 Model size and inference cost (included time, memory, operations/cw cost, comparing with SBECC and OSD decoders)

Q1 Actual decoding latency and computational cost

Reviewer comment

Although Neural BP is light in computation and parameters, the authors did not present a qualitative comparison of either latency or computational cost in actual end-to-end decoding setups. These results might clarify the limitation claims at line 475.

We agree that parameter count alone does not characterize decoding cost. In the revised manuscript, we have added Appendix A.12, which reports decoding time, peak memory usage, operation counts, and BER for the evaluated implementations, with retry and without retry decoding. For example, Table 21 presents the following results at 4 dB:

CodeMethodBERTime, ms/codewordGOps/codeword
LDPC(49,24)SB-ECC4.35e-417.60.526
LDPC(49,24)BeliefFlow, RK4, step size 0.1, retry3.00e-424974.0
LDPC(121,80)SB-ECC1.45e-420.21.04
LDPC(121,80)BeliefFlow, RK4, step size 0.1, retry4.42e-5289139

For retry decoding, operation counts are summed across trajectories. Aggregate inference time is the measured batch runtime multiplied by the number of trajectories in the batch. BeliefFlow’s GPU memory usage is reported for a single trajectory. BeliefFlow also offers accuracy-cost trade-offs through the choice of solver and step size.

We also evaluated CPU implementations of OSD and BP-OSD, as suggested by another reviewer. As expected, both methods are substantially faster than BeliefFlow on shorter codes. For longer codes, the growing search space makes high-order standalone OSD increasingly expensive.

Table 10 in Appendix A.1 also reports how often the initial attempt triggers retries. At 4 dB, the block retry rate ranges from 0.123% to 3.53% across the six codes. Although these rates do not directly quantify the additional inference cost, they indicate how frequently retry decoding is required.


Q2 Function evaluations and solver comparisons

Reviewer comment

The authors are encouraged to compare the two under the same NFE budget as well, or provide some insights into this.

The NFE values in Table 5 were obtained by evaluating the same RK4-trained weights for each code with different inference solvers. The mean of the per-setting RK4/Euler NFE ratios is approximately 3.19<4. Thus, RK4 takes fewer integration steps on average but incurs more function evaluations. Euler nevertheless achieves comparable decoding performance, with only modest differences from RK4. Appendix A.2 further compares models trained and evaluated with Euler or RK4 and supports the same observation. These results suggest that higher-order solvers are not essential for obtaining strong decoding performance in the tested settings.

For evaluation under the same NFE budget, we would like to request an exemption. In our experiments, the realized NFE is not a preset hyperparameter and is difficult to match across solvers. Each decoding attempt terminates when the parity-and-confidence criteria are met or the maximum integration horizon is reached. The reported NFE is averaged over the decoded samples.

One way to adjust the mean NFE is to vary the integration horizon. However, this adjustment only affects difficult samples that reach the horizon without satisfying the stopping criteria. With this approach, we cannot precisely control the mean NFE and can only seek an approximate match through a grid search over integration horizons.

We would also expect that extending Euler decoding to match RK4’s NFE would yield results similar to the current Euler configuration. Conversely, truncating RK4 when the NFE budget is exhausted would be expected to reduce its decoding performance.

Q3 Purpose of the continuous-time formulation

Reviewer comment

The necessity and practical benefit of the continuous-time formulation remain unclear. Equation (10) shows that Euler discretization yields a damped, weight-shared recurrent message-passing network, which can be formulated directly in discrete time with a damping coefficient.

Related critical weakness

The necessity of bringing in ODE formulation is not compelling, see question 3.

We agree with the equivalence identified by the reviewer. Applying forward Euler to the learned dynamics gives

+\Delta t\,f_\theta(M^{(\ell)};\widetilde Y),$$ which is a damped, shared-weight recurrent message-passing update that can also be defined directly in discrete time. Our results do not establish that continuous time is necessary for the reported decoding performance. The results in Table 4 and Appendix A.2 further suggest that higher-order solvers are not essential for the proposed method. Nevertheless, we would still like to emphasize the practical importance of adopting the neural ODE framework. A direct discrete implementation would require us to explicitly manage the iterative updates and unrolling during training and inference. Reproducing the current adjoint-based training procedure could also involve duplicating functionality already provided by existing neural ODE libraries. Furthermore, the duration-aware loss would need to be reformulated in terms of discrete iterations, since its current definition directly depends on integration duration. The neural ODE framework also allows us to explore accuracy-cost trade-offs by choosing different solvers and step sizes, as illustrated in Tables 4–7 and Appendix A.12. ---- ## Q4 Random features, perturbations, and attribution of the gains **Reviewer comment** > As the bank of positional embeddings is essentially frozen/learnable additive Gaussian perturbation, it is then rather to ask whether they are positional embeddings or serve as perturbation, and to propose a Gaussian noise at say level, with the same candidate and output merging setups for other compared methods. In my setups, the authors' reported -ln BER might be superseded by a 50-iteration BP decoder. We agree that the term **feature perturbations** better describes their role. The current manuscript uses this terminology. The feature vectors are reassigned to nodes for each received sample and decoding attempt, so they do not encode persistent node identities. The positional-encoding-style perturbations were originally designed to support cross-code transfer without assigning persistent identities to nodes. These features result in generating diverse decoding trajectories when their node assignments are resampled. We are also interested in whether these perturbations could be replaced by directly sampled noise and whether models trained without perturbations could benefit from perturbation-based retries. The corresponding experiments and results are presented in Appendix A.3. The results show that models trained without feature perturbations perform substantially worse than the default perturbation-trained models when given the same frozen-bank retry perturbations. This highlights the importance of training with feature perturbations before applying perturbation-based retries. Replacing frozen-bank retry perturbations with moment-matched uniform samples produces much smaller changes in performance. This suggests that the decoder learns to accommodate feature perturbations without depending on the exact frozen-bank entries used for retries. We also emphasize that, even **without feature perturbations or retries, BeliefFlow outperforms the reported SOTA baseline** in most settings in Table 2. We also agree that comparisons against unperturbed baselines alone cannot establish an advantage over other perturbation-assisted decoders. Appendix A.7 therefore compares BeliefFlow with NMS and noise-assisted NMS, as discussed under Q7. NMS provides a computationally efficient approximation to BP, and its noise-assisted retry strategy is adapted from Gao et al. \[2]. Perturbation-based retries improve both methods, and BeliefFlow generally outperforms NMS across the reported settings, both with and without retries. > [2] J. Gao, H. Wang, and K. Tian, "Noise-Assisted List Decoding for 5G LDPC Codes," IEEE VTC-Fall, 2022. As to BP-50, the BP/NMS baseline already uses up to 200 iterations with parity-based early stopping, so a 50-iteration cap would only reduce its opportunity to converge.
Reviewer_gSeVOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes BeliefFlow, a neural BP-type decoder for LDPC codes. The main idea is to replace the standard check-to-variable node update with a vector-valued state that captures continuous-time dynamics. The authors show improvements over conventional BP and neural-BP baselines, mainly on six short LDPC codes. The approach is of some interest, but the contribution is very minor.

Critical strengths

Learning more expressive message update rules while exploiting the Tanner-graph is an interesting direction. The proposed decoder improves on the selected baselines.

Critical weaknesses

The main weakness is that the contribution is too narrow and limited and lacks depth to warrant a publication at ICLR. The results are promising, but the contribution is simply too minor.

The construction is largely heuristic, with neither theoretical analysis nor insight into why its distinctive components are necessary. Indeed, the proposed architecture builds on established ideas in neural BP, continuous-time decoding, and randomized perturbations.

For a paper that evaluates the performance of LDPC decoders, the results should be reported in terms of FER curves (alternatively BER curves, but FER are preferred) as a function of Eb/N0, as it is customary in any coding paper. The current way to present the results wouldn't be acceptable in more traditional venues for coding papers, and ML conferences should not be an exception. One needs to quantify the gains (in Eb/N0) and assess the behavior in the waterfall and error floor regions.

The main experiments concern lengths 49-128. But for these block lengths, most of the codes reported are not competitive. Polar codes (with CRC and list decoding), BCH codes with OSD, or other LDPC codes (potentially with OSD), would be better alternatives. As an example, only in the early times of LDPC codes one would report results on the MacKay code. MacKay codes are not competitive and hence they should not be considered when evaluating performance.

The comparison is insufficient given the computational cost of the proposed decoder. Comparisons with LDPC+OSD and other strong alternatives such as BCH+OSD and polar codes with CRC and list decoding, and a complexity discussion, are needed.

Questions for authors

Can the authors provide FER and BER curves?

What is the complexity (computation, memory, and latency per decoded word, including retries) compared with state-of-the-art baselines such as polar codes with list decoding, BCH codes+OSD, and LDPC+OSD?

6 comments

We thank the reviewer for reviewing our manuscript and provide point-by-point responses to your comments below.

We are confident that our responses and additional results address the reviewer's questions, and we sincerely hope that you will raise the final score for our paper.


Changelog of the manuscript

Before addressing the individual comments, we summarize the main changes from the original submission. In particular, the appendix now contains 12 subsections, listed below. These provide additional experimental results and implementation details to address the questions raised during review. The revised PDF has been uploaded and is available for review.

Contents of the Appendix

  • A.1 Detailed decoding performance of BeliefFlow (included more metrics like BER, BLER, block retry rate, etc)
  • A.2 Experiments on training with different solvers and step sizes
  • A.3 Coupling between decoder training and retry perturbations (explored what the retry perturbation should be like)
  • A.4 Results on different retry voting strategies (bitwise voting vs. framewise voting)
  • A.5 Effect of the kinetic-energy regularization term (which has a modest effect and is retained following common neural ODE practice)
  • A.6 Ablation study of variable-node attention and check-node gating (variable-node attention is much more important than check-node gating)
  • A.7 Comparison with perturbation-based BP decoders (compared with NMS and perturbed-NMS)
  • A.8 Comparison with OSD-based decoders (compared with OSD, BP-OSD under a comparable nominal candidate budget)
  • A.9 Preliminary results on longer codes (updated results after more training epochs, BF leads BP by a large margin on longer codes)
  • A.10 Confidence bounds when no bit errors are observed
  • A.11 Implementation details of the neural update (included missing default hyperparameters)
  • A.12 Model size and inference cost (included time, memory, operations/cw cost, comparing with SBECC and OSD decoders)

W1 Research question and contribution

Reviewer comment

The main weakness is that the contribution is too narrow and limited and lacks depth to warrant a publication at ICLR. The results are promising, but the contribution is simply too minor. Indeed, the proposed architecture builds on established ideas in neural BP, continuous-time decoding, and randomized perturbations.

We thank the reviewer for recognizing the promising decoding results. We disagree with the assessment that the contribution is simply too minor. The contribution should be assessed through its specific architecture, learned dynamics, and training–inference mechanism.

First, BeliefFlow introduces a specific attention-based Tanner-edge update. It combines attention over destination-excluded neighborhoods with gated, elementwise box-plus interactions and vector-valued edge states. This construction provides expressive learned updates while retaining BP’s computational structure. Reviewer ou8J also highlighted its “Useful graph structure”. The existence of earlier neural BP methods does not, by itself, make this particular architecture an established method.

Second, BeliefFlow learns continuous-time dynamics directly in Tanner-edge message space. Earlier work, such as Mostafa et al. (2016), examines a continuous-time method from the perspective of analog hardware implementation. BeliefFlow learns a shared neural vector field over vector-valued Tanner-edge states, optimized using decoding data and trajectory-based training objectives. Its contribution therefore concerns learned message dynamics and their decoding performance, beyond expressing conventional BP in continuous time.

[1] Mohamad Mostafa, Giuseppe Oliveri, Werner G. Teich, and Jürgen Lindner. “A Continuous-Time Recurrent Neural Network for Joint Equalization and Decoding-Analog Hardware Implementation Aspects.” In Artificial Neural Networks-Models and Applications, 2016. DOI: 10.5772/63387.

Third, the coupling between perturbations, training, and retry decoding is consequential. We acknowledge and cite earlier perturbation-assisted conventional BP methods. Appendix A.3 investigates how perturbations interact with the learned decoder. Under the same frozen-bank retry perturbations, models trained without feature perturbations underperform the default perturbation-trained models in all 12 evaluated settings. These results support the importance of training the decoder to accommodate perturbations and show that the retry mechanism is not simply an interchangeable inference-time addition.


W2 Design rationale and insight into component contributions

Reviewer comment

The construction is largely heuristic, with neither theoretical analysis nor insight into why its distinctive components are necessary.

We thank the reviewer for highlighting the need for a more comprehensive analysis of the individual components. Following the reviewers’ suggestions, we have expanded the appendix to 12 subsections (A.1–A.12), including additional component ablations.

These studies provide direct empirical evidence about the component contributions, identifying both consistent improvements and setting-dependent effects:

ComponentEvidence and interpretationLocation
Variable-node attentionRemoving it lowers scores in all 12 tested combinations of code, SNR, and retry settingA.6, Tables 16 and 17
Check-node gatingRemoving it gives comparable performance, with improvements on MacKay(96,48); its contribution is setting-dependentA.6, Tables 16 and 17
Kinetic-energy regularizationRemoving it has little effect at 4 dB and modest, setting-dependent effects at higher SNRsA.5, Table 15
Perturbations and retriesA bank alone does not consistently improve single-attempt decoding; its main observed benefit comes from its combination with retriesTable 2; A.1, Table 10
Training with perturbationsUnder the same frozen-bank retry perturbations, models trained without perturbations underperform the default in all 12 settingsA.3, Table 13
Duration-aware reweightingImproves the displayed scores or bounds for five of six codes; CCSDS provides an exceptionTable 3
Higher-order integrationEuler-trained and Euler-evaluated models achieve scores close to those of the corresponding RK4 models on the tested codesA.2, Table 11

W3 FER/BER curves, coding gains, and the evaluated error-rate range

Reviewer comment

For a paper that evaluates the performance of LDPC decoders, the results should be reported in terms of FER curves (alternatively BER curves, but FER are preferred) as a function of Eb/N0, as it is customary in any coding paper. The current way to present the results wouldn't be acceptable in more traditional venues for coding papers, and ML conferences should not be an exception. One needs to quantify the gains (in Eb/N0) and assess the behavior in the waterfall and error floor regions.

We agree that error-rate curves against Eb/N0E_b/N_0 are more informative than a table of negative log BER scores alone. The revised Appendix A.1 now provides BLER curves, corresponding to FER curves for these codeword-level evaluations. We also report detailed decoding performance metrics derived from our experimental logs, including BER, BLER, block retry rates (BLRR), and 95% confidence intervals for BER and BLER. Specifically:

  • Figure 3 compares BLER curves over 2–6 dB on LDPC(49,24) and LDPC(121,80), including BP-200, SR-MS-LLR, DDECC, SB-ECC, and BeliefFlow with and without retries.
  • Figure 2 reports BER and BLER with 95% confidence intervals for all six main codes at 4, 5, and 6 dB, with and without retries.
  • Table 10 gives the underlying reported BER, BLER, and BLRR for all six main codes at 4, 5, and 6 dB.

Using log-linear interpolation of the BLER curves, BeliefFlow achieves approximate gains over BP-200 of 0.66 and 0.59 dB on LDPC(49,24), and 0.42 and 0.42 dB on LDPC(121,80), at BLER targets of 1e-3 and 1e-4, respectively. The corresponding gains over SB-ECC are approximately 0.09/0.08 dB and 0.25/0.21 dB. The SB-ECC gains use the digitized published curve shown in Figure 3. Over the evaluated 2–6 dB range, the BLER curves show a steep waterfall decline without visible flattening. On LDPC(121,80), no frame errors were observed at 6 dB over 1e8 codewords, as indicated by the dashed arrow in Figure 3.


W4 Benchmark-code selection and longer-code evaluation

Reviewer comment

The main experiments concern lengths 49-128. But for these block lengths, most of the codes reported are not competitive. Polar codes (with CRC and list decoding), BCH codes with OSD, or other LDPC codes (potentially with OSD), would be better alternatives. As an example, only in the early times of LDPC codes one would report results on the MacKay code. MacKay codes are not competitive and hence they should not be considered when evaluating performance.

We distinguish two research questions: how to improve a decoder for a given code, and how to choose the best code-and-decoder combination for a communication system. BeliefFlow studies the first question.

The selected short LDPC codes permit comparison with published neural decoding results on the same benchmark tasks and other works on the neural decoder research line, including:

[2] Choukroun, Yoni, and Lior Wolf. "Error correction code transformer." NeurIPS 2022

[3] Choukroun, Yoni, and Lior Wolf. "Denoising diffusion error correction codes." ICLR 2023

[4] Kwak, Hee-Youl, et al. "Boosting learning for LDPC codes to improve the error-floor performance." NeurIPS 2023.

[5] Park, Seong-Joon, et al. "CrossMPT: Cross-attention message-passing transformer for error correcting codes." ICLR 2025

[6] Helvits, Alon, and Eliya Nachmani. "Score Based Error Correcting Code Decoder." ICML 2026

We disagree that a code should be excluded from decoder evaluation solely because another code family may provide a better deployment choice. For a fixed code and channel, comparing decoding algorithms remains informative about their learned representations and inference procedures. Applying the proposed exclusion criterion would also undermine comparisons among existing neural decoders evaluated on those same codes. It would not isolate the contribution of the learned decoding method. In particular, MacKay(96,48), which “should not be considered”, is presented in all the comparison results in [2-6] listed above. Further, MacKay(96,48) is one of six main benchmark codes, rather than the sole basis of the conclusions.

Moreover, the revised Appendix A.9 extends the evaluation to longer codes: WRAN(384,320), LDPC(648,540), and LDPC(1056,880):

CodeBP-200, 4 dBBeliefFlow, 4 dBBP-200, 5 dBBeliefFlow, 5 dB
WRAN(384,320)7.768.6112.0612.84
LDPC(648,540)8.8510.7016.5418.54
LDPC(1056,880)11.4213.0217.3318.57

BeliefFlow with retries improves on BP-200 in all six settings, by 0.78–2.00 in -ln(BER), corresponding to approximately 2.18–7.39 times lower BER.

Reviewer_ou8JOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces BeliefFlow, the approach maintains the structure of continuous-time belief propagation (BP) and Tanner message-passing, but replaces the scalar messages with vector-valued Tanner-edge states that evolve according to a learned ODE, dM/dt=fθ(M;Y~)−MdM/dt = f_\theta(M; \tilde{Y}) - M. Each update aggregates information from destination-excluded neighborhoods using attention mechanisms and applies a gated, element-wise box-plus operator. The training process incorporates duration-aware loss reweighting and a kinetic-energy penalty, while inference employs randomized feature-bank perturbations and retry decoding, allowing for up to 256 parallel candidates with majority voting over the accepted ones. The model consists of 0.34M parameters and achieves leading performance in 17 out of 18 code/SNR settings across six LDPC codes, as measured by −ln⁡-\lnBER scores or lower bounds. The paper also includes ablation studies on banks, retries, loss reweighting, ODE solvers, integration horizons, step sizes, and zero-shot transfer.

Critical strengths
  • Useful graph structure. The learned operations are reused across nodes and edges, while the parity-check matrix determines their connections, and this makes cross-code evaluation possible without retraining. The approach can also scale more easily, as there are only few parameters to learn.
  • Substantial empirical evaluation, carefully interpreted. The paper compares several baselines and examines banks, retries, loss reweighting, ODE solvers, integration horizons, and zero-shot transfer. The single-attempt ablation does not consistently favor banks, and the authors appropriately present their main observed benefit as retry diversity. Results are good compared to the SotA on several codes (17 of 18 settings by scores or lower bounds).
Critical weaknesses

Inference cost is unclear

A timed-out word can trigger 256 parallel retries, each involving repeated network evaluations. Parameter count and the number of evaluations per attempt do not show the total latency, compute, or memory cost per decoded word relative to the baselines.

The source of the gains is not fully isolated

There is no direct ablation of attention, gates, box-plus, or the kinetic-energy penalty, and the experiments do not establish that continuous-time integration itself is essential: what happens if we keep everything except the continuous-time approach? Euler also performs similarly to RK4, while RK4 involves a higher computing cost.

Questions for authors
  • What is the total latency, compute and memory per decoded word, with and without retries, relative to the baselines?
  • Which components are necessary (the attention, the gates, the box-plus operator, the kinetic-energy penalty), and what happens if the continuous-time dynamics is replaced by a discrete update while everything else is kept?
  • Small differences are reported in the results, could uncertainty estimates also be reported?
  • In table 2, instead of SB-ECC, won't it be best to change it with the second best results from table 1? For example on CCSDS, SR-MS-LLR is better than SB-ECC.
  • Could the perturbation/retry mechanism (and the duration-aware loss reweighting) be applied to other architectures?
  • Is the final word after majority voting guaranteed to satisfy the parity checks, and if not, how is the reported BER computed?
  • Have you considered reporting BLER alongside BER? In a coded system what ultimately matters is whether a block is decoded correctly; otherwise it is discarded and retransmission is requested, so a block-level error rate would be more informative than a bit-level one. (−ln⁡-\lnBER is also not very readable on first sight.)
  • "BP-structured" should be read narrowly: aggregated neighbors exclude the destination incidence, but the destination message enters the learned query as exposed in Figure 1 for the query. Is the update extrinsic BP?
  • Presentation of the architecture could clearly be improved: Figure 1 and the appendix help a bit, but there are a lot of different things to take into account (NN architectures, continuous-time BP, perturbation and retries, loss), and taking the time to present everything should be done. Could the dedicated appendix provide details and justification for the design choices, and could the conclusion be expanded?
12 comments

We thank the reviewer for recognizing the value of the shared graph structure, the cross-code evaluation, and the empirical decoding results. Your comments have helped us further improve the manuscript. We provide point-by-point responses to each of your concerns below.

We are confident that our responses address the concerns, and we sincerely hope that you could raise the score and support the paper in discussion. Given the paper’s current borderline standing, your support in the Reviewer-AC Discussion could be decisive for the final outcome.


Changelog of the manuscript

Before addressing the individual comments, we summarize the main changes from the original submission. In particular, the appendix now contains 12 subsections, listed below. These provide additional experimental results and implementation details to address the questions raised during review. The revised PDF has been uploaded and is available for review.

Contents of the Appendix

  • A.1 Detailed decoding performance of BeliefFlow (included more metrics like BER, BLER, block retry rate, etc)
  • A.2 Experiments on training with different solvers and step sizes
  • A.3 Coupling between decoder training and retry perturbations (explored what the retry perturbation should be like)
  • A.4 Results on different retry voting strategies (bitwise voting vs. framewise voting)
  • A.5 Effect of the kinetic-energy regularization term (which has a modest effect and is retained following common neural ODE practice)
  • A.6 Ablation study of variable-node attention and check-node gating (variable-node attention is much more important than check-node gating)
  • A.7 Comparison with perturbation-based BP decoders (compared with NMS and perturbed-NMS)
  • A.8 Comparison with OSD-based decoders (compared with OSD, BP-OSD under a comparable nominal candidate budget)
  • A.9 Preliminary results on longer codes (updated results after more training epochs, BF leads BP by a large margin on longer codes)
  • A.10 Confidence bounds when no bit errors are observed
  • A.11 Implementation details of the neural update (included missing default hyperparameters)
  • A.12 Model size and inference cost (included time, memory, operations/cw cost, comparing with SBECC and OSD decoders)

Q1 Total latency, computation, and memory

Reviewer comment

What is the total latency, compute and memory per decoded word, with and without retries, relative to the baselines?

Related critical weakness

Inference cost is unclear

A timed-out word can trigger 256 parallel retries, each involving repeated network evaluations. Parameter count and the number of evaluations per attempt do not show the total latency, compute, or memory cost per decoded word relative to the baselines.

We agree that parameter count and per-attempt NFE alone do not characterize the cost of the complete decoder. Appendix A.12, Tables 21 and 22, now compares BeliefFlow with and without retries against SB-ECC, OSD, and BP-OSD, reporting BER, aggregate inference time, operation counts per codeword, and memory usage. Selected results from Table 21 at 4 dB are shown below. Both BeliefFlow configurations use RK4 with a step size of 0.1.

CodeMethodBERAggregate time, ms/codewordGPU memory, MBGOps/codeword
LDPC(49,24)SB-ECC4.35e-417.614.30.526
LDPC(49,24)BeliefFlow, RK4, step size 0.1, no retry6.19e-456.513.73.80
LDPC(49,24)BeliefFlow, RK4, step size 0.1, retry3.00e-424913.774.0
LDPC(121,80)SB-ECC1.45e-420.215.41.04
LDPC(121,80)BeliefFlow, RK4, step size 0.1, no retry1.45e-485.125.212.1
LDPC(121,80)BeliefFlow, RK4, step size 0.1, retry4.42e-528925.2139

For retry decoding, operation counts are summed across trajectories. Aggregate inference time is the measured batch runtime multiplied by the number of trajectories in the batch. BeliefFlow's GPU memory usage is reported for a single trajectory. The neural decoders run on an NVIDIA RTX 3090 GPU.

We also evaluated OSD and BP-OSD on an Intel Xeon Silver 4215R CPU. On short codes, these implementations are substantially faster and use fewer resources than the neural decoders. Table 22 extends the comparison, including both BeliefFlow retry settings, to three longer codes. On LDPC(1056,880), BeliefFlow with retries achieves BER 2.22e-6 at 160 ms/codeword, compared with BER 5.34e-4 at 3440 ms/codeword for standalone OSD-3. BP-OSD remains faster on all three longer codes. It applies BP first and invokes OSD only after a detected BP failure, reducing its average decoding cost. The same screening strategy could also be applied to BeliefFlow.

Table 10 additionally reports how frequently retries are triggered: at 4 dB, the block retry rate ranges from 0.123% to 3.53% across the six main codes.

Q2 Component contributions and the discrete-time alternative

Reviewer comment

Which components are necessary (the attention, the gates, the box-plus operator, the kinetic-energy penalty), and what happens if the continuous-time dynamics is replaced by a discrete update while everything else is kept?

Related critical weakness

The source of the gains is not fully isolated

There is no direct ablation of attention, gates, box-plus, or the kinetic-energy penalty, and the experiments do not establish that continuous-time integration itself is essential: what happens if we keep everything except the continuous-time approach? Euler also performs similarly to RK4, while RK4 involves a higher computing cost.

We agree that ablation studies are important for understanding the contributions of these components. We began these experiments after the initial submission and now report the results in Appendices A.5 and A.6 of the revised manuscript.

Variable-node Attention and check-node gating. Appendix A.6, Tables 16 and 17, disables variable-node attention, check-node gating, or both, with and without retry decoding. Kinetic-energy regularization. Appendix A.5, Table 15, compares kinetic-energy weights of 0 and 1 under both retry settings. The main findings from these ablation experiments are summarized below:

Component or ablationObserved resultLocation
Remove variable-node attentionLower scores in all 12 tested combinations of code, SNR, and retry settingTables 16 and 17
Remove check-node gatingMixed effects on LDPC(49,24); modestly improved scores on MacKay(96,48) at all three SNRs, with and without retriesTables 16 and 17
Remove both attention and gatingLower scores than the full configuration throughout the tested settingsTables 16 and 17
Remove kinetic-energy regularizationSimilar performance at 4 dB; modest, setting-dependent differences at 5 and 6 dBTable 15

Box-plus. We retain the box-plus operator as a coding-theoretic inductive bias to preserve parity-aware interactions within BP check-node neighborhoods. We are sorry that we could not evaluate alternatives to box-plus within the short rebuttal window. However, our earlier architecture search offers a qualitative insight into the model’s sensitivity to structural choices related to BP symmetry. Specifically, models using destination-included box-plus were difficult to train to convergence. This observation suggests that retaining the destination-exclusion rule inherited from BP plays an essential role.

Continuous versus discrete time. We agree that forward Euler gives a shared-weight discrete recurrence:

+\Delta t\,f_\theta(M^{(\ell)};\widetilde Y).$$ The same recurrence can be defined directly in discrete time, yielding the same forward computation as an Euler discretization of the ODE. At a step size of 1, its forward update becomes $M^{(\ell+1)}=f_\theta(M^{(\ell)};\widetilde Y)$. This remains a learned vector-valued update, rather than conventional scalar BP. Moreover, our results do not establish that continuous time itself is necessary for the reported decoding performance. Table 4 and Appendix A.2 show that Euler achieves performance comparable to RK4, suggesting that higher-order solvers are not essential for the proposed method. Nevertheless, we would still like to emphasize the practical importance of adopting the neural ODE framework. A direct discrete implementation would require us to explicitly manage the iterative updates and unrolling during training and inference. Reproducing the current adjoint-based training procedure could also involve duplicating functionality already provided by existing neural ODE libraries. Furthermore, the duration-aware loss would need to be reformulated in terms of discrete iterations, since its current definition directly depends on integration duration. The neural ODE framework also allows us to explore accuracy-cost trade-offs by choosing different solvers and step sizes, as illustrated in Tables 4–7 and Appendix A.12.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.