acceptodds
Under review as a conference paper at ICLR 2027

ZeroCode: On-demand Error-Correcting Code Construction from the Zero Matrix via Reinforcement Learning

Abstract

Error-correcting codes (ECCs) are essential across diverse applications—from wireless communications and storage to quantum computing—yet each application imposes distinct design requirements on the parity-check matrix (PCM). To address these on-demand requirements in a unified framework, we propose \em ZeroCode, a reinforcement learning (RL)-based approach that constructs PCMs sequentially from the all-zero matrix. ZeroCode formulates construction as a discrete sequential decision-making problem and uses proximal policy optimization with action masking to select valid edges. ZeroCode achieves a gain of approximately 1 dB over the prior RL-based construction method at a bit error rate (BER) of for the (32,16) code and outperforms existing genetic, differentiable, and classical code-design methods in our experiments. Beyond optimizing decoding performance, the masking mechanism allows on-demand structural constraints—such as a maximum degree, 4-cycle-free structure, and quasi-cyclic structure—to be flexibly incorporated. Moreover, a single policy rollout yields a library of PCMs with varying edge counts, offering trade-offs between decoding performance and complexity without retraining. Overall, ZeroCode addresses diverse code-design requirements within a unified framework, providing solutions with optimized decoding performance under given constraints.

Primary area:
reinforcement learning
Keywords:
Error-Correcting Codes, LDPC Codes, Reinforcement Learning, Code Construction

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_T6N3OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper presents an RL method for constructing a binary parity check matrix for an error correcting code. The algorithm begins with an all-zero matrix, flips a zero to a one at random (or alternatively, adds an edge to the respective Tanner graph), and checks BER in the binary symmetric channel via belief propagation (I assume via repeated sampling of the noise, and running BP a certain amount of steps); that BER serves as the RL reward function, and the results are logged. Every certain number of steps, that log is used to train a function f which dictates the next edge to be flipped, and a function g which estimates the reward. The technique is also compatible with applying various constraints on the parity check matrix, such avoiding 4-cycles in the Tanner graph (known to negatively affect decoding success probability).

Critical strengths
  1. Clear and concise presentation of the solution.
  2. State-of-the-art performance in several parameter regimes.
  3. The idea of using the buffer D to train the RL module is nice.
Critical weaknesses
  1. Limited novelty – The paper appears to be a straightforward application of known RL methods to the problem of constructing parity-check matrices compatible with belief propagation decoding.

  2. Limited scope – a. The authors demonstrate improved BER over existing method in a very narrow set of parameters (n,n/2), with four possible values of n. b. The reward system is based exclusively on BP decoding. c. Insufficient comparison to other code techniques such as polar coding; some preliminary results are given in Appendix D, but I had trouble following what’s being done, or what is the equally-well-performing AI-coding method.

  3. The paper appears to present some good methodologies in terms of constructing error correcting codes for certain parameters while optimizing success of certain decoding algorithms, but lacks a generalizable method or theoretical insights that might be applied elsewhere.

  4. As such, perhaps this paper might be of little interest to the ICLR community, and might be better suited to an IEEE journal in this area.

Questions for authors
  1. The paper presents results for a very limited set of parameters. Is there a reason why this set of parameters is important, insightful, or crucial for certain application? Otherwise, is there a reason not to attempt this method for many other parameters?
  2. Are there any generalizable insights which could be drawn from the paper, which might be applicable elsewhere?
  3. Are there any form of rigorous guarantees or theoretical statements that could be established about, say, the success probability of your approach, or of the code it produces?
  4. Is there a reason that the algorithm only adds edges to the tanner graph, rather than removing them?

Some minor comments about presentation:

  1. Slight confusion: an “edge” is an element of the Tanner graph which correspond to a given parity check matrix, not an element of the matrix itself. As such, I was confused multiple times about what it means to, say, add an edge to a matrix.

  2. “valid when it has full rank over GF(2) and contains neither an all-zero row” – The word “and” here is imprecise since a full rank matrix cannot have an all-zero row.

  3. How is BER computed in (3)? Even if this is standard averaging, please say so.

10 comments
AuthorsOpenReview ↗

Novelty and Key Contributions

Thank you very much for your thoughtful questions regarding the significance and extensibility of our work. Based on your comments, we clarify how our contributions apply to other construction problems as well as the current scope of our evaluation.


Reviewer comment

Limited novelty – The paper appears to be a straightforward application of known RL methods to the problem of constructing parity-check matrices compatible with belief propagation decoding.

Response

A straightforward application of RL does not work well for this problem. The prior RL-based construction (DDPG) underperformed classical constructions, and simply replacing DDPG with PPO even degraded performance (Table 1). ZeroCode overcomes this through a new problem formulation and domain-informed action masking, and we summarize our contributions from three perspectives.

1. Problem formulation: learning a sequential PCM construction policy guided by decoding performance

ZeroCode formulates PCM construction as a discrete sequential decision-making problem, adding one edge at a time from the all-zero matrix and using the decoding performance of each partial matrix as the reward. Inspired by AlphaChip (Mirhoseini et al., 2021), which places components sequentially onto an empty canvas, ZeroCode learns the entire construction trajectory toward high-performing codes without relying on a pre-constructed PCM.

Table 1 shows the importance of this formulation. Replacing DDPG with PPO alone increased the BER from 1.0×10−31.0\times10^{-3} to 2.1×10−32.1\times10^{-3}, whereas the largest gain came from shifting from modifying a pre-constructed PCM to constructing one from the all-zero matrix. This shows that how the construction problem is formulated is as critical as the choice of RL algorithm. Moreover, because intermediate PCMs are also optimized, this process yields a library of codes spanning different edge counts and decoding complexities.

2. Action masking: incorporating channel-coding domain knowledge into construction

ZeroCode uses action masking to embed channel-coding domain knowledge directly into the search, for example by preventing 4-cycles, which degrade BP performance, or by preserving a QC structure. This optimizes decoding performance while satisfying structural constraints by construction, without modifying the training procedure, and provides a foundation for integrating other established code-design principles into RL.

3. Performance: empirical gains beyond classical construction methods

LDPC code design has long relied on classical constructions grounded in coding theory. To the best of our knowledge, ZeroCode is the first RL-based PCM construction method to outperform classical constructions across multiple block lengths. This shows that RL, combined with an appropriate problem formulation and domain knowledge, is effective for practical code design. Furthermore, ZeroCode supports optimization under diverse structural constraints, which is difficult for classical methods, and is expected to be useful for designing high-performing codes in emerging domains such as quantum computing, data storage, and cryptography.

AuthorsOpenReview ↗

Experimental Scope

Reviewer comment

Limited scope – a. The authors demonstrate improved BER over existing method in a very narrow set of parameters (n,n/2), with four possible values of n. b. The reward system is based exclusively on BP decoding. c. Insufficient comparison to other code techniques such as polar coding; some preliminary results are given in Appendix D, but I had trouble following what's being done, or what is the equally-well-performing AI-coding method.

The paper presents results for a very limited set of parameters. Is there a reason why this set of parameters is important, insightful, or crucial for certain application? Otherwise, is there a reason not to attempt this method for many other parameters?

Response

Code rates and comparison scope

We chose rate 1/21/2 because it is the most representative setting for evaluating short-block code design, and prior studies have primarily been evaluated in this regime: Tian et al. (2025) reported results for (32,16)(32,16), and Dufrène et al. (2025) reported results for (32,16)(32,16), (64,32)(64,32), and (128,64)(128,64). Following these settings enables a direct comparison with existing methods.

However, nothing in ZeroCode is specific to this choice; the code rate and block length are simply input parameters and can be changed freely. In Section 5.6, we constructed a PCM under the structural constraints of the rate-1/51/5 (100,20)(100,20) 5G LDPC code and achieved a gain of more than 2 dB over the 5G LDPC code at BER=10−3\mathrm{BER} = 10^{-3}. In addition, we constructed PCMs with ZeroCode for the (64,16)(64,16) and (64,48)(64,48) codes, corresponding to code rates 1/41/4 and 3/43/4, and compared them with the Classical, DL, and GA baselines under identical decoding settings. The table below reports the BER at Eb/N0=5.0E_b/N_0 = 5.0 dB.

(n,k)(n,k)ClassicalDLGAZeroCode
(64,16)(64,16)1.1×10−31.1\times10^{-3}2.0×10−32.0\times10^{-3}1.7×10−41.7\times10^{-4}1.8×10−41.8\times10^{-4}
(64,48)(64,48)7.6×10−47.6\times10^{-4}4.0×10−44.0\times10^{-4}2.3×10−42.3\times10^{-4}1.7×10−41.7\times10^{-4}

Whereas the relative performance of the other methods varies substantially with the code rate, ZeroCode maintains a consistently low BER across both code rates.

BP-decoding-based reward

The reward in ZeroCode is not tied to a specific decoder; any error rate measured by a given decoder can be used. In the main text, we used BP as a representative example, and in the polar-code experiment in the Appendix, we used the FER measured by an SCL decoder as the reward. Furthermore, following your comment, to verify operation with a non-BP decoder, we constructed a new (32,16)(32,16) code over a binary symmetric channel using the FER measured by a hard-decision bit-flipping (BF) decoder as the reward. When evaluated with the same BF decoder, ZeroCode achieves a lower FER than both Reed–Muller (RM) and extended quadratic-residue (QR) codes across the entire evaluated range. We have added these results to Appendix E Figure 10c of the revised manuscript.

Eb/N0E_b/N_0RMExtended QRZeroCode
73.2×10−23.2\times10^{-2}1.6×10−21.6\times10^{-2}2.7×10−32.7\times10^{-3}
87.6×10−37.6\times10^{-3}3.6×10−33.6\times10^{-3}4.0×10−44.0\times10^{-4}
91.3×10−31.3\times10^{-3}5.5×10−45.5\times10^{-4}4.2×10−54.2\times10^{-5}

Polar-code experimental setup and comparison method

Our description of Appendix D in the submitted manuscript was insufficient. This preliminary experiment examines whether the sequential selection approach used for PCM construction can also be applied to polar-code construction. Constructing a (128,64)(128,64) polar code requires selecting 64 information-bit positions out of 128 input positions. ZeroCode starts with no positions selected and, at each step, adds one unselected position to the information-bit set. Construction terminates once 64 positions have been selected, and the remaining 64 positions become frozen-bit positions. Thus, in this experiment, the policy learns which positions to assign as information bits.

The AI-based method that achieved comparable performance to ZeroCode is AI Coding (Huang et al., 2020), which searches for the information-bit set by applying crossover and mutation to existing candidate sets via a genetic algorithm. In Figure 10b, we compare the FER curve of the ZeroCode polar code against that reported by Huang et al. (2020); ZeroCode matches the performance of AI Coding, and both outperform the conventional construction methods PW and DE/GA. We have clarified this experimental setup and comparison procedure in Appendix E of the revised manuscript.

AuthorsOpenReview ↗

Generalizable Insights

Reviewer comment

The paper appears to present some good methodologies in terms of constructing error correcting codes for certain parameters while optimizing success of certain decoding algorithms, but lacks a generalizable method or theoretical insights that might be applied elsewhere.

Are there any generalizable insights which could be drawn from the paper, which might be applicable elsewhere?

Response

While this work focuses on PCM design, a PCM is equivalent to the edge configuration of a Tanner graph, making ZeroCode fundamentally a graph-optimization method. This is analogous to AlphaChip, which places components one by one onto an empty canvas; both methods permit only feasible selections given the current state, evaluate the outcome at each step, and construct a solution incrementally. To our knowledge, ZeroCode is the first work to apply this paradigm to PCM design. Accordingly, the same framework can be applied to general graph-optimization problems that require deciding pairwise connections sequentially to optimize an objective while satisfying structural constraints such as degree bounds, cycle constraints, and symmetry. In particular, the ability to enforce diverse constraints via action masking without altering the training procedure offers broader insights for graph optimization, and the same principle can be extended to sequential discrete decision problems in communications such as pilot assignment or antenna selection. Thank you for this insightful question that encouraged us to view our work from a broader perspective.

Reviewer_8v4KOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper describes a method for constructing Error Correcting Codes (ECCs) and more specifically Linear Block Codes (LBCs). In this learning problem, the solution space is a (n,m)(n,m) matrix with elements in GF(2)GF(2): the Parity-Check (PC) matrix. The PC matrix describes how binary codes of size nn are built from binary messages of size mm.

The paper's proposal, namely ZeroCode, is a Reinforcement-Learning (RL) approach that models the construction of the PC matrix as a sequential decision-making process, similarly to the DDPG/GNN approach (K.Tian et al.) presented in the related works. At each step tt,  the algorithm chooses to flip a zero-valued element of the PC matrix (action ata_t) which leads to an updated PC matrix (new state st+1s_{t+1}), starting with the all-zeros PC matrix. The evaluation of matrices uses a standard ECC metric: the Bit Error Rate (BER) after decoding with a standard Belief-Propagation (BP) decoding algorithm.

The underlying RL modeling is quite standard: the policy function over the admissible action space and the value function are learned using Proximal Policy Optimization (PPO). One interesting element of the proposed approach, however, is the addition of domain specific expertise in the process using a masking strategy over the action set: actions that would violate expert rules ("good practices") are forbidden. Such expert rules includes the rejection of actions that would yield a PC matrix with short cycles (4-cycles) that are detrimental to BP-based decoders or those that would yield variable-nodes of the Tanner Graph exceeding a given degree threshold.

The action policy and state value are estimated using Multi-Layer Perceptron (MLP) neural networks from the PC matrix. The use of MLP is briefly motivated, a comparison with Graph-Neural Network (GNN) variants, as used in the most direct related work, is provided.

At a higher level, the training process is organized in episodes where the initial PC (state Ht=0H_{t=0}) is initialized to the all-zeros matrix 0m×n0_{m \times n}. At each time step tt, an action (bit-flip) is sampled from a masked action policy at∼πθ˜(.∣Ht)a_t \sim \~{\pi_\theta}(.|H_t) and the PC matrix is updated (Ht+1H_{t+1}). The new PC matrix is evaluated using a standard BP-decoder to evaluated the cumulative reward RtR_t. A roll-out buffer D\mathcal{D} is filled with the sampled actions and rewards. Once full, PPO is performed and the MLP parameters are updated.

The approach has been evaluated on small codes (length n≤256n\leq256) with a fixed code rate mn=0.5\frac{m}{n} = 0.5. The evaluation of the PC matrices is carried assuming a standard BP decoder and a standard AWGN channel (with given SNR or Eb/N0). The BP algorithm is an iterative decoding process and the number of iterations is set to 8 for n=32n=32 and 5 for all other nn values. The reward used in the learning process is deduced from a single channel configuration (EbN0=5dB\frac{E_b}{N_0}= 5\text{dB}). The performance comparison involves 6 contenders: a RL-based approach (DDPG) which is a direct competitor (RL-based, same modeling, different action state space), a Random Search (same iterative construction of a PC matrix with random additions of ones in the matrix), a Genetic Algorithm approach, A "Deep-Learning" approach, a LDPC baseline using the PEG algorithm and a baseline from the CCSDS codes.

Beyond BER performance, a complexity evaluation through the proxy of the computation time required by each learning algorithm.

Finally and interestingly, the usefulness of the masking strategy is exemplified with the application to 5G LDPC structured codes.

Overall, the approach is a variant to the K.Tian et al paper with 2 significant updates: the masking strategy and the direct evaluation of the policy and state value from the PC matrix through MLPs, instead of a GNN applied to a lattice graph representation of the state (MPMNN).

Critical strengths

In my opinion, the use of a masking strategy to account for best practices in linear block code design is clever and well implemented in the paper. The results seem promising.

Critical weaknesses

In my opinion, there are potential major flaws in the performance comparison and the SotA comparison lacks recent references.

Major flaws in the performance comparison:

This paper has a potential major evaluation flaw: it compares different search approaches based on their best results obtained with different search budgets.

Page 15 paragraph "Baseline implementation" states that DDPG results are taken directly from the publicly available implementation. The number of evaluated matrices is not disclosed and the paper omits this discussion. Looking at the DDPG implementation (https://github.com/KouTian1999/GNN-DRL-Auto-encoder/blob/main/ae_i1_ldpc32.ipynb), DDPG has been trained with 1000 episodes and 25 steps per episodes. This amounts to 25 000 different PC matrices (at most).

In contrast, this paper proposal (ZeroCode) is trained with K=10K=10 epochs and  T=3×106T=3\times 10^6 steps, amounting to 1200 times more potential PC matrices evaluated and it's unclear if the results presented are from the matrices evaluated during the training or from the exploitation of the learned policy, i.e. after the training has been completed.

Also the Training Cost discussion is rather thin. Reckoning that the approach creates PC matrix of increasing complexity (+1 edge at each step), the authors claim that their training process generate a pool of N(=41 in the example) PC matrices in a single run while competing approaches need to re-run N(=41) times to reach the same number of matrices. This assumes that competing approaches cannot be adapted to do the same, e.g. by iterative elimination of edges in the Tanner Graph.

Additionally, the comparison with the Random Search is not conclusive. The paper does not explicit how the Random Search is performed: what is the admissible action space? Is it equivalent to a fixed policy where each action is equally likely? Do the authors implement the same masking strategy? How many steps, episodes have been carried? In the current step, the readers are not presented evidences of that claim.

SotA is missing references

A review of the SotA for DL based methods shows that the paper is missing recent papers. In particular, a paper from 2025: L. -A. Dufrène, Q. Lampin and G. Larue, "Learning Linear Block Codes With Gradient Quantization," in IEEE Transactions on Communications, doi: 10.1109/TCOMM.2025.3615681. cites the same DL baseline chosen by the author Y. Choukroun and L. Wolf. Factor graph optimization of error-correcting codes for belief propagation decoding and reports favorable results (0.5dB gain for target BLER=10−4BLER=10^{-4} ) with very small numbers of evaluated PC matrix (≤1000\leq 1000), orders of magnitude less than advertised in the proposal. Similarly, it features a complexity/computation cost study and provides a comparison with a random search approach. The paper approach and this reference seem quite relatable and should be compared.

Questions for authors

Clarification requests:

  • The comparison with the random search should be better explained. In particular, I'm expecting the disclosure of:
    • number of episodes, steps.
    • admissible actions (use of masking?)
  • The use of MLPs for Policy and Value networks should be further motivated, in particular the choice of the activation function (tanhtanh) vs other popular choices (relu2relu^2, gelugelu).
  • Performance comparison with SotA needs an update.
11 comments
AuthorsOpenReview ↗

Fairness of Search Budget

Thank you very much for carefully examining our experimental design and providing constructive feedback. Below, we address your comments regarding the DDPG comparison, library construction cost, Random Search generation rules, MLP architecture choice, and related work.


Reviewer comment

This paper has a potential major evaluation flaw: it compares different search approaches based on their best results obtained with different search budgets. Page 15 paragraph "Baseline implementation" states that DDPG results are taken directly from the publicly available implementation. The number of evaluated matrices is not disclosed and the paper omits this discussion. Looking at the DDPG implementation (https://github.com/KouTian1999/GNN-DRL-Auto-encoder/blob/main/ae_i1_ldpc32.ipynb), DDPG has been trained with 1000 episodes and 25 steps per episodes. This amounts to 25 000 different PC matrices (at most). In contrast, this paper proposal (ZeroCode) is trained with K=10K=10 epochs and T=3×106T=3 \times 10^6 steps, amounting to 1200 times more potential PC matrices evaluated and it's unclear if the results presented are from the matrices evaluated during the training or from the exploitation of the learned policy, i.e. after the training has been completed.

Response

We agree that the search budget and the candidate selection procedure should be clearly distinguished. In short, even when DDPG is given the same search budget as ZeroCode, it does not find a better PCM, so the performance gap stems from the problem formulation rather than the search budget.

We also note that the actual search budget of ZeroCode is 3M environment steps rather than 30M. The 10 PPO epochs in Table 2 denote the number of optimization passes over the already collected rollout buffer and do not correspond to additional environment steps. In addition, the reported final ZeroCode PCMs were selected from those evaluated during training, whereas the code libraries were constructed from a rollout of the trained policy. We have explicitly stated this selection procedure in Appendix A.1 of the revised manuscript.

To examine the effect of the budget difference, we ran DDPG on the (32,16)(32,16) code for 120,000 episodes with 25 steps per episode, allocating the same 3M environment steps as ZeroCode. The table below reports the best-PCM reward obtained in each training interval.

Environment steps0–1M1M–2M2M–3M
DDPG6.96.36.3
ZeroCode8.28.48.4

For DDPG, the overall best PCM was obtained within the first 1M-step interval, and no better PCM was found in subsequent intervals. Moreover, the performance of this best PCM matched that of the PCM reported in the original DDPG paper. In other words, increasing the search budget did not enable DDPG to find a better PCM.

AuthorsOpenReview ↗

Library Construction via Edge Elimination

Reviewer comment

Also the Training Cost discussion is rather thin. Reckoning that the approach creates PC matrix of increasing complexity (+1 edge at each step), the authors claim that their training process generate a pool of N(=41 in the example) PC matrices in a single run while competing approaches need to re-run N(=41) times to reach the same number of matrices. This assumes that competing approaches cannot be adapted to do the same, e.g. by iterative elimination of edges in the Tanner Graph.

Response

The 246 hours for GA and 82 hours for DL in Table 3 are estimates assuming independent optimization for each of the 41 target edge counts. Specifically, GA evolves a population of PCMs using error performance as fitness and DL updates a PCM via gradient-based optimization, so both optimize only the final PCM; in contrast, only ZeroCode learns a sequential policy to produce meaningful intermediate PCMs. Without preference scores for individual edges, competing approaches cannot be readily adapted to construct a library via iterative edge elimination in the Tanner graph.

Following your suggestion, we evaluated iterative edge elimination starting from the final PCMs on the (32,16)(32,16) code. Because existing methods do not provide a per-edge selection criterion, we performed this elimination uniformly at random (random pruning). Specifically, starting from either the final 76-edge PCM constructed by ZeroCode or the 78-edge PCM optimized by GA, we generated ten random pruning trajectories with different random seeds for each initial matrix by removing one remaining edge at random at each step, and compared them against the original ZeroCode construction trajectory across edge counts. Comparing the required Eb/N0E_b/N_0 to achieve BER=10−3\mathrm{BER} = 10^{-3} at 65 edges yields the following results: Fig. 6

TrajectoryZeroCodeRandom pruning (ZeroCode)Random pruning (GA)
Eb/N0E_b/N_04.6 dB6.2 dB6.3 dB

This gap arises because ZeroCode receives the decoding performance of intermediate PCMs as rewards and trains its policy across the entire construction trajectory, whereas random pruning does not account for the performance of intermediate PCMs. As a result, the learned policy provides a per-edge selection criterion based on edge preference scores (action probabilities) at each step, yielding meaningful intermediate PCMs alongside the final PCM in a single rollout. Thus, even if the initial PCM performs well, simply sparsifying it is unlikely to yield a library comparable to the ZeroCode construction trajectory. We have added these results to Appendix A.3 (Figure 6) of the revised manuscript. Thank you for the helpful comment that allowed us to further validate our library-cost estimation.

AuthorsOpenReview ↗

Random Search Details

Reviewer comment

Additionally, the comparison with the Random Search is not conclusive. The paper does not explicit how the Random Search is performed: what is the admissible action space? Is it equivalent to a fixed policy where each action is equally likely? Do the authors implement the same masking strategy? How many steps, episodes have been carried? In the current step, the readers are not presented evidences of that claim.

The comparison with the random search should be better explained. In particular, I'm expecting the disclosure of: number of episodes, steps, admissible actions (use of masking?)

Response

As you pointed out, our description of Random Search in the submitted manuscript was insufficient. Its exact procedure is as follows:

  • Action space: At each step, an entry (i,j)(i,j) that is still 00 is selected and set to Hij=1H_{ij}=1, which is identical to the action space of ZeroCode.
  • Policy: A fixed policy is used that selects one of the admissible actions uniformly at random.
  • Steps and episodes: As in ZeroCode, an episode terminates when no admissible actions remain, and a total budget of 3,000,000 environment steps is allocated.
  • Masking: No masking was applied in the submitted manuscript. To examine the effect of masking, we additionally conducted an experiment applying 4-cycle masking to Random Search as well.

The BER values in the table below are measured at Eb/N0=5E_b/N_0 = 5 dB.

4-cycle maskStepsEpisodesRandom Search BERZeroCode BER
Without3,000,0005,8596.6×10−46.6\times10^{-4}2.2×10−42.2\times10^{-4}
With3,000,00031,7033.5×10−43.5\times10^{-4}1.8×10−41.8\times10^{-4}

Under the same action space and search budget, ZeroCode achieves a lower BER than Random Search in both settings. This demonstrates that the performance gain is not merely the result of evaluating many candidates, but stems from the learned policy. We have added these experimental settings and results to Appendix D of the revised manuscript. Thank you for pointing out this missing detail in our comparison.

Reviewer_TsfNOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper propose the ZeroCode method which use reinforcement learning  to generate parity check matrices for error correcting codes. The proposed ZeroCode constructs the parity check matrices with sequential discrete MDP. The generation is with agent that adds edges using PPO. The actor and the critic is two layer MLPs with inputs of binary parity check matrices and reward of minus log BER when using belief propagation decoder on AWGN. The proposed method can enforce no 4-cycles, quasi-cyclic (QC) block structure and more. Experiments shows improvement like 1 dB gain over DDPG on the (32, 16) code .

Critical strengths
  1. The formulation of the parity check matrices as sequential of single edges from the all-zero matrix with discrete Masked PPO is very novel and good for the problem.

  2. The ablation in table 1 breaks the gain from each component nicely - PPO, starting from the all-zero matrix, single-edge updates

  3. The action masking algorithm is flexible and well-suited for code design. Enforcing 4-cycle free graphs, max degree limits  and so on is very good feature

Critical weaknesses

It seems that the BP decoding algorithm run until 100 frame error rate to compute the reward for single edge. for the first n-1 steps that matrix is all zero and not full rank which is very wasteful when the code is not valid yet.

Questions for authors
  1. Please report the wall clock training time for n equals 64, 128, and 256  like in table 3 for the small (32, 16) code.

  2. The method use 5 iteration of BP in 5db. How good is the code for 50 BP iteration or neural BP decoders?

  3. What is the performance of ZeroCode with only 4-cycle masking in terms of BER curves like in figure 3?

6 comments
AuthorsOpenReview ↗

Cost of Evaluating Partial Matrices

Thank you very much for carefully reviewing our paper and raising these specific questions. Below, we address your comments regarding the evaluation cost of initial partial matrices, training time across block lengths, performance with 50 BP iterations and neural BP, and 4-cycle masking.


Reviewer comment

It seems that the BP decoding algorithm run until 100 frame error rate to compute the reward for single edge. for the first n-1 steps that matrix is all zero and not full rank which is very wasteful when the code is not valid yet.

Response

Thank you for raising this point. In short, evaluating the initial partial matrices is an intentional design choice that provides a reward signal for every edge selection, and its computational overhead is small because these sparse matrices reach 100 frame errors almost immediately.

As you noted, during the first n−1n-1 steps at least one column remains empty, so the partial matrices in this phase are not valid PCMs. However, whether a matrix is a valid PCM is the criterion for selecting candidates for the final comparison, which is separate from the criterion used to compute the training reward. During training, we fix the channel noise variance based on the target code rate k/nk/n and compute the reward from BP decoding on the partial matrix at each step, while only valid PCMs are used for the final performance comparison.

Skipping BP evaluation in this initial phase would reduce computation, but it would also leave the policy without any reward signal until a valid PCM is formed. Because early edge selections shape the structure of the PCM built afterward, training without rewards in this phase would make credit assignment difficult and would likely degrade both policy learning and the final PCM performance.

Regarding cost, the reward is estimated by simulating until 100 frame errors are observed. When the matrix is sparse, nearly every frame is decoded in error, so the simulation for these steps terminates after roughly 100 frames. Most of the decoding cost is therefore spent on later steps, where the BER is low and many frames are needed to observe 100 errors.

AuthorsOpenReview ↗

Training Time Across Block Lengths

Reviewer comment

Please report the wall clock training time for n equals 64, 128, and 256 like in table 3 for the small (32, 16) code.

Response

Thank you for the suggestion. The wall-clock training times for all block lengths are reported below. The training time increases moderately with the block length, from 4 hours at n=32n=32 to 28 hours at n=256n=256.

nn3264128256
Wall-clock time4 hours16 hours23 hours28 hours

The total training time is primarily governed by the BP decoding simulations used to compute the rewards. All experiments were run on a single NVIDIA RTX A5000 GPU. Note that the submitted manuscript erroneously stated the training time as approximately 4 to 18 hours in Appendix A.2; we have corrected this in the revised manuscript.

AuthorsOpenReview ↗

Performance with 50 BP Iterations and Neural BP

Reviewer comment

The method use 5 iteration of BP in 5db. How good is the code for 50 BP iteration or neural BP decoders?

Response

For the (32,16)(32,16) code, the table below summarizes the BER measured at 8 and 50 BP iterations for the PCM constructed by ZeroCode (trained with rewards from 8-iteration BP) and the PCMs obtained by GA, DDPG, and DL. We also report the decoding performance of a neural BP (NBP) decoder trained on each PCM. All BER values were measured at Eb/N0=5E_b/N_0 = 5 dB.

MethodBP-8NBPBP-50
ZeroCode1.8×10−41.8\times10^{-4}1.7×10−41.7\times10^{-4}1.5×10−41.5\times10^{-4}
GA1.9×10−41.9\times10^{-4}1.7×10−41.7\times10^{-4}1.6×10−41.6\times10^{-4}
DDPG1.0×10−31.0\times10^{-3}6.6×10−46.6\times10^{-4}6.1×10−46.1\times10^{-4}
DL5.7×10−45.7\times10^{-4}3.6×10−43.6\times10^{-4}3.8×10−43.8\times10^{-4}

The NBP decoder was trained over Eb/N0=4E_b/N_0 = 4–66 dB using a feed-forward architecture with multi-loss.

The results show that the PCM constructed by ZeroCode achieves the best performance across all evaluated decoders. This demonstrates that the performance gain of ZeroCode does not overfit to the decoding setting used during training (8 BP iterations) and remains effective under other decoding environments. Thank you for suggesting this experiment, which further demonstrates the generality of ZeroCode.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Discussion and Revisions

Thank you to the Area Chair and reviewers for the review process.

We summarize the key issues raised during the review into four points:

  • Novelty and suitability for ICLR: Is the contribution fundamentally different from existing methods, and does it offer sufficient insight for a general ML audience?
  • Fairness of comparison and latest SOTA: How does ZeroCode perform against DDPG and Random Search under the same budget, and against the latest method (GQLA)?
  • Broader experimental scope: Is the method effective beyond rate 1/2 and BP decoding, across other code rates and decoders?
  • Training cost and library efficiency: Are the training times and the library construction cost relative to GA reasonable?

1. Core Contributions and Novelty

ZeroCode's contribution is not a straightforward application of existing RL methods, but a new formulation of the code design problem itself. Whereas existing methods (GA, DL, and prior RL (Tian et al., 2025)) start from a pre-constructed PCM, ZeroCode starts from an all-zero matrix and adds one edge at a time, using action masking to guarantee structural constraints (e.g., 4-cycle avoidance, maximum degree, QC structure) by construction.

In the Table 1 ablation, simply replacing DDPG with PPO actually worsened the BER. Performance that surpasses classical and existing methods emerges only when all-zero initialization, single-edge updates, and action masking work together with PPO.

ZeroCode also offers three insights for a general ML audience.

  • Policy reuse: New constraints are handled at inference time through masking alone, without retraining.
  • Performance–complexity trade-offs: A single training run yields a library of codes.
  • Domain knowledge plus reward learning: Masking encodes code-design knowledge while reward learning drives performance, so the approach adapts to diverse constraints without changing the training procedure.

2. Summary of Additional Experiments and Validation

We addressed the gaps raised during review with the following experiments.

IssueAdditional experimentKey result
Fairness of search budgetRe-ran DDPG on (32,16)(32,16) with the same 33M environment steps (Figure R2)Under the same budget, ZeroCode is superior in both reward and BER over the entire range
Random Search validationCompared with and without 4-cycle masking under the same 33M budget (Appendix D)ZeroCode achieves lower BER than Random Search both with and without masking
Comparison with latest SOTACompared with GQLA (Dufrène et al., 2025) at n=32,64,128n=32, 64, 128 (Figure 3, 5a, 5b)Lower BER at all block lengths
Code rate extensionRates 1/41/4 (64,16)(64,16) and 3/43/4 (64,48)(64,48)Consistently lower BER than Classical, DL, and GA across code rates
Generalization across decodersReward learning with a BSC hard-decision BF decoder (Figure 10c)Codes trained with the BF decoder outperform existing codes (RM, extended QR)
Fairness of library constructionMeasured comparison against GA (Elkelesh et al., 2019) run separately under fixed edge-count constraints at 88 points (Figure R1)Building the 4141-code library takes GA about 219h, versus a single 4h training run for ZeroCode, which yields a similar performance–complexity curve

3. Request for Reconsideration

The reviews can be summarized as two questions. One is whether ZeroCode's contribution goes beyond the application of existing reinforcement learning methods. The other is whether the fairness and scope of the experiments sufficiently support the conclusions. The two questions ultimately converge into one. Whether the performance advantage stems from the problem formulation can be gauged by whether it persists when experimental conditions change.

This rebuttal process was precisely a test of this point. We matched the search budget, varied the code rates and decoders, and added the latest method and GA with per-target edge counts as new baselines; ZeroCode's advantages persisted.

We therefore respectfully ask that you reconsider this work as research that, rather than being a straightforward application of existing methods, proposes a new problem formulation demonstrating a performance advantage even under rigorous comparison conditions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.