acceptodds
Under review as a conference paper at ICLR 2027

FlexH: Conditional Structural Capacity for Compact Transformer Decoding

Abstract

Transformer-based decoders have achieved strong error-correction performance, but their feed-forward networks (FFNs) remain a major parameter bottleneck. In this paper, we propose FlexH, a plug-in structural preprocessor that addresses this bottleneck by adapting the parity-check representation to each received word. Using channel reliabilities, FlexH prioritizes unreliable positions when selecting linearly independent columns and turns the selected columns into an identity block. This preserves the code and syndrome constraints and provides conditional structural capacity without additional trainable parameters. This allows the FFN expansion factor to be reduced from four to two within otherwise unchanged Transformer decoders. Experiments across three Transformer decoders on BCH, Polar, and punctured Reed–Muller (PRM) codes show average reductions of roughly two orders of magnitude in both bit error rate (BER) and block error rate (BLER) at 6 dB. At the same time, parameters and FLOPs decrease by about one third, with lower peak GPU memory. FlexH–EfficientMPT also surpasses Chase-II in nearly all matched BCH comparisons, with BLER approaching available maximum-likelihood (ML) references on BCH and Polar. Ablation studies further support reliability-guided structural adaptation as a practical way to improve decoding performance with smaller FFNs.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Neural Decoding, Transformer-Based Decoders, Error-Correcting Codes, Parity-Check Matrix Reparameterization, Reliability-Conditioned Decoding, Model Compression

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_pUxSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper concerns Transformer decoding of binary error-correcting codes. FlexH constructs a parity-check representation separately for each received word: it greedily selects a full column basis in increasing order of channel-reliability magnitude, then applies invertible GF(2) row operations to turn those columns into an identity block. Applying the same transformation to the syndrome preserves the decoding constraints. The representation is computed once and held fixed throughout neural inference. This preprocessing improves decoding enough to reduce the usual FFN expansion from four to two.

Critical strengths
  1. Improvements under reasonably controlled neural comparisons. The paired models use the same systematic starting matrix, depth, embedding dimension, training budget, target, and loss (Section 4.1).

  2. The structure–width ablation addresses an important confound. Table 3 compares Static-H and FlexH at both 2d and 4d.

  3. The parameter reduction follows directly from the narrower FFNs, and preprocessing introduces no learned parameters.

  4. Insightful paper that is fun to read

Critical weaknesses
  1. Comparisons with the reliability-based methods ABP [1] and OSD [2] are missing or incomplete. FlexH’s preprocessing is not an alternative to these methods but the same operation: up to tie conventions, it coincides with the first ABP adaptation step and with the dual of OSD’s most-reliable-basis elimination. The reported comparisons do not address this. ABP appears only as a standalone iterative decoder (at most 20 iterations, α = 0.1), which mixes representation, inference procedure, and training, and OSD is absent from the paper. Three plug-in comparisons are missing. First, OSD-0 produces a codeword directly from (H_y, s_y) with no learned parameters; it is the first control, followed by low-order OSD performance–cost curves on the same code constructions. Second, the ABP paper’s own variations of the reduction, degree-2 random connection and multiple groupings of the unreliable bits [1, Section IV], yield different adapted matrices that could replace FlexH’s identity form, whereas Table 3 tests only the authors’ own basis rules. Third, ABP re-adapts the matrix as reliabilities are updated, whereas FlexH freezes it, and the effect of that choice is not tested. MM-ECCT [3] is also a meaningful missing matched baseline.

  2. Results are compared with the authors’ own baseline runs rather than with published numbers. ECCT, CrossMPT, and EfficientMPT are retrained by the authors from a systematic matrix, and Hyper-BP, ABP, Chase-II, and RPA are also run by the authors. The only external values are the ML references and the Cyclic CrossMPT results, which appear to be taken from its paper. No table lists the published results of the three backbones, or of other published Transformer decoders such as DM ECCT [4] and DDECC [5].

[1] J. Jiang and K. R. Narayanan, "Iterative soft-input soft-output decoding of Reed–Solomon codes by adapting the parity-check matrix," IEEE Transactions on Information Theory, vol. 52, no. 8, pp. 3746–3756, 2006. doi:10.1109/TIT.2006.878176; arXiv:cs/0506073.

[2] M. P. C. Fossorier and S. Lin, "Soft-decision decoding of linear block codes based on ordered statistics," IEEE Transactions on Information Theory, vol. 41, no. 5, pp. 1379–1396, 1995. doi:10.1109/18.412683.

[3] S.-J. Park, H.-Y. Kwak, S.-H. Kim, S. Kim, Y. Kim, and J.-S. No, "Multiple-masks error correction code transformer for short block codes," IEEE Journal on Selected Areas in Communications, vol. 43, no. 7, pp. 2518–2529, 2025. doi:10.1109/JSAC.2025.3559154.

[4] S.-J. Park, H.-Y. Kwak, S.-H. Kim, S. Kim, Y. Kim, and J.-S. No, "How to mask in error correction code transformer: Systematic and double masking," arXiv:2308.08128, 2023.

[5] Y. Choukroun and L. Wolf, "Denoising diffusion error correction codes," in International Conference on Learning Representations (ICLR), 2023. arXiv:2209.13533.

Questions for authors

Please address the weaknesses

Reviewer_jWWuOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper presents a simple and potentially useful structural preprocessor that substantially improves several Transformer-based decoders while allowing smaller feed-forward networks (FFNs). The matched architecture comparisons and structure–width ablations provide meaningful evidence for this empirical contribution.

Critical strengths
  1. The main performance gains are substantial and supported by useful controls. The evaluation spans three Transformer architectures and three code families. On EfficientMPT and three representative codes at 6 dB, Table 3 crosses Static-H/FlexH with FFN widths of 2d/4d, showing that narrowing the FFN alone does not explain the improvement. Random-full and Most-reliable-full controls further support the importance of reliability-guided basis selection.

  2. The evaluation considers complete-codeword recovery and model cost. Reporting both BER and BLER makes the performance evidence more informative than bit accuracy alone. Parameter savings are explicit, and the authors state that preprocessing is included in their FLOP counts. The appendices also disclose limitations, including a peak-memory increase for one long BCH configuration and FFN update changes that are not uniformly decreasing across layers.

Critical weaknesses
  1. The distinction from the paper's existing approaches and the capacity interpretation need further clarification. Section 3.2 states that FlexH uses matrix operations similar to ABP and distinguishes its one-shot construction from iterative matrix adaptation. A step-by-step comparison with the ABP baseline implemented in the experiments would help clarify which aspects of basis selection, matrix transformation, and decoder integration differ, and where the proposed contribution lies. This is a request for clarification, not a conclusion that the two algorithms are equivalent. A direct comparison with MM-ECCT, which motivates the paper's discussion of identity-column coverage, would also strengthen the experimental positioning.

  2. Some close baseline comparisons need stronger statistical support and budget analysis. Appendix H.1 specifies one training seed, and repeated-training variability is not reported. This concern matters most for small differences; it is unlikely to explain the multi-order gains over the fixed-graph baselines. Similarly, selected points are numerically close to the external ML references, but their statistical precision should be reported before drawing stronger conclusions about closeness to ML performance.

Questions for authors
  1. Section 3.2 notes a relationship between FlexH's matrix operations and ABP. For the ABP baseline used in your experiments, which steps of basis selection and matrix transformation differ from FlexH when the initial matrix, reliability order, and coordinate convention are matched? Is the main contribution the transformation itself, the one-shot Transformer interface, its interaction with FFN compression, or a combination of these? Can you include a direct MM-ECCT comparison?

  2. Can you report end-to-end latency, throughput, and preprocessing/inference cost breakdowns at batch size one and representative larger batches, including long-code configurations? How are GF(2), sorting, and dynamic structure operations included in the reported FLOP counts?

  3. Can you provide actual evaluated-word and block-error counts, confidence intervals, and repeated-training results for the main comparisons? How do the conclusions change across ABP iteration limits and Chase-II search budgets, particularly at the operating points with small reported differences?

Reviewer_ZWe9OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes FlexH, a preprocessing step for transformer decoders. For each received word, it uses channel reliability to choose an equivalent parity-check matrix and changes the syndrome accordingly. The decoder then uses this new representation with a smaller FFN.

Critical strengths

Table 3 is the most convincing part for me. Making the FFN smaller hurts the fixed-matrix baseline. With FlexH, the smaller FFN does much better and stays close to the wider version. This gives a clear reason to consider the method.

The gains are shown on three Transformer decoders and several code families. The random-basis and most-reliable-basis controls are also useful. They show that changing the matrix alone is not enough; how the columns are selected matters.

Critical weaknesses

I'm not fully clear on how much of the method is new. ABP already uses reliability information to turn unreliable columns into an identity block. I understand that FlexH does this once before the neural decoder, while ABP updates the matrix during decoding. The main contribution seems to be using this idea with Transformer decoders and showing that the FFN can be smaller. I think the paper should make this more explicit. The explanation about reducing the FFN's burden is reasonable, but the feature-update results alone do not fully explain why the method works.

The FLOP savings do not yet tell us how much faster the decoder is. The paper includes preprocessing in its counts, but sorting and binary elimination may have different costs from the neural network operations. Actual timing for the whole pipeline would help, especially on longer codes. The mixed results on long Polar codes also suggest that the benefit depends on the setting.

Questions for authors
  1. Is the matrix construction equivalent to one reliability-based adaptation step in ABP? What parts are different?
  2. Can you report latency or throughput for one short code and one longer code, including sorting, matrix adaptation, and neural decoding?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.