Parameter-Efficient Error Correction Code Transformers

Error correction coding is a cornerstone of reliable communication systems, ensuring data integrity over noisy channels. The Error Correction Code Transformer (ECCT) and its successors have demonstrated that Transformer-based neural decoders can outperform classical decoding methods for short and medium linear block codes. However, existing Transformer-based decoders suffer from high parameter counts and computational costs, both of which scale linearly with model depth, constraining their deployment scenarios and performance potential. In this work, we propose two architectures to tackle the parameter efficiency challenge. First, we introduce the Shared Error Correction Code Transformer (SECCT), which employs cross-layer parameter sharing, effectively collapsing the model to a single Transformer layer that is applied repeatedly. By preserving decoding depth through iteration rather than stacked distinct layers, SECCT improves the bit error rate (BER) over the published ECCT baseline on all 39 code–signal-to-noise ratio (SNR) settings we evaluate while using less than 20% of the parameters of that baseline. The price is only a modest increase in computational cost due to additional forward passes. This provides a viable solution with performance guarantees for memory-constrained edge deployment scenarios. Second, we propose the Hybrid Error Correction Code Transformer (HECCT), which employs Grouped-Query Attention (GQA) and a bottleneck feed-forward structure to simplify each Transformer layer. HECCT achieves lower parameter count and computational cost than the published ECCT baseline, while delivering better BER performance, establishing a new Pareto-optimal frontier in the accuracy–efficiency trade-off. Extensive experiments on 13 codes from five families validate the effectiveness of both architectures, which provide a reliable and efficient solution for next-generation communication systems.