Transformers Miss the Middle Digits: Repairing Long Multiplication by Recursion and Robust Recombination

Transformers trained to multiply long integers tend to get the first and last digits of the product right and the middle ones wrong. We show that this is structure, not noise. For a flat model that writes the product directly, the five central digits of $5\times 5$ products are predicted at chance level (10–13%) while the first digit is still $74\%$ accurate, and the failing band widens with the number of digits. The triangular profile of column interactions motivates this diagnosis. Under an explicit residual-channel model, we prove a digitwise error law and conditions for a central error peak and a widening band near chance. We quantify corrections for carry errors, dependence, and decoder mismatch. We also prove an exact local recombination rule and attention-leakage and margin conditions for its stable execution. We build the Recursive Arithmetic Transformer(RAT), a single Transformer that is never supervised on products beyond $2\times 2$ digits. Exact oracles for Split move its accuracy on 9- and 10-digit products by at most 4.2 points, and training Combine on wider inputs raises exact match from $73\%$ to $86\%$ on $10\times 10$ and from $36\%$ to $55\%$ on $18\times 18$ multiplication, with a gain at every length from 8 to 18 digits. Long-multiplication failures should be diagnosed, and repaired, one stage at a time.