A new physical architecture that general intelligence needs

Transformer-type models face high computational overhead due to quadratic complexity when handling ultra-long contexts. Existing sparse attention methods lose global dependency and perform poorly on long-text inference tasks. This paper proposes automatic route attention, a new physical architecture that does not require large computing power to obtain attention, compressing computational load while obtaining global information, reduces inference complexity from quadratic growth with sequence length to linear growth with sequence length. Under the current Transformer architecture, the supported context length is strictly limited by computing power. The current common order of magnitude is 1 million. The new physical architecture in this paper increase context length by four orders of magnitude under current computing and storage conditions. This architecture can help realize artificial general intelligence.