acceptodds
Under review as a conference paper at ICLR 2027

An Algebra of Transformers

Abstract

On the theoretic analyses on transforms, they mainly focus on the computability analysis, such the Turing-completeness of unbounded precision transformers, the advantage of parallelizability of unbounded precision transformers and the limitations of bounded precision transformers. This paper give transformers an algebra for reasoning, which relies heavily on the functional decomposition results of transformers. This algebra uses functional decompositions as atomic actions, includes sequential composition and parallel composition, recursion for feedback and abstraction for hidden layers. As we know, it is the first attempt to model transformers logically and can be used to verify the correctness of transformer based applications.

Primary area:
foundation or frontier models, including LLMs
Keywords:
transformers, algebra, sequential composition, parallel composition

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_XQaeOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper is totally AI-written and is longer than the page limit. It should be desk reject.

Critical strengths

none

Critical weaknesses

This paper is totally AI-written and is longer than the page limit. It should be desk reject.

Questions for authors

This paper is totally AI-written and is longer than the page limit. It should be desk reject.

Reviewer_PS4pOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

I don't quite follow this paper because the explanations are too short. From the introduction, it seems like this paper is doing a follow up on (Yang et al. 2025) which breaks down the Transformer architecture into atomic operation. Maybe the author of this paper is to develop a more efficient framework? I don't know

Yang et al. The Transformer cookbook. 2025

Critical strengths

No strength

Critical weaknesses
  • poor organization
  • poor explanation
Questions for authors

no questions

Reviewer_BhZ8OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes a process algebra for Transformer computations, using operations from the Transformer Cookbook as atomic actions. It introduces sequential and parallel composition, guards, recursion, and abstraction, and claims soundness and completeness with respect to pomset bisimulation

Critical strengths

None

Critical weaknesses

This paper is quite difficult to follow. It reads more like a preliminary technical note than a research paper. Moreover, I have two major factual concerns based on just the first paragraph of the introduction.

  1. “Because of the parallelizability of transformers Vaswani et al. (2023), transformers are becoming the mainstream techniques in large language models (LLMs)”: I do not think so.
  2. “They mainly focus on the computability analysis”: there are already many papers discussing efficiency issues. In general, the discussion of related work is too limited.
Questions for authors

None.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.