- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper proposes FLINT, an event-conditioned recurrent state tracker, and EGAM, a fixed-capacity associative memory, to reduce state-transition computation and memory-projection costs while preserving language modeling quality.
Critical strengths
- The paper presents a principled decomposition of recurrent state tracking and associative recall, addressing complementary limitations of efficient sequence models.
- FLINT introduces an exact event-boundary execution schedule that reduces expensive state-transition work in proportion to the number of events.
- Experiments demonstrate substantial reductions in mixer MACs and estimated arithmetic energy with minimal perplexity degradation across two language modeling benchmarks.
Critical weaknesses
- The language modeling experiments use relatively small models and limited training budgets, with only one seed reported for the FineWeb-Edu evaluation.
- The arithmetic energy improvements are based on an analytical cost model rather than direct hardware energy measurements, leaving the practical deployment benefits insufficiently established.
- State-tracking accuracy degrades substantially under longer evaluation sequences, while the associative memory exhibits limited performance on repeated-key rebinding tasks.
Questions for authors
- How does the event-boundary execution schedule perform in end-to-end inference latency against optimized baselines when using learned gates and realistic batch sizes?
- Can the authors validate the claimed energy savings using direct hardware measurements that include memory access, control overhead, and other unaccounted operations?
- What modifications could improve long-context state retention and repeated-key rebinding without increasing the fixed memory budget or computational cost?
- Why do the authors not compare EGAM with Gated Slot Attention?
5 comments
Response to Reviewer Zt9f (2/3), Q2 direct hardware measurements
Q2 / W2
Can the authors validate the claimed energy savings using direct hardware measurements that include memory access, control overhead, and other unaccounted operations?
We agree that the analytic model alone does not establish deployment benefits, and we kept as the labeled arithmetic subtotal (Appendix A). We added three direct measurements.
(i) The hardware that the two-term projections target, an FPGA-prototyped RV32 core at 100 MHz with a multiplier-free datapath. Rail power comes from the board's PMBus monitors, and every checksum is bit-exact against a NumPy model. The module and mixer rows count every cycle on the core, so memory access and control are included.
| On the RV32 core | int8 MAC | two-term shift-add | two-term / int8 |
|---|---|---|---|
| EGAM key projection, cycles per weight | 99.5 | 38.0 | 0.38x |
| EGAM key projection, dynamic energy per weight | 170.8 nJ | 62.2 nJ | 0.36x |
| EGAM module, cycles per token (equal dynamic power) | 40.9M | 16.6M | 0.41x |
| Combined mixer (FLINT + EGAM), cycles per token | 68.6M | 44.3M | 0.65x |
| Attention mixer, cycles per token at L = 256 / 1024 / 2048 | 133.8M / 216.6M / 327.0M | two-term combined mixer at 0.33x / 0.20x / 0.14x of attention |
(ii) GPU energy per forward on the GPU of Q1, where two-term weights run through the same floating-point units as FP weights. We read the NVML total-energy counter around synchronized groups of forward calls, at least 2 s per group and the median of five groups, with idle power measured on the free device and subtracted. The tables give mJ per forward, total with the idle-subtracted value in parentheses.
| WikiText-103 (50.7M), mJ per forward at B x L, total (idle subtracted) | 1 x 1024 | 16 x 1024 | 1 x 4096 | 16 x 4096 |
|---|---|---|---|---|
| Diagonal reference | 2,636 (1,229) | 17,512 (14,313) | 10,260 (5,010) | 76,439 (62,627) |
| Diagonal + FLINT (ours) | 3,599 (1,536) | 18,629 (14,880) | 12,246 (5,638) | 80,822 (65,299) |
| Diagonal + FLINT + EGAM (ours) | 3,943 (1,890) | 20,706 (17,000) | 13,719 (7,600) | 104,910 (87,590) |
| DeltaNet | 3,634 (1,513) | 18,891 (15,174) | 11,745 (5,574) | 82,776 (66,423) |
| Gated DeltaNet-2 | 3,787 (1,664) | 20,176 (16,325) | 12,027 (5,963) | 92,165 (74,658) |
| Gated DeltaProduct (k=2) | 3,670 (1,485) | 19,191 (15,497) | 12,107 (5,850) | 85,362 (68,639) |
| Mamba2 | 3,414 (1,497) | 18,379 (14,887) | 11,274 (5,373) | 79,390 (64,002) |
| xLSTM adaptation | 5,092 (2,501) | 40,273 (31,934) | - | - |
| Attention | 3,181 (1,421) | 19,327 (15,392) | 11,515 (5,446) | 88,356 (70,659) |
| FineWeb-Edu (122.5M), mJ per forward at B x L, total (idle subtracted) | 1 x 1024 | 16 x 1024 | 1 x 4096 | 16 x 4096 |
|---|---|---|---|---|
| Diagonal + FLINT + EGAM (ours) | 6,521 (3,074) | 43,792 (35,985) | 22,827 (12,218) | 204,502 (167,663) |
| Gated DeltaNet-2 | 6,471 (3,137) | 42,434 (35,388) | 21,525 (12,400) | 189,946 (157,583) |
| Mamba2 | 5,576 (2,658) | 37,586 (31,112) | 18,815 (9,869) | 164,806 (135,908) |
| Attention | 5,606 (2,995) | 39,429 (32,401) | 19,745 (10,945) | 186,165 (153,512) |
On WikiText-103 the combined model uses 1.09 to 1.32x the total energy per forward of DeltaNet and Mamba2, and its perplexity is 4.0 lower than DeltaNet's. On FineWeb-Edu it uses 1.01 to 1.08x the total energy of Gated DeltaNet-2 and 1.17 to 1.24x that of Mamba2, at 1.2 and 1.4 lower perplexity.
(iii) A Jetson Orin (JetPack 6.2). The CPU+GPU rail energy comes from the board's INA3221 monitors with idle subtracted, and the tables give the median ms and the idle-subtracted mJ per forward.
| WikiText-103 (50.7M), ms / mJ per forward at B x L | 1 x 1024 | 4 x 1024 | 1 x 4096 |
|---|---|---|---|
| Diagonal reference | 241 / 759 | 425 / 2,581 | 934 / 3,205 |
| Diagonal + FLINT + EGAM (ours) | 295 / 961 | 425 / 2,930 | 1,067 / 3,809 |
| DeltaNet | 276 / 903 | 391 / 2,866 | 974 / 3,386 |
| Gated DeltaNet-2 | 276 / 941 | 385 / 2,950 | 941 / 3,452 |
| Gated DeltaProduct (k=2) | 265 / 883 | 362 / 2,735 | 921 / 3,165 |
| Mamba2 | 260 / 864 | 370 / 2,758 | 899 / 3,232 |
| xLSTM adaptation | 357 / 1,228 | 707 / 4,954 | - / - |
| Attention | 253 / 862 | 411 / 2,877 | 987 / 3,558 |
| FineWeb-Edu (122.5M), ms / mJ per forward at B x L | 1 x 1024 | 4 x 1024 | 1 x 4096 |
|---|---|---|---|
| Diagonal + FLINT + EGAM (ours) | 446 / 1,802 | 754 / 5,855 | 1,639 / 7,145 |
| Gated DeltaNet-2 | 425 / 1,913 | 710 / 6,095 | 1,545 / 6,941 |
| Mamba2 | 392 / 1,753 | 704 / 5,759 | 1,449 / 6,589 |
| Attention | 401 / 1,799 | 767 / 6,228 | 1,558 / 7,457 |
On the edge device the combined model uses 1.02 to 1.18x the rail energy of DeltaNet and Mamba2 on WikiText-103. On FineWeb-Edu it uses 1.02 to 1.08x that of Mamba2, 0.94 to 1.03x that of Gated DeltaNet-2 and 0.94 to 1.00x that of attention. The core measurements are in Appendix A, and the GPU energy and Orin tables are in Appendix E.4.
Q3 and Q4 follow in the next comment.
Response to Reviewer Zt9f (3/3), Q3 length and rebinding, Q4 Gated Slot Attention
Q3 / W3
What modifications could improve long-context state retention and repeated-key rebinding without increasing the fixed memory budget or computational cost?
Thank you for raising both. Each has a measured mitigation that keeps the fixed memory budget. Training at the target length leaves inference unchanged. Restoring the state from the store costs one write and one read per gap, and the erase-then-write rule costs one masked subtraction per admitted write.
Length
The degradation is at 4x the training length. Training at L = 512 gives 0.987 to 1.000 at that length (Appendix D.1), so training at the target length recovers the accuracy without a larger state. The joint store-once task, trained at L = 128, reaches 0.969 to 0.992 query accuracy at L = 512 with learned gates (Table 3). A second option uses the existing store across long gaps. Appendix D.4 writes the FLINT state into the associative store at a segment boundary and restores it after the gap, with a zero-overlap address code selected from 2,048 random candidates. Final-segment accuracy rises from 0.095 / 0.184 to 1.000 / 1.000 across one 4,096-token gap and from 0.013 / 0.006 to 0.860 / 0.893 across four (shadow-state / true-state gates). Both results match an exact copy of the state. This intervention uses oracle timing and non-leaky continuous storage.
In the language model, the combined WikiText-103 model reaches 26.78 / 26.82 / 26.64 PPL at L = 8192, 8x the training length, against 26.78 / 26.72 / 26.62 at L = 1024. The change is within 0.4% in every seed, while the attention model goes from 28.83 to 48.70 (new, Appendix E.4).
Rebinding
The additive store averages repeated writes to one key, which is the failure Table 3 isolates. The store update of Sec. 3.2 adds every admitted write into the selected slots,
so after STORE$_k$ with value and a later STORE$_k$ with the slots of key hold and QUERY$_k$ reads their mean. The erase-then-write rule replaces the content of the selected slots instead,
so the selected slots hold after the write. The write counter becomes an occupancy flag, , and the read averages the matched occupied slots. Addressing uses a fixed disjoint codebook. The 256 slots are split into four blocks of 64, STORE$_k$ writes to block and QUERY$_k$ reads block , so repeated writes to one key land on the same slots and never touch another key's slots. The state is unchanged (256 slots, no extra accumulator), and the write costs one masked subtraction more than the additive write. The table (new, Appendix D.3) gives rebinding query accuracy on the Table 3 protocol with oracle gates and this codebook, four cells per row (two learning rates x two seeds). Both rules use the codebook, so the additive row differs from the rebinding row of Table 3 (0.835 to 0.905 at L = 128), which uses the addressing of Sec. 3.2.
| Rebinding query accuracy | L = 128 | L = 512 |
|---|---|---|
| Additive write | 0.809 to 0.835 | 0.285 to 0.341 |
| Erase-then-write | 0.999 to 1.000 | 0.674 to 0.788 |
With the erase-then-write rule, rebinding queries are answered at 0.999 to 1.000 at the training length. At 4x that length the accuracy is at least 0.674, against at most 0.341 for the additive write. The language-model store keeps the additive write.
Q4
Why do the authors not compare EGAM with Gated Slot Attention?
Thank you for the pointer. It is the right baseline for a fixed-slot memory, and we added it with the same backbone, the same parameter matching and the Mamba2/xLSTM selection procedure (fla implementation, 4 heads x 128 slots). We selected the learning rate on seed 1234 from 3e-4, 6e-4 and 1.2e-3, which gave 31.607, 28.622 and 27.973 PPL, then trained seeds 1235 and 1236 at 1.2e-3 (28.133 and 27.873). The table gives the three-seed means.
| Model (WikiText-103, 3 seeds) | PPL FP | PPL two-term | Mixer MACs (FP / two-term) | State |
|---|---|---|---|---|
| Gated Slot Attention (4 heads x 128 slots) | 27.993 | 28.017 | 1.90M / 1.90M | 131,072 |
| Diagonal + FLINT + EGAM | 26.705 | 26.709 | 0.865M / 0.469M | 131,584 |
| Diagonal + EGAM, parameter-matched (no FLINT) | 26.487 | 26.496 | 0.601M / 0.204M | 131,456 |
At nearly the same state size the combined block is 1.3 PPL lower than GSA with 0.865M FP mixer MACs against 1.90M (0.469M with two-term weights). The memory-only control, Diagonal + EGAM, is 1.5 PPL lower than GSA with 0.601M. On the zero-shot extraction probes of Table 4, GSA scores 2.01 on SWDE and 3.07 on SQuAD against 2.01 and 5.47 for the combined model (FP, three seeds). The GSA row is in the new Table 5 and Sec. 5.4.
We are grateful for the constructive questions and hope that the measurements above address them.
Response to Reviewer Zt9f (1/3), W1 seeds and scale, Q1 end-to-end latency with learned gates
Thank you, Reviewer Zt9f, for the constructive review and the four specific questions. We measured each one and report the results below. This comment covers W1 and Q1, the second covers Q2 and the third covers Q3 and Q4. Everything is in the revised PDF, in the new Table 5 (the controls table, which moves the FineWeb table to Table 6) and in Appendices A, C.5, D.3 and E.4. In that PDF, new content is blue and wording edits to the submitted text are green.
W1
Thank you. We agree that one seed is not enough. We trained two more FineWeb-Edu seeds for all four models at the FineWeb-Edu recipes. The FP perplexities for seeds 1234 / 1235 / 1236 are 29.328 / 29.367 / 29.370 for the combined model, 30.431 / 30.565 / 30.560 for Gated DeltaNet-2, 30.698 / 30.854 / 30.807 for Mamba2 and 31.213 / 31.241 / 31.188 for attention. The ordering of the four models is the same in every seed, with the combined model 1.1 to 1.2 PPL below the next model, and the paired two-term change is at most 0.040 PPL across the twelve runs. Table 6 now reports three-seed means, and the per-seed values are in Appendix C.5. On scale, 50.7M parameters at 236M tokens and 122.5M at 1.0B tokens are the budgets at which every baseline could be trained on one GPU with identical data order. The two-term change of the combined model stays small at both, 0.004 PPL at the smaller scale (Table 4) and 0.011 PPL at the larger one (Table 6).
Q1 / W2
Thank you for asking for the full-model number. We measured the end-to-end latency of every Table 4 model and every FineWeb-Edu model with its learned gates. The runs use the same checkpoints and settings as those tables, embedding through vocabulary head, in bf16 on the GPU of Table 1. FLINT runs in the fused state path of Table 1, EGAM runs as compiled PyTorch, the baselines run in their library kernels, and no CUDA graphs are used. The tables give the median ms per forward for each B×L (batch size x sequence length). The xLSTM adaptation is configured for context 1024 and does not run at 4096.
The complete model takes 1.2 to 1.3x the time of DeltaNet and Mamba2 on WikiText-103. On FineWeb-Edu it takes 1.0 to 1.1x the time of Gated DeltaNet-2 and 1.1 to 1.3x that of Mamba2. Without EGAM, Diagonal + FLINT takes 1.01 to 1.10x the time of DeltaNet and 0.93 to 1.04x that of Gated DeltaNet-2.
Appendix E.3 isolates the schedule by running one trained recurrence, with weights, inputs and gates frozen, token by token and by event boundaries. The recurrence speedup of the event schedule grows with length and reaches 1.84x at L = 4096 with batch 16 and 2.73x with batch 1 (Table 15). The recurrence is one part of the forward pass. The MLP costs about 2.1M MACs per token and layer against 0.865M for the mixer (Appendix C.4). The savings of two-term weights are measured in Q2, on the hardware they target. The grid is in Appendix E.4, with one sentence in Sec. 5.1.
Q2 follows in the next comment.