- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes EVER, a sequence model in which each step performs Gaussian prior–evidence fusion. The previous state serves as the prior mean with a learned, time-invariant variance , the input yields a learned Gaussian evidence factor, and a sigmoid gate controls how much of the posterior mean is adopted. The resulting update is a diagonal affine recurrence , and the authors call its associative closure VCC and compute it with a parallel prefix scan. Experiments report gains over parameter-matched Transformer and cross-attention baselines on small-scale tasks, but EVER underperforms Mamba-2 on FineWeb language modeling.
Critical strengths
- The formulation is simple and analytically transparent. The fusion step yields closed-form coefficients that guarantee , and the Triton scan is verified against the sequential recurrence to floating-point precision.
- The experimental protocol is careful in places and reports unfavorable results openly. The WikiText-2 control matches architecture and parameter count exactly, uses paired seeds and a frozen test split, and the paper does not hide its losses to Mamba-2 or the retrained ablations that match or beat the full model.
Critical weaknesses
-
The abstract is very poorly organized. It packs in experimental details such as exact parameter counts, p-values, and seed counts, and it frames a clear loss to Mamba-2 as "an architectural diagnosis". The core idea and contribution are hard to extract from it.
-
Large parts of the appendix read as unedited text. Around L1080–1100, sentences are broken into isolated one-line fragments such as "evidence value, and relative credibility of the evidence". The same pattern appears in F.1 and G.2, where single numbers sit on their own lines, and in standalone filler sentences such as "This result quantifies how well the variants compensate after retraining." The text also refers to "the notebook implementation". Together these suggest the manuscript was largely AI-generated and not carefully reviewed by the authors before submission.
-
The proposed transition reduces to a standard gated linear RNN. Because is time-invariant and the posterior variance is never propagated, is simply an input-dependent sigmoid. The gate feature is a deterministic function of . The update therefore has the same form as minGRU, differing only in how the gate is parameterized. VCC is the well-known associativity of affine maps that underlies existing parallel scans. Closely related work such as minGRU, S5, LRU, and HGRN is not cited.
-
The paper's own ablations undermine the claimed mechanism. On PTB, a constant-variance model retrained from scratch reaches 4.9902, better than the full model at 5.0127. The variant with in H.4, which removes the adoption gate the authors call central, outperforms canonical EVER by over 0.4 nats. The only architecture-matched gain is 0.03 nats, which is attributable to gate parameterization rather than to belief revision.
-
The baselines are weak. Mamba-2 beats EVER by about 1.5 nats at 120M, and EVER's NLL worsens as model size grows. The Transformer baseline is non-monotonic in size and trails Mamba-2 by nearly 2 nats, which suggests it is undertrained. The generalist suite includes no RNN or SSM baseline. The core-budget control uses a 10× smaller learning rate with persistent gradient clipping for the widened model, so it cannot support the conclusion about state geometry.
Questions for authors
-
Beyond gate parameterization, how does Affine-EVE differ from minGRU or other gated linear RNNs? Since is fixed and posterior variance is not propagated, in what sense does represent a belief? A direct comparison with minGRU, LRU, or S5 under the same setup would be needed to support the claimed contribution.
-
The deterministic control NLL in Table 3 on WikiText-2 and the direct deterministic transition NLL in Table 8 on PTB are both 5.0540. Please clarify why two experiments on different datasets report the same value.