acceptodds
Withdrawn from ICLR 2027

Activation Density in Transformer Feed-Forward Networks Is Set by the Nonlinearity, Not the Width

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Two thirds of a Transformer's parameters sit in its feed-forward blocks, whose width follows from an expansion ratio fixed at 4 since 2017. Work on activation sparsity has tied that width to the fraction of the block a model uses. We find the two nearly independent. We measure the fraction of FFN units carrying the output, the activation density, with a participation ratio, which needs no threshold and so works for gated activations. Across nine Transformers trained to differ only in , density moves 7% over a fourfold range, reconciling an earlier report of sparser wide MLPs that confounded width with model scale. Density is set by the nonlinearity. Two moments of the pre-activation, with the activation function, predict it to 0.49% across non-gated nonlinearities whose densities differ by a factor of 3.6; gated activations carry a residual we quantify and leave open. The prediction's closed forms, such as for ReLU, hold only at zero pre-activation mean, which training destroys. Constraining the first-layer rows to average to zero restores that mean: ReLU density returns to within 0.65%, and the width slope vanishes, as pre-registered, for both activations whose gate is positively homogeneous. Applied to a covariance spectrum, the same participation ratio understates effective rank by a median factor of four, and by up to twenty-six, because trained networks learn a handful of massive activations. Utilization stays put while width changes, so it cannot be what sets the width; any case for a ratio must be made on capacity.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
activation sparsity, activation functions, feed-forward networks, Transformer architecture, expansion ratio, effective rank, massive activations

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_LwZFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies FFN activation density, defined as a width-normalized participation ratio of activation magnitudes. Controlled TinyStories experiments vary the expansion ratio while holding the training recipe fixed, and measurements on eleven pretrained models extend the analysis. The paper expresses density as a functional of the empirical pre-activation distribution, tests a Gaussian moment model, and uses a zero-mean-row constraint to investigate the effect of pre-activation mean. It finds weak width dependence for its SwiGLU sweep, accurate moment-conditioned predictions for the controlled non-gated activations, and larger residuals for gated models. It also compares spectral participation ratio with entropy-based effective rank.

Critical strengths
  1. Controlled experiments isolate a relevant architectural variable. Holding data order, model dimension, depth, and training schedule fixed makes the expansion-ratio sweep more interpretable than a comparison across unrelated pretrained models. Reporting seed variability, validation loss, and the hardware sensitivity of the slopes makes the empirical evidence easier to assess (Tables 4 and 10; Appendix K).

  2. The intervention tests a specific mechanism. The zero-mean-row experiment goes beyond fitting observed activation statistics. The different predictions for homogeneous and non-homogeneous gates are useful controls: ReLU density returns close to its zero-mean prediction, while SwiGLU retains a positive slope. The paper also reports the gated residuals and failed initial prediction rather than presenting the Gaussian approximation as universally accurate.

Critical weaknesses
  1. The measured density does not establish the claimed notion of FFN utilization. Equation 2 uses activation magnitudes but ignores the norms and directions of the output-projection columns appearing in Equation 1. Consequently, a unit with a large activation need not contribute proportionally to the FFN output, and cancellation is not represented. For a ReLU FFN, positive per-unit rescaling of the input weights and bias, compensated by inverse rescaling of the corresponding output column, preserves the network function but can change this density. The statistic is still useful descriptively, but the inference that utilization cannot inform width or exploitable sparsity is too broad without an output-aware or ablation-based check. Relatedly, disagreement between spectral participation ratio and entropy effective rank does not alone establish that the former underestimates a ground-truth effective dimension: the two measures weight a nonuniform spectrum differently (Section 3.1; Appendix A).

  2. Width independence is more conditional than the headline conclusion suggests. The 7% change comes from small SwiGLU models trained on TinyStories. The paper’s unconstrained ReLU sweep shows a reduction from 0.1954 to 0.1569 as the ratio increases from 2 to 8, approximately 20% (Table 10). The 0.49% predictive error is an average over the two controlled non-gated arms, whereas pretrained SwiGLU errors reach 26–35%. Proposition 1 is an exact reformulation of the statistic; the invariance corollary additionally assumes convergence to a width-independent distribution and uniform integrability. The shared-component measurement addresses one dependence model, but does not establish all those conditions, particularly tail control in the presence of massive activations. These results support a useful conditional account, but not yet the general claim that width cannot affect utilization or available sparsity.

Questions for authors
  1. Do the width trends persist when density is computed from the norms of per-unit output contributions, or when utilization is measured by the fraction of units that can be removed at a fixed loss increase? Such a check would directly address the link between Equation 2 and the efficiency conclusions.

  2. How should the title and the claim that changing width cannot increase available sparsity be reconciled with the approximately 20% density decrease in the unconstrained ReLU sweep? Which conclusions are intended specifically for SwiGLU, and which are supported across activations?

  3. Beyond the equicorrelated shared-component proxy, is there evidence that the relevant empirical moments and tail contributions stabilize across the controlled width sweep? Please distinguish direct support for the corollary’s assumptions from observations that are merely consistent with them.

Reviewer_VcjkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The authors study the expansion rate of the FFN in a transformer in relation to activation sparsity. This factor determines how much wider the fully connected classifier is compared to the backbone. A previous work has linked an increase of the expansion rate to a higher activation sparsity -- the percentage of 0-valued activations in the final layers, which the authors strive to disprove. By using various experiments over different expansion rates with different models, they show that activation sparsity is not significantly tied to expansion rate, but rather that the change is more-or-less flat, having only a minor upward slope. Instead, they determine that the most impacting factor on the activation sparsity is the activation function used in the FFN, where RELU provided far more sparsity than function such as GELU.

Critical strengths
  • Clean experimental sweeps. The authors stave their hypothesis on a large variety of models, aiding in demonstrating the general applicability of their results. Then in the follow-up analysis the authors go into substantive depth to prove the point of the paper, by exploring different activation functions in more detail,
Critical weaknesses
  • Readability: The narrative of the paper is sometimes difficult to follow, this lies in the specific structure chosen to present the facts, including the mention of a failed hypothesis midway throughout the structure (sec 5.4), and the offloading of tables and figures important to the argument to the appendix. This is especially egregious with sec 4. experiments. In my opinion, the paper should be restructured significantly to make the story self-contained in the main body by adding the tables / figures necessary, while the appendix serves as additional illustration. Another example of this, is the section 3.1 which goes over participation rate of spectra which is not the main subject of the paper, but rather activations are.

  • Citations and related work. Several statements are made without supporting references, e.g. "... yet the ratio has scarcely moved." and "Much of the work on the FFN..." in the introduction. Furthermore, the related work is very sparse (six references split over three groups) and does not do an adequate job of sketching the existing work on activation sparsity in transformers, which makes it highly difficult to estimate the impact of this paper in its context.

Questions for authors
  • You show that activation sparsity is more or less independent of expansion ratio r, however refrain from making any practical recommendations on what it should be. Can you give some insights in how to optimize this hyperparameter, seeing as it represents a significant amount of the total parameters of a transformer. This would improve the significance of the paper, by translating these observations and theoretical results in actual practical applicability. I would also like for you to better position this paper and some of its claims to related work.
Reviewer_g9ukOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Many people assume that a wider feed-forward layer uses a smaller share of its units. The authors test this by training small models that differ only in width. Under their setting, the share of units that carry the output (i.e., density) changes by only 7% when width grows four times. It follows the activation function instead. The prediction holds for ReLU and GELU but is less accurate for gated activations.

Critical strengths
  1. The paper tests its explanation with an intervention. When the mean is set to zero, ReLU density matches the theory, and the width effect disappears where the theory expects it. This is stronger evidence than correlation alone.
  2. The experiments are controlled and easy to check. One factor changes at a time, each seed sees the same data. A failed prediction is reported and explained.
Critical weaknesses
  1. The models are small. They have only six layers. The authors say they cannot rule out different behavior at larger scale, but the title and conclusion make a general claim. It is unclear what this says about real large models.

  2. Training is short. Each model is trained for only 2,500 steps, and training length is never varied. Density depends on how the unit inputs change during training, so the result may change with longer training.

  3. The dataset is narrow. All causal results use one synthetic corpus of children's stories. It is unclear how far they apply to real text.

  4. The evidence on real models is confounded. The pretrained models differ in many ways besides width.

  5. "Utilization" is measured in one way. Density is only one definition of how much of the layer is used, and the authors say another measure might depend on width. The conclusion that usage cannot set the width is broader than what is measured.

Questions for authors
  1. Does the result hold in much larger models?

  2. Does the result hold when models are trained much longer?

  3. Can you show the result on more varied, real text?

  4. Are there pretrained models that differ only in width, so the effect can be checked outside your own runs?

  5. Can density be predicted for gated activations to the same accuracy as for ReLU and GELU?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.