acceptodds
Withdrawn from ICLR 2027

Monotone FFNs and the Quality–Sensitivity Trade-off in Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

How does a simple local architectural constraint change a language model's sensitivity to token substitutions? We study nonnegative weights in Transformer feed-forward networks (FFNs), while leaving attention, normalization, and other components unconstrained. With a non-decreasing activation such as ReLU, the constrained FFN branch is coordinatewise monotone in its own input, providing a local order-preserving bias without making the full Transformer monotone. We ask what this constraint changes empirically, what quality cost it incurs, and how far its local guarantees extend. On , fixed-gradient HotFlip with a nominal five-token budget produces a mean reference-loss increase of 0.14 nats for the constrained model, compared with 0.35 for an unconstrained fine-tuned baseline; the fraction of examples exceeding a 10% loss increase falls from 59.5% to 17.0% (200 examples, one training seed). This reduced sensitivity comes with a cost: the constrained model has higher clean and attacked loss, and clean ROUGE-L decreases by approximately 2–5% relative across two full test sets and five training seeds. To clarify what the architectural constraint does and does not imply, we analyze branch Jacobians and bound the local contributions of saturated units under explicit assumptions, and separately test agreement with semantic order using post-hoc probes, finding no consistent improvement. An exploratory three-run Pythia-1.4B study with nonnegative FFN weights and GELU likewise finds smaller substitution-induced loss increments, under an evaluation objective that also scores edited tokens. Together, the results identify a quality–sensitivity trade-off associated with nonnegative FFN weights while separating coordinatewise branch monotonicity, semantic ordering, and full-model robustness. The current experiments do not isolate persistent nonnegativity from the accompanying initialization change.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Robustness, Large Language Models, Adversarial Attacks

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_1k8SOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies whether imposing nonnegative weights on Transformer FFNs changes sensitivity to token substitutions. For T5-small with ReLU, the resulting FFN branch is coordinatewise monotone in its own input, although the full Transformer is not monotone. The main experiment reports smaller HotFlip-induced loss increases for the constrained model, at the cost of worse clean and attacked performance. The paper also presents local Jacobian arguments, semantic-order probes, and an exploratory Pythia-1.4B study.

Critical strengths
  1. The authors correctly distinguish local FFN monotonicity from full-model monotonicity and explicitly acknowledge that nonnegative weights do not imply a Lipschitz or robustness guarantee. This is an important conceptual clarification and prevents several common overclaims.
  2. The paper consistently reports that the constrained model has worse clean quality and higher absolute attacked loss, which makes the trade-off transparent. For example, on CNN/DailyMail, ROUGE-L drops from 33.2 to 31.5, while attacked loss is 3.53 versus 3.12 for the baseline.

These are good qualities of the study, but they are not sufficient for acceptance given the issues below.

Critical weaknesses
  1. The constrained model is initialized approximately from ∣Wpre∣|W_{\mathrm{pre}}|, which removes all negative pretrained weight signs and therefore substantially changes the pretrained function before fine-tuning. The paper's own control indicates that an unconstrained model initialized from pretrained magnitudes reaches essentially the same validation loss as the constrained model, suggesting that destruction of the pretrained sign structure is already a major factor. Crucially, attack results for this control are unavailable. Therefore the key comparison does not tell us whether the reduced HotFlip sensitivity comes from persistent nonnegativity/monotonicity, from the initialization change, from optimization differences, or simply from obtaining a substantially worse model. This is not a minor ablation: it directly affects the interpretation of the paper's main result.
  2. The main T5 result is based on only one training seed and 200 examples, using a fixed-gradient HotFlip variant that does not recompute gradients after substitutions and does not even verify the final token distance after decode/re-encode. The paper also lacks paired per-example losses for statistical testing. The Pythia experiments do not resolve this concern. Pythia uses GELU, so the FFNs are not actually monotone; the constrained and baseline models receive unequal training schedules; and the attack objective scores the substituted tokens themselves. 47380_Monotone_FFNs_and_the_Qu Moreover, random substitutions damage the baseline more than the fixed-gradient HotFlip attack, which the authors themselves identify as evidence that HotFlip is a weak search procedure in this setting.
Questions for authors
  1. Can the authors run the same T5 attack evaluation on the unconstrained magnitude-initialized and sign-frozen controls? In my view, this is essential. If an unconstrained ∣Wpre∣|W_{\mathrm{pre}}|-initialized model shows the same reduction in attack-induced loss, the main effect cannot reasonably be attributed to persistent nonnegativity.
  2. Does the T5 result persist across multiple independently trained seeds under a stronger adaptive attack that recomputes gradients after each substitution and enforces the actual token-edit budget? A paired per-example evaluation would also be important. A positive result here could materially change my assessment.

I would not phrase these as optional future work; both are directly relevant to the current claims.

Reviewer_LYTROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose switching to only non-negative weights in FFN in T5 model and test how this changes the language model sensitivity to token substitutions. On T5, the loss growth during the attack decrease from 0.35 to 0.14, but the model quality is getting worse both before and after the attack. Additional experiments were conducted on Pythia. The authors also analyze the mathematical properties of this restriction.

Critical strengths

A substantial reduction in loss growth was achieved for T5. The authors also demonstrate the cost of this change - the drop in standard quality was verified on two full test sets and five training runs. Also I found no significant errors in math. The limitations are openly described.

Critical weaknesses

My main concern is whether maintaining nonnegative weights throughout training provides any benefit beyond the change in initialization. If I understand correctly, Table 7 contains a model in which the original FFN weights were first replaced with their absolute values and then trained without sign restriction. It achieved the same rounded validation loss of 2.51 as the proposed model, but the attack results for it are not shown. Without this comparison, it is difficult to separate the effect of the restriction from the consequences of changing the original weights and the degraded performance.

The main T5 evaluation uses 200 examples, one training seed, and one gradient calculation per example. As stated in Appendix C.3, the final token distance may differ from the intended five-token budget due to retokenization. So it is unclear whether the results would hold under a stronger attack with a verified edit budget.

In Pythia, the loss calculation takes into account the replaced tokens themselves. Therefore, the results do not allow us to separate the prediction error of the substitutions themselves from their impact on the rest of the text (Section 5.4).

Questions for authors
  1. What are the results of the abs-init-free and sign-frozen controls from Table 7 under the same T5 HotFlip protocol? Does the constrained model show a smaller loss increase than abs-init-free?

  2. Does the T5 result persist when recalculating the gradient after each replacement, checking the number of modified tokens, and repeating on multiple training seeds?

  3. How will Pythia results change if we calculate the loss only on identical, unmodified target tokens?

Reviewer_JXAFOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper is trying to see if constraining the feedforward layers of a transformer to be non-negative would make the model more robust. The main motivation for the authors is that this might make the weights coordinate-wise monotonic.

There are some adversarial attacks that have been tested, like hotflip, and models like T5-small and Pythia were trained. There is some theoretical study that's already been done in this paper.

Overall the paper reads largely as AI-generated. I found it very difficult to make any sense of it and it was unusually difficult to follow despite putting in substantial effort. I have used an LM to help me process and simplify the text. That is to say my scientific claims and understanding are after extensive simplification of this work

Critical strengths

The paper investigates an interesting question: whether small changes in the architecture can make the model more robust and less sensitive to token changes

Critical weaknesses

First of all the paper provides very weak empirical evidence on its main claim of making models more robust on a small sample of hotflip-based attacks (even where the performance is not significantly stronger as compared to the unconstrained model)

Secondly, the paper is extremely hard to follow, and the presentation is very challenging.

While the amount of performance drop might be lesser, causing the model to be less sensitive, the trade-off is not interesting because the models are overall weaker. Being less sensitive on a weaker model is not as useful as being stronger in general.

Questions for authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.